From nobody Fri Jun 19 09:05:21 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03FCA3D813E; Fri, 24 Apr 2026 14:01:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039285; cv=none; b=ZqctjhGCP7HQQNngdZvWA8Tmmjt1spwvF7bXgadhTvVT09u63H3vG6iCVt7vxxoHSk4K4+zEVSevQEeZs8OlX5zdcmnyUjtdE9emkVAYTzx15nG72edm00nF2kC+R7seXTqAQnQzbeQkvVenwt2YpZEb2aP1G4bpOiYTTSdU6+c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039285; c=relaxed/simple; bh=jn2J3yA0J0jkmQB7t5ls5b/AEhdqnAVp7d4GDg/Z45k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OoPUkkQxqpe4QLuKBbz5fhxUJbvZd48MtYLXsJmBBcPoorEhTnj9iHJ+dS8HoYqHqmJTbgO4/N5Gd7GK9k1xrkVxp1v6vuae+ni5eH1iJkI6Dg80WBkchdI72z2GHNRFXzElRqG87GrBJvhuABHUg2yiHGNSsQTSVoUcQWH9eWU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=N4LFlvWX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="N4LFlvWX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 927D0C2BCB2; Fri, 24 Apr 2026 14:01:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777039284; bh=jn2J3yA0J0jkmQB7t5ls5b/AEhdqnAVp7d4GDg/Z45k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N4LFlvWXRa+nC2LrxZiweANVn2sW/mtDJtn9Hc8U2IyjZIgSy4LNgQs+IcGbxKtKB SQXc/kEvUje04JOPH7+CFjBAkhuNvJ1+mJpMKSbR3Ape53qEz2rSthY8WkOHr+x/AD NerivrUMmvutjugXmGXWN2xhWSt5Eg1Z+HGbi1TwI2Sj41+fdZ0AVNy+sVEUVCKVPG tLwhV6Z90OosV7D6kFMR/VPDcZwTKg+fgxFgCAHVlS42dHsnn/IJeYntXseH2xsUih orNQgIAhSM/STtN+hJVaSrAZAogD0kKuva7+i6hCEbPLvJar+PoD4pQMRXPnUSHvQp ApS4f2wWVf0jQ== From: Sasha Levin To: akpm@linux-foundation.org, david@kernel.org, corbet@lwn.net Cc: ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, skhan@linuxfoundation.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sasha Levin , Sanif Veeras , "Claude:claude-opus-4-7" Subject: [RFC 1/7] mm: add generic dual-bitmap consistency primitives Date: Fri, 24 Apr 2026 10:00:50 -0400 Message-ID: <20260424140056.2094777-2-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424140056.2094777-1-sashal@kernel.org> References: <20260424140056.2094777-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sasha Levin Add a header-only library implementing a pair-of-complementary-bitmaps integrity primitive: maintain two bitmaps where primary[i] =3D=3D !secondar= y[i] for every bit i, and detect corruption by checking that invariant. The motivation (silent metadata corruption that KASAN/KFENCE cannot see, plus the functional-safety argument for wanting this in the kernel) is described in the cover letter; this patch only introduces the building block. The primary bitmap uses 1 for "allocated" and 0 for "free"; the secondary uses the opposite convention. dual_bitmap_set() and dual_bitmap_clear() update both bitmaps and return the previous primary bit so callers can distinguish a real state transition from a double-alloc or double-free. dual_bitmap_validate() walks every word and returns the number of words that fail the invariant. Concurrency note: set and clear perform two independent atomic bit operations against the primary and secondary bitmaps, so the invariant is transiently violated between those two ops. A concurrent reader can observe an inconsistent pair on a healthy kernel. The validation helpers absorb this by retrying a small number of times with cpu_relax() and, after the retries, issuing an smp_rmb() and re-reading. Real corruption is persistent and survives the retries; transient races resolve within a few cpu_relax() loops. This keeps the update path lock-free at the cost of a bounded false-positive probability under extreme write rates, which is acceptable for a fail-stop integrity check. Based-on-patch-by: Sanif Veeras Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Sasha Levin --- MAINTAINERS | 10 ++ include/linux/dual_bitmap.h | 216 ++++++++++++++++++++++++++++++++++++ 2 files changed, 226 insertions(+) create mode 100644 include/linux/dual_bitmap.h diff --git a/MAINTAINERS b/MAINTAINERS index d1cc0e12fe1f..81b1f44215b3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19972,6 +19972,16 @@ F: mm/page-writeback.c F: mm/readahead.c F: mm/truncate.c =20 +PAGE CONSISTENCY CHECKER +M: Sasha Levin +L: linux-mm@kvack.org +S: Maintained +F: Documentation/mm/page_consistency.rst +F: include/linux/dual_bitmap.h +F: include/linux/page_consistency.h +F: mm/page_consistency.c +F: mm/page_consistency_test.c + PAGE POOL M: Jesper Dangaard Brouer M: Ilias Apalodimas diff --git a/include/linux/dual_bitmap.h b/include/linux/dual_bitmap.h new file mode 100644 index 000000000000..136822267be1 --- /dev/null +++ b/include/linux/dual_bitmap.h @@ -0,0 +1,216 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Dual-bitmap consistency primitives + * + * Provides a generic library for maintaining dual bitmaps with the invari= ant + * that (primary =3D=3D ~secondary). This pattern is useful for detecting + * single-bit memory corruption in bitmap-based data structures. + * + * Based on NVIDIA safety research. + */ +#ifndef _LINUX_DUAL_BITMAP_H +#define _LINUX_DUAL_BITMAP_H + +#include +#include +#include +#include +#include +#include + +/* Number of retries for transient inconsistencies from concurrent updates= */ +#define DUAL_BITMAP_RETRY_COUNT 3 + +/* Bitmap indices */ +enum dual_bitmap_index { + DUAL_BITMAP_PRIMARY =3D 0, /* 0=3Dfree, 1=3Dallocated */ + DUAL_BITMAP_SECONDARY =3D 1, /* 0=3Dallocated, 1=3Dfree (complement) */ + DUAL_BITMAP_COUNT =3D 2 +}; + +/** + * struct dual_bitmap - Dual bitmap structure + * @bitmap: Array of two bitmap pointers [PRIMARY, SECONDARY] + * @nbits: Number of bits in each bitmap + */ +struct dual_bitmap { + unsigned long *bitmap[DUAL_BITMAP_COUNT]; + unsigned int nbits; +}; + +/** + * dual_bitmap_consistent_word - Check if a word pair maintains the invari= ant + * @primary: Primary bitmap word + * @secondary: Secondary bitmap word + * + * Returns true if primary =3D=3D ~secondary + */ +static inline bool dual_bitmap_consistent_word(unsigned long primary, + unsigned long secondary) +{ + return primary =3D=3D ~secondary; +} + +/** + * dual_bitmap_set - Set bit in dual bitmap (mark as allocated) + * @db: Dual bitmap structure + * @bit: Bit position to set + * + * Sets bit in primary and clears corresponding bit in secondary. + * Returns the old value of the primary bit (true if was already set). + */ +static inline bool dual_bitmap_set(struct dual_bitmap *db, unsigned long b= it) +{ + bool was_set; + + if (WARN_ON_ONCE(bit >=3D db->nbits)) + return false; + + was_set =3D test_and_set_bit(bit, db->bitmap[DUAL_BITMAP_PRIMARY]); + test_and_clear_bit(bit, db->bitmap[DUAL_BITMAP_SECONDARY]); + + return was_set; +} + +/** + * dual_bitmap_clear - Clear bit in dual bitmap (mark as free) + * @db: Dual bitmap structure + * @bit: Bit position to clear + * + * Clears bit in primary and sets corresponding bit in secondary. + * Returns the old value of the primary bit (true if was set). + */ +static inline bool dual_bitmap_clear(struct dual_bitmap *db, unsigned long= bit) +{ + bool was_set; + + if (WARN_ON_ONCE(bit >=3D db->nbits)) + return false; + + was_set =3D test_and_clear_bit(bit, db->bitmap[DUAL_BITMAP_PRIMARY]); + test_and_set_bit(bit, db->bitmap[DUAL_BITMAP_SECONDARY]); + + return was_set; +} + +/** + * dual_bitmap_test - Test if bit is set in primary bitmap + * @db: Dual bitmap structure + * @bit: Bit position to test + * + * Returns true if bit is set in primary (allocated), false if clear (free= ). + */ +static inline bool dual_bitmap_test(const struct dual_bitmap *db, + unsigned long bit) +{ + if (WARN_ON_ONCE(bit >=3D db->nbits)) + return false; + + return test_bit(bit, db->bitmap[DUAL_BITMAP_PRIMARY]); +} + +/** + * dual_bitmap_consistent - Check consistency of a single bit + * @db: Dual bitmap structure + * @bit: Bit position to check + * + * Returns true if the bit values are consistent (primary !=3D secondary). + * Uses retry logic to handle transient inconsistencies from concurrent + * updates - real corruption persists while races resolve quickly. + */ +static inline bool dual_bitmap_consistent(const struct dual_bitmap *db, + unsigned long bit) +{ + int retries =3D DUAL_BITMAP_RETRY_COUNT; + + if (WARN_ON_ONCE(bit >=3D db->nbits)) + return false; + + do { + bool primary =3D test_bit(bit, db->bitmap[DUAL_BITMAP_PRIMARY]); + bool secondary =3D test_bit(bit, db->bitmap[DUAL_BITMAP_SECONDARY]); + + if (primary !=3D secondary) + return true; /* Consistent */ + + /* Inconsistent - could be transient race, retry */ + cpu_relax(); + } while (--retries > 0); + + /* + * Inconsistent after retries. Issue a read barrier and check + * one last time to rule out stale/reordered reads. + * + * Note: the two test_bit() calls are still non-atomic w.r.t. + * each other, so a concurrent set/clear between them can cause + * a transient false positive. This is acceptable because real + * corruption is persistent and will be caught on the next check. + */ + smp_rmb(); + return test_bit(bit, db->bitmap[DUAL_BITMAP_PRIMARY]) !=3D + test_bit(bit, db->bitmap[DUAL_BITMAP_SECONDARY]); +} + +/** + * dual_bitmap_validate - Validate entire dual bitmap + * @db: Dual bitmap structure + * + * Checks that the invariant (primary =3D=3D ~secondary) holds for all wor= ds. + * Uses retry logic to handle transient inconsistencies from concurrent + * updates - real corruption persists while races resolve quickly. + * Returns the number of inconsistent words found (0 =3D all consistent). + * + * Note: this is a cold-path diagnostic function kept inline for + * header-only library simplicity. It should not be called in hot paths. + */ +static inline unsigned long dual_bitmap_validate(const struct dual_bitmap = *db) +{ + unsigned int words =3D BITS_TO_LONGS(db->nbits); + unsigned long violations =3D 0; + unsigned int i; + + for (i =3D 0; i < words; i++) { + unsigned long primary, secondary; + int retries =3D DUAL_BITMAP_RETRY_COUNT; + + do { + primary =3D READ_ONCE(db->bitmap[DUAL_BITMAP_PRIMARY][i]); + secondary =3D READ_ONCE(db->bitmap[DUAL_BITMAP_SECONDARY][i]); + + if (dual_bitmap_consistent_word(primary, secondary)) + break; /* Consistent, move to next word */ + + cpu_relax(); + } while (--retries > 0); + + if (retries =3D=3D 0) { + /* + * Inconsistent after retries. Issue a read + * barrier and re-read to rule out stale/reordered + * memory views before declaring corruption. + */ + smp_rmb(); + primary =3D READ_ONCE(db->bitmap[DUAL_BITMAP_PRIMARY][i]); + secondary =3D READ_ONCE(db->bitmap[DUAL_BITMAP_SECONDARY][i]); + if (!dual_bitmap_consistent_word(primary, secondary)) + violations++; + } + } + + return violations; +} + +/** + * dual_bitmap_init - Initialize dual bitmap to empty state + * @db: Dual bitmap structure + * + * Sets primary to all zeros (nothing allocated) and secondary to all ones. + * The bitmaps must already be allocated before calling this. + */ +static inline void dual_bitmap_init(struct dual_bitmap *db) +{ + bitmap_zero(db->bitmap[DUAL_BITMAP_PRIMARY], db->nbits); + bitmap_fill(db->bitmap[DUAL_BITMAP_SECONDARY], db->nbits); +} + +#endif /* _LINUX_DUAL_BITMAP_H */ --=20 2.53.0 From nobody Fri Jun 19 09:05:21 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB2B03DBD4F; Fri, 24 Apr 2026 14:01:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039287; cv=none; b=h4nS3OmwzcxBjnAW2slg0QcMhe6vL6cGBXjhLtK0P4H4WLjhp57A0feXFuooBJ6JuyS75AXuKwYVlw7+5ksDX4jN+SOL95uvG+Afi4xH5YXWdyjYnHpKDyD6+tJvTI9LEzrFeG8pWgbLCvg3wYttTuHYbiOl3NQ04Yr1vbzYBMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039287; c=relaxed/simple; bh=OMk8G83SwVj9TZi6pSd7RKvp6NWeeiZOhOo5iK9u9hw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DcWNwikIx1QhSeni1TJYwb4YUt3Xu6eEIKmM2hDILQ6slLtd89edS0DwF0BJTKWUE44XNM4gaFCs5G/0DO8jzzMAzgU9u3XmmBTsHBJuk7l/m7MMAuR2lu0X/3MuHS/MbhKikHd5CG37gFN+vI/5sAyUeBsa/OEz6ob0pp4QstU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g3/mycHd; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g3/mycHd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93D1CC2BCB5; Fri, 24 Apr 2026 14:01:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777039286; bh=OMk8G83SwVj9TZi6pSd7RKvp6NWeeiZOhOo5iK9u9hw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=g3/mycHdG1w/wHSq1REskac8ra3x8JGBb1lHFVdN7P1hhPz4Y0FiwTGP9U5OFZxIi dYhnt5nfG9T3kn2yS21djgpX2Ka6gViUNDTCqV8LO7TWnxnfM9q4jlg2LxGQNcPlvK Ib2XS1nIVnxGi2NIi9+0gs3M7ijhNPin3sYOoANBE2DQOul5r2n5IaV1r+N0vcLiXr nK3vUQ/xjUNY5n7Kn4nTCwnXXNTci6GmJREghaIm2+UQYMn/YZOo5a/GD5qiOvMTLh TGcGpTLT/bfxLUj6ha8qaheYrBhIZg9m7mMYIpD2urYuIaLP8Hk4kpelirNJPO0Tsx kGLemIdZExQEw== From: Sasha Levin To: akpm@linux-foundation.org, david@kernel.org, corbet@lwn.net Cc: ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, skhan@linuxfoundation.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sasha Levin , Sanif Veeras , "Claude:claude-opus-4-7" Subject: [RFC 2/7] mm: add page consistency checker header Date: Fri, 24 Apr 2026 10:00:51 -0400 Message-ID: <20260424140056.2094777-3-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424140056.2094777-1-sashal@kernel.org> References: <20260424140056.2094777-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sasha Levin Define the interface for CONFIG_DEBUG_PAGE_CONSISTENCY. The API mirrors the pattern used by page_table_check: inline wrapper functions check a static key before calling the out-of-line tracking implementation, so that callers in the allocator hot path only pay for a predicted-not-taken branch when the feature is built in but not active. The header is kept separate from the implementation so the hooks in page_alloc.c can be added without pulling in implementation details. Based-on-patch-by: Sanif Veeras Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Sasha Levin --- include/linux/page_consistency.h | 84 ++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 include/linux/page_consistency.h diff --git a/include/linux/page_consistency.h b/include/linux/page_consiste= ncy.h new file mode 100644 index 000000000000..f335fa3d6c5d --- /dev/null +++ b/include/linux/page_consistency.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Dual-bitmap page consistency checking + * + * Provides corruption detection for page allocations using complementary + * bitmaps where the invariant (primary =3D=3D ~secondary) must hold. + * + * Based on NVIDIA safety research. + */ +#ifndef _LINUX_PAGE_CONSISTENCY_H +#define _LINUX_PAGE_CONSISTENCY_H + +#include +#include + +/* Return codes for page consistency checking */ +enum page_consistency_result { + PAGE_CONSISTENCY_OK =3D 0, + PAGE_CONSISTENCY_MISMATCH, + PAGE_CONSISTENCY_NOT_TRACKED, +}; + +#ifdef CONFIG_DEBUG_PAGE_CONSISTENCY + +#include +DECLARE_STATIC_KEY_FALSE(page_consistency_enabled); + +/* Initialization - called during mm_core_init() */ +void __init page_consistency_init(void); + +/* Core tracking functions */ +void __page_consistency_alloc(struct page *page, unsigned int order); +void __page_consistency_free(struct page *page, unsigned int order); + +/* Validation functions */ +enum page_consistency_result page_consistency_check_page(struct page *page= ); +enum page_consistency_result page_consistency_validate_all(void); + +/** + * page_consistency_alloc - Track page allocation + * @page: Allocated page + * @order: Allocation order + * + * Called from post_alloc_hook() to track page allocations. + * The static key avoids the out-of-line tracking call until initializatio= n. + */ +static inline void page_consistency_alloc(struct page *page, unsigned int = order) +{ + if (static_branch_unlikely(&page_consistency_enabled)) + __page_consistency_alloc(page, order); +} + +/** + * page_consistency_free - Track page free + * @page: Page being freed + * @order: Free order + * + * Called from free_pages_prepare() to track page frees. + * The static key avoids the out-of-line tracking call until initializatio= n. + */ +static inline void page_consistency_free(struct page *page, unsigned int o= rder) +{ + if (static_branch_unlikely(&page_consistency_enabled)) + __page_consistency_free(page, order); +} + +#else /* !CONFIG_DEBUG_PAGE_CONSISTENCY */ + +static inline void __init page_consistency_init(void) {} +static inline void page_consistency_alloc(struct page *page, unsigned int = order) {} +static inline void page_consistency_free(struct page *page, unsigned int o= rder) {} + +static inline enum page_consistency_result page_consistency_check_page(str= uct page *page) +{ + return PAGE_CONSISTENCY_OK; +} + +static inline enum page_consistency_result page_consistency_validate_all(v= oid) +{ + return PAGE_CONSISTENCY_OK; +} + +#endif /* CONFIG_DEBUG_PAGE_CONSISTENCY */ +#endif /* _LINUX_PAGE_CONSISTENCY_H */ --=20 2.53.0 From nobody Fri Jun 19 09:05:21 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD4913DBD47; Fri, 24 Apr 2026 14:01:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039288; cv=none; b=lkFlMsc58cGi9PRdCbx5D2iVl07UY0L/sdoedcSZYhQs9fYIrltI7dZ0sLOrxvbu8CMuqRLqh+BU5y4VYk3bizLq3q+xXMhTd7xWUH4Sa+GekwS7GN1Gk1anW9dH2cJE5HxjxI7DXSRXQX63YbnmPCFyuXYq0AeADjI/xRwh2F4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039288; c=relaxed/simple; bh=ne0w3U/qb9Eq3ShDvj1y+mTm0Iqj6lSthxxEzyDc60g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T8zVoWeyViUDS/QUNgVgSEdr4rgr7Ur1cNoXCZiALw1Y7zxXfnmQ/d0FFd1OOwx4bZHnumSDJ/oLhM6f0j0flqmNHTUYl94ANkwZhb9x0n6/gcg+xTpofvkNg/Souyxt3bQmlqa+Tbd+oKIaIfMLqC8XsRxWTh4aj1ZW3g6HDp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=I9H0hbfM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="I9H0hbfM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93534C2BCB6; Fri, 24 Apr 2026 14:01:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777039288; bh=ne0w3U/qb9Eq3ShDvj1y+mTm0Iqj6lSthxxEzyDc60g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I9H0hbfMFtXWEg5HWINI/FHkJ9LkKBaBFKX3V7MsIYLgPac0I5xlVQGxhaQg3naj3 4G7bPpahxNPtNYrVS34oz8M6Mt1E/hqS5TOb8hhX7QJ0F40hOFGnWicy9BpiRNdGw9 9uHnRGZ0npVlu/iBL3NdQ0Vk+rR5LUa92Ms2AMVwhpuzjdcdv/diAhFHwWsUtBLC3+ WmpWHc0/+CeSQPFp/s2Piq1QaaAtajpXxEndvi/wEia9+Kk/cg1rYvon+mveRlHCGp glaFm1BtTZ8LRj+tx5XUp10+9q48mgjIX3MbdLDlZVggK989NoBkAXb1/IkYh/n01R TUcfBxG0cHyig== From: Sasha Levin To: akpm@linux-foundation.org, david@kernel.org, corbet@lwn.net Cc: ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, skhan@linuxfoundation.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sasha Levin , Sanif Veeras , "Claude:claude-opus-4-7" Subject: [RFC 3/7] mm: add Kconfig options for page consistency checker Date: Fri, 24 Apr 2026 10:00:52 -0400 Message-ID: <20260424140056.2094777-4-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424140056.2094777-1-sashal@kernel.org> References: <20260424140056.2094777-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sasha Levin Add two configuration options for the dual-bitmap page consistency checker. DEBUG_PAGE_CONSISTENCY enables the feature itself. It depends on DEBUG_KERNEL since this is a debugging tool, and selects DEBUG_FS to provide the statistics interface. Memory overhead is two bits per physical page frame across two bitmaps, so about 1 MB for a 16 GB system. The bitmaps are statically sized at boot from memblock, so memory hotplug is not supported and the option depends on !MEMORY_HOTPLUG. DEBUG_PAGE_CONSISTENCY_PANIC controls the response to a detected violation. When enabled (the default) the kernel panics on double-alloc, double-free, or bitmap corruption; when disabled it logs a warning and continues. Based-on-patch-by: Sanif Veeras Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Sasha Levin --- mm/Kconfig.debug | 59 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/mm/Kconfig.debug b/mm/Kconfig.debug index 7638d75b27db..a005c904677c 100644 --- a/mm/Kconfig.debug +++ b/mm/Kconfig.debug @@ -144,6 +144,65 @@ config PAGE_TABLE_CHECK_ENFORCED =20 If unsure say "n". =20 +config DEBUG_PAGE_CONSISTENCY + bool "Debug page allocator with dual-bitmap consistency checking" + depends on DEBUG_KERNEL + depends on !MEMORY_HOTPLUG + select DEBUG_FS + help + Enable dual-bitmap tracking of page allocations for corruption + detection. Uses two complementary bitmaps where the invariant + (primary =3D=3D ~secondary) must hold. Any bit flip in either bitmap + will be detected. + + This is useful for safety-critical systems requiring Freedom From + Interference (FFI) guarantees per ISO 26262 (ASIL-D) and IEC 61508 + (SIL-3). + + When disabled, the hooks compile away. When enabled, a static key + gates tracking until initialization succeeds. The bitmaps are flat, + covering the entire PFN range from memblock_start_of_DRAM() to + memblock_end_of_DRAM() including any holes. This is deliberate: + simple (pfn - min_pfn) indexing is trivially auditable and avoids + auxiliary data structures that could themselves be subject to + corruption. Memory overhead is two bits per PFN in the spanned + range, e.g. ~4 MB total for a 64 GB system. Waste from holes is + typically under 2%. + + Based on NVIDIA safety research. + + If unsure, say N. + +config DEBUG_PAGE_CONSISTENCY_PANIC + bool "Panic on page consistency failure" + depends on DEBUG_PAGE_CONSISTENCY + default y + help + If enabled, the kernel will panic when a page consistency + violation is detected, such as double-alloc or double-free. + + If disabled, a WARN with a stack trace is emitted and execution + continues. + + For safety-critical systems, say Y. + For debugging/development, say N. + +config DEBUG_PAGE_CONSISTENCY_KUNIT_TEST + tristate "KUnit tests for dual-bitmap consistency primitives" if !KUNIT_A= LL_TESTS + depends on KUNIT + default KUNIT_ALL_TESTS + help + Enable KUnit tests for the dual-bitmap primitives defined in + . These tests verify the core algorithm: + setting and clearing bits in complementary bitmaps, detecting + double-set and double-clear conditions, and detecting simulated + corruption. + + The tests exercise only the header-only dual_bitmap library and + do not require CONFIG_DEBUG_PAGE_CONSISTENCY. + + If unsure, say N. + config PAGE_POISONING bool "Poison pages after freeing" help --=20 2.53.0 From nobody Fri Jun 19 09:05:21 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADBD13DBD76; Fri, 24 Apr 2026 14:01:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039290; cv=none; b=NYtkyBltZRNsJqumazY0qjIN3eElntWqpQlM6n0fTnR5cHlfHxbhD7G3PggYjyqn2CB2q6/Nm4rdTbqf/kQeUjAbxG1xU9TiWaZVqSTJCp8n76g3a3ijWUWfl3WOFRzBroalvSCVVa5H5sB8pnVrP89wgPhSCiZU7pFron40jSU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039290; c=relaxed/simple; bh=2y/qJ7rBs64ooX/1bvrjs/dZgg1uSC/mWgnTi28gXUE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WVmM0mDxBCyQ0qZI3NAMyOwPiJ4fQe6VV1uDDNo77oOF1+6LHcgUxJE7Qvt4JKJvkKfQ3nYXYqSD59zjlIts0ieydLpE91WwvL0/ixf45f+GLP1RhE7zThkzc1Jdyq9YJjkVqyD3Ukoajma/BNjKDmigGErGEZXk/zsD9hcb/7U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OBSDW6+7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OBSDW6+7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 96F1FC2BCB9; Fri, 24 Apr 2026 14:01:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777039290; bh=2y/qJ7rBs64ooX/1bvrjs/dZgg1uSC/mWgnTi28gXUE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OBSDW6+7Vi2Cc7APkzmfTr+0wXlEOeiHYpKiq0go50uYLvH3Bf9CaSxciXpC2/o5P +rnSpIjvgAbBCv1qOHj8FTjemiFbkayuTod2iuJBXxR3ySZ6ROqjlMyNUkLsA5RG8Z TVEvBZW3SRQ+DUjATvg5giL6SNMjSBKBHj06pAR7EqB9Nnzlhkab1xj/P5FD+DSD3V xHRiyyFjI1NPqhvuTT5B9+9iub5jgRXj4x+pi/ldTKoB2/7G9w4JvVIsecuWCfPXgc oFdTNXffHxD37p4LHl7JOAsbuunKSRCFwmtqZlA5tm1XsjL1og2STjxF0wznjsJw4h l9VEajX0l2NlQ== From: Sasha Levin To: akpm@linux-foundation.org, david@kernel.org, corbet@lwn.net Cc: ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, skhan@linuxfoundation.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sasha Levin , Sanif Veeras , "Claude:claude-opus-4-7" Subject: [RFC 4/7] mm: add page consistency checker implementation Date: Fri, 24 Apr 2026 10:00:53 -0400 Message-ID: <20260424140056.2094777-5-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424140056.2094777-1-sashal@kernel.org> References: <20260424140056.2094777-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sasha Levin This is the core implementation of the dual-bitmap page allocator consistency checker. During initialization, two bitmaps are allocated covering the physical memory range reported by memblock. The primary bitmap starts zeroed (all pages free) and the secondary bitmap starts filled with ones (the complement). As pages are allocated and freed, both bitmaps are updated atomically using test_and_set_bit / test_and_clear_bit. The mark_page_state() helper is the heart of the checker. When allocating, it sets the bit in primary and clears it in secondary. If the primary bit was already set, that indicates a double-alloc and triggers a panic (or warning, depending on config). The free path updates the bitmaps but defers double-free detection until after boot completes, since reserved boot memory pages are legitimately freed via free_reserved_area() and free_initmem() without ever being allocated through the buddy allocator. mark_page_state() returns whether the bitmap state actually changed, and the pages_tracked counter is only updated for real transitions. This keeps the counter an accurate reflection of the number of bits currently set in the primary bitmap, rather than a signed delta that can go negative during boot because of the reserved-area / initmem frees described above. The same property means post-boot "freeing of untracked pages" (e.g. a driver unloading a region it received via memblock) is detected as a real violation; by construction this code path remains a very small surface. Initialization validates that the spanned PFN range fits in an unsigned int (bitmap_bytes and the bitmap APIs are bounded by that) and disables the feature if it does not; a zero-span memblock is treated the same way. A debugfs interface is provided at /sys/kernel/debug/page_consistency/ with two files: "stats" shows counters for allocations, frees, and violations detected, while writing to "validate" triggers a full scan that checks the complement invariant holds for every bitmap word. The enable check at the debugfs late_initcall uses static_key_enabled() rather than the static_branch_unlikely() hot-path helper, which is the idiomatic form for a cold init path. Based-on-patch-by: Sanif Veeras Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Sasha Levin --- mm/Makefile | 1 + mm/page_consistency.c | 360 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 361 insertions(+) create mode 100644 mm/page_consistency.c diff --git a/mm/Makefile b/mm/Makefile index 8ad2ab08244e..2ee360001456 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -128,6 +128,7 @@ obj-$(CONFIG_NUMA_EMU) +=3D numa_emulation.o obj-$(CONFIG_BALLOON) +=3D balloon.o obj-$(CONFIG_PAGE_EXTENSION) +=3D page_ext.o obj-$(CONFIG_PAGE_TABLE_CHECK) +=3D page_table_check.o +obj-$(CONFIG_DEBUG_PAGE_CONSISTENCY) +=3D page_consistency.o obj-$(CONFIG_CMA_DEBUGFS) +=3D cma_debug.o obj-$(CONFIG_SECRETMEM) +=3D secretmem.o obj-$(CONFIG_CMA_SYSFS) +=3D cma_sysfs.o diff --git a/mm/page_consistency.c b/mm/page_consistency.c new file mode 100644 index 000000000000..f98059a1dcc0 --- /dev/null +++ b/mm/page_consistency.c @@ -0,0 +1,360 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Dual-bitmap page allocation consistency checker + * + * Provides corruption detection for page allocations using complementary + * bitmaps. The invariant (primary =3D=3D ~secondary) detects any single-b= it + * corruption in either bitmap. + * + * Based on NVIDIA safety research. + */ + +#define pr_fmt(fmt) "page_consistency: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +DEFINE_STATIC_KEY_FALSE(page_consistency_enabled); + +struct page_consistency_stats { + atomic64_t pages_tracked; + atomic64_t alloc_count; + atomic64_t free_count; + atomic64_t violations_detected; +}; + +static struct page_consistency_stats page_consistency_stats; + +/* Internal state */ +static struct { + struct dual_bitmap db; + unsigned long min_pfn; + unsigned long max_pfn; +} pc_state __ro_after_init; + +/** + * pfn_to_bit - Convert PFN to bitmap bit index + * @pfn: Page frame number + * + * Returns the bit index in the bitmap for the given PFN. + */ +static inline unsigned long pfn_to_bit(unsigned long pfn) +{ + return pfn - pc_state.min_pfn; +} + +/** + * pfn_in_range - Check if PFN is within tracked range + * @pfn: Page frame number to check + * + * Returns true if the PFN is within the range being tracked. + */ +static inline bool pfn_in_range(unsigned long pfn) +{ + return pfn >=3D pc_state.min_pfn && pfn < pc_state.max_pfn; +} + +/** + * mark_page_state - Update both bitmaps for a page state change + * @pfn: Page frame number + * @is_alloc: true for allocation, false for free + * + * Updates both bitmaps atomically and detects double-alloc/double-free. + * Double-free detection is deferred until system_state reaches SYSTEM_RUN= NING + * because reserved boot memory pages may be freed via free_reserved_area() + * and free_initmem() without ever being allocated through the buddy alloc= ator. + * + * Returns true if the primary bit actually transitioned to the requested + * state (0->1 for alloc, 1->0 for free), false if it was already in that + * state. Callers use this to keep pages_tracked an accurate reflection of + * the number of bits set in the primary bitmap. + */ +static bool mark_page_state(unsigned long pfn, bool is_alloc) +{ + unsigned long bit =3D pfn_to_bit(pfn); + bool was_allocated; + + /* + * Check the complement invariant before the update. The dual bitops + * below unconditionally write the secondary bit, so a corruption + * confined to the secondary bitmap would be silently erased by the + * very next alloc/free on that PFN. Primary-only corruption is still + * caught via the was_allocated check; this pre-check closes the gap + * for the secondary side so that corruption is reported symmetrically. + */ + if (unlikely(!dual_bitmap_consistent(&pc_state.db, bit))) { + atomic64_inc(&page_consistency_stats.violations_detected); +#ifdef CONFIG_DEBUG_PAGE_CONSISTENCY_PANIC + panic("page_consistency: bitmap corruption at PFN %lu before %s\n", + pfn, is_alloc ? "alloc" : "free"); +#else + WARN(1, "page_consistency: bitmap corruption at PFN %lu before %s\n", + pfn, is_alloc ? "alloc" : "free"); +#endif + } + + if (is_alloc) { + was_allocated =3D dual_bitmap_set(&pc_state.db, bit); + if (unlikely(was_allocated)) { + atomic64_inc(&page_consistency_stats.violations_detected); +#ifdef CONFIG_DEBUG_PAGE_CONSISTENCY_PANIC + panic("page_consistency: DOUBLE-ALLOC detected: PFN %lu\n", + pfn); +#else + WARN(1, "page_consistency: DOUBLE-ALLOC detected: PFN %lu\n", + pfn); +#endif + return false; + } + return true; + } + + was_allocated =3D dual_bitmap_clear(&pc_state.db, bit); + if (!was_allocated) { + /* + * Only flag double-free after system is fully running. + * During boot, free_reserved_area() and free_initmem() free + * pages never allocated through the buddy allocator - these + * are not bugs. system_state reaches SYSTEM_RUNNING only after + * all such freeing is complete. + */ + if (unlikely(system_state >=3D SYSTEM_RUNNING)) { + atomic64_inc(&page_consistency_stats.violations_detected); +#ifdef CONFIG_DEBUG_PAGE_CONSISTENCY_PANIC + panic("page_consistency: DOUBLE-FREE detected: PFN %lu\n", + pfn); +#else + WARN(1, "page_consistency: DOUBLE-FREE detected: PFN %lu\n", + pfn); +#endif + } + return false; + } + return true; +} + +/** + * __page_consistency_alloc - Track page allocation + * @page: Allocated page + * @order: Allocation order + * + * Called from post_alloc_hook() when page_consistency_enabled is true. + */ +void __page_consistency_alloc(struct page *page, unsigned int order) +{ + unsigned long pfn =3D page_to_pfn(page); + unsigned int nr_pages =3D 1U << order; + unsigned long last_pfn =3D pfn + nr_pages - 1; + unsigned int i, transitions =3D 0; + + if (!pfn_in_range(pfn) || !pfn_in_range(last_pfn)) + return; + + for (i =3D 0; i < nr_pages; i++) + if (mark_page_state(pfn + i, true)) + transitions++; + + atomic64_add(transitions, &page_consistency_stats.pages_tracked); + atomic64_inc(&page_consistency_stats.alloc_count); +} + +/** + * __page_consistency_free - Track page free + * @page: Page being freed + * @order: Free order + * + * Called from free_pages_prepare() when page_consistency_enabled is true. + */ +void __page_consistency_free(struct page *page, unsigned int order) +{ + unsigned long pfn =3D page_to_pfn(page); + unsigned int nr_pages =3D 1U << order; + unsigned long last_pfn =3D pfn + nr_pages - 1; + unsigned int i, transitions =3D 0; + + if (!pfn_in_range(pfn) || !pfn_in_range(last_pfn)) + return; + + for (i =3D 0; i < nr_pages; i++) + if (mark_page_state(pfn + i, false)) + transitions++; + + atomic64_sub(transitions, &page_consistency_stats.pages_tracked); + atomic64_inc(&page_consistency_stats.free_count); +} + +/** + * page_consistency_check_page - Check consistency for a single page + * @page: Page to check + * + * Returns PAGE_CONSISTENCY_OK if consistent, PAGE_CONSISTENCY_MISMATCH + * if corruption detected, or PAGE_CONSISTENCY_NOT_TRACKED if outside rang= e. + */ +enum page_consistency_result page_consistency_check_page(struct page *page) +{ + unsigned long pfn =3D page_to_pfn(page); + unsigned long bit; + + if (!pfn_in_range(pfn)) + return PAGE_CONSISTENCY_NOT_TRACKED; + + bit =3D pfn_to_bit(pfn); + + if (!dual_bitmap_consistent(&pc_state.db, bit)) { + atomic64_inc(&page_consistency_stats.violations_detected); + pr_err("Consistency violation for PFN %lu\n", pfn); + return PAGE_CONSISTENCY_MISMATCH; + } + + return PAGE_CONSISTENCY_OK; +} + +/** + * page_consistency_validate_all - Validate entire bitmap + * + * Performs a full consistency check of all bitmap words. + * Returns PAGE_CONSISTENCY_OK if all consistent, PAGE_CONSISTENCY_MISMATCH + * if any violations found. + */ +enum page_consistency_result page_consistency_validate_all(void) +{ + unsigned long violations; + + violations =3D dual_bitmap_validate(&pc_state.db); + + if (violations) { + /* + * violations counts inconsistent words, not bits. One word + * could contain up to BITS_PER_LONG corrupted bits. + */ + atomic64_add(violations, &page_consistency_stats.violations_detected); + pr_err("Validation found %lu inconsistent words\n", violations); + return PAGE_CONSISTENCY_MISMATCH; + } + + pr_info("Validation passed: %u bits checked\n", pc_state.db.nbits); + return PAGE_CONSISTENCY_OK; +} + +#ifdef CONFIG_DEBUG_FS +/* Debugfs interface */ + +static int stats_show(struct seq_file *m, void *v) +{ + seq_printf(m, "pages_tracked: %lld\n", + atomic64_read(&page_consistency_stats.pages_tracked)); + seq_printf(m, "alloc_count: %lld\n", + atomic64_read(&page_consistency_stats.alloc_count)); + seq_printf(m, "free_count: %lld\n", + atomic64_read(&page_consistency_stats.free_count)); + seq_printf(m, "violations_detected: %lld\n", + atomic64_read(&page_consistency_stats.violations_detected)); + seq_printf(m, "bitmap_size_bits: %u\n", pc_state.db.nbits); + seq_printf(m, "pfn_range: [%lu-%lu)\n", + pc_state.min_pfn, pc_state.max_pfn); + return 0; +} +DEFINE_SHOW_ATTRIBUTE(stats); + +static ssize_t validate_write(struct file *file, const char __user *buf, + size_t count, loff_t *ppos) +{ + int result =3D page_consistency_validate_all(); + + return result =3D=3D PAGE_CONSISTENCY_OK ? count : -EIO; +} + +static const struct file_operations validate_fops =3D { + .write =3D validate_write, + .llseek =3D noop_llseek, +}; + +static int __init page_consistency_debugfs_init(void) +{ + struct dentry *dir; + + if (!static_key_enabled(&page_consistency_enabled.key)) + return 0; + + dir =3D debugfs_create_dir("page_consistency", NULL); + debugfs_create_file("stats", 0444, dir, NULL, &stats_fops); + debugfs_create_file("validate", 0200, dir, NULL, &validate_fops); + + return 0; +} +late_initcall(page_consistency_debugfs_init); +#endif /* CONFIG_DEBUG_FS */ + +/** + * page_consistency_init - Initialize the page consistency checker + * + * Called during mm initialization to set up the dual bitmap tracking. + * Must be called while memblock is still active (before memblock_free_all= ()). + */ +void __init page_consistency_init(void) +{ + unsigned long spanned_pfns; + size_t bitmap_bytes; + + /* + * Size bitmaps to cover the full PFN range including any holes. + * Holes waste a few bits but a flat bitmap keeps the indexing + * trivial (pfn - min_pfn) and avoids additional data structures + * that would themselves be subject to corruption. This matches + * the approach used by pageblock_flags. + */ + pc_state.min_pfn =3D PHYS_PFN(memblock_start_of_DRAM()); + pc_state.max_pfn =3D PHYS_PFN(memblock_end_of_DRAM()); + spanned_pfns =3D pc_state.max_pfn - pc_state.min_pfn; + if (!spanned_pfns || spanned_pfns > UINT_MAX) { + pr_err("PFN span %lu cannot be represented by bitmap APIs, feature disab= led\n", + spanned_pfns); + return; + } + + pc_state.db.nbits =3D spanned_pfns; + + bitmap_bytes =3D BITS_TO_LONGS(pc_state.db.nbits) * sizeof(unsigned long); + + pr_info("Initializing: PFN range [%lu-%lu), %u bits (%zu KB per bitmap)\n= ", + pc_state.min_pfn, pc_state.max_pfn, pc_state.db.nbits, + bitmap_bytes / 1024); + + /* Allocate primary bitmap (zeroed by memblock_alloc) */ + pc_state.db.bitmap[DUAL_BITMAP_PRIMARY] =3D + memblock_alloc(bitmap_bytes, SMP_CACHE_BYTES); + if (!pc_state.db.bitmap[DUAL_BITMAP_PRIMARY]) { + pr_err("Failed to allocate primary bitmap, feature disabled\n"); + return; + } + + /* Allocate secondary bitmap */ + pc_state.db.bitmap[DUAL_BITMAP_SECONDARY] =3D + memblock_alloc(bitmap_bytes, SMP_CACHE_BYTES); + if (!pc_state.db.bitmap[DUAL_BITMAP_SECONDARY]) { + pr_err("Failed to allocate secondary bitmap, feature disabled\n"); + memblock_free(pc_state.db.bitmap[DUAL_BITMAP_PRIMARY], + bitmap_bytes); + pc_state.db.bitmap[DUAL_BITMAP_PRIMARY] =3D NULL; + return; + } + + /* + * Initialize: primary all zeros (already done by memblock_alloc), + * secondary all ones. Use dual_bitmap_init() for consistency. + */ + dual_bitmap_init(&pc_state.db); + + /* Enable tracking */ + static_branch_enable(&page_consistency_enabled); + pr_info("Initialized successfully, tracking enabled\n"); +} --=20 2.53.0 From nobody Fri Jun 19 09:05:21 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB7733DBD55; Fri, 24 Apr 2026 14:01:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039292; cv=none; b=Hci/wqhvMRY5Z1UXLz+U9QuF0SYiD4MA/SHNmZXfSyKYHOj8I5ExD/mD0jOcyy++dv111SDXtd/14xNwUzf0ap/0ol/yesfb0GcuTevDo+aFeIDfkeYkjWo6E9V/Y8TxjTXe9pOEmKHZJTCX2UMttZ3CjeyLwLXiUBPvZsMlcek= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039292; c=relaxed/simple; bh=ArDqKUGEZwiJhWfrZPmHU41Dcmah9WrzPqg0XPgBvRA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mm8DSY+uCvWrH64HOLpctsbBAgPNzJ+YIvUoFeIrH5Nm3Q4pctqWDoqP8QYvwgYP6uXQfTno588LeYbDUjRNL8Oi+6s394OZ2I590+WyMRwBb3QmGnFJqYkCuhE0/ARgVeDT5Zr63FdlQVF4oNcfiBYCzVaEPqpULwnRGRVs+kg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CaG0oB7y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CaG0oB7y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E47CC19425; Fri, 24 Apr 2026 14:01:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777039292; bh=ArDqKUGEZwiJhWfrZPmHU41Dcmah9WrzPqg0XPgBvRA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CaG0oB7yE/8MMkLVNptpROnS8QY0hY7zRDigwMad2xHJ4WdtE7xWL9Qy53kDSK4I+ vW8Zc/EkjFXCWK59rgGUEcHesVVv+GWXB6weOest7y+gCIf+ljKWTNcOkHW09GAHAD fPcJPxYVJ9g7qDykJeX/fVPlBS11BRHXyzJGxNBBAO9ClIyUOFu0u+o/BY5UItOpfy KOD7k7XOZkMkMqnnwxHCOmjXFqoXjCSEN0X2HwAP+tGW8z1DiPGx81VY8JjyttZGmQ XDrPZR9EsX6fJwfl2Nwy+pawBhFinnvEg+H6Lmgy8aIrTEOqvEvExqFwDHdFpGqGjq BCeAktUcwFnWg== From: Sasha Levin To: akpm@linux-foundation.org, david@kernel.org, corbet@lwn.net Cc: ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, skhan@linuxfoundation.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sasha Levin , Sanif Veeras , "Claude:claude-opus-4-7" Subject: [RFC 5/7] mm/page_alloc: integrate page consistency hooks Date: Fri, 24 Apr 2026 10:00:54 -0400 Message-ID: <20260424140056.2094777-6-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424140056.2094777-1-sashal@kernel.org> References: <20260424140056.2094777-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sasha Levin Wire up the page consistency checker with the page allocator by adding tracking hooks in the allocation and free paths. The hooks follow the same pattern already established by page_owner and page_table_check, inserting calls at the points where page state transitions occur. In post_alloc_hook(), a call to page_consistency_alloc() is added after the page_table_check_alloc() call. This records the allocation in both bitmaps, setting the primary bit and clearing the secondary bit for each page in the allocation. In __free_pages_prepare(), calls to page_consistency_free() are added in both the early-return path for the hwpoison check and the normal exit path. These calls clear the primary bitmap and set the secondary bitmap, maintaining the complementary relationship that enables corruption detection. The free hook lives in the internal __free_pages_prepare() rather than its free_pages_prepare() wrapper so that every free path (including the bulk folio free path) is observed exactly once. Initialization is hooked into mm_core_init() immediately before memblock_free_all(), while memblock is still active so it can use memblock_alloc() for the bitmaps, and after kho_memory_init() so that any memory handed back by a kexec-handover source has already been accounted. Based-on-patch-by: Sanif Veeras Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Sasha Levin --- mm/mm_init.c | 9 +++++++++ mm/page_alloc.c | 4 ++++ 2 files changed, 13 insertions(+) diff --git a/mm/mm_init.c b/mm/mm_init.c index df34797691bd..4d9495fb8789 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -2717,6 +2718,14 @@ void __init mm_core_init(void) */ kho_memory_init(); =20 + /* + * page_consistency_init() must run while memblock is active so it + * can use memblock_alloc() for the bitmaps. Boot-time reserved pages + * may be freed before SYSTEM_RUNNING without ever having been allocated + * through the buddy allocator, so the checker suppresses double-free + * reports until boot has completed. + */ + page_consistency_init(); memblock_free_all(); mem_init(); kmem_cache_init(); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2d4b6f1a554e..ae8f619875e9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -46,6 +46,7 @@ #include #include #include +#include #include #include #include @@ -1374,6 +1375,7 @@ __always_inline bool __free_pages_prepare(struct page= *page, /* Do not let hwpoison pages hit pcplists/buddy */ reset_page_owner(page, order); page_table_check_free(page, order); + page_consistency_free(page, order); pgalloc_tag_sub(page, 1 << order); =20 /* @@ -1432,6 +1434,7 @@ __always_inline bool __free_pages_prepare(struct page= *page, page->private =3D 0; reset_page_owner(page, order); page_table_check_free(page, order); + page_consistency_free(page, order); pgalloc_tag_sub(page, 1 << order); =20 if (!PageHighMem(page) && !(fpi_flags & FPI_TRYLOCK)) { @@ -1888,6 +1891,7 @@ inline void post_alloc_hook(struct page *page, unsign= ed int order, =20 set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); + page_consistency_alloc(page, order); pgalloc_tag_add(page, current, 1 << order); } =20 --=20 2.53.0 From nobody Fri Jun 19 09:05:21 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 935B73DC4CA; Fri, 24 Apr 2026 14:01:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039294; cv=none; b=blCkSy6s7V41uTPn4zpSvoxO7Hn+DxZcaBHo306lQc1GV+9KV6QOxT5nyqukDaBt+HoTlWxpna1DywE5tl6A4DwTi8UM6dbap74Wv/lCJaUJrbomLAXxSMjWilkxfniczL+xMMqXFClxPYMtDte2nrfFvunq0hNxsQcddtmNEIM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039294; c=relaxed/simple; bh=TBKrtjv7+bLe3/01VBt+c7tfJ2DPrZzmNjtiYrqNDtg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t/WpFcEfp+S5lJ9wY56TDmTBKf5aTqVNmXBm/PlyATOQz4XMk29akJ9SF2TXqR/0B/mUd9Jbt9c+ipDHkuER/AdH8OPgHqwToBiUDC05NjqPbyThKC/tMhcpwEroEmfMivGvvzPbjg8PXbQoIovLyY71SLM5R5aPJNHcj+jRjnM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lWd+BY+X; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lWd+BY+X" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F2C1C2BCB2; Fri, 24 Apr 2026 14:01:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777039294; bh=TBKrtjv7+bLe3/01VBt+c7tfJ2DPrZzmNjtiYrqNDtg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lWd+BY+X4r0dkO9VbE+etNI+pKWnytTiTzOSKdmSp+7vmV8Ojf342Ge77uF1k9BlB KvsGTjRejWodx3rv4U/vgPAv06riz63gFti0yJDRlZaV0f5eFxbxSn4x+8NaHsruIP PAliqU2OUAkrWZY9MGTnmX4oXUQ06hc+3sn1vxskkPZyRc/w3dmlAQjI2UA9elNsZr /NYOB58iwuXMHJqYGpnWPYQHhQnPJle+sp8cmcNE7p2n597BYz77oP5zmcE8KvkzEh WXtqjH7wrTrbiaSIK1yG8EFZglKjmXS75h/uCNTg8ww/n+6QdnLiwOxIo4ZqeWJwJD D0ujBonCYLlGA== From: Sasha Levin To: akpm@linux-foundation.org, david@kernel.org, corbet@lwn.net Cc: ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, skhan@linuxfoundation.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sasha Levin , Sanif Veeras , "Claude:claude-opus-4-7" Subject: [RFC 6/7] Documentation/mm: add page consistency checker documentation Date: Fri, 24 Apr 2026 10:00:55 -0400 Message-ID: <20260424140056.2094777-7-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424140056.2094777-1-sashal@kernel.org> References: <20260424140056.2094777-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sasha Levin Add documentation for the page consistency checker feature. The document explains the dual-bitmap algorithm, describes the configuration options, and covers the debugfs interface for monitoring and validation. The algorithm section explains how the complementary bitmaps work: the primary bitmap uses 1 for allocated and 0 for free, while the secondary bitmap uses the opposite convention. This redundancy means any single-bit corruption in either bitmap will cause a detectable violation of the invariant that primary[bit] must equal ~secondary[bit]. The document also explains the intentional limitation around double-free detection. During boot, free_reserved_area() releases pages that were never allocated through the buddy allocator. Flagging these as errors would generate many false positives, so double-free detection is deferred until after boot completes. Based-on-patch-by: Sanif Veeras Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Sasha Levin --- Documentation/mm/index.rst | 1 + Documentation/mm/page_consistency.rst | 211 ++++++++++++++++++++++++++ 2 files changed, 212 insertions(+) create mode 100644 Documentation/mm/page_consistency.rst diff --git a/Documentation/mm/index.rst b/Documentation/mm/index.rst index 7aa2a8886908..bef6c9bbc976 100644 --- a/Documentation/mm/index.rst +++ b/Documentation/mm/index.rst @@ -57,6 +57,7 @@ documentation, or deleted if it has served its purpose. page_frags page_owner page_table_check + page_consistency remap_file_pages split_page_table_lock transhuge diff --git a/Documentation/mm/page_consistency.rst b/Documentation/mm/page_= consistency.rst new file mode 100644 index 000000000000..dd1bde68f1a5 --- /dev/null +++ b/Documentation/mm/page_consistency.rst @@ -0,0 +1,211 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Page Consistency Checker +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The page consistency checker is a debugging feature that uses dual +complementary bitmaps to detect corruption in page allocation tracking. +It maintains the invariant that for every bit position, the primary +bitmap value equals the bitwise complement of the secondary bitmap value. + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +Memory corruption can silently flip bits in kernel data structures, +leading to difficult-to-diagnose failures. The page consistency checker +addresses this by maintaining redundant tracking of page allocation +state. Any single-bit corruption in either bitmap will cause a detectable +inconsistency, allowing the corruption to be caught rather than causing +silent data corruption or mysterious crashes later. + +The bitmaps are flat, covering the entire PFN range from +``memblock_start_of_DRAM()`` to ``memblock_end_of_DRAM()`` including any +holes in physical memory. This is a deliberate design choice: simple +``pfn - min_pfn`` indexing is trivially auditable, which matters for a +safety mechanism. Sparse or section-aware indexing would add auxiliary +data structures that could themselves be subject to corruption. See +`Limitations`_ for a detailed analysis of memory overhead including +holes. + +The approach is based on NVIDIA safety research and is +particularly useful for safety-critical systems requiring Freedom From +Interference (FFI) guarantees per ISO 26262 (ASIL-D) and IEC 61508 +(SIL-3). + +Algorithm +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The checker maintains two bitmaps tracking page allocation state: + +Primary bitmap + Bit set to 1 when page is allocated, 0 when free. + +Secondary bitmap + Bit set to 0 when page is allocated, 1 when free. + +The invariant that must always hold is:: + + primary[bit] =3D=3D ~secondary[bit] + +When a page is allocated, the checker sets the bit in the primary bitmap +and clears it in the secondary bitmap. When freed, it clears in primary +and sets in secondary. If the operation finds the bit already in the +expected final state, a double-allocation or double-free has occurred. + +Full validation can be performed by checking that every word in the +primary bitmap equals the bitwise complement of the corresponding word +in the secondary bitmap. + +Concurrency Handling +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The dual-bitmap update operations (set/clear) modify both bitmaps with +separate atomic operations. This creates a brief window where a concurrent +validation could observe a transient inconsistency. + +The implementation handles this by retrying validation when an inconsisten= cy +is detected. Real memory corruption is persistent and will fail all retrie= s. +Transient inconsistencies from concurrent updates resolve quickly and pass +on retry. + +Double-Free Detection +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Double-free detection is deferred until the system is fully running. During +boot, free_reserved_area() and free_initmem() release memory pages that we= re +never allocated through the buddy allocator. These would appear as double-= frees +but are expected behavior. + +The checker uses ``system_state >=3D SYSTEM_RUNNING`` to determine when bo= ot +is complete. This state is reached only after all init memory has been fre= ed, +ensuring no false positives from legitimate boot-time freeing. Any attempt= to +free a page that is not marked as allocated after this point will be flagg= ed +as a violation. + +Configuration +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The feature is controlled by two Kconfig options: + +``CONFIG_DEBUG_PAGE_CONSISTENCY`` + Enable the page consistency checker. Memory overhead is two bits per + PFN in the spanned range (start to end of DRAM, including holes), + roughly 4 MB total for a 64 GB system. When this option is disabled, + the allocator hooks compile away. When enabled, a static key gates + tracking until initialization succeeds. + +``CONFIG_DEBUG_PAGE_CONSISTENCY_PANIC`` + When enabled, the kernel will panic immediately upon detecting a + consistency violation. When disabled, a warning with a stack trace + is emitted and execution continues. Safety-critical systems should + enable this option. + +Debugfs Interface +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +When CONFIG_DEBUG_FS is enabled, the checker exposes files under +``/sys/kernel/debug/page_consistency/``: + +``stats`` + Read-only file showing tracking statistics:: + + pages_tracked: 12345 + alloc_count: 67890 + free_count: 55545 + violations_detected: 0 + bitmap_size_bits: 1048576 + pfn_range: [256-1048831] + +``validate`` + Write-only file. Writing any value triggers a full validation of + all bitmap words. Returns success if all words are consistent, + or -EIO if any violations are found. + +Usage +=3D=3D=3D=3D=3D + +To use the page consistency checker: + +1. Enable ``CONFIG_DEBUG_PAGE_CONSISTENCY`` in your kernel configuration. + +2. Optionally enable ``CONFIG_DEBUG_PAGE_CONSISTENCY_PANIC`` if you want + the kernel to halt immediately upon detecting corruption. + +3. Boot the kernel. The checker will automatically initialize and begin + tracking page allocations. + +4. Monitor statistics via debugfs:: + + cat /sys/kernel/debug/page_consistency/stats + +5. Trigger manual validation:: + + echo 1 > /sys/kernel/debug/page_consistency/validate + +Limitations +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +As described in `Overview`_, the bitmaps use a flat layout covering the +entire spanned PFN range, including any holes. Bits corresponding to +holes are initialized to the free state and remain inert; they maintain +the complement invariant and never trigger false positives. The kernel's +own ``pageblock_flags`` bitmaps use the same flat approach, sizing to +``zone->spanned_pages`` which includes holes. + +Memory overhead +--------------- + +The cost is 2 bits per PFN in the range (1 bit per bitmap x 2 bitmaps), +allocated via ``memblock_alloc()`` before the buddy allocator is +available. A hole wastes ``hole_size / PAGE_SIZE / 8`` bytes per bitmap. +In practice the waste from holes is negligible:: + + System Holes Per-bitmap size Hole waste Waste/bitmap + ----------- ------ --------------- ---------- ------------ + 64 GB, flat none 2 MB 0 0% + 256 GB, flat none 8 MB 0 0% + 256 GB 4 GB 8.1 MB 128 KB 1.5% + 1 TB 16 GB 32.5 MB 512 KB 1.5% + +On x86_64 the typical hole between low memory (below 4 GB) and high +memory is the largest source of waste. On arm64 with +``memblock_start_of_DRAM()`` typically at 0x80000000 (2 GB), holes +within the DRAM range are generally small or absent. + +Other limitations +----------------- + +The feature is incompatible with ``CONFIG_MEMORY_HOTPLUG`` because the +bitmaps are sized at boot based on the initial physical memory range. +Hot-added memory would fall outside the tracked PFN range and be silently +ignored. + +Boot-time reserved pages are not tracked as allocations. Freeing such a +page before ``SYSTEM_RUNNING`` is expected and is ignored by the +double-free detector. Freeing an untracked reserved page after boot is +reported as a double-free. + +The feature detects corruption in the tracking bitmaps themselves, not +corruption in the actual page contents. For page content verification, +see CONFIG_PAGE_POISONING. + +Implementation Details +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The checker hooks into the page allocator at two points: + +- ``post_alloc_hook()`` calls ``page_consistency_alloc()`` after a + successful allocation. + +- ``free_pages_prepare()`` calls ``page_consistency_free()`` when pages + are being returned to the allocator. + +Both hooks use static keys (``static_branch_unlikely``) so the overhead +is a single no-op when the feature is disabled. + +The bitmaps are allocated during ``mm_core_init()`` using +``memblock_alloc()`` before ``memblock_free_all()`` releases memblock +memory to the buddy allocator. The secondary bitmap is initialized with +all bits set to 1, establishing the initial complementary relationship +with the zeroed primary bitmap. --=20 2.53.0 From nobody Fri Jun 19 09:05:21 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1B0F3DB641; Fri, 24 Apr 2026 14:01:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039296; cv=none; b=mN7aygucZiS6qRpi42YIC7rjy5wzXg7FEM3yJrS3g3xpfUTZqdmuXcpeBvf4avhuvfvs1kqAkOzOHnOS+ik+1i3p7+7/IZxa5XZYyPA9qflJxObyxnO/p1bQ0TCLtvbKMfZsVmO1BFI4KSUP42+V5Qe35pBb45ejHhC6zM5TlSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777039296; c=relaxed/simple; bh=gwxMHRj8xZWjbOPpjwiAWwZJt+gKEClkO6VJvO7mI2Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U7BmvlOof36kbkwvvMk1ngjmpYAtai61RwW4sLGXSkelEr91yqD21EqD0kqsn/QqAzzDjnzjaAFOhSL0vqMa5GUywkK/5Wwk8Xr0oCFTHYOCKmq0Z7iD1W6bFiZH31NSlD6ZLS8xEpTbCmtaQfZ1eQvzV68xDsmQMYV3Lw+B46I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=i1SHrCEX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="i1SHrCEX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E5A0C19425; Fri, 24 Apr 2026 14:01:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777039296; bh=gwxMHRj8xZWjbOPpjwiAWwZJt+gKEClkO6VJvO7mI2Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i1SHrCEXRbPW4i0FC6+hGykEQzaarjMtbE/zFBFVEziiPwgJTwjYdtV/iQBP7pubm 9qbG8iE8eXHWezGjh4Jg8AAKoYThApjtNHa/ex5k3hY3flw0VibgOxpOEW+MuO+gSb ncjbv0WpYcaOFCsvThPJ/64n3RNw/kOKlOtsFIy0I7U8PpJ2Dyan7zZyfPtjarevMW pnHup9+o0D/oSQS4CofK5Zsp0Z6LjoRfswjABP3aqqXWDZfhTrWlNDpuAnaX87wOdS PSAhy5olALkX2a/B6tBaf+HWeuQte5UHAWc/NAwqx1x4eDriYDLxGFC4KqRQ1Rb9Fl feb1a3zyL9CPg== From: Sasha Levin To: akpm@linux-foundation.org, david@kernel.org, corbet@lwn.net Cc: ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, skhan@linuxfoundation.org, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Sasha Levin , Sanif Veeras , "Claude:claude-opus-4-7" Subject: [RFC 7/7] mm/page_consistency: add KUnit tests for dual-bitmap primitives Date: Fri, 24 Apr 2026 10:00:56 -0400 Message-ID: <20260424140056.2094777-8-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260424140056.2094777-1-sashal@kernel.org> References: <20260424140056.2094777-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sasha Levin Add a KUnit test suite that exercises the dual-bitmap algorithm used by the page consistency checker. The tests verify that the core invariant is maintained through various operations and that corruption can be reliably detected. The test suite covers several scenarios. The initial-state test confirms that a freshly initialized dual bitmap with zeroed primary and filled secondary passes validation. The set and clear tests verify that normal operations maintain the complementary relationship between bitmaps. The double-set and double-clear tests confirm that attempts to set an already-set bit or clear an already-clear bit are properly detected and reported through the return value. The corruption detection tests are particularly important for validating the safety guarantees. These tests directly manipulate one bitmap without updating its complement, simulating what would happen if a memory error flipped a bit. Both primary and secondary corruption scenarios are tested, confirming that either type is caught by validation. The suite also includes boundary condition tests covering the first bit, last bit, and word boundaries to ensure the bit manipulation logic handles edge cases correctly. Based-on-patch-by: Sanif Veeras Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Sasha Levin --- mm/Makefile | 1 + mm/page_consistency_test.c | 274 +++++++++++++++++++++++++++++++++++++ 2 files changed, 275 insertions(+) create mode 100644 mm/page_consistency_test.c diff --git a/mm/Makefile b/mm/Makefile index 2ee360001456..7106aeb79cf5 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -129,6 +129,7 @@ obj-$(CONFIG_BALLOON) +=3D balloon.o obj-$(CONFIG_PAGE_EXTENSION) +=3D page_ext.o obj-$(CONFIG_PAGE_TABLE_CHECK) +=3D page_table_check.o obj-$(CONFIG_DEBUG_PAGE_CONSISTENCY) +=3D page_consistency.o +obj-$(CONFIG_DEBUG_PAGE_CONSISTENCY_KUNIT_TEST) +=3D page_consistency_test= .o obj-$(CONFIG_CMA_DEBUGFS) +=3D cma_debug.o obj-$(CONFIG_SECRETMEM) +=3D secretmem.o obj-$(CONFIG_CMA_SYSFS) +=3D cma_sysfs.o diff --git a/mm/page_consistency_test.c b/mm/page_consistency_test.c new file mode 100644 index 000000000000..6cd587f8146f --- /dev/null +++ b/mm/page_consistency_test.c @@ -0,0 +1,274 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit tests for dual-bitmap primitives + * + * Tests the dual-bitmap consistency checking algorithm used by the page + * consistency checker. These tests verify the core invariant maintenance + * and corruption detection logic. + */ + +#include +#include + +#define TEST_BITMAP_BITS 256 + +struct dual_bitmap_test_context { + struct dual_bitmap db; + unsigned long primary[BITS_TO_LONGS(TEST_BITMAP_BITS)]; + unsigned long secondary[BITS_TO_LONGS(TEST_BITMAP_BITS)]; +}; + +static int dual_bitmap_test_init(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx; + + ctx =3D kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + if (!ctx) + return -ENOMEM; + + ctx->db.bitmap[DUAL_BITMAP_PRIMARY] =3D ctx->primary; + ctx->db.bitmap[DUAL_BITMAP_SECONDARY] =3D ctx->secondary; + ctx->db.nbits =3D TEST_BITMAP_BITS; + + /* Initialize: primary all zeros, secondary all ones */ + dual_bitmap_init(&ctx->db); + + test->priv =3D ctx; + return 0; +} + +static void test_initial_state_consistent(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + unsigned long violations; + + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); +} + +static void test_set_maintains_consistency(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + unsigned long violations; + bool was_set; + + /* Set bit 42 */ + was_set =3D dual_bitmap_set(&ctx->db, 42); + KUNIT_EXPECT_FALSE(test, was_set); + + /* Verify consistency */ + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); + + /* Verify individual bit consistency */ + KUNIT_EXPECT_TRUE(test, dual_bitmap_consistent(&ctx->db, 42)); +} + +static void test_clear_maintains_consistency(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + unsigned long violations; + bool was_set; + + /* First set the bit */ + dual_bitmap_set(&ctx->db, 100); + + /* Now clear it */ + was_set =3D dual_bitmap_clear(&ctx->db, 100); + KUNIT_EXPECT_TRUE(test, was_set); + + /* Verify consistency */ + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); +} + +static void test_double_set_detected(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + bool was_set; + + /* Set bit 50 */ + was_set =3D dual_bitmap_set(&ctx->db, 50); + KUNIT_EXPECT_FALSE(test, was_set); + + /* Try to set it again - should report it was already set */ + was_set =3D dual_bitmap_set(&ctx->db, 50); + KUNIT_EXPECT_TRUE(test, was_set); +} + +static void test_double_clear_detected(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + bool was_set; + + /* Clear bit 60 which is already clear (never set) */ + was_set =3D dual_bitmap_clear(&ctx->db, 60); + KUNIT_EXPECT_FALSE(test, was_set); +} + +static void test_corruption_in_primary_detected(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + unsigned long violations; + + /* Corrupt the primary bitmap directly */ + set_bit(75, ctx->primary); + + /* Validation should detect the corruption */ + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_GT(test, violations, 0UL); + + /* Individual bit check should also fail */ + KUNIT_EXPECT_FALSE(test, dual_bitmap_consistent(&ctx->db, 75)); +} + +static void test_corruption_in_secondary_detected(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + unsigned long violations; + + /* Corrupt the secondary bitmap directly */ + clear_bit(80, ctx->secondary); + + /* Validation should detect the corruption */ + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_GT(test, violations, 0UL); + + /* Individual bit check should also fail */ + KUNIT_EXPECT_FALSE(test, dual_bitmap_consistent(&ctx->db, 80)); +} + +static void test_multiple_operations(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + unsigned long violations; + unsigned long i; + + /* Set bits 0-63 */ + for (i =3D 0; i < 64; i++) + dual_bitmap_set(&ctx->db, i); + + /* Clear bits 32-63 */ + for (i =3D 32; i < 64; i++) + dual_bitmap_clear(&ctx->db, i); + + /* Validate entire bitmap */ + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); + + /* Verify expected state: bits 0-31 set, rest clear */ + for (i =3D 0; i < 32; i++) + KUNIT_EXPECT_TRUE(test, test_bit(i, ctx->primary)); + for (i =3D 32; i < TEST_BITMAP_BITS; i++) + KUNIT_EXPECT_FALSE(test, test_bit(i, ctx->primary)); +} + +static void test_boundary_bits(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + unsigned long violations; + + /* Test first bit */ + dual_bitmap_set(&ctx->db, 0); + KUNIT_EXPECT_TRUE(test, dual_bitmap_consistent(&ctx->db, 0)); + + /* Test last bit */ + dual_bitmap_set(&ctx->db, TEST_BITMAP_BITS - 1); + KUNIT_EXPECT_TRUE(test, dual_bitmap_consistent(&ctx->db, TEST_BITMAP_BITS= - 1)); + + /* Test word boundary (last bit of first word / first bit of second word)= */ + dual_bitmap_set(&ctx->db, BITS_PER_LONG - 1); + dual_bitmap_set(&ctx->db, BITS_PER_LONG); + KUNIT_EXPECT_TRUE(test, dual_bitmap_consistent(&ctx->db, BITS_PER_LONG - = 1)); + KUNIT_EXPECT_TRUE(test, dual_bitmap_consistent(&ctx->db, BITS_PER_LONG)); + + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); +} + +static void test_dual_bitmap_test_func(struct kunit *test) +{ + struct dual_bitmap_test_context *ctx =3D test->priv; + + /* Initially all bits should be clear (not allocated) */ + KUNIT_EXPECT_FALSE(test, dual_bitmap_test(&ctx->db, 10)); + + /* After setting, bit should be set */ + dual_bitmap_set(&ctx->db, 10); + KUNIT_EXPECT_TRUE(test, dual_bitmap_test(&ctx->db, 10)); + + /* After clearing, bit should be clear again */ + dual_bitmap_clear(&ctx->db, 10); + KUNIT_EXPECT_FALSE(test, dual_bitmap_test(&ctx->db, 10)); +} + +/* Test with non-word-aligned nbits to exercise partial-word handling */ +#define TEST_UNALIGNED_BITS 100 /* not a multiple of BITS_PER_LONG */ + +struct dual_bitmap_unaligned_context { + struct dual_bitmap db; + unsigned long primary[BITS_TO_LONGS(TEST_UNALIGNED_BITS)]; + unsigned long secondary[BITS_TO_LONGS(TEST_UNALIGNED_BITS)]; +}; + +static void test_non_aligned_nbits(struct kunit *test) +{ + struct dual_bitmap_unaligned_context *ctx; + unsigned long violations; + unsigned long i; + + ctx =3D kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx); + + ctx->db.bitmap[DUAL_BITMAP_PRIMARY] =3D ctx->primary; + ctx->db.bitmap[DUAL_BITMAP_SECONDARY] =3D ctx->secondary; + ctx->db.nbits =3D TEST_UNALIGNED_BITS; + + dual_bitmap_init(&ctx->db); + + /* Initial state should be consistent */ + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); + + /* Set and clear bits near the non-aligned boundary */ + for (i =3D TEST_UNALIGNED_BITS - 5; i < TEST_UNALIGNED_BITS; i++) { + dual_bitmap_set(&ctx->db, i); + KUNIT_EXPECT_TRUE(test, dual_bitmap_consistent(&ctx->db, i)); + } + + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); + + /* Clear them back */ + for (i =3D TEST_UNALIGNED_BITS - 5; i < TEST_UNALIGNED_BITS; i++) + dual_bitmap_clear(&ctx->db, i); + + violations =3D dual_bitmap_validate(&ctx->db); + KUNIT_EXPECT_EQ(test, violations, 0UL); +} + +static struct kunit_case dual_bitmap_test_cases[] =3D { + KUNIT_CASE(test_initial_state_consistent), + KUNIT_CASE(test_set_maintains_consistency), + KUNIT_CASE(test_clear_maintains_consistency), + KUNIT_CASE(test_double_set_detected), + KUNIT_CASE(test_double_clear_detected), + KUNIT_CASE(test_corruption_in_primary_detected), + KUNIT_CASE(test_corruption_in_secondary_detected), + KUNIT_CASE(test_multiple_operations), + KUNIT_CASE(test_boundary_bits), + KUNIT_CASE(test_dual_bitmap_test_func), + KUNIT_CASE(test_non_aligned_nbits), + {}, +}; + +static struct kunit_suite dual_bitmap_test_suite =3D { + .name =3D "dual_bitmap", + .init =3D dual_bitmap_test_init, + .test_cases =3D dual_bitmap_test_cases, +}; + +kunit_test_suites(&dual_bitmap_test_suite); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("KUnit tests for dual-bitmap consistency primitives"); --=20 2.53.0