From nobody Tue Feb 10 01:14:43 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 797CC2980CD for ; Fri, 11 Apr 2025 09:17:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744363081; cv=none; b=UW7wqff7CroM3jOmB6bhHUInFou2uqIxjK7j4eBIc5SvbmgkJ66DpXUypOBYf6399fljukqHbpyqhUMu8CgvbUy+wd/iMue0wHNpifPxpqb0CyfuoopwojD5N6hNyrQD0PreZzzz6qINbu3IC6r2+95Unb8vdnDyMgyFy6wWT24= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744363081; c=relaxed/simple; bh=T4PHwd4xb690wqOB/sKABtQU4LmnjW69VzWcVu/Hc5g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=A+0FT4ZgMegukc6PtKhgrMeakRZyW5YjLcyfO+fkV9VBeO+vJ1U1KWeGtq1PNDJh2n7hAVMTKMZuvW/Lfnp9HxbfWZsk7bkQglKiigSHWEY+yDwMF8NvYvGFzifRMrUg15F/j/Bq2y9JlVF4PZJmWD1/ljlhNcoTShcuVc1FgMk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9EFFC106F; Fri, 11 Apr 2025 02:17:58 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D6AEE3F6A8; Fri, 11 Apr 2025 02:17:54 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 15/18] arm64: mm: Guard page table writes with kpkeys Date: Fri, 11 Apr 2025 10:16:28 +0100 Message-ID: <20250411091631.954228-16-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When CONFIG_KPKEYS_HARDENED_PGTABLES is enabled, page tables (both user and kernel) are mapped with a privileged pkey in the linear mapping. As a result, they can only be written at an elevated kpkeys level. Introduce a kpkeys guard that sets POR_EL1 appropriately to allow writing to page tables, and use this guard wherever necessary. The scope is kept as small as possible, so that POR_EL1 is quickly reset to its default value. Where atomics are involved, the guard's scope encompasses the whole loop to avoid switching POR_EL1 unnecessarily. This patch is a no-op if CONFIG_KPKEYS_HARDENED_PGTABLES is disabled (default). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/pgtable.h | 22 +++++++++++++++++++++- arch/arm64/mm/fault.c | 2 ++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 6c780c428a6d..7929b79cd6b1 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -39,6 +39,14 @@ #include #include #include +#include + +#ifdef CONFIG_KPKEYS_HARDENED_PGTABLES +KPKEYS_GUARD_COND(kpkeys_hardened_pgtables, KPKEYS_LVL_PGTABLES, + kpkeys_hardened_pgtables_enabled()) +#else +KPKEYS_GUARD_NOOP(kpkeys_hardened_pgtables) +#endif =20 static inline void emit_pte_barriers(void) { @@ -366,6 +374,7 @@ static inline pte_t pte_clear_uffd_wp(pte_t pte) =20 static inline void __set_pte_nosync(pte_t *ptep, pte_t pte) { + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*ptep, pte); } =20 @@ -843,6 +852,7 @@ static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) } #endif /* __PAGETABLE_PMD_FOLDED */ =20 + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pmdp, pmd); =20 if (pmd_valid(pmd)) @@ -909,6 +919,7 @@ static inline void set_pud(pud_t *pudp, pud_t pud) return; } =20 + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pudp, pud); =20 if (pud_valid(pud)) @@ -990,6 +1001,7 @@ static inline void set_p4d(p4d_t *p4dp, p4d_t p4d) return; } =20 + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*p4dp, p4d); queue_pte_barriers(); } @@ -1118,6 +1130,7 @@ static inline void set_pgd(pgd_t *pgdp, pgd_t pgd) return; } =20 + guard(kpkeys_hardened_pgtables)(); WRITE_ONCE(*pgdp, pgd); queue_pte_barriers(); } @@ -1317,6 +1330,7 @@ static inline int __ptep_test_and_clear_young(struct = vm_area_struct *vma, { pte_t old_pte, pte; =20 + guard(kpkeys_hardened_pgtables)(); pte =3D __ptep_get(ptep); do { old_pte =3D pte; @@ -1363,7 +1377,10 @@ static inline int pmdp_test_and_clear_young(struct v= m_area_struct *vma, static inline pte_t ptep_get_and_clear_anysz(struct mm_struct *mm, pte_t *= ptep, unsigned long pgsize) { - pte_t pte =3D __pte(xchg_relaxed(&pte_val(*ptep), 0)); + pte_t pte; + + scoped_guard(kpkeys_hardened_pgtables) + pte =3D __pte(xchg_relaxed(&pte_val(*ptep), 0)); =20 switch (pgsize) { case PAGE_SIZE: @@ -1434,6 +1451,7 @@ static inline void ___ptep_set_wrprotect(struct mm_st= ruct *mm, { pte_t old_pte; =20 + guard(kpkeys_hardened_pgtables)(); do { old_pte =3D pte; pte =3D pte_wrprotect(pte); @@ -1467,6 +1485,7 @@ static inline void __clear_young_dirty_pte(struct vm_= area_struct *vma, { pte_t old_pte; =20 + guard(kpkeys_hardened_pgtables)(); do { old_pte =3D pte; =20 @@ -1514,6 +1533,7 @@ static inline pmd_t pmdp_establish(struct vm_area_str= uct *vma, unsigned long address, pmd_t *pmdp, pmd_t pmd) { page_table_check_pmd_set(vma->vm_mm, pmdp, pmd); + guard(kpkeys_hardened_pgtables)(); return __pmd(xchg_relaxed(&pmd_val(*pmdp), pmd_val(pmd))); } #endif diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index ec0a337891dd..5681553e2db9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -220,6 +220,8 @@ int __ptep_set_access_flags(struct vm_area_struct *vma, if (pte_same(pte, entry)) return 0; =20 + guard(kpkeys_hardened_pgtables)(); + /* only preserve the access flags and write permission */ pte_val(entry) &=3D PTE_RDONLY | PTE_AF | PTE_WRITE | PTE_DIRTY; =20 --=20 2.47.0