From nobody Thu Sep 24 15:10:58 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D89723E5592; Tue, 22 Sep 2026 17:32:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098384; cv=none; b=Yoe1gLqRJ9V8ADruaH13aakYL+wluVsjaHjpLJ1qcKU6OKaWoxBeUiMLKaVgYjSAou008GctWgpHpnhXWg/KdhW6xQE+AdVW5e3noi9g+OwG5SnVdgnyS9r51M5ca/YqC2y2e7TKkr0IbaKNwVLpKeGduGAeuv45I+hU9tzCgyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098384; c=relaxed/simple; bh=hIZ/hiH2SCX47+Q9tkaaUal44tf7eoSyrdUhb3hE0eI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oazUAekwIT4CyzStc77pmQeUAvr6Ksiil0ep5WPXtx3bddVyv2Oit6BeVOK9sayVkcIqBihMZcbxaAMQ/v9ErqNIG4NUIzEOEu5WxLtGKxZLuS+mdgkqqHSqwQ1dkkTf0R8FmeMphTDn1n/SHkHY8VWOC3Li4JZ0X/P1R1fPZSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=e0Ddf46c; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="e0Ddf46c" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 87D8F1595; Tue, 22 Sep 2026 10:32:50 -0700 (PDT) Received: from [127.0.1.1] (e142334-100.cambridge.arm.com [10.2.198.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 604A43F86C; Tue, 22 Sep 2026 10:32:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1790098374; bh=hIZ/hiH2SCX47+Q9tkaaUal44tf7eoSyrdUhb3hE0eI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=e0Ddf46cdbBKTsIXQoDqYQ1biN6PpHlZlKsAObtO3Ox9C8LMNj6L6vFxBiwd39DrM hNakk2zXKnMhkbdptEJkCN6cwdIM0GvBKgAMxVwwMn8ZrmrN/QVOL/fp+BmtsV4gEt jSQS1AXE2J1l/vkWJ2tR3RFEXXRE1rrTaMnmmRrQ= From: Muhammad Usama Anjum Date: Tue, 22 Sep 2026 18:31:56 +0100 Subject: [PATCH v2 1/6] arm64: use hw_pte_t for HW PTE pointers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-pte0_arm-v2-1-a3f1ddff0a8a@arm.com> References: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> In-Reply-To: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> To: Catalin Marinas , Will Deacon , Mark Rutland , Ryan Roberts , Ard Biesheuvel , Ilias Apalodimas , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, Muhammad Usama Anjum X-Mailer: b4 0.16.0 With pte_t * used for HW PTE pointers, the compiler cannot distinguish them from SW PTE value pointers. The generic HW PTE interfaces already use hw_pte_t *, so arm64 HW PTE pointers must use that type before ARCH_HAS_HW_PTE_T can make it distinct. Convert HW PTE pointers to hw_pte_t *, retaining pte_t for SW PTE values. Helpers shared with PMD and PUD operations keep explicit casts because this series does not introduce distinct entry types for those levels. arm64 still uses the hw_pte_t alias in this patch. The new pointer spelling therefore denotes the same type until the final opt-in. Include linux/pgtable_types.h in pi.h to make hw_pte_t visible. Signed-off-by: Muhammad Usama Anjum Reviewed-by: Ryan Roberts --- arch/arm64/include/asm/hugetlb.h | 19 +++++---- arch/arm64/include/asm/pgalloc.h | 2 +- arch/arm64/include/asm/pgtable.h | 143 ++++++++++++++++++++++++++++++++++= ------------------------------ arch/arm64/include/asm/vmalloc.h | 2 +- arch/arm64/kernel/efi.c | 3 +- arch/arm64/kernel/pi/map_kernel.c | 4 +- arch/arm64/kernel/pi/map_range.c | 9 ++-- arch/arm64/kernel/pi/pi.h | 3 +- arch/arm64/mm/contpte.c | 49 +++++++++++----------- arch/arm64/mm/fault.c | 5 ++- arch/arm64/mm/fixmap.c | 6 +-- arch/arm64/mm/hugetlbpage.c | 48 +++++++++++----------- arch/arm64/mm/kasan_init.c | 4 +- arch/arm64/mm/mmu.c | 46 ++++++++++++--------- arch/arm64/mm/pageattr.c | 4 +- arch/arm64/mm/trans_pgd.c | 4 +- 16 files changed, 190 insertions(+), 161 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/huge= tlb.h index d038ff14d16ca..15b3cf53aa2ca 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -36,33 +36,34 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shif= t, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz); + hw_pte_t *ptep, pte_t pte, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS extern int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t pte, int dirty); #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long a= ddr, - pte_t *ptep, unsigned long sz); + hw_pte_t *ptep, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT extern void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep); + unsigned long addr, hw_pte_t *ptep); #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); + unsigned long addr, hw_pte_t *ptep); #define __HAVE_ARCH_HUGE_PTE_CLEAR extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz); + hw_pte_t *ptep, unsigned long sz); #define __HAVE_ARCH_HUGE_PTEP_GET -extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t= *ptep); +extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, + hw_pte_t *ptep); =20 #define huge_ptep_modify_prot_start huge_ptep_modify_prot_start extern pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); + unsigned long addr, hw_pte_t *ptep); =20 #define huge_ptep_modify_prot_commit huge_ptep_modify_prot_commit extern void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t old_pte, pte_t new_pte); =20 #include diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgal= loc.h index 1b4509d3382c6..80063dd4d2722 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -106,7 +106,7 @@ static inline void __pmd_populate(pmd_t *pmdp, phys_add= r_t ptep, * of the mm address space. */ static inline void -pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, pte_t *ptep) +pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, hw_pte_t *ptep) { VM_BUG_ON(mm && mm !=3D &init_mm); __pmd_populate(pmdp, __pa(ptep), diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index e89ec5f4787b4..652ce413be389 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -358,7 +358,7 @@ static inline pte_t pte_clear_uffd(pte_t pte) } #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ =20 -static inline void __set_pte_nosync(pte_t *ptep, pte_t pte) +static inline void __set_pte_nosync(hw_pte_t *ptep, pte_t pte) { WRITE_ONCE(*ptep, pte); } @@ -373,13 +373,13 @@ static inline void __set_pte_complete(pte_t pte) queue_pte_barriers(); } =20 -static inline void __set_pte(pte_t *ptep, pte_t pte) +static inline void __set_pte(hw_pte_t *ptep, pte_t pte) { __set_pte_nosync(ptep, pte); __set_pte_complete(pte); } =20 -static inline pte_t __ptep_get(pte_t *ptep) +static inline pte_t __ptep_get(hw_pte_t *ptep) { return READ_ONCE(*ptep); } @@ -403,7 +403,8 @@ bool pgattr_change_is_safe(pteval_t old, pteval_t new); * PTE_DIRTY || (PTE_WRITE && !PTE_RDONLY) */ =20 -static inline void __check_safe_pte_update(struct mm_struct *mm, pte_t *pt= ep, +static inline void __check_safe_pte_update(struct mm_struct *mm, + hw_pte_t *ptep, pte_t pte) { pte_t old_pte; @@ -669,7 +670,8 @@ static inline pgprot_t pud_pgprot(pud_t pud) } =20 static inline void __set_ptes_anysz(struct mm_struct *mm, unsigned long ad= dr, - pte_t *ptep, pte_t pte, unsigned int nr, + hw_pte_t *ptep, pte_t pte, + unsigned int nr, unsigned long pgsize) { unsigned long stride =3D pgsize >> PAGE_SHIFT; @@ -707,7 +709,7 @@ static inline void __set_ptes_anysz(struct mm_struct *m= m, unsigned long addr, } =20 static inline void __set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned int nr) + hw_pte_t *ptep, pte_t pte, unsigned int nr) { __set_ptes_anysz(mm, addr, ptep, pte, nr, PAGE_SIZE); } @@ -715,14 +717,16 @@ static inline void __set_ptes(struct mm_struct *mm, u= nsigned long addr, static inline void __set_pmds(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd, unsigned int nr) { - __set_ptes_anysz(mm, addr, (pte_t *)pmdp, pmd_pte(pmd), nr, PMD_SIZE); + __set_ptes_anysz(mm, addr, (hw_pte_t *)pmdp, pmd_pte(pmd), nr, + PMD_SIZE); } #define set_pmd_at(mm, addr, pmdp, pmd) __set_pmds(mm, addr, pmdp, pmd, 1) =20 static inline void __set_puds(struct mm_struct *mm, unsigned long addr, pud_t *pudp, pud_t pud, unsigned int nr) { - __set_ptes_anysz(mm, addr, (pte_t *)pudp, pud_pte(pud), nr, PUD_SIZE); + __set_ptes_anysz(mm, addr, (hw_pte_t *)pudp, pud_pte(pud), nr, + PUD_SIZE); } #define set_pud_at(mm, addr, pudp, pud) __set_puds(mm, addr, pudp, pud, 1) =20 @@ -850,16 +854,16 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) } =20 /* Find an entry in the third-level page table. */ -#define pte_offset_phys(dir,addr) (pmd_page_paddr(READ_ONCE(*(dir))) + pte= _index(addr) * sizeof(pte_t)) +#define pte_offset_phys(dir, addr) (pmd_page_paddr(READ_ONCE(*(dir))) + pt= e_index(addr) * sizeof(hw_pte_t)) =20 -#define pte_set_fixmap(addr) ((pte_t *)set_fixmap_offset(FIX_PTE, addr)) +#define pte_set_fixmap(addr) ((hw_pte_t *)set_fixmap_offset(FIX_PTE, addr= )) #define pte_set_fixmap_offset(pmd, addr) pte_set_fixmap(pte_offset_phys(pm= d, addr)) #define pte_clear_fixmap() clear_fixmap(FIX_PTE) =20 #define pmd_page(pmd) phys_to_page(__pmd_to_phys(pmd)) =20 /* use ONLY for statically allocated translation tables */ -#define pte_offset_kimg(dir,addr) ((pte_t *)__phys_to_kimg(pte_offset_phys= ((dir), (addr)))) +#define pte_offset_kimg(dir, addr) ((hw_pte_t *)__phys_to_kimg(pte_offset_= phys((dir), (addr)))) =20 #if CONFIG_PGTABLE_LEVELS > 2 =20 @@ -1238,12 +1242,14 @@ static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t = newprot) } =20 extern int __ptep_set_access_flags_anysz(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, + unsigned long address, + hw_pte_t *ptep, pte_t entry, int dirty, unsigned long pgsize); =20 static inline int __ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, + unsigned long address, + hw_pte_t *ptep, pte_t entry, int dirty) { return __ptep_set_access_flags_anysz(vma, address, ptep, entry, dirty, @@ -1256,7 +1262,7 @@ static inline int pmdp_set_access_flags(struct vm_are= a_struct *vma, unsigned long address, pmd_t *pmdp, pmd_t entry, int dirty) { - return __ptep_set_access_flags_anysz(vma, address, (pte_t *)pmdp, + return __ptep_set_access_flags_anysz(vma, address, (hw_pte_t *)pmdp, pmd_pte(entry), dirty, PMD_SIZE); } #endif @@ -1283,13 +1289,13 @@ static inline bool pud_user_accessible_page(struct = mm_struct *mm, unsigned long */ =20 static inline void __pte_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { __set_pte(ptep, __pte(0)); } =20 static inline bool __ptep_test_and_clear_young(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep) + unsigned long address, hw_pte_t *ptep) { pte_t old_pte, pte; =20 @@ -1305,7 +1311,7 @@ static inline bool __ptep_test_and_clear_young(struct= vm_area_struct *vma, } =20 static inline bool __ptep_clear_flush_young(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep) + unsigned long address, hw_pte_t *ptep) { bool young =3D __ptep_test_and_clear_young(vma, address, ptep); =20 @@ -1331,13 +1337,13 @@ static inline bool pmdp_test_and_clear_young(struct= vm_area_struct *vma, { /* Operation applies to PMD table entry only if FEAT_HAFT is enabled */ VM_WARN_ON(pmd_table(READ_ONCE(*pmdp)) && !system_supports_haft()); - return __ptep_test_and_clear_young(vma, address, (pte_t *)pmdp); + return __ptep_test_and_clear_young(vma, address, (hw_pte_t *)pmdp); } #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG= */ =20 static inline pte_t __ptep_get_and_clear_anysz(struct mm_struct *mm, unsigned long address, - pte_t *ptep, + hw_pte_t *ptep, unsigned long pgsize) { pte_t pte =3D __pte(xchg_relaxed(&pte_val(*ptep), 0)); @@ -1362,13 +1368,13 @@ static inline pte_t __ptep_get_and_clear_anysz(stru= ct mm_struct *mm, } =20 static inline pte_t __ptep_get_and_clear(struct mm_struct *mm, - unsigned long address, pte_t *ptep) + unsigned long address, hw_pte_t *ptep) { return __ptep_get_and_clear_anysz(mm, address, ptep, PAGE_SIZE); } =20 static inline void __clear_full_ptes(struct mm_struct *mm, unsigned long a= ddr, - pte_t *ptep, unsigned int nr, int full) + hw_pte_t *ptep, unsigned int nr, int full) { for (;;) { __ptep_get_and_clear(mm, addr, ptep); @@ -1380,7 +1386,7 @@ static inline void __clear_full_ptes(struct mm_struct= *mm, unsigned long addr, } =20 static inline pte_t __get_and_clear_full_ptes(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr, int full) { pte_t pte, tmp_pte; @@ -1403,12 +1409,12 @@ static inline pte_t __get_and_clear_full_ptes(struc= t mm_struct *mm, static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long address, pmd_t *pmdp) { - return pte_pmd(__ptep_get_and_clear_anysz(mm, address, (pte_t *)pmdp, PMD= _SIZE)); + return pte_pmd(__ptep_get_and_clear_anysz(mm, address, (hw_pte_t *)pmdp, = PMD_SIZE)); } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 static inline void ___ptep_set_wrprotect(struct mm_struct *mm, - unsigned long address, pte_t *ptep, + unsigned long address, hw_pte_t *ptep, pte_t pte) { pte_t old_pte; @@ -1426,13 +1432,13 @@ static inline void ___ptep_set_wrprotect(struct mm_= struct *mm, * dirty status (PTE_DBM && !PTE_RDONLY) to the software PTE_DIRTY bit. */ static inline void __ptep_set_wrprotect(struct mm_struct *mm, - unsigned long address, pte_t *ptep) + unsigned long address, hw_pte_t *ptep) { ___ptep_set_wrprotect(mm, address, ptep, __ptep_get(ptep)); } =20 static inline void __wrprotect_ptes(struct mm_struct *mm, unsigned long ad= dress, - pte_t *ptep, unsigned int nr) + hw_pte_t *ptep, unsigned int nr) { unsigned int i; =20 @@ -1441,7 +1447,7 @@ static inline void __wrprotect_ptes(struct mm_struct = *mm, unsigned long address, } =20 static inline void __clear_young_dirty_pte(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t pte, cydp_t flags) { pte_t old_pte; @@ -1460,7 +1466,8 @@ static inline void __clear_young_dirty_pte(struct vm_= area_struct *vma, } =20 static inline void __clear_young_dirty_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, + hw_pte_t *ptep, unsigned int nr, cydp_t flags) { pte_t pte; @@ -1485,7 +1492,7 @@ static inline void __clear_young_dirty_ptes(struct vm= _area_struct *vma, static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long address, pmd_t *pmdp) { - __ptep_set_wrprotect(mm, address, (pte_t *)pmdp); + __ptep_set_wrprotect(mm, address, (hw_pte_t *)pmdp); } =20 #define pmdp_establish pmdp_establish @@ -1558,7 +1565,8 @@ extern void arch_swap_restore(swp_entry_t entry, stru= ct folio *folio); * On AArch64, the cache coherency is handled via the __set_ptes() functio= n. */ static inline void update_mmu_cache_range(struct vm_fault *vmf, - struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, + struct vm_area_struct *vma, unsigned long addr, + hw_pte_t *ptep, unsigned int nr) { /* @@ -1613,21 +1621,21 @@ static inline bool pud_sect_supported(void) #define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION #define ptep_modify_prot_start ptep_modify_prot_start extern pte_t ptep_modify_prot_start(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep); + unsigned long addr, hw_pte_t *ptep); =20 #define ptep_modify_prot_commit ptep_modify_prot_commit extern void ptep_modify_prot_commit(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t old_pte, pte_t new_pte); =20 #define modify_prot_start_ptes modify_prot_start_ptes extern pte_t modify_prot_start_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr); =20 #define modify_prot_commit_ptes modify_prot_commit_ptes extern void modify_prot_commit_ptes(struct vm_area_struct *vma, unsigned l= ong addr, - pte_t *ptep, pte_t old_pte, pte_t pte, + hw_pte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr); =20 #ifdef CONFIG_ARM64_CONTPTE @@ -1638,33 +1646,35 @@ extern void modify_prot_commit_ptes(struct vm_area_= struct *vma, unsigned long ad * a private implementation detail of the public ptep API (see below). */ extern void __contpte_try_fold(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte); + hw_pte_t *ptep, pte_t pte); extern void __contpte_try_unfold(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte); -extern pte_t contpte_ptep_get(pte_t *ptep, pte_t orig_pte); -extern pte_t contpte_ptep_get_lockless(pte_t *orig_ptep); + hw_pte_t *ptep, pte_t pte); +extern pte_t contpte_ptep_get(hw_pte_t *ptep, pte_t orig_pte); +extern pte_t contpte_ptep_get_lockless(hw_pte_t *orig_ptep); extern void contpte_set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned int nr); + hw_pte_t *ptep, pte_t pte, unsigned int nr); extern void contpte_clear_full_ptes(struct mm_struct *mm, unsigned long ad= dr, - pte_t *ptep, unsigned int nr, int full); + hw_pte_t *ptep, unsigned int nr, int full); extern pte_t contpte_get_and_clear_full_ptes(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr, int full); bool contpte_test_and_clear_young_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned int nr); + unsigned long addr, hw_pte_t *ptep, + unsigned int nr); bool contpte_clear_flush_young_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned int nr); + unsigned long addr, hw_pte_t *ptep, + unsigned int nr); extern void contpte_wrprotect_ptes(struct mm_struct *mm, unsigned long add= r, - pte_t *ptep, unsigned int nr); + hw_pte_t *ptep, unsigned int nr); extern int contpte_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t entry, int dirty); extern void contpte_clear_young_dirty_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr, cydp_t flags); =20 static __always_inline void contpte_try_fold(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, pte_t pte) + unsigned long addr, hw_pte_t *ptep, pte_t pte) { /* * Only bother trying if both the virtual and physical addresses are @@ -1687,14 +1697,14 @@ static __always_inline void contpte_try_fold(struct= mm_struct *mm, } =20 static __always_inline void contpte_try_unfold(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, pte_t pte) + unsigned long addr, hw_pte_t *ptep, pte_t pte) { if (unlikely(pte_valid_cont(pte))) __contpte_try_unfold(mm, addr, ptep, pte); } =20 #define pte_batch_hint pte_batch_hint -static inline unsigned int pte_batch_hint(pte_t *ptep, pte_t pte) +static inline unsigned int pte_batch_hint(hw_pte_t *ptep, pte_t pte) { if (!pte_valid_cont(pte)) return 1; @@ -1721,7 +1731,7 @@ static inline unsigned int pte_batch_hint(pte_t *ptep= , pte_t pte) */ =20 #define ptep_get ptep_get -static inline pte_t ptep_get(pte_t *ptep) +static inline pte_t ptep_get(hw_pte_t *ptep) { pte_t pte =3D __ptep_get(ptep); =20 @@ -1732,7 +1742,7 @@ static inline pte_t ptep_get(pte_t *ptep) } =20 #define ptep_get_lockless ptep_get_lockless -static inline pte_t ptep_get_lockless(pte_t *ptep) +static inline pte_t ptep_get_lockless(hw_pte_t *ptep) { pte_t pte =3D __ptep_get(ptep); =20 @@ -1742,7 +1752,7 @@ static inline pte_t ptep_get_lockless(pte_t *ptep) return contpte_ptep_get_lockless(ptep); } =20 -static inline void set_pte(pte_t *ptep, pte_t pte) +static inline void set_pte(hw_pte_t *ptep, pte_t pte) { /* * We don't have the mm or vaddr so cannot unfold contig entries (since @@ -1759,7 +1769,7 @@ static inline void set_pte(pte_t *ptep, pte_t pte) =20 #define set_ptes set_ptes static __always_inline void set_ptes(struct mm_struct *mm, unsigned long a= ddr, - pte_t *ptep, pte_t pte, unsigned int nr) + hw_pte_t *ptep, pte_t pte, unsigned int nr) { pte =3D pte_mknoncont(pte); =20 @@ -1773,7 +1783,7 @@ static __always_inline void set_ptes(struct mm_struct= *mm, unsigned long addr, } =20 static inline void pte_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { contpte_try_unfold(mm, addr, ptep, __ptep_get(ptep)); __pte_clear(mm, addr, ptep); @@ -1781,7 +1791,7 @@ static inline void pte_clear(struct mm_struct *mm, =20 #define clear_full_ptes clear_full_ptes static inline void clear_full_ptes(struct mm_struct *mm, unsigned long add= r, - pte_t *ptep, unsigned int nr, int full) + hw_pte_t *ptep, unsigned int nr, int full) { if (likely(nr =3D=3D 1)) { contpte_try_unfold(mm, addr, ptep, __ptep_get(ptep)); @@ -1793,7 +1803,7 @@ static inline void clear_full_ptes(struct mm_struct *= mm, unsigned long addr, =20 #define get_and_clear_full_ptes get_and_clear_full_ptes static inline pte_t get_and_clear_full_ptes(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr, int full) { pte_t pte; @@ -1810,13 +1820,13 @@ static inline pte_t get_and_clear_full_ptes(struct = mm_struct *mm, =20 #define __HAVE_ARCH_PTEP_GET_AND_CLEAR static inline pte_t ptep_get_and_clear(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { contpte_try_unfold(mm, addr, ptep, __ptep_get(ptep)); return __ptep_get_and_clear(mm, addr, ptep); } =20 -static inline bool ptep_try_set(pte_t *ptep, pte_t new_pte) +static inline bool ptep_try_set(hw_pte_t *ptep, pte_t new_pte) { pteval_t old =3D 0; =20 @@ -1846,7 +1856,7 @@ static inline void flush_tlb_before_set(unsigned long= addr) =20 #define test_and_clear_young_ptes test_and_clear_young_ptes static inline bool test_and_clear_young_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned int nr) + unsigned long addr, hw_pte_t *ptep, unsigned int nr) { if (likely(nr =3D=3D 1 && !pte_cont(__ptep_get(ptep)))) return __ptep_test_and_clear_young(vma, addr, ptep); @@ -1856,14 +1866,14 @@ static inline bool test_and_clear_young_ptes(struct= vm_area_struct *vma, =20 #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG static inline bool ptep_test_and_clear_young(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { return test_and_clear_young_ptes(vma, addr, ptep, 1); } =20 #define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH static inline bool ptep_clear_flush_young(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { pte_t orig_pte =3D __ptep_get(ptep); =20 @@ -1875,7 +1885,7 @@ static inline bool ptep_clear_flush_young(struct vm_a= rea_struct *vma, =20 #define clear_flush_young_ptes clear_flush_young_ptes static inline bool clear_flush_young_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned int nr) + unsigned long addr, hw_pte_t *ptep, unsigned int nr) { if (likely(nr =3D=3D 1 && !pte_cont(__ptep_get(ptep)))) return __ptep_clear_flush_young(vma, addr, ptep); @@ -1885,7 +1895,8 @@ static inline bool clear_flush_young_ptes(struct vm_a= rea_struct *vma, =20 #define wrprotect_ptes wrprotect_ptes static __always_inline void wrprotect_ptes(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, unsigned int nr) + unsigned long addr, hw_pte_t *ptep, + unsigned int nr) { if (likely(nr =3D=3D 1)) { /* @@ -1909,14 +1920,14 @@ static __always_inline void wrprotect_ptes(struct m= m_struct *mm, =20 #define __HAVE_ARCH_PTEP_SET_WRPROTECT static inline void ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { wrprotect_ptes(mm, addr, ptep, 1); } =20 #define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS static inline int ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t entry, int dirty) { pte_t orig_pte =3D __ptep_get(ptep); @@ -1931,7 +1942,7 @@ static inline int ptep_set_access_flags(struct vm_are= a_struct *vma, =20 #define clear_young_dirty_ptes clear_young_dirty_ptes static inline void clear_young_dirty_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr, cydp_t flags) { if (likely(nr =3D=3D 1 && !pte_cont(__ptep_get(ptep)))) diff --git a/arch/arm64/include/asm/vmalloc.h b/arch/arm64/include/asm/vmal= loc.h index 4ec1acd3c1b34..28d3e89f90e43 100644 --- a/arch/arm64/include/asm/vmalloc.h +++ b/arch/arm64/include/asm/vmalloc.h @@ -45,7 +45,7 @@ static inline unsigned long arch_vmap_pte_range_map_size(= unsigned long addr, =20 #define arch_vmap_pte_range_unmap_size arch_vmap_pte_range_unmap_size static inline unsigned long arch_vmap_pte_range_unmap_size(unsigned long a= ddr, - pte_t *ptep) + hw_pte_t *ptep) { /* * The caller handles alignment so it's sufficient just to check diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index 0ec90fd1754ed..68ff21310604d 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -111,7 +111,8 @@ struct set_perm_data { bool has_bti; }; =20 -static int __init set_permissions(pte_t *ptep, unsigned long addr, void *d= ata) +static int __init set_permissions(hw_pte_t *ptep, unsigned long addr, + void *data) { struct set_perm_data *spd =3D data; const efi_memory_desc_t *md =3D spd->md; diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_k= ernel.c index fb44cbdd2f292..2b7e7be5fd6f3 100644 --- a/arch/arm64/kernel/pi/map_kernel.c +++ b/arch/arm64/kernel/pi/map_kernel.c @@ -26,7 +26,7 @@ static void __init map_segment(pgd_t *pg_dir, phys_addr_t= *pgd, u64 va_offset, { map_range(pgd, ((u64)start + va_offset) & ~PAGE_OFFSET, ((u64)end + va_offset) & ~PAGE_OFFSET, (u64)start, - prot, root_level, (pte_t *)pg_dir, may_use_cont, 0); + prot, root_level, (hw_pte_t *)pg_dir, may_use_cont, 0); } =20 static void __init unmap_segment(pgd_t *pg_dir, u64 va_offset, void *start, @@ -209,7 +209,7 @@ static void *__init map_fdt(phys_addr_t fdt) */ map_range(&ptep, fdt, (u64)_text > fdt ? min((u64)_text, efdt) : efdt, fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL, - (pte_t *)init_idmap_pg_dir, false, 0); + (hw_pte_t *)init_idmap_pg_dir, false, 0); dsb(ishst); =20 return (void *)fdt; diff --git a/arch/arm64/kernel/pi/map_range.c b/arch/arm64/kernel/pi/map_ra= nge.c index 761b14893f748..94f06da76e333 100644 --- a/arch/arm64/kernel/pi/map_range.c +++ b/arch/arm64/kernel/pi/map_range.c @@ -27,7 +27,8 @@ * in the VA space */ void __init map_range(phys_addr_t *pte, u64 start, u64 end, phys_addr_t pa, - pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, + pgprot_t prot, int level, hw_pte_t *tbl, + bool may_use_cont, u64 va_offset) { u64 cmask =3D (level =3D=3D 3) ? CONT_PTE_SIZE - 1 : U64_MAX; @@ -62,7 +63,7 @@ void __init map_range(phys_addr_t *pte, u64 start, u64 en= d, phys_addr_t pa, *pte +=3D PTRS_PER_PTE * sizeof(pte_t); } map_range(pte, start, next, pa, prot, level + 1, - (pte_t *)(__pte_to_phys(*tbl) + va_offset), + (hw_pte_t *)(__pte_to_phys(*tbl) + va_offset), may_use_cont, va_offset); } else { /* @@ -100,10 +101,10 @@ asmlinkage phys_addr_t __init create_init_idmap(pgd_t= *pg_dir, ptval_t clrmask) /* MMU is off; pointer casts to phys_addr_t are safe */ map_range(&ptep, (u64)_stext, (u64)__initdata_begin, (phys_addr_t)_stext, text_prot, IDMAP_ROOT_LEVEL, - (pte_t *)pg_dir, false, 0); + (hw_pte_t *)pg_dir, false, 0); map_range(&ptep, (u64)__initdata_begin, (u64)_end, (phys_addr_t)__initdata_begin, data_prot, IDMAP_ROOT_LEVEL, - (pte_t *)pg_dir, false, 0); + (hw_pte_t *)pg_dir, false, 0); =20 return ptep; } diff --git a/arch/arm64/kernel/pi/pi.h b/arch/arm64/kernel/pi/pi.h index 5dfd8484d2005..a7d09a2fa8f9f 100644 --- a/arch/arm64/kernel/pi/pi.h +++ b/arch/arm64/kernel/pi/pi.h @@ -2,6 +2,7 @@ // Copyright 2023 Google LLC // Author: Ard Biesheuvel =20 +#include #include =20 #define __prel64_initconst __section(".init.rodata.prel64") @@ -30,7 +31,7 @@ void relocate_kernel(u64 offset); int scs_patch(const u8 eh_frame[], int size, bool skip_dry_run); =20 void map_range(phys_addr_t *pte, u64 start, u64 end, phys_addr_t pa, - pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, + pgprot_t prot, int level, hw_pte_t *tbl, bool may_use_cont, u64 va_offset); =20 asmlinkage void early_map_kernel(u64 boot_status, phys_addr_t fdt); diff --git a/arch/arm64/mm/contpte.c b/arch/arm64/mm/contpte.c index 0acab179fc1a5..753267c0b41e5 100644 --- a/arch/arm64/mm/contpte.c +++ b/arch/arm64/mm/contpte.c @@ -21,13 +21,14 @@ static inline bool mm_is_user(struct mm_struct *mm) return mm !=3D &init_mm; } =20 -static inline pte_t *contpte_align_down(pte_t *ptep) +static inline hw_pte_t *contpte_align_down(hw_pte_t *ptep) { return PTR_ALIGN_DOWN(ptep, sizeof(*ptep) * CONT_PTES); } =20 -static inline pte_t *contpte_align_addr_ptep(unsigned long *start, - unsigned long *end, pte_t *ptep, +static inline hw_pte_t *contpte_align_addr_ptep(unsigned long *start, + unsigned long *end, + hw_pte_t *ptep, unsigned int nr) { /* @@ -47,7 +48,7 @@ static inline pte_t *contpte_align_addr_ptep(unsigned lon= g *start, } =20 static void contpte_try_unfold_partial(struct mm_struct *mm, unsigned long= addr, - pte_t *ptep, unsigned int nr) + hw_pte_t *ptep, unsigned int nr) { /* * Unfold any partially covered contpte block at the beginning and end @@ -59,7 +60,7 @@ static void contpte_try_unfold_partial(struct mm_struct *= mm, unsigned long addr, =20 if (ptep + nr !=3D contpte_align_down(ptep + nr)) { unsigned long last_addr =3D addr + PAGE_SIZE * (nr - 1); - pte_t *last_ptep =3D ptep + nr - 1; + hw_pte_t *last_ptep =3D ptep + nr - 1; =20 contpte_try_unfold(mm, last_addr, last_ptep, __ptep_get(last_ptep)); @@ -67,11 +68,11 @@ static void contpte_try_unfold_partial(struct mm_struct= *mm, unsigned long addr, } =20 static void contpte_convert(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + hw_pte_t *ptep, pte_t pte) { struct vm_area_struct vma =3D TLB_FLUSH_VMA(mm, 0); unsigned long start_addr; - pte_t *start_ptep; + hw_pte_t *start_ptep; int i; =20 start_ptep =3D ptep =3D contpte_align_down(ptep); @@ -227,7 +228,7 @@ static void contpte_convert(struct mm_struct *mm, unsig= ned long addr, } =20 void __contpte_try_fold(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + hw_pte_t *ptep, pte_t pte) { /* * We have already checked that the virtual and pysical addresses are @@ -253,7 +254,7 @@ void __contpte_try_fold(struct mm_struct *mm, unsigned = long addr, struct folio *folio; struct page *page; unsigned long pfn; - pte_t *orig_ptep; + hw_pte_t *orig_ptep; pgprot_t prot; =20 int i; @@ -291,7 +292,7 @@ void __contpte_try_fold(struct mm_struct *mm, unsigned = long addr, EXPORT_SYMBOL_GPL(__contpte_try_fold); =20 void __contpte_try_unfold(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte) + hw_pte_t *ptep, pte_t pte) { /* * We have already checked that the ptes are contiguous in @@ -305,7 +306,7 @@ void __contpte_try_unfold(struct mm_struct *mm, unsigne= d long addr, } EXPORT_SYMBOL_GPL(__contpte_try_unfold); =20 -pte_t contpte_ptep_get(pte_t *ptep, pte_t orig_pte) +pte_t contpte_ptep_get(hw_pte_t *ptep, pte_t orig_pte) { /* * Gather access/dirty bits, which may be populated in any of the ptes @@ -362,7 +363,7 @@ static inline bool contpte_is_consistent(pte_t pte, uns= igned long pfn, pgprot_val(prot) =3D=3D pgprot_val(orig_prot); } =20 -pte_t contpte_ptep_get_lockless(pte_t *orig_ptep) +pte_t contpte_ptep_get_lockless(hw_pte_t *orig_ptep) { /* * The ptep_get_lockless() API requires us to read and return *orig_ptep @@ -384,7 +385,7 @@ pte_t contpte_ptep_get_lockless(pte_t *orig_ptep) pgprot_t orig_prot; unsigned long pfn; pte_t orig_pte; - pte_t *ptep; + hw_pte_t *ptep; pte_t pte; int i; =20 @@ -445,7 +446,8 @@ pte_t contpte_ptep_get_lockless(pte_t *orig_ptep) EXPORT_SYMBOL_GPL(contpte_ptep_get_lockless); =20 void contpte_set_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned int nr) + hw_pte_t *ptep, pte_t pte, + unsigned int nr) { unsigned long next; unsigned long end; @@ -488,7 +490,7 @@ void contpte_set_ptes(struct mm_struct *mm, unsigned lo= ng addr, EXPORT_SYMBOL_GPL(contpte_set_ptes); =20 void contpte_clear_full_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned int nr, int full) + hw_pte_t *ptep, unsigned int nr, int full) { contpte_try_unfold_partial(mm, addr, ptep, nr); __clear_full_ptes(mm, addr, ptep, nr, full); @@ -496,7 +498,7 @@ void contpte_clear_full_ptes(struct mm_struct *mm, unsi= gned long addr, EXPORT_SYMBOL_GPL(contpte_clear_full_ptes); =20 pte_t contpte_get_and_clear_full_ptes(struct mm_struct *mm, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr, int full) { contpte_try_unfold_partial(mm, addr, ptep, nr); @@ -505,7 +507,7 @@ pte_t contpte_get_and_clear_full_ptes(struct mm_struct = *mm, EXPORT_SYMBOL_GPL(contpte_get_and_clear_full_ptes); =20 bool contpte_test_and_clear_young_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned int nr) + unsigned long addr, hw_pte_t *ptep, unsigned int nr) { /* * ptep_clear_flush_young() technically requires us to clear the access @@ -531,7 +533,7 @@ bool contpte_test_and_clear_young_ptes(struct vm_area_s= truct *vma, EXPORT_SYMBOL_GPL(contpte_test_and_clear_young_ptes); =20 bool contpte_clear_flush_young_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned int nr) + unsigned long addr, hw_pte_t *ptep, unsigned int nr) { bool young; =20 @@ -554,7 +556,7 @@ bool contpte_clear_flush_young_ptes(struct vm_area_stru= ct *vma, EXPORT_SYMBOL_GPL(contpte_clear_flush_young_ptes); =20 void contpte_wrprotect_ptes(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned int nr) + hw_pte_t *ptep, unsigned int nr) { /* * If wrprotecting an entire contig range, we can avoid unfolding. Just @@ -572,7 +574,7 @@ void contpte_wrprotect_ptes(struct mm_struct *mm, unsig= ned long addr, EXPORT_SYMBOL_GPL(contpte_wrprotect_ptes); =20 void contpte_clear_young_dirty_ptes(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, unsigned int nr, cydp_t flags) { /* @@ -593,9 +595,10 @@ void contpte_clear_young_dirty_ptes(struct vm_area_str= uct *vma, } EXPORT_SYMBOL_GPL(contpte_clear_young_dirty_ptes); =20 -static bool contpte_all_subptes_match_access_flags(pte_t *ptep, pte_t entr= y) +static bool contpte_all_subptes_match_access_flags(hw_pte_t *ptep, + pte_t entry) { - pte_t *cont_ptep =3D contpte_align_down(ptep); + hw_pte_t *cont_ptep =3D contpte_align_down(ptep); /* * PFNs differ per sub-PTE. Match only bits consumed by * __ptep_set_access_flags(): AF, DIRTY and write permission. @@ -615,7 +618,7 @@ static bool contpte_all_subptes_match_access_flags(pte_= t *ptep, pte_t entry) } =20 int contpte_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t entry, int dirty) { unsigned long start_addr; diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 0b52557652be6..b77f4be88e3ea 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -162,7 +162,8 @@ static void show_pte(unsigned long addr) p4d_t *p4dp, p4d; pud_t *pudp, pud; pmd_t *pmdp, pmd; - pte_t *ptep, pte; + hw_pte_t *ptep; + pte_t pte; =20 if (pgd_none(pgd) || pgd_bad(pgd)) break; @@ -208,7 +209,7 @@ static void show_pte(unsigned long addr) * Returns whether or not the PTE actually changed. */ int __ptep_set_access_flags_anysz(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, + unsigned long address, hw_pte_t *ptep, pte_t entry, int dirty, unsigned long pgsize) { pteval_t old_pteval, pteval; diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index f66a0016dd02d..237a9136bc73b 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -35,7 +35,7 @@ static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss= _pgtbl; static pmd_t bm_pmd[PTRS_PER_PMD] __bss_pgtbl __maybe_unused; static pud_t bm_pud[PTRS_PER_PUD] __bss_pgtbl __maybe_unused; =20 -static inline pte_t *fixmap_pte(unsigned long addr) +static inline hw_pte_t *fixmap_pte(unsigned long addr) { return &bm_pte[BM_PTE_TABLE_IDX(addr)][pte_index(addr)]; } @@ -43,7 +43,7 @@ static inline pte_t *fixmap_pte(unsigned long addr) static void __init early_fixmap_init_pte(pmd_t *pmdp, unsigned long addr) { pmd_t pmd =3D READ_ONCE(*pmdp); - pte_t *ptep; + hw_pte_t *ptep; =20 if (pmd_none(pmd)) { ptep =3D bm_pte[BM_PTE_TABLE_IDX(addr)]; @@ -120,7 +120,7 @@ void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t flags) { unsigned long addr =3D __fix_to_virt(idx); - pte_t *ptep; + hw_pte_t *ptep; =20 BUG_ON(idx <=3D FIX_HOLE || idx >=3D __end_of_fixed_addresses); =20 diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 8e799c1fe0aa6..560c0f16cd8e4 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -76,7 +76,7 @@ bool arch_hugetlb_migration_supported(struct hstate *h) #endif =20 static int find_num_contig(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, size_t *pgsize) + hw_pte_t *ptep, size_t *pgsize) { pgd_t *pgdp =3D pgd_offset(mm, addr); p4d_t *p4dp; @@ -87,7 +87,7 @@ static int find_num_contig(struct mm_struct *mm, unsigned= long addr, p4dp =3D p4d_offset(pgdp, addr); pudp =3D pud_offset(p4dp, addr); pmdp =3D pmd_offset(pudp, addr); - if ((pte_t *)PTR_ALIGN_DOWN(pmdp, sizeof(*pmdp) * CONT_PMDS) =3D=3D ptep)= { + if ((hw_pte_t *)PTR_ALIGN_DOWN(pmdp, sizeof(*pmdp) * CONT_PMDS) =3D=3D pt= ep) { *pgsize =3D PMD_SIZE; return CONT_PMDS; } @@ -116,7 +116,7 @@ static inline int num_contig_ptes(unsigned long size, s= ize_t *pgsize) return contig_ptes; } =20 -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep) +pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, hw_pte_t *pt= ep) { int ncontig, i; size_t pgsize; @@ -148,7 +148,7 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long= addr, pte_t *ptep) */ static pte_t get_clear_contig(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, + hw_pte_t *ptep, unsigned long pgsize, unsigned long ncontig) { @@ -173,7 +173,7 @@ static pte_t get_clear_contig(struct mm_struct *mm, =20 static pte_t get_clear_contig_flush(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, + hw_pte_t *ptep, unsigned long pgsize, unsigned long ncontig) { @@ -196,7 +196,7 @@ static pte_t get_clear_contig_flush(struct mm_struct *m= m, */ static void clear_flush(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, + hw_pte_t *ptep, unsigned long pgsize, unsigned long ncontig) { @@ -213,7 +213,7 @@ static void clear_flush(struct mm_struct *mm, } =20 void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t pte, unsigned long sz) + hw_pte_t *ptep, pte_t pte, unsigned long sz) { size_t pgsize; int i; @@ -234,14 +234,14 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned l= ong addr, __set_ptes_anysz(mm, addr, ptep, pte, ncontig, pgsize); } =20 -pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, +hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) { pgd_t *pgdp; p4d_t *p4dp; pud_t *pudp; pmd_t *pmdp; - pte_t *ptep =3D NULL; + hw_pte_t *ptep =3D NULL; =20 pgdp =3D pgd_offset(mm, addr); p4dp =3D p4d_alloc(mm, pgdp, addr); @@ -253,7 +253,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_a= rea_struct *vma, return NULL; =20 if (sz =3D=3D PUD_SIZE) { - ptep =3D (pte_t *)pudp; + ptep =3D (hw_pte_t *)pudp; } else if (sz =3D=3D (CONT_PTE_SIZE)) { pmdp =3D pmd_alloc(mm, pudp, addr); if (!pmdp) @@ -265,17 +265,17 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm= _area_struct *vma, if (want_pmd_share(vma, addr) && pud_none(READ_ONCE(*pudp))) ptep =3D huge_pmd_share(mm, vma, addr, pudp); else - ptep =3D (pte_t *)pmd_alloc(mm, pudp, addr); + ptep =3D (hw_pte_t *)pmd_alloc(mm, pudp, addr); } else if (sz =3D=3D (CONT_PMD_SIZE)) { pmdp =3D pmd_alloc(mm, pudp, addr); WARN_ON(addr & (sz - 1)); - return (pte_t *)pmdp; + return (hw_pte_t *)pmdp; } =20 return ptep; } =20 -pte_t *huge_pte_offset(struct mm_struct *mm, +hw_pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, unsigned long sz) { pgd_t *pgdp; @@ -297,7 +297,7 @@ pte_t *huge_pte_offset(struct mm_struct *mm, return NULL; /* hugepage or swap? */ if (pud_leaf(pud) || !pud_present(pud)) - return (pte_t *)pudp; + return (hw_pte_t *)pudp; /* table; check the next level */ =20 if (sz =3D=3D CONT_PMD_SIZE) @@ -309,7 +309,7 @@ pte_t *huge_pte_offset(struct mm_struct *mm, pmd_none(pmd)) return NULL; if (pmd_leaf(pmd) || !pmd_present(pmd)) - return (pte_t *)pmdp; + return (hw_pte_t *)pmdp; =20 if (sz =3D=3D CONT_PTE_SIZE) return pte_offset_huge(pmdp, (addr & CONT_PTE_MASK)); @@ -367,7 +367,7 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shif= t, vm_flags_t flags) } =20 void huge_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) + hw_pte_t *ptep, unsigned long sz) { int i, ncontig; size_t pgsize; @@ -379,7 +379,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long= addr, } =20 pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, unsigned long sz) + hw_pte_t *ptep, unsigned long sz) { int ncontig; size_t pgsize; @@ -397,7 +397,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, uns= igned long addr, * all the contiguous ptes we need to check whether or not there is a * discrepancy between dirty or young. */ -static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) +static int __cont_access_flags_changed(hw_pte_t *ptep, pte_t pte, int ncon= tig) { int i; =20 @@ -418,7 +418,7 @@ static int __cont_access_flags_changed(pte_t *ptep, pte= _t pte, int ncontig) } =20 int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, + unsigned long addr, hw_pte_t *ptep, pte_t pte, int dirty) { int ncontig; @@ -451,7 +451,7 @@ int huge_ptep_set_access_flags(struct vm_area_struct *v= ma, } =20 void huge_ptep_set_wrprotect(struct mm_struct *mm, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { int ncontig; size_t pgsize; @@ -474,7 +474,7 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm, } =20 pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep) + unsigned long addr, hw_pte_t *ptep) { struct mm_struct *mm =3D vma->vm_mm; size_t pgsize; @@ -514,7 +514,8 @@ bool __init arch_hugetlb_valid_size(unsigned long size) return __hugetlb_valid_size(size); } =20 -pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned lon= g addr, pte_t *ptep) +pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma, unsigned lon= g addr, + hw_pte_t *ptep) { unsigned long psize =3D huge_page_size(hstate_vma(vma)); =20 @@ -530,7 +531,8 @@ pte_t huge_ptep_modify_prot_start(struct vm_area_struct= *vma, unsigned long addr return huge_ptep_get_and_clear(vma->vm_mm, addr, ptep, psize); } =20 -void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned lon= g addr, pte_t *ptep, +void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned lon= g addr, + hw_pte_t *ptep, pte_t old_pte, pte_t pte) { unsigned long psize =3D huge_page_size(hstate_vma(vma)); diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 45fbdce684c89..dccb736b32e9b 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -59,7 +59,7 @@ static phys_addr_t __init kasan_alloc_raw_page(int node) return __pa(p); } =20 -static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int= node, +static hw_pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, = int node, bool early) { if (pmd_none(READ_ONCE(*pmdp))) { @@ -116,7 +116,7 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsi= gned long addr, unsigned long end, int node, bool early) { unsigned long next; - pte_t *ptep =3D kasan_pte_offset(pmdp, addr, node, early); + hw_pte_t *ptep =3D kasan_pte_offset(pmdp, addr, node, early); =20 do { phys_addr_t page_phys =3D early ? diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 79d90226fd5dc..f02b974ad64df 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -159,7 +159,7 @@ static void init_clear_pgtable(void *table) dsb(ishst); } =20 -static void init_pte(pte_t *ptep, unsigned long addr, unsigned long end, +static void init_pte(hw_pte_t *ptep, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot) { do { @@ -182,7 +182,7 @@ static void init_pte(pte_t *ptep, unsigned long addr, u= nsigned long end, } while (ptep++, addr +=3D PAGE_SIZE, addr !=3D end); } =20 -static bool pte_range_has_valid_noncont(pte_t *ptep) +static bool pte_range_has_valid_noncont(hw_pte_t *ptep) { for (int i =3D 0; i < CONT_PTES; i++) { pte_t pte =3D __ptep_get(&ptep[i]); @@ -201,7 +201,7 @@ static int alloc_init_cont_pte(pmd_t *pmdp, unsigned lo= ng addr, { unsigned long next; pmd_t pmd =3D READ_ONCE(*pmdp); - pte_t *ptep; + hw_pte_t *ptep; =20 BUG_ON(pmd_leaf(pmd)); if (pmd_none(pmd)) { @@ -592,7 +592,7 @@ pgd_pgtable_alloc_special_mm(enum pgtable_level pgtable= _level) return __pgd_pgtable_alloc(NULL, GFP_PGTABLE_KERNEL, pgtable_level); } =20 -static void split_contpte(pte_t *ptep) +static void split_contpte(hw_pte_t *ptep) { int i; =20 @@ -607,13 +607,13 @@ static int split_pmd(pmd_t *pmdp, pmd_t pmd, gfp_t gf= p, bool to_cont) unsigned long pfn =3D pmd_pfn(pmd); pgprot_t prot =3D pmd_pgprot(pmd); phys_addr_t pte_phys; - pte_t *ptep; + hw_pte_t *ptep; int i; =20 pte_phys =3D pgd_pgtable_alloc_init_mm_gfp(PGTABLE_LEVEL_PTE, gfp); if (pte_phys =3D=3D INVALID_PHYS_ADDR) return -ENOMEM; - ptep =3D (pte_t *)phys_to_virt(pte_phys); + ptep =3D (hw_pte_t *)phys_to_virt(pte_phys); =20 if (pgprot_val(prot) & PMD_SECT_PXN) tableprot |=3D PMD_TABLE_PXN; @@ -691,7 +691,8 @@ static int split_kernel_leaf_mapping_locked(unsigned lo= ng addr) p4d_t *p4dp, p4d; pud_t *pudp, pud; pmd_t *pmdp, pmd; - pte_t *ptep, pte; + hw_pte_t *ptep; + pte_t pte; int ret =3D 0; =20 /* @@ -904,7 +905,7 @@ static int split_to_ptes_pmd_entry(pmd_t *pmdp, unsigne= d long addr, return ret; } =20 -static int split_to_ptes_pte_entry(pte_t *ptep, unsigned long addr, +static int split_to_ptes_pte_entry(hw_pte_t *ptep, unsigned long addr, unsigned long next, struct mm_walk *walk) { pte_t pte =3D __ptep_get(ptep); @@ -1438,7 +1439,8 @@ static void __init declare_kernel_vmas(void) } =20 void __pi_map_range(phys_addr_t *pte, u64 start, u64 end, phys_addr_t pa, - pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, + pgprot_t prot, int level, hw_pte_t *tbl, + bool may_use_cont, u64 va_offset); =20 static u8 idmap_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __r= o_after_init, @@ -1451,7 +1453,7 @@ static void __init create_idmap(void) phys_addr_t ptep =3D __pa_symbol(idmap_ptes); =20 __pi_map_range(&ptep, start, end, start, PAGE_KERNEL_ROX, - IDMAP_ROOT_LEVEL, (pte_t *)idmap_pg_dir, false, + IDMAP_ROOT_LEVEL, (hw_pte_t *)idmap_pg_dir, false, __phys_to_virt(ptep) - ptep); =20 if (linear_map_requires_bbml3 || @@ -1466,7 +1468,8 @@ static void __init create_idmap(void) */ ptep =3D __pa_symbol(kpti_bbml3_ptes); __pi_map_range(&ptep, pa, pa + sizeof(u32), pa, PAGE_KERNEL, - IDMAP_ROOT_LEVEL, (pte_t *)idmap_pg_dir, false, + IDMAP_ROOT_LEVEL, (hw_pte_t *)idmap_pg_dir, + false, __phys_to_virt(ptep) - ptep); } } @@ -1522,7 +1525,8 @@ static void unmap_hotplug_pte_range(pmd_t *pmdp, unsi= gned long addr, unsigned long end, bool free_mapped, struct vmem_altmap *altmap) { - pte_t *ptep, pte; + hw_pte_t *ptep; + pte_t pte; =20 do { ptep =3D pte_offset_kernel(pmdp, addr); @@ -1665,7 +1669,8 @@ static void free_empty_pte_table(pmd_t *pmdp, unsigne= d long addr, unsigned long end, unsigned long floor, unsigned long ceiling) { - pte_t *ptep, pte; + hw_pte_t *ptep; + pte_t pte; unsigned long i, start =3D addr; =20 do { @@ -1914,7 +1919,7 @@ int pmd_clear_huge(pmd_t *pmdp) =20 int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) { - pte_t *table; + hw_pte_t *table; pmd_t pmd; =20 pmd =3D READ_ONCE(*pmdp); @@ -2056,7 +2061,8 @@ static bool addr_splits_kernel_leaf(unsigned long add= r) p4d_t *p4dp, p4d; pud_t *pudp, pud; pmd_t *pmdp, pmd; - pte_t *ptep, pte; + hw_pte_t *ptep; + pte_t pte; =20 /* * If the given address points at a the start address of @@ -2284,7 +2290,7 @@ early_initcall(prevent_memory_remove_init); #endif =20 pte_t modify_prot_start_ptes(struct vm_area_struct *vma, unsigned long add= r, - pte_t *ptep, unsigned int nr) + hw_pte_t *ptep, unsigned int nr) { pte_t pte =3D get_and_clear_ptes(vma->vm_mm, addr, ptep, nr); =20 @@ -2302,19 +2308,21 @@ pte_t modify_prot_start_ptes(struct vm_area_struct = *vma, unsigned long addr, return pte; } =20 -pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long add= r, pte_t *ptep) +pte_t ptep_modify_prot_start(struct vm_area_struct *vma, unsigned long add= r, + hw_pte_t *ptep) { return modify_prot_start_ptes(vma, addr, ptep, 1); } =20 void modify_prot_commit_ptes(struct vm_area_struct *vma, unsigned long add= r, - pte_t *ptep, pte_t old_pte, pte_t pte, + hw_pte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr) { set_ptes(vma->vm_mm, addr, ptep, pte, nr); } =20 -void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long add= r, pte_t *ptep, +void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long add= r, + hw_pte_t *ptep, pte_t old_pte, pte_t pte) { modify_prot_commit_ptes(vma, addr, ptep, old_pte, pte, 1); diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index 132938b32eb16..895014212a81c 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -68,7 +68,7 @@ static int pageattr_pmd_entry(pmd_t *pmd, unsigned long a= ddr, return 0; } =20 -static int pageattr_pte_entry(pte_t *pte, unsigned long addr, +static int pageattr_pte_entry(hw_pte_t *pte, unsigned long addr, unsigned long next, struct mm_walk *walk) { pte_t val =3D __ptep_get(pte); @@ -379,7 +379,7 @@ bool kernel_page_present(struct page *page) p4d_t *p4dp; pud_t *pudp, pud; pmd_t *pmdp, pmd; - pte_t *ptep; + hw_pte_t *ptep; unsigned long addr =3D (unsigned long)page_address(page); =20 pgdp =3D pgd_offset_k(addr); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index cca9706a875c3..845417792ad48 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -34,8 +34,8 @@ static void *trans_alloc(struct trans_pgd_info *info) static int copy_pte(struct trans_pgd_info *info, pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, unsigned long end) { - pte_t *src_ptep; - pte_t *dst_ptep; + hw_pte_t *src_ptep; + hw_pte_t *dst_ptep; unsigned long addr =3D start; =20 dst_ptep =3D trans_alloc(info); --=20 2.47.3 From nobody Thu Sep 24 15:10:58 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CA8B3576EAC; Tue, 22 Sep 2026 17:32:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098386; cv=none; b=ul3I85KGvjAOv14B18jLwtjV6R+ChPfnMc+RmxibMJ9RACsVIxwfoou9lIKOvzvblnoi+TLE5Eg2sjWTjDWdvmzTlCLMHa19qDzoPztc2nyyCEIAm3u1scDO3v8nQc51l+zd2au6yWbBx5vKK54WDjwSCDdeLwXRaek7okGDWDo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098386; c=relaxed/simple; bh=xTJL2byDqGx0AB4Z6L+IiKln34gIro0/pV4Ued2u1UM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bjhnSXH3VziQtQRag1eG9rRpISsXTMpBjG27crkFUs1OZ/M+cL53gcVEXfH/nLoU9EWKDTXEG9sdMlj8/z0x/LG+Dg0Mvpqo5AHts9JsfWpAU9H6dcDIEAj8q5AbufRTOkgsGoiOocupKpsyB7HU1/W9Uc4MPE8LhPla+p9wsuQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=Rr/9WR4u; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="Rr/9WR4u" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4CC811576; Tue, 22 Sep 2026 10:32:54 -0700 (PDT) Received: from [127.0.1.1] (e142334-100.cambridge.arm.com [10.2.198.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 633063F86C; Tue, 22 Sep 2026 10:32:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1790098377; bh=xTJL2byDqGx0AB4Z6L+IiKln34gIro0/pV4Ued2u1UM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Rr/9WR4uGteoys9nZHG7+FeU3UTiy8Locu6YSlv8Os75MOwYw1jqPExvnMg9LRzUJ 2y6T0AwNdB9W5yWJSKEUKDK4zrqxaAO4cewAugA1YMhbN8da8UqEMmzaUGtYMbZYGI 8EZouveWU4RVVr0bxL6+D614EYrFlIzanIIkQzLg= From: Muhammad Usama Anjum Date: Tue, 22 Sep 2026 18:31:57 +0100 Subject: [PATCH v2 2/6] arm64: use hw_pte_val for HW PTE atomics Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-pte0_arm-v2-2-a3f1ddff0a8a@arm.com> References: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> In-Reply-To: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> To: Catalin Marinas , Will Deacon , Mark Rutland , Ryan Roberts , Ard Biesheuvel , Ilias Apalodimas , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, Muhammad Usama Anjum X-Mailer: b4 0.16.0 The low-level atomic PTE update paths take the address of pte_val(*ptep). That assumes HW PTE storage can be passed directly to the SW PTE value accessor and no longer type-checks when hw_pte_t is distinct. Use hw_pte_val() for the lvalues passed to compare-exchange and exchange operations. The atomic operations and their ordering are unchanged. Signed-off-by: Muhammad Usama Anjum Reviewed-by: Ryan Roberts --- Chagnes in v2: - Remove comment change - Move hw_pte_val() to generic series --- arch/arm64/include/asm/pgtable.h | 10 +++++----- arch/arm64/mm/fault.c | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 652ce413be389..4768ec59de555 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1303,7 +1303,7 @@ static inline bool __ptep_test_and_clear_young(struct= vm_area_struct *vma, do { old_pte =3D pte; pte =3D pte_mkold(pte); - pte_val(pte) =3D cmpxchg_relaxed(&pte_val(*ptep), + pte_val(pte) =3D cmpxchg_relaxed(&hw_pte_val(*ptep), pte_val(old_pte), pte_val(pte)); } while (pte_val(pte) !=3D pte_val(old_pte)); =20 @@ -1346,7 +1346,7 @@ static inline pte_t __ptep_get_and_clear_anysz(struct= mm_struct *mm, hw_pte_t *ptep, unsigned long pgsize) { - pte_t pte =3D __pte(xchg_relaxed(&pte_val(*ptep), 0)); + pte_t pte =3D __pte(xchg_relaxed(&hw_pte_val(*ptep), 0)); =20 switch (pgsize) { case PAGE_SIZE: @@ -1422,7 +1422,7 @@ static inline void ___ptep_set_wrprotect(struct mm_st= ruct *mm, do { old_pte =3D pte; pte =3D pte_wrprotect(pte); - pte_val(pte) =3D cmpxchg_relaxed(&pte_val(*ptep), + pte_val(pte) =3D cmpxchg_relaxed(&hw_pte_val(*ptep), pte_val(old_pte), pte_val(pte)); } while (pte_val(pte) !=3D pte_val(old_pte)); } @@ -1460,7 +1460,7 @@ static inline void __clear_young_dirty_pte(struct vm_= area_struct *vma, if (flags & CYDP_CLEAR_DIRTY) pte =3D pte_mkclean(pte); =20 - pte_val(pte) =3D cmpxchg_relaxed(&pte_val(*ptep), + pte_val(pte) =3D cmpxchg_relaxed(&hw_pte_val(*ptep), pte_val(old_pte), pte_val(pte)); } while (pte_val(pte) !=3D pte_val(old_pte)); } @@ -1830,7 +1830,7 @@ static inline bool ptep_try_set(hw_pte_t *ptep, pte_t= new_pte) { pteval_t old =3D 0; =20 - if (!try_cmpxchg(&pte_val(*ptep), &old, pte_val(new_pte))) + if (!try_cmpxchg(&hw_pte_val(*ptep), &old, pte_val(new_pte))) return false; =20 /* diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index b77f4be88e3ea..3322f4e7174dc 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -235,7 +235,7 @@ int __ptep_set_access_flags_anysz(struct vm_area_struct= *vma, pteval ^=3D PTE_RDONLY; pteval |=3D pte_val(entry); pteval ^=3D PTE_RDONLY; - pteval =3D cmpxchg_relaxed(&pte_val(*ptep), old_pteval, pteval); + pteval =3D cmpxchg_relaxed(&hw_pte_val(*ptep), old_pteval, pteval); } while (pteval !=3D old_pteval); =20 /* --=20 2.47.3 From nobody Thu Sep 24 15:10:58 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7CA0252268A; Tue, 22 Sep 2026 17:33:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098391; cv=none; b=E0MhHNBelC1EPTU3e++yIfyLmK9Z/VdxjpiXJuWL5L0zE+m49NIF3uaTJIAic2lnqOvaFe+I8yR+KuvEINwyupNLZRv6uCzhVj1ZtuzF4iIbg4oMeMG66y4TMx42GUKnx7SrbY2wzGWsiGJu/s+/3klya6TH3dfjjrND53vM5aI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098391; c=relaxed/simple; bh=67NvP4PmpCnowWkBeuojZg3SSMqBMwtXDXBIbaFjYiY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=uHwq+7aCuo45ROmWiI7pMvoBq3Kz6aezIBYKao3qQjX8cuTuFrf36vCYU0RADACHzgRXR81V9vSNn1LBzEJYvGtIkes6ZvxGt3gStGEzRoyb7NOtUAyEgsd8oISKF/iWBRH6ZAX3Ev5X0z9F/GDwpa+wxfCfsbjvCXTqQBD0IcY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=DgfsXpYX; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="DgfsXpYX" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0F2AC1AED; Tue, 22 Sep 2026 10:32:58 -0700 (PDT) Received: from [127.0.1.1] (e142334-100.cambridge.arm.com [10.2.198.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 269643F86C; Tue, 22 Sep 2026 10:32:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1790098381; bh=67NvP4PmpCnowWkBeuojZg3SSMqBMwtXDXBIbaFjYiY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=DgfsXpYXf+AULhg9SSync9g0KRdJYDwFtPkkElJ/a+wWRq0XRp4XhA/+AQcIjVicE yO9uEDNDRHS6qYUxgDmH4JNy09PlmpfEqfv0MLS+ymlFYbWBDTSK3ieWYmPBLASNUU 5DDokOnKTfjJP4/bugtIcH+JIC+awj5G0TJ4BSC4= From: Muhammad Usama Anjum Date: Tue, 22 Sep 2026 18:31:58 +0100 Subject: [PATCH v2 3/6] arm64: use hw_pte_val for HW PTE loads and stores Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-pte0_arm-v2-3-a3f1ddff0a8a@arm.com> References: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> In-Reply-To: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> To: Catalin Marinas , Will Deacon , Mark Rutland , Ryan Roberts , Ard Biesheuvel , Ilias Apalodimas , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, Muhammad Usama Anjum X-Mailer: b4 0.16.0 __ptep_get() returns a SW PTE value, while __set_pte_nosync() accepts one. When HW PTE storage uses a distinct hw_pte_t, reading or writing *ptep as a pte_t no longer satisfies those interfaces. Use hw_pte_val() to expose the pteval_t stored in either the hw_pte_t wrapper or its pte_t alias. __set_pte_nosync() writes pte_val(pte) to that storage, while __ptep_get() reads it and reconstructs a pte_t with __pte(). This avoids imposing a generic conversion interface between HW and SW PTE representations, which may require architecture-specific context in the future. Signed-off-by: Muhammad Usama Anjum Reviewed-by: Ryan Roberts --- arch/arm64/include/asm/pgtable.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 4768ec59de555..ce6580ab63128 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -360,7 +360,7 @@ static inline pte_t pte_clear_uffd(pte_t pte) =20 static inline void __set_pte_nosync(hw_pte_t *ptep, pte_t pte) { - WRITE_ONCE(*ptep, pte); + WRITE_ONCE(hw_pte_val(*ptep), pte_val(pte)); } =20 static inline void __set_pte_complete(pte_t pte) @@ -381,7 +381,7 @@ static inline void __set_pte(hw_pte_t *ptep, pte_t pte) =20 static inline pte_t __ptep_get(hw_pte_t *ptep) { - return READ_ONCE(*ptep); + return __pte(READ_ONCE(hw_pte_val(*ptep))); } =20 extern void __sync_icache_dcache(pte_t pteval); --=20 2.47.3 From nobody Thu Sep 24 15:10:58 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6EC60544D73; Tue, 22 Sep 2026 17:33:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098395; cv=none; b=E9r9/CMIMvepfvfUVN/BorgJlVwpXeDtdGIuLW6qIWyW5aD1kyAnKBFxiYGQA2tb5xZselRfysqYPXKsdGXpvwGsf23VYTzzOjNC3yWoXtQtecWNaYcNt3spati/URMBiDoKJKTgUdOyu+YO5zqMFQSmV2Uyg2rVL1f5C3dMYJw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098395; c=relaxed/simple; bh=shLHKCHn3ZdXre7hsjCS5Jwr6cqRreukZfraeH8Cy3s=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cK8QUj+MF2hzlH8TdikRF2/iuWs3PTWrPsxVpOsT0ijWxYfkBdWdQRMo045StzJOe3DoTDra2KOH71RLAkEpNIF0pENhxiY4Jjxt7vA58m33zy5qAcvPAVVuR4IG154mXdMrD7P4ZEEGCAtPTYsTtwwOoa2itSQ8HqTo++jlCP8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=SgM0Qx+n; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="SgM0Qx+n" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C4C601595; Tue, 22 Sep 2026 10:33:01 -0700 (PDT) Received: from [127.0.1.1] (e142334-100.cambridge.arm.com [10.2.198.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DD49E3F86C; Tue, 22 Sep 2026 10:33:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1790098385; bh=shLHKCHn3ZdXre7hsjCS5Jwr6cqRreukZfraeH8Cy3s=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=SgM0Qx+n9SIyEwK6aMAq6Ppwz1dhcVVmU+Lr/ToZEX+hmzKievt4rrjFlrNonyWHj Kx0OuVjhvw7BEYioGKvMfa7FL7lAcKc8dmLMmbyM40JqkH/qOYzITwD4t/d5FVCOgL ym3nNFw6xAU08NSMAjwa2v6IMkLgy4NQLbeBKLcY= From: Muhammad Usama Anjum Date: Tue, 22 Sep 2026 18:31:59 +0100 Subject: [PATCH v2 4/6] arm64: use hw_pte_t for fixmap HW PTEs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-pte0_arm-v2-4-a3f1ddff0a8a@arm.com> References: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> In-Reply-To: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> To: Catalin Marinas , Will Deacon , Mark Rutland , Ryan Roberts , Ard Biesheuvel , Ilias Apalodimas , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, Muhammad Usama Anjum X-Mailer: b4 0.16.0 fixmap_pte() returns a pointer into bm_pte, and early_fixmap_init_pte() installs those arrays as page tables. Their elements are therefore HW PTEs. Change the element type of bm_pte to hw_pte_t so it matches the HW PTE pointers returned and passed to the accessors. This is needed before ARCH_HAS_HW_PTE_T makes HW PTEs and SW PTE values distinct types; the array dimensions and placement are unchanged. Signed-off-by: Muhammad Usama Anjum Reviewed-by: Ryan Roberts --- arch/arm64/mm/fixmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index 237a9136bc73b..709a97fe327d8 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -31,7 +31,7 @@ static_assert(NR_BM_PMD_TABLES =3D=3D 1); =20 #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) =20 -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss_pgtbl; +static hw_pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __bss_pgtbl; static pmd_t bm_pmd[PTRS_PER_PMD] __bss_pgtbl __maybe_unused; static pud_t bm_pud[PTRS_PER_PUD] __bss_pgtbl __maybe_unused; =20 --=20 2.47.3 From nobody Thu Sep 24 15:10:58 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E72BB576EB7; Tue, 22 Sep 2026 17:33:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098400; cv=none; b=nE16i2PBxEypA3F4x7Nw+Aae8GuReO1lZ8rHjyj83jVag/ZDpOPjYr0WhsQKj4pxk7QWjvUjVX7ElCmNDK9t/pI63H3Lr/jgk+xdgpXZvKMVikJZ/ekpnTYprrJt3LixdN+QuZBr9mc71evVN4VuZeLNU0lr6P+oTz0qfLz/TpM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098400; c=relaxed/simple; bh=lGoyQT/IUQtR3cNCOID4as8wOLQ/VDmkytsu/6BddYQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FGouXsBvBXPaN4Yy/07B0zDmwBWpccLKRPFQC9mAWkEQoPprPQoVX95NvQsRJ8xSUbHqfR1XbOWM1QfgC4LOtkaiFzvy2xsSAhc1N9qIuoqFA1XUV67EAJyb79qTgY8GuH+liBwOTdOvTj3veF1M1qVFbOTV5vlWET9dykrDBXg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=mNoQ9q5B; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="mNoQ9q5B" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 85DBA1576; Tue, 22 Sep 2026 10:33:05 -0700 (PDT) Received: from [127.0.1.1] (e142334-100.cambridge.arm.com [10.2.198.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9EBCD3F86C; Tue, 22 Sep 2026 10:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1790098389; bh=lGoyQT/IUQtR3cNCOID4as8wOLQ/VDmkytsu/6BddYQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=mNoQ9q5BtvoMd+Rw12q0kZB49nZuw7UiCE1KeciQ3TnFAZnWIjbD+UV3pSFN6VIrW HiDynCz+zSRp1JmOxEnCoMWiBzqNZqS42/vU0/wJuCDlZ01zEtC5LxR2cFN0m6O57w DIbzipLnMXJ/EX/qZe4oH777ESIeNsTvQu0X9Inw= From: Muhammad Usama Anjum Date: Tue, 22 Sep 2026 18:32:00 +0100 Subject: [PATCH v2 5/6] arm64: use HW PTE accessors in early map_range() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-pte0_arm-v2-5-a3f1ddff0a8a@arm.com> References: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> In-Reply-To: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> To: Catalin Marinas , Will Deacon , Mark Rutland , Ryan Roberts , Ard Biesheuvel , Ilias Apalodimas , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, Muhammad Usama Anjum X-Mailer: b4 0.16.0 map_range() uses HW PTE pointers to build early translation tables at each level. Directly testing, decoding, or assigning *tbl treats an HW PTE as a SW PTE value, which requires explicit conversion once hw_pte_t becomes distinct. Use __ptep_get() to load a SW PTE value for the empty-entry check and child-table address, and __set_pte_nosync() to convert SW PTE values to HW PTEs on stores. The no-sync helper performs WRITE_ONCE() without adding the completion barriers used by __set_pte(). Reviewed-by: Ryan Roberts Signed-off-by: Muhammad Usama Anjum --- arch/arm64/kernel/pi/map_range.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/pi/map_range.c b/arch/arm64/kernel/pi/map_ra= nge.c index 94f06da76e333..9c4c8060c5df4 100644 --- a/arch/arm64/kernel/pi/map_range.c +++ b/arch/arm64/kernel/pi/map_range.c @@ -51,19 +51,21 @@ void __init map_range(phys_addr_t *pte, u64 start, u64 = end, phys_addr_t pa, =20 while (start < end) { u64 next =3D min((start | lmask) + 1, PAGE_ALIGN(end)); + pte_t entry =3D __ptep_get(tbl); =20 if (level < 2 || (level =3D=3D 2 && (start | next | pa) & lmask)) { /* * This chunk needs a finer grained mapping. Create a * table mapping if necessary and recurse. */ - if (pte_none(*tbl)) { - *tbl =3D __pte(__phys_to_pte_val(*pte) | + if (pte_none(entry)) { + entry =3D __pte(__phys_to_pte_val(*pte) | PMD_TYPE_TABLE | PMD_TABLE_UXN); - *pte +=3D PTRS_PER_PTE * sizeof(pte_t); + __set_pte_nosync(tbl, entry); + *pte +=3D PTRS_PER_PTE * sizeof(hw_pte_t); } map_range(pte, start, next, pa, prot, level + 1, - (hw_pte_t *)(__pte_to_phys(*tbl) + va_offset), + (hw_pte_t *)(__pte_to_phys(entry) + va_offset), may_use_cont, va_offset); } else { /* @@ -81,7 +83,8 @@ void __init map_range(phys_addr_t *pte, u64 start, u64 en= d, phys_addr_t pa, protval &=3D ~PTE_CONT; =20 /* Put down a block or page mapping */ - *tbl =3D __pte(__phys_to_pte_val(pa) | protval); + entry =3D __pte(__phys_to_pte_val(pa) | protval); + __set_pte_nosync(tbl, entry); } pa +=3D next - start; start =3D next; --=20 2.47.3 From nobody Thu Sep 24 15:10:58 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8D69F55C30D; Tue, 22 Sep 2026 17:33:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098400; cv=none; b=AW0i/erb7U05yZWFnYvAmXN0oFszyHabXi8YCuyp3dPoCHZ1iev3MlRiz58poIzawmiAXRUH461p+5oahUFDWj/0RfdpsekY2ohSEXdogVXuc7XvWIFxSSbwm9wSWtNJNdCv0i88py10HI/hblhZtal2c61osiovKP9Wpv5c3NI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790098400; c=relaxed/simple; bh=PeJQ7fYXLGcw4P0kvfY82twCI/k6VJB344U1K7tkEMM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MHr6VSwUWuL0nrikyk8x09865keW9lYaR69SbA/WCgiiA7E5TRe5MZuQj9GeL2SSTLPTonB1LZLn8XttUCao7ipD+rg8KTcEZK89eVZiOuJYUtLRcyBwNVphQH3Rue4Ifh29ffkfDnlVTrzxr3kh0o3p1rH1BA/Lv619p2oiL/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=d3xeuRYT; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="d3xeuRYT" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 49DA01AED; Tue, 22 Sep 2026 10:33:09 -0700 (PDT) Received: from [127.0.1.1] (e142334-100.cambridge.arm.com [10.2.198.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 616B73F86C; Tue, 22 Sep 2026 10:33:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1790098392; bh=PeJQ7fYXLGcw4P0kvfY82twCI/k6VJB344U1K7tkEMM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=d3xeuRYT+klOzNDSyNdM3rcbkRZj3fdN5cBa1D1YwCIZyxd+7f7Sy8BgbNUmU5al1 3s/0EsafMWeY/ZZrmT+pw9w6TaZJeLQe8kr+7niH1ZmkhigzCVK9Oe/kkhM5HFTKdQ LdRNH7rjihzI3dDQ1qhnr2R8GM6p9/Ohat1pmPv0= From: Muhammad Usama Anjum Date: Tue, 22 Sep 2026 18:32:01 +0100 Subject: [PATCH v2 6/6] arm64: enable a distinct type for HW PTEs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-pte0_arm-v2-6-a3f1ddff0a8a@arm.com> References: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> In-Reply-To: <20260922-pte0_arm-v2-0-a3f1ddff0a8a@arm.com> To: Catalin Marinas , Will Deacon , Mark Rutland , Ryan Roberts , Ard Biesheuvel , Ilias Apalodimas , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org, Muhammad Usama Anjum X-Mailer: b4 0.16.0 With HW PTE pointers and direct HW PTE accesses converted, arm64 can use the generic hw_pte_t wrapper instead of the pte_t alias. Select ARCH_HAS_HW_PTE_T to give HW PTEs and SW PTE values distinct C types. Reviewed-by: Ryan Roberts Signed-off-by: Muhammad Usama Anjum --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 2bbeded33da0d..42841d5e2ed26 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -31,6 +31,7 @@ config ARM64 select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL select ARCH_HAS_GIGANTIC_PAGE + select ARCH_HAS_HW_PTE_T select ARCH_HAS_KCOV select ARCH_HAS_KERNEL_FPU_SUPPORT if KERNEL_MODE_NEON select ARCH_HAS_KEEPINITRD --=20 2.47.3