From nobody Wed Dec 10 20:12:36 2025 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DFD82E03E1 for ; Thu, 13 Nov 2025 01:47:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762998437; cv=none; b=ivAzTc5fgKmVuIuooHS/wVQhhwhtrWBzwijGe+QLr18Sa/kDpWhwtV62t47XmzwcMHneI3zu6hQbSYBWsdCeYiEIWQzBxDp7i6zqbkphfBseqe5dlYWfNgEWS7pvTDsa2xNx0xUpXfIyogVWGNvA16s5Snv2r2r2QyBsd4fRxUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762998437; c=relaxed/simple; bh=86DWhTWxpA+Wn6OoSODTxcACeGJ6z7eq0QJodNfrB5g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bJQ9JY9LcnGsAysVVoRVkM2/NST765R0RYQ3AEdFkBZuWmT5O0A81yDVKLrYv+Fg+ewt2TZD9eX6u+Toro/vbhpnt9weFd8sSFOe3966m5kx5lkRGoyHXLh1SGnt5Iy1ESKwwz2B5XAngItd67rLOO4LymSLkJziFRprDfrRpJs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=h9BbaI20; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="h9BbaI20" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-7ad1cd0db3bso217560b3a.1 for ; Wed, 12 Nov 2025 17:47:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1762998434; x=1763603234; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=onXS1ZyyTIk7E4Phjch+Gs0PpMpoB1JPFOZexalawi0=; b=h9BbaI20ErXz7k96wyZ4jxSnKtn+lcez+ImlndUqs1usZiu6F8k8KatSou3fHsfogG DnDBnkpxDkAQyVm3OgdOcv5LoG8lk+UUNUFVEC4XeXzaIZ3R7Xf+DeZwckgAJlXfXGbl DQUrQiMYg6qj0f9bUwiFfizAB+GaF2uxnLlgH19UxvGUlh82R/GSOJjmtb5obCd4m1v9 KlqwiayULydoNS+fRpE+dd3oRC6getLmEu95kpOVtXAjAzbLPKNte7ZEM013EUeM2WE4 QuSIAPTGdtAspfZiFku+a55vi67F3qNeXDiSnte56gBCiiJ9RyD9ulxobPP4Ayeili1j 2ncQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762998434; x=1763603234; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=onXS1ZyyTIk7E4Phjch+Gs0PpMpoB1JPFOZexalawi0=; b=DldyaZr2Llgy5/oLSp0mnPNZ3ZChNFkRI/+I1RGK3F0JYUROXmiXJv2fJ6hFECgpZK nPuLvUjvVB3SiV8DGPAzwaTDw7tv9VWIL/+7USjNQxCcWG/eIogD/wtZ6jwcWmELnAOy AVTrMyo7NzJZuf0a3cXShNllV+Jbb9JTQMw3TKER7lLcY2ev5i1as6U+TA/tZVYoWI4B tLMaZdoUstUUIJH0M4dXPqfAXwsgFOb5rP6Zs3AJv5+sx7j++Fm8QNUCG4ZNrJ6+NQbx NksjJrQz3OgmuBhoYye9dKYWRWPRq1C/200P1+5s04ZK0wQ8gkzD3kJLuo3FHCoiXMgj 68eA== X-Forwarded-Encrypted: i=1; AJvYcCWm/fVUocn4DT5vRk9VeWik9cvxFJu7vRiQ/AXVRrZaLo6vJmtTF5eBxljBq4hiaJgI/j2j2Wt3XAjpsS0=@vger.kernel.org X-Gm-Message-State: AOJu0Ywzs4o8CR+HbkOQG9d/o7n78zA3aG/6LmjyqeneK1FA3ehddLHg DTuo4bVecu12e7ZteQ3nOORCnMzTxkI/U1wXzlMXOd1vAT8AZ0pFjqKziJx0XqDGpVY= X-Gm-Gg: ASbGnctuYQlvRRILRbpr5n0Q81RT53v55FC7CXgvzg5PLJ834TS068xCJpTMW/HBG9T jJGSxsqFn77NcORBzkLAYfULbF6Qw5mc58nlYSvzp5/781n1TFtsYgmxvakEoKEWMOUWlIUncv7 7UEcjtPJcg62KN3vLYAVZLh8vDax2ViKrnd2EI5DpoauZzHs3ow7azM7hRe6hjen27LE+SysEJY WXPqjq3m2hIZt79pznObHUgB2wWLUSAXz/E4qHjkSF/h9NNvuEJFjFuxK66T+dXQYlPGpankc19 FZ38GnxRURskbhcQ6NLBryj85g6+b3WGOL3I6cN7c+DG1D2fRioO2dAIeYa8rg1fdja2pW6S9mL d98iqfvOqxsucHSDVsxlK5dP+w5J6L5RIKL29LhnP2pqSuG0WaQvN+98x6Q+xZz8Sln2HwAjyhc m5edAc9++Ai7Ek5RuZBK8VMA== X-Google-Smtp-Source: AGHT+IEPUQpGJSnlpoH1wKnMxfvk73znW6zYh27mDLEJKnxaxWn+hw/P1W+C7Nu+pbcglz2Jny/U/g== X-Received: by 2002:a17:902:cccf:b0:297:f8d9:aad7 with SMTP id d9443c01a7336-2984ede5755mr62262705ad.50.1762998434336; Wed, 12 Nov 2025 17:47:14 -0800 (PST) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2985c2ccae8sm4986485ad.98.2025.11.12.17.47.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Nov 2025 17:47:14 -0800 (PST) From: Samuel Holland To: Palmer Dabbelt , Paul Walmsley , linux-riscv@lists.infradead.org, Andrew Morton , David Hildenbrand , linux-mm@kvack.org Cc: devicetree@vger.kernel.org, Suren Baghdasaryan , linux-kernel@vger.kernel.org, Mike Rapoport , Michal Hocko , Conor Dooley , Lorenzo Stoakes , Krzysztof Kozlowski , Alexandre Ghiti , Emil Renner Berthing , Rob Herring , Vlastimil Babka , "Liam R . Howlett" , Samuel Holland Subject: [PATCH v3 10/22] riscv: mm: Always use page table accessor functions Date: Wed, 12 Nov 2025 17:45:23 -0800 Message-ID: <20251113014656.2605447-11-samuel.holland@sifive.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20251113014656.2605447-1-samuel.holland@sifive.com> References: <20251113014656.2605447-1-samuel.holland@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the semantically appropriate accessor function instead of a raw pointer dereference. This will become important once these functions start transforming the PTE value on some platforms. Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - New patch for v2 arch/riscv/include/asm/pgtable.h | 8 ++-- arch/riscv/kvm/gstage.c | 6 +-- arch/riscv/mm/init.c | 68 +++++++++++++++++--------------- arch/riscv/mm/pgtable.c | 9 +++-- 4 files changed, 49 insertions(+), 42 deletions(-) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgta= ble.h index 5a08eb5fe99f..acfd48f92010 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -952,7 +952,7 @@ static inline pud_t pudp_huge_get_and_clear(struct mm_s= truct *mm, #ifdef CONFIG_SMP pud_t pud =3D __pud(xchg(&pudp->pud, 0)); #else - pud_t pud =3D *pudp; + pud_t pud =3D pudp_get(pudp); =20 pud_clear(pudp); #endif @@ -1129,13 +1129,15 @@ extern unsigned long empty_zero_page[PAGE_SIZE / si= zeof(unsigned long)]; */ #define set_p4d_safe(p4dp, p4d) \ ({ \ - WARN_ON_ONCE(p4d_present(*p4dp) && !p4d_same(*p4dp, p4d)); \ + p4d_t old =3D p4dp_get(p4dp); \ + WARN_ON_ONCE(p4d_present(old) && !p4d_same(old, p4d)); \ set_p4d(p4dp, p4d); \ }) =20 #define set_pgd_safe(pgdp, pgd) \ ({ \ - WARN_ON_ONCE(pgd_present(*pgdp) && !pgd_same(*pgdp, pgd)); \ + pgd_t old =3D pgdp_get(pgdp); \ + WARN_ON_ONCE(pgd_present(old) && !pgd_same(old, pgd)); \ set_pgd(pgdp, pgd); \ }) #endif /* !__ASSEMBLER__ */ diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c index b67d60d722c2..297744e2ab5d 100644 --- a/arch/riscv/kvm/gstage.c +++ b/arch/riscv/kvm/gstage.c @@ -154,7 +154,7 @@ int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage, ptep =3D &next_ptep[gstage_pte_index(map->addr, current_level)]; } =20 - if (pte_val(*ptep) !=3D pte_val(map->pte)) { + if (pte_val(ptep_get(ptep)) !=3D pte_val(map->pte)) { set_pte(ptep, map->pte); if (gstage_pte_leaf(ptep)) gstage_tlb_flush(gstage, current_level, map->addr); @@ -241,12 +241,12 @@ void kvm_riscv_gstage_op_pte(struct kvm_gstage *gstag= e, gpa_t addr, if (op =3D=3D GSTAGE_OP_CLEAR) put_page(virt_to_page(next_ptep)); } else { - old_pte =3D *ptep; + old_pte =3D ptep_get(ptep); if (op =3D=3D GSTAGE_OP_CLEAR) set_pte(ptep, __pte(0)); else if (op =3D=3D GSTAGE_OP_WP) set_pte(ptep, __pte(pte_val(ptep_get(ptep)) & ~_PAGE_WRITE)); - if (pte_val(*ptep) !=3D pte_val(old_pte)) + if (pte_val(ptep_get(ptep)) !=3D pte_val(old_pte)) gstage_tlb_flush(gstage, ptep_level, addr); } } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index d85efe74a4b6..ac686c1b2f85 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -459,8 +459,8 @@ static void __meminit create_pte_mapping(pte_t *ptep, u= intptr_t va, phys_addr_t =20 BUG_ON(sz !=3D PAGE_SIZE); =20 - if (pte_none(ptep[pte_idx])) - ptep[pte_idx] =3D pfn_pte(PFN_DOWN(pa), prot); + if (pte_none(ptep_get(ptep + pte_idx))) + set_pte(ptep + pte_idx, pfn_pte(PFN_DOWN(pa), prot)); } =20 #ifndef __PAGETABLE_PMD_FOLDED @@ -542,18 +542,19 @@ static void __meminit create_pmd_mapping(pmd_t *pmdp, uintptr_t pmd_idx =3D pmd_index(va); =20 if (sz =3D=3D PMD_SIZE) { - if (pmd_none(pmdp[pmd_idx])) - pmdp[pmd_idx] =3D pfn_pmd(PFN_DOWN(pa), prot); + if (pmd_none(pmdp_get(pmdp + pmd_idx))) + set_pmd(pmdp + pmd_idx, pfn_pmd(PFN_DOWN(pa), prot)); return; } =20 - if (pmd_none(pmdp[pmd_idx])) { + if (pmd_none(pmdp_get(pmdp + pmd_idx))) { pte_phys =3D pt_ops.alloc_pte(va); - pmdp[pmd_idx] =3D pfn_pmd(PFN_DOWN(pte_phys), PAGE_TABLE); + set_pmd(pmdp + pmd_idx, + pfn_pmd(PFN_DOWN(pte_phys), PAGE_TABLE)); ptep =3D pt_ops.get_pte_virt(pte_phys); memset(ptep, 0, PAGE_SIZE); } else { - pte_phys =3D PFN_PHYS(_pmd_pfn(pmdp[pmd_idx])); + pte_phys =3D PFN_PHYS(_pmd_pfn(pmdp_get(pmdp + pmd_idx))); ptep =3D pt_ops.get_pte_virt(pte_phys); } =20 @@ -644,18 +645,19 @@ static void __meminit create_pud_mapping(pud_t *pudp,= uintptr_t va, phys_addr_t uintptr_t pud_index =3D pud_index(va); =20 if (sz =3D=3D PUD_SIZE) { - if (pud_val(pudp[pud_index]) =3D=3D 0) - pudp[pud_index] =3D pfn_pud(PFN_DOWN(pa), prot); + if (pud_val(pudp_get(pudp + pud_index)) =3D=3D 0) + set_pud(pudp + pud_index, pfn_pud(PFN_DOWN(pa), prot)); return; } =20 - if (pud_val(pudp[pud_index]) =3D=3D 0) { + if (pud_val(pudp_get(pudp + pud_index)) =3D=3D 0) { next_phys =3D pt_ops.alloc_pmd(va); - pudp[pud_index] =3D pfn_pud(PFN_DOWN(next_phys), PAGE_TABLE); + set_pud(pudp + pud_index, + pfn_pud(PFN_DOWN(next_phys), PAGE_TABLE)); nextp =3D pt_ops.get_pmd_virt(next_phys); memset(nextp, 0, PAGE_SIZE); } else { - next_phys =3D PFN_PHYS(_pud_pfn(pudp[pud_index])); + next_phys =3D PFN_PHYS(_pud_pfn(pudp_get(pudp + pud_index))); nextp =3D pt_ops.get_pmd_virt(next_phys); } =20 @@ -670,18 +672,19 @@ static void __meminit create_p4d_mapping(p4d_t *p4dp,= uintptr_t va, phys_addr_t uintptr_t p4d_index =3D p4d_index(va); =20 if (sz =3D=3D P4D_SIZE) { - if (p4d_val(p4dp[p4d_index]) =3D=3D 0) - p4dp[p4d_index] =3D pfn_p4d(PFN_DOWN(pa), prot); + if (p4d_val(p4dp_get(p4dp + p4d_index)) =3D=3D 0) + set_p4d(p4dp + p4d_index, pfn_p4d(PFN_DOWN(pa), prot)); return; } =20 - if (p4d_val(p4dp[p4d_index]) =3D=3D 0) { + if (p4d_val(p4dp_get(p4dp + p4d_index)) =3D=3D 0) { next_phys =3D pt_ops.alloc_pud(va); - p4dp[p4d_index] =3D pfn_p4d(PFN_DOWN(next_phys), PAGE_TABLE); + set_p4d(p4dp + p4d_index, + pfn_p4d(PFN_DOWN(next_phys), PAGE_TABLE)); nextp =3D pt_ops.get_pud_virt(next_phys); memset(nextp, 0, PAGE_SIZE); } else { - next_phys =3D PFN_PHYS(_p4d_pfn(p4dp[p4d_index])); + next_phys =3D PFN_PHYS(_p4d_pfn(p4dp_get(p4dp + p4d_index))); nextp =3D pt_ops.get_pud_virt(next_phys); } =20 @@ -727,18 +730,19 @@ void __meminit create_pgd_mapping(pgd_t *pgdp, uintpt= r_t va, phys_addr_t pa, phy uintptr_t pgd_idx =3D pgd_index(va); =20 if (sz =3D=3D PGDIR_SIZE) { - if (pgd_val(pgdp[pgd_idx]) =3D=3D 0) - pgdp[pgd_idx] =3D pfn_pgd(PFN_DOWN(pa), prot); + if (pgd_val(pgdp_get(pgdp + pgd_idx)) =3D=3D 0) + set_pgd(pgdp + pgd_idx, pfn_pgd(PFN_DOWN(pa), prot)); return; } =20 - if (pgd_val(pgdp[pgd_idx]) =3D=3D 0) { + if (pgd_val(pgdp_get(pgdp + pgd_idx)) =3D=3D 0) { next_phys =3D alloc_pgd_next(va); - pgdp[pgd_idx] =3D pfn_pgd(PFN_DOWN(next_phys), PAGE_TABLE); + set_pgd(pgdp + pgd_idx, + pfn_pgd(PFN_DOWN(next_phys), PAGE_TABLE)); nextp =3D get_pgd_next_virt(next_phys); memset(nextp, 0, PAGE_SIZE); } else { - next_phys =3D PFN_PHYS(_pgd_pfn(pgdp[pgd_idx])); + next_phys =3D PFN_PHYS(_pgd_pfn(pgdp_get(pgdp + pgd_idx))); nextp =3D get_pgd_next_virt(next_phys); } =20 @@ -1574,14 +1578,14 @@ struct execmem_info __init *execmem_arch_setup(void) #ifdef CONFIG_MEMORY_HOTPLUG static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd) { - struct page *page =3D pmd_page(*pmd); + struct page *page =3D pmd_page(pmdp_get(pmd)); struct ptdesc *ptdesc =3D page_ptdesc(page); pte_t *pte; int i; =20 for (i =3D 0; i < PTRS_PER_PTE; i++) { pte =3D pte_start + i; - if (!pte_none(*pte)) + if (!pte_none(ptep_get(pte))) return; } =20 @@ -1595,14 +1599,14 @@ static void __meminit free_pte_table(pte_t *pte_sta= rt, pmd_t *pmd) =20 static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud, bool is= _vmemmap) { - struct page *page =3D pud_page(*pud); + struct page *page =3D pud_page(pudp_get(pud)); struct ptdesc *ptdesc =3D page_ptdesc(page); pmd_t *pmd; int i; =20 for (i =3D 0; i < PTRS_PER_PMD; i++) { pmd =3D pmd_start + i; - if (!pmd_none(*pmd)) + if (!pmd_none(pmdp_get(pmd))) return; } =20 @@ -1617,13 +1621,13 @@ static void __meminit free_pmd_table(pmd_t *pmd_sta= rt, pud_t *pud, bool is_vmemm =20 static void __meminit free_pud_table(pud_t *pud_start, p4d_t *p4d) { - struct page *page =3D p4d_page(*p4d); + struct page *page =3D p4d_page(p4dp_get(p4d)); pud_t *pud; int i; =20 for (i =3D 0; i < PTRS_PER_PUD; i++) { pud =3D pud_start + i; - if (!pud_none(*pud)) + if (!pud_none(pudp_get(pud))) return; } =20 @@ -1668,7 +1672,7 @@ static void __meminit remove_pte_mapping(pte_t *pte_b= ase, unsigned long addr, un =20 ptep =3D pte_base + pte_index(addr); pte =3D ptep_get(ptep); - if (!pte_present(*ptep)) + if (!pte_present(ptep_get(ptep))) continue; =20 pte_clear(&init_mm, addr, ptep); @@ -1698,7 +1702,7 @@ static void __meminit remove_pmd_mapping(pmd_t *pmd_b= ase, unsigned long addr, un continue; } =20 - pte_base =3D (pte_t *)pmd_page_vaddr(*pmdp); + pte_base =3D (pte_t *)pmd_page_vaddr(pmdp_get(pmdp)); remove_pte_mapping(pte_base, addr, next, is_vmemmap, altmap); free_pte_table(pte_base, pmdp); } @@ -1777,10 +1781,10 @@ static void __meminit remove_pgd_mapping(unsigned l= ong va, unsigned long end, bo next =3D pgd_addr_end(addr, end); pgd =3D pgd_offset_k(addr); =20 - if (!pgd_present(*pgd)) + if (!pgd_present(pgdp_get(pgd))) continue; =20 - if (pgd_leaf(*pgd)) + if (pgd_leaf(pgdp_get(pgd))) continue; =20 p4d_base =3D p4d_offset(pgd, 0); diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 8b6c0a112a8d..c4b85a828797 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -95,8 +95,8 @@ int pud_free_pmd_page(pud_t *pud, unsigned long addr) flush_tlb_kernel_range(addr, addr + PUD_SIZE); =20 for (i =3D 0; i < PTRS_PER_PMD; i++) { - if (!pmd_none(pmd[i])) { - pte_t *pte =3D (pte_t *)pmd_page_vaddr(pmd[i]); + if (!pmd_none(pmdp_get(pmd + i))) { + pte_t *pte =3D (pte_t *)pmd_page_vaddr(pmdp_get(pmd + i)); =20 pte_free_kernel(NULL, pte); } @@ -158,8 +158,9 @@ pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, pud_t *pudp) { - VM_WARN_ON_ONCE(!pud_present(*pudp)); - pud_t old =3D pudp_establish(vma, address, pudp, pud_mkinvalid(*pudp)); + VM_WARN_ON_ONCE(!pud_present(pudp_get(pudp))); + pud_t old =3D pudp_establish(vma, address, pudp, + pud_mkinvalid(pudp_get(pudp))); =20 flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE); return old; --=20 2.47.2