From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EBE0559CB7; Tue, 22 Sep 2026 15:36:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091420; cv=none; b=fz2W38Hm6IjustFRk77FRKOBy0LPWsV/rhD0+9jNrlldZbAd/X0l49E1iZdZGXsRImELNNDjQgyPvPqd3/XdapbIwQcHwymVcHqCJ2fMoXIN7d1pMXtgqz3nL2rlf5gjhx7knDameF0MddiwUpN0tX967n1cQnpMnSmXcVDw5zg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091420; c=relaxed/simple; bh=henzipa1JQvP3ZJSUdVpc3i3Gyb5R1tpkdjoqO30NaM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=VXw5IjyZKES30noxXQ936KOiU7Z63B1BcXmmJ0/2M16uUVfT+QHZMQBgRKkUaI80Ye5gkN+MCq95R8OKjK9OrfBOwGPKqC6leXGkYeBVDNvX1Tp712w/21yKL8h//ytP1Ol4YnrNOwqOVha2YV9E2KVr9FMfrktnKOpvdwpEqJE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=O5OnRqsH; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="O5OnRqsH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 324F91F00893; Tue, 22 Sep 2026 15:36:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091418; bh=J1kULK2uo5iIlycPGFV3SVU1kRYTeuiXaUNtJjPTvlY=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=O5OnRqsHWVSboGiycATXii3kxCgUoPSM2nTvyWAlnmjffTTXW+gJ0VBID4vvQUgI/ O9lywv7inXl6KjGb7/0VlcptxGCYMi4pmk8E1K0Key0xXSKimb8qT2xAQNzcqn3bgc NZ8SfEmhiX7Y6teqyn3WgbVZVXuwQ21QpRixohrxv/tDwY33XXtc8Vonec6u9sD/gt UBZbxXOIRgMuJj27Gvy7eryEkVKiqbSFWzONLl9IV9J/OcuxzDqnw+q8cSlW/JFZYS AGVcvEeYheEIUqWRkuYddTr13SdWfC/C4XoFemKzJCBJh9/mvAoKeri9l3zT04ZF4l EG7/hz9SzpOrg== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:32 +0100 Subject: [PATCH v4 01/12] mm/khugepaged: deposit a newly allocated page table on collapse Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-1-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4993; i=ljs@kernel.org; h=from:subject:message-id; bh=henzipa1JQvP3ZJSUdVpc3i3Gyb5R1tpkdjoqO30NaM=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2LciKU2R7ePSEh5juaqtbvSfO7/tsdvP2okrVGpW4O S1pJfNPdZSyMIhxMciKKbI8/yK+P0gkbF7nBX83mDmsTCBDGLg4BWAiVy8z/E9ov/RlY+ua9MdM 6+s1Yp/Jfrky6d3F/7/bC1/kWQaf9Y5hZLj38RDHHLVN1kKTwqNu25zf+0q/e6HuefG01W+Lpf+ b9LAAAA== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 collapse_huge_page() deposits a PTE page table on PMD collapse in order that it can be utilised for subsequent split operations, meaning that those operations do not need to perform an allocation (as they are in a context where it might be unwise). However the PTE page table which is deposited is the one which is currently mapped by the PMD entry that is in the process of being collapsed. Once deposited, the PTE page table may be used in a split of any other unrelated PMD entry. This is currently not an issue as this operation is performed with VMA/mmap write lock + anon rmap locks held, so ordinary page table walkers will never accidentally end up walking the wrong thing, and GUP-fast is protected by an IPI via tlb_remove_table_sync_one(). However, the series to which this commit belongs implements RCU-safe page table traversal, at which point this becomes problematic. This can be resolved by using pte_offset_map_lock() which gates on a PTE PTL and a pmd_same() check, but lockless walks are unsafe as things stand. Resolve this by simply allocating a new, zeroed, PTE page table to deposit at the point of collapse. This path is already costly and an allocation has already been performed for the huge folio, so this allocation is statistical noise in terms of performance and memory usage at this point. With this PTE page table deposited, RCU-free the existing PTE page table so it is safe for page table walkers to traverse within a grace period. This also brings this deposit case in line with all other page table deposit logic which deposit a fresh page table. Additionally, this was the only place in the kernel that displaced a page table like this, so eliminating it also helps consistency. An edge case for deposit exists for powerpc and its hash-based MMU - it stores hash slot data in deposited page tables and zeroes them on withdraw, so a zeroed deposited page table works correctly for it. Since khugepaged runs as a kernel thread, do a little dance in alloc_deposit_pte() to correctly charge the allocation. This is already done for the folio allocation via alloc_charge_folio() but no such wrapper exists for a page table allocation. Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Reviewed-by: Baolin Wang Reviewed-by: Lance Yang --- mm/khugepaged.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f49a6710933b..dab421f8233e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1278,6 +1278,23 @@ static enum scan_result alloc_charge_folio(struct fo= lio **foliop, struct mm_stru return SCAN_SUCCEED; } =20 +static pgtable_t alloc_deposit_pte(struct mm_struct *mm) +{ + /* + * khugepaged is run from a kernel thread, so need to manually set the + * correct memcg so the allocation gets charged correctly. + */ + struct mem_cgroup *memcg =3D get_mem_cgroup_from_mm(mm); + struct mem_cgroup *old_memcg =3D set_active_memcg(memcg); + pgtable_t pgtable; + + pgtable =3D pte_alloc_one(mm); + + set_active_memcg(old_memcg); + mem_cgroup_put(memcg); + return pgtable; +} + /* * collapse_huge_page() expects the mmap_lock to be unlocked before enteri= ng and * will always return with the lock unlocked, to avoid holding the mmap_lo= ck @@ -1293,7 +1310,7 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long s LIST_HEAD(compound_pagelist); pmd_t *pmd, _pmd; pte_t *pte =3D NULL; - pgtable_t pgtable; + pgtable_t pgtable =3D NULL; struct folio *folio; spinlock_t *pmd_ptl, *pte_ptl; enum scan_result result =3D SCAN_FAIL; @@ -1310,6 +1327,12 @@ static enum scan_result collapse_huge_page(struct mm= _struct *mm, unsigned long s goto out_nolock; } =20 + if (is_pmd_order(order)) { + pgtable =3D alloc_deposit_pte(mm); + if (!pgtable) + goto out_nolock; + } + mmap_read_lock(mm); result =3D hugepage_vma_revalidate(mm, pmd_addr, /*expect_anon=3D*/ true, &vma, cc, order); @@ -1433,8 +1456,8 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long s spin_lock(pmd_ptl); VM_WARN_ON_ONCE(!pmd_none(*pmd)); if (is_pmd_order(order)) { - pgtable =3D pmd_pgtable(_pmd); pgtable_trans_huge_deposit(mm, pmd, pgtable); + pgtable =3D NULL; map_anon_folio_pmd_nopf(folio, pmd, vma, pmd_addr); } else { /* @@ -1453,6 +1476,9 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long s } spin_unlock(pmd_ptl); =20 + if (is_pmd_order(order)) + pte_free_defer(mm, pmd_pgtable(_pmd)); + folio =3D NULL; =20 result =3D SCAN_SUCCEED; @@ -1463,6 +1489,8 @@ static enum scan_result collapse_huge_page(struct mm_= struct *mm, unsigned long s anon_vma_unlock_write(vma->anon_vma); mmap_write_unlock(mm); out_nolock: + if (pgtable) + pte_free(mm, pgtable); if (folio) folio_put(folio); trace_mm_collapse_huge_page(mm, result =3D=3D SCAN_SUCCEED, result, order= ); --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3CB5755820D; Tue, 22 Sep 2026 15:37:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091442; cv=none; b=hicN0/Gh5PAfA9+0mDBDWt2LblfbDSIuC30IlPqMA8cRfAJoPSFb89/gMsDeRqn8xOaZZPDkXKPhnwvBU2yE10nM+jMoRxYjAfhHQup3Od9K0jiHaKWe5zRJJ4cDjPv64eg/sf6gwZRGI2TMfkfBjGdm/xJsCSXnOysXF3Qmrg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091442; c=relaxed/simple; bh=bwuaeHFGwuLQFcHN0hOhjdTFxvn+j0g8fIWNrCdz0hk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=k4w1LO0ZYceJOF3cfweE5kiUXuU3FuGJQkFQP7tt8Oruo7BhZnThrfZn8oQqayoQkuCR+0ON5Y8UuOMGLzz4yFVAolk67kctqPJD1F7Fd+70fO5gls0UskITGeU10NwuKI4ArCkf6GsU+moJ8A1SOpsp3lH0+jSgsIMvB9tZ9GY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=A8I7oZL7; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="A8I7oZL7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4F4FC1F00899; Tue, 22 Sep 2026 15:36:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091440; bh=EytwztXVP9+HAqnBzZ3KHLRHej5X7gcr6/abe7XoyR8=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=A8I7oZL76bgc/K4OnZ0JyOMCleMrv8FdbRx4ZImWL6TzUOPy27OJNysbdg4leGGta Lx4vSFm7V5JKYxrVGD86zIiIioAiFtHv9yN3wsYYID68QpWVVMN3tQku4l98Qy4zgw qF27kTj7WZm0/RooazviLjtxmwgEqpzuJcU5c7iIeBV8ngO+1ipt0+uSiSCb1hYR9b FeqTZuRHz8OvBlBx/kNDAhn+A5qaMG/FK1Kz/HdRa6QnK91S6WhAahUdZR52ic+UQ6 m4kfJDCDMoP3ZT2i41bOmIUPh/ZPP4TnOLJOrrC+n/gp89K3MgMVpMT2NZUEDoQZj+ OSQ/pfkvGm+LQ== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:33 +0100 Subject: [PATCH v4 02/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for most 2-level architectures Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-2-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=3613; i=ljs@kernel.org; h=from:subject:message-id; bh=bwuaeHFGwuLQFcHN0hOhjdTFxvn+j0g8fIWNrCdz0hk=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2Lch++epj9MNnHpEe9yYlHD30J+h8XF7vnb1CF/a1l 1Vpn6xe2FHKwiDGxSArpsjy/Iv4/iCRsHmdF/zdYOawMoEMYeDiFICJNO1n+J9tETupvXJjzarT rwX+hry6vcpP5urt1Q9MTtum66QcKIlk+B9bHRHH1mdsW7N44armiqcpC9oZ5195ZXdRTZPRdKv ybXYA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Commit e3ecf7c7d082 ("mm: pgtable: convert some architectures to use tlb_remove_ptdesc()") updated a number of architectures from using pagetable_dtor() + tlb_remove_page_ptdesc() to using tlb_remove_ptdesc() in __pte_free_tlb(). This is meaningful as tlb_remove_ptdesc() allows for RCU page table freeing if CONFIG_MMU_GATHER_RCU_TABLE_FREE is specified. The csky, hexagon, nios2, openrisc, sh (except X2) and m68k-sun3 architectures all have 2 levels of page tables, so the only page tables ever freed by mmu_gather are PTEs, so this update suffices to ensure that every page table freed by the mmu_gather mechanism is freed under RCU. Therefore, update all of these architectures to select CONFIG_MMU_GATHER_RCU_TABLE_FREE. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Reviewed-by: Lance Yang --- arch/csky/Kconfig | 1 + arch/hexagon/Kconfig | 1 + arch/m68k/Kconfig | 1 + arch/nios2/Kconfig | 1 + arch/openrisc/Kconfig | 1 + arch/sh/Kconfig | 1 + 6 files changed, 6 insertions(+) diff --git a/arch/csky/Kconfig b/arch/csky/Kconfig index 4331313a42ff..80f89ef1d962 100644 --- a/arch/csky/Kconfig +++ b/arch/csky/Kconfig @@ -96,6 +96,7 @@ config CSKY select HAVE_SYSCALL_TRACEPOINTS select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE select MAY_HAVE_SPARSE_IRQ select MODULES_USE_ELF_RELA if MODULES select OF diff --git a/arch/hexagon/Kconfig b/arch/hexagon/Kconfig index b48491140013..d9b3fb86556b 100644 --- a/arch/hexagon/Kconfig +++ b/arch/hexagon/Kconfig @@ -23,6 +23,7 @@ config HEXAGON # select HAVE_CLK select GENERIC_ATOMIC64 select HAVE_PERF_EVENTS + select MMU_GATHER_RCU_TABLE_FREE # GENERIC_ALLOCATOR is used by dma_alloc_coherent() select GENERIC_ALLOCATOR select GENERIC_IRQ_PROBE diff --git a/arch/m68k/Kconfig b/arch/m68k/Kconfig index 11835eb59d94..e29610fd1240 100644 --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -36,6 +36,7 @@ config M68K select HAVE_MOD_ARCH_SPECIFIC select HAVE_UID16 select MMU_GATHER_NO_RANGE if MMU + select MMU_GATHER_RCU_TABLE_FREE if MMU && SUN3 select MODULES_USE_ELF_REL select MODULES_USE_ELF_RELA select NO_DMA if !MMU && !COLDFIRE diff --git a/arch/nios2/Kconfig b/arch/nios2/Kconfig index 9c0e6eaeb005..b0ccfc3b7a7e 100644 --- a/arch/nios2/Kconfig +++ b/arch/nios2/Kconfig @@ -19,6 +19,7 @@ config NIOS2 select HAVE_PAGE_SIZE_4KB select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select OF select OF_EARLY_FLATTREE diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig index 5eb995c13074..d90b24dd3bce 100644 --- a/arch/openrisc/Kconfig +++ b/arch/openrisc/Kconfig @@ -35,6 +35,7 @@ config OPENRISC select GENERIC_ATOMIC64 select GENERIC_CLOCKEVENTS_BROADCAST select GENERIC_SMP_IDLE_THREAD + select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select HAVE_DEBUG_STACKOVERFLOW select OR1K_PIC diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig index d60f1d5a94c0..204f64912f0e 100644 --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -61,6 +61,7 @@ config SUPERH select HAVE_SYSCALL_TRACEPOINTS select IRQ_FORCED_THREADING select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE if MMU && !X2TLB select MODULES_USE_ELF_RELA select NEED_SG_DMA_LENGTH select NO_DMA if !MMU && !DMA_COHERENT --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3103955C1D5; Tue, 22 Sep 2026 15:37:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091464; cv=none; b=epzAVfxYyFNEiuP7ZTgVmtv96xyFAHDQcuzSBZLKno3FFd9hHTR/NZjN0Ud/fR1+mnieo0Zztd/skUKTw/i+uXXze+kD43JzhoshnZokcL2ukaOWOk8IdzF1RxwBRswiY4CWXtQ00bBB0DwfArNn1zmjXfb1iYT+2VIqLIMPeEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091464; c=relaxed/simple; bh=Ejk+F/Wux5yTvQzPF878nAZnzdraz7QYUlz/y227QJE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aKTDlLRj6WJ168zhyn8jU+0CkTqHHPAhCi2EoIA/xVzgsS5a1k/Jo5W7CR9PqoSlZyVqF9AviDScWholowT3yzUcr/OxAv9XGfmGJIhd0ieWKMUuhYMDrYzHB0cQdt0JAzE2kw9ZnG9oSXv1sdip9+0jJ95Rx5412YxlhskSW6s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UNlcNUEG; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UNlcNUEG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6DA911F000FF; Tue, 22 Sep 2026 15:37:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091462; bh=dWYqXXcVhqhum5QoL45NNyuXa4EfpjGCy/961eqdzvI=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=UNlcNUEGwtkyvsYR4zmSeGCvMkPLe+/ezusdX20O1bWYskX7cvdvIJuhhjuVtChmb L5+b54M4DXoNchVsv0nJ54pnhIsdEfA5oI47YpiDtjUztTpT4qzogzlKrUnLoaDon0 Hi/50oU8SeABFn/+BsGw1RL0cXsZ9uni0c7ibjurbKnftlBLuqaA2M5V5zWFn9PE1b n2j5Mady9bfFc8fKJ22XaOT4EwjJPIZOiQbv+8Od9Wmuwkvc0lPJh+Kjy/qRzctIDv BhT9VIJmX/JVBTgDo4khmxS4qLbtj4ncYW3ywiP/LSPDFHak7FA0nynQk8qUTpmRo7 4FhJkMkThxUjw== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:34 +0100 Subject: [PATCH v4 03/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for MMU riscv Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-3-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1601; i=ljs@kernel.org; h=from:subject:message-id; bh=Ejk+F/Wux5yTvQzPF878nAZnzdraz7QYUlz/y227QJE=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2LchuC1lS+yl/saWZ4KXsS/+f/mMRKmHc9UBbod6B2 YU98CxHRykLgxgXg6yYIsvzL+L7g0TC5nVe8HeDmcPKBDKEgYtTACYiP5Phv+8ioR33clR2yb5m l1T66KMtEtO5qfBsf+mk6EU66XL/njAy7OdZ56cxmWV7qvxM/cnbUvUPnrh05fThTHmL9/6/nNY oMgEA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Currently riscv gates MMU_GATHER_RCU_TABLE_FREE on CONFIG_SMP and CONFIG_MMU. Commit 69be3fb111e7 ("riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU") enabled CONFIG_MMU_GATHER_RCU_TABLE_FREE for CONFIG_SMP, CONFIG_MMU riscv builds. This is expressly for the safety of GUP-fast walkers (CONFIG_HAVE_GUP_FAST is enabled if CONFIG_MMU is enabled). Naturally a single core system does not encounter issues with software page table walkers being correctly synchronised across cores, as there is only a single core. However, CONFIG_PREEMPT_RCU is still available on a riscv UP system, so for a future RCU-only page table walker, this guarantee is required to prevent concurrent page table teardown. All page table freeing is already done via tlb_remove_ptdesc() so the conditions of CONFIG_MMU_GATHER_RCU_TABLE_FREE are already met. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Lance Yang Tested-by: Lance Yang --- arch/riscv/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 505eed4af932..3529ed1861ce 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -208,7 +208,7 @@ config RISCV select IRQ_FORCED_THREADING select KASAN_VMALLOC if KASAN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if SMP && MMU + select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA if MODULES select OF select OF_EARLY_FLATTREE --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCA1E55C1C2; Tue, 22 Sep 2026 15:38:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091486; cv=none; b=TcV60232nidtwsCDEpWRakkjye7WTNz3ekN+m4ZSBSzIQdbPWsYHN9YBw+PuA3X2Tb1t5XEGPZ4gPoyanrLyhJ9CVPUSU8wl+xae0rup9QQXtYT/Nm3TeMBlWdj5YgaQwZVFtEhzMEaEGSeLauxWmDV6tmRuRw7RYleVETDxfUc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091486; c=relaxed/simple; bh=DfBAaXdC7t7vNoXwDFH4KxY7MB20la/qqFRKCtU73zE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jHQqV644E2haJeiZRhMC90KRzAL2JALLSkiXaDD2J59yeVmZRnjdSpfHgeGnRffpY37b5gjSetzT1244tlsFZ7C74iloKXEFw6d8eYCL3K9zQq19NaIP/liZi9Ip3tbngVN08upQNtXdP936YRP8N/O5+NfPoHs5Wtk9qTq2jlk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=baLVS6KH; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="baLVS6KH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4F1AD1F00893; Tue, 22 Sep 2026 15:37:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091484; bh=3qIS4A8O4N5dYDsCKKL4hiQNRSaqHHy/s6/A7yaKLWg=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=baLVS6KH22XEePkcCK/XQL5WUTQhEfQO+oVeyTZCmHU5hvgW/sC6ri+yz+fZAEvUu 4AB4f9B49fUmMJLa00WCl/Lq29SV/3o2xefFCut0zjtkzvHJxT2gLbBl6t/7FWSah2 cSFBx9oSTs3cHExoGi7w0wc7bej6do9PXjbWtdyMa5VjjS6sF0POU26Vm/6/jtKgE2 NUet2CcbMqFq9W3zLDl2+ZbTlY14Hv3I9LNx4pdfIGzHNrM7J3UDOAF8heZnHAptmF QbMCEdCV0Sqooq2HWiC0wBKEP4F5ZEAKre1J+LA0yZT213KYQ7ZAoYR+7OowJbWVy/ GA+TVgecxHh8Q== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:35 +0100 Subject: [PATCH v4 04/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for MMU arm Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-4-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1983; i=ljs@kernel.org; h=from:subject:message-id; bh=DfBAaXdC7t7vNoXwDFH4KxY7MB20la/qqFRKCtU73zE=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2LcgWNO2ZyhxnUcn2yn+6ab7p3uPZ3pZnN3/ZwLHOx /+0md2JjlIWBjEuBlkxRZbnX8T3B4mEzeu84O8GM4eVCWQIAxenAEzkeT7D/5S0tP6NOvsf9/jM NlwxdcVR6zZvveSOBtuop/zxvYaTGhn+KX384XMub+sLLqYbIeFdF9Ts3numFxt4mpcFR7aFZ6x gBgA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Commit a0ad5496b2b3 ("arm: mm: enable HAVE_RCU_TABLE_FREE logic") enabled CONFIG_MMU_GATHER_RCU_TABLE_FREE (then named HAVE_RCU_TABLE_FREE) for SMP arm architectures with LPAE enabled. Regardless of whether CONFIG_ARM_LPAE is enabled or not, the same page table freeing functions __pte_free_tlb() and __pmd_free_tlb() are used. Non-LPAE PMD page tables are folded into the PGD and freed by pgd_free() (PGD freeing is not part of mmu_gather page table freeing in any case), so this is a noop in this case. Since commit 358d1c39c82a ("arm: convert various functions to use ptdescs") both LPAE and non-LPAE PTE page table freeing uses tlb_remove_ptdesc(). Thus all page table freeing is performed under RCU with CONFIG_MMU_GATHER_RCU_TABLE_FREE enabled for LPAE and non-LPAE and thus it need not be gated on LPAE. A UP arm system can set CONFIG_PREEMPT_RCU, so a future pure RCU page table walker requires MMU_GATHER_RCU_TABLE_FREE to be enabled on UP as well, even if concurrent GUP fast is not possible there. Therefore, it is both safe and desirable to set CONFIG_MMU_GATHER_RCU_TABLE_FREE for all MMU arm architectures (nommu does not perform mmu_gather operations). This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Lance Yang Tested-by: Lance Yang --- arch/arm/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 408aa58a2a5b..72b9afc6ae10 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -134,7 +134,7 @@ config ARM select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP select HAVE_POSIX_CPU_TIMERS_TASK_WORK - select MMU_GATHER_RCU_TABLE_FREE if SMP && ARM_LPAE + select MMU_GATHER_RCU_TABLE_FREE if MMU select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RSEQ select HAVE_RUST if CPU_LITTLE_ENDIAN && CPU_32v7 && !KASAN --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0585C47D92B; Tue, 22 Sep 2026 15:38:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091508; cv=none; b=LlSK7+WpXyLzWI3bVCLpAB1mkw9bzqZA+ZqZmVcgp3g0fi9oietl31BDsNQZeeIswebDWsYMTAZimlEMCwr7cjPWTT8LnVwWWYhnFOYAQl5IH+8cIiq0d3AMqmnhINKvd7zcbNSEwzNmLtW8XurJy/Xcvv0F7UuAgymz/Q1Njd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091508; c=relaxed/simple; bh=uQXm1gV0hImCHIW2c39SJsX9YEkAV3QIf8B4ruAyqy0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kNL5RgfwYtX7KCt7l1XVdR5udhe6c2/pIExAWkQUzhPnenfWHTRtDdu/70yr4ioPx7ScEQ1hDSsaap/r5xx04YNYplvHQ+Lq/tn90h+qat2E5yQ35+/f/8/08oWqUBjBnCg2L8muYx/kX9RTzv+GGokNG2tomAPI8CIIWBWM8Sk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hczF2y71; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hczF2y71" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30E801F000FF; Tue, 22 Sep 2026 15:38:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091506; bh=BckCZCGc4vzG9wqcN+JQhNHkdFRqc5QwTzQVV8XEoLM=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=hczF2y71AlJMYpu4N67vVKHZULkCEsC1jdewMZf7zTZ68jmEJbkuYDjvGHkzAlrru VVTdGRic7q6bI25hOgzKaKwMw1bldXcdOufOkY/y9kjsEE31UuSkp1gvLn2NcaKv3/ 6JD+BuUCvN7AGim/QrSZ2leQdE8ROIUA54ZaSPB0+pmV/wH6SN54d4ek6/0IDIuLN5 OQ8VIEYaHaNBcfKUBUIJ4RnRmH+5ZkL1rbIqcZqm3fTH42Wba2SJklOLkUaC+5ffP0 2NWVTmu21ufgyXLytMnKYyfbzPaGAjDW1X9n1/hKR8g84+eML9mer0DECeMEkjiACf VOOKF2njdqfCA== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:36 +0100 Subject: [PATCH v4 05/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for arc, microblaze, xtensa Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-5-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4685; i=ljs@kernel.org; h=from:subject:message-id; bh=uQXm1gV0hImCHIW2c39SJsX9YEkAV3QIf8B4ruAyqy0=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2LcjeUX3q7EYBFY/bNxY03exr0ZGNiVeaKWfIdXDJF m4n8YynHaUsDGJcDLJiiizPv4jvDxIJm9d5wd8NZg4rE8gQBi5OAZjILjuGf9pveG59W2cgdnDr 3ZkaK3JCSw2vTI6ZuPlekWtIEVtr8wRGhslfOw5PWHx2+0o25qi9LnzJ5S9mv1t6/i+7ibx3QKn fCVYA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Each of these architectures directly free page tables without routing these changes through tlb_remove_ptdesc(). The use of tlb_remove_ptdesc() is required for CONFIG_MMU_GATHER_RCU_TABLE_FREE to correctly free page tables under RCU, so simply update these architectures to use these functions. Since none of the architectures share page tables or do anything unusual, nothing complicated is required here. Therefore this is simply a mechanical change - convert __pud_free_tlb(), __pmd_free_tlb() and __pte_free_tlb() to use tlb_remove_ptdesc() as required. At the point this is in place, all mmu_gather page table freeing is performed under RCU, and thus MMU_GATHER_RCU_TABLE_FREE is selected for each architecture. Note that CONFIG_MMU_GATHER_RCU_TABLE_FREE is dependent on CONFIG_MMU for xtensa to reflect the fact that nommu does not implement page table gathering. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Lance Yang --- arch/arc/Kconfig | 1 + arch/arc/include/asm/pgalloc.h | 6 +++--- arch/microblaze/Kconfig | 1 + arch/microblaze/include/asm/pgalloc.h | 2 +- arch/xtensa/Kconfig | 1 + arch/xtensa/include/asm/tlb.h | 2 +- 6 files changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig index 2ed7186c81c5..7a7542b61823 100644 --- a/arch/arc/Kconfig +++ b/arch/arc/Kconfig @@ -47,6 +47,7 @@ config ARC select HAVE_SYSCALL_TRACEPOINTS select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select OF select OF_EARLY_FLATTREE diff --git a/arch/arc/include/asm/pgalloc.h b/arch/arc/include/asm/pgalloc.h index dfae070fe8d5..9b6c37f92e97 100644 --- a/arch/arc/include/asm/pgalloc.h +++ b/arch/arc/include/asm/pgalloc.h @@ -72,7 +72,7 @@ static inline void p4d_populate(struct mm_struct *mm, p4d= _t *p4dp, pud_t *pudp) set_p4d(p4dp, __p4d((unsigned long)pudp)); } =20 -#define __pud_free_tlb(tlb, pmd, addr) pud_free((tlb)->mm, pmd) +#define __pud_free_tlb(tlb, pmd, addr) tlb_remove_ptdesc((tlb), virt_to_p= tdesc(pmd)) =20 #endif =20 @@ -83,10 +83,10 @@ static inline void pud_populate(struct mm_struct *mm, p= ud_t *pudp, pmd_t *pmdp) set_pud(pudp, __pud((unsigned long)pmdp)); } =20 -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) tlb_remove_ptdesc((tlb), virt_to_p= tdesc(pmd)) =20 #endif =20 -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, addr) tlb_remove_ptdesc((tlb), page_ptde= sc(pte)) =20 #endif /* _ASM_ARC_PGALLOC_H */ diff --git a/arch/microblaze/Kconfig b/arch/microblaze/Kconfig index 484ebb3baedf..af7e821e96c1 100644 --- a/arch/microblaze/Kconfig +++ b/arch/microblaze/Kconfig @@ -41,6 +41,7 @@ config MICROBLAZE select PCI_SYSCALL if PCI select CPU_NO_EFFICIENT_FFS select MMU_GATHER_NO_RANGE + select MMU_GATHER_RCU_TABLE_FREE select SPARSE_IRQ select ZONE_DMA select TRACE_IRQFLAGS_SUPPORT diff --git a/arch/microblaze/include/asm/pgalloc.h b/arch/microblaze/includ= e/asm/pgalloc.h index 084a8a0dc239..ffee6a009219 100644 --- a/arch/microblaze/include/asm/pgalloc.h +++ b/arch/microblaze/include/asm/pgalloc.h @@ -25,7 +25,7 @@ extern void __bad_pte(pmd_t *pmd); =20 extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm); =20 -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, (pte)) +#define __pte_free_tlb(tlb, pte, addr) tlb_remove_ptdesc((tlb), page_ptdes= c(pte)) =20 #define pmd_populate(mm, pmd, pte) \ (pmd_val(*(pmd)) =3D (unsigned long)page_address(pte)) diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig index f2f9cd9cde50..33c4caee30e2 100644 --- a/arch/xtensa/Kconfig +++ b/arch/xtensa/Kconfig @@ -55,6 +55,7 @@ config XTENSA select HAVE_VIRT_CPU_ACCOUNTING_GEN select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA select PERF_USE_VMALLOC select TRACE_IRQFLAGS_SUPPORT diff --git a/arch/xtensa/include/asm/tlb.h b/arch/xtensa/include/asm/tlb.h index 8c3ceb427018..6fb7b78154f6 100644 --- a/arch/xtensa/include/asm/tlb.h +++ b/arch/xtensa/include/asm/tlb.h @@ -16,7 +16,7 @@ =20 #include =20 -#define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, address) tlb_remove_ptdesc((tlb), page_pt= desc(pte)) =20 void check_tlb_sanity(void); =20 --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB54E47D92B; Tue, 22 Sep 2026 15:38:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091530; cv=none; b=I9stJWpMHbBRk3x6Kt5NoFj9F7KZx1FP1NbTFNzFMbW79VNS+vIazwoY9VxwPaucoOF6qB16slFimLdqe2gJ8Ry0DtSPR8O6CSyZru+k3Whl6B8aGdKLcguVtQUU1Mm3iGsCG2LVkZBCyoVXnMxniGWQgWx5mjBWCCt3gn6Xl2k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091530; c=relaxed/simple; bh=56BKeeSS+gh75h2n7cGmrCDtrwD4bozOQj93IG348BU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=A5Aa4OajoZsqv9lAhDB2BiS7sYme92uk1NOot8/ALn3zcYXNziV0gdhHASKaMY1yiVdY/T2dWqGF/G/h1TG2Hvzq1mGKVbqw8jUOKU9Dz/s4E4/yrUzmcSgKSgb0ZtJyWyH2ehNXg/2gYz3WbuNtsij8LrgPkjUaF+eDtJVhZ1s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aBRxSFnR; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aBRxSFnR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3BCC91F00893; Tue, 22 Sep 2026 15:38:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091528; bh=88XZdMnGcr4EoxTKx6ho5XutcNaS7FDI1j17HitUiT4=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=aBRxSFnR+u4B1t5yIchx2A0ERyJsj4YEBy2FNdmU26YxeMEhi9gcCb9tHneGyWX7V nJA2n0nAZt6niS45hsrEtz5mRv+bmTHWLyL+oJf7iHXJbtTqWg6U1myGyYN+KuL2bF WAcY219MdKdF4UBWJmkkHs8N79H5ZCyLjW9lswGRy9mgt5BsV1MqRC/Dtd2Cv4O/uK bHQtLRZzDaNCaipCAP2+QUoU6EvGdFIo3ByBFB0P5J2sY22d0mc7G4O3QBgiFgcVA0 yHQ8960BuIKrhx2onEMQBi4KdiPkZDUhRy0xiR0ugjg9aW3G3HPY34Ob27RIyRZOuj 7Eruk8db7fl8g== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:37 +0100 Subject: [PATCH v4 06/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for sparc64 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-6-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2472; i=ljs@kernel.org; h=from:subject:message-id; bh=56BKeeSS+gh75h2n7cGmrCDtrwD4bozOQj93IG348BU=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2Lcje8/i97r23G/ndN52avMTmW9S+qlW/r8/evv/+e b8tfz5qqXeUsjCIcTHIiimyPP8ivj9IJGxe5wV/N5g5rEwgQxi4OAVgIhKrGf7KiPTw12vyPt4o 9mE5o2e9KNNMm0SDRRbfuZ/6560W6Qph+MN9YMfdhZ5fxJqmhNXvCP069wGvroPxyimr7v2dvTv UbB4XAA== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Commit 4a0100f7546f ("sparc64: use RCU page table freeing") enabled CONFIG_MMU_GATHER_RCU_TABLE_FREE for SMP sparc64 architectures, expressly for GUP-fast page table walkers. Naturally, UP systems do not have to worry about concurrent GUP fast operations. However, CONFIG_PREEMPT_RCU is also available even on a UP system, so a future pure-RCU page table walker requires MMU_GATHER_RCU_TABLE_FREE to be enabled on UP, even if concurrent GUP fast is not possible there. To enable future pure-RCU page table walkers, enable MMU_GATHER_RCU_TABLE_FREE unconditionally. With this change, it is no longer necessary to have !CONFIG_SMP pgtable_free_tlb(), so also remove this now dead code. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Lance Yang Tested-by: Lance Yang --- arch/sparc/Kconfig | 4 ++-- arch/sparc/include/asm/pgalloc_64.h | 8 -------- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index ab77d3f2536e..8d42ebc6d302 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -75,8 +75,8 @@ config SPARC64 select HAVE_FUNCTION_GRAPH_TRACER select HAVE_KRETPROBES select HAVE_KPROBES - select MMU_GATHER_RCU_TABLE_FREE if SMP - select HAVE_ARCH_TLB_REMOVE_TABLE if SMP + select MMU_GATHER_RCU_TABLE_FREE + select HAVE_ARCH_TLB_REMOVE_TABLE select MMU_GATHER_MERGE_VMAS select MMU_GATHER_NO_FLUSH_CACHE select HAVE_ARCH_TRANSPARENT_HUGEPAGE diff --git a/arch/sparc/include/asm/pgalloc_64.h b/arch/sparc/include/asm/p= galloc_64.h index caa7632be4c2..b5055d259b74 100644 --- a/arch/sparc/include/asm/pgalloc_64.h +++ b/arch/sparc/include/asm/pgalloc_64.h @@ -74,8 +74,6 @@ void pte_free_defer(struct mm_struct *mm, pgtable_t pgtab= le); =20 void pgtable_free(void *table, bool is_page); =20 -#ifdef CONFIG_SMP - struct mmu_gather; void tlb_remove_table(struct mmu_gather *, void *); =20 @@ -96,12 +94,6 @@ static inline void __tlb_remove_table(void *_table) is_page =3D true; pgtable_free(table, is_page); } -#else /* CONFIG_SMP */ -static inline void pgtable_free_tlb(struct mmu_gather *tlb, void *table, b= ool is_page) -{ - pgtable_free(table, is_page); -} -#endif /* !CONFIG_SMP */ =20 static inline void __pte_free_tlb(struct mmu_gather *tlb, pte_t *pte, unsigned long address) --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95E2E5616C7; Tue, 22 Sep 2026 15:39:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091552; cv=none; b=naTmZNTs6cnEZN9DWcma6Omph/hEbURqO4EaoFKJrElXK/daf83yF+/RctIXvZRlUAzSoLGD2NyVCiOKCb23hX0eu0Rez8vzzYs4JYpvAelMIDZIz78MMJauu6udoAz6fwWKjQLj/8qAiOWjRtNZ5nEcEs6zjttyl9j9gdytzMo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091552; c=relaxed/simple; bh=BvcyYBL/6DvMpTsf26rSU6vpsqM4Qkuzt9TqOJJPzlE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=U5BBkmo8UP72OuMynv3Iz+OLLriCuEcQiCvpAiLx3Xx5/43FNrsgoLkExaPEFI+Kk+vSgt4GPFTLFABImqNt04qYx+ydaVFhFwj4D2+jhtwPq2MzRD1dGis9jhd5H0O/DqyW23MD4nobCcqPw32yescJvYZb5ZkIuvEQ69h5F1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YMqHdadW; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YMqHdadW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 104DB1F000FF; Tue, 22 Sep 2026 15:38:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091550; bh=AAMKCdWX5FPiRKf/xR0S9vZ7DBMPzl1KsARK3tocs/E=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=YMqHdadWNJEO3nv+PyQF5bVEZubijamwPceNJ4F8tT2+2ZTV+UP1+hpAtqqBUk1Fd voiXsP6pb6YTENL+WTRClpOH0TMQ40BAFAqSgDKZIOVDwLm1HJoQ6G0eRWVohQdgum Nl8dEFGs1KeJ0KvfjLwvm1mRbV/OC+0QARjQRhXrmu+CUCXPtNUIlItZ9fmFFh/4lk Cd+7gDypesGbNgTkvr4sgDfjCXnp+ZHQNy74dphXcSKDVwEt33Sx2mZ81N4Kql5n8E VPYMPcglZkBnOox3M7uLnWPrS83jiJKvEcBCwqKNcVq2pwp+Ns6KW1BlRLT+YsBTdv pSwqpEDGHSJtg== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:38 +0100 Subject: [PATCH v4 07/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for m68k-coldfire Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-7-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1893; i=ljs@kernel.org; h=from:subject:message-id; bh=BvcyYBL/6DvMpTsf26rSU6vpsqM4Qkuzt9TqOJJPzlE=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2LchO7lPnkNFb6m/L/nyu7j/vVyJfNO9dt7n/f2af5 sKaK4e6O0pZGMS4GGTFFFmefxHfHyQSNq/zgr8bzBxWJpAhDFycAjCR1ycY/nsf0VwbWJQ+SXe1 Udg/Z9f7lsvu9GmKiXzd+u5tv/JZQW6G/6GVnR+mLzi8ee7FGXqSBg6bjuWJnz3tl357s8ulqb5 F/lwA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Similar to sun3, the coldfire variant of m68k uses 2-level page tables. Update its __pte_free_tlb() function to use tlb_remove_ptdesc() in order that, with CONFIG_MMU_GATHER_RCU_TABLE_FREE, page tables are freed under RCU. The page tables occupy a page each and have no odd semantics, so this change suffices to allow enabling of CONFIG_MMU_GATHER_RCU_TABLE_FREE for m68k-coldfire, so do so. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Greg Ungerer Acked-by: Lance Yang Tested-by: Greg Ungerer --- arch/m68k/Kconfig | 2 +- arch/m68k/include/asm/mcf_pgalloc.h | 5 +---- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/m68k/Kconfig b/arch/m68k/Kconfig index e29610fd1240..6b8ec67c86fd 100644 --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -36,7 +36,7 @@ config M68K select HAVE_MOD_ARCH_SPECIFIC select HAVE_UID16 select MMU_GATHER_NO_RANGE if MMU - select MMU_GATHER_RCU_TABLE_FREE if MMU && SUN3 + select MMU_GATHER_RCU_TABLE_FREE if MMU && (SUN3 || COLDFIRE) select MODULES_USE_ELF_REL select MODULES_USE_ELF_RELA select NO_DMA if !MMU && !COLDFIRE diff --git a/arch/m68k/include/asm/mcf_pgalloc.h b/arch/m68k/include/asm/mc= f_pgalloc.h index fc5454d37da3..b53ff0950db2 100644 --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -39,10 +39,7 @@ extern inline pmd_t *pmd_alloc_kernel(pgd_t *pgd, unsign= ed long address) static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtabl= e, unsigned long address) { - struct ptdesc *ptdesc =3D virt_to_ptdesc(pgtable); - - pagetable_dtor(ptdesc); - pagetable_free(ptdesc); + tlb_remove_ptdesc(tlb, virt_to_ptdesc(pgtable)); } =20 static inline pgtable_t pte_alloc_one(struct mm_struct *mm) --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDFF1556BA2; Tue, 22 Sep 2026 15:39:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091574; cv=none; b=iT7E5dxqMJobxKMyhozLU0/93KbL9lBnVtDaRdSvf0rNJ8Mev1U2acpr/kaukcpNojw9YfJQVqkfEwK+F8+L6kPBnc9OZ6gRMT/eAtTbLQLx+3UEQQCe0/G/xIYSWFvCo3SO4NAP7sXB1GOzIK0Am8Twgz7drJy8mprudFyHHj8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091574; c=relaxed/simple; bh=vMbfUQkzgTSKIjU6KPYoUnxQRGxa3kcmbXWa6+LkIAs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZMSTvRI+Wvq0EABOWoLniUfdrphH2gH4+gEdk1fYAs5xoW9PmIK6LArGGj9ZeL73mLR2acMf8xkuKArwKQxPuHWlUiG/WYwn3OyE7vQjikQsMPW2J8Lwy3Bs0vFI0x5ComX+KKIsvQdyseUIMnflPie/1rG3qCx53WSrtoQtpUw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bm22U7Fi; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bm22U7Fi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DC2761F00893; Tue, 22 Sep 2026 15:39:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091572; bh=ZF+xTol/hcpNyDp5nDHTBbzlkakusmjcTTc5URGnJdE=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=bm22U7FiSQN+ggAF3r9V2Q0hTdDX+haEkgL46cvJgXYTzo9vM+dDmqmTp/9VihJx9 SAYWsTV+vy+dbQpZKb9sP8GwOF+y1UmnWNMyeMBUje5FYSeaC4hKYJ5rn1h1QVROYs efwCayHiTBPyeuycqYp1H+zPMa928CepF/CWHc5i6rvbMd6bsdfXEJ/1JT/ma/gslH 9SxWv5o4OHw2VFBuo+ZThwnCN4O+Y0kIv9egppeagC7qOn9OEZjvPf/H2h8AtIeOgN azEjXleHU0E17mzmNlt7cHeEN8qRySe/5AzeXWgm5MV/ZURMFf47PIrjLE+QUEhKvU go/z03ESbnwtw== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:39 +0100 Subject: [PATCH v4 08/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for sh-X2 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-8-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4177; i=ljs@kernel.org; h=from:subject:message-id; bh=vMbfUQkzgTSKIjU6KPYoUnxQRGxa3kcmbXWa6+LkIAs=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2LchJrVl3zeaaXwmbwql3cltt8pgMlDSuhvuk5AVm7 6w8HqDaUcrCIMbFICumyPL8i/j+IJGweZ0X/N1g5rAygQxh4OIUgIn828LI8CH2f8W8VZv/XLr3 dUp+cWMge9j7g/ffCDZ7Lv2fsPD4iu+MDM+t3WJETxq7bDm//Y1TqcBOTyNBzhqvo3VPjkyVn/d 9Ig8A X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Currently, non-x2 sh specifies CONFIG_MMU_GATHER_RCU_TABLE_FREE allowing RCU page table freeing. sh-X2 is problematic because it utilises slab-allocated PMD page tables, and thus tlb_remove_ptdesc() cannot be used in these cases. All other sh variants are fine as commit e3ecf7c7d082 ("mm: pgtable: convert some architectures to use tlb_remove_ptdesc()") already converted page table freeing to use tlb_remove_ptdesc(), which does so after an RCU grace period when CONFIG_MMU_GATHER_RCU_TABLE_FREE is specified. Resolve this issue by firstly specifying CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE for sh-X2, so the arch can provide its own __tlb_remove_table() implementation (called after the RCU grace period). Then, convert __pmd_free_tlb() to tag the pointer to the PMD, and have __tlb_remove_table() check this tag to determine whether to free via the slab or to use pagetable_dtor_free(). This follows the pattern used by sparc64 as implemented in commit 4a0100f7546f ("sparc64: use RCU page table freeing"). Previously __pmd_free_tlb() freed PMD page tables immediately, before any TLB flush IPI. This seems to be a pre-existing bug, which this change also resolves. CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE is only specified for sh-X2, as setting it disables CONFIG_PT_RECLAIM and causes __tlb_remove_table_one() to call tlb_remove_table_sync_rcu() and synchronize_rcu() in turn, and this is not necessary for other sh variants. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Lance Yang --- arch/sh/Kconfig | 3 ++- arch/sh/include/asm/pgalloc.h | 6 +++++- arch/sh/mm/pgtable.c | 20 ++++++++++++++++++++ 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig index 204f64912f0e..75236bef6f16 100644 --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -33,6 +33,7 @@ config SUPERH select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_KGDB select HAVE_ARCH_SECCOMP_FILTER + select HAVE_ARCH_TLB_REMOVE_TABLE if X2TLB select HAVE_ARCH_TRACEHOOK select HAVE_DEBUG_BUGVERBOSE select HAVE_DEBUG_KMEMLEAK @@ -61,7 +62,7 @@ config SUPERH select HAVE_SYSCALL_TRACEPOINTS select IRQ_FORCED_THREADING select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU && !X2TLB + select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA select NEED_SG_DMA_LENGTH select NO_DMA if !MMU && !DMA_COHERENT diff --git a/arch/sh/include/asm/pgalloc.h b/arch/sh/include/asm/pgalloc.h index 6fe7123d38fa..67ce7fa23fa1 100644 --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -17,7 +17,11 @@ extern void pgd_free(struct mm_struct *mm, pgd_t *pgd); extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); -#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) +extern void __tlb_remove_table(void *table); + +/* PMDs are slab-allocated, tag so they are freed correctly. */ +#define __pmd_free_tlb(tlb, pmdp, addr) \ + tlb_remove_table((tlb), (void *)((unsigned long)(pmdp) | 1)) #endif =20 static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, diff --git a/arch/sh/mm/pgtable.c b/arch/sh/mm/pgtable.c index 3a4085ea0161..f6184b86b89c 100644 --- a/arch/sh/mm/pgtable.c +++ b/arch/sh/mm/pgtable.c @@ -56,4 +56,24 @@ void pmd_free(struct mm_struct *mm, pmd_t *pmd) { kmem_cache_free(pmd_cachep, pmd); } + +static void __tlb_remove_table_slab(void *table) +{ + kmem_cache_free(pmd_cachep, table); +} + +static void __tlb_remove_table_pgtable(void *table) +{ + pagetable_dtor_free(table); +} + +void __tlb_remove_table(void *table) +{ + const unsigned long addr =3D (unsigned long)table; + + if (addr & 1) + __tlb_remove_table_slab((void *)(addr & ~1UL)); + else + __tlb_remove_table_pgtable(table); +} #endif /* PAGETABLE_LEVELS > 2 */ --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DFF355D888; Tue, 22 Sep 2026 15:39:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091596; cv=none; b=ttbDJbxlDxzLN6a/69nyjJhkzAexqLilzWsF2kuzOkrF/adv0ab3fbX67v96pKM/QFLx1Lm65qWxp0Ppmz1ZE1tPbO2lOuD9WWtuiaL6RG23HYr9SabVNwVu0iFVmxmSeWzw4DQdkp7OG8Z7Sru/5CaYrmJ+ZL6gIw2vAXlRots= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091596; c=relaxed/simple; bh=p817yyNbZ4NhaQkL3kSpCEKgVmtWbKpmnpJsqVcbxII=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=uBIJr0IXwbHl5hl8bt1I51ZdEHbeMmEeuYutsLriPeMgsKNN7BxlZO2qda5rJU9Xz8YuWa8YXznWpf66HHTOB6TFbngqnjk7LdrZXw6Ffcnoi0juvNxynDgjnLuh5Yi+ajnd03UqpnJzGL2Xwn0jCh08KvvhepIg+W7L5mROmKw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hhpAWdpD; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hhpAWdpD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E77EE1F000FF; Tue, 22 Sep 2026 15:39:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091594; bh=rBG5/twDrSmSB3MHNHrdw9YcNjORS2eLnr9jQpeNRXk=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=hhpAWdpD0bvsD7jM4OC98q7pXiJr60RL902irU0pwWrUMQNgqCkrex1OAAWUhA7zb QWSVQsz5dFD73V3wdjQffK6t5uwzud1jYGqled6O803NYgWk8LrnvJopfonxPmQCh+ s2hoTuZcFAHRcG8RlhB6T4SNNqcBW3/TvzUlXdFeKmcdZmd2sVihHXMayer6EMPoX1 u7teLNtM+miWVxsFbLnT6Z/DpxS9PGMhUt1K47heJPk/L7LHNrSgavildIKajWjNEg AEd3DtJTM+9XeLYbviilSP63Rjfed5m4kxtMUkZlVdkkEqeIxYfKjM0uVqk3P9d70L xv0InEHt6GXiA== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:40 +0100 Subject: [PATCH v4 09/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for m68k-motorola Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-9-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9564; i=ljs@kernel.org; h=from:subject:message-id; bh=p817yyNbZ4NhaQkL3kSpCEKgVmtWbKpmnpJsqVcbxII=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2Lcip3tumej7jIp+UTvbmZoGMBZV5JuG/VW2rWM7dL GyPYbnUUcrCIMbFICumyPL8i/j+IJGweZ0X/N1g5rAygQxh4OIUgImknWJkOHJ21naDb5WmR9O8 Lps/arzI6LI47GdwbnjJMrHCu5vLbzL8j/Yw8DBQKnbIfX2/uX9PrILm2U8Tp8WeF1975vTB1xP 3sQAA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 sun3 and coldfire are already supported, however motorola requires a little more care. Here, custom table removal logic is required, so CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE is enabled for m68k-motorola. Firstly as part of this change, the page table level must be communicated to the underlying __tlb_remove_table() implementation. Take advantage of the fact that page tables are aligned by more than enough to permit setting TABLE_PTE or TABLE_PMD in the low bits of the pointer, and store this there. Then update __pte_free_tlb() and __pmd_free_tlb() to pass this through, then have __tlb_remove_table() decode this and pass it to free_pointer_table(). The page table freeing is performed via call_rcu(), so free_pointer_table() now will be invoked from softirq context, and as such may be re-entrant. Introduce an irq save/restore spinlock to handle this, and hold it over the time a given ptable entry is being referenced in both get_pointer_table() and free_pointer_table(). In order to make things a little easier in this respect, separate out the logic for adding a new ptable entry into add_pointer_table() and only hold the lock during ptable entry insertion in this case. Note that original list_add_tail(new, dp) added new prior to dp, which is ptable_list[type].next, i.e. after ptable_list[type]. The equivalent therefore is list_add(new, &ptable_list[type]), which adds new after ptable_list[type], only without needing to make reference to dp. Note that, as m68k-motorola specifies CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE, it does not enable CONFIG_PT_RECLAIM. This isn't meaningfully impactful. With this applied, all of m68k implements CONFIG_MMU_GATHER_RCU_TABLE_FREE. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Lance Yang Tested-by: Lance Yang --- arch/m68k/Kconfig | 3 +- arch/m68k/include/asm/motorola_pgalloc.h | 9 ++- arch/m68k/mm/motorola.c | 121 ++++++++++++++++++++-------= ---- 3 files changed, 86 insertions(+), 47 deletions(-) diff --git a/arch/m68k/Kconfig b/arch/m68k/Kconfig index 6b8ec67c86fd..fa5d39549da9 100644 --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -29,6 +29,7 @@ config M68K select HAVE_ARCH_LIBGCC_H select HAVE_ARCH_SECCOMP select HAVE_ARCH_SECCOMP_FILTER + select HAVE_ARCH_TLB_REMOVE_TABLE if MMU_MOTOROLA select HAVE_ASM_MODVERSIONS select HAVE_DEBUG_BUGVERBOSE select HAVE_EFFICIENT_UNALIGNED_ACCESS if !CPU_HAS_NO_UNALIGNED @@ -36,7 +37,7 @@ config M68K select HAVE_MOD_ARCH_SPECIFIC select HAVE_UID16 select MMU_GATHER_NO_RANGE if MMU - select MMU_GATHER_RCU_TABLE_FREE if MMU && (SUN3 || COLDFIRE) + select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_REL select MODULES_USE_ELF_RELA select NO_DMA if !MMU && !COLDFIRE diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/a= sm/motorola_pgalloc.h index 1091fb0affbe..dcde40e8b5c6 100644 --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -17,6 +17,7 @@ enum m68k_table_types { extern void init_pointer_table(void *table, int type); extern void *get_pointer_table(struct mm_struct *mm, int type); extern int free_pointer_table(void *table, int type); +extern void __tlb_remove_table(void *table); =20 /* * Allocate and free page tables. The xxx_kernel() versions are @@ -47,7 +48,7 @@ static inline void pte_free(struct mm_struct *mm, pgtable= _t pgtable) static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtabl= e, unsigned long address) { - free_pointer_table(pgtable, TABLE_PTE); + tlb_remove_table(tlb, (void *)((unsigned long)pgtable | TABLE_PTE)); } =20 =20 @@ -61,10 +62,10 @@ static inline int pmd_free(struct mm_struct *mm, pmd_t = *pmd) return free_pointer_table(pmd, TABLE_PMD); } =20 -static inline int __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, - unsigned long address) +static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, + unsigned long address) { - return free_pointer_table(pmd, TABLE_PMD); + tlb_remove_table(tlb, (void *)((unsigned long)pmd | TABLE_PMD)); } =20 =20 diff --git a/arch/m68k/mm/motorola.c b/arch/m68k/mm/motorola.c index b30aa69a73a6..ffc80483440b 100644 --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 #include #include @@ -103,6 +104,8 @@ static struct list_head ptable_list[3] =3D { LIST_HEAD_INIT(ptable_list[2]), }; =20 +static DEFINE_SPINLOCK(ptable_lock); + #define PD_PTABLE(ptdesc) ((ptable_desc *)&(virt_to_ptdesc((void *)(ptdesc= ))->pt_list)) #define PD_PTDESC(ptable) (list_entry(ptable, struct ptdesc, pt_list)) #define PD_MARKBITS(dp) (*(unsigned int *)&PD_PTDESC(dp)->pt_index) @@ -139,52 +142,66 @@ void __init init_pointer_table(void *table, int type) return; } =20 -void *get_pointer_table(struct mm_struct *mm, int type) +/* + * For a pointer table for a user process address space, a + * table is taken from a ptdesc allocated for the purpose. Each + * ptdesc can hold 8 pointer tables. The ptdesc is remapped in + * virtual address space to be noncacheable. + */ +static void *add_pointer_table(struct mm_struct *mm, int type) { - ptable_desc *dp =3D ptable_list[type].next; - unsigned int mask =3D list_empty(&ptable_list[type]) ? 0 : PD_MARKBITS(dp= ); - unsigned int tmp, off; + struct ptdesc *ptdesc; + ptable_desc *new; + void *pt_addr; =20 - /* - * For a pointer table for a user process address space, a - * table is taken from a ptdesc allocated for the purpose. Each - * ptdesc can hold 8 pointer tables. The ptdesc is remapped in - * virtual address space to be noncacheable. - */ - if (mask =3D=3D 0) { - struct ptdesc *ptdesc; - ptable_desc *new; - void *pt_addr; - - ptdesc =3D pagetable_alloc(GFP_KERNEL | __GFP_ZERO, 0); - if (!ptdesc) - return NULL; - - pt_addr =3D ptdesc_address(ptdesc); - - switch (type) { - case TABLE_PTE: - /* - * m68k doesn't have SPLIT_PTE_PTLOCKS for not having - * SMP. - */ - pagetable_pte_ctor(mm, ptdesc); - break; - case TABLE_PMD: - pagetable_pmd_ctor(mm, ptdesc); - break; - case TABLE_PGD: - pagetable_pgd_ctor(ptdesc); - break; - } + ptdesc =3D pagetable_alloc(GFP_KERNEL | __GFP_ZERO, 0); + if (!ptdesc) + return NULL; + + pt_addr =3D ptdesc_address(ptdesc); + + switch (type) { + case TABLE_PTE: + /* + * m68k doesn't have SPLIT_PTE_PTLOCKS for not having + * SMP. + */ + pagetable_pte_ctor(mm, ptdesc); + break; + case TABLE_PMD: + pagetable_pmd_ctor(mm, ptdesc); + break; + case TABLE_PGD: + pagetable_pgd_ctor(ptdesc); + break; + } + + mmu_page_ctor(pt_addr); + + new =3D PD_PTABLE(pt_addr); =20 - mmu_page_ctor(pt_addr); + PD_MARKBITS(new) =3D ptable_mask(type) - 1; + scoped_guard(spinlock_irqsave, &ptable_lock) + list_add(new, &ptable_list[type]); =20 - new =3D PD_PTABLE(pt_addr); - PD_MARKBITS(new) =3D ptable_mask(type) - 1; - list_add_tail(new, dp); + return (pmd_t *)pt_addr; +} + +void *get_pointer_table(struct mm_struct *mm, int type) +{ + unsigned int tmp, off; + unsigned long mask; + unsigned long flags; + ptable_desc *dp; + void *ret; =20 - return (pmd_t *)pt_addr; + spin_lock_irqsave(&ptable_lock, flags); + dp =3D ptable_list[type].next; + mask =3D list_empty(&ptable_list[type]) ? 0 : PD_MARKBITS(dp); + + if (mask =3D=3D 0) { + spin_unlock_irqrestore(&ptable_lock, flags); + return add_pointer_table(mm, type); } =20 for (tmp =3D 1, off =3D 0; (mask & tmp) =3D=3D 0; tmp <<=3D 1, off +=3D p= table_size(type)) @@ -194,7 +211,10 @@ void *get_pointer_table(struct mm_struct *mm, int type) /* move to end of list */ list_move_tail(dp, &ptable_list[type]); } - return ptdesc_address(PD_PTDESC(dp)) + off; + + ret =3D ptdesc_address(PD_PTDESC(dp)) + off; + spin_unlock_irqrestore(&ptable_lock, flags); + return ret; } =20 int free_pointer_table(void *table, int type) @@ -203,6 +223,9 @@ int free_pointer_table(void *table, int type) unsigned long ptable =3D (unsigned long)table; unsigned long pt_addr =3D ptable & PAGE_MASK; unsigned int mask =3D 1U << ((ptable - pt_addr)/ptable_size(type)); + unsigned long flags; + + spin_lock_irqsave(&ptable_lock, flags); =20 dp =3D PD_PTABLE(pt_addr); if (PD_MARKBITS (dp) & mask) @@ -213,6 +236,8 @@ int free_pointer_table(void *table, int type) if (PD_MARKBITS(dp) =3D=3D ptable_mask(type)) { /* all tables in ptdesc are free, free ptdesc */ list_del(dp); + spin_unlock_irqrestore(&ptable_lock, flags); + mmu_page_dtor((void *)pt_addr); pagetable_dtor_free(virt_to_ptdesc((void *)pt_addr)); return 1; @@ -223,9 +248,21 @@ int free_pointer_table(void *table, int type) */ list_move(dp, &ptable_list[type]); } + + spin_unlock_irqrestore(&ptable_lock, flags); return 0; } =20 +void __tlb_remove_table(void *table) +{ + /* The bottom 2 bits are used to encode page table type. */ + const unsigned long encoded =3D (unsigned long)table; + void *addr =3D (void *)(encoded & ~3UL); + const int type =3D encoded & 3; + + free_pointer_table(addr, type); +} + /* size of memory already mapped in head.S */ extern __initdata unsigned long m68k_init_mapped_size; =20 --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B92B55D869; Tue, 22 Sep 2026 15:40:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091618; cv=none; b=NekYn7xzat9SAQGCVZ2tNNKl5YOe8aoTb/d1D9dO3zF0oVCkFTbVLhXK2aQ4YtPMKOwQtAwxiIbDOFcrv1NXoSE9v953V6ZoGo1RnOtdBKIKHQjayXZrYY2pqQ6xJfr9PzzWLdfAhN8NIOGdKCrPqoE1ZJLObtdHQ4TKWU8B+YA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091618; c=relaxed/simple; bh=6W0g4Ck4JPsftf63O4MoWWYbRZcMaFdvRzcMK3bJO74=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Awi9Xq6RgC1Cjj9Dg4UzQGaNovi59naiDM7qOcR2VdTef+zLD/hlyeFqkxj2Xuu3OQLE5w7pXwSB2WCwuDtL54HYo+LMVPOCzlXUanKa/WhIwY2Nn5P5d+FKLpcoRBrd+AIl8yGgQ4IWmM651rWYLEG739N7t0JYoZeqXj1ew7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oBlXszrp; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oBlXszrp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13C5A1F00893; Tue, 22 Sep 2026 15:39:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091617; bh=fUi0jBX2gNbeikyLH6NBFgw3bykuPr333IPe1Mu8mWc=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=oBlXszrpHmbC1L39oiYrfsLXYt0tWyovlepfyoh4M57W/m40ZxhvpFELjycOKusJ4 bChO11gXU/dIjDyVMo88rIR1xoXPcO2zvCnkQxXQ5l2aq4aID+jqSITDVffNy79clb jwen+K5f3UhSbVb9JA0ogEP/8RItWsa3m8snnl8vX3K1+riLCsZyK1/mTuTyXiT0YI DKHuECZMKJXuHkPPLPn0V4Vwk88BpouwpTu15w0WZHsGD8J009Tv8cac4fjE8Xua9F uNH6MrfvV8o1Ptjxi+VdVn9aE1xv316X/SEkbtaEN1dckNkXn3XmDaYdaTguzhgJDd ZMm2pJ0BS9JyQ== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:41 +0100 Subject: [PATCH v4 10/12] mm: enable MMU_GATHER_RCU_TABLE_FREE for sparc32 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-10-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=7457; i=ljs@kernel.org; h=from:subject:message-id; bh=6W0g4Ck4JPsftf63O4MoWWYbRZcMaFdvRzcMK3bJO74=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2LcjRYH1nV3JO7voClUs/hHkzJk/9ZfviaHzM7YhK2 /t/NUMTOkpZGMS4GGTFFFmefxHfHyQSNq/zgr8bzBxWJpAhDFycAjCRqVcZGfYsnXzv/5NQnYM+ H3WE66L/WH4QrNv/RrNgyzGDWnH2U7GMDNcbMyzTfn5Tt3+s6ah8975gn/q+HZE6X+w2zT3myd8 qyA4A X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Careful handling is required for sparc32 which implements page tables as part of a shared backing page. To support this, a custom __tlb_remove_table() function is required, as specified by CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE. This allows __pte_free_tlb() and __pmd_free_tlb() to specify which page table level is being freed, which is transmitted to __tlb_remove_table() through setting the lowest bit of the page table to 1 for a PMD and 0 for a PTE (the page tables are 256-byte aligned so this is safe to do). Next, since the page table freeing is done via RCU callback, and thus might be executed in softirq context, update the spin locks to IRQ save/restore. Then, in __tlb_remove_table(), figure out whether to free a PMD page table via free_pmd_fast() or a PTE via the newly introduced __pte_free() function, using the lower bit encoded in __pte_free_tlb() or __pmd_free_tlb() to determine which to call. As part of this change use this spin lock rather than mm->page_table_lock for all shared page table exclusion, as RCU freeing means that page tables can be freed from soft IRQ context so both don't have an mm and also mm->page_table_lock is not IRQ-safe. Note that the specification of CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE disables CONFIG_PT_RECLAIM for sparc32, which mirrors sparc64. This forms part of an overall effort to switch every architecture to this mode, and with it complete, means every architecture now supports CONFIG_MMU_GATHER_RCU_TABLE_FREE. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: Lance Yang Tested-by: Lance Yang --- arch/sparc/Kconfig | 2 ++ arch/sparc/include/asm/pgalloc_32.h | 7 +++++-- arch/sparc/lib/bitext.c | 14 +++++++------- arch/sparc/mm/srmmu.c | 32 +++++++++++++++++++++++++++----- 4 files changed, 41 insertions(+), 14 deletions(-) diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 8d42ebc6d302..79c09d6ee466 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -64,6 +64,8 @@ config SPARC32 select HAVE_UID16 select HAVE_PAGE_SIZE_4KB select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE + select HAVE_ARCH_TLB_REMOVE_TABLE select OLD_SIGACTION select ZONE_DMA =20 diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/p= galloc_32.h index 4f73e87b22a3..36010852ba0c 100644 --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -48,7 +48,9 @@ static inline void free_pmd_fast(pmd_t * pmd) } =20 #define pmd_free(mm, pmd) free_pmd_fast(pmd) -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) + +#define __pmd_free_tlb(tlb, pmd, addr) \ + tlb_remove_table((tlb), (void *)((unsigned long)(pmd) | 1UL)) =20 #define pmd_populate(mm, pmd, pte) pmd_set(pmd, pte) =20 @@ -72,6 +74,7 @@ static inline void free_pte_fast(pte_t *pte) #define pte_free_kernel(mm, pte) free_pte_fast(pte) =20 void pte_free(struct mm_struct * mm, pgtable_t pte); -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +void __tlb_remove_table(void *table); +#define __pte_free_tlb(tlb, pte, addr) tlb_remove_table((tlb), (void *)(pt= e)) =20 #endif /* _SPARC_PGALLOC_H */ diff --git a/arch/sparc/lib/bitext.c b/arch/sparc/lib/bitext.c index 32a5c1d9459c..c309e27973ce 100644 --- a/arch/sparc/lib/bitext.c +++ b/arch/sparc/lib/bitext.c @@ -22,8 +22,6 @@ * @align: requested alignment * * Returns offset in the map or -1 if out of space. - * - * Not safe to call from an interrupt (uses spin_lock). */ int bit_map_string_get(struct bit_map *t, int len, int align) { @@ -31,6 +29,7 @@ int bit_map_string_get(struct bit_map *t, int len, int al= ign) int off_new; int align1; int i, color; + unsigned long flags; =20 if (t->num_colors) { /* align is overloaded to be the page color */ @@ -50,7 +49,7 @@ int bit_map_string_get(struct bit_map *t, int len, int al= ign) BUG(); color &=3D align1; =20 - spin_lock(&t->lock); + spin_lock_irqsave(&t->lock, flags); if (len < t->last_size) offset =3D t->first_free; else @@ -64,7 +63,7 @@ int bit_map_string_get(struct bit_map *t, int len, int al= ign) if (offset >=3D t->size) offset =3D 0; if (count + len > t->size) { - spin_unlock(&t->lock); + spin_unlock_irqrestore(&t->lock, flags); /* P3 */ printk(KERN_ERR "bitmap out: size %d used %d off %d len %d align %d count %d\n", t->size, t->used, offset, len, align, count); @@ -90,7 +89,7 @@ int bit_map_string_get(struct bit_map *t, int len, int al= ign) t->last_off =3D 0; t->used +=3D len; t->last_size =3D len; - spin_unlock(&t->lock); + spin_unlock_irqrestore(&t->lock, flags); return offset; } } @@ -103,10 +102,11 @@ int bit_map_string_get(struct bit_map *t, int len, in= t align) void bit_map_clear(struct bit_map *t, int offset, int len) { int i; + unsigned long flags; =20 if (t->used < len) BUG(); /* Much too late to do any good, but alas... */ - spin_lock(&t->lock); + spin_lock_irqsave(&t->lock, flags); for (i =3D 0; i < len; i++) { if (test_bit(offset + i, t->map) =3D=3D 0) BUG(); @@ -115,7 +115,7 @@ void bit_map_clear(struct bit_map *t, int offset, int l= en) if (offset < t->first_free) t->first_free =3D offset; t->used -=3D len; - spin_unlock(&t->lock); + spin_unlock_irqrestore(&t->lock, flags); } =20 void bit_map_init(struct bit_map *t, unsigned long *map, int size) diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index 9a74902ad181..1c277ab3cdb8 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -340,38 +340,60 @@ pgd_t *get_pgd_fast(void) * Alignments up to the page size are the same for physical and virtual * addresses of the nocache area. */ + +static DEFINE_SPINLOCK(pte_page_lock); + pgtable_t pte_alloc_one(struct mm_struct *mm) { + unsigned long flags; pte_t *ptep; struct page *page; =20 if (!(ptep =3D pte_alloc_one_kernel(mm))) return NULL; page =3D pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); - spin_lock(&mm->page_table_lock); + spin_lock_irqsave(&pte_page_lock, flags); if (page_ref_inc_return(page) =3D=3D 2 && !pagetable_pte_ctor(mm, page_ptdesc(page))) { page_ref_dec(page); ptep =3D NULL; } - spin_unlock(&mm->page_table_lock); + spin_unlock_irqrestore(&pte_page_lock, flags); =20 return ptep; } =20 -void pte_free(struct mm_struct *mm, pgtable_t ptep) +static void __pte_free(pgtable_t ptep) { struct page *page; + unsigned long flags; =20 page =3D pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); - spin_lock(&mm->page_table_lock); + spin_lock_irqsave(&pte_page_lock, flags); if (page_ref_dec_return(page) =3D=3D 1) pagetable_dtor(page_ptdesc(page)); - spin_unlock(&mm->page_table_lock); + spin_unlock_irqrestore(&pte_page_lock, flags); =20 srmmu_free_nocache(ptep, SRMMU_PTE_TABLE_SIZE); } =20 +void pte_free(struct mm_struct *mm, pgtable_t ptep) +{ + __pte_free(ptep); +} + +void __tlb_remove_table(void *table) +{ + const unsigned long encoded =3D (unsigned long)table; + const unsigned long addr =3D encoded & ~1UL; + const bool is_pmd =3D encoded & 1; + + if (is_pmd) + free_pmd_fast((pmd_t *)addr); + else + __pte_free((pgtable_t)addr); +} + /* context handling - a dynamically sized pool is used */ #define NO_CONTEXT -1 =20 --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A97BB5616D0; Tue, 22 Sep 2026 15:40:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091642; cv=none; b=OlBEX8kPtWwOR6SPRpiI9cnFre8O41bE37zyrybDV5hUzHKAaiPmsrmunh683SRA6Y3g+DmC3k2CxUQFBO/HLDszx7igOyxfBs1aUAbIcU/ZFSoTf677vUU648ijli3BOpCvct5bI5L2XE5aeG4Q6qGebfD3zm8yzMHHQDrRcZo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091642; c=relaxed/simple; bh=BT0JleXFpz569SGOSbz6OPKRm+jGpeOzNmZQorlaS2g=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bMxlxgB7rqqOZy654dlFQh1+Un5ozm65BITzHt7bBRcwlcHqug21+vPcOXFNMLHFQSz7w1J90lPSRzKhYEoxkp/tQeCAw5p7INTJmEOR51/eBjjX5/oJWlY4w0xuYmhHTqVFo4pV/Vgnz7wTJWTBKtkK9n0toUmvqG+fTaO458A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WZHcHi4a; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WZHcHi4a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7997D1F000FF; Tue, 22 Sep 2026 15:40:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091639; bh=DP0C33MD1sJk9gftfUN3DqNmrRTthZUOvUUdUm9+5vQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=WZHcHi4aiZSJ/M5W5aVd3nsf68Y2qn1YytH1o3kF+QwTpnnUSKUMdkRkaSqWb95I8 pVA/RUXbudTN0j1vay7KKoaS/Y4r7gKaYf3SU69gSZO5GR2/phftjMPVoPE+ZljVeD Y2tFmPnTEWsWdvikTKT1eZvexav5cB8etQPBOoIMu9D0ur29/eeKbsD0psm+UDxjdw +av3SZ7/zMwznmHmrOcBJgimUeaMeJcK7voccyd3s2shSVJQe8+nCk75sE+Vcd4itz Xvr3FBre3YY7eVzznahb9RZESKfgNCEsn2O+Kdu6szVC4aB0L1LSOptQtqD3wUUY80 nKww3boW8E5xw== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:42 +0100 Subject: [PATCH v4 11/12] mm: make userland page table freeing RCU-safe Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-11-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=20537; i=ljs@kernel.org; h=from:subject:message-id; bh=BT0JleXFpz569SGOSbz6OPKRm+jGpeOzNmZQorlaS2g=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2Lcj51H7BvPYGf1VPoXqQtZ7o5Fmxsrls3YYTOjIcE 6ffube0o5SFQYyLQVZMkeX5F/H9QSJh8zov+LvBzGFlAhnCwMUpABNZ0czwP7D0f0mU2rv7d+o9 txyXn28R+qFUskvhuPz81+/cxXmfeDP84ZTJeBHh8XANV1v36RW7917QO7zz0iHnK66y5W/uz/7 6kQ8A X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Now every architecture has been converted to support CONFIG_MMU_GATHER_RCU_TABLE_FREE, this configuration option no longer makes any sense to keep around. Therefore remove it, and remove all the dead code that existed for !CONFIG_MMU_GATHER_RCU_TABLE_FREE architectures previously. Additionally, CONFIG_MMU_GATHER_TABLE_FREE is no longer necessary, as all architectures instead use CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE when a custom __tlb_remove_table() is required, so remove this too. A number of architectures only enabled CONFIG_MMU_GATHER_RCU_TABLE_FREE if CONFIG_MMU was set, however the mmu_gather logic only actually does something meaningful if CONFIG_MMU is set (mmu_gather.c is only compiled in this case, for instance). As a result, there's no need to gate any of this logic on CONFIG_MMU explicitly. CONFIG_PT_RECLAIM however does have a strict dependency on CONFIG_MMU, so make this dependency explicit. Additionally, correct comments to remove references to non-RCU page table gathering and make it clear that this is not 'semi-RCU', nor has it been since commit 1fb3d8c20bfa ("mm/mmu_gather: replace IPI with synchronize_rcu() when batch allocation fails"). With this change in place the kernel policy is now that userspace page tables are freed after an RCU grace period, and thus it is now safe to unconditionally perform page table walks under RCU, safe in the knowledge that page tables will not be freed underneath the walker. This is all that is guaranteed, however, so naturally it is still incumbent upon page table walkers to ensure that the page table entries are as expected. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Reviewed-by: Lance Yang --- arch/Kconfig | 8 ----- arch/alpha/Kconfig | 1 - arch/arc/Kconfig | 1 - arch/arm/Kconfig | 1 - arch/arm64/Kconfig | 1 - arch/csky/Kconfig | 1 - arch/hexagon/Kconfig | 1 - arch/loongarch/Kconfig | 1 - arch/m68k/Kconfig | 1 - arch/microblaze/Kconfig | 1 - arch/mips/Kconfig | 1 - arch/nios2/Kconfig | 1 - arch/openrisc/Kconfig | 1 - arch/parisc/Kconfig | 1 - arch/powerpc/Kconfig | 1 - arch/riscv/Kconfig | 1 - arch/s390/Kconfig | 1 - arch/sh/Kconfig | 1 - arch/sparc/Kconfig | 2 -- arch/sparc/include/asm/tlb_64.h | 2 -- arch/um/Kconfig | 1 - arch/x86/Kconfig | 1 - arch/xtensa/Kconfig | 1 - include/asm-generic/tlb.h | 66 ++++++-------------------------------= ---- mm/Kconfig | 2 +- mm/gup.c | 5 ++-- mm/mmu_gather.c | 30 ++++--------------- 27 files changed, 18 insertions(+), 117 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 45c657772362..6f7516916797 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -526,13 +526,6 @@ config HAVE_ARCH_JUMP_LABEL config HAVE_ARCH_JUMP_LABEL_RELATIVE bool =20 -config MMU_GATHER_TABLE_FREE - bool - -config MMU_GATHER_RCU_TABLE_FREE - bool - select MMU_GATHER_TABLE_FREE - config MMU_GATHER_PAGE_SIZE bool =20 @@ -548,7 +541,6 @@ config MMU_GATHER_MERGE_VMAS =20 config MMU_GATHER_NO_GATHER bool - depends on MMU_GATHER_TABLE_FREE =20 config ARCH_WANT_IRQS_OFF_ACTIVATE_MM bool diff --git a/arch/alpha/Kconfig b/arch/alpha/Kconfig index e53ef2d88463..9063c7bda4e4 100644 --- a/arch/alpha/Kconfig +++ b/arch/alpha/Kconfig @@ -42,7 +42,6 @@ config ALPHA select ARCH_STACKWALK select CPU_NO_EFFICIENT_FFS if !ALPHA_EV67 select MMU_GATHER_NO_RANGE - select MMU_GATHER_RCU_TABLE_FREE select SPARSEMEM_EXTREME if SPARSEMEM select ZONE_DMA select TRACE_IRQFLAGS_SUPPORT diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig index 7a7542b61823..2ed7186c81c5 100644 --- a/arch/arc/Kconfig +++ b/arch/arc/Kconfig @@ -47,7 +47,6 @@ config ARC select HAVE_SYSCALL_TRACEPOINTS select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select OF select OF_EARLY_FLATTREE diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 72b9afc6ae10..0cc289a7184a 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -134,7 +134,6 @@ config ARM select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP select HAVE_POSIX_CPU_TIMERS_TASK_WORK - select MMU_GATHER_RCU_TABLE_FREE if MMU select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RSEQ select HAVE_RUST if CPU_LITTLE_ENDIAN && CPU_32v7 && !KASAN diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 2bbeded33da0..b6c2dd8b2612 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -221,7 +221,6 @@ config ARM64 select HAVE_RELIABLE_STACKTRACE select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_FUNCTION_ARG_ACCESS_API - select MMU_GATHER_RCU_TABLE_FREE select HAVE_RSEQ select HAVE_RUST if RUSTC_SUPPORTS_ARM64 select HAVE_STACKPROTECTOR diff --git a/arch/csky/Kconfig b/arch/csky/Kconfig index 80f89ef1d962..4331313a42ff 100644 --- a/arch/csky/Kconfig +++ b/arch/csky/Kconfig @@ -96,7 +96,6 @@ config CSKY select HAVE_SYSCALL_TRACEPOINTS select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MAY_HAVE_SPARSE_IRQ select MODULES_USE_ELF_RELA if MODULES select OF diff --git a/arch/hexagon/Kconfig b/arch/hexagon/Kconfig index d9b3fb86556b..b48491140013 100644 --- a/arch/hexagon/Kconfig +++ b/arch/hexagon/Kconfig @@ -23,7 +23,6 @@ config HEXAGON # select HAVE_CLK select GENERIC_ATOMIC64 select HAVE_PERF_EVENTS - select MMU_GATHER_RCU_TABLE_FREE # GENERIC_ALLOCATOR is used by dma_alloc_coherent() select GENERIC_ALLOCATOR select GENERIC_IRQ_PROBE diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 9c5def706222..d1b23da40737 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -188,7 +188,6 @@ config LOONGARCH select IRQ_LOONGARCH_CPU select LOCK_MM_AND_FIND_VMA select MMU_GATHER_MERGE_VMAS if MMU - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA if MODULES select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK diff --git a/arch/m68k/Kconfig b/arch/m68k/Kconfig index fa5d39549da9..eb84c3af92c0 100644 --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -37,7 +37,6 @@ config M68K select HAVE_MOD_ARCH_SPECIFIC select HAVE_UID16 select MMU_GATHER_NO_RANGE if MMU - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_REL select MODULES_USE_ELF_RELA select NO_DMA if !MMU && !COLDFIRE diff --git a/arch/microblaze/Kconfig b/arch/microblaze/Kconfig index af7e821e96c1..484ebb3baedf 100644 --- a/arch/microblaze/Kconfig +++ b/arch/microblaze/Kconfig @@ -41,7 +41,6 @@ config MICROBLAZE select PCI_SYSCALL if PCI select CPU_NO_EFFICIENT_FFS select MMU_GATHER_NO_RANGE - select MMU_GATHER_RCU_TABLE_FREE select SPARSE_IRQ select ZONE_DMA select TRACE_IRQFLAGS_SUPPORT diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index e2eb9627bd14..f0c43d118ca0 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -97,7 +97,6 @@ config MIPS select IRQ_FORCED_THREADING select ISA if EISA select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_REL if MODULES select MODULES_USE_ELF_RELA if MODULES && 64BIT select PERF_USE_VMALLOC diff --git a/arch/nios2/Kconfig b/arch/nios2/Kconfig index b0ccfc3b7a7e..9c0e6eaeb005 100644 --- a/arch/nios2/Kconfig +++ b/arch/nios2/Kconfig @@ -19,7 +19,6 @@ config NIOS2 select HAVE_PAGE_SIZE_4KB select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select OF select OF_EARLY_FLATTREE diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig index d90b24dd3bce..5eb995c13074 100644 --- a/arch/openrisc/Kconfig +++ b/arch/openrisc/Kconfig @@ -35,7 +35,6 @@ config OPENRISC select GENERIC_ATOMIC64 select GENERIC_CLOCKEVENTS_BROADCAST select GENERIC_SMP_IDLE_THREAD - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select HAVE_DEBUG_STACKOVERFLOW select OR1K_PIC diff --git a/arch/parisc/Kconfig b/arch/parisc/Kconfig index d3afac2f0d9b..77f67028ad89 100644 --- a/arch/parisc/Kconfig +++ b/arch/parisc/Kconfig @@ -80,7 +80,6 @@ config PARISC select GENERIC_CLOCKEVENTS select CPU_NO_EFFICIENT_FFS select THREAD_INFO_IN_TASK - select MMU_GATHER_RCU_TABLE_FREE select NEED_DMA_MAP_STATE select NEED_SG_DMA_LENGTH select HAVE_ARCH_KGDB diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 2580e27e4328..0767cfcbaa42 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -307,7 +307,6 @@ config PPC select KASAN_VMALLOC if KASAN && EXECMEM select LOCK_MM_AND_FIND_VMA select MMU_GATHER_PAGE_SIZE - select MMU_GATHER_RCU_TABLE_FREE select HAVE_ARCH_TLB_REMOVE_TABLE select MMU_GATHER_MERGE_VMAS select MMU_LAZY_TLB_SHOOTDOWN if PPC_BOOK3S_64 diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 3529ed1861ce..7741a4287498 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -208,7 +208,6 @@ config RISCV select IRQ_FORCED_THREADING select KASAN_VMALLOC if KASAN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA if MODULES select OF select OF_EARLY_FLATTREE diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index b88b85042136..a34376c05f6e 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -267,7 +267,6 @@ config S390 select LOCK_MM_AND_FIND_VMA select MMU_GATHER_MERGE_VMAS select MMU_GATHER_NO_GATHER - select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA select NEED_DMA_MAP_STATE if PCI select NEED_PER_CPU_EMBED_FIRST_CHUNK diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig index 75236bef6f16..fe859def918c 100644 --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -62,7 +62,6 @@ config SUPERH select HAVE_SYSCALL_TRACEPOINTS select IRQ_FORCED_THREADING select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA select NEED_SG_DMA_LENGTH select NO_DMA if !MMU && !DMA_COHERENT diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 79c09d6ee466..742ffff8c37f 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -64,7 +64,6 @@ config SPARC32 select HAVE_UID16 select HAVE_PAGE_SIZE_4KB select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE select HAVE_ARCH_TLB_REMOVE_TABLE select OLD_SIGACTION select ZONE_DMA @@ -77,7 +76,6 @@ config SPARC64 select HAVE_FUNCTION_GRAPH_TRACER select HAVE_KRETPROBES select HAVE_KPROBES - select MMU_GATHER_RCU_TABLE_FREE select HAVE_ARCH_TLB_REMOVE_TABLE select MMU_GATHER_MERGE_VMAS select MMU_GATHER_NO_FLUSH_CACHE diff --git a/arch/sparc/include/asm/tlb_64.h b/arch/sparc/include/asm/tlb_6= 4.h index 3037187482db..f5f9631685d5 100644 --- a/arch/sparc/include/asm/tlb_64.h +++ b/arch/sparc/include/asm/tlb_64.h @@ -29,9 +29,7 @@ void flush_tlb_pending(void); * and therefore we don't need a TLBI when freeing page-table pages. */ =20 -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE #define tlb_needs_table_invalidate() (false) -#endif =20 #include =20 diff --git a/arch/um/Kconfig b/arch/um/Kconfig index d9541d13d9eb..94b8ff70f578 100644 --- a/arch/um/Kconfig +++ b/arch/um/Kconfig @@ -44,7 +44,6 @@ config UML select HAVE_SYSCALL_TRACEPOINTS select THREAD_INFO_IN_TASK select SPARSE_IRQ - select MMU_GATHER_RCU_TABLE_FREE =20 config MMU bool diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index a8c3b3d31a27..6e5e462ec059 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -283,7 +283,6 @@ config X86 select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP select ASYNC_KERNEL_PGTABLE_FREE if IOMMU_SVA - select MMU_GATHER_RCU_TABLE_FREE select MMU_GATHER_MERGE_VMAS select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_REGS_AND_STACK_ACCESS_API diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig index 33c4caee30e2..f2f9cd9cde50 100644 --- a/arch/xtensa/Kconfig +++ b/arch/xtensa/Kconfig @@ -55,7 +55,6 @@ config XTENSA select HAVE_VIRT_CPU_ACCOUNTING_GEN select IRQ_DOMAIN select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA select PERF_USE_VMALLOC select TRACE_IRQFLAGS_SUPPORT diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index bdcc2778ac64..9d827076db19 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -67,11 +67,8 @@ * - tlb_remove_table() * * tlb_remove_table() is the basic primitive to free page-table directo= ries - * (__p*_free_tlb()). In it's most primitive form it is an alias for - * tlb_remove_page() below, for when page directories are pages and hav= e no - * additional constraints. - * - * See also MMU_GATHER_TABLE_FREE and MMU_GATHER_RCU_TABLE_FREE. + * (__p*_free_tlb()). Page directories are freed after an RCU grace + * period - see the comment in mm/mmu_gather.c. * * - tlb_remove_page() / tlb_remove_page_size() * - __tlb_remove_folio_pages() / __tlb_remove_page_size() @@ -151,24 +148,15 @@ * This might be useful if your architecture has size specific TLB * invalidation instructions. * - * MMU_GATHER_TABLE_FREE - * - * This provides tlb_remove_table(), to be used instead of tlb_remove_pag= e() - * for page directores (__p*_free_tlb()). - * - * Useful if your architecture has non-page page directories. + * Page directories (__p*_free_tlb()) are always freed via tlb_remove_tab= le(), + * after an RCU grace period (see mm/mmu_gather.c). * - * When used, an architecture is expected to provide __tlb_remove_table()= or - * use the generic __tlb_remove_table(), which does the actual freeing of= these - * pages. + * This serialises against software page-table walkers, including archite= ctures + * which do not use IPIs for remote TLB invalidates. * - * MMU_GATHER_RCU_TABLE_FREE - * - * Like MMU_GATHER_TABLE_FREE, and adds semi-RCU semantics to the free (s= ee - * comment below). - * - * Useful if your architecture doesn't use IPIs for remote TLB invalidates - * and therefore doesn't naturally serialize with software page-table wal= kers. + * An architecture is expected to provide __tlb_remove_table() (see + * HAVE_ARCH_TLB_REMOVE_TABLE) or use the generic __tlb_remove_table(), w= hich + * does the actual freeing of these pages. * * MMU_GATHER_NO_FLUSH_CACHE * @@ -200,12 +188,8 @@ * various ptep_get_and_clear() functions. */ =20 -#ifdef CONFIG_MMU_GATHER_TABLE_FREE - struct mmu_table_batch { -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE struct rcu_head rcu; -#endif unsigned int nr; void *tables[]; }; @@ -224,23 +208,6 @@ static inline void __tlb_remove_table(void *table) =20 extern void tlb_remove_table(struct mmu_gather *tlb, void *table); =20 -#else /* !CONFIG_MMU_GATHER_TABLE_FREE */ - -static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *pa= ge); -/* - * Without MMU_GATHER_TABLE_FREE the architecture is assumed to have page = based - * page directories and we can use the normal page batching to free them. - */ -static inline void tlb_remove_table(struct mmu_gather *tlb, void *table) -{ - struct ptdesc *ptdesc =3D (struct ptdesc *)table; - - pagetable_dtor(ptdesc); - tlb_remove_page(tlb, ptdesc_page(ptdesc)); -} -#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ - -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE /* * This allows an architecture that does not use the linux page-tables for * hardware to skip the TLBI when freeing page tables. @@ -253,19 +220,6 @@ void tlb_remove_table_sync_one(void); =20 void tlb_remove_table_sync_rcu(void); =20 -#else - -#ifdef tlb_needs_table_invalidate -#error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE -#endif - -static inline void tlb_remove_table_sync_one(void) { } - -static inline void tlb_remove_table_sync_rcu(void) { } - -#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ - - #ifndef CONFIG_MMU_GATHER_NO_GATHER /* * If we can't allocate a page to make a big batch of page pointers @@ -325,9 +279,7 @@ static inline void tlb_flush_rmaps(struct mmu_gather *t= lb, struct vm_area_struct struct mmu_gather { struct mm_struct *mm; =20 -#ifdef CONFIG_MMU_GATHER_TABLE_FREE struct mmu_table_batch *batch; -#endif =20 unsigned long start; unsigned long end; diff --git a/mm/Kconfig b/mm/Kconfig index c1ddf59c0d71..bc7befafb47b 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1465,7 +1465,7 @@ config HAVE_ARCH_TLB_REMOVE_TABLE =20 config PT_RECLAIM def_bool y - depends on MMU_GATHER_RCU_TABLE_FREE && !HAVE_ARCH_TLB_REMOVE_TABLE + depends on MMU && !HAVE_ARCH_TLB_REMOVE_TABLE help Try to reclaim empty user page table pages in paths other than munmap and exit_mmap path. diff --git a/mm/gup.c b/mm/gup.c index eb898ea1ee22..63b435ec605c 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2700,8 +2700,9 @@ EXPORT_SYMBOL(get_user_pages_unlocked); * Before activating this code, please be aware that the following assumpt= ions * are currently made: * - * *) Either MMU_GATHER_RCU_TABLE_FREE is enabled, and tlb_remove_table()= is used to - * free pages containing page tables or TLB flushing requires IPI broadca= st. + * *) tlb_remove_table() is used to free pages containing page tables, wi= th + * the free deferred until an RCU grace period has elapsed (see + * mm/mmu_gather.c). * * *) ptes can be read atomically by the architecture. * diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c index 3985d856de7f..2a72a9686773 100644 --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -218,8 +218,6 @@ bool __tlb_remove_page_size(struct mmu_gather *tlb, str= uct page *page, int page_ =20 #endif /* MMU_GATHER_NO_GATHER */ =20 -#ifdef CONFIG_MMU_GATHER_TABLE_FREE - static void __tlb_remove_table_free(struct mmu_table_batch *batch) { int i; @@ -230,10 +228,8 @@ static void __tlb_remove_table_free(struct mmu_table_b= atch *batch) free_page((unsigned long)batch); } =20 -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE - /* - * Semi RCU freeing of the page directories. + * RCU freeing of the page directories. * * This is needed by some architectures to implement software pagetable wa= lkers. * @@ -259,13 +255,13 @@ static void __tlb_remove_table_free(struct mmu_table_= batch *batch) * means. * * What we do is batch the freed directory pages (tables) and RCU free the= m. - * We use the sched RCU variant, as that guarantees that IRQ/preempt disab= ling - * holds off grace periods. + * Disabling IRQs or preemption holds off RCU grace periods, so this prote= cts + * both rcu_read_lock() and IRQ-disabling walkers. * * However, in order to batch these pages we need to allocate storage, this * allocation is deep inside the MM code and can thus easily fail on memory - * pressure. To guarantee progress we fall back to single table freeing, s= ee - * the implementation of tlb_remove_table_one(). + * pressure. To guarantee progress we fall back to single table freeing, w= hich + * is also RCU-deferred - see the implementation of tlb_remove_table_one(). * */ =20 @@ -315,15 +311,6 @@ void tlb_remove_table_sync_rcu(void) synchronize_rcu(); } =20 -#else /* !CONFIG_MMU_GATHER_RCU_TABLE_FREE */ - -static void tlb_remove_table_free(struct mmu_table_batch *batch) -{ - __tlb_remove_table_free(batch); -} - -#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ - /* * If we want tlb_remove_table() to imply TLB invalidates. */ @@ -403,13 +390,6 @@ static inline void tlb_table_init(struct mmu_gather *t= lb) tlb->batch =3D NULL; } =20 -#else /* !CONFIG_MMU_GATHER_TABLE_FREE */ - -static inline void tlb_table_flush(struct mmu_gather *tlb) { } -static inline void tlb_table_init(struct mmu_gather *tlb) { } - -#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ - static void tlb_flush_mmu_free(struct mmu_gather *tlb) { tlb_table_flush(tlb); --=20 2.55.0 From nobody Thu Sep 24 15:11:49 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C9555625FB; Tue, 22 Sep 2026 15:41:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091662; cv=none; b=lKYLWg7xTgmkdX6w/IpTUhKuPBILMZ0C4pIIX/MUi8IotSiFUcdZH71kk/5xJw3WqNErPyile6DAiEu2nbUio2SrkAuAen5oSk76x90SazdTxbRZlaXoVxCV38eMZN3UXiICLi7X57rNIDkqD9BDWPYu+IxLGH2EeYvLi5r6nDQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091662; c=relaxed/simple; bh=d4o8BvEhvbpmkAFDsmC7e/OS16AthPkW67coZubsUt0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QqIGSMikRCz1vzq42pbBh8/9s+WbQSzbBmw66zP22hZvNZpf4zdgZpxPqNauzE7vJRvg/rtwwhr5opanPHhWAAbeMBClqdWnuXJOjOJPyqeqKd7zCCkd5UH3r653LXA4UcTcBQGCoeu8tQZhKYmoARy9NOYEEHo+lZyHW/kfQdU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jI1Gy7Ri; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jI1Gy7Ri" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C34FD1F0089A; Tue, 22 Sep 2026 15:40:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091661; bh=M7feebBV+4w0SqCnHHPUA0WnsaaqlXi4RbYO862HspA=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=jI1Gy7Ri0dwagnMGdXDueH8Vw/me/BZQMY1ZLvclBfYHE07TPWIYqEZKTOfDeKMM7 WWEYNmjHnqF3lwoScgBfUCpcP4Vzmd31Iqmg0Sox8i0+b3EzEc/kGtknRiFqelROnJ wcbUogIVm7JOp0s23z9uLqzTc+N99mJYI/tD1tkVSdrEOCMZ8aSwT3rqdtSH2Hyz8Z bOYZ6vTeWolpG4Fwfi6cgaTCqHY7pSa4UY721VzOIhSOaCjXR8xijEU4EtkTdNMAxs dW8uW14KTDudI1kEbfNi0V+Sn1IfY0buJZpOX2Ea2f+GpSpPZYolfJ0Ii0rrzBB1ZY xFRhD3RYFSySA== From: "Lorenzo Stoakes (ARM)" Date: Tue, 22 Sep 2026 16:35:43 +0100 Subject: [PATCH v4 12/12] mm: change the contract for free_pgtables(), update docs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260922-rcu-pagetable-freeing-v4-12-fe1ad1f1e303@kernel.org> References: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> In-Reply-To: <20260922-rcu-pagetable-freeing-v4-0-fe1ad1f1e303@kernel.org> To: Andrew Morton , David Hildenbrand , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kiryl Shutsemau , Guo Ren , Brian Cain , Geert Uytterhoeven , Dinh Nguyen , Simon Schuster , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Rich Felker , John Paul Adrian Glaubitz , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Russell King , Vineet Gupta , Michal Simek , Chris Zankel , Max Filippov , Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , "David S. Miller" , Andreas Larsson , Richard Henderson , Matt Turner , Magnus Lindholm , Catalin Marinas , Mark Rutland , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Madhavan Srinivasan , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Richard Weinberger , Anton Ivanov , Johannes Berg , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Arnd Bergmann , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Yoshinori Sato , Shakeel Butt , Jonathan Corbet , Randy Dunlap Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-snps-arc@lists.infradead.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-alpha@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, Hugh Dickins , Qi Zheng , linux-doc@vger.kernel.org, "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=3290; i=ljs@kernel.org; h=from:subject:message-id; bh=d4o8BvEhvbpmkAFDsmC7e/OS16AthPkW67coZubsUt0=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLI2Lci5KhCa0NC7PfzCodRL5yaIOr81TeH4vvJq5rn/a t8KJq161VHKwiDGxSArpsjy/Iv4/iCRsHmdF/zdYOawMoEMYeDiFICJdCYxMkyx61z519qF/Wrr nHyBV2IWzI+aa9tvzQh+L+iflK2mVczwz8bMcIvqx7JdNwMKJ/810e10aW9/OveN8PQ/uWtXX9B nZQMA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Now that page tables are freed after an RCU grace period, it is safe for read-only page table walkers to walk page table ranges that are being concurrently torn down, provided the mm is kept alive via mmgrab(). It is however unsafe for writers to do so, as they must obtain an appropriate lock to do so safely. Update the pte_offset_map_lock()'s comment block to reflect this. Similarly update the process addresses documentation. Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Lorenzo Stoakes (ARM) --- Documentation/mm/process_addrs.rst | 6 ++++++ mm/pgtable-generic.c | 15 +++++++++++---- 2 files changed, 17 insertions(+), 4 deletions(-) diff --git a/Documentation/mm/process_addrs.rst b/Documentation/mm/process_= addrs.rst index a7296f251799..b1f4f44d75eb 100644 --- a/Documentation/mm/process_addrs.rst +++ b/Documentation/mm/process_addrs.rst @@ -537,6 +537,12 @@ We establish basic locking rules when interacting with= page tables: * When changing a page table entry the page table lock for that page table **must** be held, except if you can safely assume nobody can access the = page tables concurrently (such as on invocation of :c:func:`!free_pgtables`). +* Page tables may be *walked* under RCU alone, as page tables are freed on= ly + after an RCU grace period has elapsed. However, any entry found must be + revalidated after the page table lock is taken (such as the + :c:func:`!pmd_same` recheck performed by :c:func:`!pte_offset_map_lock`) + before it is acted upon. Changing an entry requires the page table + lock and one of the locks that excludes teardown (mmap or VMA lock). * Reads from and writes to page table entries must be *appropriately* atomic. See the section on atomicity below for details. * Populating previously empty entries requires that the mmap or VMA locks = are diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index b91b1a98029c..a127e3e8f9b9 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -385,10 +385,17 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm,= pmd_t *pmd, * Note: "RO" / "RW" expresses the intended semantics, not that the *kmap*= will * be read-only/read-write protected. * - * Note that free_pgtables(), used after unmapping detached vmas, or when - * exiting the whole mm, does not take page table lock before freeing a pa= ge - * table, and may not use RCU at all: "outsiders" like khugepaged should a= void - * pte_offset_map() and co once the vma is detached from mm or mm_users is= zero. + * Note that free_pgtables(), used after unmapping detached vmas or when e= xiting + * the whole mm, does not take a page table lock before freeing a page tab= le. + * + * As page table freeing itself is RCU-safe, page table readers can safely= run + * concurrently with page table teardown. + * + * However, writers CANNOT as, without a lock being held, nothing prevents + * concurrent teardown. + * + * Also note that the PGD itself is freed at mmdrop() time, not under RCU = - so + * the walker must keep the mm alive either by pinning the mm or the VMA. */ pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) --=20 2.55.0