Kernel page table walkers fall into two broad categories - those ranges
where no exclusion is required via walk_kernel_page_table_range_lockless()
and those where exclusion is required via walk_kernel_page_table_range()
or walk_page_range_debug().

The former category is used only by arm64 arch code operating on ranges it
both wholly owns and does not concurrently write.

The latter category consists of kernel page table walkers operating on
ranges that are wholly owned (but which need exclusion against concurrent
writers).

The lock used for exclusion is the mmap lock, and for kernel ranges this
the mmap lock on init_mm.

ptdump is a special case being both the only user of
walk_page_range_debug(), and the only case in which it walks ranges it does
not own.

This presents a problem, as page tables may be freed under ptdump. And
indeed there is a use-after-free bug in the kernel as a result, which this
series addresses.

vmap promotes page tables to huge leaf entries where possible, freeing the
lower page table when it does. It does this with no meaningful locks held
against concurrent ptdump walks.

As a result, use-after-free can currently occur. This series addresses the
issue by having the vmap huge promotion logic acquire the mmap read lock
while both setting the huge page table entry and freeing the prior leaf
page table.

The ptdump code already acquires the mmap write lock, so by doing so we
ensure that the ptdump walker only ever observes either the huge page table
entry or the existing page table entry, and nothing is freed underneath it.

A mitigation for this issue was already applied for arm64 in commit
fa93b45fd397 ("arm64: Enable vmalloc-huge with ptdump"), which this series
has to deal with carefully.

This mitigation resolves the issue by acquiring the mmap read lock on
init_mm on vmap page table free if a ptdump is in progress.

However the fix in this series would cause a deadlock if we were to simply
apply it for arm64 without also reverting the change.

This is because vmap may acquire the read lock before ptdump attempts to
acquire the write lock, which then gets queued, and rwsem starvation rules
mean that the (unacknowledged) nested mmap read lock in the arm64 code
would also block, meaning the original read lock is never released and thus
deadlock.

This series works around this by #ifndef CONFIG_ARM64'ing the mmap read
lock in vmap logic, then partially reverting commit fa93b45fd397 ("arm64:
Enable vmalloc-huge with ptdump"), keeping the enablement of huge vmap
support, and removing the ifdeffery with the partial revert patch.

There are two related issues that we also address in this series:

* x86 page attribute logic, specifically Change Page Attributes (CPA),
  implements a feature whereby huge ranges can be collapsed into huge leaf
  entries. This can similarly cause a UAF when done in parallel with a
  ptdump walk, so similarly acquire the init_mm mmap lock to avoid this.

* x86 and arm64 permit walks of non-kernel mm's (both allowing efi mm
  walks, and in intel's case arbitrary mm's), so we ensure kernel mappings
  remain stable by locking the init_mm as well as the mm being walked.

The ordering of patches is established for both strict dependencies (the
arm64 partial revert in particular has to be done after the vmap changes)
and logical ones (the non-kernel mm fix only makes sense once the vmap/CPA
fixes are in place).

---
v2:
* Rebased on latest master of Linus's tree.
* Accumulated tags, thanks everybody!
* Fixed cover letter reference to arm64 partial revert as per David C.
* Combined all patches into a 4 patch series for ease of tracking/review
  and updated cover letter to reflect.
* Reordered patches in series logically - fix vmap, fix CPA issue, handle
  mm vs. init_mm then revert arm64 mitigation.
* Reworded first patch to be consistent with x86 wording to clearly
  indicate the intent of the fix is to fix ptdump UAF.
* Reworded arm64 revert patch subject as per Mike, Dev to make clear this
  is only reverting the ptdump mitigation not the vmap huge support.
* Added Fixes: tag to arm64 revert patch so we backport this also for
  neatness. Not strictly necessary, but is a better fix overall applied.
* Added note about stable dependency to arm64 revert patch.
* Updated walk_page_range_debug() to remove pointless !walk.mm check - if
  NULL mm then the mmap lock asserts would NULL pointer deref, and of
  course no caller does this anyway.
* Updated walk_page_range_debug() to always assert init_mm mmap write lock
  is held.
* Updated walk_page_range_debug() to always check for start >= end and that
  it has safe walk ops, and remove the unnecessary
  walk_kernel_page_table_range() invocation which ultimately does the same
  thing.
* Typo fixups as per Mike.
* Some small commit message/comment wording fixups.

v1:
* vmap/arm64 partial revert series:
  https://patch.msgid.link/20260710-series-vmap-race-fix-v1-0-5b3794c113fe@kernel.org
* CPA patch:
  https://patch.msgid.link/20260710-fix-cpa-ptdump-race-v1-1-d898699a7417@kernel.org
* non-init_mm patch:
  https://patch.msgid.link/20260710-b4-fix-non-init_mm-ptdump-v1-1-2d40982c98ec@kernel.org

To: Andrew Morton <akpm@linux-foundation.org>
To: Suren Baghdasaryan <surenb@google.com>
To: "Liam R. Howlett" <liam@infradead.org>
To: Vlastimil Babka <vbabka@kernel.org>
To: Shakeel Butt <shakeel.butt@linux.dev>
To: David Hildenbrand <david@kernel.org>
To: Mike Rapoport <rppt@kernel.org>
To: Michal Hocko <mhocko@suse.com>
To: Uladzislau Rezki <urezki@gmail.com>
To: Toshi Kani <toshi.kani@hpe.com>
To: Dave Hansen <dave.hansen@linux.intel.com>
To: Andy Lutomirski <luto@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>
To: Thomas Gleixner <tglx@kernel.org>
To: Ingo Molnar <mingo@redhat.com>
To: Borislav Petkov <bp@alien8.de>
To: x86@kernel.org
To: "H. Peter Anvin" <hpa@zytor.com>
To: Kiryl Shutsemau <kas@kernel.org>
To: Catalin Marinas <catalin.marinas@arm.com>
To: Will Deacon <will@kernel.org>
To: Dev Jain <dev.jain@arm.com>
To: Ryan Roberts <ryan.roberts@arm.com>
Cc: David Carlier <devnexen@gmail.com>
Cc: ljs@kernel.org
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
Cc: bpf@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org

---
Lorenzo Stoakes (4):
      mm/vmalloc: acquire init_mm lock on huge vmap to avoid ptdump UAF
      x86/mm/pat: acquire mmap lock on page table free to avoid ptdump UAF
      mm/ptdump: always stabilise against page table freeing using init_mm
      arm64: remove redundant concurrent ptdump UAF mitigation

 arch/arm64/include/asm/ptdump.h |  2 --
 arch/arm64/mm/mmu.c             | 43 ++++-------------------------------------
 arch/arm64/mm/ptdump.c          | 11 ++---------
 arch/x86/mm/pat/set_memory.c    | 14 +++++++++++---
 include/linux/mmap_lock.h       |  1 +
 mm/pagewalk.c                   | 36 ++++++++++++++++++++--------------
 mm/ptdump.c                     |  7 +++++++
 mm/vmalloc.c                    | 41 ++++++++++++++++++++++++++++++---------
 8 files changed, 78 insertions(+), 77 deletions(-)
---
base-commit: 44696aa3a489d2baf58efa61b37833f100072bee
change-id: 20260710-series-vmap-race-fix-2a4cac988938

Cheers,
-- 
Lorenzo Stoakes <ljs@kernel.org>

On Sun, Jul 12, 2026 at 11:42:23AM +0100, Lorenzo Stoakes wrote:
> This series addresses the issue by having the vmap huge promotion
> logic acquire the mmap read lock while both setting the huge page
> table entry and freeing the prior leaf page table.

Hi Lorenzo,

Before we settle on the mmap lock scheme, have you considered handling
this the way GUP-fast handles page table freeing -- RCU-defer the free
and make ptdump a lockless walker?

The locking here is inverted from what one would normally expect
(walker takes the write lock, mutators take read locks, mutators safe
against each other only by range ownership). It works, but it is
subtle, it is what produced the arm64 deadlock and the ifdeffery, and
it depends on every current and future freeing site remembering the
rule -- patch 3 exists because two walkers did not fit the scheme.

The free side looks cheap: kernel page table freeing already funnels
through pagetable_free_kernel(), which already has a deferred path
(used for IOMMU SVA). Adding a grace period there -- synchronize_rcu()
in the worker, amortized over the batch -- covers every freeing site
by construction.

On the walk side, nothing on the ptdump path can sleep -- the pagewalk
core only allocates for install_pte ops, kernel PTE level uses
pte_offset_kernel(), and the arch note_page() implementations are
seq_printf()/printk() into a preallocated buffer. So the walk could run
under rcu_read_lock() as is. The real work is bounding the read-side
sections: a full walk can take dozens of seconds on a KASAN kernel per
the comment in mm/ptdump.c, so it would need to drop RCU and
cond_resched() periodically, re-descending from the top. Note we
currently hold the init_mm mmap write lock across those same dozens of
seconds, and this series makes that load-bearing: during a long walk
every promotion trylock fails, silently degrading vmalloc to small
pages, and CPA collapse blocks.

That would give us: no inverted locking, no arm64 deadlock possibility
(your patch 4 stands on its own), the patch 3 walkers covered
structurally rather than by locking init_mm as well, and ptdump
invisible to production paths.

Given the live UAF, this could also be a follow-up rather than a
respin. But I would like to hear whether you see a fatal flaw in the
approach first.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

On Mon, Jul 13, 2026 at 02:32:09PM +0100, Kiryl Shutsemau wrote:
> On Sun, Jul 12, 2026 at 11:42:23AM +0100, Lorenzo Stoakes wrote:
> > This series addresses the issue by having the vmap huge promotion
> > logic acquire the mmap read lock while both setting the huge page
> > table entry and freeing the prior leaf page table.
>
> Hi Lorenzo,
>
> Before we settle on the mmap lock scheme, have you considered handling
> this the way GUP-fast handles page table freeing -- RCU-defer the free
> and make ptdump a lockless walker?

Overall I like the idea of eventually moving this to _all_ being
RCU-free-able :)

BUT... I don't like it as a fix for this bug that has to be backported.

I think there's a lot of subtleties to worry about and I don't want to
worry about having to maintain tht as a backport.

But also, we have an unfortunate situation with ptdump where it also walks
userland ranges on x86 (and ranges for efi_mm on both x86 and arm64).

And for userland ranges you have to have the mmap write lock to exclude a
downgraded mmap read lock munmap() operation (which gives rise to the weird
inversion you mention).

So the walker still has to take the write lock in this case.

I spoke a bit about it overall at [0].

We already have this mmap lock convention as a requirement for kernel
ranges, and it was being violated by CPA and vmalloc.

So I'd prefer we keep this as the proximate fix to solve the bug here, and
then revisit this later (alongside moving to RCU page table freeing
_overall_, though one doesn't have to block the other).

I believe Suren is looking at this (and I'm happy to look if Suren is tied
up with other things also!)

>
> The locking here is inverted from what one would normally expect
> (walker takes the write lock, mutators take read locks, mutators safe
> against each other only by range ownership). It works, but it is

Yeah :) this is a direct result of the munmap() thing mentioned above
(because ptdump can read non-kernel ranges).

Because you _have_ to take the mmap write lock there, so you _may as well_
exclude with a read lock and then get to do an mmap read trylock as an
added bonus on vmap huge page promotion.

> subtle, it is what produced the arm64 deadlock and the ifdeffery, and
> it depends on every current and future freeing site remembering the
> rule -- patch 3 exists because two walkers did not fit the scheme.

Well everybody who does things with shared resources should be aware of
locking requirements.

>
> The free side looks cheap: kernel page table freeing already funnels
> through pagetable_free_kernel(), which already has a deferred path
> (used for IOMMU SVA). Adding a grace period there -- synchronize_rcu()
> in the worker, amortized over the batch -- covers every freeing site
> by construction.

Well I did want to say btw that CPA doesn't actually mark the page tables
as kernel, was going to chase up with something on that when I got a chance
:)

synchronize_rcu() is a very bug hammer, you can just use call_rcu() (and
the ptdesc already has an rcu_head I think).


>
> On the walk side, nothing on the ptdump path can sleep -- the pagewalk
> core only allocates for install_pte ops, kernel PTE level uses
> pte_offset_kernel(), and the arch note_page() implementations are
> seq_printf()/printk() into a preallocated buffer. So the walk could run
> under rcu_read_lock() as is. The real work is bounding the read-side
> sections: a full walk can take dozens of seconds on a KASAN kernel per
> the comment in mm/ptdump.c, so it would need to drop RCU and
> cond_resched() periodically, re-descending from the top. Note we
> currently hold the init_mm mmap write lock across those same dozens of
> seconds, and this series makes that load-bearing: during a long walk
> every promotion trylock fails, silently degrading vmalloc to small
> pages, and CPA collapse blocks.
>

Yeah it all sucks, but you're really making my point that this is better to
revisit and consider carefully rather than doing as a fix.

> That would give us: no inverted locking, no arm64 deadlock possibility
> (your patch 4 stands on its own), the patch 3 walkers covered
> structurally rather than by locking init_mm as well, and ptdump
> invisible to production paths.
>
> Given the live UAF, this could also be a follow-up rather than a
> respin. But I would like to hear whether you see a fatal flaw in the
> approach first.

Yes - as above obviously I'd definitely prefer to defer this to a
follow-up.

The main concern for ptdump is properly slicing its walk to avoid
RCU stalls.

Keeping in mind that vmap can (in theory) span PUDs and even P4Ds it
becomes a bit tricky.

But also I worry about whether the entries in the page table will actually
be valid at the point the walker reads them.

For vmap/CPA pretty much yes they are, but if something was to actually
unmap them in future then that might no longer be the case. RCU will only
guarantee that the page tables stick around, not that they contain anything
valid.

In userland a lot of this is solved already by the leaf page table entry's
PTL.

Kernel page tables are a bit of a weird case, because mostly we solve
things through exclusive owners, ptdump being the weirdo outlier.

Anyway TL;DR is that - I think we need to be careful and think this through
and make changes across the board for all kernel page table users and
update the page walkers.

I'm happy to spin up an RFC for this when I get a moment :)

But in general I'd like us to solve the problem as a whole across the
kernel for both userland and kernel page table allocations, ideally!

>
> --
>   Kiryl Shutsemau / Kirill A. Shutemov

Cheers, Lorenzo

On Mon, Jul 13, 2026 at 04:54:27PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Mon, Jul 13, 2026 at 02:32:09PM +0100, Kiryl Shutsemau wrote:
> > On Sun, Jul 12, 2026 at 11:42:23AM +0100, Lorenzo Stoakes wrote:
> > > This series addresses the issue by having the vmap huge promotion
> > > logic acquire the mmap read lock while both setting the huge page
> > > table entry and freeing the prior leaf page table.
> >
> > Hi Lorenzo,
> >
> > Before we settle on the mmap lock scheme, have you considered handling
> > this the way GUP-fast handles page table freeing -- RCU-defer the free
> > and make ptdump a lockless walker?
> 
> Overall I like the idea of eventually moving this to _all_ being
> RCU-free-able :)
> 
> BUT... I don't like it as a fix for this bug that has to be backported.
> 
> I think there's a lot of subtleties to worry about and I don't want to
> worry about having to maintain tht as a backport.
> 
> But also, we have an unfortunate situation with ptdump where it also walks
> userland ranges on x86 (and ranges for efi_mm on both x86 and arm64).
> 
> And for userland ranges you have to have the mmap write lock to exclude a
> downgraded mmap read lock munmap() operation (which gives rise to the weird
> inversion you mention).
> 
> So the walker still has to take the write lock in this case.

I think the userland side is fixable too.

My first thought was to walk only VMA-backed ranges, which would make
the mmap read lock sufficient: munmap() detaches VMAs under the write
lock before downgrading, and free_pgtables() only frees tables
exclusively covering the detached range.

But that is an unnecessary limitation on MMU_GATHER_RCU_TABLE_FREE
architectures -- which is every architecture with generic ptdump
support. There, user page table freeing is already RCU-deferred; it is
what GUP-fast relies on to walk user page tables blindly with no mmap
lock. khugepaged retracting PTE tables under the pmd lock is likewise
covered by pte_free_defer().

So the downgraded munmap() teardown you mention is already safe to race
with -- for a walker inside an RCU read-side section. The write lock is
only needed today because ptdump walks outside of one, so the deferred
freeing does nothing for it.

That would make the end state: the whole of ptdump -- userland, efi_mm
and kernel ranges -- walks under RCU with no mmap lock taken at all,
and the kernel-side freeing conversion we discussed is the only
missing piece.

> I spoke a bit about it overall at [0].

I think you forgot to paste [0] :)

> We already have this mmap lock convention as a requirement for kernel
> ranges, and it was being violated by CPA and vmalloc.
> 
> So I'd prefer we keep this as the proximate fix to solve the bug here, and
> then revisit this later (alongside moving to RCU page table freeing
> _overall_, though one doesn't have to block the other).

No argument, let's do it as a follow-up.

I will give the current patchset a proper review.

> > The free side looks cheap: kernel page table freeing already funnels
> > through pagetable_free_kernel(), which already has a deferred path
> > (used for IOMMU SVA). Adding a grace period there -- synchronize_rcu()
> > in the worker, amortized over the batch -- covers every freeing site
> > by construction.
> 
> Well I did want to say btw that CPA doesn't actually mark the page tables
> as kernel, was going to chase up with something on that when I got a chance
> :)

You are right. CPA allocates the split tables with bare
pagetable_alloc(), so ptdesc_test_kernel() is false and collapse frees
them via __pagetable_free() directly. Note this also means they skip
the IOMMU SVA KVA invalidation that ASYNC_KERNEL_PGTABLE_FREE is there
for -- that looks like a bug today, independent of ptdump.

> 
> synchronize_rcu() is a very bug hammer, you can just use call_rcu() (and
> the ptdesc already has an rcu_head I think).

Sure, call_rcu().

> Keeping in mind that vmap can (in theory) span PUDs and even P4Ds it
> becomes a bit tricky.

Since the walker only reads, I think it is enough to re-descend from
the pgd with fresh READ_ONCE() at each level after dropping RCU --
whatever level got promoted in the meantime, we see either the new leaf
or the old table. But agreed this is the part that needs the most care.

> 
> But also I worry about whether the entries in the page table will actually
> be valid at the point the walker reads them.
> 
> For vmap/CPA pretty much yes they are, but if something was to actually
> unmap them in future then that might no longer be the case. RCU will only
> guarantee that the page tables stick around, not that they contain anything
> valid.

For a walker that only reads, I don't think staleness is a problem as
long as frees are RCU-deferred: the walker can only reach a table via
an entry it read within its RCU section, and freeing requires unlinking
that entry first, so the grace period covers any walker that saw the
old pointer. Stale leaf contents are harmless for a dumper. It does
become a problem for anything that dereferences through leaf entries --
agreed that the general case needs more care than ptdump does.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

On 2026-07-13 17:32 +0100, Kiryl Shutsemau wrote:
> On Mon, Jul 13, 2026 at 04:54:27PM +0100, Lorenzo Stoakes (ARM) wrote:
> > On Mon, Jul 13, 2026 at 02:32:09PM +0100, Kiryl Shutsemau wrote:
> > > On Sun, Jul 12, 2026 at 11:42:23AM +0100, Lorenzo Stoakes wrote:
> > > > This series addresses the issue by having the vmap huge promotion
> > > > logic acquire the mmap read lock while both setting the huge page
> > > > table entry and freeing the prior leaf page table.
> > >
> > > Hi Lorenzo,
> > >
> > > Before we settle on the mmap lock scheme, have you considered handling
> > > this the way GUP-fast handles page table freeing -- RCU-defer the free
> > > and make ptdump a lockless walker?
> >
> > Overall I like the idea of eventually moving this to _all_ being
> > RCU-free-able :)
> >
> > BUT... I don't like it as a fix for this bug that has to be backported.
> >
> > I think there's a lot of subtleties to worry about and I don't want to
> > worry about having to maintain tht as a backport.
> >
> > But also, we have an unfortunate situation with ptdump where it also walks
> > userland ranges on x86 (and ranges for efi_mm on both x86 and arm64).
> >
> > And for userland ranges you have to have the mmap write lock to exclude a
> > downgraded mmap read lock munmap() operation (which gives rise to the weird
> > inversion you mention).
> >
> > So the walker still has to take the write lock in this case.
>
> I think the userland side is fixable too.
>
> My first thought was to walk only VMA-backed ranges, which would make
> the mmap read lock sufficient: munmap() detaches VMAs under the write
> lock before downgrading, and free_pgtables() only frees tables
> exclusively covering the detached range.

Right, but some architectures have non-VMA ranges in userland (yuck), and
ptdump is the only case where we really do this. It sucks really.

I'm not sure if it's ok to just stop outputting this information.

But as you say, we don't actually have to worry about this if we're RCU
freeing.

>
> But that is an unnecessary limitation on MMU_GATHER_RCU_TABLE_FREE
> architectures -- which is every architecture with generic ptdump
> support. There, user page table freeing is already RCU-deferred; it is

The arches are powerpc, s390, arm64, x86 and riscv and indeed all set
MMU_GATHER_RCU_TABLE_FREE.

But it's more complicated than that unfortunately. As per mmu_gather.c -
'Semi RCU freeing of the page directories'.

And MMU_GATHER_RCU_TABLE_FREE != RCU page table freeing.

You can see the complexity in Qi's commit 718b13861d22 ("x86: mm: free page
table pages by RCU instead of semi RCU").

Basically you need the IRQs disabled to get the semi-RCU behaviour and to
be able to safely traverse page tables that way.

So with CONFIG_PT_RECLAIM you're safe to RCU traverse PTEs only.

Looking at the Kconfig entry:

config PT_RECLAIM
	def_bool y
	depends on MMU_GATHER_RCU_TABLE_FREE && !HAVE_ARCH_TLB_REMOVE_TABLE

And indeed, we fall back to just freeing PTE page tables immediately
(relying on IPI sync) if it's not set:

#ifdef CONFIG_PT_RECLAIM
static inline void __tlb_remove_table_one_rcu(struct rcu_head *head)
{
	struct ptdesc *ptdesc;

	ptdesc = container_of(head, struct ptdesc, pt_rcu_head);
	__tlb_remove_table(ptdesc);
}

static inline void __tlb_remove_table_one(void *table)
{
	struct ptdesc *ptdesc;

	ptdesc = table;
	call_rcu(&ptdesc->pt_rcu_head, __tlb_remove_table_one_rcu);
}
#else
static inline void __tlb_remove_table_one(void *table)
{
	tlb_remove_table_sync_rcu();
	__tlb_remove_table(table);
}
#endif /* CONFIG_PT_RECLAIM */

HAVE_ARCH_TLB_REMOVE_TABLE is set for powerpc, which also enables PTDUMP :)
and that's because it actually tracks multiple PTE page tables together as
a fragment.

So in general - it's unfortunately not so easy, and we can't just rely on
RCU.

Obviously the above is moreso about userland mappings, but it suggests that
_just_ removing page tables under RCU isn't sufficient here.

You'd also need a lockless ptdump walker to carefully handle the weird edge
cases as per pmdp_get_lockless() and ptep_get_lockless(), which the kernel
page table wlakers not currently using.

In the case of kernel page tables we can just always RCU defer page table
freeing (having audited all callers to make sure nobody's freeing them in a
broken way).

But I wonder how the PPC fragment page tables stuff would interact with
that, it's something that'd need to be audited.


> what GUP-fast relies on to walk user page tables blindly with no mmap
> lock. khugepaged retracting PTE tables under the pmd lock is likewise
> covered by pte_free_defer().

See above.

>
> So the downgraded munmap() teardown you mention is already safe to race
> with -- for a walker inside an RCU read-side section. The write lock is
> only needed today because ptdump walks outside of one, so the deferred
> freeing does nothing for it.

Yup.

>
> That would make the end state: the whole of ptdump -- userland, efi_mm
> and kernel ranges -- walks under RCU with no mmap lock taken at all,
> and the kernel-side freeing conversion we discussed is the only
> missing piece.

Yes the ideal situation is that we don't have to worry about an mmap or VMA
lock at all and can stabilise on RCU only.

Well only for ptdump.



>
> > I spoke a bit about it overall at [0].
>
> I think you forgot to paste [0] :)

Yeah sorry! It's kinda immaterial now anyway :P as we're discussing in
detail here.


>
> > We already have this mmap lock convention as a requirement for kernel
> > ranges, and it was being violated by CPA and vmalloc.
> >
> > So I'd prefer we keep this as the proximate fix to solve the bug here, and
> > then revisit this later (alongside moving to RCU page table freeing
> > _overall_, though one doesn't have to block the other).
>
> No argument, let's do it as a follow-up.
>
> I will give the current patchset a proper review.

Thanks!

>
> > > The free side looks cheap: kernel page table freeing already funnels
> > > through pagetable_free_kernel(), which already has a deferred path
> > > (used for IOMMU SVA). Adding a grace period there -- synchronize_rcu()
> > > in the worker, amortized over the batch -- covers every freeing site
> > > by construction.
> >
> > Well I did want to say btw that CPA doesn't actually mark the page tables
> > as kernel, was going to chase up with something on that when I got a chance
> > :)
>
> You are right. CPA allocates the split tables with bare
> pagetable_alloc(), so ptdesc_test_kernel() is false and collapse frees
> them via __pagetable_free() directly. Note this also means they skip
> the IOMMU SVA KVA invalidation that ASYNC_KERNEL_PGTABLE_FREE is there
> for -- that looks like a bug today, independent of ptdump.

Yeah it is, I'll send a patch.

>
> >
> > synchronize_rcu() is a very bug hammer, you can just use call_rcu() (and
> > the ptdesc already has an rcu_head I think).
>
> Sure, call_rcu().
>
> > Keeping in mind that vmap can (in theory) span PUDs and even P4Ds it
> > becomes a bit tricky.
>
> Since the walker only reads, I think it is enough to re-descend from
> the pgd with fresh READ_ONCE() at each level after dropping RCU --
> whatever level got promoted in the meantime, we see either the new leaf
> or the old table. But agreed this is the part that needs the most care.

There could be torn writes observed that would need careful revalidation
checks though couldn't there potentially?

See e.g. contpte_ptep_get_lockless() and pmdp_get_lockless() if
CONFIG_GUP_GET_PXX_LOW_HIGH.

And things are complicated in kernel code obviously by the fact you have no
PTL guarantees anymore.

And OK maybe we can prove the huge promotion case is OK, this assumes that
we won't in future free kernel page tables under some other
circumstances... :)

I'm not saying it's not doable though, just that we have to be _very
careful_ how we do it if we're going to try to do this under RCU!

>
> >
> > But also I worry about whether the entries in the page table will actually
> > be valid at the point the walker reads them.
> >
> > For vmap/CPA pretty much yes they are, but if something was to actually
> > unmap them in future then that might no longer be the case. RCU will only
> > guarantee that the page tables stick around, not that they contain anything
> > valid.
>
> For a walker that only reads, I don't think staleness is a problem as
> long as frees are RCU-deferred: the walker can only reach a table via
> an entry it read within its RCU section, and freeing requires unlinking
> that entry first, so the grace period covers any walker that saw the
> old pointer. Stale leaf contents are harmless for a dumper. It does
> become a problem for anything that dereferences through leaf entries --
> agreed that the general case needs more care than ptdump does.

Well I wonder about torn writes actually, you'd need to be careful about
revalidating in the walk.

>
> --
>   Kiryl Shutsemau / Kirill A. Shutemov
>

In general Ithink all the problems are ultimately solveable, they're just
fiddly :)

Cheers, Lorenzo

On Tue, Jul 14, 2026 at 05:20:29PM +0100, Lorenzo Stoakes (ARM) wrote:
> Basically you need the IRQs disabled to get the semi-RCU behaviour and to
> be able to safely traverse page tables that way.
> 
> So with CONFIG_PT_RECLAIM you're safe to RCU traverse PTEs only.

I think this is stale. Since 1fb3d8c20bfa ("mm/mmu_gather: replace IPI
with synchronize_rcu() when batch allocation fails") the !PT_RECLAIM
fallback is a real grace period -- note your own quote of
__tlb_remove_table_one() calls tlb_remove_table_sync_rcu(), which is
synchronize_rcu() nowadays, not the IPI broadcast. The name invites
the confusion.

Together with a37259732a7d ("x86/mm: Make MMU_GATHER_RCU_TABLE_FREE
unconditional") that means every freeing path through the generic
mmu_gather is genuine RCU, batched or not, all levels. A plain
rcu_read_lock() walker is protected; IRQs disabled is not required.

> HAVE_ARCH_TLB_REMOVE_TABLE is set for powerpc, which also enables PTDUMP :)
> and that's because it actually tracks multiple PTE page tables together as
> a fragment.

sparc also sets it with SMP. So the audit list for custom
tlb_remove_table() implementations is powerpc and sparc -- everything
else gets the generic behaviour above.

No disagreement on the rest: the walker needs
ptep_get_lockless()/pmdp_get_lockless(), and the ppc kernel-side
fragments need a look.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

+cc Lance for thanks :)

On Tue, Jul 14, 2026 at 06:05:01PM +0100, Kiryl Shutsemau wrote:
> On Tue, Jul 14, 2026 at 05:20:29PM +0100, Lorenzo Stoakes (ARM) wrote:
> > Basically you need the IRQs disabled to get the semi-RCU behaviour and to
> > be able to safely traverse page tables that way.
> >
> > So with CONFIG_PT_RECLAIM you're safe to RCU traverse PTEs only.
>
> I think this is stale. Since 1fb3d8c20bfa ("mm/mmu_gather: replace IPI
> with synchronize_rcu() when batch allocation fails") the !PT_RECLAIM
> fallback is a real grace period -- note your own quote of
> __tlb_remove_table_one() calls tlb_remove_table_sync_rcu(), which is
> synchronize_rcu() nowadays, not the IPI broadcast. The name invites
> the confusion.

Ahhh, nice thank you Lance :)


>
> Together with a37259732a7d ("x86/mm: Make MMU_GATHER_RCU_TABLE_FREE
> unconditional") that means every freeing path through the generic
> mmu_gather is genuine RCU, batched or not, all levels. A plain
> rcu_read_lock() walker is protected; IRQs disabled is not required.

Lovely. I'll probably send a patch to fix the comment then :)

>
> > HAVE_ARCH_TLB_REMOVE_TABLE is set for powerpc, which also enables PTDUMP :)
> > and that's because it actually tracks multiple PTE page tables together as
> > a fragment.
>
> sparc also sets it with SMP. So the audit list for custom
> tlb_remove_table() implementations is powerpc and sparc -- everything
> else gets the generic behaviour above.

Yeah.

>
> No disagreement on the rest: the walker needs
> ptep_get_lockless()/pmdp_get_lockless(), and the ppc kernel-side
> fragments need a look.

Indeed.

I know Suren's been taken a look at this, pinged him off-list (he's also
cc'd) to see where he's at but I'm going to take a general look at this.

I feel that RCU page table freeing is likely to be a requirement for
scalable CoW, so directly impacts me :)

>
> --
>   Kiryl Shutsemau / Kirill A. Shutemov

Cheers, Lorenzo