The gfn_to_pfn_cache read side is currently protected by a per-GPC
rwlock_t. That is a problem on PREEMPT_RT, which turns rwlock_t into a
sleeping lock, when the GPC is read from hardirq context (the Xen timer
callback and kvm_arch_set_irq_inatomic()) and from the scheduler path
(kvm_xen_runstate_set_preempted() via kvm_sched_out()).
As in v3, this version takes the locking away instead of working around
it. Readers now run under a dedicated per-VM SRCU domain (kvm->gpc_srcu)
— Sean was right that a system-wide RCU grace period in the refresh path
is a non-starter on large hosts. Mutators clear the valid bit, wait for
readers of only this VM's caches to drain, and only then touch anything
a reader might be looking at — exactly the pattern of a TLB shootdown,
and for the same reason.
Losing gpc->lock means the refresh's final "did an invalidation race
with my lookup?" check and its publish are no longer atomic with respect
to the mmu_notifier walk, and "valid is already clear" no longer implies
"no readers remain". Testing the earlier revisions of the conversion
with a KASAN+lockdep soak of the syzbot reproducers found the resulting
use-after-free and two further races; the cache state is now a
three-bit atomic word (VALID / BECOMING_VALID / INVALIDATING) with a
cmpxchg publish which an overlapping invalidation can veto. The details
are in patch 1's commit message. The soak, alongside continuous Xen
guest boot cycles, has been clean against the current version, and
syzbot has verified the fix against the reproducer for one of the two
reports patch 1 addresses (the other has no reproducer to test). I'll
let the soak accumulate more hours before I'd call this more than an
RFC, and review of the reader/mutator protocol in patch 1 is the thing
I most want eyes (and Sashiko) on.
This applies on top of the just-posted Xen series¹ — that is a textual
dependency only (patch 1 touches the same pfncache code its 11/11
does); there is no functional dependency. A git tree with everything in
the right order is at:
https://git.infradead.org/?p=users/dwmw2/linux.git;a=shortlog;h=refs/heads/xen-srcu-prealloc
Two further generic patches are soft dependencies — the right thing to
do, but needed neither to review nor to test this series:
• mm/mmu_notifier: Remove non_block_start/end() from notifier
invocation² — the invalidation path now waits for an SRCU grace
period, which is safe even for the OOM reaper (readers never
allocate, never take mmap_lock, never sleep), but the annotation is
coarser than the actual requirement and would splat on any voluntary
schedule.
• srcu: Keep a spare node array so srcu_gp_end() need not block in
reclaim³ — closes the one remaining allocation-in-grace-period
corner, where the one-time transition of an srcu_struct to its node
tree could otherwise block in reclaim on the workqueue the OOM
reaper's grace period is queued behind.
The standalone fix for the original syzbot reports (v3's patch 1) moved
to the Xen series¹ as its patch 11/11, since it is Cc: stable material
and ready to go while we continue to refine *this* series.
Patches 2-4 are Sean's, unchanged from v3 apart from context. Patches 5
and 6 convert the steal-time / preempted status update to use the GPC:
patch 5 moved the TLB_FLUSH request handling our of the critical
section, and patch 6 (Carsten's) then removes the last user of the old
map-on-demand path from the scheduler context.
¹ https://lore.kernel.org/all/20260811094829.98794-1-dwmw2@infradead.org/
² https://lore.kernel.org/all/a247c49dd61af1df7ddc4dcb1cccbcb36b04d9d5.camel@infradead.org/
³ https://lore.kernel.org/all/6eed3fe3461e9690b486ca98fa7563f60d3940ff.camel@infradead.org/
v3: https://lore.kernel.org/all/20260805195528.3853473-1-dwmw@amazon.co.uk/
v4:
- Switch from RCU to a dedicated per-VM SRCU domain (Sean).
- Squash the conversion into one patch, with the three race fixes
found by the reproducer soak: restore cache fields on a vetoed
publish, GPC_INVALIDATING so mutators wait out a still-running
invalidation grace period, and memory-ordering fixes for the
announce/veto handshake (all detailed in patch 1's message).
- Clear GPC_BECOMING_VALID when a refresh fails.
- Move the syzbot sequence fix to the Xen series; drop the
mmu_notifier patch to a standalone posting.
- Split the guest TLB_FLUSH request handling changes out of the
steal_time patch into their own patch.
Carsten Stollmaier (1):
KVM: x86: Use gfn_to_pfn_cache for steal time / preempted status
David Woodhouse (2):
KVM: pfncache: Use SRCU for readers instead of a rwlock
KVM: x86: Request the guest TLB flush from record_steal_time()
Sean Christopherson (3):
KVM: x86/xen: Extract delivery of event to vCPU into a separate helper
KVM: x86/xen: Explicitly tag "shared info" page as never being dirty tracked
KVM: x86/xen: Don't dirty track "vCPU info" page
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/msrs.c | 7 +-
arch/x86/kvm/x86.c | 155 ++++++++-------
arch/x86/kvm/xen.c | 267 +++++++++++++------------
include/linux/kvm_host.h | 35 ++--
include/linux/kvm_types.h | 38 +++-
virt/kvm/kvm_main.c | 9 +
virt/kvm/pfncache.c | 427 ++++++++++++++++++++++++++++++----------
8 files changed, 626 insertions(+), 314 deletions(-)