[v3] KVM: arm64: nv: Implement nested stage-2 reverse map

[PATCH v3 0/5] KVM: arm64: nv: Implement nested stage-2 reverse map

Posted by Wei-Lin Chang 1 month ago

Hi,

This is v3 of optimizing the shadow s2 mmu unmapping during MMU
notifiers.

Two new preparatory patches are added, one reduces a hole in kvm_s2_mmu
and another refactors the code a bit during s2 faults. Other changes are
listed below.

* Changes from v2 [1]:

  - Removed "polluted" teminology.

  - Use xa_{mk, to}_value() when storing and retriving values from maple
    trees.

  - Avoid using the 63rd bit in maple tree values so that xa_{mk, to}_value()
    does not lose us a bit.

  - Added reverse map removal during TLBI handling.

  - Other suggested refactorings.

Thanks!

[1]: https://lore.kernel.org/kvmarm/20260411125024.3735989-1-weilin.chang@arm.com/

Wei-Lin Chang (5):
  KVM: arm64: Use a variable for the canonical GPA in kvm_s2_fault_map()
  KVM: arm64: Move shadow_pt_debugfs_dentry to reduce holes in
    kvm_s2_mmu
  KVM: arm64: nv: Avoid full shadow s2 unmap
  KVM: arm64: nv: Remove reverse map entries during TLBI handling
  KVM: arm64: nv: Create nested IPA direct map to speed up reverse map
    removal

 arch/arm64/include/asm/kvm_host.h   |  17 +-
 arch/arm64/include/asm/kvm_nested.h |   6 +
 arch/arm64/kvm/mmu.c                |  43 +++--
 arch/arm64/kvm/nested.c             | 238 +++++++++++++++++++++++++++-
 arch/arm64/kvm/sys_regs.c           |   3 +
 5 files changed, 290 insertions(+), 17 deletions(-)

-- 
2.43.0

Re: [PATCH v3 0/5] KVM: arm64: nv: Implement nested stage-2 reverse map

Posted by Itaru Kitayama 3 weeks, 2 days ago

Hi Wei Lin,

On Sun, May 10, 2026 at 03:53:33PM +0100, Wei-Lin Chang wrote:
> Hi,
> 
> This is v3 of optimizing the shadow s2 mmu unmapping during MMU
> notifiers.
> 
> Two new preparatory patches are added, one reduces a hole in kvm_s2_mmu
> and another refactors the code a bit during s2 faults. Other changes are
> listed below.
> 
> * Changes from v2 [1]:
> 
>   - Removed "polluted" teminology.
> 
>   - Use xa_{mk, to}_value() when storing and retriving values from maple
>     trees.
> 
>   - Avoid using the 63rd bit in maple tree values so that xa_{mk, to}_value()
>     does not lose us a bit.
> 
>   - Added reverse map removal during TLBI handling.
> 
>   - Other suggested refactorings.
> 
> Thanks!

With this series applied on the recent kvmarm/fixes (minor local modification), 
your v3 hello_nested runs fine on all three page granules; without this 16KB 
kernel can't handle stage 2 unmapping paths. I tested using the latest QEMU only 
so wonder how you're testing.  

Thanks,
Itaru.

> 
> [1]: https://lore.kernel.org/kvmarm/20260411125024.3735989-1-weilin.chang@arm.com/
> 
> Wei-Lin Chang (5):
>   KVM: arm64: Use a variable for the canonical GPA in kvm_s2_fault_map()
>   KVM: arm64: Move shadow_pt_debugfs_dentry to reduce holes in
>     kvm_s2_mmu
>   KVM: arm64: nv: Avoid full shadow s2 unmap
>   KVM: arm64: nv: Remove reverse map entries during TLBI handling
>   KVM: arm64: nv: Create nested IPA direct map to speed up reverse map
>     removal
> 
>  arch/arm64/include/asm/kvm_host.h   |  17 +-
>  arch/arm64/include/asm/kvm_nested.h |   6 +
>  arch/arm64/kvm/mmu.c                |  43 +++--
>  arch/arm64/kvm/nested.c             | 238 +++++++++++++++++++++++++++-
>  arch/arm64/kvm/sys_regs.c           |   3 +
>  5 files changed, 290 insertions(+), 17 deletions(-)
> 
> -- 
> 2.43.0
>

Re: [PATCH v3 0/5] KVM: arm64: nv: Implement nested stage-2 reverse map

Posted by Marc Zyngier 2 weeks, 1 day ago

On Wed, 20 May 2026 08:31:12 +0100,
Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote:
> 
> Hi Wei Lin,
> 
> On Sun, May 10, 2026 at 03:53:33PM +0100, Wei-Lin Chang wrote:
> > Hi,
> > 
> > This is v3 of optimizing the shadow s2 mmu unmapping during MMU
> > notifiers.
> > 
> > Two new preparatory patches are added, one reduces a hole in kvm_s2_mmu
> > and another refactors the code a bit during s2 faults. Other changes are
> > listed below.
> > 
> > * Changes from v2 [1]:
> > 
> >   - Removed "polluted" teminology.
> > 
> >   - Use xa_{mk, to}_value() when storing and retriving values from maple
> >     trees.
> > 
> >   - Avoid using the 63rd bit in maple tree values so that xa_{mk, to}_value()
> >     does not lose us a bit.
> > 
> >   - Added reverse map removal during TLBI handling.
> > 
> >   - Other suggested refactorings.
> > 
> > Thanks!
> 
> With this series applied on the recent kvmarm/fixes (minor local modification), 
> your v3 hello_nested runs fine on all three page granules; without this 16KB 
> kernel can't handle stage 2 unmapping paths. I tested using the latest QEMU only 
> so wonder how you're testing.

Can you clarify this statement?

Are you saying that 16kB S2 unmap do not work with upstream? That
seems odd, as it would mean NV wouldn't work on Apple HW, which
represents two thirds of my development boxes...

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v3 0/5] KVM: arm64: nv: Implement nested stage-2 reverse map

Posted by Itaru Kitayama 2 weeks, 1 day ago

On Thu, May 28, 2026 at 11:19:34AM +0100, Marc Zyngier wrote:
> On Wed, 20 May 2026 08:31:12 +0100,
> Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote:
> > 
> > Hi Wei Lin,
> > 
> > On Sun, May 10, 2026 at 03:53:33PM +0100, Wei-Lin Chang wrote:
> > > Hi,
> > > 
> > > This is v3 of optimizing the shadow s2 mmu unmapping during MMU
> > > notifiers.
> > > 
> > > Two new preparatory patches are added, one reduces a hole in kvm_s2_mmu
> > > and another refactors the code a bit during s2 faults. Other changes are
> > > listed below.
> > > 
> > > * Changes from v2 [1]:
> > > 
> > >   - Removed "polluted" teminology.
> > > 
> > >   - Use xa_{mk, to}_value() when storing and retriving values from maple
> > >     trees.
> > > 
> > >   - Avoid using the 63rd bit in maple tree values so that xa_{mk, to}_value()
> > >     does not lose us a bit.
> > > 
> > >   - Added reverse map removal during TLBI handling.
> > > 
> > >   - Other suggested refactorings.
> > > 
> > > Thanks!
> > 
> > With this series applied on the recent kvmarm/fixes (minor local modification), 
> > your v3 hello_nested runs fine on all three page granules; without this 16KB 
> > kernel can't handle stage 2 unmapping paths. I tested using the latest QEMU only 
> > so wonder how you're testing.
> 
> Can you clarify this statement?
> 
> Are you saying that 16kB S2 unmap do not work with upstream? That
> seems odd, as it would mean NV wouldn't work on Apple HW, which
> represents two thirds of my development boxes...

Yes, I built this morning's upstream kvmarm/fixes with 16KB granule fresh, and tested
Wei Lin's propsed KVM selftest, hello_nested on QEMU with the small buildroot
filesystem and saw the above i.e., the stage 2 unmapping paths never finish.

I thought it was due to the Ubuntu userspace I was using for testing,
but not sure now.

Thanks,
Itaru.

> 
> Thanks,
> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.