arch/arm64/include/asm/kvm_host.h | 31 +++++- arch/arm64/include/asm/kvm_nested.h | 7 ++ arch/arm64/kvm/mmu.c | 109 +++++++++++++++++++--- arch/arm64/kvm/nested.c | 140 +++++++++++++++++++++++++++- 4 files changed, 269 insertions(+), 18 deletions(-)
Hi,
This is v6 of optimizing the shadow s2 mmu unmapping during MMU
notifiers.
This version fixes a few issues, and adds Marc's space optimization
for kvm_guest_s2_mapping [1]. Please see the changelog for the details.
Tested by booting L3, and running in-kernel targetted tests described
in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't
carry them over since some bugs fixed in v6 are non-obvious. Some
retest would be much appreciated!
Series based on v7.3-rc3 + Marc's nested mmu lifecycle fixes [3].
* Changes from v5 [4]:
- Align the addresses down to the mapping size when recording the
guest s2 mappings. s2fd->fault_ipa isn't necessarily PAGE_SIZE aligned.
- Record guest s2 mappings even when page table maps return -EAGAIN.
kvm_pgtable_stage2_map() can create mappings while returning -EAGAIN.
For example, a 2M block map (A) could race with a 4K page map (B):
1. (A) maps the 2M block in kvm_pgtable_visitor_cb()
2. (B) breaks that block into a table and maps 4K
3. (A) reloads and finds the table after kvm_pgtable_visitor_cb(),
then descends into it.
4. (A) maps some 4K, but before it finishes reads entry mapped by (B).
5. (A) returns -EAGAIN although it had mapped a few pages.
In this case, we don't know what subrange is mapped, just track the whole
requested mapping range.
- Don't remove tracked mappings from the interval trees if they only
partially overlap the removal range. Because of the previous bullet point
we can have a 4K shadow mapping tracked as a 2M range in the interval
trees. It would be wrong to remove the 2M range when a guest TLBI doesn't
touch the 4K mapped.
- Check mmu->pgt during mmu unmap notifier, as it could race against
MMU teardown.
- Make guest_s2_tracking_destroy() canonical mmu only. We simply don't
need to detach the nodes from the nested mmus' trees. Freeing them
during canonical mmu's teardown is enough.
Thanks!
[1]: https://lore.kernel.org/kvmarm/86h5jv7qrd.wl-maz@kernel.org/
[2]: https://lore.kernel.org/kvmarm/gerjpm62a2gszzggc6vuai22bf3prfquvfsp7ueumps7vz2ev3@odlmovaklg2b/
[3]: https://lore.kernel.org/kvmarm/20260911162203.1919330-1-maz@kernel.org/
[4]: https://lore.kernel.org/kvmarm/20260810205038.118843-1-weilin.chang@arm.com/
Marc Zyngier (1):
KVM: arm64: nv: Drop kvm_s2_mmu pointer from kvm_guest_s2_mapping
Wei-Lin Chang (6):
KVM: arm64: Use a variable for the canonical IPA in kvm_s2_fault_map()
KVM: arm64: nv: Introduce guest stage-2 tracking structures
KVM: arm64: nv: Track guest stage-2 mapping creation
KVM: arm64: nv: Track guest stage-2 mapping removal
KVM: arm64: nv: Avoid full shadow stage-2 unmap
KVM: arm64: Refactor kvm_unmap_gfn_range() with common variables
arch/arm64/include/asm/kvm_host.h | 31 +++++-
arch/arm64/include/asm/kvm_nested.h | 7 ++
arch/arm64/kvm/mmu.c | 109 +++++++++++++++++++---
arch/arm64/kvm/nested.c | 140 +++++++++++++++++++++++++++-
4 files changed, 269 insertions(+), 18 deletions(-)
--
2.43.0
On Tue, 15 Sep 2026 16:42:58 +0100, Wei-Lin Chang wrote:
> This is v6 of optimizing the shadow s2 mmu unmapping during MMU
> notifiers.
>
> This version fixes a few issues, and adds Marc's space optimization
> for kvm_guest_s2_mapping [1]. Please see the changelog for the details.
>
> Tested by booting L3, and running in-kernel targetted tests described
> in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't
> carry them over since some bugs fixed in v6 are non-obvious. Some
> retest would be much appreciated!
>
> [...]
Applied to next, thanks!
[1/7] KVM: arm64: Use a variable for the canonical IPA in kvm_s2_fault_map()
commit: 3b8ce2b5132ae4eebd81811ffd3bc2395c41ecfd
[2/7] KVM: arm64: nv: Introduce guest stage-2 tracking structures
commit: a6ca6cba59598de94eda90e4b3a4e405621aca90
[3/7] KVM: arm64: nv: Track guest stage-2 mapping creation
commit: c2778d14cd2ff757224715221a2ddff89fc6f127
[4/7] KVM: arm64: nv: Track guest stage-2 mapping removal
commit: 5491980c1a416139ea67faa6c4d52fa1ed0cd4e0
[5/7] KVM: arm64: nv: Avoid full shadow stage-2 unmap
commit: e6084aecaeefe9177280ea74c90127294260d805
[6/7] KVM: arm64: nv: Drop kvm_s2_mmu pointer from kvm_guest_s2_mapping
commit: 2064f15b08485384d4fca892c63a1c1b4f86e9c8
[7/7] KVM: arm64: Refactor kvm_unmap_gfn_range() with common variables
commit: 862fa568078514d0a41bd0946a65c23a0c0edec9
Cheers,
M.
--
Without deviation from the norm, progress is not possible.
On 9/15/26 11:42 PM, Wei-Lin Chang wrote: > Hi, > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > notifiers. > > This version fixes a few issues, and adds Marc's space optimization > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > Tested by booting L3, and running in-kernel targetted tests described > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > carry them over since some bugs fixed in v6 are non-obvious. Some > retest would be much appreciated! > > Series based on v7.3-rc3 + Marc's nested mmu lifecycle fixes [3]. > Hi Wei-Lin, Thanks for the updated series. I tested all seven patches on top of v7.3-rc3 with Marc's two nested MMU lifetime fixes, using the L1 boot workload that previously triggered soft lockups during host NUMA migration. The setup was: - ARM64 host with nested virtualization enabled - Automatic NUMA balancing enabled, with numad running - QEMU 10.2.3, virtualization=on - Ubuntu 26.04 L1 guest, 8 vCPUs and 32 GiB RAM Each trial used a fresh overlay backed by the same read-only golden image. I ran 15 boot trials, observing each for at least 600 seconds. Result Count ----------------------------- ----- Trials reaching guest login 15/15 Trials with guest soft lockups 0/15 Guest RCU stalls 0 Host hung-task reports 0 Host soft lockups 0 Tracing confirmed that host page migration and the reverse-map recording path were exercised. I did not reproduce the original soft lockup in this batch. For the series: Tested-by: Shuai Xue <xueshuai@linux.alibaba.com> Thanks, Shuai
Hi Wei-Lin, On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > Hi, > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > notifiers. > > This version fixes a few issues, and adds Marc's space optimization > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > Tested by booting L3, and running in-kernel targetted tests described > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > carry them over since some bugs fixed in v6 are non-obvious. Some > retest would be much appreciated! > > Series based on v7.3-rc3 + Marc's nested mmu lifecycle fixes [3]. Observed on QEMU (accel tcg) only, but I saw: [ 1812.052752] BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:1630 [ 1812.052896] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 596, name: shadow_stage2 The kernel is built on top of Marc's fix and your version six series, but the fix for debugfs will be posted later? Your hello_nested and shadow_stage2 ran without an issue. Tested-by: Itaru Kitayama <itaru.kitayama@fujitsu.com> Thanks, Itaru. > > * Changes from v5 [4]: > > - Align the addresses down to the mapping size when recording the > guest s2 mappings. s2fd->fault_ipa isn't necessarily PAGE_SIZE aligned. > > - Record guest s2 mappings even when page table maps return -EAGAIN. > kvm_pgtable_stage2_map() can create mappings while returning -EAGAIN. > For example, a 2M block map (A) could race with a 4K page map (B): > 1. (A) maps the 2M block in kvm_pgtable_visitor_cb() > 2. (B) breaks that block into a table and maps 4K > 3. (A) reloads and finds the table after kvm_pgtable_visitor_cb(), > then descends into it. > 4. (A) maps some 4K, but before it finishes reads entry mapped by (B). > 5. (A) returns -EAGAIN although it had mapped a few pages. > In this case, we don't know what subrange is mapped, just track the whole > requested mapping range. > > - Don't remove tracked mappings from the interval trees if they only > partially overlap the removal range. Because of the previous bullet point > we can have a 4K shadow mapping tracked as a 2M range in the interval > trees. It would be wrong to remove the 2M range when a guest TLBI doesn't > touch the 4K mapped. > > - Check mmu->pgt during mmu unmap notifier, as it could race against > MMU teardown. > > - Make guest_s2_tracking_destroy() canonical mmu only. We simply don't > need to detach the nodes from the nested mmus' trees. Freeing them > during canonical mmu's teardown is enough. > > Thanks! > > [1]: https://lore.kernel.org/kvmarm/86h5jv7qrd.wl-maz@kernel.org/ > [2]: https://lore.kernel.org/kvmarm/gerjpm62a2gszzggc6vuai22bf3prfquvfsp7ueumps7vz2ev3@odlmovaklg2b/ > [3]: https://lore.kernel.org/kvmarm/20260911162203.1919330-1-maz@kernel.org/ > [4]: https://lore.kernel.org/kvmarm/20260810205038.118843-1-weilin.chang@arm.com/ > > Marc Zyngier (1): > KVM: arm64: nv: Drop kvm_s2_mmu pointer from kvm_guest_s2_mapping > > Wei-Lin Chang (6): > KVM: arm64: Use a variable for the canonical IPA in kvm_s2_fault_map() > KVM: arm64: nv: Introduce guest stage-2 tracking structures > KVM: arm64: nv: Track guest stage-2 mapping creation > KVM: arm64: nv: Track guest stage-2 mapping removal > KVM: arm64: nv: Avoid full shadow stage-2 unmap > KVM: arm64: Refactor kvm_unmap_gfn_range() with common variables > > arch/arm64/include/asm/kvm_host.h | 31 +++++- > arch/arm64/include/asm/kvm_nested.h | 7 ++ > arch/arm64/kvm/mmu.c | 109 +++++++++++++++++++--- > arch/arm64/kvm/nested.c | 140 +++++++++++++++++++++++++++- > 4 files changed, 269 insertions(+), 18 deletions(-) > > -- > 2.43.0 >
On Wed, 16 Sep 2026 05:58:03 +0100, Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote: > > Hi Wei-Lin, > > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > Hi, > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > notifiers. > > > > This version fixes a few issues, and adds Marc's space optimization > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > Tested by booting L3, and running in-kernel targetted tests described > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > carry them over since some bugs fixed in v6 are non-obvious. Some > > retest would be much appreciated! > > > > Series based on v7.3-rc3 + Marc's nested mmu lifecycle fixes [3]. > > Observed on QEMU (accel tcg) only, but I saw: > > [ 1812.052752] BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:1630 > [ 1812.052896] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 596, name: shadow_stage2 > > The kernel is built on top of Marc's fix and your version six series, but > the fix for debugfs will be posted later? > > Your hello_nested and shadow_stage2 ran without an issue. > > Tested-by: Itaru Kitayama <itaru.kitayama@fujitsu.com> So you have established that there is a critical locking bug, make sure you don't communicate any information to help narrow where that bug is, and yet give your Tested-by: tag? What is the point? I'm baffled. Can you at least provide an indication of what you did to get there as well as the full backtrace? M. -- Without deviation from the norm, progress is not possible.
On Wed, Sep 16, 2026 at 08:04:53AM +0100, Marc Zyngier wrote: > On Wed, 16 Sep 2026 05:58:03 +0100, > Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote: > > > > Hi Wei-Lin, > > > > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > > Hi, > > > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > > notifiers. > > > > > > This version fixes a few issues, and adds Marc's space optimization > > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > > > Tested by booting L3, and running in-kernel targetted tests described > > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > > carry them over since some bugs fixed in v6 are non-obvious. Some > > > retest would be much appreciated! > > > > > > Series based on v7.3-rc3 + Marc's nested mmu lifecycle fixes [3]. > > > > Observed on QEMU (accel tcg) only, but I saw: > > > > [ 1812.052752] BUG: sleeping function called from invalid context at kernel/locking/rwsem.c:1630 > > [ 1812.052896] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 596, name: shadow_stage2 > > > > The kernel is built on top of Marc's fix and your version six series, but > > the fix for debugfs will be posted later? > > > > Your hello_nested and shadow_stage2 ran without an issue. > > > > Tested-by: Itaru Kitayama <itaru.kitayama@fujitsu.com> > > So you have established that there is a critical locking bug, make > sure you don't communicate any information to help narrow where that > bug is, and yet give your Tested-by: tag? > > What is the point? I'm baffled. > > Can you at least provide an indication of what you did to get there as > well as the full backtrace? It is certainly the known shadow ptdump sleeping-in-atomic problem [1] reported by Itaru, and not a problem of the reverse map series. I have the fix for that, will send it shortly. Thanks, Wei-Lin Chang [1]: https://lore.kernel.org/kvmarm/aiuF0KSvvv-ZozI1@sm-arm-grace07/ > > M. > > -- > Without deviation from the norm, progress is not possible.
On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > Hi, > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > notifiers. > > This version fixes a few issues, and adds Marc's space optimization > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > Tested by booting L3, and running in-kernel targetted tests described > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > carry them over since some bugs fixed in v6 are non-obvious. Some > retest would be much appreciated! > > Series based on v7.3-rc3 + Marc's nested mmu lifecycle fixes [3]. Looking pretty good to me, thanks Wei-Lin! Reviewed-by: Oliver Upton <oupton@kernel.org> Thanks, Oliver
On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > Hi, > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > notifiers. > > This version fixes a few issues, and adds Marc's space optimization > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > Tested by booting L3, and running in-kernel targetted tests described > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > carry them over since some bugs fixed in v6 are non-obvious. Some > retest would be much appreciated! How did you boot into L3, with KVM selftest or recrusively boot on HW? Thanks, Itaru. > > Series based on v7.3-rc3 + Marc's nested mmu lifecycle fixes [3]. > > * Changes from v5 [4]: > > - Align the addresses down to the mapping size when recording the > guest s2 mappings. s2fd->fault_ipa isn't necessarily PAGE_SIZE aligned. > > - Record guest s2 mappings even when page table maps return -EAGAIN. > kvm_pgtable_stage2_map() can create mappings while returning -EAGAIN. > For example, a 2M block map (A) could race with a 4K page map (B): > 1. (A) maps the 2M block in kvm_pgtable_visitor_cb() > 2. (B) breaks that block into a table and maps 4K > 3. (A) reloads and finds the table after kvm_pgtable_visitor_cb(), > then descends into it. > 4. (A) maps some 4K, but before it finishes reads entry mapped by (B). > 5. (A) returns -EAGAIN although it had mapped a few pages. > In this case, we don't know what subrange is mapped, just track the whole > requested mapping range. > > - Don't remove tracked mappings from the interval trees if they only > partially overlap the removal range. Because of the previous bullet point > we can have a 4K shadow mapping tracked as a 2M range in the interval > trees. It would be wrong to remove the 2M range when a guest TLBI doesn't > touch the 4K mapped. > > - Check mmu->pgt during mmu unmap notifier, as it could race against > MMU teardown. > > - Make guest_s2_tracking_destroy() canonical mmu only. We simply don't > need to detach the nodes from the nested mmus' trees. Freeing them > during canonical mmu's teardown is enough. > > Thanks! > > [1]: https://lore.kernel.org/kvmarm/86h5jv7qrd.wl-maz@kernel.org/ > [2]: https://lore.kernel.org/kvmarm/gerjpm62a2gszzggc6vuai22bf3prfquvfsp7ueumps7vz2ev3@odlmovaklg2b/ > [3]: https://lore.kernel.org/kvmarm/20260911162203.1919330-1-maz@kernel.org/ > [4]: https://lore.kernel.org/kvmarm/20260810205038.118843-1-weilin.chang@arm.com/ > > Marc Zyngier (1): > KVM: arm64: nv: Drop kvm_s2_mmu pointer from kvm_guest_s2_mapping > > Wei-Lin Chang (6): > KVM: arm64: Use a variable for the canonical IPA in kvm_s2_fault_map() > KVM: arm64: nv: Introduce guest stage-2 tracking structures > KVM: arm64: nv: Track guest stage-2 mapping creation > KVM: arm64: nv: Track guest stage-2 mapping removal > KVM: arm64: nv: Avoid full shadow stage-2 unmap > KVM: arm64: Refactor kvm_unmap_gfn_range() with common variables > > arch/arm64/include/asm/kvm_host.h | 31 +++++- > arch/arm64/include/asm/kvm_nested.h | 7 ++ > arch/arm64/kvm/mmu.c | 109 +++++++++++++++++++--- > arch/arm64/kvm/nested.c | 140 +++++++++++++++++++++++++++- > 4 files changed, 269 insertions(+), 18 deletions(-) > > -- > 2.43.0 >
On Wed, Sep 16, 2026 at 06:49:57AM +0900, Itaru Kitayama wrote: > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > Hi, > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > notifiers. > > > > This version fixes a few issues, and adds Marc's space optimization > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > Tested by booting L3, and running in-kernel targetted tests described > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > carry them over since some bugs fixed in v6 are non-obvious. Some > > retest would be much appreciated! > > How did you boot into L3, with KVM selftest or recrusively boot on HW? I just recrusively booted VMs on hardware. :) Thanks, Wei-Lin Chang > > Thanks, > Itaru. > [...]
On Wed, Sep 16, 2026 at 12:22:30AM +0100, Wei-Lin Chang wrote: > On Wed, Sep 16, 2026 at 06:49:57AM +0900, Itaru Kitayama wrote: > > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > > Hi, > > > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > > notifiers. > > > > > > This version fixes a few issues, and adds Marc's space optimization > > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > > > Tested by booting L3, and running in-kernel targetted tests described > > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > > carry them over since some bugs fixed in v6 are non-obvious. Some > > > retest would be much appreciated! > > > > How did you boot into L3, with KVM selftest or recrusively boot on HW? > > I just recrusively booted VMs on hardware. :) Good. Then I actually tried that on Grace CPUs, but letting L1 boot with nested mode did not print "VHE+NV2" KVM init log in dmesg, thus not able to boot L3. Do I need to add other than kvm.arm-mode=nested to command line? Thanks, Itaru. > > Thanks, > Wei-Lin Chang > > > > > Thanks, > > Itaru. > > > > [...] >
On Wed, 16 Sep 2026 00:27:22 +0100, Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote: > > On Wed, Sep 16, 2026 at 12:22:30AM +0100, Wei-Lin Chang wrote: > > On Wed, Sep 16, 2026 at 06:49:57AM +0900, Itaru Kitayama wrote: > > > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > > > Hi, > > > > > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > > > notifiers. > > > > > > > > This version fixes a few issues, and adds Marc's space optimization > > > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > > > > > Tested by booting L3, and running in-kernel targetted tests described > > > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > > > carry them over since some bugs fixed in v6 are non-obvious. Some > > > > retest would be much appreciated! > > > > > > How did you boot into L3, with KVM selftest or recrusively boot on HW? > > > > I just recrusively booted VMs on hardware. :) > > Good. Then I actually tried that on Grace CPUs, but letting L1 boot > with nested mode did not print "VHE+NV2" KVM init log in dmesg, thus not > able to boot L3. Do I need to add other than kvm.arm-mode=nested to command line? kvm-arm.mode=nested, as documented. That also puts in perspective your TB tag on the whole series... M. -- Without deviation from the norm, progress is not possible.
On Wed, Sep 16, 2026 at 08:08:20AM +0100, Marc Zyngier wrote: > On Wed, 16 Sep 2026 00:27:22 +0100, > Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote: > > > > On Wed, Sep 16, 2026 at 12:22:30AM +0100, Wei-Lin Chang wrote: > > > On Wed, Sep 16, 2026 at 06:49:57AM +0900, Itaru Kitayama wrote: > > > > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > > > > Hi, > > > > > > > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > > > > notifiers. > > > > > > > > > > This version fixes a few issues, and adds Marc's space optimization > > > > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > > > > > > > Tested by booting L3, and running in-kernel targetted tests described > > > > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > > > > carry them over since some bugs fixed in v6 are non-obvious. Some > > > > > retest would be much appreciated! > > > > > > > > How did you boot into L3, with KVM selftest or recrusively boot on HW? > > > > > > I just recrusively booted VMs on hardware. :) > > > > Good. Then I actually tried that on Grace CPUs, but letting L1 boot > > with nested mode did not print "VHE+NV2" KVM init log in dmesg, thus not > > able to boot L3. Do I need to add other than kvm.arm-mode=nested to command line? > > kvm-arm.mode=nested, as documented. That also puts in perspective your > TB tag on the whole series... Yes. Using kvmtool's --nested option did not bring vCPU with the VHE+NV2 boot log. The Host kernel is Wei-Lin's version 6 booted on Grace CPUs, and the guest kernel is today's kvmarm/fixes plus Wei-Lin's shadow stage 2 ptdump fix. Thanks, Itaru. > > M. > > > -- > Without deviation from the norm, progress is not possible.
On Thu, Sep 17, 2026 at 03:51:19PM +0900, Itaru Kitayama wrote: > On Wed, Sep 16, 2026 at 08:08:20AM +0100, Marc Zyngier wrote: > > On Wed, 16 Sep 2026 00:27:22 +0100, > > Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote: > > > > > > On Wed, Sep 16, 2026 at 12:22:30AM +0100, Wei-Lin Chang wrote: > > > > On Wed, Sep 16, 2026 at 06:49:57AM +0900, Itaru Kitayama wrote: > > > > > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > > > > > Hi, > > > > > > > > > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > > > > > notifiers. > > > > > > > > > > > > This version fixes a few issues, and adds Marc's space optimization > > > > > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > > > > > > > > > Tested by booting L3, and running in-kernel targetted tests described > > > > > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > > > > > carry them over since some bugs fixed in v6 are non-obvious. Some > > > > > > retest would be much appreciated! > > > > > > > > > > How did you boot into L3, with KVM selftest or recrusively boot on HW? > > > > > > > > I just recrusively booted VMs on hardware. :) > > > > > > Good. Then I actually tried that on Grace CPUs, but letting L1 boot > > > with nested mode did not print "VHE+NV2" KVM init log in dmesg, thus not > > > able to boot L3. Do I need to add other than kvm.arm-mode=nested to command line? > > > > kvm-arm.mode=nested, as documented. That also puts in perspective your > > TB tag on the whole series... > > Yes. Using kvmtool's --nested option did not bring vCPU with the VHE+NV2 boot > log. The Host kernel is Wei-Lin's version 6 booted on Grace CPUs, and > the guest kernel is today's kvmarm/fixes plus Wei-Lin's shadow stage 2 ptdump fix. I'm not sure what you did exactly, but just to be clear, you need to pass kvm-arm.mode=nested to L1's command line as well, in addition to kvmtool's --nested option. ./lkvm run --nested -p kvm-arm.mode=nested <...> Thanks, Wei-Lin Chang > > Thanks, > Itaru. > > > > > M. > > > > > > -- > > Without deviation from the norm, progress is not possible.
On Thu, 17 Sep 2026 07:51:19 +0100, Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote: > > On Wed, Sep 16, 2026 at 08:08:20AM +0100, Marc Zyngier wrote: > > On Wed, 16 Sep 2026 00:27:22 +0100, > > Itaru Kitayama <itaru.kitayama@fujitsu.com> wrote: > > > > > > On Wed, Sep 16, 2026 at 12:22:30AM +0100, Wei-Lin Chang wrote: > > > > On Wed, Sep 16, 2026 at 06:49:57AM +0900, Itaru Kitayama wrote: > > > > > On Tue, Sep 15, 2026 at 04:42:58PM +0100, Wei-Lin Chang wrote: > > > > > > Hi, > > > > > > > > > > > > This is v6 of optimizing the shadow s2 mmu unmapping during MMU > > > > > > notifiers. > > > > > > > > > > > > This version fixes a few issues, and adds Marc's space optimization > > > > > > for kvm_guest_s2_mapping [1]. Please see the changelog for the details. > > > > > > > > > > > > Tested by booting L3, and running in-kernel targetted tests described > > > > > > in [2]. v5 got some tested-by's from Itaru and Wang (thanks), I didn't > > > > > > carry them over since some bugs fixed in v6 are non-obvious. Some > > > > > > retest would be much appreciated! > > > > > > > > > > How did you boot into L3, with KVM selftest or recrusively boot on HW? > > > > > > > > I just recrusively booted VMs on hardware. :) > > > > > > Good. Then I actually tried that on Grace CPUs, but letting L1 boot > > > with nested mode did not print "VHE+NV2" KVM init log in dmesg, thus not > > > able to boot L3. Do I need to add other than kvm.arm-mode=nested to command line? > > > > kvm-arm.mode=nested, as documented. That also puts in perspective your > > TB tag on the whole series... > > Yes. Using kvmtool's --nested option did not bring vCPU with the VHE+NV2 boot > log. The Host kernel is Wei-Lin's version 6 booted on Grace CPUs, and > the guest kernel is today's kvmarm/fixes plus Wei-Lin's shadow stage 2 ptdump fix. What have you tested exactly? If you're not able to boot a working L1 guest as a NV2 hypervisor, you're probably doing something wrong. M. -- Without deviation from the norm, progress is not possible.
On Thu, Sep 17, 2026 at 08:56:56AM +0100, Marc Zyngier wrote: > What have you tested exactly? If you're not able to boot a working L1 > guest as a NV2 hypervisor, you're probably doing something wrong. Can you just stop being dismissive, if you can? Thanks, Itaru. > > M. > > -- > Without deviation from the norm, progress is not possible.
© 2016 - 2026 Red Hat, Inc.