This patch series makes the selftest work with NV enabled. The guest code
is run in vEL2 instead of EL1. We add a command line option to enable
testing of NV. The NV tests are disabled by default.

Modified around 12 selftests in this series.

Changes since v1:
	- Updated NV helper functions as per comments [1].
	- Modified existing testscases to run guest code in vEL2.

[1] https://lkml.iu.edu/hypermail/linux/kernel/2502.0/07001.html 

Ganapatrao Kulkarni (9):
  KVM: arm64: nv: selftests: Add support to run guest code in vEL2.
  KVM: arm64: nv: selftests: Add simple test to run guest code in vEL2
  KVM: arm64: nv: selftests: Enable hypervisor timer tests to run in
    vEL2
  KVM: arm64: nv: selftests: enable aarch32_id_regs test to run in vEL2
  KVM: arm64: nv: selftests: Enable vgic tests to run in vEL2
  KVM: arm64: nv: selftests: Enable set_id_regs test to run in vEL2
  KVM: arm64: nv: selftests: Enable test to run in vEL2
  KVM: selftests: arm64: Extend kvm_page_table_test to run guest code in
    vEL2
  KVM: arm64: nv: selftests: Enable page_fault_test test to run in vEL2

 tools/testing/selftests/kvm/Makefile.kvm      |   2 +
 tools/testing/selftests/kvm/arch_timer.c      |   8 +-
 .../selftests/kvm/arm64/aarch32_id_regs.c     |  34 ++++-
 .../testing/selftests/kvm/arm64/arch_timer.c  | 118 +++++++++++++++---
 .../selftests/kvm/arm64/nv_guest_hypervisor.c |  68 ++++++++++
 .../selftests/kvm/arm64/page_fault_test.c     |  35 +++++-
 .../testing/selftests/kvm/arm64/set_id_regs.c |  57 ++++++++-
 tools/testing/selftests/kvm/arm64/vgic_init.c |  54 +++++++-
 tools/testing/selftests/kvm/arm64/vgic_irq.c  |  27 ++--
 .../selftests/kvm/arm64/vgic_lpi_stress.c     |  19 ++-
 .../testing/selftests/kvm/guest_print_test.c  |  32 +++++
 .../selftests/kvm/include/arm64/arch_timer.h  |  16 +++
 .../kvm/include/arm64/kvm_util_arch.h         |   3 +
 .../selftests/kvm/include/arm64/nv_util.h     |  45 +++++++
 .../selftests/kvm/include/arm64/vgic.h        |   1 +
 .../testing/selftests/kvm/include/kvm_util.h  |   3 +
 .../selftests/kvm/include/timer_test.h        |   1 +
 .../selftests/kvm/kvm_page_table_test.c       |  30 ++++-
 tools/testing/selftests/kvm/lib/arm64/nv.c    |  46 +++++++
 .../selftests/kvm/lib/arm64/processor.c       |  61 ++++++---
 tools/testing/selftests/kvm/lib/arm64/vgic.c  |   8 ++
 21 files changed, 604 insertions(+), 64 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/arm64/nv_guest_hypervisor.c
 create mode 100644 tools/testing/selftests/kvm/include/arm64/nv_util.h
 create mode 100644 tools/testing/selftests/kvm/lib/arm64/nv.c

-- 
2.48.1

Hi Ganapat,

On Mon, May 12, 2025 at 03:52:42AM -0700, Ganapatrao Kulkarni wrote:
> This patch series makes the selftest work with NV enabled. The guest code
> is run in vEL2 instead of EL1. We add a command line option to enable
> testing of NV. The NV tests are disabled by default.
> 
> Modified around 12 selftests in this series.

Thanks for sharing this, we sorely need testing for NV.

I haven't looked at these patches thoroughly but I have some overall
feedback. What I'd like to see is that we force *all* KVM selftests
to run in VHE EL2 without an opt-in/opt-out. The user had to boot their
kernel with kvm-arm.mode=nested, so imposing NV testing on them feels
reasonable.

Thanks,
Oliver

Hi Ganapatrao,

> On 12 May 2025, at 10:52, Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> 
> This patch series makes the selftest work with NV enabled. The guest code
> is run in vEL2 instead of EL1. We add a command line option to enable
> testing of NV. The NV tests are disabled by default.
> 

The following two tests arch_timer and vgic_lpi_stress pass for a guest in VHE
mode but in a nVHE mode guest they are failing for me. I’ve tested them on
Marc’s repo tag 'kvmarm-fixes-6.16-1' on an AmpereOne.

Do you have plans to add nvhe mode testing to this series?

Thanks
Miguel

> Modified around 12 selftests in this series.
> 
> Changes since v1:
> - Updated NV helper functions as per comments [1].
> - Modified existing testscases to run guest code in vEL2.
> 
> [1] https://lkml.iu.edu/hypermail/linux/kernel/2502.0/07001.html 
> 
> Ganapatrao Kulkarni (9):
>  KVM: arm64: nv: selftests: Add support to run guest code in vEL2.
>  KVM: arm64: nv: selftests: Add simple test to run guest code in vEL2
>  KVM: arm64: nv: selftests: Enable hypervisor timer tests to run in
>    vEL2
>  KVM: arm64: nv: selftests: enable aarch32_id_regs test to run in vEL2
>  KVM: arm64: nv: selftests: Enable vgic tests to run in vEL2
>  KVM: arm64: nv: selftests: Enable set_id_regs test to run in vEL2
>  KVM: arm64: nv: selftests: Enable test to run in vEL2
>  KVM: selftests: arm64: Extend kvm_page_table_test to run guest code in
>    vEL2
>  KVM: arm64: nv: selftests: Enable page_fault_test test to run in vEL2
> 
> tools/testing/selftests/kvm/Makefile.kvm      |   2 +
> tools/testing/selftests/kvm/arch_timer.c      |   8 +-
> .../selftests/kvm/arm64/aarch32_id_regs.c     |  34 ++++-
> .../testing/selftests/kvm/arm64/arch_timer.c  | 118 +++++++++++++++---
> .../selftests/kvm/arm64/nv_guest_hypervisor.c |  68 ++++++++++
> .../selftests/kvm/arm64/page_fault_test.c     |  35 +++++-
> .../testing/selftests/kvm/arm64/set_id_regs.c |  57 ++++++++-
> tools/testing/selftests/kvm/arm64/vgic_init.c |  54 +++++++-
> tools/testing/selftests/kvm/arm64/vgic_irq.c  |  27 ++--
> .../selftests/kvm/arm64/vgic_lpi_stress.c     |  19 ++-
> .../testing/selftests/kvm/guest_print_test.c  |  32 +++++
> .../selftests/kvm/include/arm64/arch_timer.h  |  16 +++
> .../kvm/include/arm64/kvm_util_arch.h         |   3 +
> .../selftests/kvm/include/arm64/nv_util.h     |  45 +++++++
> .../selftests/kvm/include/arm64/vgic.h        |   1 +
> .../testing/selftests/kvm/include/kvm_util.h  |   3 +
> .../selftests/kvm/include/timer_test.h        |   1 +
> .../selftests/kvm/kvm_page_table_test.c       |  30 ++++-
> tools/testing/selftests/kvm/lib/arm64/nv.c    |  46 +++++++
> .../selftests/kvm/lib/arm64/processor.c       |  61 ++++++---
> tools/testing/selftests/kvm/lib/arm64/vgic.c  |   8 ++
> 21 files changed, 604 insertions(+), 64 deletions(-)
> create mode 100644 tools/testing/selftests/kvm/arm64/nv_guest_hypervisor.c
> create mode 100644 tools/testing/selftests/kvm/include/arm64/nv_util.h
> create mode 100644 tools/testing/selftests/kvm/lib/arm64/nv.c
> 
> -- 
> 2.48.1
> 
> 

Hi Ganapatrao,

On 5/12/25 12:52 PM, Ganapatrao Kulkarni wrote:
> This patch series makes the selftest work with NV enabled. The guest code
> is run in vEL2 instead of EL1. We add a command line option to enable
> testing of NV. The NV tests are disabled by default.

For commodity, I would add in the coverletter that for all tests
enhanced with vEL2 testing "-g 1" option shall be added to force that mode.

I don't really get how you chose tests capable to run at vEL2 and
excluded others? Wouldn't it make sense to have a way to run all tests
in either mode?

Thanks

Eric

> 
> Modified around 12 selftests in this series.
> 
> Changes since v1:
> 	- Updated NV helper functions as per comments [1].
> 	- Modified existing testscases to run guest code in vEL2.
> 
> [1] https://lkml.iu.edu/hypermail/linux/kernel/2502.0/07001.html 
> 
> Ganapatrao Kulkarni (9):
>   KVM: arm64: nv: selftests: Add support to run guest code in vEL2.
>   KVM: arm64: nv: selftests: Add simple test to run guest code in vEL2
>   KVM: arm64: nv: selftests: Enable hypervisor timer tests to run in
>     vEL2
>   KVM: arm64: nv: selftests: enable aarch32_id_regs test to run in vEL2
>   KVM: arm64: nv: selftests: Enable vgic tests to run in vEL2
>   KVM: arm64: nv: selftests: Enable set_id_regs test to run in vEL2
>   KVM: arm64: nv: selftests: Enable test to run in vEL2
>   KVM: selftests: arm64: Extend kvm_page_table_test to run guest code in
>     vEL2
>   KVM: arm64: nv: selftests: Enable page_fault_test test to run in vEL2
> 
>  tools/testing/selftests/kvm/Makefile.kvm      |   2 +
>  tools/testing/selftests/kvm/arch_timer.c      |   8 +-
>  .../selftests/kvm/arm64/aarch32_id_regs.c     |  34 ++++-
>  .../testing/selftests/kvm/arm64/arch_timer.c  | 118 +++++++++++++++---
>  .../selftests/kvm/arm64/nv_guest_hypervisor.c |  68 ++++++++++
>  .../selftests/kvm/arm64/page_fault_test.c     |  35 +++++-
>  .../testing/selftests/kvm/arm64/set_id_regs.c |  57 ++++++++-
>  tools/testing/selftests/kvm/arm64/vgic_init.c |  54 +++++++-
>  tools/testing/selftests/kvm/arm64/vgic_irq.c  |  27 ++--
>  .../selftests/kvm/arm64/vgic_lpi_stress.c     |  19 ++-
>  .../testing/selftests/kvm/guest_print_test.c  |  32 +++++
>  .../selftests/kvm/include/arm64/arch_timer.h  |  16 +++
>  .../kvm/include/arm64/kvm_util_arch.h         |   3 +
>  .../selftests/kvm/include/arm64/nv_util.h     |  45 +++++++
>  .../selftests/kvm/include/arm64/vgic.h        |   1 +
>  .../testing/selftests/kvm/include/kvm_util.h  |   3 +
>  .../selftests/kvm/include/timer_test.h        |   1 +
>  .../selftests/kvm/kvm_page_table_test.c       |  30 ++++-
>  tools/testing/selftests/kvm/lib/arm64/nv.c    |  46 +++++++
>  .../selftests/kvm/lib/arm64/processor.c       |  61 ++++++---
>  tools/testing/selftests/kvm/lib/arm64/vgic.c  |   8 ++
>  21 files changed, 604 insertions(+), 64 deletions(-)
>  create mode 100644 tools/testing/selftests/kvm/arm64/nv_guest_hypervisor.c
>  create mode 100644 tools/testing/selftests/kvm/include/arm64/nv_util.h
>  create mode 100644 tools/testing/selftests/kvm/lib/arm64/nv.c
>

Hi Eric,

On 5/28/2025 6:58 PM, Eric Auger wrote:
> Hi Ganapatrao,
> 
> On 5/12/25 12:52 PM, Ganapatrao Kulkarni wrote:
>> This patch series makes the selftest work with NV enabled. The guest code
>> is run in vEL2 instead of EL1. We add a command line option to enable
>> testing of NV. The NV tests are disabled by default.
> 
> For commodity, I would add in the coverletter that for all tests
> enhanced with vEL2 testing "-g 1" option shall be added to force that mode.

Sure, will do.

> 
> I don't really get how you chose tests capable to run at vEL2 and
> excluded others? Wouldn't it make sense to have a way to run all tests
> in either mode?
There is no selection as such. I have worked on around 50% of the tests and sent for the early review.
Yes, almost all tests can/should run in vEL2 except few.

-- 
Thanks,
Gk

On Thu, 29 May 2025 11:29:58 +0100,
Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> 
> 
> Hi Eric,
> 
> On 5/28/2025 6:58 PM, Eric Auger wrote:
> > Hi Ganapatrao,
> > 
> > On 5/12/25 12:52 PM, Ganapatrao Kulkarni wrote:
> >> This patch series makes the selftest work with NV enabled. The guest code
> >> is run in vEL2 instead of EL1. We add a command line option to enable
> >> testing of NV. The NV tests are disabled by default.
> > 
> > For commodity, I would add in the coverletter that for all tests
> > enhanced with vEL2 testing "-g 1" option shall be added to force that mode.
> 
> Sure, will do.
> 
> > 
> > I don't really get how you chose tests capable to run at vEL2 and
> > excluded others? Wouldn't it make sense to have a way to run all tests
> > in either mode?
> There is no selection as such. I have worked on around 50% of the tests and sent for the early review.
> Yes, almost all tests can/should run in vEL2 except few.

Define EL2. You are so far assuming a E2H RES1 guest, and I don't see
anything that is even trying E2H RES0. After all the complaining that
E2H0 wasn't initially supported, this is a bit... disappointing.

Also, running EL2 is the least of our worries, because that's pretty
easy to deal with. It is running at EL1/0 when EL2 is present that is
interesting, and I see no coverage on that front.

	M.

-- 
Without deviation from the norm, progress is not possible.

[Sorry for late reply]

On 5/29/2025 5:18 PM, Marc Zyngier wrote:
> On Thu, 29 May 2025 11:29:58 +0100,
> Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
>>
>>
>> Hi Eric,
>>
>> On 5/28/2025 6:58 PM, Eric Auger wrote:
>>> Hi Ganapatrao,
>>>
>>> On 5/12/25 12:52 PM, Ganapatrao Kulkarni wrote:
>>>> This patch series makes the selftest work with NV enabled. The guest code
>>>> is run in vEL2 instead of EL1. We add a command line option to enable
>>>> testing of NV. The NV tests are disabled by default.
>>>
>>> For commodity, I would add in the coverletter that for all tests
>>> enhanced with vEL2 testing "-g 1" option shall be added to force that mode.
>>
>> Sure, will do.
>>
>>>
>>> I don't really get how you chose tests capable to run at vEL2 and
>>> excluded others? Wouldn't it make sense to have a way to run all tests
>>> in either mode?
>> There is no selection as such. I have worked on around 50% of the tests and sent for the early review.
>> Yes, almost all tests can/should run in vEL2 except few.
> 
> Define EL2. You are so far assuming a E2H RES1 guest, and I don't see
> anything that is even trying E2H RES0. After all the complaining that

Sure, I will mention that default test code runs in EL2 with E2H enabled.
The tests code is in vEL2 with E2H RES1 by default, since host is booted with VHE.

> E2H0 wasn't initially supported, this is a bit... disappointing.
IIRC, I was mentioning about L1 switching between arch mmu table and guest mmu table(VMID 0).
I don't remember why It was switching.
  > 
> Also, running EL2 is the least of our worries, because that's pretty
> easy to deal with. It is running at EL1/0 when EL2 is present that is
> interesting, and I see no coverage on that front.

Sorry, I did not get this comment fully.
When we run selftest on Host with -g option, the guest code will run in vEL2 as L1.
This is implemented as per comment in V1.

When we run same selftest from L1 shell, then guest_code will be running in EL0/1 like running from L0.

-- 
Thanks,
Gk

On Thu, 19 Jun 2025 10:40:15 +0100,
Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> 
> 
> [Sorry for late reply]
> 
> On 5/29/2025 5:18 PM, Marc Zyngier wrote:
> > On Thu, 29 May 2025 11:29:58 +0100,
> > Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> >> 
> >> 
> >> Hi Eric,
> >> 
> >> On 5/28/2025 6:58 PM, Eric Auger wrote:
> >>> Hi Ganapatrao,
> >>> 
> >>> On 5/12/25 12:52 PM, Ganapatrao Kulkarni wrote:
> >>>> This patch series makes the selftest work with NV enabled. The guest code
> >>>> is run in vEL2 instead of EL1. We add a command line option to enable
> >>>> testing of NV. The NV tests are disabled by default.
> >>> 
> >>> For commodity, I would add in the coverletter that for all tests
> >>> enhanced with vEL2 testing "-g 1" option shall be added to force that mode.
> >> 
> >> Sure, will do.
> >> 
> >>> 
> >>> I don't really get how you chose tests capable to run at vEL2 and
> >>> excluded others? Wouldn't it make sense to have a way to run all tests
> >>> in either mode?
> >> There is no selection as such. I have worked on around 50% of the tests and sent for the early review.
> >> Yes, almost all tests can/should run in vEL2 except few.
> > 
> > Define EL2. You are so far assuming a E2H RES1 guest, and I don't see
> > anything that is even trying E2H RES0. After all the complaining that
> 
> Sure, I will mention that default test code runs in EL2 with E2H enabled.
> The tests code is in vEL2 with E2H RES1 by default, since host is booted with VHE.

Sight... You are still confused with what KVM does. With NV, the host
is *always* VHE. The guest can be VHE (E2H RES1) or nVHE (E2H RES0).

> > E2H0 wasn't initially supported, this is a bit... disappointing.
> IIRC, I was mentioning about L1 switching between arch mmu table and guest mmu table(VMID 0).
> I don't remember why It was switching.

-ENOPARSE.

>  > 
> > Also, running EL2 is the least of our worries, because that's pretty
> > easy to deal with. It is running at EL1/0 when EL2 is present that is
> > interesting, and I see no coverage on that front.
> 
> Sorry, I did not get this comment fully.
> When we run selftest on Host with -g option, the guest code will run in vEL2 as L1.
> This is implemented as per comment in V1.
> 
> When we run same selftest from L1 shell, then guest_code will be running in EL0/1 like running from L0.

What good does this bring us if we need to boot a full guest OS to run
tests? What we need is synthetic tests that implement the whole stack:

- L1 guest hypervisor
- L2 guest hypervisor
- L2 guest
- L3 guest hypervisor
- L3 guest
- [...]

This is *one* test. Not a test that runs in a guest. That's what I've
been asking since day one.

	M.

-- 
Without deviation from the norm, progress is not possible.

On 6/19/2025 5:15 PM, Marc Zyngier wrote:
>>   >
>>> Also, running EL2 is the least of our worries, because that's pretty
>>> easy to deal with. It is running at EL1/0 when EL2 is present that is
>>> interesting, and I see no coverage on that front.
>>
>> Sorry, I did not get this comment fully.
>> When we run selftest on Host with -g option, the guest code will run in vEL2 as L1.
>> This is implemented as per comment in V1.
>>
>> When we run same selftest from L1 shell, then guest_code will be running in EL0/1 like running from L0.
> 
> What good does this bring us if we need to boot a full guest OS to run
> tests? What we need is synthetic tests that implement the whole stack:
> 
> - L1 guest hypervisor
> - L2 guest hypervisor
> - L2 guest
> - L3 guest hypervisor
> - L3 guest
> - [...]

IIUC, selftest leverages host OS support and uses various IOCTLs to support the guest_code run. Are you saying to implement all this again (without OS help) in guest_code to run it as hypervisor and launch guest_code2 as NestedVM?. It seems to be complicated, doesn't it?

> 
> This is *one* test. Not a test that runs in a guest. That's what I've
> been asking since day one.
-- 
Thanks,
Gk

On Mon, 23 Jun 2025 11:31:32 +0100,
Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> 
> On 6/19/2025 5:15 PM, Marc Zyngier wrote:
> >>   >
> >>> Also, running EL2 is the least of our worries, because that's pretty
> >>> easy to deal with. It is running at EL1/0 when EL2 is present that is
> >>> interesting, and I see no coverage on that front.
> >> 
> >> Sorry, I did not get this comment fully.
> >> When we run selftest on Host with -g option, the guest code will run in vEL2 as L1.
> >> This is implemented as per comment in V1.
> >> 
> >> When we run same selftest from L1 shell, then guest_code will be running in EL0/1 like running from L0.
> > 
> > What good does this bring us if we need to boot a full guest OS to run
> > tests? What we need is synthetic tests that implement the whole stack:
> > 
> > - L1 guest hypervisor
> > - L2 guest hypervisor
> > - L2 guest
> > - L3 guest hypervisor
> > - L3 guest
> > - [...]
> 
> IIUC, selftest leverages host OS support and uses various IOCTLs to
> support the guest_code run. Are you saying to implement all this
> again (without OS help) in guest_code to run it as hypervisor and
> launch guest_code2 as NestedVM?.

The whole point of having small selftests is to run something that is
simpler several orders of magnitude simpler than the full blown
OS/hypervisor. So indeed, I'm asking for selftests that build chains
of guests up to some level and verify that the nesting, as described
in the architecture, works correctly.

> It seems to be complicated, doesn't it?

Yes, it is complicated. What did you expect?

	M.

-- 
Without deviation from the norm, progress is not possible.

Hi Marc,

On 6/23/2025 7:41 PM, Marc Zyngier wrote:
> On Mon, 23 Jun 2025 11:31:32 +0100,
> Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
>>
>> On 6/19/2025 5:15 PM, Marc Zyngier wrote:
>>>>    >
>>>>> Also, running EL2 is the least of our worries, because that's pretty
>>>>> easy to deal with. It is running at EL1/0 when EL2 is present that is
>>>>> interesting, and I see no coverage on that front.
>>>>
>>>> Sorry, I did not get this comment fully.
>>>> When we run selftest on Host with -g option, the guest code will run in vEL2 as L1.
>>>> This is implemented as per comment in V1.
>>>>
>>>> When we run same selftest from L1 shell, then guest_code will be running in EL0/1 like running from L0.
>>>
>>> What good does this bring us if we need to boot a full guest OS to run
>>> tests? What we need is synthetic tests that implement the whole stack:
>>>
>>> - L1 guest hypervisor
>>> - L2 guest hypervisor
>>> - L2 guest
>>> - L3 guest hypervisor
>>> - L3 guest
>>> - [...]
>>
>> IIUC, selftest leverages host OS support and uses various IOCTLs to
>> support the guest_code run. Are you saying to implement all this
>> again (without OS help) in guest_code to run it as hypervisor and
>> launch guest_code2 as NestedVM?.
> 
> The whole point of having small selftests is to run something that is
> simpler several orders of magnitude simpler than the full blown
> OS/hypervisor. So indeed, I'm asking for selftests that build chains
> of guests up to some level and verify that the nesting, as described
> in the architecture, works correctly.
> 

Do you see value in the patches as they are, without the changes to support the bare-metal hypervisor in guest, or will you only consider them if they are first reworked to support the recursive guests?

-- 
Thanks,
Gk

On Fri, 25 Jul 2025 11:01:05 +0100,
Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> 
> 
> Hi Marc,
> 
> On 6/23/2025 7:41 PM, Marc Zyngier wrote:
> > On Mon, 23 Jun 2025 11:31:32 +0100,
> > Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
> >> 
> >> On 6/19/2025 5:15 PM, Marc Zyngier wrote:
> >>>>    >
> >>>>> Also, running EL2 is the least of our worries, because that's pretty
> >>>>> easy to deal with. It is running at EL1/0 when EL2 is present that is
> >>>>> interesting, and I see no coverage on that front.
> >>>> 
> >>>> Sorry, I did not get this comment fully.
> >>>> When we run selftest on Host with -g option, the guest code will run in vEL2 as L1.
> >>>> This is implemented as per comment in V1.
> >>>> 
> >>>> When we run same selftest from L1 shell, then guest_code will be running in EL0/1 like running from L0.
> >>> 
> >>> What good does this bring us if we need to boot a full guest OS to run
> >>> tests? What we need is synthetic tests that implement the whole stack:
> >>> 
> >>> - L1 guest hypervisor
> >>> - L2 guest hypervisor
> >>> - L2 guest
> >>> - L3 guest hypervisor
> >>> - L3 guest
> >>> - [...]
> >> 
> >> IIUC, selftest leverages host OS support and uses various IOCTLs to
> >> support the guest_code run. Are you saying to implement all this
> >> again (without OS help) in guest_code to run it as hypervisor and
> >> launch guest_code2 as NestedVM?.
> > 
> > The whole point of having small selftests is to run something that is
> > simpler several orders of magnitude simpler than the full blown
> > OS/hypervisor. So indeed, I'm asking for selftests that build chains
> > of guests up to some level and verify that the nesting, as described
> > in the architecture, works correctly.
> > 
> 
> Do you see value in the patches as they are, without the changes to
> support the bare-metal hypervisor in guest or will you only consider
> them if they are first reworked to support the recursive guests?

Running existing tests at EL2 may an interesting goal, but the way
you've gone about it is really wrong. These tests are not about
"nested", they are about running at EL2. So getting rid of all the
"nested" nonsense in patch #1 and focusing on the *two* flavours of
EL2 would be a good start.

Also, EL2 tests shouldn't be optional. If EL2 is available, the test
should run, without any option, and things like

+	if (is_nested)
+		vm = nv_vm_create_with_vcpus_gic(1, &vcpu, &gic_fd, guest_main);
+	else
+		vm = vm_create_with_one_vcpu(&vcpu, guest_main);

are just non-starters. You just need to either get rid of the whole
vm_create_with_one_vcpu() nonsense, or teach it to take an execution
context.

But the real NV tests are not optional either. They need to happen,
and I don't have the confidence that they will if I agree to what you
are suggesting. Because it's been months since I asked for these
things, and not much has happened in the interval.

	M.

-- 
Without deviation from the norm, progress is not possible.