Now we have virtualization support in the GICv2 emulation,
I thought I'd have a look at how much we were still missing
for being able to enable EL2 support for AArch32.
This set of patches fixes some minor missing pieces:
 * various small bugs in cp15 registers or places where
   we were missing the 32-bit version of a 64-bit register
 * a bugfix for MSR/MRS (banked), which were not allowing
   Hyp mode to access ELR_Hyp
 * implementation of the ERET instruction for A32/T32
 * support for taking exceptions to Hyp mode (the largest
   of these missing bits)

This isn't complete, but I thought I'd push these patches
out for review. My test setup is that I have another
couple of patches, one which fixes up hw/arm/boot.c to
boot AArch32 kernels in Hyp mode if it exists, and one
which sets ARM_FEATURE_EL2 on our A15 model. With those I
can get an outer kernel to boot with KVM support and try
to run an inner guest kernel. The inner kernel boots OK
but gets random segfaults in its userspace -- I haven't
tracked down why this is yet...

Some bits that are definitely missing:
 * ATS1HR, ATS1HW address translation ops
 * I need to check that the trap semantics for AArch32
   regs line up with their AArch64 counterparts

I also noticed that we fail to implement really quite a lot
of the HCR_EL2 trap semantics for either AArch64 or AArch32,
to the extent that I'm surprised that nested guests work
under AArch64 :-)

This patchset is based on top of my target-arm.for-3.1
branch.

thanks
-- PMM

Peter Maydell (10):
  target/arm: Correct typo in HAMAIR1 regdef name
  target/arm: Add missing .cp = 15 to HMAIR1 and HAMAIR1 regdefs
  target/arm: Implement RAZ/WI HACTLR2
  target/arm: Implement AArch32 HVBAR
  target/arm: Implement AArch32 HCR and HCR2
  target/arm: Implement AArch32 Hyp FARs
  target/arm: Implement ESR_EL2/HSR for AArch32 and no-EL2
  target/arm: Permit accesses to ELR_Hyp from Hyp mode via MSR/MRS
    (banked)
  target/arm: Implement AArch32 ERET instruction
  target/arm: Implement support for taking exceptions to Hyp mode

 target/arm/helper.c    | 226 ++++++++++++++++++++++++++++++++++-------
 target/arm/op_helper.c |  22 ++--
 target/arm/translate.c |  41 +++++++-
 3 files changed, 236 insertions(+), 53 deletions(-)

-- 
2.18.0

On Tue, Aug 14, 2018 at 01:42:44PM +0100, Peter Maydell wrote:
> Now we have virtualization support in the GICv2 emulation,
> I thought I'd have a look at how much we were still missing
> for being able to enable EL2 support for AArch32.
> This set of patches fixes some minor missing pieces:
>  * various small bugs in cp15 registers or places where
>    we were missing the 32-bit version of a 64-bit register
>  * a bugfix for MSR/MRS (banked), which were not allowing
>    Hyp mode to access ELR_Hyp
>  * implementation of the ERET instruction for A32/T32
>  * support for taking exceptions to Hyp mode (the largest
>    of these missing bits)
> 
> This isn't complete, but I thought I'd push these patches
> out for review. My test setup is that I have another
> couple of patches, one which fixes up hw/arm/boot.c to
> boot AArch32 kernels in Hyp mode if it exists, and one
> which sets ARM_FEATURE_EL2 on our A15 model. With those I
> can get an outer kernel to boot with KVM support and try
> to run an inner guest kernel. The inner kernel boots OK
> but gets random segfaults in its userspace -- I haven't
> tracked down why this is yet...

Cool! :-)

> 
> Some bits that are definitely missing:
>  * ATS1HR, ATS1HW address translation ops
>  * I need to check that the trap semantics for AArch32
>    regs line up with their AArch64 counterparts
> 
> I also noticed that we fail to implement really quite a lot
> of the HCR_EL2 trap semantics for either AArch64 or AArch32,
> to the extent that I'm surprised that nested guests work
> under AArch64 :-)

Yes, we've have an entry for a while in our TODO list to improve
HCR trapping but haven't gotten around to it. IIRC, at the time
when we where bringing the EL2 stuff up, quite little was actually
being used by KVM/Xen.

Cheers,
Edgar



> 
> This patchset is based on top of my target-arm.for-3.1
> branch.
> 
> thanks
> -- PMM
> 
> Peter Maydell (10):
>   target/arm: Correct typo in HAMAIR1 regdef name
>   target/arm: Add missing .cp = 15 to HMAIR1 and HAMAIR1 regdefs
>   target/arm: Implement RAZ/WI HACTLR2
>   target/arm: Implement AArch32 HVBAR
>   target/arm: Implement AArch32 HCR and HCR2
>   target/arm: Implement AArch32 Hyp FARs
>   target/arm: Implement ESR_EL2/HSR for AArch32 and no-EL2
>   target/arm: Permit accesses to ELR_Hyp from Hyp mode via MSR/MRS
>     (banked)
>   target/arm: Implement AArch32 ERET instruction
>   target/arm: Implement support for taking exceptions to Hyp mode
> 
>  target/arm/helper.c    | 226 ++++++++++++++++++++++++++++++++++-------
>  target/arm/op_helper.c |  22 ++--
>  target/arm/translate.c |  41 +++++++-
>  3 files changed, 236 insertions(+), 53 deletions(-)
> 
> -- 
> 2.18.0
> 

On 15 August 2018 at 12:04, Edgar E. Iglesias <edgar.iglesias@xilinx.com> wrote:
> On Tue, Aug 14, 2018 at 01:42:44PM +0100, Peter Maydell wrote:
>> I also noticed that we fail to implement really quite a lot
>> of the HCR_EL2 trap semantics for either AArch64 or AArch32,
>> to the extent that I'm surprised that nested guests work
>> under AArch64 :-)
>
> Yes, we've have an entry for a while in our TODO list to improve
> HCR trapping but haven't gotten around to it. IIRC, at the time
> when we where bringing the EL2 stuff up, quite little was actually
> being used by KVM/Xen.

We're also I think not correctly implementing some of
the required upgrades of TLB maintenance ops to inner-shareable,
so I suspect that operation in SMP configs will be flaky.

thanks
-- PMM

On 14 August 2018 at 13:42, Peter Maydell <peter.maydell@linaro.org> wrote:
> Now we have virtualization support in the GICv2 emulation,
> I thought I'd have a look at how much we were still missing
> for being able to enable EL2 support for AArch32.
> This set of patches fixes some minor missing pieces:
>  * various small bugs in cp15 registers or places where
>    we were missing the 32-bit version of a 64-bit register
>  * a bugfix for MSR/MRS (banked), which were not allowing
>    Hyp mode to access ELR_Hyp
>  * implementation of the ERET instruction for A32/T32
>  * support for taking exceptions to Hyp mode (the largest
>    of these missing bits)
>
> This isn't complete, but I thought I'd push these patches
> out for review. My test setup is that I have another
> couple of patches, one which fixes up hw/arm/boot.c to
> boot AArch32 kernels in Hyp mode if it exists, and one
> which sets ARM_FEATURE_EL2 on our A15 model. With those I
> can get an outer kernel to boot with KVM support and try
> to run an inner guest kernel. The inner kernel boots OK
> but gets random segfaults in its userspace -- I haven't
> tracked down why this is yet...

I've put patches 1, 2, 4, 6, 7, 8, 9:

>   target/arm: Correct typo in HAMAIR1 regdef name
>   target/arm: Add missing .cp = 15 to HMAIR1 and HAMAIR1 regdefs
>   target/arm: Implement RAZ/WI HACTLR2
>   target/arm: Implement AArch32 HVBAR
>   target/arm: Implement AArch32 HCR and HCR2
>   target/arm: Implement AArch32 Hyp FARs
>   target/arm: Implement ESR_EL2/HSR for AArch32 and no-EL2
>   target/arm: Permit accesses to ELR_Hyp from Hyp mode via MSR/MRS
>     (banked)
>   target/arm: Implement AArch32 ERET instruction

into target-arm.next.

Patches 3, 5, 10 had issues in code review and I'll
rework those and send a v2 at some point:

>   target/arm: Implement RAZ/WI HACTLR2
>   target/arm: Implement AArch32 HCR and HCR2
>   target/arm: Implement support for taking exceptions to Hyp mode

thanks
-- PMM