[Qemu-devel] [PATCH v3 00/34] target/arm: Implement ARMv8.1-VHE

Richard Henderson posted 34 patches 4 years, 7 months ago
Test checkpatch passed
Test s390x passed
Test asan passed
Test docker-mingw@fedora passed
Test FreeBSD passed
Test docker-clang@ubuntu passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190803184800.8221-1-richard.henderson@linaro.org
Maintainers: Peter Maydell <peter.maydell@linaro.org>
There is a newer version of this series
include/exec/cpu-all.h     |   11 +
include/exec/cpu-defs.h    |    2 +
include/exec/exec-all.h    |   35 ++
include/qom/cpu.h          |    2 +
target/arm/cpu-qom.h       |    1 +
target/arm/cpu.h           |  261 ++++-----
target/arm/internals.h     |   62 ++-
target/arm/translate.h     |    2 +-
accel/tcg/cputlb.c         |   81 +++
target/arm/cpu.c           |    2 +
target/arm/cpu64.c         |   20 +
target/arm/debug_helper.c  |   50 +-
target/arm/helper-a64.c    |    2 +-
target/arm/helper.c        | 1042 +++++++++++++++++++++++++-----------
target/arm/m_helper.c      |    6 +-
target/arm/pauth_helper.c  |   13 +-
target/arm/translate-a64.c |   13 +-
target/arm/translate.c     |   17 +-
18 files changed, 1134 insertions(+), 488 deletions(-)
[Qemu-devel] [PATCH v3 00/34] target/arm: Implement ARMv8.1-VHE
Posted by Richard Henderson 4 years, 7 months ago
About half of this patch set is cleanup of the qemu tlb handling
leading up to the actual implementation of VHE, and the biggest
piece of that: The EL2&0 translation regime.

Changes since v2:
  * arm_mmu_idx was incomplete; test TGE+E2H not just E2H.
  * arm_sctlr was incomplete; now uses arm_mmu_idx to avoid
    duplication of tests.
  * Update aa64_zva_access and ctr_el0_access for EL2.

Changes since v1:
  * Merge feedback from AJB.
  * Split out 7 renaming patches from "Reorganize ARMMMUIdx".
  * Alex's MIDR patch keeps the nested KVM from spitting warnings.

I have tested 

  qemu-system-aarch64 -accel kvm -cpu host -M virt,gic-version-host \
    -m 512 -bios /usr/share/edk2/aarch64/QEMU_EFI.fd -nographic

with fedora 30 system qemu, itself booted with

  ../bld/aarch64-softmmu/qemu-system-aarch64 \
    -cpu max -M virt,gic-version=3,virtualization=on \
    -drive if=virtio,file=./f30.q,format=qcow2 \
    -m 4G -nographic

It took a while, but eventually the nested bios arrived at the
pxe boot sequence.  Thankfully (?), the f30 shipped bios has
debug enabled, so there's some sense of progress in the meantime.


r~


Alex Bennée (2):
  target/arm: check TGE and E2H flags for EL0 pauth traps
  target/arm: generate a custom MIDR for -cpu max

Richard Henderson (32):
  cputlb: Add tlb_set_asid_for_mmuidx
  cputlb: Add tlb_flush_asid_by_mmuidx and friends
  target/arm: Install ASIDs for long-form from EL1
  target/arm: Install ASIDs for short-form from EL1
  target/arm: Install ASIDs for EL2
  target/arm: Define isar_feature_aa64_vh
  target/arm: Enable HCR_E2H for VHE
  target/arm: Add CONTEXTIDR_EL2
  target/arm: Add TTBR1_EL2
  target/arm: Update CNTVCT_EL0 for VHE
  target/arm: Add the hypervisor virtual counter
  target/arm: Add VHE system register redirection and aliasing
  target/arm: Split out vae1_tlbmask, vmalle1_tlbmask
  target/arm: Simplify tlb_force_broadcast alternatives
  target/arm: Rename ARMMMUIdx*_S12NSE* to ARMMMUIdx*_E10_*
  target/arm: Rename ARMMMUIdx_S2NS to ARMMMUIdx_Stage2
  target/arm: Rename ARMMMUIdx_S1NSE* to ARMMMUIdx_Stage1_E*
  target/arm: Rename ARMMMUIdx_S1SE* to ARMMMUIdx_SE*
  target/arm: Rename ARMMMUIdx*_S1E3 to ARMMMUIdx*_SE3
  target/arm: Rename ARMMMUIdx_S1E2 to ARMMMUIdx_E2
  target/arm: Reorganize ARMMMUIdx
  target/arm: Add regime_has_2_ranges
  target/arm: Update arm_mmu_idx for VHE
  target/arm: Update arm_sctlr for VHE
  target/arm: Update aa64_zva_access for EL2
  target/arm: Update ctr_el0_access for EL2
  target/arm: Install asids for E2&0 translation regime
  target/arm: Flush tlbs for E2&0 translation regime
  target/arm: Update arm_phys_excp_target_el for TGE
  target/arm: Update regime_is_user for EL2&0
  target/arm: Update {fp,sve}_exception_el for VHE
  target/arm: Enable ARMv8.1-VHE in -cpu max

 include/exec/cpu-all.h     |   11 +
 include/exec/cpu-defs.h    |    2 +
 include/exec/exec-all.h    |   35 ++
 include/qom/cpu.h          |    2 +
 target/arm/cpu-qom.h       |    1 +
 target/arm/cpu.h           |  261 ++++-----
 target/arm/internals.h     |   62 ++-
 target/arm/translate.h     |    2 +-
 accel/tcg/cputlb.c         |   81 +++
 target/arm/cpu.c           |    2 +
 target/arm/cpu64.c         |   20 +
 target/arm/debug_helper.c  |   50 +-
 target/arm/helper-a64.c    |    2 +-
 target/arm/helper.c        | 1042 +++++++++++++++++++++++++-----------
 target/arm/m_helper.c      |    6 +-
 target/arm/pauth_helper.c  |   13 +-
 target/arm/translate-a64.c |   13 +-
 target/arm/translate.c     |   17 +-
 18 files changed, 1134 insertions(+), 488 deletions(-)

-- 
2.17.1


Re: [Qemu-devel] [PATCH v3 00/34] target/arm: Implement ARMv8.1-VHE
Posted by Alex Bennée 4 years, 7 months ago
Richard Henderson <richard.henderson@linaro.org> writes:

> About half of this patch set is cleanup of the qemu tlb handling
> leading up to the actual implementation of VHE, and the biggest
> piece of that: The EL2&0 translation regime.
>
> Changes since v2:
>   * arm_mmu_idx was incomplete; test TGE+E2H not just E2H.
>   * arm_sctlr was incomplete; now uses arm_mmu_idx to avoid
>     duplication of tests.
>   * Update aa64_zva_access and ctr_el0_access for EL2.
>
> Changes since v1:
>   * Merge feedback from AJB.
>   * Split out 7 renaming patches from "Reorganize ARMMMUIdx".
>   * Alex's MIDR patch keeps the nested KVM from spitting warnings.
>
> I have tested
>
>   qemu-system-aarch64 -accel kvm -cpu host -M virt,gic-version-host \
>     -m 512 -bios /usr/share/edk2/aarch64/QEMU_EFI.fd -nographic

So testing with a host doing:

  ./aarch64-softmmu/qemu-system-aarch64 -machine type=virt,virtualization=on -cpu cortex-a57 -serial mon:stdio -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 -device virtio-scsi-pci -drive file=/dev/zvol/hackpool-0/debian-buster-arm64,id=hd0,index=0,if=none,format=raw,discard=on -device scsi-hd,drive=hd0 -kernel ../linux.git/builds/arm64/arch/arm64/boot/Image -append "console=ttyAMA0 root=/dev/sda2" -display none -m 4096 -smp 8

And a guest doing:

  ./aarch64-softmmu/qemu-system-aarch64 -machine type=virt -cpu host
  -serial mon:stdio -nic user,model=virtio-net-pci -device
  virtio-scsi-pci -kernel /boot/vmlinuz-4.19.0-5-arm64 -append "console=ttyAMA0 panic=-1" -display none -m 256 -smp 4 --no-reboot
  --enable-kvm

I triggered:

  ERROR:/home/alex.bennee/lsrc/qemu.git/target/arm/helper.c:3436:update_lpae_el1_asid: code should not be reached
fish: “./aarch64-softmmu/qemu-system-a…” terminated by signal SIGABRT (Abort)

With -cpu max on the host it hangs the whole thing. I'm going to
continue to experiment with explicit GIC versions.

>
> with fedora 30 system qemu, itself booted with
>
>   ../bld/aarch64-softmmu/qemu-system-aarch64 \
>     -cpu max -M virt,gic-version=3,virtualization=on \
>     -drive if=virtio,file=./f30.q,format=qcow2 \
>     -m 4G -nographic
>
> It took a while, but eventually the nested bios arrived at the
> pxe boot sequence.  Thankfully (?), the f30 shipped bios has
> debug enabled, so there's some sense of progress in the meantime.
>
>
> r~
>
>
> Alex Bennée (2):
>   target/arm: check TGE and E2H flags for EL0 pauth traps
>   target/arm: generate a custom MIDR for -cpu max
>
> Richard Henderson (32):
>   cputlb: Add tlb_set_asid_for_mmuidx
>   cputlb: Add tlb_flush_asid_by_mmuidx and friends
>   target/arm: Install ASIDs for long-form from EL1
>   target/arm: Install ASIDs for short-form from EL1
>   target/arm: Install ASIDs for EL2
>   target/arm: Define isar_feature_aa64_vh
>   target/arm: Enable HCR_E2H for VHE
>   target/arm: Add CONTEXTIDR_EL2
>   target/arm: Add TTBR1_EL2
>   target/arm: Update CNTVCT_EL0 for VHE
>   target/arm: Add the hypervisor virtual counter
>   target/arm: Add VHE system register redirection and aliasing
>   target/arm: Split out vae1_tlbmask, vmalle1_tlbmask
>   target/arm: Simplify tlb_force_broadcast alternatives
>   target/arm: Rename ARMMMUIdx*_S12NSE* to ARMMMUIdx*_E10_*
>   target/arm: Rename ARMMMUIdx_S2NS to ARMMMUIdx_Stage2
>   target/arm: Rename ARMMMUIdx_S1NSE* to ARMMMUIdx_Stage1_E*
>   target/arm: Rename ARMMMUIdx_S1SE* to ARMMMUIdx_SE*
>   target/arm: Rename ARMMMUIdx*_S1E3 to ARMMMUIdx*_SE3
>   target/arm: Rename ARMMMUIdx_S1E2 to ARMMMUIdx_E2
>   target/arm: Reorganize ARMMMUIdx
>   target/arm: Add regime_has_2_ranges
>   target/arm: Update arm_mmu_idx for VHE
>   target/arm: Update arm_sctlr for VHE
>   target/arm: Update aa64_zva_access for EL2
>   target/arm: Update ctr_el0_access for EL2
>   target/arm: Install asids for E2&0 translation regime
>   target/arm: Flush tlbs for E2&0 translation regime
>   target/arm: Update arm_phys_excp_target_el for TGE
>   target/arm: Update regime_is_user for EL2&0
>   target/arm: Update {fp,sve}_exception_el for VHE
>   target/arm: Enable ARMv8.1-VHE in -cpu max
>
>  include/exec/cpu-all.h     |   11 +
>  include/exec/cpu-defs.h    |    2 +
>  include/exec/exec-all.h    |   35 ++
>  include/qom/cpu.h          |    2 +
>  target/arm/cpu-qom.h       |    1 +
>  target/arm/cpu.h           |  261 ++++-----
>  target/arm/internals.h     |   62 ++-
>  target/arm/translate.h     |    2 +-
>  accel/tcg/cputlb.c         |   81 +++
>  target/arm/cpu.c           |    2 +
>  target/arm/cpu64.c         |   20 +
>  target/arm/debug_helper.c  |   50 +-
>  target/arm/helper-a64.c    |    2 +-
>  target/arm/helper.c        | 1042 +++++++++++++++++++++++++-----------
>  target/arm/m_helper.c      |    6 +-
>  target/arm/pauth_helper.c  |   13 +-
>  target/arm/translate-a64.c |   13 +-
>  target/arm/translate.c     |   17 +-
>  18 files changed, 1134 insertions(+), 488 deletions(-)


--
Alex Bennée

Re: [Qemu-devel] [PATCH v3 00/34] target/arm: Implement ARMv8.1-VHE
Posted by Richard Henderson 4 years, 7 months ago
On 8/5/19 6:02 AM, Alex Bennée wrote:
> 
> Richard Henderson <richard.henderson@linaro.org> writes:
> 
>> About half of this patch set is cleanup of the qemu tlb handling
>> leading up to the actual implementation of VHE, and the biggest
>> piece of that: The EL2&0 translation regime.
>>
>> Changes since v2:
>>   * arm_mmu_idx was incomplete; test TGE+E2H not just E2H.
>>   * arm_sctlr was incomplete; now uses arm_mmu_idx to avoid
>>     duplication of tests.
>>   * Update aa64_zva_access and ctr_el0_access for EL2.
>>
>> Changes since v1:
>>   * Merge feedback from AJB.
>>   * Split out 7 renaming patches from "Reorganize ARMMMUIdx".
>>   * Alex's MIDR patch keeps the nested KVM from spitting warnings.
>>
>> I have tested
>>
>>   qemu-system-aarch64 -accel kvm -cpu host -M virt,gic-version-host \
>>     -m 512 -bios /usr/share/edk2/aarch64/QEMU_EFI.fd -nographic
> 
> So testing with a host doing:
> 
>   ./aarch64-softmmu/qemu-system-aarch64 -machine type=virt,virtualization=on -cpu cortex-a57 -serial mon:stdio -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 -device virtio-scsi-pci -drive file=/dev/zvol/hackpool-0/debian-buster-arm64,id=hd0,index=0,if=none,format=raw,discard=on -device scsi-hd,drive=hd0 -kernel ../linux.git/builds/arm64/arch/arm64/boot/Image -append "console=ttyAMA0 root=/dev/sda2" -display none -m 4096 -smp 8
> 
> And a guest doing:
> 
>   ./aarch64-softmmu/qemu-system-aarch64 -machine type=virt -cpu host
>   -serial mon:stdio -nic user,model=virtio-net-pci -device
>   virtio-scsi-pci -kernel /boot/vmlinuz-4.19.0-5-arm64 -append "console=ttyAMA0 panic=-1" -display none -m 256 -smp 4 --no-reboot
>   --enable-kvm
> 
> I triggered:
> 
>   ERROR:/home/alex.bennee/lsrc/qemu.git/target/arm/helper.c:3436:update_lpae_el1_asid: code should not be reached
> fish: “./aarch64-softmmu/qemu-system-a…” terminated by signal SIGABRT (Abort)

Whoops.  Rebase error while changing the signature of this function.
Thanks for re-testing the case where VHE isn't present.  :-P

> With -cpu max on the host it hangs the whole thing. I'm going to
> continue to experiment with explicit GIC versions.

Hangs the host?  Are you sure that the guest isn't just slow?
That's why I changed my testing to boot a debug edk2, which
outputs stuff much sooner than the kernel does.

Although I have no idea why the guest should be extra slow.
It does seem like it ought to be booting at the same speed as
the host.  I see tlb flushes as quite high in the profile,
and wonder if I'm doing too many of them.


r~

Re: [Qemu-devel] [PATCH v3 00/34] target/arm: Implement ARMv8.1-VHE
Posted by Alex Bennée 4 years, 7 months ago
Richard Henderson <richard.henderson@linaro.org> writes:

> On 8/5/19 6:02 AM, Alex Bennée wrote:
>>
>> Richard Henderson <richard.henderson@linaro.org> writes:
>>
>>> About half of this patch set is cleanup of the qemu tlb handling
>>> leading up to the actual implementation of VHE, and the biggest
>>> piece of that: The EL2&0 translation regime.
>>>
>>> Changes since v2:
>>>   * arm_mmu_idx was incomplete; test TGE+E2H not just E2H.
>>>   * arm_sctlr was incomplete; now uses arm_mmu_idx to avoid
>>>     duplication of tests.
>>>   * Update aa64_zva_access and ctr_el0_access for EL2.
>>>
>>> Changes since v1:
>>>   * Merge feedback from AJB.
>>>   * Split out 7 renaming patches from "Reorganize ARMMMUIdx".
>>>   * Alex's MIDR patch keeps the nested KVM from spitting warnings.
>>>
>>> I have tested
>>>
>>>   qemu-system-aarch64 -accel kvm -cpu host -M virt,gic-version-host \
>>>     -m 512 -bios /usr/share/edk2/aarch64/QEMU_EFI.fd -nographic
>>
>> So testing with a host doing:
>>
>>   ./aarch64-softmmu/qemu-system-aarch64 -machine type=virt,virtualization=on -cpu cortex-a57 -serial mon:stdio -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 -device virtio-scsi-pci -drive file=/dev/zvol/hackpool-0/debian-buster-arm64,id=hd0,index=0,if=none,format=raw,discard=on -device scsi-hd,drive=hd0 -kernel ../linux.git/builds/arm64/arch/arm64/boot/Image -append "console=ttyAMA0 root=/dev/sda2" -display none -m 4096 -smp 8
>>
>> And a guest doing:
>>
>>   ./aarch64-softmmu/qemu-system-aarch64 -machine type=virt -cpu host
>>   -serial mon:stdio -nic user,model=virtio-net-pci -device
>>   virtio-scsi-pci -kernel /boot/vmlinuz-4.19.0-5-arm64 -append "console=ttyAMA0 panic=-1" -display none -m 256 -smp 4 --no-reboot
>>   --enable-kvm
>>
>> I triggered:
>>
>>   ERROR:/home/alex.bennee/lsrc/qemu.git/target/arm/helper.c:3436:update_lpae_el1_asid: code should not be reached
>> fish: “./aarch64-softmmu/qemu-system-a…” terminated by signal SIGABRT (Abort)
>
> Whoops.  Rebase error while changing the signature of this function.
> Thanks for re-testing the case where VHE isn't present.  :-P
>
>> With -cpu max on the host it hangs the whole thing. I'm going to
>> continue to experiment with explicit GIC versions.
>
> Hangs the host?  Are you sure that the guest isn't just slow?

Ahh nested terminology - the aarch64 VHE host hangs so I can't access
it's shells either. I'll try and get a better trace once I've gotten a
reliable non-VHE guest-guest boot ;-)

> That's why I changed my testing to boot a debug edk2, which
> outputs stuff much sooner than the kernel does.

Looking via gdbstub it looked like stuff had stopped happening but you
are right it could be just slow. None of the threads seem to be doing
anything.

> Although I have no idea why the guest should be extra slow.
> It does seem like it ought to be booting at the same speed as
> the host.  I see tlb flushes as quite high in the profile,
> and wonder if I'm doing too many of them.

Hmm yeah they do seem a bit high:

TLB full flushes    8
TLB partial flushes 39036316
TLB elided flushes  256488402

... 10 seconds ...

TLB full flushes    8
TLB partial flushes 40393340
TLB elided flushes  265444803

For non-VHE that seems expected because we should be changing the entire
mapping regime as we bounce between the EL1's - shouldn't be as high for
VHE right?


>
>
> r~


--
Alex Bennée