Implement Microvm for aarch64 architecture

This series attempts to implement microvm for aarch64
architecture.

Just like how Sergio Lopez does for implementing microvm
for x86 architecture. We remove parts of emulate devices which
are not needed in microvm, compared with normal VM,
We only keep PL011 (UART), PL031 (RTC) and virtio-mmio
devices for microvm of aarch64.

We compared the boot time between virt VM and microvm With the
follow command lines:
For microvm:
qemu-system-aarch64 \
    -cpu host \
    -smp 1,sockets=1,cores=1,threads=1 \
    -m 1024M \
    -machine microvm,accel=kvm,gic-version=3 \
    -kernel vmlinux.bin \
    -drive id=test,file=ubuntu-rootfs.img,format=raw,if=none \
    -device virtio-blk-device,drive=test \
    -netdev tap,id=tap0,script=no,downscript=no \
    -device virtio-net-device,netdev=tap0 \
    -append "root=/dev/vda rw printk.time=y" \
    -nographic
Normal VM：Only changed the machine type to 'virt'
qemu-system-aarch64 \
    -cpu host \
    -smp 1,sockets=1,cores=1,threads=1 \
    -m 1024M \
    -machine virt,accel=kvm,gic-version=3 \
    -kernel vmlinux.bin \
    -drive id=test,file=ubuntu-rootfs.img,format=raw,if=none \
    -device virtio-blk-device,drive=test \
    -netdev tap,id=tap0,script=no,downscript=no \
    -device virtio-net-device,netdev=tap0 \
    -append "root=/dev/vda rw printk.time=y" \
    -nographic

This is the test results:
1. Boot time
1) boot time after dropped caches(echo 3 > /proc/sys/vm/drop_caches)
normal virt Machine: 204 ms
microvm Machine: 132 ms

2) boot time with caches
normal virt Machine: 168 ms
microvm Machine: 71 ms

2. MMIO regions
normal virt Machine:
address-space: memory
  0000000000000000-ffffffffffffffff (prio 0, i/o): system
    0000000000000000-0000000003ffffff (prio 0, romd): virt.flash0
    0000000004000000-0000000007ffffff (prio 0, romd): virt.flash1
    0000000008000000-000000000800ffff (prio 0, i/o): gicv3_dist
    0000000008080000-000000000809ffff (prio 0, i/o): gicv3_its
      0000000008080000-000000000808ffff (prio 0, i/o): control
      0000000008090000-000000000809ffff (prio 0, i/o): translation
    00000000080a0000-00000000080bffff (prio 0, i/o): gicv3_redist_region[0]
    0000000009000000-0000000009000fff (prio 0, i/o): pl011
    0000000009010000-0000000009010fff (prio 0, i/o): pl031
    0000000009020000-0000000009020007 (prio 0, i/o): fwcfg.data
    0000000009020008-0000000009020009 (prio 0, i/o): fwcfg.ctl
    0000000009020010-0000000009020017 (prio 0, i/o): fwcfg.dma
    0000000009030000-0000000009030fff (prio 0, i/o): pl061
    000000000a000000-000000000a0001ff (prio 0, i/o): virtio-mmio
 	... skipped 30 virtio-mmio region
    000000000a003e00-000000000a003fff (prio 0, i/o): virtio-mmio
    000000000c000000-000000000dffffff (prio 0, i/o): platform bus
    0000000010000000-000000003efeffff (prio 0, i/o): alias pcie-mmio @gpex_mmio 0000000010000000-000000003efeffff
    000000003eff0000-000000003effffff (prio 0, i/o): gpex_ioport
      000000003eff1000-000000003eff101f (prio 1, i/o): virtio-pci
    0000000040000000-000000007fffffff (prio 0, ram): mach-virt.ram
    0000004010000000-000000401fffffff (prio 0, i/o): alias pcie-ecam @pcie-mmcfg-mmio 0000000000000000-000000000fffffff
    0000008000000000-000000ffffffffff (prio 0, i/o): alias pcie-mmio-high @gpex_mmio 0000008000000000-000000ffffffffff

microvm Machine:
address-space: memory
  0000000000000000-ffffffffffffffff (prio 0, i/o): system
  0000000008000000-000000000800ffff (prio 0, i/o): gicv3_dist
    00000000080a0000-00000000080bffff (prio 0, i/o): gicv3_redist_region[0]
    0000000009000000-0000000009000fff (prio 0, i/o): pl011
    0000000009010000-0000000009010fff (prio 0, i/o): pl031
    000000000a000000-000000000a0001ff (prio 0, i/o): virtio-mmio
	... skipped 30 virtio-mmio region
    000000000a003e00-000000000a003fff (prio 0, i/o): virtio-mmio
    0000000040000000-000000007fffffff (prio 0, ram): mach-virt.ram

Please review this series.

Thinks,
Xu Yandong.

Xu Yandong (16):
  hw/arm/arm: Introduce ArmMachineState and ArmMachineClass
  hw/arm: move shared fdt member to ArmMachine
  hw/arm: move shared memmap member to ArmMachine
  hw/arm: move shared irqmap member to ArmMachine
  hw/arm: move shared smp_cpus member to ArmMachine
  hw/arm/virt: split MSI related codes from create_gic
  hw/arm/virt: split virt extension related codes from create_gic
  hw/arm/virt: split secure extension related codes from create_gic
  hw/arm: move shared gic member to ArmMachine
  hw/arm: split create_gic function
  hw/arm: move shared psci_enable and claim_edge_triggered_timers member
    to ArmMachine
  hw/arm: move shared devices related functions to arm.c and export them
  hw/arm: move shared fdt related functions to arm.c and export them
  hw/arm: move shared bootinfo member to ArmMachine
  hw/arm: move shared cpu related functions to arm.c and export them
  hw/arm: Introduce the microvm machine type

 default-configs/aarch64-softmmu.mak |    2 +
 hw/arm/Kconfig                      |    7 +
 hw/arm/Makefile.objs                |    3 +-
 hw/arm/arm.c                        |  687 +++++++++++++++++
 hw/arm/microvm.c                    |  303 ++++++++
 hw/arm/virt-acpi-build.c            |   48 +-
 hw/arm/virt.c                       | 1079 +++++++--------------------
 include/hw/arm/arm.h                |  152 ++++
 include/hw/arm/microvm.h            |   40 +
 include/hw/arm/virt.h               |   86 +--
 10 files changed, 1505 insertions(+), 902 deletions(-)
 create mode 100644 hw/arm/arm.c
 create mode 100644 hw/arm/microvm.c
 create mode 100644 include/hw/arm/arm.h
 create mode 100644 include/hw/arm/microvm.h

-- 
2.18.1

On Mon, 17 Feb 2020 at 07:42, Xu Yandong <xuyandong2@huawei.com> wrote:
>
> Implement Microvm for aarch64 architecture
>
> This series attempts to implement microvm for aarch64
> architecture.
>
> Just like how Sergio Lopez does for implementing microvm
> for x86 architecture. We remove parts of emulate devices which
> are not needed in microvm, compared with normal VM,
> We only keep PL011 (UART), PL031 (RTC) and virtio-mmio
> devices for microvm of aarch64.

For x86, 'microvm' makes sense, because the standard
PC models are models of real hardware with a lot of
legacy baggage. The situation is different for aarch64.
The 'virt' board is already intended as a "minimal
machine for booting a VM that knows it's a VM".
Why do we need another model that's intended for the
same purpose?

It would be more interesting to look at whether there
are reasonable places where we could allow command
line options to have the 'virt' board not provide
some devices where that makes a significant speed
improvement. Analysis of where the extra time is
actually going would also be helpful.

NB: I'm pretty firmly against dropping PCI. This is
a pluggable discoverable bus, and it's a much better
way to provide virtio than virtio-mmio.

thanks
-- PMM

Patchew URL: https://patchew.org/QEMU/1581925888-103620-1-git-send-email-xuyandong2@huawei.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  TEST    iotest-qcow2: 190
socket_accept failed: Resource temporarily unavailable
**
ERROR:/tmp/qemu-test/src/tests/qtest/libqtest.c:272:qtest_init_without_qmp_handshake: assertion failed: (s->fd >= 0 && s->qmp_fd >= 0)
/tmp/qemu-test/src/tests/qtest/libqtest.c:140: kill_qemu() tried to terminate QEMU process but encountered exit status 1 (expected 0)
ERROR - Bail out! ERROR:/tmp/qemu-test/src/tests/qtest/libqtest.c:272:qtest_init_without_qmp_handshake: assertion failed: (s->fd >= 0 && s->qmp_fd >= 0)
make: *** [check-qtest-aarch64] Error 1
make: *** Waiting for unfinished jobs....
  TEST    iotest-qcow2: 191
  TEST    iotest-qcow2: 192
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=caf02b6d3be34c0c9411c2cbdb28758d', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-2flvq84m/src/docker-src.2020-02-17-03.06.35.9047:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=caf02b6d3be34c0c9411c2cbdb28758d
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-2flvq84m/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    13m21.007s
user    0m8.792s


The full log is available at
http://patchew.org/logs/1581925888-103620-1-git-send-email-xuyandong2@huawei.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com