[PATCH v8 15/15] hw/vmapple/vmapple: Add vmapple machine type

Phil Dennis-Jordan posted 15 patches 2 weeks, 1 day ago
There is a newer version of this series
[PATCH v8 15/15] hw/vmapple/vmapple: Add vmapple machine type
Posted by Phil Dennis-Jordan 2 weeks, 1 day ago
From: Alexander Graf <graf@amazon.com>

Apple defines a new "vmapple" machine type as part of its proprietary
macOS Virtualization.Framework vmm. This machine type is similar to the
virt one, but with subtle differences in base devices, a few special
vmapple device additions and a vastly different boot chain.

This patch reimplements this machine type in QEMU. To use it, you
have to have a readily installed version of macOS for VMApple,
run on macOS with -accel hvf, pass the Virtualization.Framework
boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
and pass aux and root volume as virtio drives. In addition, you also
need to find the machine UUID and pass that as -M vmapple,uuid= parameter:

$ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
    -bios /System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
    -drive file=aux,if=pflash,format=raw \
    -drive file=root,if=pflash,format=raw \
    -drive file=aux,if=none,id=aux,format=raw \
    -device vmapple-virtio-aux,drive=aux \
    -drive file=root,if=none,id=root,format=raw \
    -device vmapple-virtio-root,drive=root

With all these in place, you should be able to see macOS booting
successfully.

Known issues:
 - Keyboard and mouse/tablet input is laggy. The reason for this is
   either that macOS's XHCI driver is broken when the device/platform
   does not support MSI/MSI-X, or there's some unfortunate interplay
   with Qemu's XHCI implementation in this scenario.
 - Currently only macOS 12 guests are supported. The boot process for
   13+ will need further investigation and adjustment.

Signed-off-by: Alexander Graf <graf@amazon.com>
Co-authored-by: Phil Dennis-Jordan <phil@philjordan.eu>
Signed-off-by: Phil Dennis-Jordan <phil@philjordan.eu>
---

v3:
 * Rebased on latest upstream, updated affinity and NIC creation
   API usage
 * Included Apple-variant virtio-blk in build dependency
 * Updated API usage for setting 'redist-region-count' array-typed property
   on GIC.
 * Switched from virtio HID devices (for which macOS 12 does not contain
   drivers) to an XHCI USB controller and USB HID devices.

v4:
 * Fixups for v4 changes to the other patches in the set.
 * Corrected the assert macro to use
 * Removed superfluous endian conversions corresponding to cfg's.
 * Init error handling improvement.
 * No need to select CPU type on TCG, as only HVF is supported.
 * Machine type version bumped to 9.2
 * #include order improved

v5:
 * Fixed memory reservation for ecam alias region.
 * Better error handling setting properties on devices.
 * Simplified the machine ECID/UUID extraction script and actually created a
   file for it rather than quoting its code in documentation.

v7:
 * Tiny error handling fix, un-inlined function.

v8:
 * Use object_property_add_uint64_ptr rather than defining custom UUID
   property get/set functions.

 MAINTAINERS                 |   1 +
 contrib/vmapple/uuid.sh     |   9 +
 docs/system/arm/vmapple.rst |  60 ++++
 docs/system/target-arm.rst  |   1 +
 hw/vmapple/Kconfig          |  20 ++
 hw/vmapple/meson.build      |   1 +
 hw/vmapple/vmapple.c        | 638 ++++++++++++++++++++++++++++++++++++
 7 files changed, 730 insertions(+)
 create mode 100755 contrib/vmapple/uuid.sh
 create mode 100644 docs/system/arm/vmapple.rst
 create mode 100644 hw/vmapple/vmapple.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 9d0d26cb65f..9591fd44a34 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2767,6 +2767,7 @@ R: Phil Dennis-Jordan <phil@philjordan.eu>
 S: Maintained
 F: hw/vmapple/*
 F: include/hw/vmapple/*
+F: docs/system/arm/vmapple.rst
 
 Subsystems
 ----------
diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.sh
new file mode 100755
index 00000000000..956e8c3afed
--- /dev/null
+++ b/contrib/vmapple/uuid.sh
@@ -0,0 +1,9 @@
+#!/bin/sh
+# Used for converting a guest provisioned using Virtualization.framework
+# for use with the QEMU 'vmapple' aarch64 machine type.
+#
+# Extracts the Machine UUID from Virtualization.framework VM JSON file.
+# (as produced by 'macosvm', passed as command line argument)
+
+plutil -extract machineId raw "$1" | base64 -d | plutil -extract ECID raw -
+
diff --git a/docs/system/arm/vmapple.rst b/docs/system/arm/vmapple.rst
new file mode 100644
index 00000000000..6a634fa4572
--- /dev/null
+++ b/docs/system/arm/vmapple.rst
@@ -0,0 +1,60 @@
+VMApple machine emulation
+========================================================================================
+
+VMApple is the device model that the macOS built-in hypervisor called "Virtualization.framework"
+exposes to Apple Silicon macOS guests. The "vmapple" machine model in QEMU implements the same
+device model, but does not use any code from Virtualization.Framework.
+
+Prerequisites
+-------------
+
+To run the vmapple machine model, you need to
+
+ * Run on Apple Silicon
+ * Run on macOS 12.0 or above
+ * Have an already installed copy of a Virtualization.Framework macOS 12 virtual machine. I will
+   assume that you installed it using the macosvm CLI.
+
+First, we need to extract the UUID from the virtual machine that you installed. You can do this
+by running the shell script in contrib/vmapple/uuid.sh on the macosvm.json file.
+
+.. code-block:: bash
+  :caption: uuid.sh script to extract the UUID from a macosvm.json file
+
+  $ contrib/vmapple/uuid.sh "path/to/macosvm.json"
+
+Now we also need to trim the aux partition. It contains metadata that we can just discard:
+
+.. code-block:: bash
+  :caption: Command to trim the aux file
+
+  $ dd if="aux.img" of="aux.img.trimmed" bs=$(( 0x4000 )) skip=1
+
+How to run
+----------
+
+Then, we can launch QEMU with the Virtualization.Framework pre-boot environment and the readily
+installed target disk images. I recommend to port forward the VM's ssh and vnc ports to the host
+to get better interactive access into the target system:
+
+.. code-block:: bash
+  :caption: Example execution command line
+
+  $ UUID=$(uuid.sh macosvm.json)
+  $ AVPBOOTER=/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin
+  $ AUX=aux.img.trimmed
+  $ DISK=disk.img
+  $ qemu-system-aarch64 \
+       -serial mon:stdio \
+       -m 4G \
+       -accel hvf \
+       -M vmapple,uuid=$UUID \
+       -bios $AVPBOOTER \
+        -drive file="$AUX",if=pflash,format=raw \
+        -drive file="$DISK",if=pflash,format=raw \
+       -drive file="$AUX",if=none,id=aux,format=raw \
+       -drive file="$DISK",if=none,id=root,format=raw \
+       -device vmapple-virtio-aux,drive=aux \
+       -device vmapple-virtio-root,drive=root \
+       -net user,ipv6=off,hostfwd=tcp::2222-:22,hostfwd=tcp::5901-:5900 \
+       -net nic,model=virtio-net-pci \
diff --git a/docs/system/target-arm.rst b/docs/system/target-arm.rst
index 9aaa9c414c9..3426f79100b 100644
--- a/docs/system/target-arm.rst
+++ b/docs/system/target-arm.rst
@@ -102,6 +102,7 @@ Board-specific documentation
    arm/stellaris
    arm/stm32
    arm/virt
+   arm/vmapple
    arm/xenpvh
    arm/xlnx-versal-virt
    arm/xlnx-zynq
diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
index bcd1be63e3c..6a4c4a7fa2e 100644
--- a/hw/vmapple/Kconfig
+++ b/hw/vmapple/Kconfig
@@ -10,3 +10,23 @@ config VMAPPLE_CFG
 config VMAPPLE_VIRTIO_BLK
     bool
 
+config VMAPPLE
+    bool
+    depends on ARM
+    depends on HVF
+    default y if ARM
+    imply PCI_DEVICES
+    select ARM_GIC
+    select PLATFORM_BUS
+    select PCI_EXPRESS
+    select PCI_EXPRESS_GENERIC_BRIDGE
+    select PL011 # UART
+    select PL031 # RTC
+    select PL061 # GPIO
+    select GPIO_PWR
+    select PVPANIC_MMIO
+    select VMAPPLE_AES
+    select VMAPPLE_BDIF
+    select VMAPPLE_CFG
+    select MAC_PVG_MMIO
+    select VMAPPLE_VIRTIO_BLK
diff --git a/hw/vmapple/meson.build b/hw/vmapple/meson.build
index bf17cf906c9..e572f7d5602 100644
--- a/hw/vmapple/meson.build
+++ b/hw/vmapple/meson.build
@@ -2,3 +2,4 @@ system_ss.add(when: 'CONFIG_VMAPPLE_AES',  if_true: files('aes.c'))
 system_ss.add(when: 'CONFIG_VMAPPLE_BDIF', if_true: files('bdif.c'))
 system_ss.add(when: 'CONFIG_VMAPPLE_CFG',  if_true: files('cfg.c'))
 system_ss.add(when: 'CONFIG_VMAPPLE_VIRTIO_BLK',  if_true: files('virtio-blk.c'))
+specific_ss.add(when: 'CONFIG_VMAPPLE',     if_true: files('vmapple.c'))
diff --git a/hw/vmapple/vmapple.c b/hw/vmapple/vmapple.c
new file mode 100644
index 00000000000..8b8710c70a2
--- /dev/null
+++ b/hw/vmapple/vmapple.c
@@ -0,0 +1,638 @@
+/*
+ * VMApple machine emulation
+ *
+ * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * VMApple is the device model that the macOS built-in hypervisor called
+ * "Virtualization.framework" exposes to Apple Silicon macOS guests. The
+ * machine model in this file implements the same device model in QEMU, but
+ * does not use any code from Virtualization.Framework.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/bitops.h"
+#include "qemu/datadir.h"
+#include "qemu/error-report.h"
+#include "qemu/guest-random.h"
+#include "qemu/help-texts.h"
+#include "qemu/log.h"
+#include "qemu/module.h"
+#include "qemu/option.h"
+#include "qemu/units.h"
+#include "monitor/qdev.h"
+#include "hw/boards.h"
+#include "hw/irq.h"
+#include "hw/loader.h"
+#include "hw/qdev-properties.h"
+#include "hw/sysbus.h"
+#include "hw/usb.h"
+#include "hw/arm/boot.h"
+#include "hw/arm/primecell.h"
+#include "hw/char/pl011.h"
+#include "hw/intc/arm_gic.h"
+#include "hw/intc/arm_gicv3_common.h"
+#include "hw/misc/pvpanic.h"
+#include "hw/pci-host/gpex.h"
+#include "hw/usb/xhci.h"
+#include "hw/virtio/virtio-pci.h"
+#include "hw/vmapple/vmapple.h"
+#include "net/net.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qlist.h"
+#include "qapi/visitor.h"
+#include "qapi/qapi-visit-common.h"
+#include "standard-headers/linux/input.h"
+#include "sysemu/hvf.h"
+#include "sysemu/kvm.h"
+#include "sysemu/reset.h"
+#include "sysemu/runstate.h"
+#include "sysemu/sysemu.h"
+#include "target/arm/internals.h"
+#include "target/arm/kvm_arm.h"
+
+struct VMAppleMachineClass {
+    MachineClass parent;
+};
+
+struct VMAppleMachineState {
+    MachineState parent;
+
+    Notifier machine_done;
+    struct arm_boot_info bootinfo;
+    MemMapEntry *memmap;
+    const int *irqmap;
+    DeviceState *gic;
+    DeviceState *cfg;
+    Notifier powerdown_notifier;
+    PCIBus *bus;
+    MemoryRegion fw_mr;
+    MemoryRegion ecam_alias;
+    uint64_t uuid;
+};
+
+#define DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, latest) \
+    static void vmapple##major##_##minor##_class_init(ObjectClass *oc, \
+                                                    void *data) \
+    { \
+        MachineClass *mc = MACHINE_CLASS(oc); \
+        vmapple_machine_##major##_##minor##_options(mc); \
+        mc->desc = "QEMU " # major "." # minor " Apple Virtual Machine"; \
+        if (latest) { \
+            mc->alias = "vmapple"; \
+        } \
+    } \
+    static const TypeInfo machvmapple##major##_##minor##_info = { \
+        .name = MACHINE_TYPE_NAME("vmapple-" # major "." # minor), \
+        .parent = TYPE_VMAPPLE_MACHINE, \
+        .class_init = vmapple##major##_##minor##_class_init, \
+    }; \
+    static void machvmapple_machine_##major##_##minor##_init(void) \
+    { \
+        type_register_static(&machvmapple##major##_##minor##_info); \
+    } \
+    type_init(machvmapple_machine_##major##_##minor##_init);
+
+#define DEFINE_VMAPPLE_MACHINE_AS_LATEST(major, minor) \
+    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, true)
+#define DEFINE_VMAPPLE_MACHINE(major, minor) \
+    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, false)
+
+#define TYPE_VMAPPLE_MACHINE   MACHINE_TYPE_NAME("vmapple")
+OBJECT_DECLARE_TYPE(VMAppleMachineState, VMAppleMachineClass, VMAPPLE_MACHINE)
+
+/* Number of external interrupt lines to configure the GIC with */
+#define NUM_IRQS 256
+
+enum {
+    VMAPPLE_FIRMWARE,
+    VMAPPLE_CONFIG,
+    VMAPPLE_MEM,
+    VMAPPLE_GIC_DIST,
+    VMAPPLE_GIC_REDIST,
+    VMAPPLE_UART,
+    VMAPPLE_RTC,
+    VMAPPLE_PCIE,
+    VMAPPLE_PCIE_MMIO,
+    VMAPPLE_PCIE_ECAM,
+    VMAPPLE_GPIO,
+    VMAPPLE_PVPANIC,
+    VMAPPLE_APV_GFX,
+    VMAPPLE_APV_IOSFC,
+    VMAPPLE_AES_1,
+    VMAPPLE_AES_2,
+    VMAPPLE_BDOOR,
+    VMAPPLE_MEMMAP_LAST,
+};
+
+static MemMapEntry memmap[] = {
+    [VMAPPLE_FIRMWARE] =           { 0x00100000, 0x00100000 },
+    [VMAPPLE_CONFIG] =             { 0x00400000, 0x00010000 },
+
+    [VMAPPLE_GIC_DIST] =           { 0x10000000, 0x00010000 },
+    [VMAPPLE_GIC_REDIST] =         { 0x10010000, 0x00400000 },
+
+    [VMAPPLE_UART] =               { 0x20010000, 0x00010000 },
+    [VMAPPLE_RTC] =                { 0x20050000, 0x00001000 },
+    [VMAPPLE_GPIO] =               { 0x20060000, 0x00001000 },
+    [VMAPPLE_PVPANIC] =            { 0x20070000, 0x00000002 },
+    [VMAPPLE_BDOOR] =              { 0x30000000, 0x00200000 },
+    [VMAPPLE_APV_GFX] =            { 0x30200000, 0x00010000 },
+    [VMAPPLE_APV_IOSFC] =          { 0x30210000, 0x00010000 },
+    [VMAPPLE_AES_1] =              { 0x30220000, 0x00004000 },
+    [VMAPPLE_AES_2] =              { 0x30230000, 0x00004000 },
+    [VMAPPLE_PCIE_ECAM] =          { 0x40000000, 0x10000000 },
+    [VMAPPLE_PCIE_MMIO] =          { 0x50000000, 0x1fff0000 },
+
+    /* Actual RAM size depends on configuration */
+    [VMAPPLE_MEM] =                { 0x70000000ULL, GiB},
+};
+
+static const int irqmap[] = {
+    [VMAPPLE_UART] = 1,
+    [VMAPPLE_RTC] = 2,
+    [VMAPPLE_GPIO] = 0x5,
+    [VMAPPLE_APV_IOSFC] = 0x10,
+    [VMAPPLE_APV_GFX] = 0x11,
+    [VMAPPLE_AES_1] = 0x12,
+    [VMAPPLE_PCIE] = 0x20,
+};
+
+#define GPEX_NUM_IRQS 16
+
+static void create_bdif(VMAppleMachineState *vms, MemoryRegion *mem)
+{
+    DeviceState *bdif;
+    SysBusDevice *bdif_sb;
+    DriveInfo *di_aux = drive_get(IF_PFLASH, 0, 0);
+    DriveInfo *di_root = drive_get(IF_PFLASH, 0, 1);
+
+    if (!di_aux) {
+        error_report("No AUX device. Please specify one as pflash drive.");
+        exit(1);
+    }
+
+    if (!di_root) {
+        /* Fall back to the first IF_VIRTIO device as root device */
+        di_root = drive_get(IF_VIRTIO, 0, 0);
+    }
+
+    if (!di_root) {
+        error_report("No root device. Please specify one as virtio drive.");
+        exit(1);
+    }
+
+    /* PV backdoor device */
+    bdif = qdev_new(TYPE_VMAPPLE_BDIF);
+    bdif_sb = SYS_BUS_DEVICE(bdif);
+    sysbus_mmio_map(bdif_sb, 0, vms->memmap[VMAPPLE_BDOOR].base);
+
+    qdev_prop_set_drive(DEVICE(bdif), "aux", blk_by_legacy_dinfo(di_aux));
+    qdev_prop_set_drive(DEVICE(bdif), "root", blk_by_legacy_dinfo(di_root));
+
+    sysbus_realize_and_unref(bdif_sb, &error_fatal);
+}
+
+static void create_pvpanic(VMAppleMachineState *vms, MemoryRegion *mem)
+{
+    SysBusDevice *cfg;
+
+    vms->cfg = qdev_new(TYPE_PVPANIC_MMIO_DEVICE);
+    cfg = SYS_BUS_DEVICE(vms->cfg);
+    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_PVPANIC].base);
+
+    sysbus_realize_and_unref(cfg, &error_fatal);
+}
+
+static void create_cfg(VMAppleMachineState *vms, MemoryRegion *mem)
+{
+    SysBusDevice *cfg;
+    MachineState *machine = MACHINE(vms);
+    uint32_t rnd = 1;
+
+    vms->cfg = qdev_new(TYPE_VMAPPLE_CFG);
+    cfg = SYS_BUS_DEVICE(vms->cfg);
+    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_CONFIG].base);
+
+    qemu_guest_getrandom_nofail(&rnd, sizeof(rnd));
+
+    qdev_prop_set_uint32(vms->cfg, "nr-cpus", machine->smp.cpus);
+    qdev_prop_set_uint64(vms->cfg, "ecid", vms->uuid);
+    qdev_prop_set_uint64(vms->cfg, "ram-size", machine->ram_size);
+    qdev_prop_set_uint32(vms->cfg, "rnd", rnd);
+
+    sysbus_realize_and_unref(cfg, &error_fatal);
+}
+
+static void create_gfx(VMAppleMachineState *vms, MemoryRegion *mem)
+{
+    int irq_gfx = vms->irqmap[VMAPPLE_APV_GFX];
+    int irq_iosfc = vms->irqmap[VMAPPLE_APV_IOSFC];
+    SysBusDevice *gfx;
+
+    gfx = SYS_BUS_DEVICE(qdev_new("apple-gfx-mmio"));
+    sysbus_mmio_map(gfx, 0, vms->memmap[VMAPPLE_APV_GFX].base);
+    sysbus_mmio_map(gfx, 1, vms->memmap[VMAPPLE_APV_IOSFC].base);
+    sysbus_connect_irq(gfx, 0, qdev_get_gpio_in(vms->gic, irq_gfx));
+    sysbus_connect_irq(gfx, 1, qdev_get_gpio_in(vms->gic, irq_iosfc));
+    sysbus_realize_and_unref(gfx, &error_fatal);
+}
+
+static void create_aes(VMAppleMachineState *vms, MemoryRegion *mem)
+{
+    int irq = vms->irqmap[VMAPPLE_AES_1];
+    SysBusDevice *aes;
+
+    aes = SYS_BUS_DEVICE(qdev_new(TYPE_APPLE_AES));
+    sysbus_mmio_map(aes, 0, vms->memmap[VMAPPLE_AES_1].base);
+    sysbus_mmio_map(aes, 1, vms->memmap[VMAPPLE_AES_2].base);
+    sysbus_connect_irq(aes, 0, qdev_get_gpio_in(vms->gic, irq));
+    sysbus_realize_and_unref(aes, &error_fatal);
+}
+
+static int arm_gic_ppi_index(int cpu_nr, int ppi_index)
+{
+    return NUM_IRQS + cpu_nr * GIC_INTERNAL + ppi_index;
+}
+
+static void create_gic(VMAppleMachineState *vms, MemoryRegion *mem)
+{
+    MachineState *ms = MACHINE(vms);
+    /* We create a standalone GIC */
+    SysBusDevice *gicbusdev;
+    QList *redist_region_count;
+    int i;
+    unsigned int smp_cpus = ms->smp.cpus;
+
+    vms->gic = qdev_new(gicv3_class_name());
+    qdev_prop_set_uint32(vms->gic, "revision", 3);
+    qdev_prop_set_uint32(vms->gic, "num-cpu", smp_cpus);
+    /*
+     * Note that the num-irq property counts both internal and external
+     * interrupts; there are always 32 of the former (mandated by GIC spec).
+     */
+    qdev_prop_set_uint32(vms->gic, "num-irq", NUM_IRQS + 32);
+
+    uint32_t redist0_capacity =
+                vms->memmap[VMAPPLE_GIC_REDIST].size / GICV3_REDIST_SIZE;
+    uint32_t redist0_count = MIN(smp_cpus, redist0_capacity);
+
+    redist_region_count = qlist_new();
+    qlist_append_int(redist_region_count, redist0_count);
+    qdev_prop_set_array(vms->gic, "redist-region-count", redist_region_count);
+
+    gicbusdev = SYS_BUS_DEVICE(vms->gic);
+    sysbus_realize_and_unref(gicbusdev, &error_fatal);
+    sysbus_mmio_map(gicbusdev, 0, vms->memmap[VMAPPLE_GIC_DIST].base);
+    sysbus_mmio_map(gicbusdev, 1, vms->memmap[VMAPPLE_GIC_REDIST].base);
+
+    /*
+     * Wire the outputs from each CPU's generic timer and the GICv3
+     * maintenance interrupt signal to the appropriate GIC PPI inputs,
+     * and the GIC's IRQ/FIQ/VIRQ/VFIQ interrupt outputs to the CPU's inputs.
+     */
+    for (i = 0; i < smp_cpus; i++) {
+        DeviceState *cpudev = DEVICE(qemu_get_cpu(i));
+
+        /* Map the virt timer to PPI 27 */
+        qdev_connect_gpio_out(cpudev, GTIMER_VIRT,
+                              qdev_get_gpio_in(vms->gic,
+                                               arm_gic_ppi_index(i, 27)));
+
+        /* Map the GIC IRQ and FIQ lines to CPU */
+        sysbus_connect_irq(gicbusdev, i, qdev_get_gpio_in(cpudev, ARM_CPU_IRQ));
+        sysbus_connect_irq(gicbusdev, i + smp_cpus,
+                           qdev_get_gpio_in(cpudev, ARM_CPU_FIQ));
+    }
+}
+
+static void create_uart(const VMAppleMachineState *vms, int uart,
+                        MemoryRegion *mem, Chardev *chr)
+{
+    hwaddr base = vms->memmap[uart].base;
+    int irq = vms->irqmap[uart];
+    DeviceState *dev = qdev_new(TYPE_PL011);
+    SysBusDevice *s = SYS_BUS_DEVICE(dev);
+
+    qdev_prop_set_chr(dev, "chardev", chr);
+    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+    memory_region_add_subregion(mem, base,
+                                sysbus_mmio_get_region(s, 0));
+    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
+}
+
+static void create_rtc(const VMAppleMachineState *vms)
+{
+    hwaddr base = vms->memmap[VMAPPLE_RTC].base;
+    int irq = vms->irqmap[VMAPPLE_RTC];
+
+    sysbus_create_simple("pl031", base, qdev_get_gpio_in(vms->gic, irq));
+}
+
+static DeviceState *gpio_key_dev;
+static void vmapple_powerdown_req(Notifier *n, void *opaque)
+{
+    /* use gpio Pin 3 for power button event */
+    qemu_set_irq(qdev_get_gpio_in(gpio_key_dev, 0), 1);
+}
+
+static void create_gpio_devices(const VMAppleMachineState *vms, int gpio,
+                                MemoryRegion *mem)
+{
+    DeviceState *pl061_dev;
+    hwaddr base = vms->memmap[gpio].base;
+    int irq = vms->irqmap[gpio];
+    SysBusDevice *s;
+
+    pl061_dev = qdev_new("pl061");
+    /* Pull lines down to 0 if not driven by the PL061 */
+    qdev_prop_set_uint32(pl061_dev, "pullups", 0);
+    qdev_prop_set_uint32(pl061_dev, "pulldowns", 0xff);
+    s = SYS_BUS_DEVICE(pl061_dev);
+    sysbus_realize_and_unref(s, &error_fatal);
+    memory_region_add_subregion(mem, base, sysbus_mmio_get_region(s, 0));
+    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
+    gpio_key_dev = sysbus_create_simple("gpio-key", -1,
+                                        qdev_get_gpio_in(pl061_dev, 3));
+}
+
+static void vmapple_firmware_init(VMAppleMachineState *vms,
+                                  MemoryRegion *sysmem)
+{
+    hwaddr size = vms->memmap[VMAPPLE_FIRMWARE].size;
+    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
+    const char *bios_name;
+    int image_size;
+    char *fname;
+
+    bios_name = MACHINE(vms)->firmware;
+    if (!bios_name) {
+        error_report("No firmware specified");
+        exit(1);
+    }
+
+    fname = qemu_find_file(QEMU_FILE_TYPE_BIOS, bios_name);
+    if (!fname) {
+        error_report("Could not find ROM image '%s'", bios_name);
+        exit(1);
+    }
+
+    memory_region_init_ram(&vms->fw_mr, NULL, "firmware", size, &error_fatal);
+    image_size = load_image_mr(fname, &vms->fw_mr);
+
+    g_free(fname);
+    if (image_size < 0) {
+        error_report("Could not load ROM image '%s'", bios_name);
+        exit(1);
+    }
+
+    memory_region_add_subregion(get_system_memory(), base, &vms->fw_mr);
+}
+
+static void create_pcie(VMAppleMachineState *vms)
+{
+    hwaddr base_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].base;
+    hwaddr size_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].size;
+    hwaddr base_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].base;
+    hwaddr size_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].size;
+    int irq = vms->irqmap[VMAPPLE_PCIE];
+    MemoryRegion *mmio_alias;
+    MemoryRegion *mmio_reg;
+    MemoryRegion *ecam_reg;
+    DeviceState *dev;
+    int i;
+    PCIHostState *pci;
+    DeviceState *usb_controller;
+    USBBus *usb_bus;
+
+    dev = qdev_new(TYPE_GPEX_HOST);
+    qdev_prop_set_uint32(dev, "num-irqs", GPEX_NUM_IRQS);
+    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+
+    /* Map only the first size_ecam bytes of ECAM space */
+    ecam_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0);
+    memory_region_init_alias(&vms->ecam_alias, OBJECT(dev), "pcie-ecam",
+                             ecam_reg, 0, size_ecam);
+    memory_region_add_subregion(get_system_memory(), base_ecam,
+                                &vms->ecam_alias);
+
+    /*
+     * Map the MMIO window from [0x50000000-0x7fff0000] in PCI space into
+     * system address space at [0x50000000-0x7fff0000].
+     */
+    mmio_alias = g_new0(MemoryRegion, 1);
+    mmio_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 1);
+    memory_region_init_alias(mmio_alias, OBJECT(dev), "pcie-mmio",
+                             mmio_reg, base_mmio, size_mmio);
+    memory_region_add_subregion(get_system_memory(), base_mmio, mmio_alias);
+
+    for (i = 0; i < GPEX_NUM_IRQS; i++) {
+        sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
+                           qdev_get_gpio_in(vms->gic, irq + i));
+        gpex_set_irq_num(GPEX_HOST(dev), i, irq + i);
+    }
+
+    pci = PCI_HOST_BRIDGE(dev);
+    vms->bus = pci->bus;
+    g_assert(vms->bus);
+
+    while ((dev = qemu_create_nic_device("virtio-net-pci", true, NULL))) {
+        qdev_realize_and_unref(dev, BUS(vms->bus), &error_fatal);
+    }
+
+    usb_controller = qdev_new(TYPE_QEMU_XHCI);
+    qdev_realize_and_unref(usb_controller, BUS(pci->bus), &error_fatal);
+
+    usb_bus = USB_BUS(object_resolve_type_unambiguous(TYPE_USB_BUS,
+                                                      &error_fatal));
+    usb_create_simple(usb_bus, "usb-kbd");
+    usb_create_simple(usb_bus, "usb-tablet");
+}
+
+static void vmapple_reset(void *opaque)
+{
+    VMAppleMachineState *vms = opaque;
+    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
+
+    cpu_set_pc(first_cpu, base);
+}
+
+static void mach_vmapple_init(MachineState *machine)
+{
+    VMAppleMachineState *vms = VMAPPLE_MACHINE(machine);
+    MachineClass *mc = MACHINE_GET_CLASS(machine);
+    const CPUArchIdList *possible_cpus;
+    MemoryRegion *sysmem = get_system_memory();
+    int n;
+    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int max_cpus = machine->smp.max_cpus;
+
+    vms->memmap = memmap;
+    machine->usb = true;
+
+    possible_cpus = mc->possible_cpu_arch_ids(machine);
+    assert(possible_cpus->len == max_cpus);
+    for (n = 0; n < possible_cpus->len; n++) {
+        Object *cpu;
+        CPUState *cs;
+
+        if (n >= smp_cpus) {
+            break;
+        }
+
+        cpu = object_new(possible_cpus->cpus[n].type);
+        object_property_set_int(cpu, "mp-affinity",
+                                possible_cpus->cpus[n].arch_id, &error_fatal);
+
+        cs = CPU(cpu);
+        cs->cpu_index = n;
+
+        numa_cpu_pre_plug(&possible_cpus->cpus[cs->cpu_index], DEVICE(cpu),
+                          &error_fatal);
+
+        if (object_property_find(cpu, "has_el3")) {
+            object_property_set_bool(cpu, "has_el3", false, &error_fatal);
+        }
+        if (object_property_find(cpu, "has_el2")) {
+            object_property_set_bool(cpu, "has_el2", false, &error_fatal);
+        }
+        object_property_set_int(cpu, "psci-conduit", QEMU_PSCI_CONDUIT_HVC,
+                                &error_fatal);
+
+        /* Secondary CPUs start in PSCI powered-down state */
+        if (n > 0) {
+            object_property_set_bool(cpu, "start-powered-off", true,
+                                     &error_fatal);
+        }
+
+        object_property_set_link(cpu, "memory", OBJECT(sysmem), &error_abort);
+        qdev_realize(DEVICE(cpu), NULL, &error_fatal);
+        object_unref(cpu);
+    }
+
+    memory_region_add_subregion(sysmem, vms->memmap[VMAPPLE_MEM].base,
+                                machine->ram);
+
+    create_gic(vms, sysmem);
+    create_bdif(vms, sysmem);
+    create_pvpanic(vms, sysmem);
+    create_aes(vms, sysmem);
+    create_gfx(vms, sysmem);
+    create_uart(vms, VMAPPLE_UART, sysmem, serial_hd(0));
+    create_rtc(vms);
+    create_pcie(vms);
+
+    create_gpio_devices(vms, VMAPPLE_GPIO, sysmem);
+
+    vmapple_firmware_init(vms, sysmem);
+    create_cfg(vms, sysmem);
+
+    /* connect powerdown request */
+    vms->powerdown_notifier.notify = vmapple_powerdown_req;
+    qemu_register_powerdown_notifier(&vms->powerdown_notifier);
+
+    vms->bootinfo.ram_size = machine->ram_size;
+    vms->bootinfo.board_id = -1;
+    vms->bootinfo.loader_start = vms->memmap[VMAPPLE_MEM].base;
+    vms->bootinfo.skip_dtb_autoload = true;
+    vms->bootinfo.firmware_loaded = true;
+    arm_load_kernel(ARM_CPU(first_cpu), machine, &vms->bootinfo);
+
+    qemu_register_reset(vmapple_reset, vms);
+}
+
+static CpuInstanceProperties
+vmapple_cpu_index_to_props(MachineState *ms, unsigned cpu_index)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    const CPUArchIdList *possible_cpus = mc->possible_cpu_arch_ids(ms);
+
+    assert(cpu_index < possible_cpus->len);
+    return possible_cpus->cpus[cpu_index].props;
+}
+
+
+static int64_t vmapple_get_default_cpu_node_id(const MachineState *ms, int idx)
+{
+    return idx % ms->numa_state->num_nodes;
+}
+
+static const CPUArchIdList *vmapple_possible_cpu_arch_ids(MachineState *ms)
+{
+    int n;
+    unsigned int max_cpus = ms->smp.max_cpus;
+
+    if (ms->possible_cpus) {
+        assert(ms->possible_cpus->len == max_cpus);
+        return ms->possible_cpus;
+    }
+
+    ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
+                                  sizeof(CPUArchId) * max_cpus);
+    ms->possible_cpus->len = max_cpus;
+    for (n = 0; n < ms->possible_cpus->len; n++) {
+        ms->possible_cpus->cpus[n].type = ms->cpu_type;
+        ms->possible_cpus->cpus[n].arch_id =
+            arm_build_mp_affinity(n, GICV3_TARGETLIST_BITS);
+        ms->possible_cpus->cpus[n].props.has_thread_id = true;
+        ms->possible_cpus->cpus[n].props.thread_id = n;
+    }
+    return ms->possible_cpus;
+}
+
+static void vmapple_machine_class_init(ObjectClass *oc, void *data)
+{
+    MachineClass *mc = MACHINE_CLASS(oc);
+
+    mc->init = mach_vmapple_init;
+    mc->max_cpus = 32;
+    mc->block_default_type = IF_VIRTIO;
+    mc->no_cdrom = 1;
+    mc->pci_allow_0_address = true;
+    mc->minimum_page_bits = 12;
+    mc->possible_cpu_arch_ids = vmapple_possible_cpu_arch_ids;
+    mc->cpu_index_to_instance_props = vmapple_cpu_index_to_props;
+    mc->default_cpu_type = ARM_CPU_TYPE_NAME("host");
+    mc->get_default_cpu_node_id = vmapple_get_default_cpu_node_id;
+    mc->default_ram_id = "mach-vmapple.ram";
+
+    object_register_sugar_prop(TYPE_VIRTIO_PCI, "disable-legacy",
+                               "on", true);
+}
+
+static void vmapple_instance_init(Object *obj)
+{
+    VMAppleMachineState *vms = VMAPPLE_MACHINE(obj);
+
+    vms->irqmap = irqmap;
+
+    object_property_add_uint64_ptr(obj, "uuid", &vms->uuid,
+                                   OBJ_PROP_FLAG_READWRITE);
+    object_property_set_description(obj, "uuid", "Machine UUID (SDOM)");
+}
+
+static const TypeInfo vmapple_machine_info = {
+    .name          = TYPE_VMAPPLE_MACHINE,
+    .parent        = TYPE_MACHINE,
+    .abstract      = true,
+    .instance_size = sizeof(VMAppleMachineState),
+    .class_size    = sizeof(VMAppleMachineClass),
+    .class_init    = vmapple_machine_class_init,
+    .instance_init = vmapple_instance_init,
+};
+
+static void machvmapple_machine_init(void)
+{
+    type_register_static(&vmapple_machine_info);
+}
+type_init(machvmapple_machine_init);
+
+static void vmapple_machine_9_2_options(MachineClass *mc)
+{
+}
+DEFINE_VMAPPLE_MACHINE_AS_LATEST(9, 2)
+
-- 
2.39.3 (Apple Git-145)


Re: [PATCH v8 15/15] hw/vmapple/vmapple: Add vmapple machine type
Posted by Akihiko Odaki 1 week, 6 days ago
On 2024/11/08 23:47, Phil Dennis-Jordan wrote:
> From: Alexander Graf <graf@amazon.com>
> 
> Apple defines a new "vmapple" machine type as part of its proprietary
> macOS Virtualization.Framework vmm. This machine type is similar to the
> virt one, but with subtle differences in base devices, a few special
> vmapple device additions and a vastly different boot chain.
> 
> This patch reimplements this machine type in QEMU. To use it, you
> have to have a readily installed version of macOS for VMApple,
> run on macOS with -accel hvf, pass the Virtualization.Framework
> boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
> and pass aux and root volume as virtio drives. In addition, you also
> need to find the machine UUID and pass that as -M vmapple,uuid= parameter:
> 
> $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
>      -bios /System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
>      -drive file=aux,if=pflash,format=raw \
>      -drive file=root,if=pflash,format=raw \
>      -drive file=aux,if=none,id=aux,format=raw \
>      -device vmapple-virtio-aux,drive=aux \
>      -drive file=root,if=none,id=root,format=raw \
>      -device vmapple-virtio-root,drive=root
> 
> With all these in place, you should be able to see macOS booting
> successfully.
> 
> Known issues:
>   - Keyboard and mouse/tablet input is laggy. The reason for this is
>     either that macOS's XHCI driver is broken when the device/platform
>     does not support MSI/MSI-X, or there's some unfortunate interplay
>     with Qemu's XHCI implementation in this scenario.
>   - Currently only macOS 12 guests are supported. The boot process for
>     13+ will need further investigation and adjustment.
> 
> Signed-off-by: Alexander Graf <graf@amazon.com>
> Co-authored-by: Phil Dennis-Jordan <phil@philjordan.eu>
> Signed-off-by: Phil Dennis-Jordan <phil@philjordan.eu>
> ---
> 
> v3:
>   * Rebased on latest upstream, updated affinity and NIC creation
>     API usage
>   * Included Apple-variant virtio-blk in build dependency
>   * Updated API usage for setting 'redist-region-count' array-typed property
>     on GIC.
>   * Switched from virtio HID devices (for which macOS 12 does not contain
>     drivers) to an XHCI USB controller and USB HID devices.
> 
> v4:
>   * Fixups for v4 changes to the other patches in the set.
>   * Corrected the assert macro to use
>   * Removed superfluous endian conversions corresponding to cfg's.
>   * Init error handling improvement.
>   * No need to select CPU type on TCG, as only HVF is supported.
>   * Machine type version bumped to 9.2
>   * #include order improved
> 
> v5:
>   * Fixed memory reservation for ecam alias region.
>   * Better error handling setting properties on devices.
>   * Simplified the machine ECID/UUID extraction script and actually created a
>     file for it rather than quoting its code in documentation.
> 
> v7:
>   * Tiny error handling fix, un-inlined function.
> 
> v8:
>   * Use object_property_add_uint64_ptr rather than defining custom UUID
>     property get/set functions.
> 
>   MAINTAINERS                 |   1 +
>   contrib/vmapple/uuid.sh     |   9 +
>   docs/system/arm/vmapple.rst |  60 ++++
>   docs/system/target-arm.rst  |   1 +
>   hw/vmapple/Kconfig          |  20 ++
>   hw/vmapple/meson.build      |   1 +
>   hw/vmapple/vmapple.c        | 638 ++++++++++++++++++++++++++++++++++++
>   7 files changed, 730 insertions(+)
>   create mode 100755 contrib/vmapple/uuid.sh
>   create mode 100644 docs/system/arm/vmapple.rst
>   create mode 100644 hw/vmapple/vmapple.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 9d0d26cb65f..9591fd44a34 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2767,6 +2767,7 @@ R: Phil Dennis-Jordan <phil@philjordan.eu>
>   S: Maintained
>   F: hw/vmapple/*
>   F: include/hw/vmapple/*
> +F: docs/system/arm/vmapple.rst
>   
>   Subsystems
>   ----------
> diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.sh
> new file mode 100755
> index 00000000000..956e8c3afed
> --- /dev/null
> +++ b/contrib/vmapple/uuid.sh
> @@ -0,0 +1,9 @@
> +#!/bin/sh
> +# Used for converting a guest provisioned using Virtualization.framework
> +# for use with the QEMU 'vmapple' aarch64 machine type.
> +#
> +# Extracts the Machine UUID from Virtualization.framework VM JSON file.
> +# (as produced by 'macosvm', passed as command line argument)
> +
> +plutil -extract machineId raw "$1" | base64 -d | plutil -extract ECID raw -
> +
> diff --git a/docs/system/arm/vmapple.rst b/docs/system/arm/vmapple.rst
> new file mode 100644
> index 00000000000..6a634fa4572
> --- /dev/null
> +++ b/docs/system/arm/vmapple.rst
> @@ -0,0 +1,60 @@
> +VMApple machine emulation
> +========================================================================================
> +
> +VMApple is the device model that the macOS built-in hypervisor called "Virtualization.framework"
> +exposes to Apple Silicon macOS guests. The "vmapple" machine model in QEMU implements the same
> +device model, but does not use any code from Virtualization.Framework.
> +
> +Prerequisites
> +-------------
> +
> +To run the vmapple machine model, you need to
> +
> + * Run on Apple Silicon
> + * Run on macOS 12.0 or above
> + * Have an already installed copy of a Virtualization.Framework macOS 12 virtual machine. I will
> +   assume that you installed it using the macosvm CLI.

Add a URL to macosvm.

> +
> +First, we need to extract the UUID from the virtual machine that you installed. You can do this
> +by running the shell script in contrib/vmapple/uuid.sh on the macosvm.json file.
> +
> +.. code-block:: bash
> +  :caption: uuid.sh script to extract the UUID from a macosvm.json file
> +
> +  $ contrib/vmapple/uuid.sh "path/to/macosvm.json"
> +
> +Now we also need to trim the aux partition. It contains metadata that we can just discard:
> +
> +.. code-block:: bash
> +  :caption: Command to trim the aux file
> +
> +  $ dd if="aux.img" of="aux.img.trimmed" bs=$(( 0x4000 )) skip=1

Quoting is inconsistent. aux.img.trimmed is not quoted below but it is 
quoted here.

> +
> +How to run
> +----------
> +
> +Then, we can launch QEMU with the Virtualization.Framework pre-boot environment and the readily
> +installed target disk images. I recommend to port forward the VM's ssh and vnc ports to the host
> +to get better interactive access into the target system:
> +
> +.. code-block:: bash
> +  :caption: Example execution command line
> +
> +  $ UUID=$(uuid.sh macosvm.json)
> +  $ AVPBOOTER=/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin
> +  $ AUX=aux.img.trimmed
> +  $ DISK=disk.img
> +  $ qemu-system-aarch64 \
> +       -serial mon:stdio \
> +       -m 4G \
> +       -accel hvf \
> +       -M vmapple,uuid=$UUID \
> +       -bios $AVPBOOTER \

$AUX and $DISK are quoted but $UUID and $AVPBOOTER are not.

> +        -drive file="$AUX",if=pflash,format=raw \
> +        -drive file="$DISK",if=pflash,format=raw \

These two lines are misaligned.

> +       -drive file="$AUX",if=none,id=aux,format=raw \
> +       -drive file="$DISK",if=none,id=root,format=raw \
> +       -device vmapple-virtio-aux,drive=aux \
> +       -device vmapple-virtio-root,drive=root \
> +       -net user,ipv6=off,hostfwd=tcp::2222-:22,hostfwd=tcp::5901-:5900 \
> +       -net nic,model=virtio-net-pci \

-net is a legacy option and creates a unnecessary indirection with hub. 
Use -netdev and -device options. Also plug virtio-net-pci into a PCIe 
root port as suggested in: docs/pcie.txt

I thought it is about time to give a try on my M2 MacBook Air, but this 
command line did not work. The serial output said iBootStage2 finished, 
but it went silent after that and the display gives not output. macosvm 
can run the VM image. I used: UniversalMac_15.1_24B83_Restore.ipsw

Do you have an idea what's wrong with my setup? Doesn't it work with 15.1?

> diff --git a/docs/system/target-arm.rst b/docs/system/target-arm.rst
> index 9aaa9c414c9..3426f79100b 100644
> --- a/docs/system/target-arm.rst
> +++ b/docs/system/target-arm.rst
> @@ -102,6 +102,7 @@ Board-specific documentation
>      arm/stellaris
>      arm/stm32
>      arm/virt
> +   arm/vmapple
>      arm/xenpvh
>      arm/xlnx-versal-virt
>      arm/xlnx-zynq
> diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
> index bcd1be63e3c..6a4c4a7fa2e 100644
> --- a/hw/vmapple/Kconfig
> +++ b/hw/vmapple/Kconfig
> @@ -10,3 +10,23 @@ config VMAPPLE_CFG
>   config VMAPPLE_VIRTIO_BLK
>       bool
>   
> +config VMAPPLE
> +    bool
> +    depends on ARM
> +    depends on HVF
> +    default y if ARM
> +    imply PCI_DEVICES
> +    select ARM_GIC
> +    select PLATFORM_BUS
> +    select PCI_EXPRESS
> +    select PCI_EXPRESS_GENERIC_BRIDGE
> +    select PL011 # UART
> +    select PL031 # RTC
> +    select PL061 # GPIO
> +    select GPIO_PWR
> +    select PVPANIC_MMIO
> +    select VMAPPLE_AES
> +    select VMAPPLE_BDIF
> +    select VMAPPLE_CFG
> +    select MAC_PVG_MMIO
> +    select VMAPPLE_VIRTIO_BLK
> diff --git a/hw/vmapple/meson.build b/hw/vmapple/meson.build
> index bf17cf906c9..e572f7d5602 100644
> --- a/hw/vmapple/meson.build
> +++ b/hw/vmapple/meson.build
> @@ -2,3 +2,4 @@ system_ss.add(when: 'CONFIG_VMAPPLE_AES',  if_true: files('aes.c'))
>   system_ss.add(when: 'CONFIG_VMAPPLE_BDIF', if_true: files('bdif.c'))
>   system_ss.add(when: 'CONFIG_VMAPPLE_CFG',  if_true: files('cfg.c'))
>   system_ss.add(when: 'CONFIG_VMAPPLE_VIRTIO_BLK',  if_true: files('virtio-blk.c'))
> +specific_ss.add(when: 'CONFIG_VMAPPLE',     if_true: files('vmapple.c'))
> diff --git a/hw/vmapple/vmapple.c b/hw/vmapple/vmapple.c
> new file mode 100644
> index 00000000000..8b8710c70a2
> --- /dev/null
> +++ b/hw/vmapple/vmapple.c
> @@ -0,0 +1,638 @@
> +/*
> + * VMApple machine emulation
> + *
> + * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + *
> + * VMApple is the device model that the macOS built-in hypervisor called
> + * "Virtualization.framework" exposes to Apple Silicon macOS guests. The
> + * machine model in this file implements the same device model in QEMU, but
> + * does not use any code from Virtualization.Framework.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/bitops.h"
> +#include "qemu/datadir.h"
> +#include "qemu/error-report.h"
> +#include "qemu/guest-random.h"
> +#include "qemu/help-texts.h"
> +#include "qemu/log.h"
> +#include "qemu/module.h"
> +#include "qemu/option.h"
> +#include "qemu/units.h"
> +#include "monitor/qdev.h"
> +#include "hw/boards.h"
> +#include "hw/irq.h"
> +#include "hw/loader.h"
> +#include "hw/qdev-properties.h"
> +#include "hw/sysbus.h"
> +#include "hw/usb.h"
> +#include "hw/arm/boot.h"
> +#include "hw/arm/primecell.h"
> +#include "hw/char/pl011.h"
> +#include "hw/intc/arm_gic.h"
> +#include "hw/intc/arm_gicv3_common.h"
> +#include "hw/misc/pvpanic.h"
> +#include "hw/pci-host/gpex.h"
> +#include "hw/usb/xhci.h"
> +#include "hw/virtio/virtio-pci.h"
> +#include "hw/vmapple/vmapple.h"
> +#include "net/net.h"
> +#include "qapi/error.h"
> +#include "qapi/qmp/qlist.h"
> +#include "qapi/visitor.h"
> +#include "qapi/qapi-visit-common.h"
> +#include "standard-headers/linux/input.h"
> +#include "sysemu/hvf.h"
> +#include "sysemu/kvm.h"
> +#include "sysemu/reset.h"
> +#include "sysemu/runstate.h"
> +#include "sysemu/sysemu.h"
> +#include "target/arm/internals.h"
> +#include "target/arm/kvm_arm.h"
> +
> +struct VMAppleMachineClass {
> +    MachineClass parent;
> +};
> +
> +struct VMAppleMachineState {
> +    MachineState parent;
> +
> +    Notifier machine_done;
> +    struct arm_boot_info bootinfo;
> +    MemMapEntry *memmap;
> +    const int *irqmap;
> +    DeviceState *gic;
> +    DeviceState *cfg;
> +    Notifier powerdown_notifier;
> +    PCIBus *bus;
> +    MemoryRegion fw_mr;
> +    MemoryRegion ecam_alias;
> +    uint64_t uuid;
> +};
> +
> +#define DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, latest) \
> +    static void vmapple##major##_##minor##_class_init(ObjectClass *oc, \
> +                                                    void *data) \
> +    { \
> +        MachineClass *mc = MACHINE_CLASS(oc); \
> +        vmapple_machine_##major##_##minor##_options(mc); \
> +        mc->desc = "QEMU " # major "." # minor " Apple Virtual Machine"; \
> +        if (latest) { \
> +            mc->alias = "vmapple"; \
> +        } \
> +    } \
> +    static const TypeInfo machvmapple##major##_##minor##_info = { \
> +        .name = MACHINE_TYPE_NAME("vmapple-" # major "." # minor), \
> +        .parent = TYPE_VMAPPLE_MACHINE, \
> +        .class_init = vmapple##major##_##minor##_class_init, \
> +    }; \
> +    static void machvmapple_machine_##major##_##minor##_init(void) \
> +    { \
> +        type_register_static(&machvmapple##major##_##minor##_info); \
> +    } \
> +    type_init(machvmapple_machine_##major##_##minor##_init);
> +
> +#define DEFINE_VMAPPLE_MACHINE_AS_LATEST(major, minor) \
> +    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, true)
> +#define DEFINE_VMAPPLE_MACHINE(major, minor) \
> +    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, false)
> +
> +#define TYPE_VMAPPLE_MACHINE   MACHINE_TYPE_NAME("vmapple")
> +OBJECT_DECLARE_TYPE(VMAppleMachineState, VMAppleMachineClass, VMAPPLE_MACHINE)
> +
> +/* Number of external interrupt lines to configure the GIC with */
> +#define NUM_IRQS 256
> +
> +enum {
> +    VMAPPLE_FIRMWARE,
> +    VMAPPLE_CONFIG,
> +    VMAPPLE_MEM,
> +    VMAPPLE_GIC_DIST,
> +    VMAPPLE_GIC_REDIST,
> +    VMAPPLE_UART,
> +    VMAPPLE_RTC,
> +    VMAPPLE_PCIE,
> +    VMAPPLE_PCIE_MMIO,
> +    VMAPPLE_PCIE_ECAM,
> +    VMAPPLE_GPIO,
> +    VMAPPLE_PVPANIC,
> +    VMAPPLE_APV_GFX,
> +    VMAPPLE_APV_IOSFC,
> +    VMAPPLE_AES_1,
> +    VMAPPLE_AES_2,
> +    VMAPPLE_BDOOR,
> +    VMAPPLE_MEMMAP_LAST,
> +};
> +
> +static MemMapEntry memmap[] = {
> +    [VMAPPLE_FIRMWARE] =           { 0x00100000, 0x00100000 },
> +    [VMAPPLE_CONFIG] =             { 0x00400000, 0x00010000 },
> +
> +    [VMAPPLE_GIC_DIST] =           { 0x10000000, 0x00010000 },
> +    [VMAPPLE_GIC_REDIST] =         { 0x10010000, 0x00400000 },
> +
> +    [VMAPPLE_UART] =               { 0x20010000, 0x00010000 },
> +    [VMAPPLE_RTC] =                { 0x20050000, 0x00001000 },
> +    [VMAPPLE_GPIO] =               { 0x20060000, 0x00001000 },
> +    [VMAPPLE_PVPANIC] =            { 0x20070000, 0x00000002 },
> +    [VMAPPLE_BDOOR] =              { 0x30000000, 0x00200000 },
> +    [VMAPPLE_APV_GFX] =            { 0x30200000, 0x00010000 },
> +    [VMAPPLE_APV_IOSFC] =          { 0x30210000, 0x00010000 },
> +    [VMAPPLE_AES_1] =              { 0x30220000, 0x00004000 },
> +    [VMAPPLE_AES_2] =              { 0x30230000, 0x00004000 },
> +    [VMAPPLE_PCIE_ECAM] =          { 0x40000000, 0x10000000 },
> +    [VMAPPLE_PCIE_MMIO] =          { 0x50000000, 0x1fff0000 },
> +
> +    /* Actual RAM size depends on configuration */
> +    [VMAPPLE_MEM] =                { 0x70000000ULL, GiB},
> +};
> +
> +static const int irqmap[] = {
> +    [VMAPPLE_UART] = 1,
> +    [VMAPPLE_RTC] = 2,
> +    [VMAPPLE_GPIO] = 0x5,
> +    [VMAPPLE_APV_IOSFC] = 0x10,
> +    [VMAPPLE_APV_GFX] = 0x11,
> +    [VMAPPLE_AES_1] = 0x12,
> +    [VMAPPLE_PCIE] = 0x20,
> +};
> +
> +#define GPEX_NUM_IRQS 16
> +
> +static void create_bdif(VMAppleMachineState *vms, MemoryRegion *mem)
> +{
> +    DeviceState *bdif;
> +    SysBusDevice *bdif_sb;
> +    DriveInfo *di_aux = drive_get(IF_PFLASH, 0, 0);
> +    DriveInfo *di_root = drive_get(IF_PFLASH, 0, 1);
> +
> +    if (!di_aux) {
> +        error_report("No AUX device. Please specify one as pflash drive.");
> +        exit(1);
> +    }
> +
> +    if (!di_root) {
> +        /* Fall back to the first IF_VIRTIO device as root device */
> +        di_root = drive_get(IF_VIRTIO, 0, 0);
> +    }
> +
> +    if (!di_root) {
> +        error_report("No root device. Please specify one as virtio drive.");
> +        exit(1);
> +    }
> +
> +    /* PV backdoor device */
> +    bdif = qdev_new(TYPE_VMAPPLE_BDIF);
> +    bdif_sb = SYS_BUS_DEVICE(bdif);
> +    sysbus_mmio_map(bdif_sb, 0, vms->memmap[VMAPPLE_BDOOR].base);
> +
> +    qdev_prop_set_drive(DEVICE(bdif), "aux", blk_by_legacy_dinfo(di_aux));
> +    qdev_prop_set_drive(DEVICE(bdif), "root", blk_by_legacy_dinfo(di_root));
> +
> +    sysbus_realize_and_unref(bdif_sb, &error_fatal);
> +}
> +
> +static void create_pvpanic(VMAppleMachineState *vms, MemoryRegion *mem)
> +{
> +    SysBusDevice *cfg;
> +
> +    vms->cfg = qdev_new(TYPE_PVPANIC_MMIO_DEVICE);
> +    cfg = SYS_BUS_DEVICE(vms->cfg);
> +    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_PVPANIC].base);
> +
> +    sysbus_realize_and_unref(cfg, &error_fatal);
> +}
> +
> +static void create_cfg(VMAppleMachineState *vms, MemoryRegion *mem)
> +{
> +    SysBusDevice *cfg;
> +    MachineState *machine = MACHINE(vms);
> +    uint32_t rnd = 1;
> +
> +    vms->cfg = qdev_new(TYPE_VMAPPLE_CFG);
> +    cfg = SYS_BUS_DEVICE(vms->cfg);
> +    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_CONFIG].base);
> +
> +    qemu_guest_getrandom_nofail(&rnd, sizeof(rnd));
> +
> +    qdev_prop_set_uint32(vms->cfg, "nr-cpus", machine->smp.cpus);
> +    qdev_prop_set_uint64(vms->cfg, "ecid", vms->uuid);
> +    qdev_prop_set_uint64(vms->cfg, "ram-size", machine->ram_size);
> +    qdev_prop_set_uint32(vms->cfg, "rnd", rnd);
> +
> +    sysbus_realize_and_unref(cfg, &error_fatal);
> +}
> +
> +static void create_gfx(VMAppleMachineState *vms, MemoryRegion *mem)
> +{
> +    int irq_gfx = vms->irqmap[VMAPPLE_APV_GFX];
> +    int irq_iosfc = vms->irqmap[VMAPPLE_APV_IOSFC];
> +    SysBusDevice *gfx;
> +
> +    gfx = SYS_BUS_DEVICE(qdev_new("apple-gfx-mmio"));
> +    sysbus_mmio_map(gfx, 0, vms->memmap[VMAPPLE_APV_GFX].base);
> +    sysbus_mmio_map(gfx, 1, vms->memmap[VMAPPLE_APV_IOSFC].base);
> +    sysbus_connect_irq(gfx, 0, qdev_get_gpio_in(vms->gic, irq_gfx));
> +    sysbus_connect_irq(gfx, 1, qdev_get_gpio_in(vms->gic, irq_iosfc));
> +    sysbus_realize_and_unref(gfx, &error_fatal);
> +}
> +
> +static void create_aes(VMAppleMachineState *vms, MemoryRegion *mem)
> +{
> +    int irq = vms->irqmap[VMAPPLE_AES_1];
> +    SysBusDevice *aes;
> +
> +    aes = SYS_BUS_DEVICE(qdev_new(TYPE_APPLE_AES));
> +    sysbus_mmio_map(aes, 0, vms->memmap[VMAPPLE_AES_1].base);
> +    sysbus_mmio_map(aes, 1, vms->memmap[VMAPPLE_AES_2].base);
> +    sysbus_connect_irq(aes, 0, qdev_get_gpio_in(vms->gic, irq));
> +    sysbus_realize_and_unref(aes, &error_fatal);
> +}
> +
> +static int arm_gic_ppi_index(int cpu_nr, int ppi_index)
> +{
> +    return NUM_IRQS + cpu_nr * GIC_INTERNAL + ppi_index;
> +}
> +
> +static void create_gic(VMAppleMachineState *vms, MemoryRegion *mem)
> +{
> +    MachineState *ms = MACHINE(vms);
> +    /* We create a standalone GIC */
> +    SysBusDevice *gicbusdev;
> +    QList *redist_region_count;
> +    int i;
> +    unsigned int smp_cpus = ms->smp.cpus;
> +
> +    vms->gic = qdev_new(gicv3_class_name());
> +    qdev_prop_set_uint32(vms->gic, "revision", 3);
> +    qdev_prop_set_uint32(vms->gic, "num-cpu", smp_cpus);
> +    /*
> +     * Note that the num-irq property counts both internal and external
> +     * interrupts; there are always 32 of the former (mandated by GIC spec).
> +     */
> +    qdev_prop_set_uint32(vms->gic, "num-irq", NUM_IRQS + 32);
> +
> +    uint32_t redist0_capacity =
> +                vms->memmap[VMAPPLE_GIC_REDIST].size / GICV3_REDIST_SIZE;
> +    uint32_t redist0_count = MIN(smp_cpus, redist0_capacity);
> +
> +    redist_region_count = qlist_new();
> +    qlist_append_int(redist_region_count, redist0_count);
> +    qdev_prop_set_array(vms->gic, "redist-region-count", redist_region_count);
> +
> +    gicbusdev = SYS_BUS_DEVICE(vms->gic);
> +    sysbus_realize_and_unref(gicbusdev, &error_fatal);
> +    sysbus_mmio_map(gicbusdev, 0, vms->memmap[VMAPPLE_GIC_DIST].base);
> +    sysbus_mmio_map(gicbusdev, 1, vms->memmap[VMAPPLE_GIC_REDIST].base);
> +
> +    /*
> +     * Wire the outputs from each CPU's generic timer and the GICv3
> +     * maintenance interrupt signal to the appropriate GIC PPI inputs,
> +     * and the GIC's IRQ/FIQ/VIRQ/VFIQ interrupt outputs to the CPU's inputs.
> +     */
> +    for (i = 0; i < smp_cpus; i++) {
> +        DeviceState *cpudev = DEVICE(qemu_get_cpu(i));
> +
> +        /* Map the virt timer to PPI 27 */
> +        qdev_connect_gpio_out(cpudev, GTIMER_VIRT,
> +                              qdev_get_gpio_in(vms->gic,
> +                                               arm_gic_ppi_index(i, 27)));
> +
> +        /* Map the GIC IRQ and FIQ lines to CPU */
> +        sysbus_connect_irq(gicbusdev, i, qdev_get_gpio_in(cpudev, ARM_CPU_IRQ));
> +        sysbus_connect_irq(gicbusdev, i + smp_cpus,
> +                           qdev_get_gpio_in(cpudev, ARM_CPU_FIQ));
> +    }
> +}
> +
> +static void create_uart(const VMAppleMachineState *vms, int uart,
> +                        MemoryRegion *mem, Chardev *chr)
> +{
> +    hwaddr base = vms->memmap[uart].base;
> +    int irq = vms->irqmap[uart];
> +    DeviceState *dev = qdev_new(TYPE_PL011);
> +    SysBusDevice *s = SYS_BUS_DEVICE(dev);
> +
> +    qdev_prop_set_chr(dev, "chardev", chr);
> +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> +    memory_region_add_subregion(mem, base,
> +                                sysbus_mmio_get_region(s, 0));
> +    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
> +}
> +
> +static void create_rtc(const VMAppleMachineState *vms)
> +{
> +    hwaddr base = vms->memmap[VMAPPLE_RTC].base;
> +    int irq = vms->irqmap[VMAPPLE_RTC];
> +
> +    sysbus_create_simple("pl031", base, qdev_get_gpio_in(vms->gic, irq));
> +}
> +
> +static DeviceState *gpio_key_dev;
> +static void vmapple_powerdown_req(Notifier *n, void *opaque)
> +{
> +    /* use gpio Pin 3 for power button event */
> +    qemu_set_irq(qdev_get_gpio_in(gpio_key_dev, 0), 1);
> +}
> +
> +static void create_gpio_devices(const VMAppleMachineState *vms, int gpio,
> +                                MemoryRegion *mem)
> +{
> +    DeviceState *pl061_dev;
> +    hwaddr base = vms->memmap[gpio].base;
> +    int irq = vms->irqmap[gpio];
> +    SysBusDevice *s;
> +
> +    pl061_dev = qdev_new("pl061");
> +    /* Pull lines down to 0 if not driven by the PL061 */
> +    qdev_prop_set_uint32(pl061_dev, "pullups", 0);
> +    qdev_prop_set_uint32(pl061_dev, "pulldowns", 0xff);
> +    s = SYS_BUS_DEVICE(pl061_dev);
> +    sysbus_realize_and_unref(s, &error_fatal);
> +    memory_region_add_subregion(mem, base, sysbus_mmio_get_region(s, 0));
> +    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
> +    gpio_key_dev = sysbus_create_simple("gpio-key", -1,
> +                                        qdev_get_gpio_in(pl061_dev, 3));
> +}
> +
> +static void vmapple_firmware_init(VMAppleMachineState *vms,
> +                                  MemoryRegion *sysmem)
> +{
> +    hwaddr size = vms->memmap[VMAPPLE_FIRMWARE].size;
> +    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
> +    const char *bios_name;
> +    int image_size;
> +    char *fname;
> +
> +    bios_name = MACHINE(vms)->firmware;
> +    if (!bios_name) {
> +        error_report("No firmware specified");
> +        exit(1);
> +    }
> +
> +    fname = qemu_find_file(QEMU_FILE_TYPE_BIOS, bios_name);
> +    if (!fname) {
> +        error_report("Could not find ROM image '%s'", bios_name);
> +        exit(1);
> +    }
> +
> +    memory_region_init_ram(&vms->fw_mr, NULL, "firmware", size, &error_fatal);
> +    image_size = load_image_mr(fname, &vms->fw_mr);
> +
> +    g_free(fname);
> +    if (image_size < 0) {
> +        error_report("Could not load ROM image '%s'", bios_name);
> +        exit(1);
> +    }
> +
> +    memory_region_add_subregion(get_system_memory(), base, &vms->fw_mr);
> +}
> +
> +static void create_pcie(VMAppleMachineState *vms)
> +{
> +    hwaddr base_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].base;
> +    hwaddr size_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].size;
> +    hwaddr base_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].base;
> +    hwaddr size_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].size;
> +    int irq = vms->irqmap[VMAPPLE_PCIE];
> +    MemoryRegion *mmio_alias;
> +    MemoryRegion *mmio_reg;
> +    MemoryRegion *ecam_reg;
> +    DeviceState *dev;
> +    int i;
> +    PCIHostState *pci;
> +    DeviceState *usb_controller;
> +    USBBus *usb_bus;
> +
> +    dev = qdev_new(TYPE_GPEX_HOST);
> +    qdev_prop_set_uint32(dev, "num-irqs", GPEX_NUM_IRQS);
> +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> +
> +    /* Map only the first size_ecam bytes of ECAM space */
> +    ecam_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0);
> +    memory_region_init_alias(&vms->ecam_alias, OBJECT(dev), "pcie-ecam",
> +                             ecam_reg, 0, size_ecam);
> +    memory_region_add_subregion(get_system_memory(), base_ecam,
> +                                &vms->ecam_alias);
> +
> +    /*
> +     * Map the MMIO window from [0x50000000-0x7fff0000] in PCI space into
> +     * system address space at [0x50000000-0x7fff0000].
> +     */
> +    mmio_alias = g_new0(MemoryRegion, 1);
> +    mmio_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 1);
> +    memory_region_init_alias(mmio_alias, OBJECT(dev), "pcie-mmio",
> +                             mmio_reg, base_mmio, size_mmio);
> +    memory_region_add_subregion(get_system_memory(), base_mmio, mmio_alias);
> +
> +    for (i = 0; i < GPEX_NUM_IRQS; i++) {
> +        sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
> +                           qdev_get_gpio_in(vms->gic, irq + i));
> +        gpex_set_irq_num(GPEX_HOST(dev), i, irq + i);
> +    }
> +
> +    pci = PCI_HOST_BRIDGE(dev);
> +    vms->bus = pci->bus;
> +    g_assert(vms->bus);
> +
> +    while ((dev = qemu_create_nic_device("virtio-net-pci", true, NULL))) {
> +        qdev_realize_and_unref(dev, BUS(vms->bus), &error_fatal);
> +    }
> +
> +    usb_controller = qdev_new(TYPE_QEMU_XHCI);
> +    qdev_realize_and_unref(usb_controller, BUS(pci->bus), &error_fatal);
> +
> +    usb_bus = USB_BUS(object_resolve_type_unambiguous(TYPE_USB_BUS,
> +                                                      &error_fatal));
> +    usb_create_simple(usb_bus, "usb-kbd");
> +    usb_create_simple(usb_bus, "usb-tablet");
> +}
> +
> +static void vmapple_reset(void *opaque)
> +{
> +    VMAppleMachineState *vms = opaque;
> +    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
> +
> +    cpu_set_pc(first_cpu, base);
> +}
> +
> +static void mach_vmapple_init(MachineState *machine)
> +{
> +    VMAppleMachineState *vms = VMAPPLE_MACHINE(machine);
> +    MachineClass *mc = MACHINE_GET_CLASS(machine);
> +    const CPUArchIdList *possible_cpus;
> +    MemoryRegion *sysmem = get_system_memory();
> +    int n;
> +    unsigned int smp_cpus = machine->smp.cpus;
> +    unsigned int max_cpus = machine->smp.max_cpus;
> +
> +    vms->memmap = memmap;
> +    machine->usb = true;
> +
> +    possible_cpus = mc->possible_cpu_arch_ids(machine);
> +    assert(possible_cpus->len == max_cpus);
> +    for (n = 0; n < possible_cpus->len; n++) {
> +        Object *cpu;
> +        CPUState *cs;
> +
> +        if (n >= smp_cpus) {
> +            break;
> +        }
> +
> +        cpu = object_new(possible_cpus->cpus[n].type);
> +        object_property_set_int(cpu, "mp-affinity",
> +                                possible_cpus->cpus[n].arch_id, &error_fatal);
> +
> +        cs = CPU(cpu);
> +        cs->cpu_index = n;
> +
> +        numa_cpu_pre_plug(&possible_cpus->cpus[cs->cpu_index], DEVICE(cpu),
> +                          &error_fatal);
> +
> +        if (object_property_find(cpu, "has_el3")) {
> +            object_property_set_bool(cpu, "has_el3", false, &error_fatal);
> +        }
> +        if (object_property_find(cpu, "has_el2")) {
> +            object_property_set_bool(cpu, "has_el2", false, &error_fatal);
> +        }
> +        object_property_set_int(cpu, "psci-conduit", QEMU_PSCI_CONDUIT_HVC,
> +                                &error_fatal);
> +
> +        /* Secondary CPUs start in PSCI powered-down state */
> +        if (n > 0) {
> +            object_property_set_bool(cpu, "start-powered-off", true,
> +                                     &error_fatal);
> +        }
> +
> +        object_property_set_link(cpu, "memory", OBJECT(sysmem), &error_abort);
> +        qdev_realize(DEVICE(cpu), NULL, &error_fatal);
> +        object_unref(cpu);
> +    }
> +
> +    memory_region_add_subregion(sysmem, vms->memmap[VMAPPLE_MEM].base,
> +                                machine->ram);
> +
> +    create_gic(vms, sysmem);
> +    create_bdif(vms, sysmem);
> +    create_pvpanic(vms, sysmem);
> +    create_aes(vms, sysmem);
> +    create_gfx(vms, sysmem);
> +    create_uart(vms, VMAPPLE_UART, sysmem, serial_hd(0));
> +    create_rtc(vms);
> +    create_pcie(vms);
> +
> +    create_gpio_devices(vms, VMAPPLE_GPIO, sysmem);
> +
> +    vmapple_firmware_init(vms, sysmem);
> +    create_cfg(vms, sysmem);
> +
> +    /* connect powerdown request */
> +    vms->powerdown_notifier.notify = vmapple_powerdown_req;
> +    qemu_register_powerdown_notifier(&vms->powerdown_notifier);
> +
> +    vms->bootinfo.ram_size = machine->ram_size;
> +    vms->bootinfo.board_id = -1;
> +    vms->bootinfo.loader_start = vms->memmap[VMAPPLE_MEM].base;
> +    vms->bootinfo.skip_dtb_autoload = true;
> +    vms->bootinfo.firmware_loaded = true;
> +    arm_load_kernel(ARM_CPU(first_cpu), machine, &vms->bootinfo);
> +
> +    qemu_register_reset(vmapple_reset, vms);
> +}
> +
> +static CpuInstanceProperties
> +vmapple_cpu_index_to_props(MachineState *ms, unsigned cpu_index)
> +{
> +    MachineClass *mc = MACHINE_GET_CLASS(ms);
> +    const CPUArchIdList *possible_cpus = mc->possible_cpu_arch_ids(ms);
> +
> +    assert(cpu_index < possible_cpus->len);
> +    return possible_cpus->cpus[cpu_index].props;
> +}
> +
> +
> +static int64_t vmapple_get_default_cpu_node_id(const MachineState *ms, int idx)
> +{
> +    return idx % ms->numa_state->num_nodes;
> +}
> +
> +static const CPUArchIdList *vmapple_possible_cpu_arch_ids(MachineState *ms)
> +{
> +    int n;
> +    unsigned int max_cpus = ms->smp.max_cpus;
> +
> +    if (ms->possible_cpus) {
> +        assert(ms->possible_cpus->len == max_cpus);
> +        return ms->possible_cpus;
> +    }
> +
> +    ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
> +                                  sizeof(CPUArchId) * max_cpus);
> +    ms->possible_cpus->len = max_cpus;
> +    for (n = 0; n < ms->possible_cpus->len; n++) {
> +        ms->possible_cpus->cpus[n].type = ms->cpu_type;
> +        ms->possible_cpus->cpus[n].arch_id =
> +            arm_build_mp_affinity(n, GICV3_TARGETLIST_BITS);
> +        ms->possible_cpus->cpus[n].props.has_thread_id = true;
> +        ms->possible_cpus->cpus[n].props.thread_id = n;
> +    }
> +    return ms->possible_cpus;
> +}
> +
> +static void vmapple_machine_class_init(ObjectClass *oc, void *data)
> +{
> +    MachineClass *mc = MACHINE_CLASS(oc);
> +
> +    mc->init = mach_vmapple_init;
> +    mc->max_cpus = 32;
> +    mc->block_default_type = IF_VIRTIO;
> +    mc->no_cdrom = 1;
> +    mc->pci_allow_0_address = true;
> +    mc->minimum_page_bits = 12;
> +    mc->possible_cpu_arch_ids = vmapple_possible_cpu_arch_ids;
> +    mc->cpu_index_to_instance_props = vmapple_cpu_index_to_props;
> +    mc->default_cpu_type = ARM_CPU_TYPE_NAME("host");
> +    mc->get_default_cpu_node_id = vmapple_get_default_cpu_node_id;
> +    mc->default_ram_id = "mach-vmapple.ram";
> +
> +    object_register_sugar_prop(TYPE_VIRTIO_PCI, "disable-legacy",
> +                               "on", true);
> +}
> +
> +static void vmapple_instance_init(Object *obj)
> +{
> +    VMAppleMachineState *vms = VMAPPLE_MACHINE(obj);
> +
> +    vms->irqmap = irqmap;
> +
> +    object_property_add_uint64_ptr(obj, "uuid", &vms->uuid,
> +                                   OBJ_PROP_FLAG_READWRITE);
> +    object_property_set_description(obj, "uuid", "Machine UUID (SDOM)");
> +}
> +
> +static const TypeInfo vmapple_machine_info = {
> +    .name          = TYPE_VMAPPLE_MACHINE,
> +    .parent        = TYPE_MACHINE,
> +    .abstract      = true,
> +    .instance_size = sizeof(VMAppleMachineState),
> +    .class_size    = sizeof(VMAppleMachineClass),
> +    .class_init    = vmapple_machine_class_init,
> +    .instance_init = vmapple_instance_init,
> +};
> +
> +static void machvmapple_machine_init(void)
> +{
> +    type_register_static(&vmapple_machine_info);
> +}
> +type_init(machvmapple_machine_init);
> +
> +static void vmapple_machine_9_2_options(MachineClass *mc)
> +{
> +}
> +DEFINE_VMAPPLE_MACHINE_AS_LATEST(9, 2)
> +


Re: [PATCH v8 15/15] hw/vmapple/vmapple: Add vmapple machine type
Posted by Phil Dennis-Jordan 1 week, 5 days ago
I've applied the majority of this feedback in the soon-to-be-posted v9
of the patch set, thanks!

Just a few extra comments/replies below:

On Sun, 10 Nov 2024 at 08:38, Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
>
> On 2024/11/08 23:47, Phil Dennis-Jordan wrote:
> > From: Alexander Graf <graf@amazon.com>
> >
> > Apple defines a new "vmapple" machine type as part of its proprietary
> > macOS Virtualization.Framework vmm. This machine type is similar to the
> > virt one, but with subtle differences in base devices, a few special
> > vmapple device additions and a vastly different boot chain.
> >
> > This patch reimplements this machine type in QEMU. To use it, you
> > have to have a readily installed version of macOS for VMApple,
> > run on macOS with -accel hvf, pass the Virtualization.Framework
> > boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
> > and pass aux and root volume as virtio drives. In addition, you also
> > need to find the machine UUID and pass that as -M vmapple,uuid= parameter:
> >
> > $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
> >      -bios /System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
> >      -drive file=aux,if=pflash,format=raw \
> >      -drive file=root,if=pflash,format=raw \
> >      -drive file=aux,if=none,id=aux,format=raw \
> >      -device vmapple-virtio-aux,drive=aux \
> >      -drive file=root,if=none,id=root,format=raw \
> >      -device vmapple-virtio-root,drive=root
> >
> > With all these in place, you should be able to see macOS booting
> > successfully.
> >
> > Known issues:
> >   - Keyboard and mouse/tablet input is laggy. The reason for this is
> >     either that macOS's XHCI driver is broken when the device/platform
> >     does not support MSI/MSI-X, or there's some unfortunate interplay
> >     with Qemu's XHCI implementation in this scenario.
> >   - Currently only macOS 12 guests are supported. The boot process for
> >     13+ will need further investigation and adjustment.
> >
> > Signed-off-by: Alexander Graf <graf@amazon.com>
> > Co-authored-by: Phil Dennis-Jordan <phil@philjordan.eu>
> > Signed-off-by: Phil Dennis-Jordan <phil@philjordan.eu>
> > ---
> >
> > v3:
> >   * Rebased on latest upstream, updated affinity and NIC creation
> >     API usage
> >   * Included Apple-variant virtio-blk in build dependency
> >   * Updated API usage for setting 'redist-region-count' array-typed property
> >     on GIC.
> >   * Switched from virtio HID devices (for which macOS 12 does not contain
> >     drivers) to an XHCI USB controller and USB HID devices.
> >
> > v4:
> >   * Fixups for v4 changes to the other patches in the set.
> >   * Corrected the assert macro to use
> >   * Removed superfluous endian conversions corresponding to cfg's.
> >   * Init error handling improvement.
> >   * No need to select CPU type on TCG, as only HVF is supported.
> >   * Machine type version bumped to 9.2
> >   * #include order improved
> >
> > v5:
> >   * Fixed memory reservation for ecam alias region.
> >   * Better error handling setting properties on devices.
> >   * Simplified the machine ECID/UUID extraction script and actually created a
> >     file for it rather than quoting its code in documentation.
> >
> > v7:
> >   * Tiny error handling fix, un-inlined function.
> >
> > v8:
> >   * Use object_property_add_uint64_ptr rather than defining custom UUID
> >     property get/set functions.
> >
> >   MAINTAINERS                 |   1 +
> >   contrib/vmapple/uuid.sh     |   9 +
> >   docs/system/arm/vmapple.rst |  60 ++++
> >   docs/system/target-arm.rst  |   1 +
> >   hw/vmapple/Kconfig          |  20 ++
> >   hw/vmapple/meson.build      |   1 +
> >   hw/vmapple/vmapple.c        | 638 ++++++++++++++++++++++++++++++++++++
> >   7 files changed, 730 insertions(+)
> >   create mode 100755 contrib/vmapple/uuid.sh
> >   create mode 100644 docs/system/arm/vmapple.rst
> >   create mode 100644 hw/vmapple/vmapple.c
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 9d0d26cb65f..9591fd44a34 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -2767,6 +2767,7 @@ R: Phil Dennis-Jordan <phil@philjordan.eu>
> >   S: Maintained
> >   F: hw/vmapple/*
> >   F: include/hw/vmapple/*
> > +F: docs/system/arm/vmapple.rst
> >
> >   Subsystems
> >   ----------
> > diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.sh
> > new file mode 100755
> > index 00000000000..956e8c3afed
> > --- /dev/null
> > +++ b/contrib/vmapple/uuid.sh
> > @@ -0,0 +1,9 @@
> > +#!/bin/sh
> > +# Used for converting a guest provisioned using Virtualization.framework
> > +# for use with the QEMU 'vmapple' aarch64 machine type.
> > +#
> > +# Extracts the Machine UUID from Virtualization.framework VM JSON file.
> > +# (as produced by 'macosvm', passed as command line argument)
> > +
> > +plutil -extract machineId raw "$1" | base64 -d | plutil -extract ECID raw -
> > +
> > diff --git a/docs/system/arm/vmapple.rst b/docs/system/arm/vmapple.rst
> > new file mode 100644
> > index 00000000000..6a634fa4572
> > --- /dev/null
> > +++ b/docs/system/arm/vmapple.rst
> > @@ -0,0 +1,60 @@
> > +VMApple machine emulation
> > +========================================================================================
> > +
> > +VMApple is the device model that the macOS built-in hypervisor called "Virtualization.framework"
> > +exposes to Apple Silicon macOS guests. The "vmapple" machine model in QEMU implements the same
> > +device model, but does not use any code from Virtualization.Framework.
> > +
> > +Prerequisites
> > +-------------
> > +
> > +To run the vmapple machine model, you need to
> > +
> > + * Run on Apple Silicon
> > + * Run on macOS 12.0 or above
> > + * Have an already installed copy of a Virtualization.Framework macOS 12 virtual machine. I will
> > +   assume that you installed it using the macosvm CLI.
>
> Add a URL to macosvm.
>
> > +
> > +First, we need to extract the UUID from the virtual machine that you installed. You can do this
> > +by running the shell script in contrib/vmapple/uuid.sh on the macosvm.json file.
> > +
> > +.. code-block:: bash
> > +  :caption: uuid.sh script to extract the UUID from a macosvm.json file
> > +
> > +  $ contrib/vmapple/uuid.sh "path/to/macosvm.json"
> > +
> > +Now we also need to trim the aux partition. It contains metadata that we can just discard:
> > +
> > +.. code-block:: bash
> > +  :caption: Command to trim the aux file
> > +
> > +  $ dd if="aux.img" of="aux.img.trimmed" bs=$(( 0x4000 )) skip=1
>
> Quoting is inconsistent. aux.img.trimmed is not quoted below but it is
> quoted here.
>
> > +
> > +How to run
> > +----------
> > +
> > +Then, we can launch QEMU with the Virtualization.Framework pre-boot environment and the readily
> > +installed target disk images. I recommend to port forward the VM's ssh and vnc ports to the host
> > +to get better interactive access into the target system:
> > +
> > +.. code-block:: bash
> > +  :caption: Example execution command line
> > +
> > +  $ UUID=$(uuid.sh macosvm.json)
> > +  $ AVPBOOTER=/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin
> > +  $ AUX=aux.img.trimmed
> > +  $ DISK=disk.img
> > +  $ qemu-system-aarch64 \
> > +       -serial mon:stdio \
> > +       -m 4G \
> > +       -accel hvf \
> > +       -M vmapple,uuid=$UUID \
> > +       -bios $AVPBOOTER \
>
> $AUX and $DISK are quoted but $UUID and $AVPBOOTER are not.
>
> > +        -drive file="$AUX",if=pflash,format=raw \
> > +        -drive file="$DISK",if=pflash,format=raw \
>
> These two lines are misaligned.
>
> > +       -drive file="$AUX",if=none,id=aux,format=raw \
> > +       -drive file="$DISK",if=none,id=root,format=raw \
> > +       -device vmapple-virtio-aux,drive=aux \
> > +       -device vmapple-virtio-root,drive=root \
> > +       -net user,ipv6=off,hostfwd=tcp::2222-:22,hostfwd=tcp::5901-:5900 \
> > +       -net nic,model=virtio-net-pci \
>
> -net is a legacy option and creates a unnecessary indirection with hub.
> Use -netdev and -device options. Also plug virtio-net-pci into a PCIe
> root port as suggested in: docs/pcie.txt

I have not had any success with booting macOS with the block devices
connected to PCIe root ports, and similarly the network doesn't come
up if I attach just the virtio-net-pci device to one.

As far as I can tell (ioreg), Apple's own VMapple machine type does
not use PCIe root ports, but I'm not 100% sure. The PCIe documentation
is generally rather focused on x86 and on running Linux in the 'virt'
machine type.

In any case, a far more pressing issue is MSI support, without which
the XHCI driver fails to receive most kinds of notifications from the
USB controller. (Therefore keyboard/mouse is currently very laggy.)
The arm 'virt' machine type supposedly supports it via the GIC ITS
device, but I've so far not managed to transfer this to 'vmapple' in a
way that makes the guest OS detect MSI support. Anyway, in its current
iteration, the machine type is good enough to be used for some tasks,
and we can always refine it later. (Another major limitation is of
course guest support for OS versions > 12 - but those issues might
even be related.)

One thing I am thinking about changing (as we've now missed the 9.2
merge window anyway, sigh) as it would be non-backwards-compatible is
replacing the vmapple-virtio-aux/vmapple-virtio-root pair of devices
with just the the base type, vmapple-virtio-blk-pci, and adding a
variant property to it. So:

       -device vmapple-virtio-blk-pci,variant=aux,drive=aux \
       -device vmapple-virtio-blk-pci,variant=root,drive=root \

I think this would make hw/vmapple/virtio-blk.c somewhat more readable
and also significantly shorter.


> I thought it is about time to give a try on my M2 MacBook Air, but this
> command line did not work. The serial output said iBootStage2 finished,
> but it went silent after that and the display gives not output. macosvm
> can run the VM image. I used: UniversalMac_15.1_24B83_Restore.ipsw
>
> Do you have an idea what's wrong with my setup? Doesn't it work with 15.1?
>
> > diff --git a/docs/system/target-arm.rst b/docs/system/target-arm.rst
> > index 9aaa9c414c9..3426f79100b 100644
> > --- a/docs/system/target-arm.rst
> > +++ b/docs/system/target-arm.rst
> > @@ -102,6 +102,7 @@ Board-specific documentation
> >      arm/stellaris
> >      arm/stm32
> >      arm/virt
> > +   arm/vmapple
> >      arm/xenpvh
> >      arm/xlnx-versal-virt
> >      arm/xlnx-zynq
> > diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
> > index bcd1be63e3c..6a4c4a7fa2e 100644
> > --- a/hw/vmapple/Kconfig
> > +++ b/hw/vmapple/Kconfig
> > @@ -10,3 +10,23 @@ config VMAPPLE_CFG
> >   config VMAPPLE_VIRTIO_BLK
> >       bool
> >
> > +config VMAPPLE
> > +    bool
> > +    depends on ARM
> > +    depends on HVF
> > +    default y if ARM
> > +    imply PCI_DEVICES
> > +    select ARM_GIC
> > +    select PLATFORM_BUS
> > +    select PCI_EXPRESS
> > +    select PCI_EXPRESS_GENERIC_BRIDGE
> > +    select PL011 # UART
> > +    select PL031 # RTC
> > +    select PL061 # GPIO
> > +    select GPIO_PWR
> > +    select PVPANIC_MMIO
> > +    select VMAPPLE_AES
> > +    select VMAPPLE_BDIF
> > +    select VMAPPLE_CFG
> > +    select MAC_PVG_MMIO
> > +    select VMAPPLE_VIRTIO_BLK
> > diff --git a/hw/vmapple/meson.build b/hw/vmapple/meson.build
> > index bf17cf906c9..e572f7d5602 100644
> > --- a/hw/vmapple/meson.build
> > +++ b/hw/vmapple/meson.build
> > @@ -2,3 +2,4 @@ system_ss.add(when: 'CONFIG_VMAPPLE_AES',  if_true: files('aes.c'))
> >   system_ss.add(when: 'CONFIG_VMAPPLE_BDIF', if_true: files('bdif.c'))
> >   system_ss.add(when: 'CONFIG_VMAPPLE_CFG',  if_true: files('cfg.c'))
> >   system_ss.add(when: 'CONFIG_VMAPPLE_VIRTIO_BLK',  if_true: files('virtio-blk.c'))
> > +specific_ss.add(when: 'CONFIG_VMAPPLE',     if_true: files('vmapple.c'))
> > diff --git a/hw/vmapple/vmapple.c b/hw/vmapple/vmapple.c
> > new file mode 100644
> > index 00000000000..8b8710c70a2
> > --- /dev/null
> > +++ b/hw/vmapple/vmapple.c
> > @@ -0,0 +1,638 @@
> > +/*
> > + * VMApple machine emulation
> > + *
> > + * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later
> > + *
> > + * VMApple is the device model that the macOS built-in hypervisor called
> > + * "Virtualization.framework" exposes to Apple Silicon macOS guests. The
> > + * machine model in this file implements the same device model in QEMU, but
> > + * does not use any code from Virtualization.Framework.
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/bitops.h"
> > +#include "qemu/datadir.h"
> > +#include "qemu/error-report.h"
> > +#include "qemu/guest-random.h"
> > +#include "qemu/help-texts.h"
> > +#include "qemu/log.h"
> > +#include "qemu/module.h"
> > +#include "qemu/option.h"
> > +#include "qemu/units.h"
> > +#include "monitor/qdev.h"
> > +#include "hw/boards.h"
> > +#include "hw/irq.h"
> > +#include "hw/loader.h"
> > +#include "hw/qdev-properties.h"
> > +#include "hw/sysbus.h"
> > +#include "hw/usb.h"
> > +#include "hw/arm/boot.h"
> > +#include "hw/arm/primecell.h"
> > +#include "hw/char/pl011.h"
> > +#include "hw/intc/arm_gic.h"
> > +#include "hw/intc/arm_gicv3_common.h"
> > +#include "hw/misc/pvpanic.h"
> > +#include "hw/pci-host/gpex.h"
> > +#include "hw/usb/xhci.h"
> > +#include "hw/virtio/virtio-pci.h"
> > +#include "hw/vmapple/vmapple.h"
> > +#include "net/net.h"
> > +#include "qapi/error.h"
> > +#include "qapi/qmp/qlist.h"
> > +#include "qapi/visitor.h"
> > +#include "qapi/qapi-visit-common.h"
> > +#include "standard-headers/linux/input.h"
> > +#include "sysemu/hvf.h"
> > +#include "sysemu/kvm.h"
> > +#include "sysemu/reset.h"
> > +#include "sysemu/runstate.h"
> > +#include "sysemu/sysemu.h"
> > +#include "target/arm/internals.h"
> > +#include "target/arm/kvm_arm.h"
> > +
> > +struct VMAppleMachineClass {
> > +    MachineClass parent;
> > +};
> > +
> > +struct VMAppleMachineState {
> > +    MachineState parent;
> > +
> > +    Notifier machine_done;
> > +    struct arm_boot_info bootinfo;
> > +    MemMapEntry *memmap;
> > +    const int *irqmap;
> > +    DeviceState *gic;
> > +    DeviceState *cfg;
> > +    Notifier powerdown_notifier;
> > +    PCIBus *bus;
> > +    MemoryRegion fw_mr;
> > +    MemoryRegion ecam_alias;
> > +    uint64_t uuid;
> > +};
> > +
> > +#define DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, latest) \
> > +    static void vmapple##major##_##minor##_class_init(ObjectClass *oc, \
> > +                                                    void *data) \
> > +    { \
> > +        MachineClass *mc = MACHINE_CLASS(oc); \
> > +        vmapple_machine_##major##_##minor##_options(mc); \
> > +        mc->desc = "QEMU " # major "." # minor " Apple Virtual Machine"; \
> > +        if (latest) { \
> > +            mc->alias = "vmapple"; \
> > +        } \
> > +    } \
> > +    static const TypeInfo machvmapple##major##_##minor##_info = { \
> > +        .name = MACHINE_TYPE_NAME("vmapple-" # major "." # minor), \
> > +        .parent = TYPE_VMAPPLE_MACHINE, \
> > +        .class_init = vmapple##major##_##minor##_class_init, \
> > +    }; \
> > +    static void machvmapple_machine_##major##_##minor##_init(void) \
> > +    { \
> > +        type_register_static(&machvmapple##major##_##minor##_info); \
> > +    } \
> > +    type_init(machvmapple_machine_##major##_##minor##_init);
> > +
> > +#define DEFINE_VMAPPLE_MACHINE_AS_LATEST(major, minor) \
> > +    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, true)
> > +#define DEFINE_VMAPPLE_MACHINE(major, minor) \
> > +    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, false)
> > +
> > +#define TYPE_VMAPPLE_MACHINE   MACHINE_TYPE_NAME("vmapple")
> > +OBJECT_DECLARE_TYPE(VMAppleMachineState, VMAppleMachineClass, VMAPPLE_MACHINE)
> > +
> > +/* Number of external interrupt lines to configure the GIC with */
> > +#define NUM_IRQS 256
> > +
> > +enum {
> > +    VMAPPLE_FIRMWARE,
> > +    VMAPPLE_CONFIG,
> > +    VMAPPLE_MEM,
> > +    VMAPPLE_GIC_DIST,
> > +    VMAPPLE_GIC_REDIST,
> > +    VMAPPLE_UART,
> > +    VMAPPLE_RTC,
> > +    VMAPPLE_PCIE,
> > +    VMAPPLE_PCIE_MMIO,
> > +    VMAPPLE_PCIE_ECAM,
> > +    VMAPPLE_GPIO,
> > +    VMAPPLE_PVPANIC,
> > +    VMAPPLE_APV_GFX,
> > +    VMAPPLE_APV_IOSFC,
> > +    VMAPPLE_AES_1,
> > +    VMAPPLE_AES_2,
> > +    VMAPPLE_BDOOR,
> > +    VMAPPLE_MEMMAP_LAST,
> > +};
> > +
> > +static MemMapEntry memmap[] = {
> > +    [VMAPPLE_FIRMWARE] =           { 0x00100000, 0x00100000 },
> > +    [VMAPPLE_CONFIG] =             { 0x00400000, 0x00010000 },
> > +
> > +    [VMAPPLE_GIC_DIST] =           { 0x10000000, 0x00010000 },
> > +    [VMAPPLE_GIC_REDIST] =         { 0x10010000, 0x00400000 },
> > +
> > +    [VMAPPLE_UART] =               { 0x20010000, 0x00010000 },
> > +    [VMAPPLE_RTC] =                { 0x20050000, 0x00001000 },
> > +    [VMAPPLE_GPIO] =               { 0x20060000, 0x00001000 },
> > +    [VMAPPLE_PVPANIC] =            { 0x20070000, 0x00000002 },
> > +    [VMAPPLE_BDOOR] =              { 0x30000000, 0x00200000 },
> > +    [VMAPPLE_APV_GFX] =            { 0x30200000, 0x00010000 },
> > +    [VMAPPLE_APV_IOSFC] =          { 0x30210000, 0x00010000 },
> > +    [VMAPPLE_AES_1] =              { 0x30220000, 0x00004000 },
> > +    [VMAPPLE_AES_2] =              { 0x30230000, 0x00004000 },
> > +    [VMAPPLE_PCIE_ECAM] =          { 0x40000000, 0x10000000 },
> > +    [VMAPPLE_PCIE_MMIO] =          { 0x50000000, 0x1fff0000 },
> > +
> > +    /* Actual RAM size depends on configuration */
> > +    [VMAPPLE_MEM] =                { 0x70000000ULL, GiB},
> > +};
> > +
> > +static const int irqmap[] = {
> > +    [VMAPPLE_UART] = 1,
> > +    [VMAPPLE_RTC] = 2,
> > +    [VMAPPLE_GPIO] = 0x5,
> > +    [VMAPPLE_APV_IOSFC] = 0x10,
> > +    [VMAPPLE_APV_GFX] = 0x11,
> > +    [VMAPPLE_AES_1] = 0x12,
> > +    [VMAPPLE_PCIE] = 0x20,
> > +};
> > +
> > +#define GPEX_NUM_IRQS 16
> > +
> > +static void create_bdif(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    DeviceState *bdif;
> > +    SysBusDevice *bdif_sb;
> > +    DriveInfo *di_aux = drive_get(IF_PFLASH, 0, 0);
> > +    DriveInfo *di_root = drive_get(IF_PFLASH, 0, 1);
> > +
> > +    if (!di_aux) {
> > +        error_report("No AUX device. Please specify one as pflash drive.");
> > +        exit(1);
> > +    }
> > +
> > +    if (!di_root) {
> > +        /* Fall back to the first IF_VIRTIO device as root device */
> > +        di_root = drive_get(IF_VIRTIO, 0, 0);
> > +    }
> > +
> > +    if (!di_root) {
> > +        error_report("No root device. Please specify one as virtio drive.");
> > +        exit(1);
> > +    }
> > +
> > +    /* PV backdoor device */
> > +    bdif = qdev_new(TYPE_VMAPPLE_BDIF);
> > +    bdif_sb = SYS_BUS_DEVICE(bdif);
> > +    sysbus_mmio_map(bdif_sb, 0, vms->memmap[VMAPPLE_BDOOR].base);
> > +
> > +    qdev_prop_set_drive(DEVICE(bdif), "aux", blk_by_legacy_dinfo(di_aux));
> > +    qdev_prop_set_drive(DEVICE(bdif), "root", blk_by_legacy_dinfo(di_root));
> > +
> > +    sysbus_realize_and_unref(bdif_sb, &error_fatal);
> > +}
> > +
> > +static void create_pvpanic(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    SysBusDevice *cfg;
> > +
> > +    vms->cfg = qdev_new(TYPE_PVPANIC_MMIO_DEVICE);
> > +    cfg = SYS_BUS_DEVICE(vms->cfg);
> > +    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_PVPANIC].base);
> > +
> > +    sysbus_realize_and_unref(cfg, &error_fatal);
> > +}
> > +
> > +static void create_cfg(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    SysBusDevice *cfg;
> > +    MachineState *machine = MACHINE(vms);
> > +    uint32_t rnd = 1;
> > +
> > +    vms->cfg = qdev_new(TYPE_VMAPPLE_CFG);
> > +    cfg = SYS_BUS_DEVICE(vms->cfg);
> > +    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_CONFIG].base);
> > +
> > +    qemu_guest_getrandom_nofail(&rnd, sizeof(rnd));
> > +
> > +    qdev_prop_set_uint32(vms->cfg, "nr-cpus", machine->smp.cpus);
> > +    qdev_prop_set_uint64(vms->cfg, "ecid", vms->uuid);
> > +    qdev_prop_set_uint64(vms->cfg, "ram-size", machine->ram_size);
> > +    qdev_prop_set_uint32(vms->cfg, "rnd", rnd);
> > +
> > +    sysbus_realize_and_unref(cfg, &error_fatal);
> > +}
> > +
> > +static void create_gfx(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    int irq_gfx = vms->irqmap[VMAPPLE_APV_GFX];
> > +    int irq_iosfc = vms->irqmap[VMAPPLE_APV_IOSFC];
> > +    SysBusDevice *gfx;
> > +
> > +    gfx = SYS_BUS_DEVICE(qdev_new("apple-gfx-mmio"));
> > +    sysbus_mmio_map(gfx, 0, vms->memmap[VMAPPLE_APV_GFX].base);
> > +    sysbus_mmio_map(gfx, 1, vms->memmap[VMAPPLE_APV_IOSFC].base);
> > +    sysbus_connect_irq(gfx, 0, qdev_get_gpio_in(vms->gic, irq_gfx));
> > +    sysbus_connect_irq(gfx, 1, qdev_get_gpio_in(vms->gic, irq_iosfc));
> > +    sysbus_realize_and_unref(gfx, &error_fatal);
> > +}
> > +
> > +static void create_aes(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    int irq = vms->irqmap[VMAPPLE_AES_1];
> > +    SysBusDevice *aes;
> > +
> > +    aes = SYS_BUS_DEVICE(qdev_new(TYPE_APPLE_AES));
> > +    sysbus_mmio_map(aes, 0, vms->memmap[VMAPPLE_AES_1].base);
> > +    sysbus_mmio_map(aes, 1, vms->memmap[VMAPPLE_AES_2].base);
> > +    sysbus_connect_irq(aes, 0, qdev_get_gpio_in(vms->gic, irq));
> > +    sysbus_realize_and_unref(aes, &error_fatal);
> > +}
> > +
> > +static int arm_gic_ppi_index(int cpu_nr, int ppi_index)
> > +{
> > +    return NUM_IRQS + cpu_nr * GIC_INTERNAL + ppi_index;
> > +}
> > +
> > +static void create_gic(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    MachineState *ms = MACHINE(vms);
> > +    /* We create a standalone GIC */
> > +    SysBusDevice *gicbusdev;
> > +    QList *redist_region_count;
> > +    int i;
> > +    unsigned int smp_cpus = ms->smp.cpus;
> > +
> > +    vms->gic = qdev_new(gicv3_class_name());
> > +    qdev_prop_set_uint32(vms->gic, "revision", 3);
> > +    qdev_prop_set_uint32(vms->gic, "num-cpu", smp_cpus);
> > +    /*
> > +     * Note that the num-irq property counts both internal and external
> > +     * interrupts; there are always 32 of the former (mandated by GIC spec).
> > +     */
> > +    qdev_prop_set_uint32(vms->gic, "num-irq", NUM_IRQS + 32);
> > +
> > +    uint32_t redist0_capacity =
> > +                vms->memmap[VMAPPLE_GIC_REDIST].size / GICV3_REDIST_SIZE;
> > +    uint32_t redist0_count = MIN(smp_cpus, redist0_capacity);
> > +
> > +    redist_region_count = qlist_new();
> > +    qlist_append_int(redist_region_count, redist0_count);
> > +    qdev_prop_set_array(vms->gic, "redist-region-count", redist_region_count);
> > +
> > +    gicbusdev = SYS_BUS_DEVICE(vms->gic);
> > +    sysbus_realize_and_unref(gicbusdev, &error_fatal);
> > +    sysbus_mmio_map(gicbusdev, 0, vms->memmap[VMAPPLE_GIC_DIST].base);
> > +    sysbus_mmio_map(gicbusdev, 1, vms->memmap[VMAPPLE_GIC_REDIST].base);
> > +
> > +    /*
> > +     * Wire the outputs from each CPU's generic timer and the GICv3
> > +     * maintenance interrupt signal to the appropriate GIC PPI inputs,
> > +     * and the GIC's IRQ/FIQ/VIRQ/VFIQ interrupt outputs to the CPU's inputs.
> > +     */
> > +    for (i = 0; i < smp_cpus; i++) {
> > +        DeviceState *cpudev = DEVICE(qemu_get_cpu(i));
> > +
> > +        /* Map the virt timer to PPI 27 */
> > +        qdev_connect_gpio_out(cpudev, GTIMER_VIRT,
> > +                              qdev_get_gpio_in(vms->gic,
> > +                                               arm_gic_ppi_index(i, 27)));
> > +
> > +        /* Map the GIC IRQ and FIQ lines to CPU */
> > +        sysbus_connect_irq(gicbusdev, i, qdev_get_gpio_in(cpudev, ARM_CPU_IRQ));
> > +        sysbus_connect_irq(gicbusdev, i + smp_cpus,
> > +                           qdev_get_gpio_in(cpudev, ARM_CPU_FIQ));
> > +    }
> > +}
> > +
> > +static void create_uart(const VMAppleMachineState *vms, int uart,
> > +                        MemoryRegion *mem, Chardev *chr)
> > +{
> > +    hwaddr base = vms->memmap[uart].base;
> > +    int irq = vms->irqmap[uart];
> > +    DeviceState *dev = qdev_new(TYPE_PL011);
> > +    SysBusDevice *s = SYS_BUS_DEVICE(dev);
> > +
> > +    qdev_prop_set_chr(dev, "chardev", chr);
> > +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> > +    memory_region_add_subregion(mem, base,
> > +                                sysbus_mmio_get_region(s, 0));
> > +    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
> > +}
> > +
> > +static void create_rtc(const VMAppleMachineState *vms)
> > +{
> > +    hwaddr base = vms->memmap[VMAPPLE_RTC].base;
> > +    int irq = vms->irqmap[VMAPPLE_RTC];
> > +
> > +    sysbus_create_simple("pl031", base, qdev_get_gpio_in(vms->gic, irq));
> > +}
> > +
> > +static DeviceState *gpio_key_dev;
> > +static void vmapple_powerdown_req(Notifier *n, void *opaque)
> > +{
> > +    /* use gpio Pin 3 for power button event */
> > +    qemu_set_irq(qdev_get_gpio_in(gpio_key_dev, 0), 1);
> > +}
> > +
> > +static void create_gpio_devices(const VMAppleMachineState *vms, int gpio,
> > +                                MemoryRegion *mem)
> > +{
> > +    DeviceState *pl061_dev;
> > +    hwaddr base = vms->memmap[gpio].base;
> > +    int irq = vms->irqmap[gpio];
> > +    SysBusDevice *s;
> > +
> > +    pl061_dev = qdev_new("pl061");
> > +    /* Pull lines down to 0 if not driven by the PL061 */
> > +    qdev_prop_set_uint32(pl061_dev, "pullups", 0);
> > +    qdev_prop_set_uint32(pl061_dev, "pulldowns", 0xff);
> > +    s = SYS_BUS_DEVICE(pl061_dev);
> > +    sysbus_realize_and_unref(s, &error_fatal);
> > +    memory_region_add_subregion(mem, base, sysbus_mmio_get_region(s, 0));
> > +    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
> > +    gpio_key_dev = sysbus_create_simple("gpio-key", -1,
> > +                                        qdev_get_gpio_in(pl061_dev, 3));
> > +}
> > +
> > +static void vmapple_firmware_init(VMAppleMachineState *vms,
> > +                                  MemoryRegion *sysmem)
> > +{
> > +    hwaddr size = vms->memmap[VMAPPLE_FIRMWARE].size;
> > +    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
> > +    const char *bios_name;
> > +    int image_size;
> > +    char *fname;
> > +
> > +    bios_name = MACHINE(vms)->firmware;
> > +    if (!bios_name) {
> > +        error_report("No firmware specified");
> > +        exit(1);
> > +    }
> > +
> > +    fname = qemu_find_file(QEMU_FILE_TYPE_BIOS, bios_name);
> > +    if (!fname) {
> > +        error_report("Could not find ROM image '%s'", bios_name);
> > +        exit(1);
> > +    }
> > +
> > +    memory_region_init_ram(&vms->fw_mr, NULL, "firmware", size, &error_fatal);
> > +    image_size = load_image_mr(fname, &vms->fw_mr);
> > +
> > +    g_free(fname);
> > +    if (image_size < 0) {
> > +        error_report("Could not load ROM image '%s'", bios_name);
> > +        exit(1);
> > +    }
> > +
> > +    memory_region_add_subregion(get_system_memory(), base, &vms->fw_mr);
> > +}
> > +
> > +static void create_pcie(VMAppleMachineState *vms)
> > +{
> > +    hwaddr base_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].base;
> > +    hwaddr size_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].size;
> > +    hwaddr base_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].base;
> > +    hwaddr size_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].size;
> > +    int irq = vms->irqmap[VMAPPLE_PCIE];
> > +    MemoryRegion *mmio_alias;
> > +    MemoryRegion *mmio_reg;
> > +    MemoryRegion *ecam_reg;
> > +    DeviceState *dev;
> > +    int i;
> > +    PCIHostState *pci;
> > +    DeviceState *usb_controller;
> > +    USBBus *usb_bus;
> > +
> > +    dev = qdev_new(TYPE_GPEX_HOST);
> > +    qdev_prop_set_uint32(dev, "num-irqs", GPEX_NUM_IRQS);
> > +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> > +
> > +    /* Map only the first size_ecam bytes of ECAM space */
> > +    ecam_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0);
> > +    memory_region_init_alias(&vms->ecam_alias, OBJECT(dev), "pcie-ecam",
> > +                             ecam_reg, 0, size_ecam);
> > +    memory_region_add_subregion(get_system_memory(), base_ecam,
> > +                                &vms->ecam_alias);
> > +
> > +    /*
> > +     * Map the MMIO window from [0x50000000-0x7fff0000] in PCI space into
> > +     * system address space at [0x50000000-0x7fff0000].
> > +     */
> > +    mmio_alias = g_new0(MemoryRegion, 1);
> > +    mmio_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 1);
> > +    memory_region_init_alias(mmio_alias, OBJECT(dev), "pcie-mmio",
> > +                             mmio_reg, base_mmio, size_mmio);
> > +    memory_region_add_subregion(get_system_memory(), base_mmio, mmio_alias);
> > +
> > +    for (i = 0; i < GPEX_NUM_IRQS; i++) {
> > +        sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
> > +                           qdev_get_gpio_in(vms->gic, irq + i));
> > +        gpex_set_irq_num(GPEX_HOST(dev), i, irq + i);
> > +    }
> > +
> > +    pci = PCI_HOST_BRIDGE(dev);
> > +    vms->bus = pci->bus;
> > +    g_assert(vms->bus);
> > +
> > +    while ((dev = qemu_create_nic_device("virtio-net-pci", true, NULL))) {
> > +        qdev_realize_and_unref(dev, BUS(vms->bus), &error_fatal);
> > +    }
> > +
> > +    usb_controller = qdev_new(TYPE_QEMU_XHCI);
> > +    qdev_realize_and_unref(usb_controller, BUS(pci->bus), &error_fatal);
> > +
> > +    usb_bus = USB_BUS(object_resolve_type_unambiguous(TYPE_USB_BUS,
> > +                                                      &error_fatal));
> > +    usb_create_simple(usb_bus, "usb-kbd");
> > +    usb_create_simple(usb_bus, "usb-tablet");
> > +}
> > +
> > +static void vmapple_reset(void *opaque)
> > +{
> > +    VMAppleMachineState *vms = opaque;
> > +    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
> > +
> > +    cpu_set_pc(first_cpu, base);
> > +}
> > +
> > +static void mach_vmapple_init(MachineState *machine)
> > +{
> > +    VMAppleMachineState *vms = VMAPPLE_MACHINE(machine);
> > +    MachineClass *mc = MACHINE_GET_CLASS(machine);
> > +    const CPUArchIdList *possible_cpus;
> > +    MemoryRegion *sysmem = get_system_memory();
> > +    int n;
> > +    unsigned int smp_cpus = machine->smp.cpus;
> > +    unsigned int max_cpus = machine->smp.max_cpus;
> > +
> > +    vms->memmap = memmap;
> > +    machine->usb = true;
> > +
> > +    possible_cpus = mc->possible_cpu_arch_ids(machine);
> > +    assert(possible_cpus->len == max_cpus);
> > +    for (n = 0; n < possible_cpus->len; n++) {
> > +        Object *cpu;
> > +        CPUState *cs;
> > +
> > +        if (n >= smp_cpus) {
> > +            break;
> > +        }
> > +
> > +        cpu = object_new(possible_cpus->cpus[n].type);
> > +        object_property_set_int(cpu, "mp-affinity",
> > +                                possible_cpus->cpus[n].arch_id, &error_fatal);
> > +
> > +        cs = CPU(cpu);
> > +        cs->cpu_index = n;
> > +
> > +        numa_cpu_pre_plug(&possible_cpus->cpus[cs->cpu_index], DEVICE(cpu),
> > +                          &error_fatal);
> > +
> > +        if (object_property_find(cpu, "has_el3")) {
> > +            object_property_set_bool(cpu, "has_el3", false, &error_fatal);
> > +        }
> > +        if (object_property_find(cpu, "has_el2")) {
> > +            object_property_set_bool(cpu, "has_el2", false, &error_fatal);
> > +        }
> > +        object_property_set_int(cpu, "psci-conduit", QEMU_PSCI_CONDUIT_HVC,
> > +                                &error_fatal);
> > +
> > +        /* Secondary CPUs start in PSCI powered-down state */
> > +        if (n > 0) {
> > +            object_property_set_bool(cpu, "start-powered-off", true,
> > +                                     &error_fatal);
> > +        }
> > +
> > +        object_property_set_link(cpu, "memory", OBJECT(sysmem), &error_abort);
> > +        qdev_realize(DEVICE(cpu), NULL, &error_fatal);
> > +        object_unref(cpu);
> > +    }
> > +
> > +    memory_region_add_subregion(sysmem, vms->memmap[VMAPPLE_MEM].base,
> > +                                machine->ram);
> > +
> > +    create_gic(vms, sysmem);
> > +    create_bdif(vms, sysmem);
> > +    create_pvpanic(vms, sysmem);
> > +    create_aes(vms, sysmem);
> > +    create_gfx(vms, sysmem);
> > +    create_uart(vms, VMAPPLE_UART, sysmem, serial_hd(0));
> > +    create_rtc(vms);
> > +    create_pcie(vms);
> > +
> > +    create_gpio_devices(vms, VMAPPLE_GPIO, sysmem);
> > +
> > +    vmapple_firmware_init(vms, sysmem);
> > +    create_cfg(vms, sysmem);
> > +
> > +    /* connect powerdown request */
> > +    vms->powerdown_notifier.notify = vmapple_powerdown_req;
> > +    qemu_register_powerdown_notifier(&vms->powerdown_notifier);
> > +
> > +    vms->bootinfo.ram_size = machine->ram_size;
> > +    vms->bootinfo.board_id = -1;
> > +    vms->bootinfo.loader_start = vms->memmap[VMAPPLE_MEM].base;
> > +    vms->bootinfo.skip_dtb_autoload = true;
> > +    vms->bootinfo.firmware_loaded = true;
> > +    arm_load_kernel(ARM_CPU(first_cpu), machine, &vms->bootinfo);
> > +
> > +    qemu_register_reset(vmapple_reset, vms);
> > +}
> > +
> > +static CpuInstanceProperties
> > +vmapple_cpu_index_to_props(MachineState *ms, unsigned cpu_index)
> > +{
> > +    MachineClass *mc = MACHINE_GET_CLASS(ms);
> > +    const CPUArchIdList *possible_cpus = mc->possible_cpu_arch_ids(ms);
> > +
> > +    assert(cpu_index < possible_cpus->len);
> > +    return possible_cpus->cpus[cpu_index].props;
> > +}
> > +
> > +
> > +static int64_t vmapple_get_default_cpu_node_id(const MachineState *ms, int idx)
> > +{
> > +    return idx % ms->numa_state->num_nodes;
> > +}
> > +
> > +static const CPUArchIdList *vmapple_possible_cpu_arch_ids(MachineState *ms)
> > +{
> > +    int n;
> > +    unsigned int max_cpus = ms->smp.max_cpus;
> > +
> > +    if (ms->possible_cpus) {
> > +        assert(ms->possible_cpus->len == max_cpus);
> > +        return ms->possible_cpus;
> > +    }
> > +
> > +    ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
> > +                                  sizeof(CPUArchId) * max_cpus);
> > +    ms->possible_cpus->len = max_cpus;
> > +    for (n = 0; n < ms->possible_cpus->len; n++) {
> > +        ms->possible_cpus->cpus[n].type = ms->cpu_type;
> > +        ms->possible_cpus->cpus[n].arch_id =
> > +            arm_build_mp_affinity(n, GICV3_TARGETLIST_BITS);
> > +        ms->possible_cpus->cpus[n].props.has_thread_id = true;
> > +        ms->possible_cpus->cpus[n].props.thread_id = n;
> > +    }
> > +    return ms->possible_cpus;
> > +}
> > +
> > +static void vmapple_machine_class_init(ObjectClass *oc, void *data)
> > +{
> > +    MachineClass *mc = MACHINE_CLASS(oc);
> > +
> > +    mc->init = mach_vmapple_init;
> > +    mc->max_cpus = 32;
> > +    mc->block_default_type = IF_VIRTIO;
> > +    mc->no_cdrom = 1;
> > +    mc->pci_allow_0_address = true;
> > +    mc->minimum_page_bits = 12;
> > +    mc->possible_cpu_arch_ids = vmapple_possible_cpu_arch_ids;
> > +    mc->cpu_index_to_instance_props = vmapple_cpu_index_to_props;
> > +    mc->default_cpu_type = ARM_CPU_TYPE_NAME("host");
> > +    mc->get_default_cpu_node_id = vmapple_get_default_cpu_node_id;
> > +    mc->default_ram_id = "mach-vmapple.ram";
> > +
> > +    object_register_sugar_prop(TYPE_VIRTIO_PCI, "disable-legacy",
> > +                               "on", true);
> > +}
> > +
> > +static void vmapple_instance_init(Object *obj)
> > +{
> > +    VMAppleMachineState *vms = VMAPPLE_MACHINE(obj);
> > +
> > +    vms->irqmap = irqmap;
> > +
> > +    object_property_add_uint64_ptr(obj, "uuid", &vms->uuid,
> > +                                   OBJ_PROP_FLAG_READWRITE);
> > +    object_property_set_description(obj, "uuid", "Machine UUID (SDOM)");
> > +}
> > +
> > +static const TypeInfo vmapple_machine_info = {
> > +    .name          = TYPE_VMAPPLE_MACHINE,
> > +    .parent        = TYPE_MACHINE,
> > +    .abstract      = true,
> > +    .instance_size = sizeof(VMAppleMachineState),
> > +    .class_size    = sizeof(VMAppleMachineClass),
> > +    .class_init    = vmapple_machine_class_init,
> > +    .instance_init = vmapple_instance_init,
> > +};
> > +
> > +static void machvmapple_machine_init(void)
> > +{
> > +    type_register_static(&vmapple_machine_info);
> > +}
> > +type_init(machvmapple_machine_init);
> > +
> > +static void vmapple_machine_9_2_options(MachineClass *mc)
> > +{
> > +}
> > +DEFINE_VMAPPLE_MACHINE_AS_LATEST(9, 2)
> > +
>
>
Re: [PATCH v8 15/15] hw/vmapple/vmapple: Add vmapple machine type
Posted by Akihiko Odaki 1 week, 5 days ago
On 2024/11/11 5:18, Phil Dennis-Jordan wrote:
> I've applied the majority of this feedback in the soon-to-be-posted v9
> of the patch set, thanks!
> 
> Just a few extra comments/replies below:
> 
> On Sun, 10 Nov 2024 at 08:38, Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
>>
>> On 2024/11/08 23:47, Phil Dennis-Jordan wrote:
>>> From: Alexander Graf <graf@amazon.com>
>>>
>>> Apple defines a new "vmapple" machine type as part of its proprietary
>>> macOS Virtualization.Framework vmm. This machine type is similar to the
>>> virt one, but with subtle differences in base devices, a few special
>>> vmapple device additions and a vastly different boot chain.
>>>
>>> This patch reimplements this machine type in QEMU. To use it, you
>>> have to have a readily installed version of macOS for VMApple,
>>> run on macOS with -accel hvf, pass the Virtualization.Framework
>>> boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
>>> and pass aux and root volume as virtio drives. In addition, you also
>>> need to find the machine UUID and pass that as -M vmapple,uuid= parameter:
>>>
>>> $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
>>>       -bios /System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
>>>       -drive file=aux,if=pflash,format=raw \
>>>       -drive file=root,if=pflash,format=raw \
>>>       -drive file=aux,if=none,id=aux,format=raw \
>>>       -device vmapple-virtio-aux,drive=aux \
>>>       -drive file=root,if=none,id=root,format=raw \
>>>       -device vmapple-virtio-root,drive=root
>>>
>>> With all these in place, you should be able to see macOS booting
>>> successfully.
>>>
>>> Known issues:
>>>    - Keyboard and mouse/tablet input is laggy. The reason for this is
>>>      either that macOS's XHCI driver is broken when the device/platform
>>>      does not support MSI/MSI-X, or there's some unfortunate interplay
>>>      with Qemu's XHCI implementation in this scenario.
>>>    - Currently only macOS 12 guests are supported. The boot process for
>>>      13+ will need further investigation and adjustment.
>>>
>>> Signed-off-by: Alexander Graf <graf@amazon.com>
>>> Co-authored-by: Phil Dennis-Jordan <phil@philjordan.eu>
>>> Signed-off-by: Phil Dennis-Jordan <phil@philjordan.eu>
>>> ---
>>>
>>> v3:
>>>    * Rebased on latest upstream, updated affinity and NIC creation
>>>      API usage
>>>    * Included Apple-variant virtio-blk in build dependency
>>>    * Updated API usage for setting 'redist-region-count' array-typed property
>>>      on GIC.
>>>    * Switched from virtio HID devices (for which macOS 12 does not contain
>>>      drivers) to an XHCI USB controller and USB HID devices.
>>>
>>> v4:
>>>    * Fixups for v4 changes to the other patches in the set.
>>>    * Corrected the assert macro to use
>>>    * Removed superfluous endian conversions corresponding to cfg's.
>>>    * Init error handling improvement.
>>>    * No need to select CPU type on TCG, as only HVF is supported.
>>>    * Machine type version bumped to 9.2
>>>    * #include order improved
>>>
>>> v5:
>>>    * Fixed memory reservation for ecam alias region.
>>>    * Better error handling setting properties on devices.
>>>    * Simplified the machine ECID/UUID extraction script and actually created a
>>>      file for it rather than quoting its code in documentation.
>>>
>>> v7:
>>>    * Tiny error handling fix, un-inlined function.
>>>
>>> v8:
>>>    * Use object_property_add_uint64_ptr rather than defining custom UUID
>>>      property get/set functions.
>>>
>>>    MAINTAINERS                 |   1 +
>>>    contrib/vmapple/uuid.sh     |   9 +
>>>    docs/system/arm/vmapple.rst |  60 ++++
>>>    docs/system/target-arm.rst  |   1 +
>>>    hw/vmapple/Kconfig          |  20 ++
>>>    hw/vmapple/meson.build      |   1 +
>>>    hw/vmapple/vmapple.c        | 638 ++++++++++++++++++++++++++++++++++++
>>>    7 files changed, 730 insertions(+)
>>>    create mode 100755 contrib/vmapple/uuid.sh
>>>    create mode 100644 docs/system/arm/vmapple.rst
>>>    create mode 100644 hw/vmapple/vmapple.c
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 9d0d26cb65f..9591fd44a34 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -2767,6 +2767,7 @@ R: Phil Dennis-Jordan <phil@philjordan.eu>
>>>    S: Maintained
>>>    F: hw/vmapple/*
>>>    F: include/hw/vmapple/*
>>> +F: docs/system/arm/vmapple.rst
>>>
>>>    Subsystems
>>>    ----------
>>> diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.sh
>>> new file mode 100755
>>> index 00000000000..956e8c3afed
>>> --- /dev/null
>>> +++ b/contrib/vmapple/uuid.sh
>>> @@ -0,0 +1,9 @@
>>> +#!/bin/sh
>>> +# Used for converting a guest provisioned using Virtualization.framework
>>> +# for use with the QEMU 'vmapple' aarch64 machine type.
>>> +#
>>> +# Extracts the Machine UUID from Virtualization.framework VM JSON file.
>>> +# (as produced by 'macosvm', passed as command line argument)
>>> +
>>> +plutil -extract machineId raw "$1" | base64 -d | plutil -extract ECID raw -
>>> +
>>> diff --git a/docs/system/arm/vmapple.rst b/docs/system/arm/vmapple.rst
>>> new file mode 100644
>>> index 00000000000..6a634fa4572
>>> --- /dev/null
>>> +++ b/docs/system/arm/vmapple.rst
>>> @@ -0,0 +1,60 @@
>>> +VMApple machine emulation
>>> +========================================================================================
>>> +
>>> +VMApple is the device model that the macOS built-in hypervisor called "Virtualization.framework"
>>> +exposes to Apple Silicon macOS guests. The "vmapple" machine model in QEMU implements the same
>>> +device model, but does not use any code from Virtualization.Framework.
>>> +
>>> +Prerequisites
>>> +-------------
>>> +
>>> +To run the vmapple machine model, you need to
>>> +
>>> + * Run on Apple Silicon
>>> + * Run on macOS 12.0 or above
>>> + * Have an already installed copy of a Virtualization.Framework macOS 12 virtual machine. I will
>>> +   assume that you installed it using the macosvm CLI.
>>
>> Add a URL to macosvm.
>>
>>> +
>>> +First, we need to extract the UUID from the virtual machine that you installed. You can do this
>>> +by running the shell script in contrib/vmapple/uuid.sh on the macosvm.json file.
>>> +
>>> +.. code-block:: bash
>>> +  :caption: uuid.sh script to extract the UUID from a macosvm.json file
>>> +
>>> +  $ contrib/vmapple/uuid.sh "path/to/macosvm.json"
>>> +
>>> +Now we also need to trim the aux partition. It contains metadata that we can just discard:
>>> +
>>> +.. code-block:: bash
>>> +  :caption: Command to trim the aux file
>>> +
>>> +  $ dd if="aux.img" of="aux.img.trimmed" bs=$(( 0x4000 )) skip=1
>>
>> Quoting is inconsistent. aux.img.trimmed is not quoted below but it is
>> quoted here.
>>
>>> +
>>> +How to run
>>> +----------
>>> +
>>> +Then, we can launch QEMU with the Virtualization.Framework pre-boot environment and the readily
>>> +installed target disk images. I recommend to port forward the VM's ssh and vnc ports to the host
>>> +to get better interactive access into the target system:
>>> +
>>> +.. code-block:: bash
>>> +  :caption: Example execution command line
>>> +
>>> +  $ UUID=$(uuid.sh macosvm.json)
>>> +  $ AVPBOOTER=/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin
>>> +  $ AUX=aux.img.trimmed
>>> +  $ DISK=disk.img
>>> +  $ qemu-system-aarch64 \
>>> +       -serial mon:stdio \
>>> +       -m 4G \
>>> +       -accel hvf \
>>> +       -M vmapple,uuid=$UUID \
>>> +       -bios $AVPBOOTER \
>>
>> $AUX and $DISK are quoted but $UUID and $AVPBOOTER are not.
>>
>>> +        -drive file="$AUX",if=pflash,format=raw \
>>> +        -drive file="$DISK",if=pflash,format=raw \
>>
>> These two lines are misaligned.
>>
>>> +       -drive file="$AUX",if=none,id=aux,format=raw \
>>> +       -drive file="$DISK",if=none,id=root,format=raw \
>>> +       -device vmapple-virtio-aux,drive=aux \
>>> +       -device vmapple-virtio-root,drive=root \
>>> +       -net user,ipv6=off,hostfwd=tcp::2222-:22,hostfwd=tcp::5901-:5900 \
>>> +       -net nic,model=virtio-net-pci \
>>
>> -net is a legacy option and creates a unnecessary indirection with hub.
>> Use -netdev and -device options. Also plug virtio-net-pci into a PCIe
>> root port as suggested in: docs/pcie.txt
> 
> I have not had any success with booting macOS with the block devices
> connected to PCIe root ports, and similarly the network doesn't come
> up if I attach just the virtio-net-pci device to one.
> 
> As far as I can tell (ioreg), Apple's own VMapple machine type does
> not use PCIe root ports, but I'm not 100% sure. The PCIe documentation
> is generally rather focused on x86 and on running Linux in the 'virt'
> machine type.

Without a PCIe root port, virtio devices will be forced to work as a PCI 
device (not PCIe) and to use its "legacy" interface. I don't think 
Virtualization.framework uses the legacy interface though.

> 
> In any case, a far more pressing issue is MSI support, without which
> the XHCI driver fails to receive most kinds of notifications from the
> USB controller. (Therefore keyboard/mouse is currently very laggy.)
> The arm 'virt' machine type supposedly supports it via the GIC ITS
> device, but I've so far not managed to transfer this to 'vmapple' in a
> way that makes the guest OS detect MSI support. Anyway, in its current
> iteration, the machine type is good enough to be used for some tasks,
> and we can always refine it later. (Another major limitation is of
> course guest support for OS versions > 12 - but those issues might
> even be related.)

Perhaps the MSI issue may be also related to PCIe.

> 
> One thing I am thinking about changing (as we've now missed the 9.2
> merge window anyway, sigh) as it would be non-backwards-compatible is
> replacing the vmapple-virtio-aux/vmapple-virtio-root pair of devices
> with just the the base type, vmapple-virtio-blk-pci, and adding a
> variant property to it. So:
> 
>         -device vmapple-virtio-blk-pci,variant=aux,drive=aux \
>         -device vmapple-virtio-blk-pci,variant=root,drive=root \
> 
> I think this would make hw/vmapple/virtio-blk.c somewhat more readable
> and also significantly shorter.

I agree. It will be a nice improvement.
Re: [PATCH v8 15/15] hw/vmapple/vmapple: Add vmapple machine type
Posted by Phil Dennis-Jordan 1 week, 5 days ago
On Mon, 11 Nov 2024 at 05:42, Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
>
> On 2024/11/11 5:18, Phil Dennis-Jordan wrote:
> > I've applied the majority of this feedback in the soon-to-be-posted v9
> > of the patch set, thanks!
> >
> > Just a few extra comments/replies below:
> >
> > On Sun, 10 Nov 2024 at 08:38, Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
> >>
> >> On 2024/11/08 23:47, Phil Dennis-Jordan wrote:
> >>> From: Alexander Graf <graf@amazon.com>
> >>>
> >>> Apple defines a new "vmapple" machine type as part of its proprietary
> >>> macOS Virtualization.Framework vmm. This machine type is similar to the
> >>> virt one, but with subtle differences in base devices, a few special
> >>> vmapple device additions and a vastly different boot chain.
> >>>
> >>> This patch reimplements this machine type in QEMU. To use it, you
> >>> have to have a readily installed version of macOS for VMApple,
> >>> run on macOS with -accel hvf, pass the Virtualization.Framework
> >>> boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
> >>> and pass aux and root volume as virtio drives. In addition, you also
> >>> need to find the machine UUID and pass that as -M vmapple,uuid= parameter:
> >>>
> >>> $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
> >>>       -bios /System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
> >>>       -drive file=aux,if=pflash,format=raw \
> >>>       -drive file=root,if=pflash,format=raw \
> >>>       -drive file=aux,if=none,id=aux,format=raw \
> >>>       -device vmapple-virtio-aux,drive=aux \
> >>>       -drive file=root,if=none,id=root,format=raw \
> >>>       -device vmapple-virtio-root,drive=root
> >>>
> >>> With all these in place, you should be able to see macOS booting
> >>> successfully.
> >>>
> >>> Known issues:
> >>>    - Keyboard and mouse/tablet input is laggy. The reason for this is
> >>>      either that macOS's XHCI driver is broken when the device/platform
> >>>      does not support MSI/MSI-X, or there's some unfortunate interplay
> >>>      with Qemu's XHCI implementation in this scenario.
> >>>    - Currently only macOS 12 guests are supported. The boot process for
> >>>      13+ will need further investigation and adjustment.
> >>>
> >>> Signed-off-by: Alexander Graf <graf@amazon.com>
> >>> Co-authored-by: Phil Dennis-Jordan <phil@philjordan.eu>
> >>> Signed-off-by: Phil Dennis-Jordan <phil@philjordan.eu>
> >>> ---
> >>>
> >>> v3:
> >>>    * Rebased on latest upstream, updated affinity and NIC creation
> >>>      API usage
> >>>    * Included Apple-variant virtio-blk in build dependency
> >>>    * Updated API usage for setting 'redist-region-count' array-typed property
> >>>      on GIC.
> >>>    * Switched from virtio HID devices (for which macOS 12 does not contain
> >>>      drivers) to an XHCI USB controller and USB HID devices.
> >>>
> >>> v4:
> >>>    * Fixups for v4 changes to the other patches in the set.
> >>>    * Corrected the assert macro to use
> >>>    * Removed superfluous endian conversions corresponding to cfg's.
> >>>    * Init error handling improvement.
> >>>    * No need to select CPU type on TCG, as only HVF is supported.
> >>>    * Machine type version bumped to 9.2
> >>>    * #include order improved
> >>>
> >>> v5:
> >>>    * Fixed memory reservation for ecam alias region.
> >>>    * Better error handling setting properties on devices.
> >>>    * Simplified the machine ECID/UUID extraction script and actually created a
> >>>      file for it rather than quoting its code in documentation.
> >>>
> >>> v7:
> >>>    * Tiny error handling fix, un-inlined function.
> >>>
> >>> v8:
> >>>    * Use object_property_add_uint64_ptr rather than defining custom UUID
> >>>      property get/set functions.
> >>>
> >>>    MAINTAINERS                 |   1 +
> >>>    contrib/vmapple/uuid.sh     |   9 +
> >>>    docs/system/arm/vmapple.rst |  60 ++++
> >>>    docs/system/target-arm.rst  |   1 +
> >>>    hw/vmapple/Kconfig          |  20 ++
> >>>    hw/vmapple/meson.build      |   1 +
> >>>    hw/vmapple/vmapple.c        | 638 ++++++++++++++++++++++++++++++++++++
> >>>    7 files changed, 730 insertions(+)
> >>>    create mode 100755 contrib/vmapple/uuid.sh
> >>>    create mode 100644 docs/system/arm/vmapple.rst
> >>>    create mode 100644 hw/vmapple/vmapple.c
> >>>
> >>> diff --git a/MAINTAINERS b/MAINTAINERS
> >>> index 9d0d26cb65f..9591fd44a34 100644
> >>> --- a/MAINTAINERS
> >>> +++ b/MAINTAINERS
> >>> @@ -2767,6 +2767,7 @@ R: Phil Dennis-Jordan <phil@philjordan.eu>
> >>>    S: Maintained
> >>>    F: hw/vmapple/*
> >>>    F: include/hw/vmapple/*
> >>> +F: docs/system/arm/vmapple.rst
> >>>
> >>>    Subsystems
> >>>    ----------
> >>> diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.sh
> >>> new file mode 100755
> >>> index 00000000000..956e8c3afed
> >>> --- /dev/null
> >>> +++ b/contrib/vmapple/uuid.sh
> >>> @@ -0,0 +1,9 @@
> >>> +#!/bin/sh
> >>> +# Used for converting a guest provisioned using Virtualization.framework
> >>> +# for use with the QEMU 'vmapple' aarch64 machine type.
> >>> +#
> >>> +# Extracts the Machine UUID from Virtualization.framework VM JSON file.
> >>> +# (as produced by 'macosvm', passed as command line argument)
> >>> +
> >>> +plutil -extract machineId raw "$1" | base64 -d | plutil -extract ECID raw -
> >>> +
> >>> diff --git a/docs/system/arm/vmapple.rst b/docs/system/arm/vmapple.rst
> >>> new file mode 100644
> >>> index 00000000000..6a634fa4572
> >>> --- /dev/null
> >>> +++ b/docs/system/arm/vmapple.rst
> >>> @@ -0,0 +1,60 @@
> >>> +VMApple machine emulation
> >>> +========================================================================================
> >>> +
> >>> +VMApple is the device model that the macOS built-in hypervisor called "Virtualization.framework"
> >>> +exposes to Apple Silicon macOS guests. The "vmapple" machine model in QEMU implements the same
> >>> +device model, but does not use any code from Virtualization.Framework.
> >>> +
> >>> +Prerequisites
> >>> +-------------
> >>> +
> >>> +To run the vmapple machine model, you need to
> >>> +
> >>> + * Run on Apple Silicon
> >>> + * Run on macOS 12.0 or above
> >>> + * Have an already installed copy of a Virtualization.Framework macOS 12 virtual machine. I will
> >>> +   assume that you installed it using the macosvm CLI.
> >>
> >> Add a URL to macosvm.
> >>
> >>> +
> >>> +First, we need to extract the UUID from the virtual machine that you installed. You can do this
> >>> +by running the shell script in contrib/vmapple/uuid.sh on the macosvm.json file.
> >>> +
> >>> +.. code-block:: bash
> >>> +  :caption: uuid.sh script to extract the UUID from a macosvm.json file
> >>> +
> >>> +  $ contrib/vmapple/uuid.sh "path/to/macosvm.json"
> >>> +
> >>> +Now we also need to trim the aux partition. It contains metadata that we can just discard:
> >>> +
> >>> +.. code-block:: bash
> >>> +  :caption: Command to trim the aux file
> >>> +
> >>> +  $ dd if="aux.img" of="aux.img.trimmed" bs=$(( 0x4000 )) skip=1
> >>
> >> Quoting is inconsistent. aux.img.trimmed is not quoted below but it is
> >> quoted here.
> >>
> >>> +
> >>> +How to run
> >>> +----------
> >>> +
> >>> +Then, we can launch QEMU with the Virtualization.Framework pre-boot environment and the readily
> >>> +installed target disk images. I recommend to port forward the VM's ssh and vnc ports to the host
> >>> +to get better interactive access into the target system:
> >>> +
> >>> +.. code-block:: bash
> >>> +  :caption: Example execution command line
> >>> +
> >>> +  $ UUID=$(uuid.sh macosvm.json)
> >>> +  $ AVPBOOTER=/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin
> >>> +  $ AUX=aux.img.trimmed
> >>> +  $ DISK=disk.img
> >>> +  $ qemu-system-aarch64 \
> >>> +       -serial mon:stdio \
> >>> +       -m 4G \
> >>> +       -accel hvf \
> >>> +       -M vmapple,uuid=$UUID \
> >>> +       -bios $AVPBOOTER \
> >>
> >> $AUX and $DISK are quoted but $UUID and $AVPBOOTER are not.
> >>
> >>> +        -drive file="$AUX",if=pflash,format=raw \
> >>> +        -drive file="$DISK",if=pflash,format=raw \
> >>
> >> These two lines are misaligned.
> >>
> >>> +       -drive file="$AUX",if=none,id=aux,format=raw \
> >>> +       -drive file="$DISK",if=none,id=root,format=raw \
> >>> +       -device vmapple-virtio-aux,drive=aux \
> >>> +       -device vmapple-virtio-root,drive=root \
> >>> +       -net user,ipv6=off,hostfwd=tcp::2222-:22,hostfwd=tcp::5901-:5900 \
> >>> +       -net nic,model=virtio-net-pci \
> >>
> >> -net is a legacy option and creates a unnecessary indirection with hub.
> >> Use -netdev and -device options. Also plug virtio-net-pci into a PCIe
> >> root port as suggested in: docs/pcie.txt
> >
> > I have not had any success with booting macOS with the block devices
> > connected to PCIe root ports, and similarly the network doesn't come
> > up if I attach just the virtio-net-pci device to one.
> >
> > As far as I can tell (ioreg), Apple's own VMapple machine type does
> > not use PCIe root ports, but I'm not 100% sure. The PCIe documentation
> > is generally rather focused on x86 and on running Linux in the 'virt'
> > machine type.
>
> Without a PCIe root port, virtio devices will be forced to work as a PCI
> device (not PCIe) and to use its "legacy" interface. I don't think
> Virtualization.framework uses the legacy interface though.

I'm not sure that's true, the GPEX controller purports to support PCIe
in some form. And even if it is falling back to legacy Virtio mode,
the guest OS doesn't appear to care. The guest's view of the PCI(e?)
setup looks extremely similar between Virtualization.framework and the
Qemu vmapple machine type we're adding here. I've attached `ioreg`
output for some relevant I/O registry node types for both Qemu
(vmapple-qemu-ioreg.txt) and macosvm (vmapple-vzfwk-ioreg.txt).

> > In any case, a far more pressing issue is MSI support, without which
> > the XHCI driver fails to receive most kinds of notifications from the
> > USB controller. (Therefore keyboard/mouse is currently very laggy.)
> > The arm 'virt' machine type supposedly supports it via the GIC ITS
> > device, but I've so far not managed to transfer this to 'vmapple' in a
> > way that makes the guest OS detect MSI support. Anyway, in its current
> > iteration, the machine type is good enough to be used for some tasks,
> > and we can always refine it later. (Another major limitation is of
> > course guest support for OS versions > 12 - but those issues might
> > even be related.)
>
> Perhaps the MSI issue may be also related to PCIe.

MSI is certainly intended to be the "native" method for implementing
interrupts on PCIe. However:

Reviewing the ioreg output, I actually see no evidence that
Virtualization.framework is even using MSIs. In particular, take note
of the IOInterruptControllers property on each IOPCIDevice. In both
VZ.fwk and Qemu vmapple, the Virtio and XHCI devices use this:

                |   "IOInterruptControllers" = ("IOInterruptController00000037")

This in turn simply corresponds to the GIC:

            +-o AppleARMGICv3  <class AppleARMGICv3, id 0x10000016b,
registered, matched, active, busy 0 (4 ms), retain 6>
                {
…
                  "InterruptControllerName" = "IOInterruptController00000037"
…

On a physical Mac, or an x86-64 VM, there will additionally be a
separate interrupt controller object for MSIs, which is a subclass of
IOPCIMessagedInterruptController.

So perhaps the problem isn't lack of MSIs after all (the guest even
may not support them) but rather that Qemu's XHCI controller doesn't
implement the case of 1 interrupter in a way that macOS's XHCI driver
supports.

(Qemu's XHCI controller defaults to 16 interrupters, with 1 MSI vector
per interrupter. When MSIs are unavailable, numintrs remains 16 even
though the legacy interrupt mechanism only provides a single IRQ line,
and when the XHCI controller tries to signal vector 1 or higher, the
guest isn't notified. Unfortunately, a quick hack that detects lack of
MSI support and sets numintrs to 1 makes things even worse. I've tried
this with an x86-64 VM as well, and the macOS guest stops being able
to drive the XHCI controller correctly when I force numintrs to 1.

Linux guests were able to cope with it if I remember correctly though,
so I'm not sure where exactly the problem lies:
 A. Does Qemu implement numintrs=1 badly, but Linux's driver is
tolerant enough to make it work regardless?
 B. Or does macOS's XHCI driver expect a very specific device
behaviour with numintrs=1, which doesn't match Qemu's?
Either way, we can't fix it on the guest side here…

Interestingly, Virtualization.framework also exposes an
AppleVirtIOUSBDeviceController to the guest. This isn't used for the
virtual keyboard and digitiser though, and I've not found any protocol
information on USB-over-Virtio, so who knows if that would be in any
way useful as a generic USB controller.)

Long story short: I think we're best off leaving the PCI(e) setup as
it is for the moment until we figure out exactly what's needed to fix
the XHCI controller.

> > One thing I am thinking about changing (as we've now missed the 9.2
> > merge window anyway, sigh) as it would be non-backwards-compatible is
> > replacing the vmapple-virtio-aux/vmapple-virtio-root pair of devices
> > with just the the base type, vmapple-virtio-blk-pci, and adding a
> > variant property to it. So:
> >
> >         -device vmapple-virtio-blk-pci,variant=aux,drive=aux \
> >         -device vmapple-virtio-blk-pci,variant=root,drive=root \
> >
> > I think this would make hw/vmapple/virtio-blk.c somewhat more readable
> > and also significantly shorter.
>
> I agree. It will be a nice improvement.
% ioreg -trc IOPCIDevice
+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci106b,1a05@0  <class IOPCIDevice, id 0x100000170, registered, matched, active, busy 0 (2 ms), retain 8>
                  {
                    "compatible" = <"pci106b,1a05","pciclass,060000">
                    "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                    "IOName" = "pci106b,1a05"
                    "reg" = <0000000000000000000000000000000000000000>
                    "device-id" = <051a0000>
                    "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                    "IOPCIResourced" = Yes
                    "revision-id" = <00000000>
                    "vendor-id" = <6b100000>
                    "pcidebug" = "0:0:0"
                    "class-code" = <00000600>
                    "name" = <"pci106b,1a05">
                  }
                  

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o ethernet@1  <class IOPCIDevice, id 0x100000171, registered, matched, active, busy 0 (14 ms), retain 9>
                | {
                |   "assigned-addresses" = <100800820080015000000000004000000000000018080082c0d50150000000004000000000000000>
                |   "IOInterruptSpecifiers" = (<4100000001000000>)
                |   "class-code" = <00000200>
                |   "IODeviceMemory" = (({"address"=1342275584,"length"=16384}),({"address"=1342297536,"length"=64}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <f41a0000>
                |   "name" = <"ethernet">
                |   "device-id" = <41100000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,41","pci1af4,1041","pciclass,020000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <41000000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:1:0"
                |   "IOName" = "ethernet"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <000800000000000000000000000000000000000010080002000000000000000000400000000000001808000200000000000000004000000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000185, registered, matched, active, busy 0 (11 ms), re$
                  +-o AppleVirtIONetwork  <class AppleVirtIONetwork, id 0x10000018e, registered, matched, active, busy 0 (0 ms), retain 9>
                    +-o en0  <class IOEthernetInterface, id 0x10000020b, registered, matched, active, busy 0 (0 ms), retain 13>
                      +-o IONetworkStack  <class IONetworkStack, id 0x1000001ba, registered, matched, active, busy 0 (0 ms), retain 12>
                        +-o IONetworkStackUserClient  <class IONetworkStackUserClient, id 0x100000291, !registered, !matched, active, busy 0$

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci1af4,1043@5  <class IOPCIDevice, id 0x100000172, registered, matched, active, busy 0 (14 ms), retain 9>
                | {
                |   "assigned-addresses" = <10280082004001500000000000400000000000001828008200d40150000000008000000000000000>
                |   "IOInterruptSpecifiers" = (<4500000001000000>)
                |   "class-code" = <00800700>
                |   "IODeviceMemory" = (({"address"=1342259200,"length"=16384}),({"address"=1342297088,"length"=128}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <f41a0000>
                |   "name" = <"pci1af4,1043">
                |   "device-id" = <43100000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,43","pci1af4,1043","pciclass,078000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <43000000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:5:0"
                |   "IOName" = "pci1af4,1043"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <002800000000000000000000000000000000000010280002000000000000000000400000000000001828000200000000000000008000000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000186, registered, matched, active, busy 0 (10 ms), re$
                  +-o AppleVirtIOConsole  <class AppleVirtIOConsole, id 0x10000018f, registered, matched, active, busy 0 (9 ms), retain 8>
                    +-o IOSerialStreamSync  <class IOSerialStreamSync, id 0x100000191, registered, matched, active, busy 0 (9 ms), retain 7>
                      +-o IOSerialBSDClient  <class IOSerialBSDClient, id 0x100000199, registered, matched, active, busy 0 (0 ms), retain 5>

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci106b,1a00@6  <class IOPCIDevice, id 0x100000173, registered, matched, active, busy 0 (89 ms), retain 10>
                | {
                |   "assigned-addresses" = <10300082000001500000000000400000000000001830008280d50150000000004000000000000000>
                |   "IOInterruptSpecifiers" = (<4600000001000000>)
                |   "class-code" = <00800100>
                |   "IODeviceMemory" = (({"address"=1342242816,"length"=16384}),({"address"=1342297472,"length"=64}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <6b100000>
                |   "name" = <"pci106b,1a00">
                |   "device-id" = <001a0000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,42","pci106b,1a00","pciclass,018000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <42000000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:6:0"
                |   "IOName" = "pci106b,1a00"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <003000000000000000000000000000000000000010300002000000000000000000400000000000001830000200000000000000004000000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000187, registered, matched, active, busy 0 (85 ms), re$
                  +-o AppleVirtIOStorageDevice  <class AppleVirtIOStorageDevice, id 0x100000190, registered, matched, active, busy 0 (84 ms)$
                    +-o AppleVirtIODiskStorageDevice@1  <class AppleVirtIODiskStorageDevice, id 0x100000192, registered, matched, active, bu$
                      +-o IOBlockStorageDriver  <class IOBlockStorageDriver, id 0x100000193, registered, matched, active, busy 0 (84 ms), re$
                        +-o Apple Inc. VirtIO Block Media  <class IOMedia, id 0x100000195, registered, matched, active, busy 0 (84 ms), reta$
                          +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001a0, registered, matched, active, busy 0 (0 ms), retain $
                          +-o IOGUIDPartitionScheme  <class IOGUIDPartitionScheme, id 0x1000001e2, !registered, !matched, active, busy 0 (77$
                            +-o iBootSystemContainer@1  <class IOMedia, id 0x1000001f2, registered, matched, active, busy 0 (29 ms), retain $
                            | +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001f5, registered, matched, active, busy 0 (0 ms), ret$
                            | +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001f6, !registered, !matched, active, $
                            |   +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001fb, registered, matched, active, busy 0 (28 ms), reta$
                            |     +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x1000001fc, registered, matched, active, $
                            |     +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x100000218, registered, matched, active, busy 0 (0 $
                            |       +-o iSCPreboot@1  <class AppleAPFSVolume, id 0x10000021d, registered, matched, active, busy 0 (0 ms), re$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000221, registered, matched, ac$
                            |       +-o xART@2  <class AppleAPFSVolume, id 0x10000021e, registered, matched, active, busy 0 (0 ms), retain 1$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000223, registered, matched, ac$
                            |       +-o Hardware@3  <class AppleAPFSVolume, id 0x10000021f, registered, matched, active, busy 0 (0 ms), reta$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000225, registered, matched, ac$
                            |       +-o Recovery@4  <class AppleAPFSVolume, id 0x100000220, registered, matched, active, busy 0 (0 ms), reta$
                            |         +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000227, registered, matched, ac$
                            +-o Container@2  <class IOMedia, id 0x1000001f3, registered, matched, active, busy 0 (77 ms), retain 12>
                            | +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001f7, registered, matched, active, busy 0 (0 ms), ret$
                            | +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001f8, !registered, !matched, active, $
                            |   +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001ff, registered, matched, active, busy 0 (77 ms), reta$
                            |     +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x100000200, registered, matched, active, $
                            |     +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x10000021a, registered, matched, active, busy 0 (1 $
                            |       +-o Macintosh HD@1  <class AppleAPFSVolume, id 0x10000022f, registered, matched, active, busy 0 (0 ms), $
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000235, registered, matched, ac$
                            |       | +-o com.apple.os.update-7E5F621E717386C0F6DD5D927F23C2AFB35C42381A5EA61C993232428F0E6E55@1  <class App$
                            |       |   +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x100000243, registered, matched, active, busy 0 ($
                            |       +-o Preboot@2  <class AppleAPFSVolume, id 0x100000230, registered, matched, active, busy 0 (0 ms), retai$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000237, registered, matched, ac$
                            |       +-o Recovery@3  <class AppleAPFSVolume, id 0x100000231, registered, matched, active, busy 0 (0 ms), reta$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000239, registered, matched, ac$
                            |       +-o Update@4  <class AppleAPFSVolume, id 0x100000232, registered, matched, active, busy 0 (0 ms), retain$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000023b, registered, matched, ac$
                            |       +-o Data@5  <class AppleAPFSVolume, id 0x100000233, registered, matched, active, busy 0 (0 ms), retain 1$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000023e, registered, matched, ac$
                            |       +-o VM@6  <class AppleAPFSVolume, id 0x100000234, registered, matched, active, busy 0 (0 ms), retain 11>
                            |         +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000240, registered, matched, ac$
                            +-o RecoveryOSContainer@3  <class IOMedia, id 0x1000001f4, registered, matched, active, busy 0 (44 ms), retain 1$
                              +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001f9, registered, matched, active, busy 0 (0 ms), ret$
                              +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001fa, !registered, !matched, active, $
                                +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x100000203, registered, matched, active, busy 0 (43 ms), reta$
                                  +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x100000204, registered, matched, active, $
                                  +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x10000021c, registered, matched, active, busy 0 (0 $
                                    +-o Recovery@1  <class AppleAPFSVolume, id 0x100000229, registered, matched, active, busy 0 (0 ms), reta$
                                    | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000022b, registered, matched, ac$
                                    +-o Update@2  <class AppleAPFSVolume, id 0x10000022a, registered, matched, active, busy 0 (0 ms), retain$
                                      +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000022d, registered, matched, ac$

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci106b,1a00@7  <class IOPCIDevice, id 0x100000174, registered, matched, active, busy 0 (7 ms), retain 10>
                | {
                |   "assigned-addresses" = <1038008200c000500000000000400000000000001838008240d50150000000004000000000000000>
                |   "IOInterruptSpecifiers" = (<4700000001000000>)
                |   "class-code" = <00800100>
                |   "IODeviceMemory" = (({"address"=1342226432,"length"=16384}),({"address"=1342297408,"length"=64}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <6b100000>
                |   "name" = <"pci106b,1a00">
                |   "device-id" = <001a0000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,42","pci106b,1a00","pciclass,018000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <42000000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:7:0"
                |   "IOName" = "pci106b,1a00"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <003800000000000000000000000000000000000010380002000000000000000000400000000000001838000200000000000000004000000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000188, registered, matched, active, busy 0 (3 ms), ret$
                  +-o AppleVirtIOStorageDevice  <class AppleVirtIOStorageDevice, id 0x100000194, registered, matched, active, busy 0 (0 ms),$
                    +-o AppleVirtIOUtilStorageDevice  <class AppleVirtIOUtilStorageDevice, id 0x100000198, registered, matched, active, busy$
                      +-o AppleEmbeddedSimpleSPINORFlasherDriver  <class AppleEmbeddedSimpleSPINORFlasherDriver, id 0x10000019a, registered,$
                      +-o AppleVirtIONVRAM  <class AppleVirtIONVRAM, id 0x10000019b, registered, matched, active, busy 0 (0 ms), retain 5>
                      +-o AppleNVMeNamespaceDevice  <class AppleNVMeNamespaceDevice, id 0x1000001a9, registered, matched, active, busy 0 (0 $
                      +-o AppleNVMeEAN  <class AppleNVMeEAN, id 0x1000001ac, registered, matched, active, busy 0 (0 ms), retain 6>

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci1af4,1044@8  <class IOPCIDevice, id 0x100000175, registered, matched, active, busy 0 (5 ms), retain 9>
                | {
                |   "assigned-addresses" = <10400082008000500000000000400000000000001840008200d50150000000004000000000000000>
                |   "IOInterruptSpecifiers" = (<4800000001000000>)
                |   "class-code" = <00001000>
                |   "IODeviceMemory" = (({"address"=1342210048,"length"=16384}),({"address"=1342297344,"length"=64}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <f41a0000>
                |   "name" = <"pci1af4,1044">
                |   "device-id" = <44100000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,44","pci1af4,1044","pciclass,100000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <44000000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:8:0"
                |   "IOName" = "pci1af4,1044"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <004000000000000000000000000000000000000010400002000000000000000000400000000000001840000200000000000000004000000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000189, registered, matched, active, busy 0 (2 ms), ret$
                  +-o AppleVirtIOEntropy  <class AppleVirtIOEntropy, id 0x100000196, registered, matched, active, busy 0 (0 ms), retain 6>

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci106b,1a06@9  <class IOPCIDevice, id 0x100000176, registered, matched, active, busy 0 (143 ms), retain 12>
                | {
                |   "compatible" = <"pci106b,1a06","pciclass,0c0330">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "IOName" = "pci106b,1a06"
                |   "reg" = <004800000000000000000000000000000000000010480002000000000000000000100000000000001448000200000000000000000004000$
                |   "assigned-addresses" = <1048008200c001500000000000100000000000001448008200d00150000000000004000000000000>
                |   "device-id" = <061a0000>
                |   "IOPowerManagement" = {"ChildrenPowerState"=2,"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxP$
                |   "IOPCIResourced" = Yes
                |   "IODeviceMemory" = (({"address"=1342291968,"length"=4096}),({"address"=1342296064,"length"=1024}))
                |   "revision-id" = <00000000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <6b100000>
                |   "pcidebug" = "0:9:0"
                |   "class-code" = <30030c00>
                |   "IOInterruptSpecifiers" = (<4900000001000000>)
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "IOReportLegendPublic" = Yes
                |   "name" = <"pci106b,1a06">
                | }
                | 
                +-o AppleUSBXHCIPCI@09000000  <class AppleUSBXHCIPCI, id 0x10000018a, registered, matched, active, busy 0 (136 ms), retain 1$
                  +-o AppleUSB30XHCIPort@09100000  <class AppleUSB30XHCIPort, id 0x1000001c3, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09200000  <class AppleUSB30XHCIPort, id 0x1000001c4, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09300000  <class AppleUSB30XHCIPort, id 0x1000001c5, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09400000  <class AppleUSB30XHCIPort, id 0x1000001c6, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09500000  <class AppleUSB30XHCIPort, id 0x1000001c7, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09600000  <class AppleUSB30XHCIPort, id 0x1000001c8, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09700000  <class AppleUSB30XHCIPort, id 0x1000001c9, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09800000  <class AppleUSB30XHCIPort, id 0x1000001ca, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09900000  <class AppleUSB20XHCIPort, id 0x1000001cb, registered, matched, active, busy 0 (133 ms), $
                  | +-o Virtual USB Keyboard@09900000  <class IOUSBHostDevice, id 0x10000026b, registered, matched, active, busy 0 (132 ms),$
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x10000026e, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x10000026f, registered, matched, active, busy 0 (132 ms), ret$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x1000002c3, registered, matched, active, busy 0 (2 ms)$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002d3, registered, matched, active, busy 0 (2 ms), retain 10>
                  |         +-o AppleUserHIDEventDriver  <class AppleUserHIDEventService, id 0x1000002d5, registered, matched, active, busy $
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002d9, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@09a00000  <class AppleUSB20XHCIPort, id 0x1000001cc, registered, matched, active, busy 0 (131 ms), $
                  | +-o Virtual USB Digitizer@09a00000  <class IOUSBHostDevice, id 0x100000266, registered, matched, active, busy 0 (130 ms)$
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x100000269, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x10000026a, registered, matched, active, busy 0 (128 ms), ret$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x1000002c2, registered, matched, active, busy 0 (1 ms)$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002cb, registered, matched, active, busy 0 (1 ms), retain 8>
                  |         +-o AppleVirtualPlatformHIDEventDriver  <class AppleVirtualPlatformHIDEventDriver, id 0x1000002cc, registered, m$
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002d0, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@09b00000  <class AppleUSB20XHCIPort, id 0x1000001cd, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09c00000  <class AppleUSB20XHCIPort, id 0x1000001ce, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09d00000  <class AppleUSB20XHCIPort, id 0x1000001cf, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09e00000  <class AppleUSB20XHCIPort, id 0x1000001d0, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09f00000  <class AppleUSB20XHCIPort, id 0x1000001d1, registered, matched, active, busy 0 (0 ms), re$

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci106b,1a01@A  <class IOPCIDevice, id 0x100000177, registered, matched, active, busy 0 (13 ms), retain 9>
                | {
                |   "assigned-addresses" = <105000820040005000000000004000000000000018500082c0d40150000000004000000000000000>
                |   "IOInterruptSpecifiers" = (<4a00000001000000>)
                |   "class-code" = <fe030c00>
                |   "IODeviceMemory" = (({"address"=1342193664,"length"=16384}),({"address"=1342297280,"length"=64}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <6b100000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <6b100000>
                |   "name" = <"pci106b,1a01">
                |   "device-id" = <011a0000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci106b,1a01","pciclass,0c03fe">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <011a0000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:10:0"
                |   "IOName" = "pci106b,1a01"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <005000000000000000000000000000000000000010500002000000000000000000400000000000001850000200000000000000004000000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x10000018b, registered, matched, active, busy 0 (9 ms), ret$
                  +-o AppleVirtIOUSBConfiguration  <class AppleVirtIOUSBConfiguration, id 0x100000197, registered, matched, active, busy 0 ($
                    +-o AppleVirtIOUSBDeviceController@0  <class AppleVirtIOUSBDeviceController, id 0x1000001a2, registered, matched, active$
                      +-o IOUSBDeviceConfigurator  <class IOUSBDeviceConfigurator, id 0x1000001a3, !registered, !matched, active, busy 0, re$
                      +-o AppleUSBNCMControl@0  <class IOUSBDeviceInterface, id 0x1000001a5, registered, matched, active, busy 0 (0 ms), ret$
                      | +-o AppleUSBDeviceNCMControl@0  <class AppleUSBDeviceNCMControl, id 0x1000001aa, registered, matched, active, busy 0$
                      +-o AppleUSBNCMData@1  <class IOUSBDeviceInterface, id 0x1000001a6, registered, matched, active, busy 0 (7 ms), retain$
                      | +-o AppleUSBDeviceNCMData  <class AppleUSBDeviceNCMData, id 0x1000001ad, registered, matched, active, busy 0 (0 ms),$
                      |   +-o en1  <class IOEthernetInterface, id 0x10000020d, registered, matched, active, busy 0 (0 ms), retain 11>
                      |     +-o IONetworkStack  <class IONetworkStack, id 0x1000001ba, registered, matched, active, busy 0 (0 ms), retain 12$
                      |       +-o IONetworkStackUserClient  <class IONetworkStackUserClient, id 0x100000291, !registered, !matched, active, $
                      +-o AppleUSBNCMControlAux@2  <class IOUSBDeviceInterface, id 0x1000001a7, registered, matched, active, busy 0 (1 ms), $
                      | +-o AppleUSBDeviceNCMControl@2  <class AppleUSBDeviceNCMControl, id 0x1000001b6, registered, matched, active, busy 0$
                      +-o AppleUSBNCMDataAux@3  <class IOUSBDeviceInterface, id 0x1000001a8, registered, matched, active, busy 0 (4 ms), ret$
                        +-o AppleUSBDeviceNCMData  <class AppleUSBDeviceNCMData, id 0x1000001b0, registered, matched, active, busy 0 (1 ms),$
                          +-o anpi0  <class AppleUSBDeviceNCMPrivateEthernetInterface, id 0x1000001e1, registered, matched, active, busy 0 ($
                            +-o IONetworkStack  <class IONetworkStack, id 0x1000001ba, registered, matched, active, busy 0 (0 ms), retain 12$
                              +-o IONetworkStackUserClient  <class IONetworkStackUserClient, id 0x100000291, !registered, !matched, active, $

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8861 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (285 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (227 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (224 ms), ret$
              +-o pci106b,1a04@B  <class IOPCIDevice, id 0x100000178, registered, matched, active, busy 0 (4 ms), retain 8>
                  {
                    "subsystem-vendor-id" = <6b100000>
                    "compatible" = <"pci106b,1a04","pciclass,000000">
                    "IOName" = "pci106b,1a04"
                    "reg" = <005800000000000000000000000000000000000010580002000000000000000000400000000000001858000200000000000000004000000$
                    "assigned-addresses" = <10580082000000500000000000400000000000001858008280d40150000000004000000000000000>
                    "device-id" = <041a0000>
                    "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                    "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                    "IOPCIResourced" = Yes
                    "revision-id" = <01000000>
                    "IODeviceMemory" = (({"address"=1342177280,"length"=16384}),({"address"=1342297216,"length"=64}))
                    "vendor-id" = <6b100000>
                    "pcidebug" = "0:11:0"
                    "class-code" = <00000000>
                    "subsystem-id" = <041a0000>
                    "name" = <"pci106b,1a04">
                  }
                  


% ioreg -trc IOInterruptController
+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8862 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (286 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (284 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (281 ms), retain 12$
          +-o gic@0  <class AppleARMIODevice, id 0x100000117, registered, matched, active, busy 0 (5 ms), retain 9>
            +-o AppleARMGICv3  <class AppleARMGICv3, id 0x10000016b, registered, matched, active, busy 0 (4 ms), retain 6>
                {
                  "IOClass" = "AppleARMGICv3"
                  "CFBundleIdentifier" = "com.apple.driver.AppleARMGIC"
                  "IOProviderClass" = "AppleARMIODevice"
                  "SystemPowerProfileOverrideDict" = {"System Sleep Timer"=2147483647}
                  "IOProbeScore" = 1000
                  "IONameMatch" = "ARM,gicv3"
                  "IOMatchedAtBoot" = Yes
                  "IOMatchCategory" = "IODefaultMatchCategory"
                  "IONameMatched" = "ARM,gicv3"
                  "IOPersonalityPublisher" = "com.apple.driver.AppleARMGIC"
                  "InterruptControllerName" = "IOInterruptController00000037"
                  "CFBundleIdentifierKernel" = "com.apple.driver.AppleARMGIC"
                }
                

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (8862 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (286 ms), retain 21$
      +-o PassthruInterruptController  <class PassthruInterruptController, id 0x100000167, registered, matched, active, busy 0 (0 ms), retai$
          {
            "IOPlatformInterruptController" = Yes
          }

% ioreg -trc AppleUSBXHCI
+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (6657 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (310 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (309 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (306 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (253 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (250 ms), ret$
              +-o pci106b,1a06@9  <class IOPCIDevice, id 0x100000176, registered, matched, active, busy 0 (184 ms), retain 12>
                +-o AppleUSBXHCIPCI@09000000  <class AppleUSBXHCIPCI, id 0x10000018a, registered, matched, active, busy 0 (170 ms), retain 1$
                  | {
                  |   "IOClass" = "AppleUSBXHCIPCI"
                  |   "CFBundleIdentifier" = "com.apple.driver.usb.AppleUSBXHCIPCI"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "kUSBSleepSupported" = No
                  |   "IOPCIClassMatch" = "0x0c033000"
                  |   "IOPCITunnelCompatible" = Yes
                  |   "IOPowerManagement" = {"ChildrenPowerState"=3,"DevicePowerState"=0,"CurrentPowerState"=3,"CapabilityFlags"=32768,"MaxP$
                  |   "IOProbeScore" = 0
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOPCIPauseCompatible" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "locationID" = 150994944
                  |   "Revision" = <0103>
                  |   "controller-statistics" = {"kControllerStatIOCount"=62,"kControllerStatPowerStateTime"={"kPowerStateOff"="0ms (0%)","k$
                  |   "IOPersonalityPublisher" = "com.apple.driver.usb.AppleUSBXHCIPCI"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.usb.AppleUSBXHCIPCI"
                  |   "UsbHostControllerStreamPolicy" = 0
                  |   "IOPCIUseDeviceMapper" = Yes
                  | }
                  | 
                  +-o AppleUSB30XHCIPort@09100000  <class AppleUSB30XHCIPort, id 0x1000001ff, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09200000  <class AppleUSB30XHCIPort, id 0x100000200, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09300000  <class AppleUSB30XHCIPort, id 0x100000201, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09400000  <class AppleUSB30XHCIPort, id 0x100000202, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09500000  <class AppleUSB30XHCIPort, id 0x100000203, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09600000  <class AppleUSB30XHCIPort, id 0x100000204, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09700000  <class AppleUSB30XHCIPort, id 0x100000205, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@09800000  <class AppleUSB30XHCIPort, id 0x100000206, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09900000  <class AppleUSB20XHCIPort, id 0x100000207, registered, matched, active, busy 0 (169 ms), $
                  | +-o Virtual USB Keyboard@09900000  <class IOUSBHostDevice, id 0x100000266, registered, matched, active, busy 0 (169 ms),$
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x100000269, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x10000026a, registered, matched, active, busy 0 (167 ms), ret$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x1000002b2, registered, matched, active, busy 0 (2 ms)$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002bf, registered, matched, active, busy 0 (2 ms), retain 10>
                  |         +-o AppleUserHIDEventDriver  <class AppleUserHIDEventService, id 0x1000002c6, registered, matched, active, busy $
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002ca, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@09a00000  <class AppleUSB20XHCIPort, id 0x100000208, registered, matched, active, busy 0 (166 ms), $
                  | +-o Virtual USB Digitizer@09a00000  <class IOUSBHostDevice, id 0x10000026c, registered, matched, active, busy 0 (166 ms)$
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x10000026f, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x100000270, registered, matched, active, busy 0 (165 ms), ret$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x1000002b1, registered, matched, active, busy 0 (1 ms)$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002bd, registered, matched, active, busy 0 (1 ms), retain 8>
                  |         +-o AppleVirtualPlatformHIDEventDriver  <class AppleVirtualPlatformHIDEventDriver, id 0x1000002c0, registered, m$
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002c5, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@09b00000  <class AppleUSB20XHCIPort, id 0x100000209, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09c00000  <class AppleUSB20XHCIPort, id 0x10000020a, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09d00000  <class AppleUSB20XHCIPort, id 0x10000020b, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09e00000  <class AppleUSB20XHCIPort, id 0x10000020c, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@09f00000  <class AppleUSB20XHCIPort, id 0x10000020d, registered, matched, active, busy 0 (0 ms), re$



% ioreg -trc AppleVirtIOPCITransport
+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (6662 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (311 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (310 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (307 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (253 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (250 ms), ret$
              +-o ethernet@1  <class IOPCIDevice, id 0x100000171, registered, matched, active, busy 0 (14 ms), retain 9>
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000185, registered, matched, active, busy 0 (10 ms), re$
                  | {
                  |   "IOPCIPrimaryMatch" = "0x00001af4&0x0000FFFF"
                  |   "CFBundleIdentifier" = "com.apple.driver.AppleVirtIO"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "IOClass" = "AppleVirtIOPCITransport"
                  |   "IOPersonalityPublisher" = "com.apple.driver.AppleVirtIO"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.AppleVirtIO"
                  |   "IOProbeScore" = 10000
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "built-in" = <00>
                  | }
                  | 
                  +-o AppleVirtIONetwork  <class AppleVirtIONetwork, id 0x10000018e, registered, matched, active, busy 0 (0 ms), retain 9>
                    +-o en0  <class IOEthernetInterface, id 0x1000001f7, registered, matched, active, busy 0 (0 ms), retain 13>
                      +-o IONetworkStack  <class IONetworkStack, id 0x1000001cb, registered, matched, active, busy 0 (0 ms), retain 12>
                        +-o IONetworkStackUserClient  <class IONetworkStackUserClient, id 0x100000296, !registered, !matched, active, busy 0$

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (6662 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (311 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (310 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (307 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (253 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (250 ms), ret$
              +-o pci1af4,1043@5  <class IOPCIDevice, id 0x100000172, registered, matched, active, busy 0 (14 ms), retain 9>
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000186, registered, matched, active, busy 0 (10 ms), re$
                  | {
                  |   "IOPCIPrimaryMatch" = "0x00001af4&0x0000FFFF"
                  |   "CFBundleIdentifier" = "com.apple.driver.AppleVirtIO"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "IOClass" = "AppleVirtIOPCITransport"
                  |   "IOPersonalityPublisher" = "com.apple.driver.AppleVirtIO"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.AppleVirtIO"
                  |   "IOProbeScore" = 10000
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "built-in" = <00>
                  | }
                  | 
                  +-o AppleVirtIOConsole  <class AppleVirtIOConsole, id 0x10000018f, registered, matched, active, busy 0 (8 ms), retain 8>
                    +-o IOSerialStreamSync  <class IOSerialStreamSync, id 0x100000191, registered, matched, active, busy 0 (8 ms), retain 7>
                      +-o IOSerialBSDClient  <class IOSerialBSDClient, id 0x10000019b, registered, matched, active, busy 0 (0 ms), retain 5>

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (6662 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (311 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (310 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (307 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (253 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (250 ms), ret$
              +-o pci106b,1a00@6  <class IOPCIDevice, id 0x100000173, registered, matched, active, busy 0 (80 ms), retain 10>
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000187, registered, matched, active, busy 0 (76 ms), re$
                  | {
                  |   "IOPCIPrimaryMatch" = "0x1a00106b&0xFFFFFFFF"
                  |   "CFBundleIdentifier" = "com.apple.driver.AppleVirtIO"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "IOClass" = "AppleVirtIOPCITransport"
                  |   "IOPersonalityPublisher" = "com.apple.iokit.AppleVirtIOStorage"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.AppleVirtIO"
                  |   "IOProbeScore" = 10000
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "built-in" = <00>
                  | }
                  | 
                  +-o AppleVirtIOStorageDevice  <class AppleVirtIOStorageDevice, id 0x100000190, registered, matched, active, busy 0 (74 ms)$
                    +-o AppleVirtIODiskStorageDevice@1  <class AppleVirtIODiskStorageDevice, id 0x100000193, registered, matched, active, bu$
                      +-o IOBlockStorageDriver  <class IOBlockStorageDriver, id 0x100000194, registered, matched, active, busy 0 (74 ms), re$
                        +-o Apple Inc. VirtIO Block Media  <class IOMedia, id 0x100000195, registered, matched, active, busy 0 (74 ms), reta$
                          +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001a0, registered, matched, active, busy 0 (1 ms), retain $
                          +-o IOGUIDPartitionScheme  <class IOGUIDPartitionScheme, id 0x1000001b0, !registered, !matched, active, busy 0 (72$
                            +-o iBootSystemContainer@1  <class IOMedia, id 0x1000001bb, registered, matched, active, busy 0 (17 ms), retain $
                            | +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001be, registered, matched, active, busy 0 (0 ms), ret$
                            | +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001bf, !registered, !matched, active, $
                            |   +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001c0, registered, matched, active, busy 0 (16 ms), reta$
                            |     +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x1000001cf, registered, matched, active, $
                            |     +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x1000001ea, registered, matched, active, busy 0 (0 $
                            |       +-o iSCPreboot@1  <class AppleAPFSVolume, id 0x10000021d, registered, matched, active, busy 0 (0 ms), re$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000221, registered, matched, ac$
                            |       +-o xART@2  <class AppleAPFSVolume, id 0x10000021e, registered, matched, active, busy 0 (0 ms), retain 1$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000223, registered, matched, ac$
                            |       +-o Hardware@3  <class AppleAPFSVolume, id 0x10000021f, registered, matched, active, busy 0 (0 ms), reta$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000225, registered, matched, ac$
                            |       +-o Recovery@4  <class AppleAPFSVolume, id 0x100000220, registered, matched, active, busy 0 (0 ms), reta$
                            |         +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000227, registered, matched, ac$
                            +-o Container@2  <class IOMedia, id 0x1000001bc, registered, matched, active, busy 0 (72 ms), retain 12>
                            | +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001da, registered, matched, active, busy 0 (0 ms), ret$
                            | +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001db, !registered, !matched, active, $
                            |   +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001e2, registered, matched, active, busy 0 (70 ms), reta$
                            |     +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x1000001e3, registered, matched, active, $
                            |     +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x1000001f3, registered, matched, active, busy 0 (1 $
                            |       +-o Macintosh HD@1  <class AppleAPFSVolume, id 0x10000022f, registered, matched, active, busy 0 (1 ms), $
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000235, registered, matched, ac$
                            |       | +-o com.apple.os.update-7E5F621E717386C0F6DD5D927F23C2AFB35C42381A5EA61C993232428F0E6E55@1  <class App$
                            |       |   +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x100000243, registered, matched, active, busy 0 ($
                            |       +-o Preboot@2  <class AppleAPFSVolume, id 0x100000230, registered, matched, active, busy 0 (0 ms), retai$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000237, registered, matched, ac$
                            |       +-o Recovery@3  <class AppleAPFSVolume, id 0x100000231, registered, matched, active, busy 0 (0 ms), reta$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000239, registered, matched, ac$
                            |       +-o Update@4  <class AppleAPFSVolume, id 0x100000232, registered, matched, active, busy 0 (0 ms), retain$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000023b, registered, matched, ac$
                            |       +-o Data@5  <class AppleAPFSVolume, id 0x100000233, registered, matched, active, busy 0 (0 ms), retain 1$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000240, registered, matched, ac$
                            |       +-o VM@6  <class AppleAPFSVolume, id 0x100000234, registered, matched, active, busy 0 (0 ms), retain 11>
                            |         +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000023e, registered, matched, ac$
                            +-o RecoveryOSContainer@3  <class IOMedia, id 0x1000001bd, registered, matched, active, busy 0 (40 ms), retain 1$
                              +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001d8, registered, matched, active, busy 0 (0 ms), ret$
                              +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001d9, !registered, !matched, active, $
                                +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001dd, registered, matched, active, busy 0 (38 ms), reta$
                                  +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x1000001de, registered, matched, active, $
                                  +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x1000001f0, registered, matched, active, busy 0 (0 $
                                    +-o Recovery@1  <class AppleAPFSVolume, id 0x100000229, registered, matched, active, busy 0 (0 ms), reta$
                                    | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000022b, registered, matched, ac$
                                    +-o Update@2  <class AppleAPFSVolume, id 0x10000022a, registered, matched, active, busy 0 (0 ms), retain$
                                      +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000022d, registered, matched, ac$

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (6662 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (311 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (310 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (307 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (253 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (250 ms), ret$
              +-o pci106b,1a00@7  <class IOPCIDevice, id 0x100000174, registered, matched, active, busy 0 (7 ms), retain 10>
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000188, registered, matched, active, busy 0 (3 ms), ret$
                  | {
                  |   "IOPCIPrimaryMatch" = "0x1a00106b&0xFFFFFFFF"
                  |   "CFBundleIdentifier" = "com.apple.driver.AppleVirtIO"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "IOClass" = "AppleVirtIOPCITransport"
                  |   "IOPersonalityPublisher" = "com.apple.iokit.AppleVirtIOStorage"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.AppleVirtIO"
                  |   "IOProbeScore" = 10000
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "built-in" = <00>
                  | }
                  | 
                  +-o AppleVirtIOStorageDevice  <class AppleVirtIOStorageDevice, id 0x100000192, registered, matched, active, busy 0 (0 ms),$
                    +-o AppleVirtIOUtilStorageDevice  <class AppleVirtIOUtilStorageDevice, id 0x100000197, registered, matched, active, busy$
                      +-o AppleEmbeddedSimpleSPINORFlasherDriver  <class AppleEmbeddedSimpleSPINORFlasherDriver, id 0x100000198, registered,$
                      +-o AppleVirtIONVRAM  <class AppleVirtIONVRAM, id 0x100000199, registered, matched, active, busy 0 (0 ms), retain 5>
                      +-o AppleNVMeNamespaceDevice  <class AppleNVMeNamespaceDevice, id 0x1000001a4, registered, matched, active, busy 0 (0 $
                      +-o AppleNVMeEAN  <class AppleNVMeEAN, id 0x1000001a5, registered, matched, active, busy 0 (0 ms), retain 6>

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (6662 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (311 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (310 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (307 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (253 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (250 ms), ret$
              +-o pci1af4,1044@8  <class IOPCIDevice, id 0x100000175, registered, matched, active, busy 0 (6 ms), retain 9>
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x100000189, registered, matched, active, busy 0 (2 ms), ret$
                  | {
                  |   "IOPCIPrimaryMatch" = "0x00001af4&0x0000FFFF"
                  |   "CFBundleIdentifier" = "com.apple.driver.AppleVirtIO"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "IOClass" = "AppleVirtIOPCITransport"
                  |   "IOPersonalityPublisher" = "com.apple.driver.AppleVirtIO"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.AppleVirtIO"
                  |   "IOProbeScore" = 10000
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "built-in" = <00>
                  | }
                  | 
                  +-o AppleVirtIOEntropy  <class AppleVirtIOEntropy, id 0x100000196, registered, matched, active, busy 0 (0 ms), retain 6>

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000135, registered, matched, active, busy 0 (6662 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x100000139, registered, matched, active, busy 0 (311 ms), retain 21$
      +-o arm-io  <class IOPlatformDevice, id 0x100000115, registered, matched, active, busy 0 (310 ms), retain 15>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000161, !registered, !matched, active, busy 0 (307 ms), retain 12$
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (253 ms), retain 17>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000016e, registered, matched, active, busy 0 (250 ms), ret$
              +-o pci106b,1a01@A  <class IOPCIDevice, id 0x100000177, registered, matched, active, busy 0 (14 ms), retain 9>
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x10000018b, registered, matched, active, busy 0 (9 ms), ret$
                  | {
                  |   "IOPCIPrimaryMatch" = "0x1a01106b"
                  |   "CFBundleIdentifier" = "com.apple.driver.AppleVirtIO"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "IOClass" = "AppleVirtIOPCITransport"
                  |   "IOPersonalityPublisher" = "com.apple.driver.AppleVirtualPlatform"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.AppleVirtIO"
                  |   "IOProbeScore" = 10000
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "built-in" = <00>
                  | }
                  | 
                  +-o AppleVirtIOUSBConfiguration  <class AppleVirtIOUSBConfiguration, id 0x10000019a, registered, matched, active, busy 0 ($
                    +-o AppleVirtIOUSBDeviceController@0  <class AppleVirtIOUSBDeviceController, id 0x1000001a6, registered, matched, active$
                      +-o IOUSBDeviceConfigurator  <class IOUSBDeviceConfigurator, id 0x1000001a8, !registered, !matched, active, busy 0, re$
                      +-o AppleUSBNCMControl@0  <class IOUSBDeviceInterface, id 0x1000001a9, registered, matched, active, busy 0 (0 ms), ret$
                      | +-o AppleUSBDeviceNCMControl@0  <class AppleUSBDeviceNCMControl, id 0x1000001ae, registered, matched, active, busy 0$
                      +-o AppleUSBNCMData@1  <class IOUSBDeviceInterface, id 0x1000001aa, registered, matched, active, busy 0 (7 ms), retain$
                      | +-o AppleUSBDeviceNCMData  <class AppleUSBDeviceNCMData, id 0x1000001b1, registered, matched, active, busy 0 (0 ms),$
                      |   +-o en1  <class IOEthernetInterface, id 0x1000001f9, registered, matched, active, busy 0 (0 ms), retain 11>
                      |     +-o IONetworkStack  <class IONetworkStack, id 0x1000001cb, registered, matched, active, busy 0 (0 ms), retain 12$
                      |       +-o IONetworkStackUserClient  <class IONetworkStackUserClient, id 0x100000296, !registered, !matched, active, $
                      +-o AppleUSBNCMControlAux@2  <class IOUSBDeviceInterface, id 0x1000001ab, registered, matched, active, busy 0 (0 ms), $
                      | +-o AppleUSBDeviceNCMControl@2  <class AppleUSBDeviceNCMControl, id 0x1000001b3, registered, matched, active, busy 0$
                      +-o AppleUSBNCMDataAux@3  <class IOUSBDeviceInterface, id 0x1000001ac, registered, matched, active, busy 0 (8 ms), ret$
                        +-o AppleUSBDeviceNCMData  <class AppleUSBDeviceNCMData, id 0x1000001b5, registered, matched, active, busy 0 (0 ms),$
                          +-o anpi0  <class AppleUSBDeviceNCMPrivateEthernetInterface, id 0x1000001fb, registered, matched, active, busy 0 ($
                            +-o IONetworkStack  <class IONetworkStack, id 0x1000001cb, registered, matched, active, busy 0 (0 ms), retain 12$
                              +-o IONetworkStackUserClient  <class IONetworkStackUserClient, id 0x100000296, !registered, !matched, active, $

% ioreg -trc IOPCIDevice          
+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o arm-io  <class IOPlatformDevice, id 0x100000117, registered, matched, active, busy 0 (24452 ms), retain 16>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000166, !registered, !matched, active, busy 0 (24449 ms), retain $
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011d, registered, matched, active, busy 0 (24433 ms), retain 13>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000017b, registered, matched, active, busy 0 (24432 ms), r$
              +-o pci1b36,8@0  <class IOPCIDevice, id 0x10000017d, registered, matched, active, busy 0 (2 ms), retain 8>
                  {
                    "subsystem-vendor-id" = <f41a0000>
                    "compatible" = <"pci1af4,1100","pci1b36,8","pciclass,060000">
                    "IOName" = "pci1b36,8"
                    "reg" = <0000000000000000000000000000000000000000>
                    "device-id" = <08000000>
                    "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                    "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                    "IOPCIResourced" = Yes
                    "revision-id" = <00000000>
                    "vendor-id" = <361b0000>
                    "pcidebug" = "0:0:0"
                    "class-code" = <00000600>
                    "subsystem-id" = <00110000>
                    "name" = <"pci1b36,8">
                  }
                  

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o arm-io  <class IOPlatformDevice, id 0x100000117, registered, matched, active, busy 0 (24452 ms), retain 16>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000166, !registered, !matched, active, busy 0 (24449 ms), retain $
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011d, registered, matched, active, busy 0 (24433 ms), retain 13>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000017b, registered, matched, active, busy 0 (24432 ms), r$
              +-o pci1b36,d@1  <class IOPCIDevice, id 0x10000017e, registered, matched, active, busy 0 (24419 ms), retain 12>
                | {
                |   "assigned-addresses" = <1008008200c00050000000000040000000000000>
                |   "IOInterruptSpecifiers" = (<4100000001000000>)
                |   "class-code" = <30030c00>
                |   "IODeviceMemory" = (({"address"=1342226432,"length"=16384}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"ChildrenPowerState"=2,"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxP$
                |   "subsystem-vendor-id" = <f41a0000>
                |   "pci-aspm-default" = 1
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <361b0000>
                |   "name" = <"pci1b36,d">
                |   "device-id" = <0d000000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,1100","pci1b36,d","pciclass,0c0330">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "IOPCIExpressLinkCapabilities" = 1041
                |   "subsystem-id" = <00110000>
                |   "revision-id" = <01000000>
                |   "IOPCIExpressLinkStatus" = 17
                |   "IOPCIExpressCapabilities" = 146
                |   "pcidebug" = "0:1:0"
                |   "IOName" = "pci1b36,d"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <00080000000000000000000000000000000000001008000200000000000000000040000000000000>
                | }
                | 
                +-o AppleUSBXHCIPCI@01000000  <class AppleUSBXHCIPCI, id 0x10000018a, registered, matched, active, busy 0 (24406 ms), retain$
                  +-o AppleUSB20XHCIPort@01500000  <class AppleUSB20XHCIPort, id 0x1000001bc, registered, matched, active, busy 0 (20485 ms)$
                  | +-o QEMU USB Keyboard@01500000  <class IOUSBHostDevice, id 0x100000280, registered, matched, active, busy 0 (20485 ms), $
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x1000002de, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x1000002e8, registered, matched, active, busy 0 (14355 ms), r$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x1000002e9, registered, matched, active, busy 0 (10240$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002ef, registered, matched, active, busy 0 (10240 ms), retain $
                  |         +-o AppleUserHIDEventDriver  <class AppleUserHIDEventService, id 0x1000002f0, registered, matched, active, busy $
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002f5, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@01600000  <class AppleUSB20XHCIPort, id 0x1000001bd, registered, matched, active, busy 0 (3920 ms),$
                  | +-o QEMU USB Tablet@01600000  <class IOUSBHostDevice, id 0x10000024c, registered, matched, active, busy 0 (3920 ms), ret$
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x10000027f, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x10000028a, registered, matched, active, busy 0 (2050 ms), re$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x10000028b, registered, matched, active, busy 0 (2 ms)$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002a2, registered, matched, active, busy 0 (2 ms), retain 10>
                  |         +-o AppleUserHIDEventDriver  <class AppleUserHIDEventService, id 0x1000002a3, registered, matched, active, busy $
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002a7, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@01700000  <class AppleUSB20XHCIPort, id 0x1000001be, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@01800000  <class AppleUSB20XHCIPort, id 0x1000001bf, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01100000  <class AppleUSB30XHCIPort, id 0x1000001c0, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01200000  <class AppleUSB30XHCIPort, id 0x1000001c1, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01300000  <class AppleUSB30XHCIPort, id 0x1000001c2, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01400000  <class AppleUSB30XHCIPort, id 0x1000001c3, registered, matched, active, busy 0 (0 ms), re$

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o arm-io  <class IOPlatformDevice, id 0x100000117, registered, matched, active, busy 0 (24452 ms), retain 16>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000166, !registered, !matched, active, busy 0 (24449 ms), retain $
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011d, registered, matched, active, busy 0 (24433 ms), retain 13>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000017b, registered, matched, active, busy 0 (24432 ms), r$
              +-o scsi@2  <class IOPCIDevice, id 0x10000017f, registered, matched, active, busy 0 (13 ms), retain 10>
                | {
                |   "assigned-addresses" = <1410008200200150000000000010000000000000201000c200800050000000000040000000000000>
                |   "IOInterruptSpecifiers" = (<4200000001000000>)
                |   "class-code" = <00000100>
                |   "IODeviceMemory" = (({"address"=1342251008,"length"=4096}),({"address"=1342210048,"length"=16384}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <6b100000>
                |   "name" = <"scsi">
                |   "device-id" = <001a0000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,1100","pci106b,1a00","pciclass,010000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <00110000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:2:0"
                |   "IOName" = "scsi"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <001000000000000000000000000000000000000014100002000000000000000000100000000000002010004200000000000000000040000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x10000018b, registered, matched, active, busy 0 (9 ms), ret$
                  +-o AppleVirtIOStorageDevice  <class AppleVirtIOStorageDevice, id 0x100000191, registered, matched, active, busy 0 (0 ms),$
                    +-o AppleVirtIOUtilStorageDevice  <class AppleVirtIOUtilStorageDevice, id 0x100000192, registered, matched, active, busy$
                      +-o AppleEmbeddedSimpleSPINORFlasherDriver  <class AppleEmbeddedSimpleSPINORFlasherDriver, id 0x100000193, registered,$
                      +-o AppleVirtIONVRAM  <class AppleVirtIONVRAM, id 0x100000194, registered, matched, active, busy 0 (0 ms), retain 5>
                      +-o AppleNVMeNamespaceDevice  <class AppleNVMeNamespaceDevice, id 0x1000001b2, registered, matched, active, busy 0 (0 $
                      +-o AppleNVMeEAN  <class AppleNVMeEAN, id 0x1000001b3, registered, matched, active, busy 0 (0 ms), retain 6>

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o arm-io  <class IOPlatformDevice, id 0x100000117, registered, matched, active, busy 0 (24452 ms), retain 16>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000166, !registered, !matched, active, busy 0 (24449 ms), retain $
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011d, registered, matched, active, busy 0 (24433 ms), retain 13>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000017b, registered, matched, active, busy 0 (24432 ms), r$
              +-o scsi@3  <class IOPCIDevice, id 0x100000180, registered, matched, active, busy 0 (26 ms), retain 10>
                | {
                |   "assigned-addresses" = <1418008200100150000000000010000000000000201800c200400050000000000040000000000000>
                |   "IOInterruptSpecifiers" = (<4300000001000000>)
                |   "class-code" = <00000100>
                |   "IODeviceMemory" = (({"address"=1342246912,"length"=4096}),({"address"=1342193664,"length"=16384}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <6b100000>
                |   "name" = <"scsi">
                |   "device-id" = <001a0000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,1100","pci106b,1a00","pciclass,010000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <00110000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:3:0"
                |   "IOName" = "scsi"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <001800000000000000000000000000000000000014180002000000000000000000100000000000002018004200000000000000000040000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x10000018c, registered, matched, active, busy 0 (22 ms), re$
                  +-o AppleVirtIOStorageDevice  <class AppleVirtIOStorageDevice, id 0x100000195, registered, matched, active, busy 0 (21 ms)$
                    +-o AppleVirtIODiskStorageDevice@1  <class AppleVirtIODiskStorageDevice, id 0x100000196, registered, matched, active, bu$
                      +-o IOBlockStorageDriver  <class IOBlockStorageDriver, id 0x100000197, registered, matched, active, busy 0 (21 ms), re$
                        +-o Apple Inc. VirtIO Block Media  <class IOMedia, id 0x100000198, registered, matched, active, busy 0 (21 ms), reta$
                          +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x10000019a, registered, matched, active, busy 0 (0 ms), retain $
                          +-o IOGUIDPartitionScheme  <class IOGUIDPartitionScheme, id 0x1000001cc, !registered, !matched, active, busy 0 (11$
                            +-o iBootSystemContainer@1  <class IOMedia, id 0x1000001d0, registered, matched, active, busy 0 (4 ms), retain 1$
                            | +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001d3, registered, matched, active, busy 0 (0 ms), ret$
                            | +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001d4, !registered, !matched, active, $
                            |   +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001d9, registered, matched, active, busy 0 (4 ms), retai$
                            |     +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x1000001da, registered, matched, active, $
                            |     +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x1000001ef, registered, matched, active, busy 0 (0 $
                            |       +-o iSCPreboot@1  <class AppleAPFSVolume, id 0x1000001f4, registered, matched, active, busy 0 (0 ms), re$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x1000001f8, registered, matched, ac$
                            |       +-o xART@2  <class AppleAPFSVolume, id 0x1000001f5, registered, matched, active, busy 0 (0 ms), retain 1$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x1000001fa, registered, matched, ac$
                            |       +-o Hardware@3  <class AppleAPFSVolume, id 0x1000001f6, registered, matched, active, busy 0 (0 ms), reta$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x1000001fc, registered, matched, ac$
                            |       +-o Recovery@4  <class AppleAPFSVolume, id 0x1000001f7, registered, matched, active, busy 0 (0 ms), reta$
                            |         +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x1000001fe, registered, matched, ac$
                            +-o Container@2  <class IOMedia, id 0x1000001d1, registered, matched, active, busy 0 (11 ms), retain 12>
                            | +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001d5, registered, matched, active, busy 0 (0 ms), ret$
                            | +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001d6, !registered, !matched, active, $
                            |   +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001dd, registered, matched, active, busy 0 (11 ms), reta$
                            |     +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x1000001de, registered, matched, active, $
                            |     +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x1000001f3, registered, matched, active, busy 0 (1 $
                            |       +-o Macintosh HD@1  <class AppleAPFSVolume, id 0x100000206, registered, matched, active, busy 0 (1 ms), $
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000020c, registered, matched, ac$
                            |       | +-o com.apple.os.update-7E5F621E717386C0F6DD5D927F23C2AFB35C42381A5EA61C993232428F0E6E55@1  <class App$
                            |       |   +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x10000021a, registered, matched, active, busy 0 ($
                            |       +-o Preboot@2  <class AppleAPFSVolume, id 0x100000207, registered, matched, active, busy 0 (0 ms), retai$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x10000020f, registered, matched, ac$
                            |       +-o Recovery@3  <class AppleAPFSVolume, id 0x100000208, registered, matched, active, busy 0 (0 ms), reta$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000211, registered, matched, ac$
                            |       +-o Update@4  <class AppleAPFSVolume, id 0x100000209, registered, matched, active, busy 0 (0 ms), retain$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000213, registered, matched, ac$
                            |       +-o Data@5  <class AppleAPFSVolume, id 0x10000020a, registered, matched, active, busy 0 (0 ms), retain 1$
                            |       | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000215, registered, matched, ac$
                            |       +-o VM@6  <class AppleAPFSVolume, id 0x10000020b, registered, matched, active, busy 0 (0 ms), retain 11>
                            |         +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000217, registered, matched, ac$
                            +-o RecoveryOSContainer@3  <class IOMedia, id 0x1000001d2, registered, matched, active, busy 0 (5 ms), retain 11$
                              +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x1000001d7, registered, matched, active, busy 0 (0 ms), ret$
                              +-o AppleAPFSContainerScheme  <class AppleAPFSContainerScheme, id 0x1000001d8, !registered, !matched, active, $
                                +-o AppleAPFSMedia  <class AppleAPFSMedia, id 0x1000001e1, registered, matched, active, busy 0 (5 ms), retai$
                                  +-o AppleAPFSMediaBSDClient  <class AppleAPFSMediaBSDClient, id 0x1000001e2, registered, matched, active, $
                                  +-o AppleAPFSContainer  <class AppleAPFSContainer, id 0x1000001f1, registered, matched, active, busy 0 (0 $
                                    +-o Recovery@1  <class AppleAPFSVolume, id 0x100000200, registered, matched, active, busy 0 (0 ms), reta$
                                    | +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000202, registered, matched, ac$
                                    +-o Update@2  <class AppleAPFSVolume, id 0x100000201, registered, matched, active, busy 0 (0 ms), retain$
                                      +-o AppleAPFSVolumeBSDClient  <class AppleAPFSVolumeBSDClient, id 0x100000204, registered, matched, ac$

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o arm-io  <class IOPlatformDevice, id 0x100000117, registered, matched, active, busy 0 (24452 ms), retain 16>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000166, !registered, !matched, active, busy 0 (24449 ms), retain $
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011d, registered, matched, active, busy 0 (24433 ms), retain 13>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000017b, registered, matched, active, busy 0 (24432 ms), r$
              +-o ethernet@4  <class IOPCIDevice, id 0x100000181, registered, matched, active, busy 0 (12 ms), retain 9>
                | {
                |   "assigned-addresses" = <1420008200000150000000000010000000000000202000c200000050000000000040000000000000>
                |   "IOInterruptSpecifiers" = (<4400000001000000>)
                |   "class-code" = <00000200>
                |   "IODeviceMemory" = (({"address"=1342242816,"length"=4096}),({"address"=1342177280,"length"=16384}))
                |   "IOReportLegendPublic" = Yes
                |   "IOPowerManagement" = {"CurrentPowerState"=2,"CapabilityFlags"=258,"ChildProxyPowerState"=2,"MaxPowerState"=3}
                |   "subsystem-vendor-id" = <f41a0000>
                |   "IOInterruptControllers" = ("IOInterruptController00000037")
                |   "vendor-id" = <f41a0000>
                |   "name" = <"ethernet">
                |   "device-id" = <41100000>
                |   "IOPCIResourced" = Yes
                |   "compatible" = <"pci1af4,1100","pci1af4,1041","pciclass,020000">
                |   "IOServiceDEXTEntitlements" = (("com.apple.developer.driverkit.transport.pci"))
                |   "subsystem-id" = <00110000>
                |   "revision-id" = <01000000>
                |   "pcidebug" = "0:4:0"
                |   "IOName" = "ethernet"
                |   "IOReportLegend" = ({"IOReportGroupName"="Interrupt Statistics (by index)","IOReportChannels"=((5291294645182005248,4295$
                |   "reg" = <002000000000000000000000000000000000000014200002000000000000000000100000000000002020004200000000000000000040000$
                | }
                | 
                +-o AppleVirtIOPCITransport  <class AppleVirtIOPCITransport, id 0x10000018d, registered, matched, active, busy 0 (8 ms), ret$
                  +-o AppleVirtIONetwork  <class AppleVirtIONetwork, id 0x100000199, registered, matched, active, busy 0 (0 ms), retain 10>
                    +-o en0  <class IOEthernetInterface, id 0x1000001b6, registered, matched, active, busy 0 (0 ms), retain 13>
                      +-o IONetworkStack  <class IONetworkStack, id 0x1000001a7, registered, matched, active, busy 0 (0 ms), retain 8>
                        +-o IONetworkStackUserClient  <class IONetworkStackUserClient, id 0x100000265, !registered, !matched, active, busy 0$


% ioreg -trc IOInterruptController
+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o arm-io  <class IOPlatformDevice, id 0x100000117, registered, matched, active, busy 0 (24452 ms), retain 16>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000166, !registered, !matched, active, busy 0 (24449 ms), retain $
          +-o gic@0  <class AppleARMIODevice, id 0x10000011a, registered, matched, active, busy 0 (4 ms), retain 9>
            +-o AppleARMGICv3  <class AppleARMGICv3, id 0x100000177, registered, matched, active, busy 0 (4 ms), retain 6>
                {
                  "IOClass" = "AppleARMGICv3"
                  "CFBundleIdentifier" = "com.apple.driver.AppleARMGIC"
                  "IOProviderClass" = "AppleARMIODevice"
                  "SystemPowerProfileOverrideDict" = {"System Sleep Timer"=2147483647}
                  "IOProbeScore" = 1000
                  "IONameMatch" = "ARM,gicv3"
                  "IOMatchedAtBoot" = Yes
                  "IOMatchCategory" = "IODefaultMatchCategory"
                  "IONameMatched" = "ARM,gicv3"
                  "IOPersonalityPublisher" = "com.apple.driver.AppleARMGIC"
                  "InterruptControllerName" = "IOInterruptController00000037"
                  "CFBundleIdentifierKernel" = "com.apple.driver.AppleARMGIC"
                }
                

+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o PassthruInterruptController  <class PassthruInterruptController, id 0x10000016f, registered, matched, active, busy 0 (0 ms), retai$
          {
            "IOPlatformInterruptController" = Yes
          }


% ioreg -trc AppleUSBXHCI         
+-o Root  <class IORegistryEntry, id 0x100000100, retain 7>
  +-o VMA2MACOSAP  <class IOPlatformExpertDevice, id 0x100000138, registered, matched, active, busy 0 (30766 ms), retain 26>
    +-o AppleVirtualPlatformARMPE  <class AppleVirtualPlatformARMPE, id 0x10000013c, registered, matched, active, busy 0 (24454 ms), retain $
      +-o arm-io  <class IOPlatformDevice, id 0x100000117, registered, matched, active, busy 0 (24453 ms), retain 16>
        +-o AppleVirtualPlatformIO  <class AppleVirtualPlatformIO, id 0x100000166, !registered, !matched, active, busy 0 (24450 ms), retain $
          +-o pcie@30000000  <class AppleARMIODevice, id 0x10000011d, registered, matched, active, busy 0 (24433 ms), retain 13>
            +-o AppleVirtualPlatformPCIE  <class AppleVirtualPlatformPCIE, id 0x10000017b, registered, matched, active, busy 0 (24432 ms), r$
              +-o pci1b36,d@1  <class IOPCIDevice, id 0x10000017e, registered, matched, active, busy 0 (24419 ms), retain 12>
                +-o AppleUSBXHCIPCI@01000000  <class AppleUSBXHCIPCI, id 0x10000018a, registered, matched, active, busy 0 (24406 ms), retain$
                  | {
                  |   "IOClass" = "AppleUSBXHCIPCI"
                  |   "CFBundleIdentifier" = "com.apple.driver.usb.AppleUSBXHCIPCI"
                  |   "IOProviderClass" = "IOPCIDevice"
                  |   "kUSBSleepSupported" = No
                  |   "IOPCIClassMatch" = "0x0c033000"
                  |   "IOPCITunnelCompatible" = Yes
                  |   "IOPowerManagement" = {"ChildrenPowerState"=3,"DevicePowerState"=0,"CurrentPowerState"=3,"CapabilityFlags"=32768,"MaxP$
                  |   "IOProbeScore" = 0
                  |   "IOMatchedAtBoot" = Yes
                  |   "IOPCIPauseCompatible" = Yes
                  |   "IOMatchCategory" = "IODefaultMatchCategory"
                  |   "locationID" = 16777216
                  |   "Revision" = <0003>
                  |   "controller-statistics" = {"kControllerStatIOCount"=2030,"kControllerStatPowerStateTime"={"kPowerStateOff"="0ms (0%)",$
                  |   "IOPersonalityPublisher" = "com.apple.driver.usb.AppleUSBXHCIPCI"
                  |   "CFBundleIdentifierKernel" = "com.apple.driver.usb.AppleUSBXHCIPCI"
                  |   "IOPCIUseDeviceMapper" = Yes
                  | }
                  | 
                  +-o AppleUSB20XHCIPort@01500000  <class AppleUSB20XHCIPort, id 0x1000001bc, registered, matched, active, busy 0 (20485 ms)$
                  | +-o QEMU USB Keyboard@01500000  <class IOUSBHostDevice, id 0x100000280, registered, matched, active, busy 0 (20485 ms), $
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x1000002de, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x1000002e8, registered, matched, active, busy 0 (14355 ms), r$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x1000002e9, registered, matched, active, busy 0 (10240$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002ef, registered, matched, active, busy 0 (10240 ms), retain $
                  |         +-o AppleUserHIDEventDriver  <class AppleUserHIDEventService, id 0x1000002f0, registered, matched, active, busy $
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002f5, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@01600000  <class AppleUSB20XHCIPort, id 0x1000001bd, registered, matched, active, busy 0 (3920 ms),$
                  | +-o QEMU USB Tablet@01600000  <class IOUSBHostDevice, id 0x10000024c, registered, matched, active, busy 0 (3920 ms), ret$
                  |   +-o AppleUSBHostCompositeDevice  <class AppleUSBHostCompositeDevice, id 0x10000027f, !registered, !matched, active, bu$
                  |   +-o IOUSBHostInterface@0  <class IOUSBHostInterface, id 0x10000028a, registered, matched, active, busy 0 (2050 ms), re$
                  |     +-o AppleUserUSBHostHIDDevice  <class AppleUserHIDDevice, id 0x10000028b, registered, matched, active, busy 0 (2 ms)$
                  |       +-o IOHIDInterface  <class IOHIDInterface, id 0x1000002a2, registered, matched, active, busy 0 (2 ms), retain 10>
                  |         +-o AppleUserHIDEventDriver  <class AppleUserHIDEventService, id 0x1000002a3, registered, matched, active, busy $
                  |           +-o IOHIDEventServiceUserClient  <class IOHIDEventServiceUserClient, id 0x1000002a7, !registered, !matched, ac$
                  +-o AppleUSB20XHCIPort@01700000  <class AppleUSB20XHCIPort, id 0x1000001be, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB20XHCIPort@01800000  <class AppleUSB20XHCIPort, id 0x1000001bf, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01100000  <class AppleUSB30XHCIPort, id 0x1000001c0, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01200000  <class AppleUSB30XHCIPort, id 0x1000001c1, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01300000  <class AppleUSB30XHCIPort, id 0x1000001c2, registered, matched, active, busy 0 (0 ms), re$
                  +-o AppleUSB30XHCIPort@01400000  <class AppleUSB30XHCIPort, id 0x1000001c3, registered, matched, active, busy 0 (0 ms), re$

Re: [PATCH v8 15/15] hw/vmapple/vmapple: Add vmapple machine type
Posted by Phil Dennis-Jordan 1 week, 6 days ago
On Sun, 10 Nov 2024 at 08:38, Akihiko Odaki <akihiko.odaki@daynix.com> wrote:
>
> On 2024/11/08 23:47, Phil Dennis-Jordan wrote:
> > From: Alexander Graf <graf@amazon.com>
> >
> > Apple defines a new "vmapple" machine type as part of its proprietary
> > macOS Virtualization.Framework vmm. This machine type is similar to the
> > virt one, but with subtle differences in base devices, a few special
> > vmapple device additions and a vastly different boot chain.
> >
> > This patch reimplements this machine type in QEMU. To use it, you
> > have to have a readily installed version of macOS for VMApple,
> > run on macOS with -accel hvf, pass the Virtualization.Framework
> > boot rom (AVPBooter) in via -bios, pass the aux and root volume as pflash
> > and pass aux and root volume as virtio drives. In addition, you also
> > need to find the machine UUID and pass that as -M vmapple,uuid= parameter:
> >
> > $ qemu-system-aarch64 -accel hvf -M vmapple,uuid=0x1234 -m 4G \
> >      -bios /System/Library/Frameworks/Virtualization.framework/Versions/A/Resources/AVPBooter.vmapple2.bin
> >      -drive file=aux,if=pflash,format=raw \
> >      -drive file=root,if=pflash,format=raw \
> >      -drive file=aux,if=none,id=aux,format=raw \
> >      -device vmapple-virtio-aux,drive=aux \
> >      -drive file=root,if=none,id=root,format=raw \
> >      -device vmapple-virtio-root,drive=root
> >
> > With all these in place, you should be able to see macOS booting
> > successfully.
> >
> > Known issues:
> >   - Keyboard and mouse/tablet input is laggy. The reason for this is
> >     either that macOS's XHCI driver is broken when the device/platform
> >     does not support MSI/MSI-X, or there's some unfortunate interplay
> >     with Qemu's XHCI implementation in this scenario.
> >   - Currently only macOS 12 guests are supported. The boot process for
> >     13+ will need further investigation and adjustment.
> >
> > Signed-off-by: Alexander Graf <graf@amazon.com>
> > Co-authored-by: Phil Dennis-Jordan <phil@philjordan.eu>
> > Signed-off-by: Phil Dennis-Jordan <phil@philjordan.eu>
> > ---
> >
> > v3:
> >   * Rebased on latest upstream, updated affinity and NIC creation
> >     API usage
> >   * Included Apple-variant virtio-blk in build dependency
> >   * Updated API usage for setting 'redist-region-count' array-typed property
> >     on GIC.
> >   * Switched from virtio HID devices (for which macOS 12 does not contain
> >     drivers) to an XHCI USB controller and USB HID devices.
> >
> > v4:
> >   * Fixups for v4 changes to the other patches in the set.
> >   * Corrected the assert macro to use
> >   * Removed superfluous endian conversions corresponding to cfg's.
> >   * Init error handling improvement.
> >   * No need to select CPU type on TCG, as only HVF is supported.
> >   * Machine type version bumped to 9.2
> >   * #include order improved
> >
> > v5:
> >   * Fixed memory reservation for ecam alias region.
> >   * Better error handling setting properties on devices.
> >   * Simplified the machine ECID/UUID extraction script and actually created a
> >     file for it rather than quoting its code in documentation.
> >
> > v7:
> >   * Tiny error handling fix, un-inlined function.
> >
> > v8:
> >   * Use object_property_add_uint64_ptr rather than defining custom UUID
> >     property get/set functions.
> >
> >   MAINTAINERS                 |   1 +
> >   contrib/vmapple/uuid.sh     |   9 +
> >   docs/system/arm/vmapple.rst |  60 ++++
> >   docs/system/target-arm.rst  |   1 +
> >   hw/vmapple/Kconfig          |  20 ++
> >   hw/vmapple/meson.build      |   1 +
> >   hw/vmapple/vmapple.c        | 638 ++++++++++++++++++++++++++++++++++++
> >   7 files changed, 730 insertions(+)
> >   create mode 100755 contrib/vmapple/uuid.sh
> >   create mode 100644 docs/system/arm/vmapple.rst
> >   create mode 100644 hw/vmapple/vmapple.c
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 9d0d26cb65f..9591fd44a34 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -2767,6 +2767,7 @@ R: Phil Dennis-Jordan <phil@philjordan.eu>
> >   S: Maintained
> >   F: hw/vmapple/*
> >   F: include/hw/vmapple/*
> > +F: docs/system/arm/vmapple.rst
> >
> >   Subsystems
> >   ----------
> > diff --git a/contrib/vmapple/uuid.sh b/contrib/vmapple/uuid.sh
> > new file mode 100755
> > index 00000000000..956e8c3afed
> > --- /dev/null
> > +++ b/contrib/vmapple/uuid.sh
> > @@ -0,0 +1,9 @@
> > +#!/bin/sh
> > +# Used for converting a guest provisioned using Virtualization.framework
> > +# for use with the QEMU 'vmapple' aarch64 machine type.
> > +#
> > +# Extracts the Machine UUID from Virtualization.framework VM JSON file.
> > +# (as produced by 'macosvm', passed as command line argument)
> > +
> > +plutil -extract machineId raw "$1" | base64 -d | plutil -extract ECID raw -
> > +
> > diff --git a/docs/system/arm/vmapple.rst b/docs/system/arm/vmapple.rst
> > new file mode 100644
> > index 00000000000..6a634fa4572
> > --- /dev/null
> > +++ b/docs/system/arm/vmapple.rst
> > @@ -0,0 +1,60 @@
> > +VMApple machine emulation
> > +========================================================================================
> > +
> > +VMApple is the device model that the macOS built-in hypervisor called "Virtualization.framework"
> > +exposes to Apple Silicon macOS guests. The "vmapple" machine model in QEMU implements the same
> > +device model, but does not use any code from Virtualization.Framework.
> > +
> > +Prerequisites
> > +-------------
> > +
> > +To run the vmapple machine model, you need to
> > +
> > + * Run on Apple Silicon
> > + * Run on macOS 12.0 or above
> > + * Have an already installed copy of a Virtualization.Framework macOS 12 virtual machine. I will
> > +   assume that you installed it using the macosvm CLI.
>
> Add a URL to macosvm.
>
> > +
> > +First, we need to extract the UUID from the virtual machine that you installed. You can do this
> > +by running the shell script in contrib/vmapple/uuid.sh on the macosvm.json file.
> > +
> > +.. code-block:: bash
> > +  :caption: uuid.sh script to extract the UUID from a macosvm.json file
> > +
> > +  $ contrib/vmapple/uuid.sh "path/to/macosvm.json"
> > +
> > +Now we also need to trim the aux partition. It contains metadata that we can just discard:
> > +
> > +.. code-block:: bash
> > +  :caption: Command to trim the aux file
> > +
> > +  $ dd if="aux.img" of="aux.img.trimmed" bs=$(( 0x4000 )) skip=1
>
> Quoting is inconsistent. aux.img.trimmed is not quoted below but it is
> quoted here.

Quoting is fairly unusual in shell variable assignments I guess?

> > +
> > +How to run
> > +----------
> > +
> > +Then, we can launch QEMU with the Virtualization.Framework pre-boot environment and the readily
> > +installed target disk images. I recommend to port forward the VM's ssh and vnc ports to the host
> > +to get better interactive access into the target system:
> > +
> > +.. code-block:: bash
> > +  :caption: Example execution command line
> > +
> > +  $ UUID=$(uuid.sh macosvm.json)
> > +  $ AVPBOOTER=/System/Library/Frameworks/Virtualization.framework/Resources/AVPBooter.vmapple2.bin
> > +  $ AUX=aux.img.trimmed
> > +  $ DISK=disk.img
> > +  $ qemu-system-aarch64 \
> > +       -serial mon:stdio \
> > +       -m 4G \
> > +       -accel hvf \
> > +       -M vmapple,uuid=$UUID \
> > +       -bios $AVPBOOTER \
>
> $AUX and $DISK are quoted but $UUID and $AVPBOOTER are not.
>
> > +        -drive file="$AUX",if=pflash,format=raw \
> > +        -drive file="$DISK",if=pflash,format=raw \
>
> These two lines are misaligned.
>
> > +       -drive file="$AUX",if=none,id=aux,format=raw \
> > +       -drive file="$DISK",if=none,id=root,format=raw \
> > +       -device vmapple-virtio-aux,drive=aux \
> > +       -device vmapple-virtio-root,drive=root \
> > +       -net user,ipv6=off,hostfwd=tcp::2222-:22,hostfwd=tcp::5901-:5900 \
> > +       -net nic,model=virtio-net-pci \
>
> -net is a legacy option and creates a unnecessary indirection with hub.
> Use -netdev and -device options. Also plug virtio-net-pci into a PCIe
> root port as suggested in: docs/pcie.txt
>
> I thought it is about time to give a try on my M2 MacBook Air, but this
> command line did not work. The serial output said iBootStage2 finished,
> but it went silent after that and the display gives not output. macosvm
> can run the VM image. I used: UniversalMac_15.1_24B83_Restore.ipsw
>
> Do you have an idea what's wrong with my setup? Doesn't it work with 15.1?

Right now it only works with macOS 12.x as guest. I guess this needs
to be called out to be more explicit than just "Have an already
installed copy of a Virtualization.Framework macOS 12 virtual
machine."

> > diff --git a/docs/system/target-arm.rst b/docs/system/target-arm.rst
> > index 9aaa9c414c9..3426f79100b 100644
> > --- a/docs/system/target-arm.rst
> > +++ b/docs/system/target-arm.rst
> > @@ -102,6 +102,7 @@ Board-specific documentation
> >      arm/stellaris
> >      arm/stm32
> >      arm/virt
> > +   arm/vmapple
> >      arm/xenpvh
> >      arm/xlnx-versal-virt
> >      arm/xlnx-zynq
> > diff --git a/hw/vmapple/Kconfig b/hw/vmapple/Kconfig
> > index bcd1be63e3c..6a4c4a7fa2e 100644
> > --- a/hw/vmapple/Kconfig
> > +++ b/hw/vmapple/Kconfig
> > @@ -10,3 +10,23 @@ config VMAPPLE_CFG
> >   config VMAPPLE_VIRTIO_BLK
> >       bool
> >
> > +config VMAPPLE
> > +    bool
> > +    depends on ARM
> > +    depends on HVF
> > +    default y if ARM
> > +    imply PCI_DEVICES
> > +    select ARM_GIC
> > +    select PLATFORM_BUS
> > +    select PCI_EXPRESS
> > +    select PCI_EXPRESS_GENERIC_BRIDGE
> > +    select PL011 # UART
> > +    select PL031 # RTC
> > +    select PL061 # GPIO
> > +    select GPIO_PWR
> > +    select PVPANIC_MMIO
> > +    select VMAPPLE_AES
> > +    select VMAPPLE_BDIF
> > +    select VMAPPLE_CFG
> > +    select MAC_PVG_MMIO
> > +    select VMAPPLE_VIRTIO_BLK
> > diff --git a/hw/vmapple/meson.build b/hw/vmapple/meson.build
> > index bf17cf906c9..e572f7d5602 100644
> > --- a/hw/vmapple/meson.build
> > +++ b/hw/vmapple/meson.build
> > @@ -2,3 +2,4 @@ system_ss.add(when: 'CONFIG_VMAPPLE_AES',  if_true: files('aes.c'))
> >   system_ss.add(when: 'CONFIG_VMAPPLE_BDIF', if_true: files('bdif.c'))
> >   system_ss.add(when: 'CONFIG_VMAPPLE_CFG',  if_true: files('cfg.c'))
> >   system_ss.add(when: 'CONFIG_VMAPPLE_VIRTIO_BLK',  if_true: files('virtio-blk.c'))
> > +specific_ss.add(when: 'CONFIG_VMAPPLE',     if_true: files('vmapple.c'))
> > diff --git a/hw/vmapple/vmapple.c b/hw/vmapple/vmapple.c
> > new file mode 100644
> > index 00000000000..8b8710c70a2
> > --- /dev/null
> > +++ b/hw/vmapple/vmapple.c
> > @@ -0,0 +1,638 @@
> > +/*
> > + * VMApple machine emulation
> > + *
> > + * Copyright © 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> > + * See the COPYING file in the top-level directory.
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later
> > + *
> > + * VMApple is the device model that the macOS built-in hypervisor called
> > + * "Virtualization.framework" exposes to Apple Silicon macOS guests. The
> > + * machine model in this file implements the same device model in QEMU, but
> > + * does not use any code from Virtualization.Framework.
> > + */
> > +
> > +#include "qemu/osdep.h"
> > +#include "qemu/bitops.h"
> > +#include "qemu/datadir.h"
> > +#include "qemu/error-report.h"
> > +#include "qemu/guest-random.h"
> > +#include "qemu/help-texts.h"
> > +#include "qemu/log.h"
> > +#include "qemu/module.h"
> > +#include "qemu/option.h"
> > +#include "qemu/units.h"
> > +#include "monitor/qdev.h"
> > +#include "hw/boards.h"
> > +#include "hw/irq.h"
> > +#include "hw/loader.h"
> > +#include "hw/qdev-properties.h"
> > +#include "hw/sysbus.h"
> > +#include "hw/usb.h"
> > +#include "hw/arm/boot.h"
> > +#include "hw/arm/primecell.h"
> > +#include "hw/char/pl011.h"
> > +#include "hw/intc/arm_gic.h"
> > +#include "hw/intc/arm_gicv3_common.h"
> > +#include "hw/misc/pvpanic.h"
> > +#include "hw/pci-host/gpex.h"
> > +#include "hw/usb/xhci.h"
> > +#include "hw/virtio/virtio-pci.h"
> > +#include "hw/vmapple/vmapple.h"
> > +#include "net/net.h"
> > +#include "qapi/error.h"
> > +#include "qapi/qmp/qlist.h"
> > +#include "qapi/visitor.h"
> > +#include "qapi/qapi-visit-common.h"
> > +#include "standard-headers/linux/input.h"
> > +#include "sysemu/hvf.h"
> > +#include "sysemu/kvm.h"
> > +#include "sysemu/reset.h"
> > +#include "sysemu/runstate.h"
> > +#include "sysemu/sysemu.h"
> > +#include "target/arm/internals.h"
> > +#include "target/arm/kvm_arm.h"
> > +
> > +struct VMAppleMachineClass {
> > +    MachineClass parent;
> > +};
> > +
> > +struct VMAppleMachineState {
> > +    MachineState parent;
> > +
> > +    Notifier machine_done;
> > +    struct arm_boot_info bootinfo;
> > +    MemMapEntry *memmap;
> > +    const int *irqmap;
> > +    DeviceState *gic;
> > +    DeviceState *cfg;
> > +    Notifier powerdown_notifier;
> > +    PCIBus *bus;
> > +    MemoryRegion fw_mr;
> > +    MemoryRegion ecam_alias;
> > +    uint64_t uuid;
> > +};
> > +
> > +#define DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, latest) \
> > +    static void vmapple##major##_##minor##_class_init(ObjectClass *oc, \
> > +                                                    void *data) \
> > +    { \
> > +        MachineClass *mc = MACHINE_CLASS(oc); \
> > +        vmapple_machine_##major##_##minor##_options(mc); \
> > +        mc->desc = "QEMU " # major "." # minor " Apple Virtual Machine"; \
> > +        if (latest) { \
> > +            mc->alias = "vmapple"; \
> > +        } \
> > +    } \
> > +    static const TypeInfo machvmapple##major##_##minor##_info = { \
> > +        .name = MACHINE_TYPE_NAME("vmapple-" # major "." # minor), \
> > +        .parent = TYPE_VMAPPLE_MACHINE, \
> > +        .class_init = vmapple##major##_##minor##_class_init, \
> > +    }; \
> > +    static void machvmapple_machine_##major##_##minor##_init(void) \
> > +    { \
> > +        type_register_static(&machvmapple##major##_##minor##_info); \
> > +    } \
> > +    type_init(machvmapple_machine_##major##_##minor##_init);
> > +
> > +#define DEFINE_VMAPPLE_MACHINE_AS_LATEST(major, minor) \
> > +    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, true)
> > +#define DEFINE_VMAPPLE_MACHINE(major, minor) \
> > +    DEFINE_VMAPPLE_MACHINE_LATEST(major, minor, false)
> > +
> > +#define TYPE_VMAPPLE_MACHINE   MACHINE_TYPE_NAME("vmapple")
> > +OBJECT_DECLARE_TYPE(VMAppleMachineState, VMAppleMachineClass, VMAPPLE_MACHINE)
> > +
> > +/* Number of external interrupt lines to configure the GIC with */
> > +#define NUM_IRQS 256
> > +
> > +enum {
> > +    VMAPPLE_FIRMWARE,
> > +    VMAPPLE_CONFIG,
> > +    VMAPPLE_MEM,
> > +    VMAPPLE_GIC_DIST,
> > +    VMAPPLE_GIC_REDIST,
> > +    VMAPPLE_UART,
> > +    VMAPPLE_RTC,
> > +    VMAPPLE_PCIE,
> > +    VMAPPLE_PCIE_MMIO,
> > +    VMAPPLE_PCIE_ECAM,
> > +    VMAPPLE_GPIO,
> > +    VMAPPLE_PVPANIC,
> > +    VMAPPLE_APV_GFX,
> > +    VMAPPLE_APV_IOSFC,
> > +    VMAPPLE_AES_1,
> > +    VMAPPLE_AES_2,
> > +    VMAPPLE_BDOOR,
> > +    VMAPPLE_MEMMAP_LAST,
> > +};
> > +
> > +static MemMapEntry memmap[] = {
> > +    [VMAPPLE_FIRMWARE] =           { 0x00100000, 0x00100000 },
> > +    [VMAPPLE_CONFIG] =             { 0x00400000, 0x00010000 },
> > +
> > +    [VMAPPLE_GIC_DIST] =           { 0x10000000, 0x00010000 },
> > +    [VMAPPLE_GIC_REDIST] =         { 0x10010000, 0x00400000 },
> > +
> > +    [VMAPPLE_UART] =               { 0x20010000, 0x00010000 },
> > +    [VMAPPLE_RTC] =                { 0x20050000, 0x00001000 },
> > +    [VMAPPLE_GPIO] =               { 0x20060000, 0x00001000 },
> > +    [VMAPPLE_PVPANIC] =            { 0x20070000, 0x00000002 },
> > +    [VMAPPLE_BDOOR] =              { 0x30000000, 0x00200000 },
> > +    [VMAPPLE_APV_GFX] =            { 0x30200000, 0x00010000 },
> > +    [VMAPPLE_APV_IOSFC] =          { 0x30210000, 0x00010000 },
> > +    [VMAPPLE_AES_1] =              { 0x30220000, 0x00004000 },
> > +    [VMAPPLE_AES_2] =              { 0x30230000, 0x00004000 },
> > +    [VMAPPLE_PCIE_ECAM] =          { 0x40000000, 0x10000000 },
> > +    [VMAPPLE_PCIE_MMIO] =          { 0x50000000, 0x1fff0000 },
> > +
> > +    /* Actual RAM size depends on configuration */
> > +    [VMAPPLE_MEM] =                { 0x70000000ULL, GiB},
> > +};
> > +
> > +static const int irqmap[] = {
> > +    [VMAPPLE_UART] = 1,
> > +    [VMAPPLE_RTC] = 2,
> > +    [VMAPPLE_GPIO] = 0x5,
> > +    [VMAPPLE_APV_IOSFC] = 0x10,
> > +    [VMAPPLE_APV_GFX] = 0x11,
> > +    [VMAPPLE_AES_1] = 0x12,
> > +    [VMAPPLE_PCIE] = 0x20,
> > +};
> > +
> > +#define GPEX_NUM_IRQS 16
> > +
> > +static void create_bdif(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    DeviceState *bdif;
> > +    SysBusDevice *bdif_sb;
> > +    DriveInfo *di_aux = drive_get(IF_PFLASH, 0, 0);
> > +    DriveInfo *di_root = drive_get(IF_PFLASH, 0, 1);
> > +
> > +    if (!di_aux) {
> > +        error_report("No AUX device. Please specify one as pflash drive.");
> > +        exit(1);
> > +    }
> > +
> > +    if (!di_root) {
> > +        /* Fall back to the first IF_VIRTIO device as root device */
> > +        di_root = drive_get(IF_VIRTIO, 0, 0);
> > +    }
> > +
> > +    if (!di_root) {
> > +        error_report("No root device. Please specify one as virtio drive.");
> > +        exit(1);
> > +    }
> > +
> > +    /* PV backdoor device */
> > +    bdif = qdev_new(TYPE_VMAPPLE_BDIF);
> > +    bdif_sb = SYS_BUS_DEVICE(bdif);
> > +    sysbus_mmio_map(bdif_sb, 0, vms->memmap[VMAPPLE_BDOOR].base);
> > +
> > +    qdev_prop_set_drive(DEVICE(bdif), "aux", blk_by_legacy_dinfo(di_aux));
> > +    qdev_prop_set_drive(DEVICE(bdif), "root", blk_by_legacy_dinfo(di_root));
> > +
> > +    sysbus_realize_and_unref(bdif_sb, &error_fatal);
> > +}
> > +
> > +static void create_pvpanic(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    SysBusDevice *cfg;
> > +
> > +    vms->cfg = qdev_new(TYPE_PVPANIC_MMIO_DEVICE);
> > +    cfg = SYS_BUS_DEVICE(vms->cfg);
> > +    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_PVPANIC].base);
> > +
> > +    sysbus_realize_and_unref(cfg, &error_fatal);
> > +}
> > +
> > +static void create_cfg(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    SysBusDevice *cfg;
> > +    MachineState *machine = MACHINE(vms);
> > +    uint32_t rnd = 1;
> > +
> > +    vms->cfg = qdev_new(TYPE_VMAPPLE_CFG);
> > +    cfg = SYS_BUS_DEVICE(vms->cfg);
> > +    sysbus_mmio_map(cfg, 0, vms->memmap[VMAPPLE_CONFIG].base);
> > +
> > +    qemu_guest_getrandom_nofail(&rnd, sizeof(rnd));
> > +
> > +    qdev_prop_set_uint32(vms->cfg, "nr-cpus", machine->smp.cpus);
> > +    qdev_prop_set_uint64(vms->cfg, "ecid", vms->uuid);
> > +    qdev_prop_set_uint64(vms->cfg, "ram-size", machine->ram_size);
> > +    qdev_prop_set_uint32(vms->cfg, "rnd", rnd);
> > +
> > +    sysbus_realize_and_unref(cfg, &error_fatal);
> > +}
> > +
> > +static void create_gfx(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    int irq_gfx = vms->irqmap[VMAPPLE_APV_GFX];
> > +    int irq_iosfc = vms->irqmap[VMAPPLE_APV_IOSFC];
> > +    SysBusDevice *gfx;
> > +
> > +    gfx = SYS_BUS_DEVICE(qdev_new("apple-gfx-mmio"));
> > +    sysbus_mmio_map(gfx, 0, vms->memmap[VMAPPLE_APV_GFX].base);
> > +    sysbus_mmio_map(gfx, 1, vms->memmap[VMAPPLE_APV_IOSFC].base);
> > +    sysbus_connect_irq(gfx, 0, qdev_get_gpio_in(vms->gic, irq_gfx));
> > +    sysbus_connect_irq(gfx, 1, qdev_get_gpio_in(vms->gic, irq_iosfc));
> > +    sysbus_realize_and_unref(gfx, &error_fatal);
> > +}
> > +
> > +static void create_aes(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    int irq = vms->irqmap[VMAPPLE_AES_1];
> > +    SysBusDevice *aes;
> > +
> > +    aes = SYS_BUS_DEVICE(qdev_new(TYPE_APPLE_AES));
> > +    sysbus_mmio_map(aes, 0, vms->memmap[VMAPPLE_AES_1].base);
> > +    sysbus_mmio_map(aes, 1, vms->memmap[VMAPPLE_AES_2].base);
> > +    sysbus_connect_irq(aes, 0, qdev_get_gpio_in(vms->gic, irq));
> > +    sysbus_realize_and_unref(aes, &error_fatal);
> > +}
> > +
> > +static int arm_gic_ppi_index(int cpu_nr, int ppi_index)
> > +{
> > +    return NUM_IRQS + cpu_nr * GIC_INTERNAL + ppi_index;
> > +}
> > +
> > +static void create_gic(VMAppleMachineState *vms, MemoryRegion *mem)
> > +{
> > +    MachineState *ms = MACHINE(vms);
> > +    /* We create a standalone GIC */
> > +    SysBusDevice *gicbusdev;
> > +    QList *redist_region_count;
> > +    int i;
> > +    unsigned int smp_cpus = ms->smp.cpus;
> > +
> > +    vms->gic = qdev_new(gicv3_class_name());
> > +    qdev_prop_set_uint32(vms->gic, "revision", 3);
> > +    qdev_prop_set_uint32(vms->gic, "num-cpu", smp_cpus);
> > +    /*
> > +     * Note that the num-irq property counts both internal and external
> > +     * interrupts; there are always 32 of the former (mandated by GIC spec).
> > +     */
> > +    qdev_prop_set_uint32(vms->gic, "num-irq", NUM_IRQS + 32);
> > +
> > +    uint32_t redist0_capacity =
> > +                vms->memmap[VMAPPLE_GIC_REDIST].size / GICV3_REDIST_SIZE;
> > +    uint32_t redist0_count = MIN(smp_cpus, redist0_capacity);
> > +
> > +    redist_region_count = qlist_new();
> > +    qlist_append_int(redist_region_count, redist0_count);
> > +    qdev_prop_set_array(vms->gic, "redist-region-count", redist_region_count);
> > +
> > +    gicbusdev = SYS_BUS_DEVICE(vms->gic);
> > +    sysbus_realize_and_unref(gicbusdev, &error_fatal);
> > +    sysbus_mmio_map(gicbusdev, 0, vms->memmap[VMAPPLE_GIC_DIST].base);
> > +    sysbus_mmio_map(gicbusdev, 1, vms->memmap[VMAPPLE_GIC_REDIST].base);
> > +
> > +    /*
> > +     * Wire the outputs from each CPU's generic timer and the GICv3
> > +     * maintenance interrupt signal to the appropriate GIC PPI inputs,
> > +     * and the GIC's IRQ/FIQ/VIRQ/VFIQ interrupt outputs to the CPU's inputs.
> > +     */
> > +    for (i = 0; i < smp_cpus; i++) {
> > +        DeviceState *cpudev = DEVICE(qemu_get_cpu(i));
> > +
> > +        /* Map the virt timer to PPI 27 */
> > +        qdev_connect_gpio_out(cpudev, GTIMER_VIRT,
> > +                              qdev_get_gpio_in(vms->gic,
> > +                                               arm_gic_ppi_index(i, 27)));
> > +
> > +        /* Map the GIC IRQ and FIQ lines to CPU */
> > +        sysbus_connect_irq(gicbusdev, i, qdev_get_gpio_in(cpudev, ARM_CPU_IRQ));
> > +        sysbus_connect_irq(gicbusdev, i + smp_cpus,
> > +                           qdev_get_gpio_in(cpudev, ARM_CPU_FIQ));
> > +    }
> > +}
> > +
> > +static void create_uart(const VMAppleMachineState *vms, int uart,
> > +                        MemoryRegion *mem, Chardev *chr)
> > +{
> > +    hwaddr base = vms->memmap[uart].base;
> > +    int irq = vms->irqmap[uart];
> > +    DeviceState *dev = qdev_new(TYPE_PL011);
> > +    SysBusDevice *s = SYS_BUS_DEVICE(dev);
> > +
> > +    qdev_prop_set_chr(dev, "chardev", chr);
> > +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> > +    memory_region_add_subregion(mem, base,
> > +                                sysbus_mmio_get_region(s, 0));
> > +    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
> > +}
> > +
> > +static void create_rtc(const VMAppleMachineState *vms)
> > +{
> > +    hwaddr base = vms->memmap[VMAPPLE_RTC].base;
> > +    int irq = vms->irqmap[VMAPPLE_RTC];
> > +
> > +    sysbus_create_simple("pl031", base, qdev_get_gpio_in(vms->gic, irq));
> > +}
> > +
> > +static DeviceState *gpio_key_dev;
> > +static void vmapple_powerdown_req(Notifier *n, void *opaque)
> > +{
> > +    /* use gpio Pin 3 for power button event */
> > +    qemu_set_irq(qdev_get_gpio_in(gpio_key_dev, 0), 1);
> > +}
> > +
> > +static void create_gpio_devices(const VMAppleMachineState *vms, int gpio,
> > +                                MemoryRegion *mem)
> > +{
> > +    DeviceState *pl061_dev;
> > +    hwaddr base = vms->memmap[gpio].base;
> > +    int irq = vms->irqmap[gpio];
> > +    SysBusDevice *s;
> > +
> > +    pl061_dev = qdev_new("pl061");
> > +    /* Pull lines down to 0 if not driven by the PL061 */
> > +    qdev_prop_set_uint32(pl061_dev, "pullups", 0);
> > +    qdev_prop_set_uint32(pl061_dev, "pulldowns", 0xff);
> > +    s = SYS_BUS_DEVICE(pl061_dev);
> > +    sysbus_realize_and_unref(s, &error_fatal);
> > +    memory_region_add_subregion(mem, base, sysbus_mmio_get_region(s, 0));
> > +    sysbus_connect_irq(s, 0, qdev_get_gpio_in(vms->gic, irq));
> > +    gpio_key_dev = sysbus_create_simple("gpio-key", -1,
> > +                                        qdev_get_gpio_in(pl061_dev, 3));
> > +}
> > +
> > +static void vmapple_firmware_init(VMAppleMachineState *vms,
> > +                                  MemoryRegion *sysmem)
> > +{
> > +    hwaddr size = vms->memmap[VMAPPLE_FIRMWARE].size;
> > +    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
> > +    const char *bios_name;
> > +    int image_size;
> > +    char *fname;
> > +
> > +    bios_name = MACHINE(vms)->firmware;
> > +    if (!bios_name) {
> > +        error_report("No firmware specified");
> > +        exit(1);
> > +    }
> > +
> > +    fname = qemu_find_file(QEMU_FILE_TYPE_BIOS, bios_name);
> > +    if (!fname) {
> > +        error_report("Could not find ROM image '%s'", bios_name);
> > +        exit(1);
> > +    }
> > +
> > +    memory_region_init_ram(&vms->fw_mr, NULL, "firmware", size, &error_fatal);
> > +    image_size = load_image_mr(fname, &vms->fw_mr);
> > +
> > +    g_free(fname);
> > +    if (image_size < 0) {
> > +        error_report("Could not load ROM image '%s'", bios_name);
> > +        exit(1);
> > +    }
> > +
> > +    memory_region_add_subregion(get_system_memory(), base, &vms->fw_mr);
> > +}
> > +
> > +static void create_pcie(VMAppleMachineState *vms)
> > +{
> > +    hwaddr base_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].base;
> > +    hwaddr size_mmio = vms->memmap[VMAPPLE_PCIE_MMIO].size;
> > +    hwaddr base_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].base;
> > +    hwaddr size_ecam = vms->memmap[VMAPPLE_PCIE_ECAM].size;
> > +    int irq = vms->irqmap[VMAPPLE_PCIE];
> > +    MemoryRegion *mmio_alias;
> > +    MemoryRegion *mmio_reg;
> > +    MemoryRegion *ecam_reg;
> > +    DeviceState *dev;
> > +    int i;
> > +    PCIHostState *pci;
> > +    DeviceState *usb_controller;
> > +    USBBus *usb_bus;
> > +
> > +    dev = qdev_new(TYPE_GPEX_HOST);
> > +    qdev_prop_set_uint32(dev, "num-irqs", GPEX_NUM_IRQS);
> > +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> > +
> > +    /* Map only the first size_ecam bytes of ECAM space */
> > +    ecam_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 0);
> > +    memory_region_init_alias(&vms->ecam_alias, OBJECT(dev), "pcie-ecam",
> > +                             ecam_reg, 0, size_ecam);
> > +    memory_region_add_subregion(get_system_memory(), base_ecam,
> > +                                &vms->ecam_alias);
> > +
> > +    /*
> > +     * Map the MMIO window from [0x50000000-0x7fff0000] in PCI space into
> > +     * system address space at [0x50000000-0x7fff0000].
> > +     */
> > +    mmio_alias = g_new0(MemoryRegion, 1);
> > +    mmio_reg = sysbus_mmio_get_region(SYS_BUS_DEVICE(dev), 1);
> > +    memory_region_init_alias(mmio_alias, OBJECT(dev), "pcie-mmio",
> > +                             mmio_reg, base_mmio, size_mmio);
> > +    memory_region_add_subregion(get_system_memory(), base_mmio, mmio_alias);
> > +
> > +    for (i = 0; i < GPEX_NUM_IRQS; i++) {
> > +        sysbus_connect_irq(SYS_BUS_DEVICE(dev), i,
> > +                           qdev_get_gpio_in(vms->gic, irq + i));
> > +        gpex_set_irq_num(GPEX_HOST(dev), i, irq + i);
> > +    }
> > +
> > +    pci = PCI_HOST_BRIDGE(dev);
> > +    vms->bus = pci->bus;
> > +    g_assert(vms->bus);
> > +
> > +    while ((dev = qemu_create_nic_device("virtio-net-pci", true, NULL))) {
> > +        qdev_realize_and_unref(dev, BUS(vms->bus), &error_fatal);
> > +    }
> > +
> > +    usb_controller = qdev_new(TYPE_QEMU_XHCI);
> > +    qdev_realize_and_unref(usb_controller, BUS(pci->bus), &error_fatal);
> > +
> > +    usb_bus = USB_BUS(object_resolve_type_unambiguous(TYPE_USB_BUS,
> > +                                                      &error_fatal));
> > +    usb_create_simple(usb_bus, "usb-kbd");
> > +    usb_create_simple(usb_bus, "usb-tablet");
> > +}
> > +
> > +static void vmapple_reset(void *opaque)
> > +{
> > +    VMAppleMachineState *vms = opaque;
> > +    hwaddr base = vms->memmap[VMAPPLE_FIRMWARE].base;
> > +
> > +    cpu_set_pc(first_cpu, base);
> > +}
> > +
> > +static void mach_vmapple_init(MachineState *machine)
> > +{
> > +    VMAppleMachineState *vms = VMAPPLE_MACHINE(machine);
> > +    MachineClass *mc = MACHINE_GET_CLASS(machine);
> > +    const CPUArchIdList *possible_cpus;
> > +    MemoryRegion *sysmem = get_system_memory();
> > +    int n;
> > +    unsigned int smp_cpus = machine->smp.cpus;
> > +    unsigned int max_cpus = machine->smp.max_cpus;
> > +
> > +    vms->memmap = memmap;
> > +    machine->usb = true;
> > +
> > +    possible_cpus = mc->possible_cpu_arch_ids(machine);
> > +    assert(possible_cpus->len == max_cpus);
> > +    for (n = 0; n < possible_cpus->len; n++) {
> > +        Object *cpu;
> > +        CPUState *cs;
> > +
> > +        if (n >= smp_cpus) {
> > +            break;
> > +        }
> > +
> > +        cpu = object_new(possible_cpus->cpus[n].type);
> > +        object_property_set_int(cpu, "mp-affinity",
> > +                                possible_cpus->cpus[n].arch_id, &error_fatal);
> > +
> > +        cs = CPU(cpu);
> > +        cs->cpu_index = n;
> > +
> > +        numa_cpu_pre_plug(&possible_cpus->cpus[cs->cpu_index], DEVICE(cpu),
> > +                          &error_fatal);
> > +
> > +        if (object_property_find(cpu, "has_el3")) {
> > +            object_property_set_bool(cpu, "has_el3", false, &error_fatal);
> > +        }
> > +        if (object_property_find(cpu, "has_el2")) {
> > +            object_property_set_bool(cpu, "has_el2", false, &error_fatal);
> > +        }
> > +        object_property_set_int(cpu, "psci-conduit", QEMU_PSCI_CONDUIT_HVC,
> > +                                &error_fatal);
> > +
> > +        /* Secondary CPUs start in PSCI powered-down state */
> > +        if (n > 0) {
> > +            object_property_set_bool(cpu, "start-powered-off", true,
> > +                                     &error_fatal);
> > +        }
> > +
> > +        object_property_set_link(cpu, "memory", OBJECT(sysmem), &error_abort);
> > +        qdev_realize(DEVICE(cpu), NULL, &error_fatal);
> > +        object_unref(cpu);
> > +    }
> > +
> > +    memory_region_add_subregion(sysmem, vms->memmap[VMAPPLE_MEM].base,
> > +                                machine->ram);
> > +
> > +    create_gic(vms, sysmem);
> > +    create_bdif(vms, sysmem);
> > +    create_pvpanic(vms, sysmem);
> > +    create_aes(vms, sysmem);
> > +    create_gfx(vms, sysmem);
> > +    create_uart(vms, VMAPPLE_UART, sysmem, serial_hd(0));
> > +    create_rtc(vms);
> > +    create_pcie(vms);
> > +
> > +    create_gpio_devices(vms, VMAPPLE_GPIO, sysmem);
> > +
> > +    vmapple_firmware_init(vms, sysmem);
> > +    create_cfg(vms, sysmem);
> > +
> > +    /* connect powerdown request */
> > +    vms->powerdown_notifier.notify = vmapple_powerdown_req;
> > +    qemu_register_powerdown_notifier(&vms->powerdown_notifier);
> > +
> > +    vms->bootinfo.ram_size = machine->ram_size;
> > +    vms->bootinfo.board_id = -1;
> > +    vms->bootinfo.loader_start = vms->memmap[VMAPPLE_MEM].base;
> > +    vms->bootinfo.skip_dtb_autoload = true;
> > +    vms->bootinfo.firmware_loaded = true;
> > +    arm_load_kernel(ARM_CPU(first_cpu), machine, &vms->bootinfo);
> > +
> > +    qemu_register_reset(vmapple_reset, vms);
> > +}
> > +
> > +static CpuInstanceProperties
> > +vmapple_cpu_index_to_props(MachineState *ms, unsigned cpu_index)
> > +{
> > +    MachineClass *mc = MACHINE_GET_CLASS(ms);
> > +    const CPUArchIdList *possible_cpus = mc->possible_cpu_arch_ids(ms);
> > +
> > +    assert(cpu_index < possible_cpus->len);
> > +    return possible_cpus->cpus[cpu_index].props;
> > +}
> > +
> > +
> > +static int64_t vmapple_get_default_cpu_node_id(const MachineState *ms, int idx)
> > +{
> > +    return idx % ms->numa_state->num_nodes;
> > +}
> > +
> > +static const CPUArchIdList *vmapple_possible_cpu_arch_ids(MachineState *ms)
> > +{
> > +    int n;
> > +    unsigned int max_cpus = ms->smp.max_cpus;
> > +
> > +    if (ms->possible_cpus) {
> > +        assert(ms->possible_cpus->len == max_cpus);
> > +        return ms->possible_cpus;
> > +    }
> > +
> > +    ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
> > +                                  sizeof(CPUArchId) * max_cpus);
> > +    ms->possible_cpus->len = max_cpus;
> > +    for (n = 0; n < ms->possible_cpus->len; n++) {
> > +        ms->possible_cpus->cpus[n].type = ms->cpu_type;
> > +        ms->possible_cpus->cpus[n].arch_id =
> > +            arm_build_mp_affinity(n, GICV3_TARGETLIST_BITS);
> > +        ms->possible_cpus->cpus[n].props.has_thread_id = true;
> > +        ms->possible_cpus->cpus[n].props.thread_id = n;
> > +    }
> > +    return ms->possible_cpus;
> > +}
> > +
> > +static void vmapple_machine_class_init(ObjectClass *oc, void *data)
> > +{
> > +    MachineClass *mc = MACHINE_CLASS(oc);
> > +
> > +    mc->init = mach_vmapple_init;
> > +    mc->max_cpus = 32;
> > +    mc->block_default_type = IF_VIRTIO;
> > +    mc->no_cdrom = 1;
> > +    mc->pci_allow_0_address = true;
> > +    mc->minimum_page_bits = 12;
> > +    mc->possible_cpu_arch_ids = vmapple_possible_cpu_arch_ids;
> > +    mc->cpu_index_to_instance_props = vmapple_cpu_index_to_props;
> > +    mc->default_cpu_type = ARM_CPU_TYPE_NAME("host");
> > +    mc->get_default_cpu_node_id = vmapple_get_default_cpu_node_id;
> > +    mc->default_ram_id = "mach-vmapple.ram";
> > +
> > +    object_register_sugar_prop(TYPE_VIRTIO_PCI, "disable-legacy",
> > +                               "on", true);
> > +}
> > +
> > +static void vmapple_instance_init(Object *obj)
> > +{
> > +    VMAppleMachineState *vms = VMAPPLE_MACHINE(obj);
> > +
> > +    vms->irqmap = irqmap;
> > +
> > +    object_property_add_uint64_ptr(obj, "uuid", &vms->uuid,
> > +                                   OBJ_PROP_FLAG_READWRITE);
> > +    object_property_set_description(obj, "uuid", "Machine UUID (SDOM)");
> > +}
> > +
> > +static const TypeInfo vmapple_machine_info = {
> > +    .name          = TYPE_VMAPPLE_MACHINE,
> > +    .parent        = TYPE_MACHINE,
> > +    .abstract      = true,
> > +    .instance_size = sizeof(VMAppleMachineState),
> > +    .class_size    = sizeof(VMAppleMachineClass),
> > +    .class_init    = vmapple_machine_class_init,
> > +    .instance_init = vmapple_instance_init,
> > +};
> > +
> > +static void machvmapple_machine_init(void)
> > +{
> > +    type_register_static(&vmapple_machine_info);
> > +}
> > +type_init(machvmapple_machine_init);
> > +
> > +static void vmapple_machine_9_2_options(MachineClass *mc)
> > +{
> > +}
> > +DEFINE_VMAPPLE_MACHINE_AS_LATEST(9, 2)
> > +
>
>