[PATCH] qemu: add pcie-to-pci-bridge for q35 machines

Oleg Vasilev posted 1 patch 1 year, 4 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20221104104300.1893081-1-oleg.vasilev@virtuozzo.com
src/qemu/qemu_domain.c                        | 19 +++--
.../boot-floppy-q35.x86_64-latest.args        |  5 +-
.../cpu-host-model.x86_64-4.2.0.args          |  9 +-
.../cpu-host-model.x86_64-5.0.0.args          |  9 +-
.../cpu-host-model.x86_64-5.1.0.args          |  9 +-
.../cpu-host-model.x86_64-5.2.0.args          |  9 +-
.../cpu-host-model.x86_64-6.0.0.args          |  9 +-
.../cpu-host-model.x86_64-6.1.0.args          |  9 +-
.../cpu-host-model.x86_64-latest.args         |  9 +-
.../disk-floppy-q35.x86_64-latest.args        |  5 +-
...are-auto-bios-stateless.x86_64-latest.args |  2 +
.../firmware-auto-bios.x86_64-latest.args     |  2 +
...-auto-efi-enrolled-keys.x86_64-latest.args |  2 +
...-auto-efi-loader-secure.x86_64-latest.args |  2 +
...to-efi-no-enrolled-keys.x86_64-latest.args |  2 +
...are-auto-efi-no-secboot.x86_64-latest.args |  2 +
...firmware-auto-efi-nvram.x86_64-latest.args |  2 +
...rmware-auto-efi-secboot.x86_64-latest.args |  2 +
...ware-auto-efi-stateless.x86_64-latest.args |  2 +
.../firmware-auto-efi.x86_64-latest.args      |  2 +
...dev-scsi-vhost-scsi-pcie.x86_64-4.2.0.args |  5 +-
...ev-scsi-vhost-scsi-pcie.x86_64-latest.args |  5 +-
.../intel-iommu-aw-bits.x86_64-latest.args    |  2 +
.../qemuxml2argvdata/intel-iommu-aw-bits.xml  |  9 ++
...ntel-iommu-caching-mode.x86_64-latest.args |  2 +
.../intel-iommu-caching-mode.xml              |  9 ++
...ntel-iommu-device-iotlb.x86_64-latest.args |  2 +
.../intel-iommu-device-iotlb.xml              |  9 ++
.../intel-iommu-eim.x86_64-latest.args        |  2 +
tests/qemuxml2argvdata/intel-iommu-eim.xml    |  9 ++
.../intel-iommu.x86_64-latest.args            |  2 +
tests/qemuxml2argvdata/intel-iommu.xml        |  9 ++
.../machine-smm-off.x86_64-latest.args        |  2 +
.../machine-smm-on.x86_64-latest.args         |  2 +
...q35-default-devices-only.x86_64-4.2.0.args |  9 +-
...35-default-devices-only.x86_64-latest.args |  9 +-
.../q35-pcie-autoadd.x86_64-4.2.0.args        | 53 ++++++------
.../q35-pcie-autoadd.x86_64-latest.args       | 53 ++++++------
.../q35-virt-manager-basic.x86_64-4.2.0.args  | 21 ++---
.../q35-virt-manager-basic.x86_64-latest.args | 21 ++---
.../tseg-explicit-size.x86_64-latest.args     |  9 +-
.../user-aliases2.x86_64-latest.args          |  5 +-
.../virtio-iommu-x86_64.x86_64-latest.args    |  4 +-
...virtio-non-transitional.x86_64-latest.args | 37 ++++----
.../virtio-transitional.x86_64-latest.args    |  3 +-
...default-cpu-kvm-q35-4.2.x86_64-latest.args | 13 +--
...default-cpu-tcg-q35-4.2.x86_64-latest.args | 13 +--
.../x86_64-q35-graphics.x86_64-latest.args    | 25 +++---
.../x86_64-q35-headless.x86_64-latest.args    | 25 +++---
...ware-auto-bios-stateless.x86_64-latest.xml |  9 ++
.../firmware-auto-bios.x86_64-latest.xml      |  9 ++
...e-auto-efi-enrolled-keys.x86_64-latest.xml |  9 ++
...e-auto-efi-loader-secure.x86_64-latest.xml |  9 ++
...uto-efi-no-enrolled-keys.x86_64-latest.xml |  9 ++
...ware-auto-efi-no-secboot.x86_64-latest.xml |  9 ++
.../firmware-auto-efi-nvram.x86_64-latest.xml |  9 ++
...irmware-auto-efi-secboot.x86_64-latest.xml |  9 ++
.../firmware-auto-efi.x86_64-latest.xml       |  9 ++
...dev-scsi-vhost-scsi-pcie.x86_64-latest.xml | 10 ++-
.../machine-smm-off.x86_64-latest.xml         |  9 ++
.../machine-smm-on.x86_64-latest.xml          |  9 ++
.../net-isolated-port.x86_64-latest.xml       | 18 ++--
...q35-default-devices-only.x86_64-latest.xml | 18 ++--
.../q35-pcie-autoadd.x86_64-latest.xml        | 84 ++++++++++---------
.../q35-virt-manager-basic.x86_64-latest.xml  | 36 ++++----
.../tseg-explicit-size.x86_64-latest.xml      | 18 ++--
.../virtio-iommu-x86_64.x86_64-latest.xml     | 11 ++-
.../virtio-non-transitional.x86_64-latest.xml | 60 ++++++-------
.../virtio-transitional.x86_64-latest.xml     |  7 +-
...-default-cpu-kvm-q35-4.2.x86_64-latest.xml | 24 +++---
...-default-cpu-tcg-q35-4.2.x86_64-latest.xml | 24 +++---
.../x86_64-q35-graphics.x86_64-latest.xml     | 46 +++++-----
.../x86_64-q35-headless.x86_64-latest.xml     | 46 +++++-----
73 files changed, 621 insertions(+), 361 deletions(-)
[PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Oleg Vasilev 1 year, 4 months ago
Hotplugging PCI devices on pc-i440fx machines is supported without
additional configuration. On q35, pcie-to-pci-bridge needs to be added
prior to the machine startup in order to support hotplug [1].

The idea is to support the same workflow for creating VMs in q35, as was
in pc. Namely, do not require additional configuration when hotplugging
is needed. Otherwise, all libvirt clients need to be updated which there
are a lot and they are maintained by different parties.

Instead, a pcie-to-pci-bridge better be created by default, so that PCI
slots are readily available. Might be a good idea to make it configurable
in the future.

Previously there was a pci-bridge present by default, but was removed in
  d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)

[1]: https://libvirt.org/pci-hotplug.html

Signed-off-by: Oleg Vasilev <oleg.vasilev@virtuozzo.com>
---
 src/qemu/qemu_domain.c                        | 19 +++--
 .../boot-floppy-q35.x86_64-latest.args        |  5 +-
 .../cpu-host-model.x86_64-4.2.0.args          |  9 +-
 .../cpu-host-model.x86_64-5.0.0.args          |  9 +-
 .../cpu-host-model.x86_64-5.1.0.args          |  9 +-
 .../cpu-host-model.x86_64-5.2.0.args          |  9 +-
 .../cpu-host-model.x86_64-6.0.0.args          |  9 +-
 .../cpu-host-model.x86_64-6.1.0.args          |  9 +-
 .../cpu-host-model.x86_64-latest.args         |  9 +-
 .../disk-floppy-q35.x86_64-latest.args        |  5 +-
 ...are-auto-bios-stateless.x86_64-latest.args |  2 +
 .../firmware-auto-bios.x86_64-latest.args     |  2 +
 ...-auto-efi-enrolled-keys.x86_64-latest.args |  2 +
 ...-auto-efi-loader-secure.x86_64-latest.args |  2 +
 ...to-efi-no-enrolled-keys.x86_64-latest.args |  2 +
 ...are-auto-efi-no-secboot.x86_64-latest.args |  2 +
 ...firmware-auto-efi-nvram.x86_64-latest.args |  2 +
 ...rmware-auto-efi-secboot.x86_64-latest.args |  2 +
 ...ware-auto-efi-stateless.x86_64-latest.args |  2 +
 .../firmware-auto-efi.x86_64-latest.args      |  2 +
 ...dev-scsi-vhost-scsi-pcie.x86_64-4.2.0.args |  5 +-
 ...ev-scsi-vhost-scsi-pcie.x86_64-latest.args |  5 +-
 .../intel-iommu-aw-bits.x86_64-latest.args    |  2 +
 .../qemuxml2argvdata/intel-iommu-aw-bits.xml  |  9 ++
 ...ntel-iommu-caching-mode.x86_64-latest.args |  2 +
 .../intel-iommu-caching-mode.xml              |  9 ++
 ...ntel-iommu-device-iotlb.x86_64-latest.args |  2 +
 .../intel-iommu-device-iotlb.xml              |  9 ++
 .../intel-iommu-eim.x86_64-latest.args        |  2 +
 tests/qemuxml2argvdata/intel-iommu-eim.xml    |  9 ++
 .../intel-iommu.x86_64-latest.args            |  2 +
 tests/qemuxml2argvdata/intel-iommu.xml        |  9 ++
 .../machine-smm-off.x86_64-latest.args        |  2 +
 .../machine-smm-on.x86_64-latest.args         |  2 +
 ...q35-default-devices-only.x86_64-4.2.0.args |  9 +-
 ...35-default-devices-only.x86_64-latest.args |  9 +-
 .../q35-pcie-autoadd.x86_64-4.2.0.args        | 53 ++++++------
 .../q35-pcie-autoadd.x86_64-latest.args       | 53 ++++++------
 .../q35-virt-manager-basic.x86_64-4.2.0.args  | 21 ++---
 .../q35-virt-manager-basic.x86_64-latest.args | 21 ++---
 .../tseg-explicit-size.x86_64-latest.args     |  9 +-
 .../user-aliases2.x86_64-latest.args          |  5 +-
 .../virtio-iommu-x86_64.x86_64-latest.args    |  4 +-
 ...virtio-non-transitional.x86_64-latest.args | 37 ++++----
 .../virtio-transitional.x86_64-latest.args    |  3 +-
 ...default-cpu-kvm-q35-4.2.x86_64-latest.args | 13 +--
 ...default-cpu-tcg-q35-4.2.x86_64-latest.args | 13 +--
 .../x86_64-q35-graphics.x86_64-latest.args    | 25 +++---
 .../x86_64-q35-headless.x86_64-latest.args    | 25 +++---
 ...ware-auto-bios-stateless.x86_64-latest.xml |  9 ++
 .../firmware-auto-bios.x86_64-latest.xml      |  9 ++
 ...e-auto-efi-enrolled-keys.x86_64-latest.xml |  9 ++
 ...e-auto-efi-loader-secure.x86_64-latest.xml |  9 ++
 ...uto-efi-no-enrolled-keys.x86_64-latest.xml |  9 ++
 ...ware-auto-efi-no-secboot.x86_64-latest.xml |  9 ++
 .../firmware-auto-efi-nvram.x86_64-latest.xml |  9 ++
 ...irmware-auto-efi-secboot.x86_64-latest.xml |  9 ++
 .../firmware-auto-efi.x86_64-latest.xml       |  9 ++
 ...dev-scsi-vhost-scsi-pcie.x86_64-latest.xml | 10 ++-
 .../machine-smm-off.x86_64-latest.xml         |  9 ++
 .../machine-smm-on.x86_64-latest.xml          |  9 ++
 .../net-isolated-port.x86_64-latest.xml       | 18 ++--
 ...q35-default-devices-only.x86_64-latest.xml | 18 ++--
 .../q35-pcie-autoadd.x86_64-latest.xml        | 84 ++++++++++---------
 .../q35-virt-manager-basic.x86_64-latest.xml  | 36 ++++----
 .../tseg-explicit-size.x86_64-latest.xml      | 18 ++--
 .../virtio-iommu-x86_64.x86_64-latest.xml     | 11 ++-
 .../virtio-non-transitional.x86_64-latest.xml | 60 ++++++-------
 .../virtio-transitional.x86_64-latest.xml     |  7 +-
 ...-default-cpu-kvm-q35-4.2.x86_64-latest.xml | 24 +++---
 ...-default-cpu-tcg-q35-4.2.x86_64-latest.xml | 24 +++---
 .../x86_64-q35-graphics.x86_64-latest.xml     | 46 +++++-----
 .../x86_64-q35-headless.x86_64-latest.xml     | 46 +++++-----
 73 files changed, 621 insertions(+), 361 deletions(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 9ef6c8bb64..52a393e07f 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -3890,6 +3890,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
     bool addImplicitSATA = false;
     bool addPCIRoot = false;
     bool addPCIeRoot = false;
+    bool addPCIeToPCIBridge = false;
     bool addDefaultMemballoon = true;
     bool addDefaultUSBKBD = false;
     bool addDefaultUSBMouse = false;
@@ -3910,6 +3911,8 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
         if (qemuDomainIsQ35(def)) {
             addPCIeRoot = true;
             addImplicitSATA = true;
+            if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PCIE_PCI_BRIDGE))
+                addPCIeToPCIBridge = true;
 
             /* Prefer adding a USB3 controller if supported, fall back
              * to USB2 if there is no USB3 available, and if that's
@@ -4046,11 +4049,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
         }
     }
 
-    /* When a machine has a pcie-root, make sure that there is always
-     * a dmi-to-pci-bridge controller added as bus 1, and a pci-bridge
-     * as bus 2, so that standard PCI devices can be connected
-     *
-     * NB: any machine that sets addPCIeRoot to true must also return
+    /* NB: any machine that sets addPCIeRoot to true must also return
      * true from the function qemuDomainSupportsPCI().
      */
     if (addPCIeRoot) {
@@ -4069,6 +4068,16 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
         }
     }
 
+    /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
+    if (addPCIeToPCIBridge) {
+        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
+                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
+            return -1;
+        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
+                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
+            return -1;
+    }
+
     if (addDefaultMemballoon && !def->memballoon) {
         virDomainMemballoonDef *memballoon;
         memballoon = g_new0(virDomainMemballoonDef, 1);
diff --git a/tests/qemuxml2argvdata/boot-floppy-q35.x86_64-latest.args b/tests/qemuxml2argvdata/boot-floppy-q35.x86_64-latest.args
index 7f34c7a965..75aae3254c 100644
--- a/tests/qemuxml2argvdata/boot-floppy-q35.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/boot-floppy-q35.x86_64-latest.args
@@ -28,8 +28,9 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -device '{"driver":"isa-fdc","bootindexA":1}' \
 -blockdev '{"driver":"file","filename":"/tmp/firmware.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \
diff --git a/tests/qemuxml2argvdata/cpu-host-model.x86_64-4.2.0.args b/tests/qemuxml2argvdata/cpu-host-model.x86_64-4.2.0.args
index 4b2b55be62..577ea9cd8a 100644
--- a/tests/qemuxml2argvdata/cpu-host-model.x86_64-4.2.0.args
+++ b/tests/qemuxml2argvdata/cpu-host-model.x86_64-4.2.0.args
@@ -27,10 +27,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
--device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \
--device qemu-xhci,id=usb,bus=pci.1,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-root-port,port=10,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x2 \
+-device qemu-xhci,id=usb,bus=pci.3,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.0.0.args b/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.0.0.args
index 2ea818cc13..838265acbf 100644
--- a/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.0.0.args
+++ b/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.0.0.args
@@ -27,10 +27,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
--device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \
--device qemu-xhci,id=usb,bus=pci.1,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-root-port,port=10,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x2 \
+-device qemu-xhci,id=usb,bus=pci.3,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.1.0.args b/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.1.0.args
index c6e2f345b8..50b8d56390 100644
--- a/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.1.0.args
+++ b/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.1.0.args
@@ -27,10 +27,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
--device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \
--device qemu-xhci,id=usb,bus=pci.1,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-root-port,port=10,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x2 \
+-device qemu-xhci,id=usb,bus=pci.3,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.2.0.args b/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.2.0.args
index 5bb6545e62..40302a86ee 100644
--- a/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.2.0.args
+++ b/tests/qemuxml2argvdata/cpu-host-model.x86_64-5.2.0.args
@@ -28,10 +28,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
--device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \
--device qemu-xhci,id=usb,bus=pci.1,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-root-port,port=10,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x2 \
+-device qemu-xhci,id=usb,bus=pci.3,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.0.0.args b/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.0.0.args
index 57ae8a19d8..d39171386f 100644
--- a/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.0.0.args
+++ b/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.0.0.args
@@ -28,10 +28,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
--device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \
--device qemu-xhci,id=usb,bus=pci.1,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-root-port,port=10,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x2 \
+-device qemu-xhci,id=usb,bus=pci.3,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.1.0.args b/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.1.0.args
index baf0955027..3b0b09a93c 100644
--- a/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.1.0.args
+++ b/tests/qemuxml2argvdata/cpu-host-model.x86_64-6.1.0.args
@@ -28,10 +28,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
--device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \
--device qemu-xhci,id=usb,bus=pci.1,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-root-port,port=10,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x2 \
+-device qemu-xhci,id=usb,bus=pci.3,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/cpu-host-model.x86_64-latest.args b/tests/qemuxml2argvdata/cpu-host-model.x86_64-latest.args
index 853f0d9505..4a57f1dc98 100644
--- a/tests/qemuxml2argvdata/cpu-host-model.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/cpu-host-model.x86_64-latest.args
@@ -28,10 +28,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.2","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.4","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/disk-floppy-q35.x86_64-latest.args b/tests/qemuxml2argvdata/disk-floppy-q35.x86_64-latest.args
index 810cc0796d..59da018898 100644
--- a/tests/qemuxml2argvdata/disk-floppy-q35.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/disk-floppy-q35.x86_64-latest.args
@@ -28,8 +28,9 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -device '{"driver":"isa-fdc","bootindexA":1}' \
 -blockdev '{"driver":"file","filename":"/tmp/firmware.img","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"raw","file":"libvirt-2-storage"}' \
diff --git a/tests/qemuxml2argvdata/firmware-auto-bios-stateless.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-bios-stateless.x86_64-latest.args
index 1d45a8cfba..a27bfa997d 100644
--- a/tests/qemuxml2argvdata/firmware-auto-bios-stateless.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-bios-stateless.x86_64-latest.args
@@ -27,6 +27,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-bios.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-bios.x86_64-latest.args
index 1d45a8cfba..a27bfa997d 100644
--- a/tests/qemuxml2argvdata/firmware-auto-bios.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-bios.x86_64-latest.args
@@ -27,6 +27,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi-enrolled-keys.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi-enrolled-keys.x86_64-latest.args
index 0b02464963..7dbed3af6e 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi-enrolled-keys.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi-enrolled-keys.x86_64-latest.args
@@ -31,6 +31,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi-loader-secure.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi-loader-secure.x86_64-latest.args
index 0b02464963..7dbed3af6e 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi-loader-secure.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi-loader-secure.x86_64-latest.args
@@ -31,6 +31,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.args
index a1ef95ee03..9b24126035 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.args
@@ -30,6 +30,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi-no-secboot.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi-no-secboot.x86_64-latest.args
index a1ef95ee03..9b24126035 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi-no-secboot.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi-no-secboot.x86_64-latest.args
@@ -30,6 +30,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi-nvram.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi-nvram.x86_64-latest.args
index 3659889bf3..78871166a7 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi-nvram.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi-nvram.x86_64-latest.args
@@ -31,6 +31,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi-secboot.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi-secboot.x86_64-latest.args
index 0b02464963..7dbed3af6e 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi-secboot.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi-secboot.x86_64-latest.args
@@ -31,6 +31,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi-stateless.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi-stateless.x86_64-latest.args
index b79c141b18..5e36ff4854 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi-stateless.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi-stateless.x86_64-latest.args
@@ -28,6 +28,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/firmware-auto-efi.x86_64-latest.args b/tests/qemuxml2argvdata/firmware-auto-efi.x86_64-latest.args
index 0b02464963..7dbed3af6e 100644
--- a/tests/qemuxml2argvdata/firmware-auto-efi.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/firmware-auto-efi.x86_64-latest.args
@@ -31,6 +31,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-fedora/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-4.2.0.args b/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-4.2.0.args
index 992d8ec905..eec461d679 100644
--- a/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-4.2.0.args
+++ b/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-4.2.0.args
@@ -27,8 +27,9 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest2/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device vhost-scsi-pci,wwpn=naa.5123456789abcde0,vhostfd=3,id=hostdev0,bus=pci.1,addr=0x0 \
+-device vhost-scsi-pci,wwpn=naa.5123456789abcde0,vhostfd=3,id=hostdev0,bus=pci.3,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.args b/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.args
index 98b5e9b0ad..af1afcc455 100644
--- a/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.args
@@ -28,8 +28,9 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest2/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"vhost-scsi-pci","wwpn":"naa.5123456789abcde0","vhostfd":"3","id":"hostdev0","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"vhost-scsi-pci","wwpn":"naa.5123456789abcde0","vhostfd":"3","id":"hostdev0","bus":"pci.3","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/intel-iommu-aw-bits.x86_64-latest.args b/tests/qemuxml2argvdata/intel-iommu-aw-bits.x86_64-latest.args
index 417b5fb9dc..dbcb8d7e93 100644
--- a/tests/qemuxml2argvdata/intel-iommu-aw-bits.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/intel-iommu-aw-bits.x86_64-latest.args
@@ -28,6 +28,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"intel-iommu","id":"iommu0","intremap":"on","aw-bits":48}' \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/intel-iommu-aw-bits.xml b/tests/qemuxml2argvdata/intel-iommu-aw-bits.xml
index b4eb81ff5d..6db13b476c 100644
--- a/tests/qemuxml2argvdata/intel-iommu-aw-bits.xml
+++ b/tests/qemuxml2argvdata/intel-iommu-aw-bits.xml
@@ -21,6 +21,15 @@
   <devices>
     <emulator>/usr/bin/qemu-system-x86_64</emulator>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <controller type='usb' index='0' model='none'/>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
diff --git a/tests/qemuxml2argvdata/intel-iommu-caching-mode.x86_64-latest.args b/tests/qemuxml2argvdata/intel-iommu-caching-mode.x86_64-latest.args
index bfcfc35e28..bc98aeb691 100644
--- a/tests/qemuxml2argvdata/intel-iommu-caching-mode.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/intel-iommu-caching-mode.x86_64-latest.args
@@ -28,6 +28,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"intel-iommu","id":"iommu0","intremap":"on","caching-mode":true}' \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/intel-iommu-caching-mode.xml b/tests/qemuxml2argvdata/intel-iommu-caching-mode.xml
index 4aa4bacf5b..9fb1bcdf8a 100644
--- a/tests/qemuxml2argvdata/intel-iommu-caching-mode.xml
+++ b/tests/qemuxml2argvdata/intel-iommu-caching-mode.xml
@@ -21,6 +21,15 @@
   <devices>
     <emulator>/usr/bin/qemu-system-x86_64</emulator>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <controller type='usb' index='0' model='none'/>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
diff --git a/tests/qemuxml2argvdata/intel-iommu-device-iotlb.x86_64-latest.args b/tests/qemuxml2argvdata/intel-iommu-device-iotlb.x86_64-latest.args
index 95cd9d26ef..9787ba9b1e 100644
--- a/tests/qemuxml2argvdata/intel-iommu-device-iotlb.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/intel-iommu-device-iotlb.x86_64-latest.args
@@ -28,6 +28,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"intel-iommu","id":"iommu0","intremap":"on","device-iotlb":true}' \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/intel-iommu-device-iotlb.xml b/tests/qemuxml2argvdata/intel-iommu-device-iotlb.xml
index e106737e6a..577bf9bbbd 100644
--- a/tests/qemuxml2argvdata/intel-iommu-device-iotlb.xml
+++ b/tests/qemuxml2argvdata/intel-iommu-device-iotlb.xml
@@ -21,6 +21,15 @@
   <devices>
     <emulator>/usr/bin/qemu-system-x86_64</emulator>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <controller type='usb' index='0' model='none'/>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
diff --git a/tests/qemuxml2argvdata/intel-iommu-eim.x86_64-latest.args b/tests/qemuxml2argvdata/intel-iommu-eim.x86_64-latest.args
index 12113a2800..a27b8c581b 100644
--- a/tests/qemuxml2argvdata/intel-iommu-eim.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/intel-iommu-eim.x86_64-latest.args
@@ -28,6 +28,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"intel-iommu","id":"iommu0","intremap":"on","eim":"on"}' \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/intel-iommu-eim.xml b/tests/qemuxml2argvdata/intel-iommu-eim.xml
index c4942e4c79..76a99bc7f9 100644
--- a/tests/qemuxml2argvdata/intel-iommu-eim.xml
+++ b/tests/qemuxml2argvdata/intel-iommu-eim.xml
@@ -21,6 +21,15 @@
   <devices>
     <emulator>/usr/bin/qemu-system-x86_64</emulator>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <controller type='usb' index='0' model='none'/>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
diff --git a/tests/qemuxml2argvdata/intel-iommu.x86_64-latest.args b/tests/qemuxml2argvdata/intel-iommu.x86_64-latest.args
index c99efd229b..53c92f69c1 100644
--- a/tests/qemuxml2argvdata/intel-iommu.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/intel-iommu.x86_64-latest.args
@@ -28,6 +28,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"intel-iommu","id":"iommu0"}' \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/intel-iommu.xml b/tests/qemuxml2argvdata/intel-iommu.xml
index 8badf6c970..2aa3c5a3c0 100644
--- a/tests/qemuxml2argvdata/intel-iommu.xml
+++ b/tests/qemuxml2argvdata/intel-iommu.xml
@@ -18,6 +18,15 @@
   <devices>
     <emulator>/usr/bin/qemu-system-x86_64</emulator>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <controller type='usb' index='0' model='none'/>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
diff --git a/tests/qemuxml2argvdata/machine-smm-off.x86_64-latest.args b/tests/qemuxml2argvdata/machine-smm-off.x86_64-latest.args
index 46f6a4a70c..737ed31919 100644
--- a/tests/qemuxml2argvdata/machine-smm-off.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/machine-smm-off.x86_64-latest.args
@@ -27,6 +27,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-shutdown \
 -no-acpi \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/machine-smm-on.x86_64-latest.args b/tests/qemuxml2argvdata/machine-smm-on.x86_64-latest.args
index 6fde375523..ced719d4e2 100644
--- a/tests/qemuxml2argvdata/machine-smm-on.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/machine-smm-on.x86_64-latest.args
@@ -27,6 +27,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-shutdown \
 -no-acpi \
 -boot strict=on \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-4.2.0.args b/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-4.2.0.args
index a4de318a05..d2995d16d0 100644
--- a/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-4.2.0.args
+++ b/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-4.2.0.args
@@ -27,10 +27,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-q35-test/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 \
--device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 \
--device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 \
--device qemu-xhci,id=usb,bus=pci.1,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=9,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x1 \
+-device pcie-root-port,port=10,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x2 \
+-device qemu-xhci,id=usb,bus=pci.3,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-latest.args b/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-latest.args
index ce5b2a216e..d3889717f4 100644
--- a/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/q35-default-devices-only.x86_64-latest.args
@@ -28,10 +28,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-q35-test/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.2","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.4","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-4.2.0.args b/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-4.2.0.args
index 787cbfa511..1f629a0371 100644
--- a/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-4.2.0.args
+++ b/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-4.2.0.args
@@ -27,39 +27,40 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-q35-test/.config \
 -no-acpi \
 -boot strict=on \
 -device pcie-root-port,port=16,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \
--device pcie-root-port,port=17,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
--device pcie-root-port,port=18,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
--device pcie-root-port,port=19,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
--device pcie-root-port,port=20,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
--device pcie-root-port,port=21,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
--device pcie-root-port,port=22,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \
--device pcie-root-port,port=23,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 \
--device pcie-root-port,port=24,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x3 \
--device pcie-root-port,port=25,chassis=10,id=pci.10,bus=pcie.0,addr=0x3.0x1 \
--device pcie-root-port,port=26,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x2 \
--device pcie-root-port,port=27,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x3 \
--device pcie-root-port,port=28,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x4 \
--device pcie-root-port,port=29,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x5 \
--device nec-usb-xhci,id=usb,bus=pci.6,addr=0x0 \
--device virtio-scsi-pci,id=scsi0,bus=pci.5,addr=0x0 \
--device virtio-serial-pci,id=virtio-serial0,bus=pci.4,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=17,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x1 \
+-device pcie-root-port,port=18,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x2 \
+-device pcie-root-port,port=19,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x3 \
+-device pcie-root-port,port=20,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x4 \
+-device pcie-root-port,port=21,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x5 \
+-device pcie-root-port,port=22,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x6 \
+-device pcie-root-port,port=23,chassis=9,id=pci.9,bus=pcie.0,addr=0x2.0x7 \
+-device pcie-root-port,port=24,chassis=10,id=pci.10,bus=pcie.0,multifunction=on,addr=0x3 \
+-device pcie-root-port,port=25,chassis=11,id=pci.11,bus=pcie.0,addr=0x3.0x1 \
+-device pcie-root-port,port=26,chassis=12,id=pci.12,bus=pcie.0,addr=0x3.0x2 \
+-device pcie-root-port,port=27,chassis=13,id=pci.13,bus=pcie.0,addr=0x3.0x3 \
+-device pcie-root-port,port=28,chassis=14,id=pci.14,bus=pcie.0,addr=0x3.0x4 \
+-device pcie-root-port,port=29,chassis=15,id=pci.15,bus=pcie.0,addr=0x3.0x5 \
+-device nec-usb-xhci,id=usb,bus=pci.8,addr=0x0 \
+-device virtio-scsi-pci,id=scsi0,bus=pci.7,addr=0x0 \
+-device virtio-serial-pci,id=virtio-serial0,bus=pci.6,addr=0x0 \
 -blockdev '{"driver":"host_device","filename":"/dev/HostVG/QEMUGuest1","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \
--device virtio-blk-pci,scsi=off,bus=pci.7,addr=0x0,drive=libvirt-1-format,id=virtio-disk1,bootindex=1 \
+-device virtio-blk-pci,scsi=off,bus=pci.9,addr=0x0,drive=libvirt-1-format,id=virtio-disk1,bootindex=1 \
 -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \
--device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,bus=pci.1,addr=0x0 \
+-device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=/import/from/host,bus=pci.3,addr=0x0 \
 -netdev user,id=hostnet0 \
--device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.2,addr=0x0 \
+-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.4,addr=0x0 \
 -netdev user,id=hostnet1 \
--device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.3,addr=0x0 \
--device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.10,addr=0x0 \
--device virtio-mouse-pci,id=input1,bus=pci.11,addr=0x0 \
--device virtio-keyboard-pci,id=input2,bus=pci.12,addr=0x0 \
--device virtio-tablet-pci,id=input3,bus=pci.13,addr=0x0 \
+-device e1000e,netdev=hostnet1,id=net1,mac=00:11:22:33:44:66,bus=pci.5,addr=0x0 \
+-device virtio-input-host-pci,id=input0,evdev=/dev/input/event1234,bus=pci.12,addr=0x0 \
+-device virtio-mouse-pci,id=input1,bus=pci.13,addr=0x0 \
+-device virtio-keyboard-pci,id=input2,bus=pci.14,addr=0x0 \
+-device virtio-tablet-pci,id=input3,bus=pci.15,addr=0x0 \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -device virtio-vga,id=video0,max_outputs=1,bus=pcie.0,addr=0x1 \
--device virtio-balloon-pci,id=balloon0,bus=pci.8,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.10,addr=0x0 \
 -object rng-random,id=objrng0,filename=/dev/urandom \
--device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,bus=pci.9,addr=0x0 \
+-device virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=123,period=1234,bus=pci.11,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-latest.args b/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-latest.args
index 7992c33ace..4dd06294f5 100644
--- a/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/q35-pcie-autoadd.x86_64-latest.args
@@ -28,39 +28,40 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-q35-test/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \
--device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \
--device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \
--device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \
--device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \
--device '{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}' \
--device '{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}' \
--device '{"driver":"pcie-root-port","port":23,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x7"}' \
--device '{"driver":"pcie-root-port","port":24,"chassis":9,"id":"pci.9","bus":"pcie.0","multifunction":true,"addr":"0x3"}' \
--device '{"driver":"pcie-root-port","port":25,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x3.0x1"}' \
--device '{"driver":"pcie-root-port","port":26,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x3.0x2"}' \
--device '{"driver":"pcie-root-port","port":27,"chassis":12,"id":"pci.12","bus":"pcie.0","addr":"0x3.0x3"}' \
--device '{"driver":"pcie-root-port","port":28,"chassis":13,"id":"pci.13","bus":"pcie.0","addr":"0x3.0x4"}' \
--device '{"driver":"pcie-root-port","port":29,"chassis":14,"id":"pci.14","bus":"pcie.0","addr":"0x3.0x5"}' \
--device '{"driver":"nec-usb-xhci","id":"usb","bus":"pci.6","addr":"0x0"}' \
--device '{"driver":"virtio-scsi-pci","id":"scsi0","bus":"pci.5","addr":"0x0"}' \
--device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.4","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":17,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x1"}' \
+-device '{"driver":"pcie-root-port","port":18,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x2"}' \
+-device '{"driver":"pcie-root-port","port":19,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x3"}' \
+-device '{"driver":"pcie-root-port","port":20,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x4"}' \
+-device '{"driver":"pcie-root-port","port":21,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x5"}' \
+-device '{"driver":"pcie-root-port","port":22,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x6"}' \
+-device '{"driver":"pcie-root-port","port":23,"chassis":9,"id":"pci.9","bus":"pcie.0","addr":"0x2.0x7"}' \
+-device '{"driver":"pcie-root-port","port":24,"chassis":10,"id":"pci.10","bus":"pcie.0","multifunction":true,"addr":"0x3"}' \
+-device '{"driver":"pcie-root-port","port":25,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x3.0x1"}' \
+-device '{"driver":"pcie-root-port","port":26,"chassis":12,"id":"pci.12","bus":"pcie.0","addr":"0x3.0x2"}' \
+-device '{"driver":"pcie-root-port","port":27,"chassis":13,"id":"pci.13","bus":"pcie.0","addr":"0x3.0x3"}' \
+-device '{"driver":"pcie-root-port","port":28,"chassis":14,"id":"pci.14","bus":"pcie.0","addr":"0x3.0x4"}' \
+-device '{"driver":"pcie-root-port","port":29,"chassis":15,"id":"pci.15","bus":"pcie.0","addr":"0x3.0x5"}' \
+-device '{"driver":"nec-usb-xhci","id":"usb","bus":"pci.8","addr":"0x0"}' \
+-device '{"driver":"virtio-scsi-pci","id":"scsi0","bus":"pci.7","addr":"0x0"}' \
+-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.6","addr":"0x0"}' \
 -blockdev '{"driver":"host_device","filename":"/dev/HostVG/QEMUGuest1","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \
--device '{"driver":"virtio-blk-pci","bus":"pci.7","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk1","bootindex":1}' \
+-device '{"driver":"virtio-blk-pci","bus":"pci.9","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk1","bootindex":1}' \
 -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/to/guest \
--device '{"driver":"virtio-9p-pci","id":"fs0","fsdev":"fsdev-fs0","mount_tag":"/import/from/host","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"virtio-9p-pci","id":"fs0","fsdev":"fsdev-fs0","mount_tag":"/import/from/host","bus":"pci.3","addr":"0x0"}' \
 -netdev user,id=hostnet0 \
--device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"00:11:22:33:44:55","bus":"pci.2","addr":"0x0"}' \
+-device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"00:11:22:33:44:55","bus":"pci.4","addr":"0x0"}' \
 -netdev user,id=hostnet1 \
--device '{"driver":"e1000e","netdev":"hostnet1","id":"net1","mac":"00:11:22:33:44:66","bus":"pci.3","addr":"0x0"}' \
--device '{"driver":"virtio-input-host-pci","id":"input0","evdev":"/dev/input/event1234","bus":"pci.10","addr":"0x0"}' \
--device '{"driver":"virtio-mouse-pci","id":"input1","bus":"pci.11","addr":"0x0"}' \
--device '{"driver":"virtio-keyboard-pci","id":"input2","bus":"pci.12","addr":"0x0"}' \
--device '{"driver":"virtio-tablet-pci","id":"input3","bus":"pci.13","addr":"0x0"}' \
+-device '{"driver":"e1000e","netdev":"hostnet1","id":"net1","mac":"00:11:22:33:44:66","bus":"pci.5","addr":"0x0"}' \
+-device '{"driver":"virtio-input-host-pci","id":"input0","evdev":"/dev/input/event1234","bus":"pci.12","addr":"0x0"}' \
+-device '{"driver":"virtio-mouse-pci","id":"input1","bus":"pci.13","addr":"0x0"}' \
+-device '{"driver":"virtio-keyboard-pci","id":"input2","bus":"pci.14","addr":"0x0"}' \
+-device '{"driver":"virtio-tablet-pci","id":"input3","bus":"pci.15","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -device '{"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.8","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.10","addr":"0x0"}' \
 -object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
--device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","max-bytes":123,"period":1234,"bus":"pci.9","addr":"0x0"}' \
+-device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","max-bytes":123,"period":1234,"bus":"pci.11","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-4.2.0.args b/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-4.2.0.args
index 14093a3f80..b67790f066 100644
--- a/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-4.2.0.args
+++ b/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-4.2.0.args
@@ -29,18 +29,19 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-virt-manager-basic/.config \
 -global ICH9-LPC.disable_s4=1 \
 -boot strict=on \
 -device pcie-root-port,port=16,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \
--device pcie-root-port,port=17,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
--device pcie-root-port,port=18,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
--device pcie-root-port,port=19,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
--device pcie-root-port,port=20,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
--device pcie-root-port,port=21,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
--device nec-usb-xhci,id=usb,bus=pci.2,addr=0x0 \
--device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 \
+-device pcie-pci-bridge,id=pci.2,bus=pci.1,addr=0x0 \
+-device pcie-root-port,port=17,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x1 \
+-device pcie-root-port,port=18,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x2 \
+-device pcie-root-port,port=19,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x3 \
+-device pcie-root-port,port=20,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x4 \
+-device pcie-root-port,port=21,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x5 \
+-device nec-usb-xhci,id=usb,bus=pci.4,addr=0x0 \
+-device virtio-serial-pci,id=virtio-serial0,bus=pci.5,addr=0x0 \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/basic.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage"}' \
--device virtio-blk-pci,scsi=off,bus=pci.4,addr=0x0,drive=libvirt-1-format,id=virtio-disk0,bootindex=1 \
+-device virtio-blk-pci,scsi=off,bus=pci.6,addr=0x0,drive=libvirt-1-format,id=virtio-disk0,bootindex=1 \
 -netdev user,id=hostnet0 \
--device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:9a:e6:c6,bus=pci.1,addr=0x0 \
+-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:9a:e6:c6,bus=pci.3,addr=0x0 \
 -chardev pty,id=charserial0 \
 -device isa-serial,chardev=charserial0,id=serial0,index=0 \
 -chardev socket,id=charchannel0,fd=1729,server=on,wait=off \
@@ -57,6 +58,6 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-virt-manager-basic/.config \
 -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 \
 -chardev spicevmc,id=charredir1,name=usbredir \
 -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 \
--device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.7,addr=0x0 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-latest.args b/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-latest.args
index 77a7cf5e4f..02f3ef021f 100644
--- a/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/q35-virt-manager-basic.x86_64-latest.args
@@ -30,18 +30,19 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-virt-manager-basic/.config \
 -global ICH9-LPC.disable_s4=1 \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \
--device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \
--device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \
--device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \
--device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \
--device '{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}' \
--device '{"driver":"nec-usb-xhci","id":"usb","bus":"pci.2","addr":"0x0"}' \
--device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":17,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x1"}' \
+-device '{"driver":"pcie-root-port","port":18,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x2"}' \
+-device '{"driver":"pcie-root-port","port":19,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x3"}' \
+-device '{"driver":"pcie-root-port","port":20,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x4"}' \
+-device '{"driver":"pcie-root-port","port":21,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x5"}' \
+-device '{"driver":"nec-usb-xhci","id":"usb","bus":"pci.4","addr":"0x0"}' \
+-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.5","addr":"0x0"}' \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/basic.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage"}' \
--device '{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
+-device '{"driver":"virtio-blk-pci","bus":"pci.6","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
 -netdev user,id=hostnet0 \
--device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:9a:e6:c6","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:9a:e6:c6","bus":"pci.3","addr":"0x0"}' \
 -chardev pty,id=charserial0 \
 -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
 -chardev socket,id=charchannel0,fd=1729,server=on,wait=off \
@@ -58,6 +59,6 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-virt-manager-basic/.config \
 -device '{"driver":"usb-redir","chardev":"charredir0","id":"redir0","bus":"usb.0","port":"2"}' \
 -chardev spicevmc,id=charredir1,name=usbredir \
 -device '{"driver":"usb-redir","chardev":"charredir1","id":"redir1","bus":"usb.0","port":"3"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.7","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/tseg-explicit-size.x86_64-latest.args b/tests/qemuxml2argvdata/tseg-explicit-size.x86_64-latest.args
index 3371199be5..a3b6fab557 100644
--- a/tests/qemuxml2argvdata/tseg-explicit-size.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/tseg-explicit-size.x86_64-latest.args
@@ -29,10 +29,11 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.2","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.4","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/user-aliases2.x86_64-latest.args b/tests/qemuxml2argvdata/user-aliases2.x86_64-latest.args
index 7f34c7a965..75aae3254c 100644
--- a/tests/qemuxml2argvdata/user-aliases2.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/user-aliases2.x86_64-latest.args
@@ -28,8 +28,9 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -device '{"driver":"isa-fdc","bootindexA":1}' \
 -blockdev '{"driver":"file","filename":"/tmp/firmware.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \
diff --git a/tests/qemuxml2argvdata/virtio-iommu-x86_64.x86_64-latest.args b/tests/qemuxml2argvdata/virtio-iommu-x86_64.x86_64-latest.args
index 744367c6c0..0149b63cd5 100644
--- a/tests/qemuxml2argvdata/virtio-iommu-x86_64.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/virtio-iommu-x86_64.x86_64-latest.args
@@ -26,7 +26,9 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -rtc base=utc \
 -no-shutdown \
 -boot strict=on \
--device '{"driver":"virtio-iommu","id":"iommu0","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"virtio-iommu","id":"iommu0","bus":"pcie.0","addr":"0x2"}' \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/virtio-non-transitional.x86_64-latest.args b/tests/qemuxml2argvdata/virtio-non-transitional.x86_64-latest.args
index 541c074871..18f83e6fe4 100644
--- a/tests/qemuxml2argvdata/virtio-non-transitional.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/virtio-non-transitional.x86_64-latest.args
@@ -28,29 +28,30 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-acpi \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
--device '{"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"}' \
--device '{"driver":"pcie-root-port","port":12,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x4"}' \
--device '{"driver":"pcie-root-port","port":13,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x5"}' \
--device '{"driver":"pcie-root-port","port":14,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x6"}' \
--device '{"driver":"pcie-root-port","port":15,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x1.0x7"}' \
--device '{"driver":"pcie-root-port","port":16,"chassis":9,"id":"pci.9","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \
--device '{"driver":"pcie-root-port","port":17,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x2.0x1"}' \
--device '{"driver":"virtio-scsi-pci-non-transitional","id":"scsi0","bus":"pci.4","addr":"0x0"}' \
--device '{"driver":"virtio-serial-pci-non-transitional","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
+-device '{"driver":"pcie-root-port","port":11,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x3"}' \
+-device '{"driver":"pcie-root-port","port":12,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x4"}' \
+-device '{"driver":"pcie-root-port","port":13,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x5"}' \
+-device '{"driver":"pcie-root-port","port":14,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x1.0x6"}' \
+-device '{"driver":"pcie-root-port","port":15,"chassis":9,"id":"pci.9","bus":"pcie.0","addr":"0x1.0x7"}' \
+-device '{"driver":"pcie-root-port","port":16,"chassis":10,"id":"pci.10","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \
+-device '{"driver":"pcie-root-port","port":17,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x2.0x1"}' \
+-device '{"driver":"virtio-scsi-pci-non-transitional","id":"scsi0","bus":"pci.6","addr":"0x0"}' \
+-device '{"driver":"virtio-serial-pci-non-transitional","id":"virtio-serial0","bus":"pci.5","addr":"0x0"}' \
 -blockdev '{"driver":"host_device","filename":"/dev/HostVG/QEMUGuest1","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"raw","file":"libvirt-1-storage"}' \
--device '{"driver":"virtio-blk-pci-non-transitional","bus":"pci.5","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
+-device '{"driver":"virtio-blk-pci-non-transitional","bus":"pci.7","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
 -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/export/fs1 \
--device '{"driver":"virtio-9p-pci-non-transitional","id":"fs0","fsdev":"fsdev-fs0","mount_tag":"fs1","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"virtio-9p-pci-non-transitional","id":"fs0","fsdev":"fsdev-fs0","mount_tag":"fs1","bus":"pci.3","addr":"0x0"}' \
 -netdev user,id=hostnet0 \
--device '{"driver":"virtio-net-pci-non-transitional","netdev":"hostnet0","id":"net0","mac":"00:11:22:33:44:55","bus":"pci.2","addr":"0x0"}' \
+-device '{"driver":"virtio-net-pci-non-transitional","netdev":"hostnet0","id":"net0","mac":"00:11:22:33:44:55","bus":"pci.4","addr":"0x0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"vhost-scsi-pci-non-transitional","wwpn":"naa.5123456789abcde0","vhostfd":"3","id":"hostdev0","bus":"pci.6","addr":"0x0"}' \
--device '{"driver":"virtio-balloon-pci-non-transitional","id":"balloon0","bus":"pci.7","addr":"0x0"}' \
+-device '{"driver":"vhost-scsi-pci-non-transitional","wwpn":"naa.5123456789abcde0","vhostfd":"3","id":"hostdev0","bus":"pci.8","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci-non-transitional","id":"balloon0","bus":"pci.9","addr":"0x0"}' \
 -object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
--device '{"driver":"virtio-rng-pci-non-transitional","rng":"objrng0","id":"rng0","bus":"pci.8","addr":"0x0"}' \
+-device '{"driver":"virtio-rng-pci-non-transitional","rng":"objrng0","id":"rng0","bus":"pci.10","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
--device '{"driver":"vhost-vsock-pci-non-transitional","id":"vsock0","guest-cid":4,"vhostfd":"6789","bus":"pci.9","addr":"0x0"}' \
+-device '{"driver":"vhost-vsock-pci-non-transitional","id":"vsock0","guest-cid":4,"vhostfd":"6789","bus":"pci.11","addr":"0x0"}' \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/virtio-transitional.x86_64-latest.args b/tests/qemuxml2argvdata/virtio-transitional.x86_64-latest.args
index 6d18698ec7..a5730e9059 100644
--- a/tests/qemuxml2argvdata/virtio-transitional.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/virtio-transitional.x86_64-latest.args
@@ -27,9 +27,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
 -no-shutdown \
 -no-acpi \
 -boot strict=on \
--device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
+-device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
 -device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
 -device '{"driver":"virtio-scsi-pci-transitional","id":"scsi0","bus":"pci.2","addr":"0x4"}' \
 -device '{"driver":"virtio-serial-pci-transitional","id":"virtio-serial0","bus":"pci.2","addr":"0x3"}' \
 -blockdev '{"driver":"host_device","filename":"/dev/HostVG/QEMUGuest1","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
diff --git a/tests/qemuxml2argvdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.args b/tests/qemuxml2argvdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.args
index 0368ecc699..b5cf29038f 100644
--- a/tests/qemuxml2argvdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.args
@@ -27,14 +27,15 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-guest/.config \
 -no-shutdown \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
--device '{"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
+-device '{"driver":"pcie-root-port","port":11,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x3"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/guest.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage"}' \
--device '{"driver":"virtio-blk-pci","bus":"pci.2","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
+-device '{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.3","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.args b/tests/qemuxml2argvdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.args
index c2eb8066df..4a3169e2a8 100644
--- a/tests/qemuxml2argvdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.args
@@ -27,14 +27,15 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-guest/.config \
 -no-shutdown \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
--device '{"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"}' \
--device '{"driver":"qemu-xhci","id":"usb","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
+-device '{"driver":"pcie-root-port","port":11,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x3"}' \
+-device '{"driver":"qemu-xhci","id":"usb","bus":"pci.3","addr":"0x0"}' \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/guest.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage"}' \
--device '{"driver":"virtio-blk-pci","bus":"pci.2","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
+-device '{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.3","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/x86_64-q35-graphics.x86_64-latest.args b/tests/qemuxml2argvdata/x86_64-q35-graphics.x86_64-latest.args
index 37a1afb897..31796d9fb6 100644
--- a/tests/qemuxml2argvdata/x86_64-q35-graphics.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/x86_64-q35-graphics.x86_64-latest.args
@@ -30,19 +30,20 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-guest/.config \
 -global ICH9-LPC.disable_s4=1 \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \
--device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \
--device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \
--device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \
--device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \
--device '{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}' \
--device '{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}' \
--device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"}' \
--device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":17,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x1"}' \
+-device '{"driver":"pcie-root-port","port":18,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x2"}' \
+-device '{"driver":"pcie-root-port","port":19,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x3"}' \
+-device '{"driver":"pcie-root-port","port":20,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x4"}' \
+-device '{"driver":"pcie-root-port","port":21,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x5"}' \
+-device '{"driver":"pcie-root-port","port":22,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x6"}' \
+-device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.4","addr":"0x0"}' \
+-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.5","addr":"0x0"}' \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/guest.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage"}' \
--device '{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
+-device '{"driver":"virtio-blk-pci","bus":"pci.6","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
 -netdev user,id=hostnet0 \
--device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:d2:70:0b","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:d2:70:0b","bus":"pci.3","addr":"0x0"}' \
 -chardev pty,id=charserial0 \
 -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
 -chardev socket,id=charchannel0,fd=1729,server=on,wait=off \
@@ -51,8 +52,8 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-guest/.config \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -vnc 127.0.0.1:0,audiodev=audio1 \
 -device '{"driver":"qxl-vga","id":"video0","max_outputs":1,"ram_size":67108864,"vram_size":67108864,"vram64_size_mb":0,"vgamem_mb":16,"bus":"pcie.0","addr":"0x1"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.7","addr":"0x0"}' \
 -object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
--device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.6","addr":"0x0"}' \
+-device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.8","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/x86_64-q35-headless.x86_64-latest.args b/tests/qemuxml2argvdata/x86_64-q35-headless.x86_64-latest.args
index b4d90dff5e..6efcbf7943 100644
--- a/tests/qemuxml2argvdata/x86_64-q35-headless.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/x86_64-q35-headless.x86_64-latest.args
@@ -31,26 +31,27 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-guest/.config \
 -global ICH9-LPC.disable_s4=1 \
 -boot strict=on \
 -device '{"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}' \
--device '{"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"}' \
--device '{"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"}' \
--device '{"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"}' \
--device '{"driver":"pcie-root-port","port":12,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x4"}' \
--device '{"driver":"pcie-root-port","port":13,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x5"}' \
--device '{"driver":"pcie-root-port","port":14,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x6"}' \
--device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"}' \
--device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}' \
+-device '{"driver":"pcie-pci-bridge","id":"pci.2","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"pcie-root-port","port":9,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x1"}' \
+-device '{"driver":"pcie-root-port","port":10,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x2"}' \
+-device '{"driver":"pcie-root-port","port":11,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x3"}' \
+-device '{"driver":"pcie-root-port","port":12,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x4"}' \
+-device '{"driver":"pcie-root-port","port":13,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x5"}' \
+-device '{"driver":"pcie-root-port","port":14,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x1.0x6"}' \
+-device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.4","addr":"0x0"}' \
+-device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.5","addr":"0x0"}' \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/guest.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage"}' \
--device '{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
+-device '{"driver":"virtio-blk-pci","bus":"pci.6","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}' \
 -netdev user,id=hostnet0 \
--device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:09:a4:37","bus":"pci.1","addr":"0x0"}' \
+-device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:09:a4:37","bus":"pci.3","addr":"0x0"}' \
 -chardev pty,id=charserial0 \
 -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \
 -chardev socket,id=charchannel0,fd=1729,server=on,wait=off \
 -device '{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.7","addr":"0x0"}' \
 -object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
--device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.6","addr":"0x0"}' \
+-device '{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.8","addr":"0x0"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-bios-stateless.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-bios-stateless.x86_64-latest.xml
index f1b5516ce4..88d9581502 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-bios-stateless.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-bios-stateless.x86_64-latest.xml
@@ -26,6 +26,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-bios.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-bios.x86_64-latest.xml
index 722294089e..0c789d4af2 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-bios.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-bios.x86_64-latest.xml
@@ -25,6 +25,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-efi-enrolled-keys.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-efi-enrolled-keys.x86_64-latest.xml
index 8dcc741c1e..015fadb74d 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-efi-enrolled-keys.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-efi-enrolled-keys.x86_64-latest.xml
@@ -29,6 +29,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-efi-loader-secure.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-efi-loader-secure.x86_64-latest.xml
index 26e30d7f64..c5674b3da3 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-efi-loader-secure.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-efi-loader-secure.x86_64-latest.xml
@@ -26,6 +26,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.xml
index e5d307e0b2..573f86be71 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-efi-no-enrolled-keys.x86_64-latest.xml
@@ -28,6 +28,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-efi-no-secboot.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-efi-no-secboot.x86_64-latest.xml
index 1528ebcfe0..a9c556c573 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-efi-no-secboot.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-efi-no-secboot.x86_64-latest.xml
@@ -28,6 +28,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-efi-nvram.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-efi-nvram.x86_64-latest.xml
index abd6ec079d..ed24ef11a4 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-efi-nvram.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-efi-nvram.x86_64-latest.xml
@@ -26,6 +26,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-efi-secboot.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-efi-secboot.x86_64-latest.xml
index ef24e04b61..f4c61b67b0 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-efi-secboot.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-efi-secboot.x86_64-latest.xml
@@ -28,6 +28,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/firmware-auto-efi.x86_64-latest.xml b/tests/qemuxml2xmloutdata/firmware-auto-efi.x86_64-latest.xml
index 7e2e40036e..c0a39927aa 100644
--- a/tests/qemuxml2xmloutdata/firmware-auto-efi.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/firmware-auto-efi.x86_64-latest.xml
@@ -25,6 +25,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.xml b/tests/qemuxml2xmloutdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.xml
index 032b4ce030..2e0f52891f 100644
--- a/tests/qemuxml2xmloutdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/hostdev-scsi-vhost-scsi-pcie.x86_64-latest.xml
@@ -27,9 +27,13 @@
       <target chassis='1' port='0x8'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
+    <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='2' port='0x9'/>
+      <target chassis='3' port='0x9'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
     </controller>
     <input type='mouse' bus='ps2'/>
@@ -37,7 +41,7 @@
     <audio id='1' type='none'/>
     <hostdev mode='subsystem' type='scsi_host' managed='no'>
       <source protocol='vhost' wwpn='naa.5123456789abcde0'/>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </hostdev>
     <memballoon model='none'/>
   </devices>
diff --git a/tests/qemuxml2xmloutdata/machine-smm-off.x86_64-latest.xml b/tests/qemuxml2xmloutdata/machine-smm-off.x86_64-latest.xml
index 77b0b39750..bd493f411f 100644
--- a/tests/qemuxml2xmloutdata/machine-smm-off.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/machine-smm-off.x86_64-latest.xml
@@ -25,6 +25,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/machine-smm-on.x86_64-latest.xml b/tests/qemuxml2xmloutdata/machine-smm-on.x86_64-latest.xml
index 7c7df9e06d..e43cc6500f 100644
--- a/tests/qemuxml2xmloutdata/machine-smm-on.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/machine-smm-on.x86_64-latest.xml
@@ -25,6 +25,15 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
diff --git a/tests/qemuxml2xmloutdata/net-isolated-port.x86_64-latest.xml b/tests/qemuxml2xmloutdata/net-isolated-port.x86_64-latest.xml
index b2eb8516b8..7072879f25 100644
--- a/tests/qemuxml2xmloutdata/net-isolated-port.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/net-isolated-port.x86_64-latest.xml
@@ -25,7 +25,7 @@
     </disk>
     <controller type='pci' index='0' model='pcie-root'/>
     <controller type='usb' index='0' model='qemu-xhci'>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
@@ -35,14 +35,18 @@
       <target chassis='1' port='0x10'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x11'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0x12'/>
+      <target chassis='3' port='0x11'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
+    </controller>
+    <controller type='pci' index='4' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='4' port='0x12'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
     </controller>
     <interface type='network'>
@@ -50,7 +54,7 @@
       <source network='default'/>
       <port isolated='yes'/>
       <model type='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </interface>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
diff --git a/tests/qemuxml2xmloutdata/q35-default-devices-only.x86_64-latest.xml b/tests/qemuxml2xmloutdata/q35-default-devices-only.x86_64-latest.xml
index 5540ad1147..386df5920c 100644
--- a/tests/qemuxml2xmloutdata/q35-default-devices-only.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/q35-default-devices-only.x86_64-latest.xml
@@ -18,7 +18,7 @@
   <devices>
     <emulator>/usr/bin/qemu-system-x86_64</emulator>
     <controller type='usb' index='0' model='qemu-xhci'>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
@@ -29,21 +29,25 @@
       <target chassis='1' port='0x8'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x9'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0xa'/>
+      <target chassis='3' port='0x9'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    </controller>
+    <controller type='pci' index='4' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='4' port='0xa'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
     </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </memballoon>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/q35-pcie-autoadd.x86_64-latest.xml b/tests/qemuxml2xmloutdata/q35-pcie-autoadd.x86_64-latest.xml
index 3037902354..725756b8fc 100644
--- a/tests/qemuxml2xmloutdata/q35-pcie-autoadd.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/q35-pcie-autoadd.x86_64-latest.xml
@@ -21,16 +21,16 @@
       <driver name='qemu' type='raw'/>
       <source dev='/dev/HostVG/QEMUGuest1'/>
       <target dev='vdb' bus='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
     </disk>
     <controller type='virtio-serial' index='0'>
-      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
     </controller>
     <controller type='scsi' index='0' model='virtio-scsi'>
-      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
     </controller>
     <controller type='usb' index='0' model='nec-xhci'>
-      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
@@ -41,98 +41,102 @@
       <target chassis='1' port='0x10'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x11'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0x12'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
+      <target chassis='3' port='0x11'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
     </controller>
     <controller type='pci' index='4' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='4' port='0x13'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
+      <target chassis='4' port='0x12'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
     </controller>
     <controller type='pci' index='5' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='5' port='0x14'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
+      <target chassis='5' port='0x13'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
     </controller>
     <controller type='pci' index='6' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='6' port='0x15'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
+      <target chassis='6' port='0x14'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
     </controller>
     <controller type='pci' index='7' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='7' port='0x16'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
+      <target chassis='7' port='0x15'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
     </controller>
     <controller type='pci' index='8' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='8' port='0x17'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
+      <target chassis='8' port='0x16'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
     </controller>
     <controller type='pci' index='9' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='9' port='0x18'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
+      <target chassis='9' port='0x17'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
     </controller>
     <controller type='pci' index='10' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='10' port='0x19'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
+      <target chassis='10' port='0x18'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
     </controller>
     <controller type='pci' index='11' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='11' port='0x1a'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
+      <target chassis='11' port='0x19'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
     </controller>
     <controller type='pci' index='12' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='12' port='0x1b'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
+      <target chassis='12' port='0x1a'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
     </controller>
     <controller type='pci' index='13' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='13' port='0x1c'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
+      <target chassis='13' port='0x1b'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
     </controller>
     <controller type='pci' index='14' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='14' port='0x1d'/>
+      <target chassis='14' port='0x1c'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
+    </controller>
+    <controller type='pci' index='15' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='15' port='0x1d'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x5'/>
     </controller>
     <filesystem type='mount' accessmode='passthrough'>
       <source dir='/export/to/guest'/>
       <target dir='/import/from/host'/>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </filesystem>
     <interface type='user'>
       <mac address='00:11:22:33:44:55'/>
       <model type='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </interface>
     <interface type='user'>
       <mac address='00:11:22:33:44:66'/>
       <model type='e1000e'/>
-      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
     </interface>
     <input type='passthrough' bus='virtio'>
       <source evdev='/dev/input/event1234'/>
-      <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>
     </input>
     <input type='mouse' bus='virtio'>
-      <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/>
     </input>
     <input type='keyboard' bus='virtio'>
-      <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/>
     </input>
     <input type='tablet' bus='virtio'>
-      <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0f' slot='0x00' function='0x0'/>
     </input>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
@@ -142,12 +146,12 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
     </video>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
     </memballoon>
     <rng model='virtio'>
       <rate bytes='123' period='1234'/>
       <backend model='random'>/dev/urandom</backend>
-      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
     </rng>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/q35-virt-manager-basic.x86_64-latest.xml b/tests/qemuxml2xmloutdata/q35-virt-manager-basic.x86_64-latest.xml
index 62289d4800..40e5fdd975 100644
--- a/tests/qemuxml2xmloutdata/q35-virt-manager-basic.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/q35-virt-manager-basic.x86_64-latest.xml
@@ -34,16 +34,16 @@
       <driver name='qemu' type='qcow2'/>
       <source file='/var/lib/libvirt/images/basic.qcow2'/>
       <target dev='vda' bus='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
     </disk>
     <controller type='usb' index='0' model='nec-xhci'>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='virtio-serial' index='0'>
-      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
     <controller type='pci' index='1' model='pcie-root-port'>
@@ -51,35 +51,39 @@
       <target chassis='1' port='0x10'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x11'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0x12'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
+      <target chassis='3' port='0x11'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
     </controller>
     <controller type='pci' index='4' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='4' port='0x13'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
+      <target chassis='4' port='0x12'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
     </controller>
     <controller type='pci' index='5' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='5' port='0x14'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
+      <target chassis='5' port='0x13'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
     </controller>
     <controller type='pci' index='6' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='6' port='0x15'/>
+      <target chassis='6' port='0x14'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
+    </controller>
+    <controller type='pci' index='7' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='7' port='0x15'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
     </controller>
     <interface type='user'>
       <mac address='52:54:00:9a:e6:c6'/>
       <model type='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </interface>
     <serial type='pty'>
       <target type='isa-serial' port='0'>
@@ -121,7 +125,7 @@
       <address type='usb' bus='0' port='3'/>
     </redirdev>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
     </memballoon>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/tseg-explicit-size.x86_64-latest.xml b/tests/qemuxml2xmloutdata/tseg-explicit-size.x86_64-latest.xml
index 78ffe820cf..6339300694 100644
--- a/tests/qemuxml2xmloutdata/tseg-explicit-size.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/tseg-explicit-size.x86_64-latest.xml
@@ -23,7 +23,7 @@
   <devices>
     <emulator>/usr/bin/qemu-system-x86_64</emulator>
     <controller type='usb' index='0' model='qemu-xhci'>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
@@ -34,21 +34,25 @@
       <target chassis='1' port='0x8'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x9'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0xa'/>
+      <target chassis='3' port='0x9'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    </controller>
+    <controller type='pci' index='4' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='4' port='0xa'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
     </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </memballoon>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/virtio-iommu-x86_64.x86_64-latest.xml b/tests/qemuxml2xmloutdata/virtio-iommu-x86_64.x86_64-latest.xml
index ad3a702b0b..e116aa9603 100644
--- a/tests/qemuxml2xmloutdata/virtio-iommu-x86_64.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/virtio-iommu-x86_64.x86_64-latest.xml
@@ -25,12 +25,21 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x8'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
     <memballoon model='none'/>
     <iommu model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
     </iommu>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/virtio-non-transitional.x86_64-latest.xml b/tests/qemuxml2xmloutdata/virtio-non-transitional.x86_64-latest.xml
index 26f6c6d34e..a6f8662f5b 100644
--- a/tests/qemuxml2xmloutdata/virtio-non-transitional.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/virtio-non-transitional.x86_64-latest.xml
@@ -21,13 +21,13 @@
       <driver name='qemu' type='raw'/>
       <source dev='/dev/HostVG/QEMUGuest1'/>
       <target dev='vda' bus='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
     </disk>
     <controller type='virtio-serial' index='0' model='virtio-non-transitional'>
-      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
     </controller>
     <controller type='scsi' index='0' model='virtio-non-transitional'>
-      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
     </controller>
     <controller type='usb' index='0' model='none'/>
     <controller type='sata' index='0'>
@@ -39,78 +39,82 @@
       <target chassis='1' port='0x8'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x9'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0xa'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+      <target chassis='3' port='0x9'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
     </controller>
     <controller type='pci' index='4' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='4' port='0xb'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
+      <target chassis='4' port='0xa'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
     </controller>
     <controller type='pci' index='5' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='5' port='0xc'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
+      <target chassis='5' port='0xb'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
     </controller>
     <controller type='pci' index='6' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='6' port='0xd'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
+      <target chassis='6' port='0xc'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
     </controller>
     <controller type='pci' index='7' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='7' port='0xe'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
+      <target chassis='7' port='0xd'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
     </controller>
     <controller type='pci' index='8' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='8' port='0xf'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
+      <target chassis='8' port='0xe'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
     </controller>
     <controller type='pci' index='9' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='9' port='0x10'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
+      <target chassis='9' port='0xf'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
     </controller>
     <controller type='pci' index='10' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='10' port='0x11'/>
+      <target chassis='10' port='0x10'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
+    </controller>
+    <controller type='pci' index='11' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='11' port='0x11'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
     </controller>
     <filesystem type='mount' accessmode='passthrough' model='virtio-non-transitional'>
       <source dir='/export/fs1'/>
       <target dir='fs1'/>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </filesystem>
     <interface type='user'>
       <mac address='00:11:22:33:44:55'/>
       <model type='virtio-non-transitional'/>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </interface>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
     <hostdev mode='subsystem' type='scsi_host' managed='no' model='virtio-non-transitional'>
       <source protocol='vhost' wwpn='naa.5123456789abcde0'/>
-      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
     </hostdev>
     <memballoon model='virtio-non-transitional'>
-      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
     </memballoon>
     <rng model='virtio-non-transitional'>
       <backend model='random'>/dev/urandom</backend>
-      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
     </rng>
     <vsock model='virtio-non-transitional'>
       <cid auto='no' address='4'/>
-      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
     </vsock>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/virtio-transitional.x86_64-latest.xml b/tests/qemuxml2xmloutdata/virtio-transitional.x86_64-latest.xml
index 8e9ecd8cf3..e920147f16 100644
--- a/tests/qemuxml2xmloutdata/virtio-transitional.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/virtio-transitional.x86_64-latest.xml
@@ -37,17 +37,12 @@
     <controller type='pci' index='1' model='pcie-root-port'>
       <model name='pcie-root-port'/>
       <target chassis='1' port='0x8'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
     </controller>
     <controller type='pci' index='2' model='pcie-to-pci-bridge'>
       <model name='pcie-pci-bridge'/>
       <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
-    <controller type='pci' index='3' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='3' port='0x9'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
-    </controller>
     <filesystem type='mount' accessmode='passthrough' model='virtio-transitional'>
       <source dir='/export/fs1'/>
       <target dir='fs1'/>
diff --git a/tests/qemuxml2xmloutdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.xml b/tests/qemuxml2xmloutdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.xml
index 61b9077597..d014ecdc14 100644
--- a/tests/qemuxml2xmloutdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/x86_64-default-cpu-kvm-q35-4.2.x86_64-latest.xml
@@ -25,10 +25,10 @@
       <driver name='qemu' type='qcow2'/>
       <source file='/var/lib/libvirt/images/guest.qcow2'/>
       <target dev='vda' bus='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </disk>
     <controller type='usb' index='0' model='qemu-xhci'>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
@@ -39,26 +39,30 @@
       <target chassis='1' port='0x8'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x9'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0xa'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+      <target chassis='3' port='0x9'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
     </controller>
     <controller type='pci' index='4' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='4' port='0xb'/>
+      <target chassis='4' port='0xa'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+    </controller>
+    <controller type='pci' index='5' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='5' port='0xb'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
     </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
     </memballoon>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.xml b/tests/qemuxml2xmloutdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.xml
index 04335f6244..e1ea284491 100644
--- a/tests/qemuxml2xmloutdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/x86_64-default-cpu-tcg-q35-4.2.x86_64-latest.xml
@@ -25,10 +25,10 @@
       <driver name='qemu' type='qcow2'/>
       <source file='/var/lib/libvirt/images/guest.qcow2'/>
       <target dev='vda' bus='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </disk>
     <controller type='usb' index='0' model='qemu-xhci'>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
@@ -39,26 +39,30 @@
       <target chassis='1' port='0x8'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x9'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0xa'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+      <target chassis='3' port='0x9'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
     </controller>
     <controller type='pci' index='4' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='4' port='0xb'/>
+      <target chassis='4' port='0xa'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+    </controller>
+    <controller type='pci' index='5' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='5' port='0xb'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
     </controller>
     <input type='mouse' bus='ps2'/>
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
     </memballoon>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/x86_64-q35-graphics.x86_64-latest.xml b/tests/qemuxml2xmloutdata/x86_64-q35-graphics.x86_64-latest.xml
index 02485adb98..589e1f01f9 100644
--- a/tests/qemuxml2xmloutdata/x86_64-q35-graphics.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/x86_64-q35-graphics.x86_64-latest.xml
@@ -38,57 +38,61 @@
       <driver name='qemu' type='qcow2'/>
       <source file='/var/lib/libvirt/images/guest.qcow2'/>
       <target dev='vda' bus='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
     </disk>
     <controller type='usb' index='0' model='qemu-xhci' ports='15'>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
-    <controller type='virtio-serial' index='0'>
-      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
-    </controller>
     <controller type='pci' index='1' model='pcie-root-port'>
       <model name='pcie-root-port'/>
       <target chassis='1' port='0x10'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x11'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
+    <controller type='virtio-serial' index='0'>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0x12'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
+      <target chassis='3' port='0x11'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
     </controller>
     <controller type='pci' index='4' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='4' port='0x13'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
+      <target chassis='4' port='0x12'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
     </controller>
     <controller type='pci' index='5' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='5' port='0x14'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
+      <target chassis='5' port='0x13'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
     </controller>
     <controller type='pci' index='6' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='6' port='0x15'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
+      <target chassis='6' port='0x14'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
     </controller>
     <controller type='pci' index='7' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='7' port='0x16'/>
+      <target chassis='7' port='0x15'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
+    </controller>
+    <controller type='pci' index='8' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='8' port='0x16'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
     </controller>
     <interface type='user'>
       <mac address='52:54:00:d2:70:0b'/>
       <model type='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </interface>
     <serial type='pty'>
       <target type='isa-serial' port='0'>
@@ -114,11 +118,11 @@
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
     </video>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
     </memballoon>
     <rng model='virtio'>
       <backend model='random'>/dev/urandom</backend>
-      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
     </rng>
   </devices>
 </domain>
diff --git a/tests/qemuxml2xmloutdata/x86_64-q35-headless.x86_64-latest.xml b/tests/qemuxml2xmloutdata/x86_64-q35-headless.x86_64-latest.xml
index 0eedb748dd..8f7eec34f3 100644
--- a/tests/qemuxml2xmloutdata/x86_64-q35-headless.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/x86_64-q35-headless.x86_64-latest.xml
@@ -38,57 +38,61 @@
       <driver name='qemu' type='qcow2'/>
       <source file='/var/lib/libvirt/images/guest.qcow2'/>
       <target dev='vda' bus='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
     </disk>
     <controller type='usb' index='0' model='qemu-xhci' ports='15'>
-      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
     </controller>
     <controller type='sata' index='0'>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
     </controller>
     <controller type='pci' index='0' model='pcie-root'/>
-    <controller type='virtio-serial' index='0'>
-      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
-    </controller>
     <controller type='pci' index='1' model='pcie-root-port'>
       <model name='pcie-root-port'/>
       <target chassis='1' port='0x8'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
     </controller>
-    <controller type='pci' index='2' model='pcie-root-port'>
-      <model name='pcie-root-port'/>
-      <target chassis='2' port='0x9'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    <controller type='pci' index='2' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
+    <controller type='virtio-serial' index='0'>
+      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
     </controller>
     <controller type='pci' index='3' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='3' port='0xa'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+      <target chassis='3' port='0x9'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
     </controller>
     <controller type='pci' index='4' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='4' port='0xb'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
+      <target chassis='4' port='0xa'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
     </controller>
     <controller type='pci' index='5' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='5' port='0xc'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
+      <target chassis='5' port='0xb'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
     </controller>
     <controller type='pci' index='6' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='6' port='0xd'/>
-      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
+      <target chassis='6' port='0xc'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
     </controller>
     <controller type='pci' index='7' model='pcie-root-port'>
       <model name='pcie-root-port'/>
-      <target chassis='7' port='0xe'/>
+      <target chassis='7' port='0xd'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
+    </controller>
+    <controller type='pci' index='8' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='8' port='0xe'/>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
     </controller>
     <interface type='user'>
       <mac address='52:54:00:09:a4:37'/>
       <model type='virtio'/>
-      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
     </interface>
     <serial type='pty'>
       <target type='isa-serial' port='0'>
@@ -106,11 +110,11 @@
     <input type='keyboard' bus='ps2'/>
     <audio id='1' type='none'/>
     <memballoon model='virtio'>
-      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
     </memballoon>
     <rng model='virtio'>
       <backend model='random'>/dev/urandom</backend>
-      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
+      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
     </rng>
   </devices>
 </domain>
-- 
2.38.1
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Peter Krempa 1 year, 4 months ago
On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
> Hotplugging PCI devices on pc-i440fx machines is supported without
> additional configuration. On q35, pcie-to-pci-bridge needs to be added
> prior to the machine startup in order to support hotplug [1].
> 
> The idea is to support the same workflow for creating VMs in q35, as was
> in pc. Namely, do not require additional configuration when hotplugging
> is needed. Otherwise, all libvirt clients need to be updated which there
> are a lot and they are maintained by different parties.
> 
> Instead, a pcie-to-pci-bridge better be created by default, so that PCI
> slots are readily available. Might be a good idea to make it configurable
> in the future.
> 
> Previously there was a pci-bridge present by default, but was removed in
>   d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
> 
> [1]: https://libvirt.org/pci-hotplug.html
> 
> Signed-off-by: Oleg Vasilev <oleg.vasilev@virtuozzo.com>
> ---

[...]

>  73 files changed, 621 insertions(+), 361 deletions(-)
> 
> diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
> index 9ef6c8bb64..52a393e07f 100644
> --- a/src/qemu/qemu_domain.c
> +++ b/src/qemu/qemu_domain.c
> @@ -3890,6 +3890,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>      bool addImplicitSATA = false;
>      bool addPCIRoot = false;
>      bool addPCIeRoot = false;
> +    bool addPCIeToPCIBridge = false;
>      bool addDefaultMemballoon = true;
>      bool addDefaultUSBKBD = false;
>      bool addDefaultUSBMouse = false;
> @@ -3910,6 +3911,8 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>          if (qemuDomainIsQ35(def)) {
>              addPCIeRoot = true;
>              addImplicitSATA = true;
> +            if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PCIE_PCI_BRIDGE))
> +                addPCIeToPCIBridge = true;
>  
>              /* Prefer adding a USB3 controller if supported, fall back
>               * to USB2 if there is no USB3 available, and if that's
> @@ -4046,11 +4049,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>          }
>      }
>  
> -    /* When a machine has a pcie-root, make sure that there is always
> -     * a dmi-to-pci-bridge controller added as bus 1, and a pci-bridge
> -     * as bus 2, so that standard PCI devices can be connected
> -     *
> -     * NB: any machine that sets addPCIeRoot to true must also return
> +    /* NB: any machine that sets addPCIeRoot to true must also return
>       * true from the function qemuDomainSupportsPCI().
>       */
>      if (addPCIeRoot) {
> @@ -4069,6 +4068,16 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>          }
>      }
>  
> +    /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
> +    if (addPCIeToPCIBridge) {
> +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
> +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
> +            return -1;
> +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
> +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
> +            return -1;
> +    }

Doing this unconditionally breaks ABI of the VM, as new guest visible
devices appear without any change in configuration. What's worse this
implementation would also break migration compatibility from older
versions of libvirt as there isn't any form of explicit handling for it.

I unfortunately don't see any reasonable solution originating in
libvirt. The bridge will need to be added by users or higher level
management tools (e.g. virt-install) as libvirt explicitly wants to
support use cases when VMs are started without persisting the XML
definition (e.g. always re-define the XML before start or use the
'createXML' api).
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Andrea Bolognani 1 year, 4 months ago
On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
> > Hotplugging PCI devices on pc-i440fx machines is supported without
> > additional configuration. On q35, pcie-to-pci-bridge needs to be added
> > prior to the machine startup in order to support hotplug [1].
> >
> > The idea is to support the same workflow for creating VMs in q35, as was
> > in pc. Namely, do not require additional configuration when hotplugging
> > is needed. Otherwise, all libvirt clients need to be updated which there
> > are a lot and they are maintained by different parties.
> >
> > Instead, a pcie-to-pci-bridge better be created by default, so that PCI
> > slots are readily available. Might be a good idea to make it configurable
> > in the future.
> >
> > Previously there was a pci-bridge present by default, but was removed in
> >   d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
> >
> > [1]: https://libvirt.org/pci-hotplug.html
> >
> > +    /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
> > +    if (addPCIeToPCIBridge) {
> > +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
> > +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
> > +            return -1;
> > +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
> > +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
> > +            return -1;
> > +    }
>
> Doing this unconditionally breaks ABI of the VM, as new guest visible
> devices appear without any change in configuration. What's worse this
> implementation would also break migration compatibility from older
> versions of libvirt as there isn't any form of explicit handling for it.
>
> I unfortunately don't see any reasonable solution originating in
> libvirt. The bridge will need to be added by users or higher level
> management tools (e.g. virt-install) as libvirt explicitly wants to
> support use cases when VMs are started without persisting the XML
> definition (e.g. always re-define the XML before start or use the
> 'createXML' api).

Additionally, after this change it would be impossible to create a
q35 VM *without* the additional bridge. Most users of the q35 machine
type are likely using PCIe devices exclusively, and such a change
would negatively impact them.

In the (hopefully rare) cases in which hotplugging traditional PCI
devices is actually required, enabling that at the time when the
domain is defined is already straightforward: just include the
relevant controller in the XML.

Unfortunately q35, unlike i440fx, doesn't support hotplugging of any
kind out of the box, so it's up to the user / management application
to ensure that the necessary controllers are added. This is not
ideal, but there's no way around it, and it's still preferable to
forcing unavoidable, potentially useless extra controllers on
everybody.

-- 
Andrea Bolognani / Red Hat / Virtualization
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Oleg Vasilev 1 year, 4 months ago

On 04.11.2022 20:19, Andrea Bolognani wrote:
> On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
>> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
>>> Hotplugging PCI devices on pc-i440fx machines is supported without
>>> additional configuration. On q35, pcie-to-pci-bridge needs to be added
>>> prior to the machine startup in order to support hotplug [1].
>>>
>>> The idea is to support the same workflow for creating VMs in q35, as was
>>> in pc. Namely, do not require additional configuration when hotplugging
>>> is needed. Otherwise, all libvirt clients need to be updated which there
>>> are a lot and they are maintained by different parties.
>>>
>>> Instead, a pcie-to-pci-bridge better be created by default, so that PCI
>>> slots are readily available. Might be a good idea to make it configurable
>>> in the future.
>>>
>>> Previously there was a pci-bridge present by default, but was removed in
>>>    d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
>>>
>>> [1]: https://libvirt.org/pci-hotplug.html
>>>
>>> +    /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
>>> +    if (addPCIeToPCIBridge) {
>>> +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
>>> +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
>>> +            return -1;
>>> +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
>>> +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
>>> +            return -1;
>>> +    }
>>
>> Doing this unconditionally breaks ABI of the VM, as new guest visible
>> devices appear without any change in configuration. What's worse this
>> implementation would also break migration compatibility from older
>> versions of libvirt as there isn't any form of explicit handling for it.
>>
>> I unfortunately don't see any reasonable solution originating in
>> libvirt. The bridge will need to be added by users or higher level
>> management tools (e.g. virt-install) as libvirt explicitly wants to
>> support use cases when VMs are started without persisting the XML
>> definition (e.g. always re-define the XML before start or use the
>> 'createXML' api).
> 
> Additionally, after this change it would be impossible to create a
> q35 VM *without* the additional bridge. Most users of the q35 machine
> type are likely using PCIe devices exclusively, and such a change
> would negatively impact them.
> 
> In the (hopefully rare) cases in which hotplugging traditional PCI
> devices is actually required, enabling that at the time when the
> domain is defined is already straightforward: just include the
> relevant controller in the XML.
> 
> Unfortunately q35, unlike i440fx, doesn't support hotplugging of any
> kind out of the box, so it's up to the user / management application
> to ensure that the necessary controllers are added. This is not
> ideal, but there's no way around it, and it's still preferable to
> forcing unavoidable, potentially useless extra controllers on
> everybody.

Apparently, there is a way to hotplug pcie-to-pci-bridge described in 
qemu doc [1]. Do you think this approach with bus-reserve=1 would be 
acceptable to support in libvirt by default?

[1]: https://github.com/qemu/qemu/blob/master/docs/pcie_pci_bridge.txt#L25

>
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Andrea Bolognani 1 year, 4 months ago
On Fri, Nov 04, 2022 at 08:23:52PM +0600, Oleg Vasilev wrote:
> On 04.11.2022 20:19, Andrea Bolognani wrote:
> > Additionally, after this change it would be impossible to create a
> > q35 VM *without* the additional bridge. Most users of the q35 machine
> > type are likely using PCIe devices exclusively, and such a change
> > would negatively impact them.
> >
> > In the (hopefully rare) cases in which hotplugging traditional PCI
> > devices is actually required, enabling that at the time when the
> > domain is defined is already straightforward: just include the
> > relevant controller in the XML.
> >
> > Unfortunately q35, unlike i440fx, doesn't support hotplugging of any
> > kind out of the box, so it's up to the user / management application
> > to ensure that the necessary controllers are added. This is not
> > ideal, but there's no way around it, and it's still preferable to
> > forcing unavoidable, potentially useless extra controllers on
> > everybody.
>
> Apparently, there is a way to hotplug pcie-to-pci-bridge described in qemu
> doc [1]. Do you think this approach with bus-reserve=1 would be acceptable
> to support in libvirt by default?
>
> [1]: https://github.com/qemu/qemu/blob/master/docs/pcie_pci_bridge.txt#L25

I don't think so.

First of all, the document says that the feature is only supported by
SeaBIOS, whereas most q35 guests are likely using UEFI firmware.

Moreover, we wouldn't be able to just set bus_reserve=1 for all
pcie-root-ports, as that would result in a massive waste of PCI bus
numbers. So we'd have to select a single pcie-root-port and make sure
we don't hotplug anything but a pcie-to-pci-bridge in there. I can't
imagine a way to implement this that's not going to be both
incredibly complicated and extremely fragile.

At the end of the day, q35 just requires to plan the hotplug
capabilities of the VM ahead of time, and this is true both for PCIe
and traditional PCI hotplug. At the libvirt level, I don't really see
a way around that.

Higher-level tools are of course free to offer a more opinionated out
of the box experience. I believe both virt-manager and OpenStack nova
allocate a few pcie-root-ports by default to ensure PCIe hotplug can
work. Perhaps they'd be okay adding a pcie-to-pci-bridge by default
too, making the experience for end users a less frustrating one.

-- 
Andrea Bolognani / Red Hat / Virtualization
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Laine Stump 1 year, 4 months ago
On 11/4/22 11:05 AM, Andrea Bolognani wrote:
> On Fri, Nov 04, 2022 at 08:23:52PM +0600, Oleg Vasilev wrote:
>> On 04.11.2022 20:19, Andrea Bolognani wrote:
>>> Additionally, after this change it would be impossible to create a
>>> q35 VM *without* the additional bridge. Most users of the q35 machine
>>> type are likely using PCIe devices exclusively, and such a change
>>> would negatively impact them.
>>>
>>> In the (hopefully rare) cases in which hotplugging traditional PCI
>>> devices is actually required, enabling that at the time when the
>>> domain is defined is already straightforward: just include the
>>> relevant controller in the XML.
>>>
>>> Unfortunately q35, unlike i440fx, doesn't support hotplugging of any
>>> kind out of the box, so it's up to the user / management application
>>> to ensure that the necessary controllers are added. This is not
>>> ideal, but there's no way around it, and it's still preferable to
>>> forcing unavoidable, potentially useless extra controllers on
>>> everybody.
>>
>> Apparently, there is a way to hotplug pcie-to-pci-bridge described in qemu
>> doc [1]. Do you think this approach with bus-reserve=1 would be acceptable
>> to support in libvirt by default?
>>
>> [1]: https://github.com/qemu/qemu/blob/master/docs/pcie_pci_bridge.txt#L25
> 
> I don't think so.
> 
> First of all, the document says that the feature is only supported by
> SeaBIOS, whereas most q35 guests are likely using UEFI firmware.
> 
> Moreover, we wouldn't be able to just set bus_reserve=1 for all
> pcie-root-ports, as that would result in a massive waste of PCI bus
> numbers. So we'd have to select a single pcie-root-port and make sure
> we don't hotplug anything but a pcie-to-pci-bridge in there. I can't
> imagine a way to implement this that's not going to be both
> incredibly complicated and extremely fragile.

Yes, my recollection of the description of how to *potentially* support 
hotplug of a pcie-to-pci-bridge into a running VM (as explained by 
Marcel, back when he was working on this stuff) is that 1) it would take 
a lot of work (and hand waving) in the guest OS as well as in qemu *and* 
in libvirt, and 2) it would be fragile, and likely not work in exactly 
the scenarios where people wanted it to work. In the end, the return on 
investment just seemed incredibly low.

> 
> At the end of the day, q35 just requires to plan the hotplug
> capabilities of the VM ahead of time, and this is true both for PCIe
> and traditional PCI hotplug. At the libvirt level, I don't really see
> a way around that.
> 
> Higher-level tools are of course free to offer a more opinionated out
> of the box experience. I believe both virt-manager and OpenStack nova
> allocate a few pcie-root-ports by default to ensure PCIe hotplug can
> work. Perhaps they'd be okay adding a pcie-to-pci-bridge by default
> too, making the experience for end users a less frustrating one.
> 

Don't ever let Marcel hear that! :-)

Seriously though, there is almost 0 reason to need a conventional PCI 
device under *any* circumstances, much less needing to hotplug one. If 
you pick the correct model of device, all your devices will be PCIe, and 
in that case having a pcie-to-pci-bridge hanging around in every single 
domain definition will just be wasting resources for no reason at all 
(not even a *bad* reason).
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Daniel P. Berrangé 1 year, 4 months ago
On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
> > Hotplugging PCI devices on pc-i440fx machines is supported without
> > additional configuration. On q35, pcie-to-pci-bridge needs to be added
> > prior to the machine startup in order to support hotplug [1].
> > 
> > The idea is to support the same workflow for creating VMs in q35, as was
> > in pc. Namely, do not require additional configuration when hotplugging
> > is needed. Otherwise, all libvirt clients need to be updated which there
> > are a lot and they are maintained by different parties.
> > 
> > Instead, a pcie-to-pci-bridge better be created by default, so that PCI
> > slots are readily available. Might be a good idea to make it configurable
> > in the future.

The goal of q35 is to provide a PCI-e topology though, and we want
devices to be exposed as PCI-e endpoints, with no legacy PCI usage
by default.

When a guest XML is provided for cold boot, libvirt will ensure
that all devices are placed onto the PCI-e bus via pcie-root-ports

With this proposal hotplugging a device will cause it to be placed
on a PCI bus instead, if there are no spare pcie-root-port available.

This behavioural difference between cold plug and hot plug is very
undesirable IMHO, as it is counter to our goal of avoiding legacy
PCI on PCI-e machines.

If a mgmt app wants to enable this mis-matched behaviour they can
add a pcie-to-pci-bridge when initially cold booting the VM, but
we shouldn't add one by default IMHO. We really recommend that
apps just pre-populate 10-30 pcie-root-port devices when creating
a VM, so there is plenty of space for hotplugging devices later.

> > Previously there was a pci-bridge present by default, but was removed in
> >   d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)



With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Denis V. Lunev 1 year, 4 months ago
On 11/4/22 14:41, Daniel P. Berrangé wrote:
> On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
>> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
>>> Hotplugging PCI devices on pc-i440fx machines is supported without
>>> additional configuration. On q35, pcie-to-pci-bridge needs to be added
>>> prior to the machine startup in order to support hotplug [1].
>>>
>>> The idea is to support the same workflow for creating VMs in q35, as was
>>> in pc. Namely, do not require additional configuration when hotplugging
>>> is needed. Otherwise, all libvirt clients need to be updated which there
>>> are a lot and they are maintained by different parties.
>>>
>>> Instead, a pcie-to-pci-bridge better be created by default, so that PCI
>>> slots are readily available. Might be a good idea to make it configurable
>>> in the future.
> The goal of q35 is to provide a PCI-e topology though, and we want
> devices to be exposed as PCI-e endpoints, with no legacy PCI usage
> by default.
>
> When a guest XML is provided for cold boot, libvirt will ensure
> that all devices are placed onto the PCI-e bus via pcie-root-ports
>
> With this proposal hotplugging a device will cause it to be placed
> on a PCI bus instead, if there are no spare pcie-root-port available.
>
> This behavioural difference between cold plug and hot plug is very
> undesirable IMHO, as it is counter to our goal of avoiding legacy
> PCI on PCI-e machines.

that is an interesting point which I have initially missed. I have
checked our investigation log and found that we have
potentially made a mistake with making compatibility
check with e1000 device instead of virtio, which is
our default. This could be the reason.

>
> If a mgmt app wants to enable this mis-matched behaviour they can
> add a pcie-to-pci-bridge when initially cold booting the VM, but
> we shouldn't add one by default IMHO. We really recommend that
> apps just pre-populate 10-30 pcie-root-port devices when creating
> a VM, so there is plenty of space for hotplugging devices later.

this potentially an option, but I think that these port should be
created automatically by libvirt to avoid major client rewrites.
Will the option to keep the amount of spare ports in /etc/libvirt/qemu.conf
work for you?

Den

>>> Previously there was a pci-bridge present by default, but was removed in
>>>    d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
>
>
> With regards,
> Daniel
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Laine Stump 1 year, 4 months ago
On 11/4/22 10:01 AM, Denis V. Lunev wrote:
> On 11/4/22 14:41, Daniel P. Berrangé wrote:
>> On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
>>> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
>>>> Hotplugging PCI devices on pc-i440fx machines is supported without
>>>> additional configuration. On q35, pcie-to-pci-bridge needs to be added
>>>> prior to the machine startup in order to support hotplug [1].
>>>>
>>>> The idea is to support the same workflow for creating VMs in q35, as 
>>>> was
>>>> in pc. Namely, do not require additional configuration when hotplugging
>>>> is needed. Otherwise, all libvirt clients need to be updated which 
>>>> there
>>>> are a lot and they are maintained by different parties.
>>>>
>>>> Instead, a pcie-to-pci-bridge better be created by default, so that PCI
>>>> slots are readily available. Might be a good idea to make it 
>>>> configurable
>>>> in the future.
>> The goal of q35 is to provide a PCI-e topology though, and we want
>> devices to be exposed as PCI-e endpoints, with no legacy PCI usage
>> by default.
>>
>> When a guest XML is provided for cold boot, libvirt will ensure
>> that all devices are placed onto the PCI-e bus via pcie-root-ports
>>
>> With this proposal hotplugging a device will cause it to be placed
>> on a PCI bus instead, if there are no spare pcie-root-port available.

Actually if the device being hotplugged is a PCIe device (e.g. a 
virtio-net device, or e1000e) what will instead happen is that libvirt 
will attempt to find an empty pcie-root-port and place the device there 
if one is found. If it can't find any empty pcie-root-ports (or 
pcie-downstream-ports) the hotplug will fail because there are no 
available PCIe slots - libvirt will never auto-assign a PCIe device to a 
conventional PCI slot, or vice-versa - it will only auto-assign a PCIe 
device to a PCIe slot (e.g. a a pcie-root-port), and will only 
auto-assign a conventional PCI device to a conventional PCI address 
(e.g. a slot on a pcie-to-pci-bridge). If you want it to do anything 
else, you need to supply explicit PCI addresses.



>>
>> This behavioural difference between cold plug and hot plug is very
>> undesirable IMHO, as it is counter to our goal of avoiding legacy
>> PCI on PCI-e machines.
> 
> that is an interesting point which I have initially missed. I have
> checked our investigation log and found that we have
> potentially made a mistake with making compatibility
> check with e1000 device instead of virtio, which is
> our default. This could be the reason.

Yes, with the default PCI bus topology created by libvirt, you would be 
unable to hotplug even one e1000 device, because the e1000 is a 
conventional PCI device, *not* PCIe, and by default libvirt doesn't add 
*any* conventional PCI slots to a domain based on Q35. If you must use 
an emulated device based on real hardware instead of virtio, always use 
the e1000e device (which is PCIe) for Q35 machinetypes, not e1000 (which 
is conventional PCI). If your test had specified e1000e, then you should 
have been able to hotplug one device, as libvirt makes sure there is a 
single empty pcie-root-port in any domain definition where it needs to 
auto-construct the PCI topology.

You really should avoid *any* emulated device that is conventional PCI 
only in a Q35 vm; fortunately I think the only type of device that 
doesn't have a PCIe flavor is the watchdog timers (I could be 
remembering incorrectly, but I think that was the one).

Also, I don't think anyone has mentioned it so far anywhere in this 
thread, but I just wanted to point out that if you plug a PCIe device 
into a conventional PCI slot (such as those provided on a 
pci-to-pcie-bridge) then the extended PCIe config/capabilities will be 
inaccessible for that device, which could impact performance - I recall 
one issue where the bus width/speed/something-like-that of a device was 
inaccessible, leading to problems because either the guest couldn't 
determine the value of the setting, or because it couldn't change it (I 
forget which, and the exact answer isn't important enough to spend an 
hour on mailing-list forensics to find it :-))

> this potentially an option, but I think that these port should be
> created automatically by libvirt to avoid major client rewrites.

As a sort of compromise, libvirt will always attempt to have one empty 
pcie-root-port when it's building the PCI controller topology. Anything 
beyond this requires intervention by the user/management application. If 
you want "lots" of open ports for hotplug, the simplest remedy is to 
include several <controller type='pci' model='pcie-root-port'/> in your 
initial definition.

Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Daniel P. Berrangé 1 year, 4 months ago
On Fri, Nov 04, 2022 at 11:51:47AM -0400, Laine Stump wrote:
> You really should avoid *any* emulated device that is conventional PCI only
> in a Q35 vm; fortunately I think the only type of device that doesn't have a
> PCIe flavor is the watchdog timers (I could be remembering incorrectly, but
> I think that was the one).

And there shouldn';t be any need to add watchdogs, since Q35 has a builtin
iTCO watchdog in the chipset... just missing the libvirt bits to configure
the action which is being worked on.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Daniel P. Berrangé 1 year, 4 months ago
On Fri, Nov 04, 2022 at 03:01:38PM +0100, Denis V. Lunev wrote:
> On 11/4/22 14:41, Daniel P. Berrangé wrote:
> > On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
> > > On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
> > > > Hotplugging PCI devices on pc-i440fx machines is supported without
> > > > additional configuration. On q35, pcie-to-pci-bridge needs to be added
> > > > prior to the machine startup in order to support hotplug [1].
> > > > 
> > > > The idea is to support the same workflow for creating VMs in q35, as was
> > > > in pc. Namely, do not require additional configuration when hotplugging
> > > > is needed. Otherwise, all libvirt clients need to be updated which there
> > > > are a lot and they are maintained by different parties.
> > > > 
> > > > Instead, a pcie-to-pci-bridge better be created by default, so that PCI
> > > > slots are readily available. Might be a good idea to make it configurable
> > > > in the future.
> > The goal of q35 is to provide a PCI-e topology though, and we want
> > devices to be exposed as PCI-e endpoints, with no legacy PCI usage
> > by default.
> > 
> > When a guest XML is provided for cold boot, libvirt will ensure
> > that all devices are placed onto the PCI-e bus via pcie-root-ports
> > 
> > With this proposal hotplugging a device will cause it to be placed
> > on a PCI bus instead, if there are no spare pcie-root-port available.
> > 
> > This behavioural difference between cold plug and hot plug is very
> > undesirable IMHO, as it is counter to our goal of avoiding legacy
> > PCI on PCI-e machines.
> 
> that is an interesting point which I have initially missed. I have
> checked our investigation log and found that we have
> potentially made a mistake with making compatibility
> check with e1000 device instead of virtio, which is
> our default. This could be the reason.

virtio devices are particularly sensitive to PCI vs PCI-e placement,
because if placed on PCI, they get setup in transitional-mode, where
they advertize the legacy PCI IDs and virtio 0.9/1.0 is negotiated,
while if placed on PCI-E, they get unconditionally setup in modern
mode with the modern PCI IDs and virtio 1.0 only.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Denis V. Lunev 1 year, 4 months ago
On 11/4/22 15:11, Daniel P. Berrangé wrote:
> On Fri, Nov 04, 2022 at 03:01:38PM +0100, Denis V. Lunev wrote:
>> On 11/4/22 14:41, Daniel P. Berrangé wrote:
>>> On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
>>>> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
>>>>> Hotplugging PCI devices on pc-i440fx machines is supported without
>>>>> additional configuration. On q35, pcie-to-pci-bridge needs to be added
>>>>> prior to the machine startup in order to support hotplug [1].
>>>>>
>>>>> The idea is to support the same workflow for creating VMs in q35, as was
>>>>> in pc. Namely, do not require additional configuration when hotplugging
>>>>> is needed. Otherwise, all libvirt clients need to be updated which there
>>>>> are a lot and they are maintained by different parties.
>>>>>
>>>>> Instead, a pcie-to-pci-bridge better be created by default, so that PCI
>>>>> slots are readily available. Might be a good idea to make it configurable
>>>>> in the future.
>>> The goal of q35 is to provide a PCI-e topology though, and we want
>>> devices to be exposed as PCI-e endpoints, with no legacy PCI usage
>>> by default.
>>>
>>> When a guest XML is provided for cold boot, libvirt will ensure
>>> that all devices are placed onto the PCI-e bus via pcie-root-ports
>>>
>>> With this proposal hotplugging a device will cause it to be placed
>>> on a PCI bus instead, if there are no spare pcie-root-port available.
>>>
>>> This behavioural difference between cold plug and hot plug is very
>>> undesirable IMHO, as it is counter to our goal of avoiding legacy
>>> PCI on PCI-e machines.
>> that is an interesting point which I have initially missed. I have
>> checked our investigation log and found that we have
>> potentially made a mistake with making compatibility
>> check with e1000 device instead of virtio, which is
>> our default. This could be the reason.
> virtio devices are particularly sensitive to PCI vs PCI-e placement,
> because if placed on PCI, they get setup in transitional-mode, where
> they advertize the legacy PCI IDs and virtio 0.9/1.0 is negotiated,
> while if placed on PCI-E, they get unconditionally setup in modern
> mode with the modern PCI IDs and virtio 1.0 only.
Thanks for a note!
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Daniel P. Berrangé 1 year, 4 months ago
On Fri, Nov 04, 2022 at 03:01:38PM +0100, Denis V. Lunev wrote:
> On 11/4/22 14:41, Daniel P. Berrangé wrote:
> > On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
> > > On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
> > > > Hotplugging PCI devices on pc-i440fx machines is supported without
> > > > additional configuration. On q35, pcie-to-pci-bridge needs to be added
> > > > prior to the machine startup in order to support hotplug [1].
> > > > 
> > > > The idea is to support the same workflow for creating VMs in q35, as was
> > > > in pc. Namely, do not require additional configuration when hotplugging
> > > > is needed. Otherwise, all libvirt clients need to be updated which there
> > > > are a lot and they are maintained by different parties.
> > > > 
> > > > Instead, a pcie-to-pci-bridge better be created by default, so that PCI
> > > > slots are readily available. Might be a good idea to make it configurable
> > > > in the future.
> > The goal of q35 is to provide a PCI-e topology though, and we want
> > devices to be exposed as PCI-e endpoints, with no legacy PCI usage
> > by default.
> > 
> > When a guest XML is provided for cold boot, libvirt will ensure
> > that all devices are placed onto the PCI-e bus via pcie-root-ports
> > 
> > With this proposal hotplugging a device will cause it to be placed
> > on a PCI bus instead, if there are no spare pcie-root-port available.
> > 
> > This behavioural difference between cold plug and hot plug is very
> > undesirable IMHO, as it is counter to our goal of avoiding legacy
> > PCI on PCI-e machines.
> 
> that is an interesting point which I have initially missed. I have
> checked our investigation log and found that we have
> potentially made a mistake with making compatibility
> check with e1000 device instead of virtio, which is
> our default. This could be the reason.
> 
> > 
> > If a mgmt app wants to enable this mis-matched behaviour they can
> > add a pcie-to-pci-bridge when initially cold booting the VM, but
> > we shouldn't add one by default IMHO. We really recommend that
> > apps just pre-populate 10-30 pcie-root-port devices when creating
> > a VM, so there is plenty of space for hotplugging devices later.
> 
> this potentially an option, but I think that these port should be
> created automatically by libvirt to avoid major client rewrites.
> Will the option to keep the amount of spare ports in /etc/libvirt/qemu.conf
> work for you?

Our general view is that qemu.conf should not cause changes that
affect the guest ABI.

For an given input XML, we want applications to have a predictable
guest ABI created. Influencing the guest ABI from qemu.conf settings
would mean applications have uncertain results for a given XML config,
so what works on one host may break on another host.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Denis V. Lunev 1 year, 4 months ago
On 11/4/22 13:03, Peter Krempa wrote:
> On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
>> Hotplugging PCI devices on pc-i440fx machines is supported without
>> additional configuration. On q35, pcie-to-pci-bridge needs to be added
>> prior to the machine startup in order to support hotplug [1].
>>
>> The idea is to support the same workflow for creating VMs in q35, as was
>> in pc. Namely, do not require additional configuration when hotplugging
>> is needed. Otherwise, all libvirt clients need to be updated which there
>> are a lot and they are maintained by different parties.
>>
>> Instead, a pcie-to-pci-bridge better be created by default, so that PCI
>> slots are readily available. Might be a good idea to make it configurable
>> in the future.
>>
>> Previously there was a pci-bridge present by default, but was removed in
>>    d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
>>
>> [1]: https://libvirt.org/pci-hotplug.html
>>
>> Signed-off-by: Oleg Vasilev <oleg.vasilev@virtuozzo.com>
>> ---
> [...]
>
>>   73 files changed, 621 insertions(+), 361 deletions(-)
>>
>> diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
>> index 9ef6c8bb64..52a393e07f 100644
>> --- a/src/qemu/qemu_domain.c
>> +++ b/src/qemu/qemu_domain.c
>> @@ -3890,6 +3890,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>>       bool addImplicitSATA = false;
>>       bool addPCIRoot = false;
>>       bool addPCIeRoot = false;
>> +    bool addPCIeToPCIBridge = false;
>>       bool addDefaultMemballoon = true;
>>       bool addDefaultUSBKBD = false;
>>       bool addDefaultUSBMouse = false;
>> @@ -3910,6 +3911,8 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>>           if (qemuDomainIsQ35(def)) {
>>               addPCIeRoot = true;
>>               addImplicitSATA = true;
>> +            if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PCIE_PCI_BRIDGE))
>> +                addPCIeToPCIBridge = true;
>>   
>>               /* Prefer adding a USB3 controller if supported, fall back
>>                * to USB2 if there is no USB3 available, and if that's
>> @@ -4046,11 +4049,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>>           }
>>       }
>>   
>> -    /* When a machine has a pcie-root, make sure that there is always
>> -     * a dmi-to-pci-bridge controller added as bus 1, and a pci-bridge
>> -     * as bus 2, so that standard PCI devices can be connected
>> -     *
>> -     * NB: any machine that sets addPCIeRoot to true must also return
>> +    /* NB: any machine that sets addPCIeRoot to true must also return
>>        * true from the function qemuDomainSupportsPCI().
>>        */
>>       if (addPCIeRoot) {
>> @@ -4069,6 +4068,16 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
>>           }
>>       }
>>   
>> +    /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
>> +    if (addPCIeToPCIBridge) {
>> +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
>> +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
>> +            return -1;
>> +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
>> +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
>> +            return -1;
>> +    }
> Doing this unconditionally breaks ABI of the VM, as new guest visible
> devices appear without any change in configuration. What's worse this
> implementation would also break migration compatibility from older
> versions of libvirt as there isn't any form of explicit handling for it.

Hmm, I am not so sure here. Doing this we goes exactly in the same way
as done for balloon and thus my expectations are that the device will
appear in the configuration.

Quick check over domain.xml seems that my opinion is correct. I am
able to find the following device

     <controller type='pci' index='2' model='pcie-to-pci-bridge'>
       <model name='pcie-pci-bridge'/>
       <alias name='pci.2'/>
       <address type='pci' domain='0x0000' bus='0x01' slot='0x00' 
function='0x0'/>
     </controller>

> I unfortunately don't see any reasonable solution originating in
> libvirt. The bridge will need to be added by users or higher level
> management tools (e.g. virt-install) as libvirt explicitly wants to
> support use cases when VMs are started without persisting the XML
> definition (e.g. always re-define the XML before start or use the
> 'createXML' api).
>
That approach seems cumbersome. Even in Virtuozzo we have
* QA test client, which creates VMs on its own
* 'virt-install' based software
* 'prl-disp-service' based software
* OpenStack based clusters

With your proposal we would need to implement a code,
which will check that the VM is created with Q35
machine type and in that case the bridge is added
each time the virtual machine is created. There are a lot
of software doing that and the process seems to complex.
In general, old PC based software have supported ordinary
VirtIO network device hotplug without any VM specific
pre-configuration and that was brilliant. Please correct
me if I am wrong.

Proposed approach requires a lot of efforts and seems
error prone.

On the other hand I could propose at least 2 options:
* hotplugging of PCIe-to-PCI bridge at the time
    of the first hotlug to Q35 VM
* put the behavior of creating default bridge under
    the option in libvirt configuration

In general, hotplugging of bridge seems much more
guest specific but potentially could work.

Den
Re: [PATCH] qemu: add pcie-to-pci-bridge for q35 machines
Posted by Peter Krempa 1 year, 4 months ago
On Fri, Nov 04, 2022 at 14:16:44 +0100, Denis V. Lunev wrote:
> On 11/4/22 13:03, Peter Krempa wrote:
> > On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
> > > Hotplugging PCI devices on pc-i440fx machines is supported without
> > > additional configuration. On q35, pcie-to-pci-bridge needs to be added
> > > prior to the machine startup in order to support hotplug [1].
> > > 
> > > The idea is to support the same workflow for creating VMs in q35, as was
> > > in pc. Namely, do not require additional configuration when hotplugging
> > > is needed. Otherwise, all libvirt clients need to be updated which there
> > > are a lot and they are maintained by different parties.
> > > 
> > > Instead, a pcie-to-pci-bridge better be created by default, so that PCI
> > > slots are readily available. Might be a good idea to make it configurable
> > > in the future.
> > > 
> > > Previously there was a pci-bridge present by default, but was removed in
> > >    d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
> > > 
> > > [1]: https://libvirt.org/pci-hotplug.html
> > > 
> > > Signed-off-by: Oleg Vasilev <oleg.vasilev@virtuozzo.com>
> > > ---
> > [...]
> > 
> > >   73 files changed, 621 insertions(+), 361 deletions(-)
> > > 
> > > diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
> > > index 9ef6c8bb64..52a393e07f 100644
> > > --- a/src/qemu/qemu_domain.c
> > > +++ b/src/qemu/qemu_domain.c
> > > @@ -3890,6 +3890,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
> > >       bool addImplicitSATA = false;
> > >       bool addPCIRoot = false;
> > >       bool addPCIeRoot = false;
> > > +    bool addPCIeToPCIBridge = false;
> > >       bool addDefaultMemballoon = true;
> > >       bool addDefaultUSBKBD = false;
> > >       bool addDefaultUSBMouse = false;
> > > @@ -3910,6 +3911,8 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
> > >           if (qemuDomainIsQ35(def)) {
> > >               addPCIeRoot = true;
> > >               addImplicitSATA = true;
> > > +            if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_PCIE_PCI_BRIDGE))
> > > +                addPCIeToPCIBridge = true;
> > >               /* Prefer adding a USB3 controller if supported, fall back
> > >                * to USB2 if there is no USB3 available, and if that's
> > > @@ -4046,11 +4049,7 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
> > >           }
> > >       }
> > > -    /* When a machine has a pcie-root, make sure that there is always
> > > -     * a dmi-to-pci-bridge controller added as bus 1, and a pci-bridge
> > > -     * as bus 2, so that standard PCI devices can be connected
> > > -     *
> > > -     * NB: any machine that sets addPCIeRoot to true must also return
> > > +    /* NB: any machine that sets addPCIeRoot to true must also return
> > >        * true from the function qemuDomainSupportsPCI().
> > >        */
> > >       if (addPCIeRoot) {
> > > @@ -4069,6 +4068,16 @@ qemuDomainDefAddDefaultDevices(virQEMUDriver *driver,
> > >           }
> > >       }
> > > +    /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
> > > +    if (addPCIeToPCIBridge) {
> > > +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
> > > +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
> > > +            return -1;
> > > +        if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
> > > +                                       VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
> > > +            return -1;
> > > +    }
> > Doing this unconditionally breaks ABI of the VM, as new guest visible
> > devices appear without any change in configuration. What's worse this
> > implementation would also break migration compatibility from older
> > versions of libvirt as there isn't any form of explicit handling for it.
> 
> Hmm, I am not so sure here. Doing this we goes exactly in the same way
> as done for balloon and thus my expectations are that the device will
> appear in the configuration.
> 
> Quick check over domain.xml seems that my opinion is correct. I am
> able to find the following device
> 
>     <controller type='pci' index='2' model='pcie-to-pci-bridge'>
>       <model name='pcie-pci-bridge'/>
>       <alias name='pci.2'/>
>       <address type='pci' domain='0x0000' bus='0x01' slot='0x00'
> function='0x0'/>
>     </controller>

Note that balloon is not a good example here. In fact it's a bit of a
counter example to the case of yours.

The balloon device was historically auto-added by qemu unless
specifically disabled. Libvirt implemented the support for controlling
ballon some time later.

This meant that there were libvirt XMLs which resulted in the balloon
device being present in the VM ABI. To correct this we auto-add the XML
bit as we historically did. If you want to disable the balloon you must
explicitly request a config where it is disabled.

Now in the case presented here we did not historically add the device so
we must not do that unless explcitly configured.

> > I unfortunately don't see any reasonable solution originating in
> > libvirt. The bridge will need to be added by users or higher level
> > management tools (e.g. virt-install) as libvirt explicitly wants to
> > support use cases when VMs are started without persisting the XML
> > definition (e.g. always re-define the XML before start or use the
> > 'createXML' api).
> > 
> That approach seems cumbersome. Even in Virtuozzo we have
> * QA test client, which creates VMs on its own
> * 'virt-install' based software
> * 'prl-disp-service' based software
> * OpenStack based clusters
> 
> With your proposal we would need to implement a code,
> which will check that the VM is created with Q35
> machine type and in that case the bridge is added
> each time the virtual machine is created. There are a lot
> of software doing that and the process seems to complex.
> In general, old PC based software have supported ordinary
> VirtIO network device hotplug without any VM specific
> pre-configuration and that was brilliant. Please correct
> me if I am wrong.
> 
> Proposed approach requires a lot of efforts and seems
> error prone.
> 
> On the other hand I could propose at least 2 options:
> * hotplugging of PCIe-to-PCI bridge at the time
>    of the first hotlug to Q35 VM

This should be okay because an explicit configuration change request
will cause it.

> * put the behavior of creating default bridge under
>    the option in libvirt configuration

If by "libvirt configuration" you mean putting it into the
/etc/libvirt/qemu.conf or similar that is not acceptable, because it has
the same set of problems with changing ABI.

The only reasonable place is somewhere in the VM xml. In fact we already
provide that if you specify the bridge manually but I guess some form of
syntax sugar to add the bridge could be (reluctantly) acceptable.

> In general, hotplugging of bridge seems much more
> guest specific but potentially could work.