While libvirt does not support all the qemu parameters in standand
xml syntax, such as hostfwd. We have to use <qemu:commandline> to
passthrough the parameters. E.x:

<qemu:commandline>
  <qemu:arg value='-netdev'/>
    <qemu:arg value='user,id=mynet0,hostfwd=tcp::44483-:22'/>
    <qemu:arg value='-device'/>
  <qemu:arg value='virtio-net-pci,netdev=mynet0,mac=00:16:3E:68:00:10,romfile='/>
</qemu:commandline>

Then we got below error:

qemu-system-x86_64: -device {"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}: PCI: slot 1 function 0 not available for pcie-root-port, in use by virtio-net-pci

Libvirt generates qemu "-device" in JSON syntax followed by the
passthroughed "-device" in legacy syntax. But legacy syntax
described devices are assigned BDF first, this leads to conflict
with the pci devices in JSON syntax with fixed BDF.

Move legacy syntax described device creation after JSON syntax
described ones so we can start that kind of VM successfully in
libvirt.

Signed-off-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
---
 softmmu/vl.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/softmmu/vl.c b/softmmu/vl.c
index 1fe028800fdf..5cf90e279fe9 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2682,8 +2682,6 @@ static void qemu_create_cli_devices(void)
 
     /* init generic devices */
     rom_set_order_override(FW_CFG_ORDER_OVERRIDE_DEVICE);
-    qemu_opts_foreach(qemu_find_opts("device"),
-                      device_init_func, NULL, &error_fatal);
     QTAILQ_FOREACH(opt, &device_opts, next) {
         DeviceState *dev;
         loc_push_restore(&opt->loc);
@@ -2698,6 +2696,8 @@ static void qemu_create_cli_devices(void)
         object_unref(OBJECT(dev));
         loc_pop(&opt->loc);
     }
+    qemu_opts_foreach(qemu_find_opts("device"),
+                      device_init_func, NULL, &error_fatal);
     rom_reset_order_override();
 }
 
-- 
2.25.1

On Wed, Feb 23, 2022 at 17:46:49 +0800, Zhenzhong Duan wrote:
> While libvirt does not support all the qemu parameters in standand
> xml syntax, such as hostfwd. We have to use <qemu:commandline> to
> passthrough the parameters. E.x:
> 
> <qemu:commandline>
>   <qemu:arg value='-netdev'/>
>     <qemu:arg value='user,id=mynet0,hostfwd=tcp::44483-:22'/>
>     <qemu:arg value='-device'/>
>   <qemu:arg value='virtio-net-pci,netdev=mynet0,mac=00:16:3E:68:00:10,romfile='/>
> </qemu:commandline>
> 
> Then we got below error:
> 
> qemu-system-x86_64: -device {"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"}: PCI: slot 1 function 0 not available for pcie-root-port, in use by virtio-net-pci
> 
> Libvirt generates qemu "-device" in JSON syntax followed by the
> passthroughed "-device" in legacy syntax. But legacy syntax
> described devices are assigned BDF first, this leads to conflict
> with the pci devices in JSON syntax with fixed BDF.
> 
> Move legacy syntax described device creation after JSON syntax
> described ones so we can start that kind of VM successfully in
> libvirt.

As a libvirt developer I strongly suggest you post patches for your use
case to libvirt rather than trying to go behind our back with the
commandline passthrough.