Hi,

This is a follow up to the fourth RFC patchset [0] for supporting multiple
vSMMU instances and using iommufd to propagate DMA mappings to kernel for
VM-assigned host devices in a qemu VM.

This patchset implements support for specifying multiple <iommu> devices
within the VM definition when smmuv3Dev IOMMU model is specified, and is
tested with Shameer's v4 qemu RFC for HW-accelerated vSMMU devices [1]

Moreover, it adds a new 'iommufd' attribute for hostdev devices to be
associated with the iommufd object.

For instance, specifying the iommufd object and associated hostdev in a
VM definition with multiple IOMMUs, configured to be routed to
pcie-expander-bus controllers in a way where VFIO device to SMMUv3
associations are matched with the host:

  <devices>
...
    <controller type='pci' index='1' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='252'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='248'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
...
    <controller type='pci' index='21' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='21' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='22' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='22' port='0xa8'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
...
    <hostdev mode='subsystem' type='pci' managed='no'>
      <driver iommufd='yes'/>
      <source>
        <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='no'>
      <driver iommufd='yes'/>
      <source>
        <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/>
    </hostdev>
    <iommu model='smmuv3Dev'>
      <driver parentIdx='1' accel='on' ats='on' ril='on' pasid='on' oas='44'/>
    </iommu>
    <iommu model='smmuv3Dev'>
      <driver parentIdx='2' accel='on' ats='on' ril='on' pasid='on' oas='44'/>
    </iommu>
  </devices>

This would get translated to a qemu command line with the arguments below.
Note that libvirt will open the /dev/iommu and VFIO cdev, passing the
associated fd number to qemu:

 -device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
 -device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \
 -device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.21","bus":"pci.1","addr":"0x0"}' \
 -device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.22","bus":"pci.2","addr":"0x0"}' \
 -object '{"qom-type":"iommufd","id":"iommufd0","fd":"24"}' \
 -device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"smmuv3.0","accel":true,"ats":true,"ril":true,"pasid":true,"oas":44}' \
 -device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"smmuv3.1","accel":true,"ats":true,"ril":true,"pasid":true,"oas":44}' \
 -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","iommufd":"iommufd0","fd":"22","bus":"pci.21","addr":"0x0"}' \
 -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","iommufd":"iommufd0","fd":"25","bus":"pci.22","addr":"0x0"}' \

Changes from RFCv3:
- Move smmuv3Dev-specific attributes under <driver>
- Include new smmuv3Dev-specific attributes based on Shameer's v4 qemu series

This series is on Github:
https://github.com/NathanChenNVIDIA/libvirt/tree/smmuv3Dev-iommufd-10-01-25

Thanks,
Nathan

[0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/T66FIOXQEOUEFZUWAAQWEQTLIGHNKRTQ/
[1] https://lore.kernel.org/all/20250929133643.38961-1-skolothumtho@nvidia.com/#r

Signed-off-by: Nathan Chen <nathanc@nvidia.com>

Nathan Chen (7):
  qemu: add IOMMU model smmuv3Dev
  conf: Support multiple smmuv3Dev IOMMU devices
  tests: qemuxmlconfdata: provide smmuv3Dev sample XML and CLI args
  qemu: Implement support for associating iommufd to hostdev
  qemu: open iommufd FDs from libvirt backend
  qemu: Update Cgroup, namespace, and seclabel for qemu to access
    iommufd paths
  tests: qemuxmlconfdata: provide iommufd sample XML and CLI args

 docs/formatdomain.rst                         |  44 ++-
 src/conf/device_conf.c                        |   9 +
 src/conf/device_conf.h                        |   1 +
 src/conf/domain_conf.c                        | 177 +++++++++--
 src/conf/domain_conf.h                        |  16 +-
 src/conf/domain_validate.c                    |  73 ++++-
 src/conf/schemas/basictypes.rng               |   5 +
 src/conf/schemas/domaincommon.rng             |  35 ++-
 src/libvirt_private.syms                      |   2 +
 src/qemu/qemu_alias.c                         |  15 +-
 src/qemu/qemu_cgroup.c                        |  61 ++++
 src/qemu/qemu_cgroup.h                        |   1 +
 src/qemu/qemu_command.c                       | 275 +++++++++++++-----
 src/qemu/qemu_command.h                       |   3 +-
 src/qemu/qemu_domain.c                        |   8 +
 src/qemu/qemu_domain.h                        |   7 +
 src/qemu/qemu_domain_address.c                |  33 ++-
 src/qemu/qemu_driver.c                        |   8 +-
 src/qemu/qemu_hotplug.c                       |   2 +-
 src/qemu/qemu_namespace.c                     |  44 +++
 src/qemu/qemu_postparse.c                     |  11 +-
 src/qemu/qemu_process.c                       | 232 +++++++++++++++
 src/qemu/qemu_validate.c                      |  18 +-
 src/security/security_apparmor.c              |  15 +
 src/security/security_dac.c                   |  34 +++
 src/security/security_selinux.c               |  30 ++
 src/util/virpci.c                             |  68 +++++
 src/util/virpci.h                             |   1 +
 .../iommu-smmuv3Dev.aarch64-latest.args       |  39 +++
 .../iommu-smmuv3Dev.aarch64-latest.xml        |  66 +++++
 tests/qemuxmlconfdata/iommu-smmuv3Dev.xml     |  53 ++++
 .../iommufd-q35.x86_64-latest.args            |  41 +++
 .../iommufd-q35.x86_64-latest.xml             |  60 ++++
 tests/qemuxmlconfdata/iommufd-q35.xml         |  38 +++
 .../iommufd-virt.aarch64-latest.args          |  33 +++
 .../iommufd-virt.aarch64-latest.xml           |  34 +++
 tests/qemuxmlconfdata/iommufd-virt.xml        |  22 ++
 .../iommufd.x86_64-latest.args                |  35 +++
 .../qemuxmlconfdata/iommufd.x86_64-latest.xml |  38 +++
 tests/qemuxmlconfdata/iommufd.xml             |  30 ++
 tests/qemuxmlconftest.c                       |   5 +
 41 files changed, 1584 insertions(+), 138 deletions(-)
 create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3Dev.aarch64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3Dev.aarch64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3Dev.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-q35.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd.xml

-- 
2.43.0