[PATCH v6 0/7] qemu: Implement support for iommufd

Nathan Chen via Devel posted 7 patches 2 days, 9 hours ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20260130185918.4154310-1-nathanc@nvidia.com
docs/formatdomain.rst                         |   6 +
meson.build                                   |   1 +
po/POTFILES                                   |   1 +
src/bhyve/bhyve_parse_command.c               |   2 +-
src/conf/device_conf.c                        |  11 ++
src/conf/device_conf.h                        |   1 +
src/conf/domain_conf.c                        |  13 +-
src/conf/domain_conf.h                        |   5 +-
src/conf/schemas/basictypes.rng               |   5 +
src/libvirt_private.syms                      |   4 +
src/libxl/xen_common.c                        |   2 +-
src/libxl/xen_xl.c                            |   2 +-
src/lxc/lxc_native.c                          |   2 +-
src/qemu/qemu_cgroup.c                        |   3 +
src/qemu/qemu_command.c                       |  76 +++++++++++
src/qemu/qemu_domain.c                        |  41 ++++++
src/qemu/qemu_domain.h                        |  17 +++
src/qemu/qemu_namespace.c                     |   3 +
src/qemu/qemu_process.c                       | 121 ++++++++++++++++++
src/security/security_apparmor.c              |  28 ++--
src/security/security_dac.c                   |  49 +++++--
src/security/security_selinux.c               |  47 +++++--
src/security/virt-aa-helper.c                 |  33 ++++-
src/util/meson.build                          |   1 +
src/util/viriommufd.c                         | 111 ++++++++++++++++
src/util/viriommufd.h                         |  25 ++++
src/util/virpci.c                             |  39 ++++++
src/util/virpci.h                             |   2 +
src/vbox/vbox_common.c                        |   2 +-
.../iommufd-q35.x86_64-latest.args            |  41 ++++++
.../iommufd-q35.x86_64-latest.xml             |  60 +++++++++
tests/qemuxmlconfdata/iommufd-q35.xml         |  38 ++++++
...fd-virt-pci-bus-single.aarch64-latest.args |  32 +++++
...ufd-virt-pci-bus-single.aarch64-latest.xml |  31 +++++
.../iommufd-virt-pci-bus-single.xml           |  22 ++++
.../iommufd-virt.aarch64-latest.args          |  36 ++++++
.../iommufd-virt.aarch64-latest.xml           |  53 ++++++++
tests/qemuxmlconfdata/iommufd-virt.xml        |  29 +++++
.../iommufd.x86_64-latest.args                |  35 +++++
.../qemuxmlconfdata/iommufd.x86_64-latest.xml |  38 ++++++
tests/qemuxmlconfdata/iommufd.xml             |  30 +++++
tests/qemuxmlconftest.c                       |  34 +++++
tests/virhostdevtest.c                        |   2 +-
43 files changed, 1090 insertions(+), 44 deletions(-)
create mode 100644 src/util/viriommufd.c
create mode 100644 src/util/viriommufd.h
create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-q35.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt.xml
create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd.xml
[PATCH v6 0/7] qemu: Implement support for iommufd
Posted by Nathan Chen via Devel 2 days, 9 hours ago
Hi,

This is a follow up to the fifth patch series [0] for using iommufd
to propagate DMA mappings to the kernel for VM-assigned host
devices in a qemu VM.

We add a new 'iommufd' attribute for hostdev devices to be
associated with the iommufd object.

For instance, specifying the iommufd object and associated hostdev in a
VM definition:

  <devices>
...
    <hostdev mode='subsystem' type='pci' managed='no'>
      <driver iommufd='yes'/>
      <source>
        <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='no'>
      <driver iommufd='yes'/>
      <source>
        <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/>
    </hostdev>
...
  </devices>

This would get translated to a qemu command line with the arguments below.
Note that libvirt will open the /dev/iommu and VFIO cdev, passing the
associated fd number to qemu:

 -object '{"qom-type":"iommufd","id":"iommufd0","fd":"24"}' \
 -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","iommufd":"iommufd0","fd":"22","bus":"pci.21","addr":"0x0"}' \
 -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","iommufd":"iommufd0","fd":"25","bus":"pci.22","addr":"0x0"}' \

Changes from v5:
- Updated documentation to point to 12.1.0 (QEMU and KVM only)
- Move define and VIR_LOG_INIT out of #ifdef __linux__ block
  in viriommufd.h
- Remove unneeded virPCIDeviceFree() from AppArmorSetSecurityHostdevLabel 
- Always error out if per-process accounting failed

Changes from v4:
- Move qemuProcessOpenVfioFds() to qemu_process.h in patch 4/7
- Add `linux/iommufd.h` into headers list in the meson.build file
- Correct viriommufd.c pre-processor definition indentations
- Restore call to virIOMMUFDSetRLimitMode() in patch 5/7
- Fix indentations for various print function calls
- Move the qemuProcessOpenVfioFds() call to qemuProcessPrepareHost()
- Make qemuProcessOpenVfioFds() static
- Remove unnecessary formatting and comment in virPCIDeviceGetVfioPath()
- Clean up namespace and cgroup changes with conditional check for
  iommufd
- Remove virIOMMUFDSupported()
- Fix seclabel return logic
- In Apparmor seclabel logic, use g_autoptr for virPCIDevice pointer

Changes from v3:
- Resolved issue from v2 where stale FD from previous VM boot was
  in use
- Remove second approach for retrieving VFIO device path in
  virPCIDeviceGetVfioPath()
- Resolve broken build of libvirt on non-Linux platforms
- Conditionally define iommufd headers and use system headers where
  possible
- Add non-fatal handling + warning print for EPERM for the
  IOMMU_OPTION_RLIMIT_MODE ioctl
- Replace references to /dev/iommu with VIR_IOMMU_DEV_PATH
- Implement virIOMMUFDSupported(void) to check for existence of
  /dev/iommu on host
- Include tests for multiple hostdevs

Changes from v2:
- Set per-process memory accounting mode for iommufd
- Separated out formatting of iommufd object from qemuBuildHostdevCommandLine
- Placed hostdev private data implementation in a separate commit
- Allocate hostdev private data unconditionally
- Compare FDs against -1
- Integrated callback function in virQEMUDriverPrivateDataCallbacks for qemuDomainHostdevPrivateNew
- Dropped qemuProcessCloseVfioFds
- Addressed other feedback from v2 (formatting, includes, etc.)
- Revised seclabel logic to be device-specific for AppArmor and to allow paths for SELinux/DAC

This series is on Github:
https://github.com/NathanChenNVIDIA/libvirt/commits/iommufd-v6-01-26

Thanks,
Nathan

[0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/GKSDPK5VSPUKE5C73GI5IPDPUHO37D55/

Nathan Chen (7):
  qemu: Implement support for associating iommufd to hostdev
  qemu: Introduce privateData for hostdevs
  qemu: Support per-process memory accounting for iommufd
  qemu: open VFIO FDs from libvirt backend
  qemu: open iommufd FD from libvirt backend
  qemu: Update Cgroup, namespace, and seclabel for iommufd
  tests: qemuxmlconfdata: provide iommufd sample XML and CLI args

 docs/formatdomain.rst                         |   6 +
 meson.build                                   |   1 +
 po/POTFILES                                   |   1 +
 src/bhyve/bhyve_parse_command.c               |   2 +-
 src/conf/device_conf.c                        |  11 ++
 src/conf/device_conf.h                        |   1 +
 src/conf/domain_conf.c                        |  13 +-
 src/conf/domain_conf.h                        |   5 +-
 src/conf/schemas/basictypes.rng               |   5 +
 src/libvirt_private.syms                      |   4 +
 src/libxl/xen_common.c                        |   2 +-
 src/libxl/xen_xl.c                            |   2 +-
 src/lxc/lxc_native.c                          |   2 +-
 src/qemu/qemu_cgroup.c                        |   3 +
 src/qemu/qemu_command.c                       |  76 +++++++++++
 src/qemu/qemu_domain.c                        |  41 ++++++
 src/qemu/qemu_domain.h                        |  17 +++
 src/qemu/qemu_namespace.c                     |   3 +
 src/qemu/qemu_process.c                       | 121 ++++++++++++++++++
 src/security/security_apparmor.c              |  28 ++--
 src/security/security_dac.c                   |  49 +++++--
 src/security/security_selinux.c               |  47 +++++--
 src/security/virt-aa-helper.c                 |  33 ++++-
 src/util/meson.build                          |   1 +
 src/util/viriommufd.c                         | 111 ++++++++++++++++
 src/util/viriommufd.h                         |  25 ++++
 src/util/virpci.c                             |  39 ++++++
 src/util/virpci.h                             |   2 +
 src/vbox/vbox_common.c                        |   2 +-
 .../iommufd-q35.x86_64-latest.args            |  41 ++++++
 .../iommufd-q35.x86_64-latest.xml             |  60 +++++++++
 tests/qemuxmlconfdata/iommufd-q35.xml         |  38 ++++++
 ...fd-virt-pci-bus-single.aarch64-latest.args |  32 +++++
 ...ufd-virt-pci-bus-single.aarch64-latest.xml |  31 +++++
 .../iommufd-virt-pci-bus-single.xml           |  22 ++++
 .../iommufd-virt.aarch64-latest.args          |  36 ++++++
 .../iommufd-virt.aarch64-latest.xml           |  53 ++++++++
 tests/qemuxmlconfdata/iommufd-virt.xml        |  29 +++++
 .../iommufd.x86_64-latest.args                |  35 +++++
 .../qemuxmlconfdata/iommufd.x86_64-latest.xml |  38 ++++++
 tests/qemuxmlconfdata/iommufd.xml             |  30 +++++
 tests/qemuxmlconftest.c                       |  34 +++++
 tests/virhostdevtest.c                        |   2 +-
 43 files changed, 1090 insertions(+), 44 deletions(-)
 create mode 100644 src/util/viriommufd.c
 create mode 100644 src/util/viriommufd.h
 create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-q35.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.aarch64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.aarch64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd-virt.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommufd.xml

-- 
2.43.0
Re: [PATCH v6 0/7] qemu: Implement support for iommufd
Posted by Pavel Hrdina via Devel 2 days, 6 hours ago
On Fri, Jan 30, 2026 at 10:59:11AM -0800, Nathan Chen via Devel wrote:
> Hi,
> 
> This is a follow up to the fifth patch series [0] for using iommufd
> to propagate DMA mappings to the kernel for VM-assigned host
> devices in a qemu VM.
> 
> We add a new 'iommufd' attribute for hostdev devices to be
> associated with the iommufd object.
> 
> For instance, specifying the iommufd object and associated hostdev in a
> VM definition:
> 
>   <devices>
> ...
>     <hostdev mode='subsystem' type='pci' managed='no'>
>       <driver iommufd='yes'/>
>       <source>
>         <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/>
>       </source>
>       <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/>
>     </hostdev>
>     <hostdev mode='subsystem' type='pci' managed='no'>
>       <driver iommufd='yes'/>
>       <source>
>         <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/>
>       </source>
>       <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/>
>     </hostdev>
> ...
>   </devices>
> 
> This would get translated to a qemu command line with the arguments below.
> Note that libvirt will open the /dev/iommu and VFIO cdev, passing the
> associated fd number to qemu:
> 
>  -object '{"qom-type":"iommufd","id":"iommufd0","fd":"24"}' \
>  -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","iommufd":"iommufd0","fd":"22","bus":"pci.21","addr":"0x0"}' \
>  -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","iommufd":"iommufd0","fd":"25","bus":"pci.22","addr":"0x0"}' \
> 
> Changes from v5:
> - Updated documentation to point to 12.1.0 (QEMU and KVM only)
> - Move define and VIR_LOG_INIT out of #ifdef __linux__ block
>   in viriommufd.h
> - Remove unneeded virPCIDeviceFree() from AppArmorSetSecurityHostdevLabel 
> - Always error out if per-process accounting failed
> 
> Changes from v4:
> - Move qemuProcessOpenVfioFds() to qemu_process.h in patch 4/7
> - Add `linux/iommufd.h` into headers list in the meson.build file
> - Correct viriommufd.c pre-processor definition indentations
> - Restore call to virIOMMUFDSetRLimitMode() in patch 5/7
> - Fix indentations for various print function calls
> - Move the qemuProcessOpenVfioFds() call to qemuProcessPrepareHost()
> - Make qemuProcessOpenVfioFds() static
> - Remove unnecessary formatting and comment in virPCIDeviceGetVfioPath()
> - Clean up namespace and cgroup changes with conditional check for
>   iommufd
> - Remove virIOMMUFDSupported()
> - Fix seclabel return logic
> - In Apparmor seclabel logic, use g_autoptr for virPCIDevice pointer
> 
> Changes from v3:
> - Resolved issue from v2 where stale FD from previous VM boot was
>   in use
> - Remove second approach for retrieving VFIO device path in
>   virPCIDeviceGetVfioPath()
> - Resolve broken build of libvirt on non-Linux platforms
> - Conditionally define iommufd headers and use system headers where
>   possible
> - Add non-fatal handling + warning print for EPERM for the
>   IOMMU_OPTION_RLIMIT_MODE ioctl
> - Replace references to /dev/iommu with VIR_IOMMU_DEV_PATH
> - Implement virIOMMUFDSupported(void) to check for existence of
>   /dev/iommu on host
> - Include tests for multiple hostdevs
> 
> Changes from v2:
> - Set per-process memory accounting mode for iommufd
> - Separated out formatting of iommufd object from qemuBuildHostdevCommandLine
> - Placed hostdev private data implementation in a separate commit
> - Allocate hostdev private data unconditionally
> - Compare FDs against -1
> - Integrated callback function in virQEMUDriverPrivateDataCallbacks for qemuDomainHostdevPrivateNew
> - Dropped qemuProcessCloseVfioFds
> - Addressed other feedback from v2 (formatting, includes, etc.)
> - Revised seclabel logic to be device-specific for AppArmor and to allow paths for SELinux/DAC
> 
> This series is on Github:
> https://github.com/NathanChenNVIDIA/libvirt/commits/iommufd-v6-01-26
> 
> Thanks,
> Nathan
> 
> [0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/GKSDPK5VSPUKE5C73GI5IPDPUHO37D55/
> 
> Nathan Chen (7):
>   qemu: Implement support for associating iommufd to hostdev
>   qemu: Introduce privateData for hostdevs
>   qemu: Support per-process memory accounting for iommufd
>   qemu: open VFIO FDs from libvirt backend
>   qemu: open iommufd FD from libvirt backend
>   qemu: Update Cgroup, namespace, and seclabel for iommufd
>   tests: qemuxmlconfdata: provide iommufd sample XML and CLI args

Once we decide what error reporting to use in PATCH 3.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>