Hi,
This is a follow up to the third patch series [0] for using iommufd
to propagate DMA mappings to the kernel for VM-assigned host
devices in a qemu VM.
We add a new 'iommufd' attribute for hostdev devices to be
associated with the iommufd object.
For instance, specifying the iommufd object and associated hostdev in a
VM definition:
<devices>
...
<hostdev mode='subsystem' type='pci' managed='no'>
<driver iommufd='yes'/>
<source>
<address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='no'>
<driver iommufd='yes'/>
<source>
<address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/>
</hostdev>
...
</devices>
This would get translated to a qemu command line with the arguments below.
Note that libvirt will open the /dev/iommu and VFIO cdev, passing the
associated fd number to qemu:
-object '{"qom-type":"iommufd","id":"iommufd0","fd":"24"}' \
-device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","iommufd":"iommufd0","fd":"22","bus":"pci.21","addr":"0x0"}' \
-device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","iommufd":"iommufd0","fd":"25","bus":"pci.22","addr":"0x0"}' \
Changes from v3:
- Resolved issue from v2 where stale FD from previous VM boot was
in use
- Remove second approach for retrieving VFIO device path in
virPCIDeviceGetVfioPath()
- Resolve broken build of libvirt on non-Linux platforms
- Conditionally define iommufd headers and use system headers where
possible
- Add non-fatal handling + warning print for EPERM for the
IOMMU_OPTION_RLIMIT_MODE ioctl
- Replace references to /dev/iommu with VIR_IOMMU_DEV_PATH
- Implement virIOMMUFDSupported(void) to check for existence of
/dev/iommu on host
- Include tests for multiple hostdevs
Changes from v2:
- Set per-process memory accounting mode for iommufd
- Separated out formatting of iommufd object from qemuBuildHostdevCommandLine
- Placed hostdev private data implementation in a separate commit
- Allocate hostdev private data unconditionally
- Compare FDs against -1
- Integrated callback function in virQEMUDriverPrivateDataCallbacks for qemuDomainHostdevPrivateNew
- Dropped qemuProcessCloseVfioFds
- Addressed other feedback from v2 (formatting, includes, etc.)
- Revised seclabel logic to be device-specific for AppArmor and to allow paths for SELinux/DAC
This series is on Github:
https://github.com/NathanChenNVIDIA/libvirt/tree/iommufd-01-26
Thanks,
Nathan
[0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/WIBZ6HRZGVCDXJBOWYROHYAQY5XBSABW/
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Nathan Chen (7):
qemu: Implement support for associating iommufd to hostdev
qemu: Introduce privateData for hostdevs
qemu: Set per-process memory accounting for iommufd
qemu: open VFIO FDs from libvirt backend
qemu: open iommufd FD from libvirt backend
qemu: Update Cgroup, namespace, and seclabel for iommufd
tests: qemuxmlconfdata: provide iommufd sample XML and CLI args
docs/formatdomain.rst | 7 +
po/POTFILES | 1 +
src/bhyve/bhyve_parse_command.c | 2 +-
src/conf/device_conf.c | 11 ++
src/conf/device_conf.h | 1 +
src/conf/domain_conf.c | 13 +-
src/conf/domain_conf.h | 5 +-
src/conf/schemas/basictypes.rng | 5 +
src/libvirt_private.syms | 5 +
src/libxl/xen_common.c | 2 +-
src/libxl/xen_xl.c | 2 +-
src/lxc/lxc_native.c | 2 +-
src/qemu/qemu_cgroup.c | 26 ++--
src/qemu/qemu_command.c | 76 +++++++++++
src/qemu/qemu_domain.c | 41 ++++++
src/qemu/qemu_domain.h | 20 +++
src/qemu/qemu_namespace.c | 16 ++-
src/qemu/qemu_process.c | 118 ++++++++++++++++
src/security/security_apparmor.c | 32 ++++-
src/security/security_dac.c | 59 ++++++--
src/security/security_selinux.c | 57 ++++++--
src/security/virt-aa-helper.c | 33 ++++-
src/util/meson.build | 1 +
src/util/viriommufd.c | 127 ++++++++++++++++++
src/util/viriommufd.h | 27 ++++
src/util/virpci.c | 42 ++++++
src/util/virpci.h | 2 +
src/vbox/vbox_common.c | 2 +-
.../iommufd-q35.x86_64-latest.args | 41 ++++++
.../iommufd-q35.x86_64-latest.xml | 60 +++++++++
tests/qemuxmlconfdata/iommufd-q35.xml | 38 ++++++
...fd-virt-pci-bus-single.aarch64-latest.args | 33 +++++
...ufd-virt-pci-bus-single.aarch64-latest.xml | 34 +++++
.../iommufd-virt-pci-bus-single.xml | 22 +++
.../iommufd-virt.aarch64-latest.args | 37 +++++
.../iommufd-virt.aarch64-latest.xml | 56 ++++++++
tests/qemuxmlconfdata/iommufd-virt.xml | 29 ++++
.../iommufd.x86_64-latest.args | 35 +++++
.../qemuxmlconfdata/iommufd.x86_64-latest.xml | 38 ++++++
tests/qemuxmlconfdata/iommufd.xml | 30 +++++
tests/qemuxmlconftest.c | 34 +++++
tests/virhostdevtest.c | 2 +-
42 files changed, 1162 insertions(+), 62 deletions(-)
create mode 100644 src/util/viriommufd.c
create mode 100644 src/util/viriommufd.h
create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd-q35.x86_64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-q35.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt-pci-bus-single.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd-virt.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd-virt.xml
create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommufd.x86_64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommufd.xml
--
2.43.0