:p
atchew
Login
Hi, This is a follow up to the seventh patchset [0] for supporting multiple hardware-accelerated vSMMU instances. For instance, specifying hostdevs associated with multiple accelerated SMMUs, configured to be routed to pcie-expander-bus controllers in a way where VFIO device to SMMUv3 associations are matched with the host: <devices> ... <controller type='pci' index='1' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='252'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </controller> <controller type='pci' index='2' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='248'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> ... <controller type='pci' index='21' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='21' port='0x0'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='22' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='22' port='0xa8'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> ... <hostdev mode='subsystem' type='pci' managed='no'> <source> <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='no'> <source> <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/> </hostdev> <iommu model='smmuv3'> <driver pciBus='1' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> </iommu> <iommu model='smmuv3'> <driver pciBus='2' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> </iommu> </devices> This would get translated to a qemu command line with the arguments below: -device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ -device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.21","bus":"pci.1","addr":"0x0"}' \ -device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.22","bus":"pci.2","addr":"0x0"}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true,"ats":true,"ril":false,"ssidsize":20,"oas":44}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true,"ats":true,"ril":false,"ssidsize":20,"oas":44}' \ -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","bus":"pci.21","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","bus":"pci.22","addr":"0x0"}' \ Two QEMU series implementing auto support for accel SMMUv3 properties are now present in the upstream master branch [1][2]. This Libvirt series leverages the auto support: users can omit SMMUv3 properties ATS, RIL, OAS, and SSIDSIZE on the domain XML, and they will also be omitted on the QEMU command line as a result. With these new QEMU series applied, the feature property values will default to 'auto' to be resolved based on what the host SMMU supports. This series is on Github: https://github.com/NathanChenNVIDIA/libvirt/tree/smmuv3-accel-v8/ Thanks, Nathan [0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/5GG76AQTTDUHW5KRANPY3QUII4ZOEYRP/ [1] https://lore.kernel.org/all/20260323182454.1416110-1-nathanc@nvidia.com/ [2] https://lore.kernel.org/all/20260608174900.2227340-1-nathanc@nvidia.com/ Changes from v7: - Remove support for x-vpasid-cap-offset based on feedback pointing out the QEMU feature prefixed with x- is experimental and not to be supported in Libvirt - Implement support for auto SMMUv3 properties. Omitting these properties in the XML omits them on the QEMU command line as well, which results in default auto values being chosen for ATS, RIL, SSIDSIZE, and OAS - Add validation check that only allows enabling accel SMMUv3 features when accel is enabled - Add tests for accel=on but omitting accel SMMUv3 feature properties Changes from v6: - Remove smmuv3 'pasid' attribute - Add smmuv3 'ssidsize' attribute - Add support for 'vpasidCapOffset' element - Modify logic for building accel smmuv3 qemu command line to omit ATS, RIL, accel flags when not needed (enabled/disabled by default in QEMU SMMUv3) Nathan Chen (2): qemu: Add support for HW-accelerated nested SMMUv3 tests: qemuxmlconfdata: provide HW-accel smmuv3 sample XML and CLI args docs/formatdomain.rst | 29 +++++++ src/conf/domain_conf.c | 80 +++++++++++++++++++ src/conf/domain_conf.h | 5 ++ src/conf/domain_validate.c | 27 ++++++- src/conf/schemas/domaincommon.rng | 25 ++++++ src/qemu/qemu_command.c | 5 ++ ...uv3-pci-bus-accel-auto.aarch64-latest.args | 38 +++++++++ ...muv3-pci-bus-accel-auto.aarch64-latest.xml | 63 +++++++++++++++ .../iommu-smmuv3-pci-bus-accel-auto.xml | 53 ++++++++++++ ...u-smmuv3-pci-bus-accel.aarch64-latest.args | 38 +++++++++ ...mu-smmuv3-pci-bus-accel.aarch64-latest.xml | 63 +++++++++++++++ .../iommu-smmuv3-pci-bus-accel.xml | 53 ++++++++++++ tests/qemuxmlconftest.c | 2 + 13 files changed, 480 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml -- 2.43.0
From: Nathan Chen <nathanc@nvidia.com> Add support for enabling HW-accelerated nested SMMUv3 via <accel> attribute and its additional attributes for ATS, SSID, RIL, and OAS configuration. Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- docs/formatdomain.rst | 29 +++++++++++ src/conf/domain_conf.c | 80 +++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 5 ++ src/conf/domain_validate.c | 27 ++++++++++- src/conf/schemas/domaincommon.rng | 25 ++++++++++ src/qemu/qemu_command.c | 5 ++ 6 files changed, 170 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -XXX,XX +XXX,XX @@ Examples: The ``pciBus`` attribute notes the index of the controller that an IOMMU device is attached to. (QEMU/KVM and ``smmuv3`` model only) + ``accel`` + The ``accel`` attribute with possible values ``on`` and ``off`` can be used + to enable hardware acceleration support for smmuv3 IOMMU devices. + (QEMU/KVM and ``smmuv3`` model only) + + ``ats`` + The ``ats`` attribute with possible values ``on`` and ``off`` can be used + to enable reporting Address Translation Services capability to the guest + for smmuv3 IOMMU devices with ``accel`` set to ``on``, if the host + SMMUv3 supports ATS and the associated passthrough device supports ATS. + (QEMU/KVM and ``smmuv3`` model only) + + ``ril`` + The ``ril`` attribute with possible values ``on`` and ``off`` can be used + to report whether Range Invalidation for IOMMU devices with + ``accel`` set to ``on`` is compatible with host SMMUv3 support. + (QEMU/KVM and ``smmuv3`` model only) + + ``ssidSize`` + The ``ssidSize`` attribute sets the number of bits used to represent + SubstreamIDs. A value of N allows SSIDs in the range [0 .. 2^N - 1]. + The valid range is 0-20, and a value greater than 0 is required for + enabling PASID support, as doing so advertises PASID capability to + the vIOMMU. (QEMU/KVM and ``smmuv3`` model only) + + ``oas`` + The ``oas`` attribute sets the output address size in units of bits. + (QEMU/KVM and ``smmuv3`` model only) + In case of ``virtio`` IOMMU device, the ``driver`` element can optionally contain ``granule`` subelement that allows to choose which granule will be used by default. It is useful when running guests with different page size diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index XXXXXXX..XXXXXXX 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefNew(void) iommu->pci_bus = -1; + iommu->ssid_size = -1; + + iommu->oas = -1; + return g_steal_pointer(&iommu); } @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefParseXML(virDomainXMLOption *xmlopt, &iommu->pci_bus, -1) < 0) return NULL; + if (virXMLPropTristateSwitch(driver, "accel", VIR_XML_PROP_NONE, + &iommu->accel) < 0) + return NULL; + + if (virXMLPropTristateSwitch(driver, "ats", VIR_XML_PROP_NONE, + &iommu->ats) < 0) + return NULL; + + if (virXMLPropTristateSwitch(driver, "ril", VIR_XML_PROP_NONE, + &iommu->ril) < 0) + return NULL; + + if (virXMLPropInt(driver, "ssidSize", 10, VIR_XML_PROP_NONE, + &iommu->ssid_size, -1) < 0) + return NULL; + + if (virXMLPropInt(driver, "oas", 10, VIR_XML_PROP_NONE, + &iommu->oas, -1) < 0) + return NULL; + if ((granule = virXPathNode("./driver/granule", ctxt))) { g_autofree char *mode = virXMLPropString(granule, "mode"); unsigned long long size; @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, a->iotlb != b->iotlb || a->aw_bits != b->aw_bits || a->dma_translation != b->dma_translation || + a->pci_bus != b->pci_bus || + a->accel != b->accel || + a->ats != b->ats || + a->ril != b->ril || + a->ssid_size != b->ssid_size || + a->oas != b->oas || a->xtsup != b->xtsup || a->pt != b->pt || a->granule != b->granule) @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefCheckABIStability(virDomainIOMMUDef *src, dst->pci_bus, src->pci_bus); return false; } + if (src->accel != dst->accel) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device accel value '%1$d' does not match source '%2$d'"), + dst->accel, src->accel); + return false; + } + if (src->ats != dst->ats) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device ATS value '%1$d' does not match source '%2$d'"), + dst->ats, src->ats); + return false; + } + if (src->ril != dst->ril) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device ril value '%1$d' does not match source '%2$d'"), + dst->ril, src->ril); + return false; + } + if (src->ssid_size != dst->ssid_size) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device ssid_size value '%1$d' does not match source '%2$d'"), + dst->ssid_size, src->ssid_size); + return false; + } + if (src->oas != dst->oas) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device oas value '%1$d' does not match source '%2$d'"), + dst->oas, src->oas); + return false; + } if (src->dma_translation != dst->dma_translation) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Target domain IOMMU device dma translation '%1$s' does not match source '%2$s'"), @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefFormat(virBuffer *buf, virBufferAsprintf(&driverAttrBuf, " pciBus='%d'", iommu->pci_bus); } + if (iommu->accel != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " accel='%s'", + virTristateSwitchTypeToString(iommu->accel)); + } + if (iommu->ats != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " ats='%s'", + virTristateSwitchTypeToString(iommu->ats)); + } + if (iommu->ril != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " ril='%s'", + virTristateSwitchTypeToString(iommu->ril)); + } + if (iommu->ssid_size >= 0) { + virBufferAsprintf(&driverAttrBuf, " ssidSize='%d'", + iommu->ssid_size); + } + if (iommu->oas >= 0) { + virBufferAsprintf(&driverAttrBuf, " oas='%d'", + iommu->oas); + } if (iommu->granule != 0) { if (iommu->granule == -1) { virBufferAddLit(&driverChildBuf, "<granule mode='host'/>\n"); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index XXXXXXX..XXXXXXX 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -XXX,XX +XXX,XX @@ struct _virDomainIOMMUDef { virTristateSwitch xtsup; virTristateSwitch pt; int granule; /* -1 means 'host', 0 unset, page size in KiB otherwise */ + virTristateSwitch accel; + virTristateSwitch ats; + virTristateSwitch ril; + int ssid_size; + int oas; }; typedef enum { diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index XXXXXXX..XXXXXXX 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) virDomainIOMMUModelTypeToString(iommu->model)); return -1; } + if (iommu->accel != VIR_TRISTATE_SWITCH_ON && + (iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("accel must be enabled for iommu model '%1$s' when setting ats, ril, ssidSize, or oas"), + virDomainIOMMUModelTypeToString(iommu->model)); + return -1; + } break; case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: if (iommu->intremap != VIR_TRISTATE_SWITCH_ABSENT || @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->eim != VIR_TRISTATE_SWITCH_ABSENT || iommu->iotlb != VIR_TRISTATE_SWITCH_ABSENT || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || - iommu->pci_bus >= 0) { + iommu->pci_bus >= 0 || + iommu->accel != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), virDomainIOMMUModelTypeToString(iommu->model)); @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->aw_bits != 0 || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->accel != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0 || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) if (iommu->pt != VIR_TRISTATE_SWITCH_ABSENT || iommu->xtsup != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->accel != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0 || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index XXXXXXX..XXXXXXX 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -XXX,XX +XXX,XX @@ <empty/> </element> </optional> + <optional> + <attribute name="accel"> + <ref name="virOnOff"/> + </attribute> + </optional> + <optional> + <attribute name="ats"> + <ref name="virOnOff"/> + </attribute> + </optional> + <optional> + <attribute name="ril"> + <ref name="virOnOff"/> + </attribute> + </optional> + <optional> + <attribute name="ssidSize"> + <data type="int"/> + </attribute> + </optional> + <optional> + <attribute name="oas"> + <data type="int"/> + </attribute> + </optional> </element> </optional> <optional> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index XXXXXXX..XXXXXXX 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -XXX,XX +XXX,XX @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, "s:driver", "arm-smmuv3", "s:primary-bus", bus, "s:id", iommu->info.alias, + "B:accel", (iommu->accel == VIR_TRISTATE_SWITCH_ON), + "T:ats", iommu->ats, + "T:ril", iommu->ril, + "k:ssidsize", iommu->ssid_size, + "k:oas", iommu->oas, NULL) < 0) return NULL; -- 2.43.0
From: Nathan Chen <nathanc@nvidia.com> Provide sample XML and CLI args for the HW-accel smmuv3 XML schema for virt machine types. Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- ...uv3-pci-bus-accel-auto.aarch64-latest.args | 38 +++++++++++ ...muv3-pci-bus-accel-auto.aarch64-latest.xml | 63 +++++++++++++++++++ .../iommu-smmuv3-pci-bus-accel-auto.xml | 53 ++++++++++++++++ ...u-smmuv3-pci-bus-accel.aarch64-latest.args | 38 +++++++++++ ...mu-smmuv3-pci-bus-accel.aarch64-latest.xml | 63 +++++++++++++++++++ .../iommu-smmuv3-pci-bus-accel.xml | 53 ++++++++++++++++ tests/qemuxmlconftest.c | 2 + 7 files changed, 310 insertions(+) create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args @@ -XXX,XX +XXX,XX @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-guest \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-guest/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-guest/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-guest/.config \ +/usr/bin/qemu-system-aarch64 \ +-name guest=guest,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-guest/master-key.aes"}' \ +-machine virt,usb=off,gic-version=2,dump-guest-core=off,memory-backend=mach-virt.ram,acpi=off \ +-accel tcg \ +-m size=1048576k \ +-object '{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":1073741824}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid 1ccfd97d-5eb4-478a-bbe6-88d254c16db7 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=@mon-fd@,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ +-device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ +-device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.3","bus":"pci.1","addr":"0x0"}' \ +-device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.4","bus":"pci.2","addr":"0x0"}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:00.0","id":"hostdev0","bus":"pci.3","addr":"0x0"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:01.0","id":"hostdev1","bus":"pci.4","addr":"0x0"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <gic version='2'/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' index='0' model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <audio id='1' type='none'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <memballoon model='none'/> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' model='none'/> + <memballoon model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args @@ -XXX,XX +XXX,XX @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-guest \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-guest/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-guest/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-guest/.config \ +/usr/bin/qemu-system-aarch64 \ +-name guest=guest,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-guest/master-key.aes"}' \ +-machine virt,usb=off,gic-version=2,dump-guest-core=off,memory-backend=mach-virt.ram,acpi=off \ +-accel tcg \ +-m size=1048576k \ +-object '{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":1073741824}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid 1ccfd97d-5eb4-478a-bbe6-88d254c16db7 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=@mon-fd@,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ +-device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ +-device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.3","bus":"pci.1","addr":"0x0"}' \ +-device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.4","bus":"pci.2","addr":"0x0"}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true,"ats":true,"ril":false,"ssidsize":20,"oas":44}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true,"ats":true,"ril":false,"ssidsize":20,"oas":44}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:00.0","id":"hostdev0","bus":"pci.3","addr":"0x0"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:01.0","id":"hostdev1","bus":"pci.4","addr":"0x0"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <gic version='2'/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' index='0' model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <audio id='1' type='none'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <memballoon model='none'/> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' model='none'/> + <memballoon model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconftest.c b/tests/qemuxmlconftest.c index XXXXXXX..XXXXXXX 100644 --- a/tests/qemuxmlconftest.c +++ b/tests/qemuxmlconftest.c @@ -XXX,XX +XXX,XX @@ mymain(void) DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3", "aarch64"); DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus", "aarch64"); DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-single", "aarch64"); + DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-accel", "aarch64"); + DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-accel-auto", "aarch64"); DO_TEST_CAPS_LATEST("virtio-iommu-x86_64"); DO_TEST_CAPS_ARCH_LATEST("virtio-iommu-aarch64", "aarch64"); DO_TEST_CAPS_LATEST_PARSE_ERROR("virtio-iommu-wrong-machine"); -- 2.43.0
Hi, This is a follow up to the eighth patchset [0] for supporting multiple hardware-accelerated vSMMU instances. For instance, specifying hostdevs associated with multiple accelerated SMMUs, configured to be routed to pcie-expander-bus controllers in a way where VFIO device to SMMUv3 associations are matched with the host: <devices> ... <controller type='pci' index='1' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='252'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </controller> <controller type='pci' index='2' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='248'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> ... <controller type='pci' index='21' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='21' port='0x0'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='22' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='22' port='0xa8'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> ... <hostdev mode='subsystem' type='pci' managed='no'> <source> <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='no'> <source> <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/> </hostdev> <iommu model='smmuv3'> <driver pciBus='1' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> </iommu> <iommu model='smmuv3'> <driver pciBus='2' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> </iommu> </devices> This would get translated to a qemu command line with the arguments below: -device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ -device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.21","bus":"pci.1","addr":"0x0"}' \ -device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.22","bus":"pci.2","addr":"0x0"}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true,"ats":"on","ril":"off","ssidsize":"20","oas":"44"}' \ -device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true,"ats":"on","ril":"off","ssidsize":"20","oas":"44"}' \ -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","bus":"pci.21","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","bus":"pci.22","addr":"0x0"}' \ Two QEMU series implementing auto support for accel SMMUv3 properties are now present in the upstream master branch [1][2]. This Libvirt series leverages the auto support: users can omit SMMUv3 properties ATS, RIL, OAS, and SSIDSIZE on the domain XML, and they will also be omitted on the QEMU command line as a result. With these new QEMU series applied, the feature property values will default to 'auto' to be resolved based on what the host SMMU supports. This series is on Github: https://github.com/NathanChenNVIDIA/libvirt/tree/smmuv3-accel-v9/ Thanks, Nathan [0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/XETQ62LKX5LNFBHL4Z3P7NEOT4IRJHJK/ [1] https://lore.kernel.org/all/20260323182454.1416110-1-nathanc@nvidia.com/ [2] https://lore.kernel.org/all/20260608174900.2227340-1-nathanc@nvidia.com/ Changes from v8: - Changed smmuv3 attribute QEMU CLI values to strings to match QEMU 11.0 Changes from v7: - Remove support for x-vpasid-cap-offset based on feedback pointing out the QEMU feature prefixed with x- is experimental and not to be supported in Libvirt - Implement support for auto SMMUv3 properties. Omitting these properties in the XML omits them on the QEMU command line as well, which results in default auto values being chosen for ATS, RIL, SSIDSIZE, and OAS - Add validation check that only allows enabling accel SMMUv3 features when accel is enabled - Add tests for accel=on but omitting accel SMMUv3 feature properties Changes from v6: - Remove smmuv3 'pasid' attribute - Add smmuv3 'ssidsize' attribute - Add support for 'vpasidCapOffset' element - Modify logic for building accel smmuv3 qemu command line to omit ATS, RIL, accel flags when not needed (enabled/disabled by default in QEMU SMMUv3) Nathan Chen (2): qemu: Add support for HW-accelerated nested SMMUv3 tests: qemuxmlconfdata: provide HW-accel smmuv3 sample XML and CLI args docs/formatdomain.rst | 29 +++++++ src/conf/domain_conf.c | 80 +++++++++++++++++++ src/conf/domain_conf.h | 5 ++ src/conf/domain_validate.c | 27 ++++++- src/conf/schemas/domaincommon.rng | 25 ++++++ src/qemu/qemu_command.c | 15 ++++ ...uv3-pci-bus-accel-auto.aarch64-latest.args | 38 +++++++++ ...muv3-pci-bus-accel-auto.aarch64-latest.xml | 63 +++++++++++++++ .../iommu-smmuv3-pci-bus-accel-auto.xml | 53 ++++++++++++ ...u-smmuv3-pci-bus-accel.aarch64-latest.args | 38 +++++++++ ...mu-smmuv3-pci-bus-accel.aarch64-latest.xml | 63 +++++++++++++++ .../iommu-smmuv3-pci-bus-accel.xml | 53 ++++++++++++ tests/qemuxmlconftest.c | 2 + 13 files changed, 490 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml -- 2.43.0
From: Nathan Chen <nathanc@nvidia.com> Add support for enabling HW-accelerated nested SMMUv3 via <accel> attribute and its additional attributes for ATS, SSID, RIL, and OAS configuration. Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- docs/formatdomain.rst | 29 +++++++++++ src/conf/domain_conf.c | 80 +++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 5 ++ src/conf/domain_validate.c | 27 ++++++++++- src/conf/schemas/domaincommon.rng | 25 ++++++++++ src/qemu/qemu_command.c | 15 ++++++ 6 files changed, 180 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index XXXXXXX..XXXXXXX 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -XXX,XX +XXX,XX @@ Examples: The ``pciBus`` attribute notes the index of the controller that an IOMMU device is attached to. (QEMU/KVM and ``smmuv3`` model only) + ``accel`` + The ``accel`` attribute with possible values ``on`` and ``off`` can be used + to enable hardware acceleration support for smmuv3 IOMMU devices. + (QEMU/KVM and ``smmuv3`` model only) + + ``ats`` + The ``ats`` attribute with possible values ``on`` and ``off`` can be used + to enable reporting Address Translation Services capability to the guest + for smmuv3 IOMMU devices with ``accel`` set to ``on``, if the host + SMMUv3 supports ATS and the associated passthrough device supports ATS. + (QEMU/KVM and ``smmuv3`` model only) + + ``ril`` + The ``ril`` attribute with possible values ``on`` and ``off`` can be used + to report whether Range Invalidation for IOMMU devices with + ``accel`` set to ``on`` is compatible with host SMMUv3 support. + (QEMU/KVM and ``smmuv3`` model only) + + ``ssidSize`` + The ``ssidSize`` attribute sets the number of bits used to represent + SubstreamIDs. A value of N allows SSIDs in the range [0 .. 2^N - 1]. + The valid range is 0-20, and a value greater than 0 is required for + enabling PASID support, as doing so advertises PASID capability to + the vIOMMU. (QEMU/KVM and ``smmuv3`` model only) + + ``oas`` + The ``oas`` attribute sets the output address size in units of bits. + (QEMU/KVM and ``smmuv3`` model only) + In case of ``virtio`` IOMMU device, the ``driver`` element can optionally contain ``granule`` subelement that allows to choose which granule will be used by default. It is useful when running guests with different page size diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index XXXXXXX..XXXXXXX 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefNew(void) iommu->pci_bus = -1; + iommu->ssid_size = -1; + + iommu->oas = -1; + return g_steal_pointer(&iommu); } @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefParseXML(virDomainXMLOption *xmlopt, &iommu->pci_bus, -1) < 0) return NULL; + if (virXMLPropTristateSwitch(driver, "accel", VIR_XML_PROP_NONE, + &iommu->accel) < 0) + return NULL; + + if (virXMLPropTristateSwitch(driver, "ats", VIR_XML_PROP_NONE, + &iommu->ats) < 0) + return NULL; + + if (virXMLPropTristateSwitch(driver, "ril", VIR_XML_PROP_NONE, + &iommu->ril) < 0) + return NULL; + + if (virXMLPropInt(driver, "ssidSize", 10, VIR_XML_PROP_NONE, + &iommu->ssid_size, -1) < 0) + return NULL; + + if (virXMLPropInt(driver, "oas", 10, VIR_XML_PROP_NONE, + &iommu->oas, -1) < 0) + return NULL; + if ((granule = virXPathNode("./driver/granule", ctxt))) { g_autofree char *mode = virXMLPropString(granule, "mode"); unsigned long long size; @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefEquals(const virDomainIOMMUDef *a, a->iotlb != b->iotlb || a->aw_bits != b->aw_bits || a->dma_translation != b->dma_translation || + a->pci_bus != b->pci_bus || + a->accel != b->accel || + a->ats != b->ats || + a->ril != b->ril || + a->ssid_size != b->ssid_size || + a->oas != b->oas || a->xtsup != b->xtsup || a->pt != b->pt || a->granule != b->granule) @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefCheckABIStability(virDomainIOMMUDef *src, dst->pci_bus, src->pci_bus); return false; } + if (src->accel != dst->accel) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device accel value '%1$d' does not match source '%2$d'"), + dst->accel, src->accel); + return false; + } + if (src->ats != dst->ats) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device ATS value '%1$d' does not match source '%2$d'"), + dst->ats, src->ats); + return false; + } + if (src->ril != dst->ril) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device ril value '%1$d' does not match source '%2$d'"), + dst->ril, src->ril); + return false; + } + if (src->ssid_size != dst->ssid_size) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device ssid_size value '%1$d' does not match source '%2$d'"), + dst->ssid_size, src->ssid_size); + return false; + } + if (src->oas != dst->oas) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("Target domain IOMMU device oas value '%1$d' does not match source '%2$d'"), + dst->oas, src->oas); + return false; + } if (src->dma_translation != dst->dma_translation) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Target domain IOMMU device dma translation '%1$s' does not match source '%2$s'"), @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefFormat(virBuffer *buf, virBufferAsprintf(&driverAttrBuf, " pciBus='%d'", iommu->pci_bus); } + if (iommu->accel != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " accel='%s'", + virTristateSwitchTypeToString(iommu->accel)); + } + if (iommu->ats != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " ats='%s'", + virTristateSwitchTypeToString(iommu->ats)); + } + if (iommu->ril != VIR_TRISTATE_SWITCH_ABSENT) { + virBufferAsprintf(&driverAttrBuf, " ril='%s'", + virTristateSwitchTypeToString(iommu->ril)); + } + if (iommu->ssid_size >= 0) { + virBufferAsprintf(&driverAttrBuf, " ssidSize='%d'", + iommu->ssid_size); + } + if (iommu->oas >= 0) { + virBufferAsprintf(&driverAttrBuf, " oas='%d'", + iommu->oas); + } if (iommu->granule != 0) { if (iommu->granule == -1) { virBufferAddLit(&driverChildBuf, "<granule mode='host'/>\n"); diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index XXXXXXX..XXXXXXX 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -XXX,XX +XXX,XX @@ struct _virDomainIOMMUDef { virTristateSwitch xtsup; virTristateSwitch pt; int granule; /* -1 means 'host', 0 unset, page size in KiB otherwise */ + virTristateSwitch accel; + virTristateSwitch ats; + virTristateSwitch ril; + int ssid_size; + int oas; }; typedef enum { diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c index XXXXXXX..XXXXXXX 100644 --- a/src/conf/domain_validate.c +++ b/src/conf/domain_validate.c @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) virDomainIOMMUModelTypeToString(iommu->model)); return -1; } + if (iommu->accel != VIR_TRISTATE_SWITCH_ON && + (iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("accel must be enabled for iommu model '%1$s' when setting ats, ril, ssidSize, or oas"), + virDomainIOMMUModelTypeToString(iommu->model)); + return -1; + } break; case VIR_DOMAIN_IOMMU_MODEL_VIRTIO: if (iommu->intremap != VIR_TRISTATE_SWITCH_ABSENT || @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->eim != VIR_TRISTATE_SWITCH_ABSENT || iommu->iotlb != VIR_TRISTATE_SWITCH_ABSENT || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || - iommu->pci_bus >= 0) { + iommu->pci_bus >= 0 || + iommu->accel != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), virDomainIOMMUModelTypeToString(iommu->model)); @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) iommu->aw_bits != 0 || iommu->dma_translation != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->accel != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0 || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), @@ -XXX,XX +XXX,XX @@ virDomainIOMMUDefValidate(const virDomainIOMMUDef *iommu) if (iommu->pt != VIR_TRISTATE_SWITCH_ABSENT || iommu->xtsup != VIR_TRISTATE_SWITCH_ABSENT || iommu->pci_bus >= 0 || + iommu->accel != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ats != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ril != VIR_TRISTATE_SWITCH_ABSENT || + iommu->ssid_size >= 0 || + iommu->oas >= 0 || iommu->granule != 0) { virReportError(VIR_ERR_XML_ERROR, _("iommu model '%1$s' doesn't support some additional attributes"), diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index XXXXXXX..XXXXXXX 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -XXX,XX +XXX,XX @@ <empty/> </element> </optional> + <optional> + <attribute name="accel"> + <ref name="virOnOff"/> + </attribute> + </optional> + <optional> + <attribute name="ats"> + <ref name="virOnOff"/> + </attribute> + </optional> + <optional> + <attribute name="ril"> + <ref name="virOnOff"/> + </attribute> + </optional> + <optional> + <attribute name="ssidSize"> + <data type="int"/> + </attribute> + </optional> + <optional> + <attribute name="oas"> + <data type="int"/> + </attribute> + </optional> </element> </optional> <optional> diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index XXXXXXX..XXXXXXX 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -XXX,XX +XXX,XX @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, { g_autoptr(virJSONValue) props = NULL; g_autofree char *bus = NULL; + g_autofree char *ssidsizeStr = NULL; + g_autofree char *oasStr = NULL; virPCIDeviceAddress addr = { .bus = iommu->pci_bus }; bus = qemuBuildDeviceAddressPCIGetBus(def, &addr); @@ -XXX,XX +XXX,XX @@ qemuBuildPCINestedSmmuv3DevProps(const virDomainDef *def, return NULL; } + if (iommu->ssid_size >= 0) { + ssidsizeStr = g_strdup_printf("%u", iommu->ssid_size); + } + + if (iommu->oas >= 0) { + oasStr = g_strdup_printf("%u", iommu->oas); + } + if (virJSONValueObjectAdd(&props, "s:driver", "arm-smmuv3", "s:primary-bus", bus, "s:id", iommu->info.alias, + "B:accel", (iommu->accel == VIR_TRISTATE_SWITCH_ON), + "S:ats", iommu->ats == VIR_TRISTATE_SWITCH_ON ? "on" : iommu->ats == VIR_TRISTATE_SWITCH_OFF ? "off" : NULL, + "S:ril", iommu->ril == VIR_TRISTATE_SWITCH_ON ? "on" : iommu->ril == VIR_TRISTATE_SWITCH_OFF ? "off" : NULL, + "S:ssidsize", ssidsizeStr, + "S:oas", oasStr, NULL) < 0) return NULL; -- 2.43.0
From: Nathan Chen <nathanc@nvidia.com> Provide sample XML and CLI args for the HW-accel smmuv3 XML schema for virt machine types. Signed-off-by: Nathan Chen <nathanc@nvidia.com> --- ...uv3-pci-bus-accel-auto.aarch64-latest.args | 38 +++++++++++ ...muv3-pci-bus-accel-auto.aarch64-latest.xml | 63 +++++++++++++++++++ .../iommu-smmuv3-pci-bus-accel-auto.xml | 53 ++++++++++++++++ ...u-smmuv3-pci-bus-accel.aarch64-latest.args | 38 +++++++++++ ...mu-smmuv3-pci-bus-accel.aarch64-latest.xml | 63 +++++++++++++++++++ .../iommu-smmuv3-pci-bus-accel.xml | 53 ++++++++++++++++ tests/qemuxmlconftest.c | 2 + 7 files changed, 310 insertions(+) create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.args @@ -XXX,XX +XXX,XX @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-guest \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-guest/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-guest/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-guest/.config \ +/usr/bin/qemu-system-aarch64 \ +-name guest=guest,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-guest/master-key.aes"}' \ +-machine virt,usb=off,gic-version=2,dump-guest-core=off,memory-backend=mach-virt.ram,acpi=off \ +-accel tcg \ +-m size=1048576k \ +-object '{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":1073741824}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid 1ccfd97d-5eb4-478a-bbe6-88d254c16db7 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=@mon-fd@,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ +-device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ +-device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.3","bus":"pci.1","addr":"0x0"}' \ +-device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.4","bus":"pci.2","addr":"0x0"}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:00.0","id":"hostdev0","bus":"pci.3","addr":"0x0"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:01.0","id":"hostdev1","bus":"pci.4","addr":"0x0"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.aarch64-latest.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <gic version='2'/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' index='0' model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <audio id='1' type='none'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <memballoon model='none'/> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-auto.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' model='none'/> + <memballoon model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.args @@ -XXX,XX +XXX,XX @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/var/lib/libvirt/qemu/domain--1-guest \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/var/lib/libvirt/qemu/domain--1-guest/.local/share \ +XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain--1-guest/.cache \ +XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-guest/.config \ +/usr/bin/qemu-system-aarch64 \ +-name guest=guest,debug-threads=on \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain--1-guest/master-key.aes"}' \ +-machine virt,usb=off,gic-version=2,dump-guest-core=off,memory-backend=mach-virt.ram,acpi=off \ +-accel tcg \ +-m size=1048576k \ +-object '{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":1073741824}' \ +-overcommit mem-lock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid 1ccfd97d-5eb4-478a-bbe6-88d254c16db7 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,fd=@mon-fd@,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot strict=on \ +-device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \ +-device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \ +-device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.3","bus":"pci.1","addr":"0x0"}' \ +-device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.4","bus":"pci.2","addr":"0x0"}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true,"ats":"on","ril":"off","ssidsize":"20","oas":"44"}' \ +-device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true,"ats":"on","ril":"off","ssidsize":"20","oas":"44"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:00.0","id":"hostdev0","bus":"pci.3","addr":"0x0"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:01.0","id":"hostdev1","bus":"pci.4","addr":"0x0"}' \ +-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ +-msg timestamp=on diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.aarch64-latest.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <gic version='2'/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' index='0' model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <audio id='1' type='none'/> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <memballoon model='none'/> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml new file mode 100644 index XXXXXXX..XXXXXXX --- /dev/null +++ b/tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel.xml @@ -XXX,XX +XXX,XX @@ +<domain type='qemu'> + <name>guest</name> + <uuid>1ccfd97d-5eb4-478a-bbe6-88d254c16db7</uuid> + <memory unit='KiB'>1048576</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='aarch64' machine='virt'>hvm</type> + </os> + <devices> + <emulator>/usr/bin/qemu-system-aarch64</emulator> + <controller type='usb' model='none'/> + <memballoon model='none'/> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='252'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pcie-expander-bus'> + <model name='pxb-pcie'/> + <target busNr='248'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </controller> + <controller type='pci' index='3' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='21' port='0x0'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <controller type='pci' index='4' model='pcie-root-port'> + <model name='pcie-root-port'/> + <target chassis='22' port='0xa8'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> + </controller> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> + </hostdev> + <hostdev mode='subsystem' type='pci' managed='no'> + <source> + <address domain='0x0000' bus='0x06' slot='0x01' function='0x0'/> + </source> + <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> + </hostdev> + <iommu model='smmuv3'> + <driver pciBus='1' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + <iommu model='smmuv3'> + <driver pciBus='2' accel='on' ats='on' ril='off' ssidSize='20' oas='44'/> + </iommu> + </devices> +</domain> diff --git a/tests/qemuxmlconftest.c b/tests/qemuxmlconftest.c index XXXXXXX..XXXXXXX 100644 --- a/tests/qemuxmlconftest.c +++ b/tests/qemuxmlconftest.c @@ -XXX,XX +XXX,XX @@ mymain(void) DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3", "aarch64"); DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus", "aarch64"); DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-single", "aarch64"); + DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-accel", "aarch64"); + DO_TEST_CAPS_ARCH_LATEST("iommu-smmuv3-pci-bus-accel-auto", "aarch64"); DO_TEST_CAPS_LATEST("virtio-iommu-x86_64"); DO_TEST_CAPS_ARCH_LATEST("virtio-iommu-aarch64", "aarch64"); DO_TEST_CAPS_LATEST_PARSE_ERROR("virtio-iommu-wrong-machine"); -- 2.43.0