[PATCH v4 0/3] qemu: Support vCMDQ

Nathan Chen via Devel posted 3 patches 5 days, 14 hours ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20260818183012.1260768-1-nathanc@nvidia.com
docs/formatdomain.rst                         |  8 +++
src/conf/domain_conf.c                        | 15 +++++
src/conf/domain_conf.h                        |  1 +
src/conf/domain_validate.c                    |  6 +-
src/conf/schemas/domaincommon.rng             |  5 ++
src/qemu/qemu_capabilities.c                  |  4 ++
src/qemu/qemu_capabilities.h                  |  3 +
src/qemu/qemu_command.c                       |  1 +
src/qemu/qemu_validate.c                      |  8 +++
.../caps_11.1.0_aarch64.xml                   |  1 +
...v3-pci-bus-accel-cmdqv.aarch64-latest.args | 40 +++++++++++++
...uv3-pci-bus-accel-cmdqv.aarch64-latest.xml | 59 +++++++++++++++++++
.../iommu-smmuv3-pci-bus-accel-cmdqv.xml      | 49 +++++++++++++++
tests/qemuxmlconftest.c                       |  1 +
14 files changed, 200 insertions(+), 1 deletion(-)
create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-cmdqv.aarch64-latest.args
create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-cmdqv.aarch64-latest.xml
create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-cmdqv.xml
[PATCH v4 0/3] qemu: Support vCMDQ
Posted by Nathan Chen via Devel 5 days, 14 hours ago
Hi,

This is a follow up to the previous patchset [0] for supporting Tegra241
vCMDQ.

This patchset is tested with Shameer's qemu series for vCMDQ support.
The accelerated SMMUv3 series for qemu and Libvirt are
pre-requisites for vCMDQ support, as accelerated SMMUv3 must be enabled
alongside vCMDQ.

A new 'cmdqv' <iommu> driver attribute is introduced to toggle enabling
vCMDQ support. For instance, specifying hostdevs associated with
multiple accelerated SMMUs + CMDQV enabled, configured to be routed to
pcie-expander-bus controllers in a way where VFIO device to SMMUv3
associations are matched with the host:

  <devices>
...
    <controller type='pci' index='1' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='252'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='248'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
...
    <controller type='pci' index='21' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='21' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='22' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='22' port='0xa8'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
...
    <hostdev mode='subsystem' type='pci' managed='no'>
      <source>
        <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='no'>
      <source>
        <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/>
    </hostdev>
    <iommu model='smmuv3'>
      <driver pciBus='1' accel='on' ats='on' ril='on' ssidsize='20' oas='44' cmdqv='on'/>
    </iommu>
    <iommu model='smmuv3'>
      <driver pciBus='2' accel='on' ats='on' ril='on' ssidsize='20' oas='44' cmdqv='on'/>
    </iommu>
  </devices>

This would get translated to a qemu command line with the arguments
below:

 -device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
 -device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \
 -device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.21","bus":"pci.1","addr":"0x0"}' \
 -device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.22","bus":"pci.2","addr":"0x0"}' \
 -device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true,"ats":true,"cmdqv":"on","ril":true,"pasid":true,"oas":44}' \
 -device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true,"ats":true,"cmdqv":"on","ril":true,"pasid":true,"oas":44}' \
 -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","bus":"pci.21","addr":"0x0"}' \
 -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","bus":"pci.22","addr":"0x0"}' \

This series is on Github:
https://github.com/NathanChenNVIDIA/libvirt/tree/cmdqv-v4/

Thanks,
Nathan

Changes from v3:
- Move qemuValidateDomainDeviceDefIOMMU() check from patch 1 to 2
- Mention default hypervisor value will be used in formatdomain.rst if
  cmdqv is omitted
- Make cmdqv virDomainIOMMUDefValidate() check aligned with order of all
  cmdqv checks in the function
- Include cmdqv for accel validation error message in domain_validate.c

Changes from v2:
- Add QEMU capabilities support for cmdqv
- Change QEMU command line parameter to expect OnOffAuto string

Changes from v1:
- Remove 'identifier' attribute for SMMUv3
- Change QEMU command line property from 'tegra241-cmdqv' to 'cmdqv'
- Support QEMU OnOffAuto CMDQV property instead of expecting a bool

[0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/NGSB6ECUQR6RPL7WQQRTNCSDZTSECPLH/

Nathan Chen (3):
  qemu: introduce QEMU_CAPS_ARM_SMMUV3_CMDQV
  qemu: add IOMMU attribute "cmdqv" for smmuv3
  tests: qemuxmlconfdata: provide cmdqv sample XML and CLI args

 docs/formatdomain.rst                         |  8 +++
 src/conf/domain_conf.c                        | 15 +++++
 src/conf/domain_conf.h                        |  1 +
 src/conf/domain_validate.c                    |  6 +-
 src/conf/schemas/domaincommon.rng             |  5 ++
 src/qemu/qemu_capabilities.c                  |  4 ++
 src/qemu/qemu_capabilities.h                  |  3 +
 src/qemu/qemu_command.c                       |  1 +
 src/qemu/qemu_validate.c                      |  8 +++
 .../caps_11.1.0_aarch64.xml                   |  1 +
 ...v3-pci-bus-accel-cmdqv.aarch64-latest.args | 40 +++++++++++++
 ...uv3-pci-bus-accel-cmdqv.aarch64-latest.xml | 59 +++++++++++++++++++
 .../iommu-smmuv3-pci-bus-accel-cmdqv.xml      | 49 +++++++++++++++
 tests/qemuxmlconftest.c                       |  1 +
 14 files changed, 200 insertions(+), 1 deletion(-)
 create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-cmdqv.aarch64-latest.args
 create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-cmdqv.aarch64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/iommu-smmuv3-pci-bus-accel-cmdqv.xml

-- 
2.43.0
Re: [PATCH v4 0/3] qemu: Support vCMDQ
Posted by Pavel Hrdina via Devel 4 days, 19 hours ago
On Tue, Aug 18, 2026 at 11:30:09AM -0700, Nathan Chen via Devel wrote:
> Hi,
> 
> This is a follow up to the previous patchset [0] for supporting Tegra241
> vCMDQ.
> 
> This patchset is tested with Shameer's qemu series for vCMDQ support.
> The accelerated SMMUv3 series for qemu and Libvirt are
> pre-requisites for vCMDQ support, as accelerated SMMUv3 must be enabled
> alongside vCMDQ.
> 
> A new 'cmdqv' <iommu> driver attribute is introduced to toggle enabling
> vCMDQ support. For instance, specifying hostdevs associated with
> multiple accelerated SMMUs + CMDQV enabled, configured to be routed to
> pcie-expander-bus controllers in a way where VFIO device to SMMUv3
> associations are matched with the host:
> 
>   <devices>
> ...
>     <controller type='pci' index='1' model='pcie-expander-bus'>
>       <model name='pxb-pcie'/>
>       <target busNr='252'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
>     </controller>
>     <controller type='pci' index='2' model='pcie-expander-bus'>
>       <model name='pxb-pcie'/>
>       <target busNr='248'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
>     </controller>
> ...
>     <controller type='pci' index='21' model='pcie-root-port'>
>       <model name='pcie-root-port'/>
>       <target chassis='21' port='0x0'/>
>       <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
>     </controller>
>     <controller type='pci' index='22' model='pcie-root-port'>
>       <model name='pcie-root-port'/>
>       <target chassis='22' port='0xa8'/>
>       <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
>     </controller>
> ...
>     <hostdev mode='subsystem' type='pci' managed='no'>
>       <source>
>         <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/>
>       </source>
>       <address type='pci' domain='0x0000' bus='0x15' slot='0x00' function='0x0'/>
>     </hostdev>
>     <hostdev mode='subsystem' type='pci' managed='no'>
>       <source>
>         <address domain='0x0019' bus='0x01' slot='0x00' function='0x0'/>
>       </source>
>       <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/>
>     </hostdev>
>     <iommu model='smmuv3'>
>       <driver pciBus='1' accel='on' ats='on' ril='on' ssidsize='20' oas='44' cmdqv='on'/>
>     </iommu>
>     <iommu model='smmuv3'>
>       <driver pciBus='2' accel='on' ats='on' ril='on' ssidsize='20' oas='44' cmdqv='on'/>
>     </iommu>
>   </devices>
> 
> This would get translated to a qemu command line with the arguments
> below:
> 
>  -device '{"driver":"pxb-pcie","bus_nr":252,"id":"pci.1","bus":"pcie.0","addr":"0x1"}' \
>  -device '{"driver":"pxb-pcie","bus_nr":248,"id":"pci.2","bus":"pcie.0","addr":"0x2"}' \
>  -device '{"driver":"pcie-root-port","port":0,"chassis":21,"id":"pci.21","bus":"pci.1","addr":"0x0"}' \
>  -device '{"driver":"pcie-root-port","port":168,"chassis":22,"id":"pci.22","bus":"pci.2","addr":"0x0"}' \
>  -device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"iommu0","accel":true,"ats":true,"cmdqv":"on","ril":true,"pasid":true,"oas":44}' \
>  -device '{"driver":"arm-smmuv3","primary-bus":"pci.2","id":"iommu1","accel":true,"ats":true,"cmdqv":"on","ril":true,"pasid":true,"oas":44}' \
>  -device '{"driver":"vfio-pci","host":"0009:01:00.0","id":"hostdev0","bus":"pci.21","addr":"0x0"}' \
>  -device '{"driver":"vfio-pci","host":"0019:01:00.0","id":"hostdev1","bus":"pci.22","addr":"0x0"}' \
> 
> This series is on Github:
> https://github.com/NathanChenNVIDIA/libvirt/tree/cmdqv-v4/
> 
> Thanks,
> Nathan
> 
> Changes from v3:
> - Move qemuValidateDomainDeviceDefIOMMU() check from patch 1 to 2
> - Mention default hypervisor value will be used in formatdomain.rst if
>   cmdqv is omitted
> - Make cmdqv virDomainIOMMUDefValidate() check aligned with order of all
>   cmdqv checks in the function
> - Include cmdqv for accel validation error message in domain_validate.c
> 
> Changes from v2:
> - Add QEMU capabilities support for cmdqv
> - Change QEMU command line parameter to expect OnOffAuto string
> 
> Changes from v1:
> - Remove 'identifier' attribute for SMMUv3
> - Change QEMU command line property from 'tegra241-cmdqv' to 'cmdqv'
> - Support QEMU OnOffAuto CMDQV property instead of expecting a bool
> 
> [0] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/NGSB6ECUQR6RPL7WQQRTNCSDZTSECPLH/
> 
> Nathan Chen (3):
>   qemu: introduce QEMU_CAPS_ARM_SMMUV3_CMDQV
>   qemu: add IOMMU attribute "cmdqv" for smmuv3
>   tests: qemuxmlconfdata: provide cmdqv sample XML and CLI args

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>