Here is a draft patch series for amd-iommu device. It will implement
amd-iommu device and its attributes: intremap, device-iotlb, xtsup.

However, its secret device AMDVI-PCI will occupy a PCIe slot and cause
VM failed to start:
Domain XML:
<domain type='kvm'>
...
    <video>
      <model type='virtio' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <iommu model='amd'/>
...
</domain>

➜  ~ virsh create /tmp/fedora.xml
error: Failed to create domain from /tmp/fedora.xml
error: internal error: QEMU unexpectedly closed the monitor (vm='fedora'): 2024-09-27T07:55:46.132886Z qemu-system-x86_64: -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"}: PCI: slot 1 function 0 not available for virtio-vga, in use by AMDVI-PCI,id=(null)

I'll update the series when QEMU fixes the blocker.

Han Han (5):
  qemu_capabilities: Introduce QEMU_CAPS_AMD_IOMMU
  qemu_capabilities: Introduce QEMU_CAPS_AMD_IOMMU_INTREMAP
  qemu_capabilities: Introduce QEMU_CAPS_AMD_IOMMU_DEVICE_IOTLB
  qemu_capabilities: Introduce QEMU_CAPS_AMD_IOMMU_XTSUP
  qemu: Add suppport for amd-iommu device and xtsup attrib

 docs/formatdomain.rst                         | 20 +++-
 src/conf/domain_conf.c                        | 16 +++
 src/conf/domain_conf.h                        |  2 +
 src/conf/domain_validate.c                    | 39 +++++++-
 src/conf/schemas/domaincommon.rng             |  6 ++
 src/qemu/qemu_capabilities.c                  | 16 +++
 src/qemu/qemu_capabilities.h                  |  6 ++
 src/qemu/qemu_command.c                       | 15 +++
 src/qemu/qemu_domain_address.c                |  2 +
 src/qemu/qemu_validate.c                      |  1 +
 .../caps_5.2.0_x86_64.replies                 | 90 +++++++++++------
 .../caps_5.2.0_x86_64.xml                     |  3 +
 .../caps_6.0.0_x86_64.replies                 | 90 +++++++++++------
 .../caps_6.0.0_x86_64.xml                     |  3 +
 .../caps_6.1.0_x86_64.replies                 | 90 +++++++++++------
 .../caps_6.1.0_x86_64.xml                     |  3 +
 .../caps_6.2.0_x86_64.xml                     |  3 +
 .../caps_7.0.0_x86_64.replies                 | 94 ++++++++++++------
 .../caps_7.0.0_x86_64.xml                     |  3 +
 .../caps_7.1.0_x86_64.replies                 | 94 ++++++++++++------
 .../caps_7.1.0_x86_64.xml                     |  3 +
 .../caps_7.2.0_x86_64+hvf.replies             | 94 ++++++++++++------
 .../caps_7.2.0_x86_64+hvf.xml                 |  3 +
 .../caps_7.2.0_x86_64.replies                 | 94 ++++++++++++------
 .../caps_7.2.0_x86_64.xml                     |  3 +
 .../caps_8.0.0_x86_64.replies                 | 94 ++++++++++++------
 .../caps_8.0.0_x86_64.xml                     |  3 +
 .../caps_8.1.0_x86_64.replies                 | 90 +++++++++++------
 .../caps_8.1.0_x86_64.xml                     |  3 +
 .../caps_8.2.0_x86_64.replies                 | 86 ++++++++++------
 .../caps_8.2.0_x86_64.xml                     |  3 +
 .../caps_9.0.0_x86_64.replies                 | 99 +++++++++++++------
 .../caps_9.0.0_x86_64.xml                     |  4 +
 .../caps_9.1.0_x86_64.replies                 | 79 ++++++++++-----
 .../caps_9.1.0_x86_64.xml                     |  4 +
 tests/qemuxmlconfdata/amd-iommu-aw-bits.err   |  1 +
 tests/qemuxmlconfdata/amd-iommu-aw-bits.xml   | 32 ++++++
 .../amd-iommu-caching-mode.err                |  1 +
 .../amd-iommu-caching-mode.xml                | 32 ++++++
 .../amd-iommu-device-iotlb.x86_64-latest.args | 34 +++++++
 .../amd-iommu-device-iotlb.xml                | 37 +++++++
 .../amd-iommu-dma-translation.err             |  1 +
 .../amd-iommu-dma-translation.xml             | 32 ++++++
 tests/qemuxmlconfdata/amd-iommu-eim.err       |  1 +
 tests/qemuxmlconfdata/amd-iommu-eim.xml       | 32 ++++++
 .../amd-iommu-xtsup.x86_64-latest.args        | 34 +++++++
 tests/qemuxmlconfdata/amd-iommu-xtsup.xml     | 32 ++++++
 .../amd-iommu.x86_64-latest.args              | 34 +++++++
 tests/qemuxmlconfdata/amd-iommu.xml           | 32 ++++++
 tests/qemuxmlconfdata/intel-iommu-xtsup.err   |  1 +
 tests/qemuxmlconfdata/intel-iommu-xtsup.xml   | 38 +++++++
 tests/qemuxmlconftest.c                       |  8 ++
 52 files changed, 1275 insertions(+), 365 deletions(-)
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-aw-bits.err
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-aw-bits.xml
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-caching-mode.err
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-caching-mode.xml
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-device-iotlb.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-device-iotlb.xml
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-dma-translation.err
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-dma-translation.xml
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-eim.err
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-eim.xml
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-xtsup.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/amd-iommu-xtsup.xml
 create mode 100644 tests/qemuxmlconfdata/amd-iommu.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/amd-iommu.xml
 create mode 100644 tests/qemuxmlconfdata/intel-iommu-xtsup.err
 create mode 100644 tests/qemuxmlconfdata/intel-iommu-xtsup.xml

-- 
2.46.2

On Fri, Sep 27, 2024 at 03:45:41PM +0800, Han Han wrote:
> Here is a draft patch series for amd-iommu device. It will implement
> amd-iommu device and its attributes: intremap, device-iotlb, xtsup.
> 
> However, its secret device AMDVI-PCI will occupy a PCIe slot and cause
> VM failed to start:
> Domain XML:
> <domain type='kvm'>
> ...
>     <video>
>       <model type='virtio' heads='1' primary='yes'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
>     </video>
>     <iommu model='amd'/>
> ...
> </domain>
> 
> ➜  ~ virsh create /tmp/fedora.xml
> error: Failed to create domain from /tmp/fedora.xml
> error: internal error: QEMU unexpectedly closed the monitor (vm='fedora'): 2024-09-27T07:55:46.132886Z qemu-system-x86_64: -device {"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"}: PCI: slot 1 function 0 not available for virtio-vga, in use by AMDVI-PCI,id=(null)
> 
> I'll update the series when QEMU fixes the blocker.

Is anyone even working on this problem in QEMU ?  I don't recall
seeing any patchset that claimed to address this.

The amd-iommu device as it exists today is completely unsupportable,
due to it secretly creating this PCI device behind the scenes, making
it impossible for libvirt to configure PCI placement.

IOW, this series is pretty much useless until the amd-iommu device
is re-designed, at which point you'll need to re-write it to cope
with the new design which properly exposes the PCI device (or
alternatively removes the need for a secret PCI device?).

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|