>-----Original Message-----
>From: Eric Auger <eric.auger@redhat.com>
>Subject: Re: [PATCH v6 00/19] Add a host IOMMU device abstraction to
>check with vIOMMU
>
>Hi Zhenzhong,
>
>On 6/3/24 08:10, Zhenzhong Duan wrote:
>> Hi,
>>
>> This series introduce a HostIOMMUDevice abstraction and sub-classes.
>> Also HostIOMMUDeviceCaps structure in HostIOMMUDevice and a new
>interface
>> between vIOMMU and HostIOMMUDevice.
>
>I think we should have a textual description of what is a
>HostIOMMUDevice. Because to me the terminology may be confusing as the
>reader can understand this is an abstraction for the physical IOMMU.
>
>Would it be correct to say:
>
>A HostIOMMUDevice is an abstraction for an assigned device that is
>protected by a physical IOMMU (aka host IOMMU). The userspace
>interaction with this physical IOMMU can be done either through the VFIO
>IOMMU type 1 legacy backend or the new iommufd backend. The assigned
>device can be a VFIO device or a VDPA device. The HostIOMMUDevice is
>needed to interact with the host IOMMU that protects the assigned
>device. It is especially useful when the device is also protected by a
>virtual IOMMU as this latter use the translation services of the
>physical IOMMU and is constraained by it. In that context the
>HostIOMMUDevice can be passed to the virtual IOMMU to collect physical
>IOMMU capabilities such as the supported address width. In the future,
>the virtual IOMMU will use the HostIOMMUDevice to program the guest
>page
>tables in the first translation stage of the physical IOMMU.

Great, thanks Eric.

>
>If such kind of description is correct, I would also suggest to embed it
>in the patch 1 commit msg.

Sure.

Thanks
Zhenzhong

>
>Thanks
>
>Eric
>
>
>>
>> HostIOMMUDeviceClass::realize() is introduced to initialize
>> HostIOMMUDeviceCaps and other fields of HostIOMMUDevice variants.
>>
>> HostIOMMUDeviceClass::get_cap() is introduced to query host IOMMU
>> device capabilities.
>>
>> The class tree is as below:
>>
>>                               HostIOMMUDevice
>>                                      | .caps
>>                                      | .realize()
>>                                      | .get_cap()
>>                                      |
>>             .-----------------------------------------------.
>>             |                        |                      |
>> HostIOMMUDeviceLegacyVFIO  {HostIOMMUDeviceLegacyVDPA}
>HostIOMMUDeviceIOMMUFD
>>             |                        |                      | [.iommufd]
>>                                                             | [.devid]
>>                                                             | [.ioas_id]
>>                                                             | [.attach_hwpt()]
>>                                                             | [.detach_hwpt()]
>>                                                             |
>>                                             .----------------------.
>>                                             |                      |
>>                          HostIOMMUDeviceIOMMUFDVFIO
>{HostIOMMUDeviceIOMMUFDVDPA}
>>                                           | [.vdev]                | {.vdev}
>>
>> * The attributes in [] will be implemented in nesting series.
>> * The classes in {} will be implemented in future.
>> * .vdev in different class points to different agent device,
>> * i.e., for VFIO it points to VFIODevice.
>>
>> PATCH1-4: Introduce HostIOMMUDevice and its sub classes
>> PATCH5-11: Introduce HostIOMMUDeviceCaps, implement .realize()
>and .get_cap() handler
>> PATCH12-16: Create HostIOMMUDevice instance and pass to vIOMMU
>> PATCH17-19: Implement compatibility check between host IOMMU and
>vIOMMU(intel_iommu)
>>
>> Test done:
>> make check
>> vfio device hotplug/unplug with different backend on linux
>> reboot
>> build test on linux and windows11
>>
>> Qemu code can be found at:
>>
>https://github.com/yiliu1765/qemu/tree/zhenzhong/iommufd_nesting_pre
>q_v6
>>
>> Besides the compatibility check in this series, in nesting series, this
>> host IOMMU device is extended for much wider usage. For anyone
>interested
>> on the nesting series, here is the link:
>>
>https://github.com/yiliu1765/qemu/tree/zhenzhong/iommufd_nesting_rfc
>v2
>>
>> Thanks
>> Zhenzhong
>>
>> Changelog:
>> v6:
>> - Open coded host_iommu_device_get_cap() to avoid #ifdef in
>intel_iommu.c (Cédric)
>>
>> v5:
>> - pci_device_set_iommu_device return true (Cédric)
>> - fix build failure on windows (thanks Cédric found that issue)
>>
>> v4:
>> - move properties vdev, iommufd and devid to nesting series where need it
>(Cédric)
>> - fix 32bit build with clz64 (Cédric)
>> - change check_cap naming to get_cap (Cédric)
>> - return bool if error is passed through errp (Cédric)
>> - drop HostIOMMUDevice[LegacyVFIO|IOMMUFD|IOMMUFDVFIO]
>declaration (Cédric)
>> - drop HOST_IOMMU_DEVICE_CAP_IOMMUFD (Cédric)
>> - replace include directive with forward declaration (Cédric)
>>
>> v3:
>> - refine declaration and doc for HostIOMMUDevice (Cédric, Philippe)
>> - introduce HostIOMMUDeviceCaps, .realize() and .check_cap() (Cédric)
>> - introduce helper range_get_last_bit() for range operation (Cédric)
>> - separate pci_device_get_iommu_bus_devfn() in a prereq patch (Cédric)
>> - replace HIOD_ abbreviation with HOST_IOMMU_DEVICE_ (Cédric)
>> - add header in include/sysemu/iommufd.h (Cédric)
>>
>> v2:
>> - use QOM to abstract host IOMMU device and its sub-classes (Cédric)
>> - move host IOMMU device creation in attach_device() (Cédric)
>> - refine pci_device_set/unset_iommu_device doc further (Eric)
>> - define host IOMMU info format of different backend
>> - implement get_host_iommu_info() for different backend (Cédric)
>> - drop cap/ecap update logic (MST)
>> - check aw-bits from get_host_iommu_info() in legacy mode
>>
>> v1:
>> - use HostIOMMUDevice handle instead of union in VFIODevice (Eric)
>> - change host_iommu_device_init to host_iommu_device_create
>> - allocate HostIOMMUDevice in host_iommu_device_create callback
>>   and set the VFIODevice base_hdev handle (Eric)
>> - refine pci_device_set/unset_iommu_device doc (Eric)
>> - use HostIOMMUDevice handle instead of union in VTDHostIOMMUDevice
>(Eric)
>> - convert HostIOMMUDevice to sub object pointer in vtd_check_hdev
>>
>> rfcv2:
>> - introduce common abstract HostIOMMUDevice and sub struct for
>different BEs (Eric, Cédric)
>> - remove iommufd_device.[ch] (Cédric)
>> - remove duplicate iommufd/devid define from VFIODevice (Eric)
>> - drop the p in aliased_pbus and aliased_pdevfn (Eric)
>> - assert devfn and iommu_bus in pci_device_get_iommu_bus_devfn
>(Cédric, Eric)
>> - use errp in iommufd_device_get_info (Eric)
>> - split and simplify cap/ecap check/sync code in intel_iommu.c (Cédric)
>> - move VTDHostIOMMUDevice declaration to intel_iommu_internal.h
>(Cédric)
>> - make '(vtd->cap_reg >> 16) & 0x3fULL' a MACRO and add missed '+1'
>(Cédric)
>> - block migration if vIOMMU cap/ecap updated based on host IOMMU
>cap/ecap
>> - add R-B
>>
>> Yi Liu (2):
>>   hw/pci: Introduce pci_device_[set|unset]_iommu_device()
>>   intel_iommu: Implement [set|unset]_iommu_device() callbacks
>>
>> Zhenzhong Duan (17):
>>   backends: Introduce HostIOMMUDevice abstract
>>   vfio/container: Introduce TYPE_HOST_IOMMU_DEVICE_LEGACY_VFIO
>device
>>   backends/iommufd: Introduce abstract
>TYPE_HOST_IOMMU_DEVICE_IOMMUFD
>>     device
>>   vfio/iommufd: Introduce TYPE_HOST_IOMMU_DEVICE_IOMMUFD_VFIO
>device
>>   backends/host_iommu_device: Introduce HostIOMMUDeviceCaps
>>   range: Introduce range_get_last_bit()
>>   vfio/container: Implement HostIOMMUDeviceClass::realize() handler
>>   backends/iommufd: Introduce helper function
>>     iommufd_backend_get_device_info()
>>   vfio/iommufd: Implement HostIOMMUDeviceClass::realize() handler
>>   vfio/container: Implement HostIOMMUDeviceClass::get_cap() handler
>>   backends/iommufd: Implement HostIOMMUDeviceClass::get_cap()
>handler
>>   vfio: Introduce VFIOIOMMUClass::hiod_typename attribute
>>   vfio: Create host IOMMU device instance
>>   hw/pci: Introduce helper function pci_device_get_iommu_bus_devfn()
>>   vfio/pci: Pass HostIOMMUDevice to vIOMMU
>>   intel_iommu: Extract out vtd_cap_init() to initialize cap/ecap
>>   intel_iommu: Check compatibility with host IOMMU capabilities
>>
>>  MAINTAINERS                           |   2 +
>>  hw/i386/intel_iommu_internal.h        |   9 ++
>>  include/hw/i386/intel_iommu.h         |   3 +
>>  include/hw/pci/pci.h                  |  38 ++++-
>>  include/hw/vfio/vfio-common.h         |   7 +
>>  include/hw/vfio/vfio-container-base.h |   3 +
>>  include/qemu/range.h                  |  11 ++
>>  include/sysemu/host_iommu_device.h    |  88 ++++++++++++
>>  include/sysemu/iommufd.h              |  19 +++
>>  backends/host_iommu_device.c          |  30 ++++
>>  backends/iommufd.c                    |  76 ++++++++--
>>  hw/i386/intel_iommu.c                 | 198 ++++++++++++++++++++------
>>  hw/pci/pci.c                          |  75 +++++++++-
>>  hw/vfio/common.c                      |  16 ++-
>>  hw/vfio/container.c                   |  48 ++++++-
>>  hw/vfio/iommufd.c                     |  44 +++++-
>>  hw/vfio/pci.c                         |  19 ++-
>>  backends/Kconfig                      |   5 +
>>  backends/meson.build                  |   1 +
>>  19 files changed, 623 insertions(+), 69 deletions(-)
>>  create mode 100644 include/sysemu/host_iommu_device.h
>>  create mode 100644 backends/host_iommu_device.c
>>