Hi All,

Kindly ping😊, any more comments?

Thanks
Zhenzhong

>-----Original Message-----
>From: Duan, Zhenzhong <zhenzhong.duan@intel.com>
>Subject: [PATCH v5 0/9] vfio: relax the vIOMMU check
>
>Hi,
>
>This series relax the vIOMMU check and allows live migration with vIOMMU
>without VFs using device dirty tracking. It's rewritten based on first 4
>patches of [1] from Joao.
>
>Currently what block us is the lack of dirty bitmap query with iommufd
>before unmap. By adding that query and handle some corner case we can
>relax the check.
>
>Based on vfio-next branch:
>
>patch1-2: add dirty bitmap query with iommufd
>patch3:   a ranaming cleanup
>patch4-5: unmap_bitmap optimization
>patch6-7: fixes to avoid losing dirty pages
>patch8:   add a blocker if VM memory is really quite large for unmap_bitmap
>patch9:   relax vIOMMU check
>
>
>We used [2] to test, it contains dom_switch series + this series +
>nesting series. I included nesting series just because I'd like to confirm
>the two patches optimizing out dirty tracking for readonly pages work.
>
>We tested VM live migration (running QAT workload in VM) with QAT device
>passthrough, below matrix configs with guest config 'iommu=pt' and
>'iommu=nopt':
>1.Scalable mode vIOMMU + IOMMUFD cdev mode
>2.Scalable mode vIOMMU + legacy VFIO mode
>3.legacy mode vIOMMU + IOMMUFD cdev mode
>4.legacy mode vIOMMU + legacy VFIO mode
>
>The QAT workload is a user level app that utilizes VFIO to control QAT device.
>
>Thanks
>Zhenzhong
>
>[1] https://github.com/jpemartins/qemu/commits/vfio-migration-viommu/
>[2]
>https://github.com/yiliu1765/qemu/tree/liuyi/zhenzhong/iommufd_nesting.v
>8.DS_LM.wip
>
>Changelog:
>v5:
>- drop the patch checking iommu_dirty_tracking (Avihai, Joao)
>- pass iotlb info to unmap_bitmap when switch out of system AS
>
>v4:
>- bypass memory size check for device dirty tracking as it's unrelated (Avihai)
>- split vfio_device_dirty_pages_disabled() helper out as a separate patch
>- add a patch to fix minor error on checking vbasedev->iommu_dirty_tracking
>
>v3:
>- return bitmap query failure to fail migration (Avihai)
>- refine patch7, set IOMMUFD backend 'dirty_pgsizes' and
>'max_dirty_bitmap_size' (Cedric)
>- refine patch7, calculate memory limit instead of hardcode 8TB (Liuyi)
>- refine commit log (Cedric, Liuyi)
>
>v2:
>- add backend_flag parameter to pass DIRTY_BITMAP_NO_CLEAR (Joao,
>Cedric)
>- add a cleanup patch to rename vfio_dma_unmap_bitmap (Cedric)
>- add blocker if unmap_bitmap limit check fail (Liuyi)
>
>
>
>Joao Martins (1):
>  vfio: Add a backend_flag parameter to
>    vfio_contianer_query_dirty_bitmap()
>
>Zhenzhong Duan (8):
>  vfio/iommufd: Add framework code to support getting dirty bitmap
>    before unmap
>  vfio/iommufd: Query dirty bitmap before DMA unmap
>  vfio/container-legacy: rename vfio_dma_unmap_bitmap() to
>    vfio_legacy_dma_unmap_get_dirty_bitmap()
>  vfio/iommufd: Add IOMMU_HWPT_GET_DIRTY_BITMAP_NO_CLEAR flag
>support
>  intel_iommu: Fix unmap_bitmap failure with legacy VFIO backend
>  vfio/listener: Construct iotlb entry when unmap memory address space
>  vfio/migration: Add migration blocker if VM memory is too large to
>    cause unmap_bitmap failure
>  vfio/migration: Allow live migration with vIOMMU without VFs using
>    device dirty tracking
>
> include/hw/vfio/vfio-container.h |  8 +++--
> include/hw/vfio/vfio-device.h    | 10 ++++++
> include/system/iommufd.h         |  2 +-
> backends/iommufd.c               |  5 +--
> hw/i386/intel_iommu.c            | 42 +++++++++++++++++++++++++
> hw/vfio-user/container.c         |  5 +--
> hw/vfio/container-legacy.c       | 15 +++++----
> hw/vfio/container.c              | 20 ++++++------
> hw/vfio/device.c                 |  6 ++++
> hw/vfio/iommufd.c                | 53
>+++++++++++++++++++++++++++++---
> hw/vfio/listener.c               | 21 ++++++++++---
> hw/vfio/migration.c              | 40 ++++++++++++++++++++++--
> backends/trace-events            |  2 +-
> hw/vfio/trace-events             |  2 +-
> 14 files changed, 194 insertions(+), 37 deletions(-)
>
>--
>2.47.1