This series adds support for tests that use multiple devices, and adds
one new test, vfio_pci_device_init_perf_test, which measures parallel
device initialization time to demonstrate the improvement from commit
e908f58b6beb ("vfio/pci: Separate SR-IOV VF dev_set").

This series also breaks apart the monolithic vfio_util.h and
vfio_pci_device.c into separate files, to account for all the new code.
This required quite a bit of code motion so the diffstat looks large.
The final layout is more granular and provides a better separation of
the IOMMU code from the device code.

Final layout:

  C files:
    - tools/testing/selftests/vfio/lib/iommu.c
    - tools/testing/selftests/vfio/lib/iova_allocator.c
    - tools/testing/selftests/vfio/lib/libvfio.c
    - tools/testing/selftests/vfio/lib/vfio_pci_device.c
    - tools/testing/selftests/vfio/lib/vfio_pci_driver.c

  H files:
   - tools/testing/selftests/vfio/lib/include/libvfio.h
   - tools/testing/selftests/vfio/lib/include/libvfio/assert.h
   - tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
   - tools/testing/selftests/vfio/lib/include/libvfio/iova_allocator.h
   - tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
   - tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_driver.h

Notably, vfio_util.h is now gone and replaced with libvfio.h.

This series is based on vfio/next plus Alex Mastro's series to add the
IOVA allocator [1]. It should apply cleanly to vfio/next once Alex's
series is merged into 6.18 and then into vfio/next.

This series can be found on GitHub:

  https://github.com/dmatlack/linux/tree/vfio/selftests/init_perf_test/v2

[1] https://lore.kernel.org/kvm/20251111-iova-ranges-v3-0-7960244642c5@fb.com/

Cc: Alex Mastro <amastro@fb.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Josh Hilke <jrhilke@google.com>
Cc: Raghavendra Rao Ananta <rananta@google.com>
Cc: Vipin Sharma <vipinsh@google.com>

v2:
 - Require tests to call iommu_init() and manage struct iommu objects
   rather than implicitly doing it in vfio_pci_device_init().
 - Drop all the device wrappers for IOMMU methods and require tests to
   interact with the iommu_*() helper functions directly.
 - Add a commit to eliminate INVALID_IOVA. This is a simple cleanup I've
   been meaning to make.
 - Upgrade some driver logging to error (Raghavendra)
 - Remove plurality from helper function that fetches BDF from
   environment variable (Raghavendra)
 - Fix cleanup.sh to only delete the device directory when cleaning up
   all devices (Raghavendra)

v1: https://lore.kernel.org/kvm/20251008232531.1152035-1-dmatlack@google.com/

David Matlack (18):
  vfio: selftests: Move run.sh into scripts directory
  vfio: selftests: Split run.sh into separate scripts
  vfio: selftests: Allow passing multiple BDFs on the command line
  vfio: selftests: Rename struct vfio_iommu_mode to iommu_mode
  vfio: selftests: Introduce struct iommu
  vfio: selftests: Support multiple devices in the same
    container/iommufd
  vfio: selftests: Eliminate overly chatty logging
  vfio: selftests: Prefix logs with device BDF where relevant
  vfio: selftests: Upgrade driver logging to dev_err()
  vfio: selftests: Rename struct vfio_dma_region to dma_region
  vfio: selftests: Move IOMMU library code into iommu.c
  vfio: selftests: Move IOVA allocator into iova_allocator.c
  vfio: selftests: Stop passing device for IOMMU operations
  vfio: selftests: Rename vfio_util.h to libvfio.h
  vfio: selftests: Move vfio_selftests_*() helpers into libvfio.c
  vfio: selftests: Split libvfio.h into separate header files
  vfio: selftests: Eliminate INVALID_IOVA
  vfio: selftests: Add vfio_pci_device_init_perf_test

 tools/testing/selftests/vfio/Makefile         |   9 +-
 .../selftests/vfio/lib/drivers/dsa/dsa.c      |  36 +-
 .../selftests/vfio/lib/drivers/ioat/ioat.c    |  18 +-
 .../selftests/vfio/lib/include/libvfio.h      |  26 +
 .../vfio/lib/include/libvfio/assert.h         |  54 ++
 .../vfio/lib/include/libvfio/iommu.h          |  76 +++
 .../vfio/lib/include/libvfio/iova_allocator.h |  23 +
 .../lib/include/libvfio/vfio_pci_device.h     | 125 ++++
 .../lib/include/libvfio/vfio_pci_driver.h     |  97 +++
 .../selftests/vfio/lib/include/vfio_util.h    | 331 -----------
 tools/testing/selftests/vfio/lib/iommu.c      | 465 +++++++++++++++
 .../selftests/vfio/lib/iova_allocator.c       |  94 +++
 tools/testing/selftests/vfio/lib/libvfio.c    |  78 +++
 tools/testing/selftests/vfio/lib/libvfio.mk   |   5 +-
 .../selftests/vfio/lib/vfio_pci_device.c      | 555 +-----------------
 .../selftests/vfio/lib/vfio_pci_driver.c      |  16 +-
 tools/testing/selftests/vfio/run.sh           | 109 ----
 .../testing/selftests/vfio/scripts/cleanup.sh |  41 ++
 tools/testing/selftests/vfio/scripts/lib.sh   |  42 ++
 tools/testing/selftests/vfio/scripts/run.sh   |  16 +
 tools/testing/selftests/vfio/scripts/setup.sh |  48 ++
 .../selftests/vfio/vfio_dma_mapping_test.c    |  46 +-
 .../selftests/vfio/vfio_iommufd_setup_test.c  |   2 +-
 .../vfio/vfio_pci_device_init_perf_test.c     | 167 ++++++
 .../selftests/vfio/vfio_pci_device_test.c     |  12 +-
 .../selftests/vfio/vfio_pci_driver_test.c     |  51 +-
 26 files changed, 1479 insertions(+), 1063 deletions(-)
 create mode 100644 tools/testing/selftests/vfio/lib/include/libvfio.h
 create mode 100644 tools/testing/selftests/vfio/lib/include/libvfio/assert.h
 create mode 100644 tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
 create mode 100644 tools/testing/selftests/vfio/lib/include/libvfio/iova_allocator.h
 create mode 100644 tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
 create mode 100644 tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_driver.h
 delete mode 100644 tools/testing/selftests/vfio/lib/include/vfio_util.h
 create mode 100644 tools/testing/selftests/vfio/lib/iommu.c
 create mode 100644 tools/testing/selftests/vfio/lib/iova_allocator.c
 create mode 100644 tools/testing/selftests/vfio/lib/libvfio.c
 delete mode 100755 tools/testing/selftests/vfio/run.sh
 create mode 100755 tools/testing/selftests/vfio/scripts/cleanup.sh
 create mode 100755 tools/testing/selftests/vfio/scripts/lib.sh
 create mode 100755 tools/testing/selftests/vfio/scripts/run.sh
 create mode 100755 tools/testing/selftests/vfio/scripts/setup.sh
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_device_init_perf_test.c


base-commit: 0ed3a30fd996cb0cac872432cf25185fda7e5316
prerequisite-patch-id: dcf23dcc1198960bda3102eefaa21df60b2e4c54
prerequisite-patch-id: e32e56d5bf7b6c7dd40d737aa3521560407e00f5
prerequisite-patch-id: 4f79a41bf10a4c025ba5f433551b46035aa15878
prerequisite-patch-id: f903a45f0c32319138cd93a007646ab89132b18c
-- 
2.52.0.rc1.455.g30608eb744-goog

On Wed, Nov 12, 2025 at 07:22:14PM +0000, David Matlack wrote:
> This series adds support for tests that use multiple devices, and adds
> one new test, vfio_pci_device_init_perf_test, which measures parallel
> device initialization time to demonstrate the improvement from commit
> e908f58b6beb ("vfio/pci: Separate SR-IOV VF dev_set").
 
Aside from the Makefile assignment issue in "Split run.sh into separate
scripts", LGTM.

Reviewed-by: Alex Mastro <amastro@fb.com>

On Mon, Nov 17, 2025 at 12:13 PM Alex Mastro <amastro@fb.com> wrote:
>
> On Wed, Nov 12, 2025 at 07:22:14PM +0000, David Matlack wrote:
> > This series adds support for tests that use multiple devices, and adds
> > one new test, vfio_pci_device_init_perf_test, which measures parallel
> > device initialization time to demonstrate the improvement from commit
> > e908f58b6beb ("vfio/pci: Separate SR-IOV VF dev_set").
>
> Aside from the Makefile assignment issue in "Split run.sh into separate
> scripts", LGTM.
>
> Reviewed-by: Alex Mastro <amastro@fb.com>

Thanks for the review. I'll send a v3 this week to address your comments.

On Wed, Nov 12, 2025 at 07:22:14PM +0000, David Matlack wrote:
> This series adds support for tests that use multiple devices, and adds
> one new test, vfio_pci_device_init_perf_test, which measures parallel
> device initialization time to demonstrate the improvement from commit
> e908f58b6beb ("vfio/pci: Separate SR-IOV VF dev_set").

The new test runs and passes for me.

Tested-by: Alex Mastro <amastro@fb.com>

$ vfio_pci_device_init_perf_test $(readlink -f /sys/class/vfio-dev/*/../.. | xargs -n1 basename)
Testing parallel initialization of 24 devices:
    0000:05:00.0
    0000:06:00.0
    0000:27:00.0
    0000:28:00.0
    0000:e5:00.0
    0000:e6:00.0
    0000:e7:00.0
    0000:e8:00.0
    0000:e9:00.0
    0000:ea:00.0
    0000:c5:00.0
    0000:c6:00.0
    0000:07:00.0
    0000:c7:00.0
    0000:c8:00.0
    0000:c9:00.0
    0000:ca:00.0
    0000:08:00.0
    0000:09:00.0
    0000:0a:00.0
    0000:23:00.0
    0000:24:00.0
    0000:25:00.0
    0000:26:00.0
TAP version 13
1..5
# Starting 5 tests from 5 test cases.
#  RUN           vfio_pci_device_init_perf_test.vfio_type1_iommu.init ...
Wall time: 1.440004083s
Min init time (per device): 1.236283226s
Max init time (per device): 1.440002010s
Avg init time (per device): 1.258762246s
#            OK  vfio_pci_device_init_perf_test.vfio_type1_iommu.init
ok 1 vfio_pci_device_init_perf_test.vfio_type1_iommu.init
#  RUN           vfio_pci_device_init_perf_test.vfio_type1v2_iommu.init ...
Wall time: 1.448952696s
Min init time (per device): 1.242623358s
Max init time (per device): 1.448951915s
Avg init time (per device): 1.264836317s
#            OK  vfio_pci_device_init_perf_test.vfio_type1v2_iommu.init
ok 2 vfio_pci_device_init_perf_test.vfio_type1v2_iommu.init
#  RUN           vfio_pci_device_init_perf_test.iommufd_compat_type1.init ...
Wall time: 1.446931666s
Min init time (per device): 1.242316634s
Max init time (per device): 1.446927360s
Avg init time (per device): 1.264904097s
#            OK  vfio_pci_device_init_perf_test.iommufd_compat_type1.init
ok 3 vfio_pci_device_init_perf_test.iommufd_compat_type1.init
#  RUN           vfio_pci_device_init_perf_test.iommufd_compat_type1v2.init ...
Wall time: 1.449211729s
Min init time (per device): 1.243377853s
Max init time (per device): 1.449211729s
Avg init time (per device): 1.264266785s
#            OK  vfio_pci_device_init_perf_test.iommufd_compat_type1v2.init
ok 4 vfio_pci_device_init_perf_test.iommufd_compat_type1v2.init
#  RUN           vfio_pci_device_init_perf_test.iommufd.init ...
Wall time: 1.447583702s
Min init time (per device): 1.241216318s
Max init time (per device): 1.447582350s
Avg init time (per device): 1.264159293s
#            OK  vfio_pci_device_init_perf_test.iommufd.init
ok 5 vfio_pci_device_init_perf_test.iommufd.init
# PASSED: 5 / 5 tests passed.
# Totals: pass:5 fail:0 xfail:0 xpass:0 skip:0 error:0

On Wed, Nov 12, 2025 at 2:07 PM Alex Mastro <amastro@fb.com> wrote:
>
> On Wed, Nov 12, 2025 at 07:22:14PM +0000, David Matlack wrote:
> > This series adds support for tests that use multiple devices, and adds
> > one new test, vfio_pci_device_init_perf_test, which measures parallel
> > device initialization time to demonstrate the improvement from commit
> > e908f58b6beb ("vfio/pci: Separate SR-IOV VF dev_set").
>
> The new test runs and passes for me.
>
> Tested-by: Alex Mastro <amastro@fb.com>

Thanks for testing!