This series can be found on GitHub:
https://github.com/dmatlack/linux/tree/liveupdate/vfio/cdev/v2
This series adds the base support to preserve a VFIO device file across
a Live Update. "Base support" means that this allows userspace to
safely preserve a VFIO device file with LIVEUPDATE_SESSION_PRESERVE_FD
and retrieve it with LIVEUPDATE_SESSION_RETRIEVE_FD, but the device
itself is not preserved in a fully running state across Live Update.
This series aims to provide a foundation on which to build the rest of
the device preservation infrastructure, including:
- Preservation of iommufd files [1]
- Preservation of IOMMU driver state
- Preservation of PCI state (BAR resources, device state, bridge state, ...)
- Preservation of vfio-pci driver state
Testing
-------
The patches at the end of this series provide comprehensive selftests
for the new code added by this series. The selftests have been validated
in both a VM environment using a virtio-net PCIe device, and in a
baremetal environment on an Intel EMR server with an Intel DSA PCIe
device.
Here is an example of how to run the new selftests:
vfio_pci_liveupdate_uapi_test:
$ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
$ tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test 0000:00:04.0
$ tools/testing/selftests/vfio/scripts/cleanup.sh
vfio_pci_liveupdate_kexec_test:
$ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
$ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 1 0000:00:04.0
$ kexec ...
$ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
$ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 2 0000:00:04.0
$ tools/testing/selftests/vfio/scripts/cleanup.sh
It is also possible to run vfio_pci_liveupdate_kexec_test multiple times
to preserve multiple devices simultaneously across a Live Update. This
series has been tested with up to 8 devices concurrently preserved.
Changelog
---------
v2:
- Rebase on top of linux-next (tag: next-20260115)
- Add missing EXPORT_SYMBOL_GPLs in LUO (Zhu)
- Add Missing EXPORT_SYMBOL_GPLs for vfio_device_fops (Zhu)
- Fix circular dependency between vfio-pci-core and vfio-pci (Zhu)
- Handle pci=assign-busses (Lukas)
- Drop driver_override patch (Jason)
- Use kho_alloc_preserve(), kho_unpreserve_free(), kho_restore_free() (Pasha)
- Don't access PCI FLB after device initialization (Jason)
- Fix folio leak in vfio_pci_liveupdate_retrieve() (Alex)
- Add Documentation (Pasha)
v1: https://lore.kernel.org/kvm/20251126193608.2678510-1-dmatlack@google.com/
rfc: https://lore.kernel.org/kvm/20251018000713.677779-1-vipinsh@google.com/
Cc: Saeed Mahameed <saeedm@nvidia.com>
Cc: Adithya Jayachandran <ajayachandra@nvidia.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Parav Pandit <parav@nvidia.com>
Cc: Leon Romanovsky <leonro@nvidia.com>
Cc: William Tu <witu@nvidia.com>
Cc: Jacob Pan <jacob.pan@linux.microsoft.com>
Cc: Lukas Wunner <lukas@wunner.de>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Samiullah Khawaja <skhawaja@google.com>
Cc: Chris Li <chrisl@kernel.org>
Cc: Josh Hilke <jrhilke@google.com>
Cc: David Rientjes <rientjes@google.com>
[1] https://lore.kernel.org/linux-iommu/20251202230303.1017519-1-skhawaja@google.com/
David Matlack (13):
liveupdate: Export symbols needed by modules
PCI: Add API to track PCI devices preserved across Live Update
PCI: Inherit bus numbers from previous kernel during Live Update
vfio/pci: Notify PCI subsystem about devices preserved across Live
Update
vfio: Enforce preserved devices are retrieved via
LIVEUPDATE_SESSION_RETRIEVE_FD
vfio/pci: Store incoming Live Update state in struct
vfio_pci_core_device
docs: liveupdate: Document VFIO device file preservation
vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED
vfio: selftests: Add vfio_pci_liveupdate_uapi_test
vfio: selftests: Expose iommu_modes to tests
vfio: selftests: Expose low-level helper routines for setting up
struct vfio_pci_device
vfio: selftests: Verify that opening VFIO device fails during Live
Update
vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test
Vipin Sharma (9):
vfio/pci: Register a file handler with Live Update Orchestrator
vfio/pci: Preserve vfio-pci device files across Live Update
vfio/pci: Retrieve preserved device files after Live Update
vfio/pci: Skip reset of preserved device after Live Update
selftests/liveupdate: Move luo_test_utils.* into a reusable library
selftests/liveupdate: Add helpers to preserve/retrieve FDs
vfio: selftests: Build liveupdate library in VFIO selftests
vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD
vfio: selftests: Add vfio_pci_liveupdate_kexec_test
Documentation/userspace-api/liveupdate.rst | 144 ++++++++++
MAINTAINERS | 1 +
drivers/pci/Makefile | 1 +
drivers/pci/liveupdate.c | 212 +++++++++++++++
drivers/pci/probe.c | 23 +-
drivers/vfio/device_cdev.c | 25 +-
drivers/vfio/group.c | 9 +
drivers/vfio/pci/Makefile | 1 +
drivers/vfio/pci/vfio_pci.c | 11 +-
drivers/vfio/pci/vfio_pci_core.c | 24 +-
drivers/vfio/pci/vfio_pci_liveupdate.c | 246 +++++++++++++++++
drivers/vfio/pci/vfio_pci_priv.h | 16 ++
drivers/vfio/vfio.h | 13 -
drivers/vfio/vfio_main.c | 23 +-
include/linux/kho/abi/pci.h | 55 ++++
include/linux/kho/abi/vfio_pci.h | 45 +++
include/linux/pci.h | 47 ++++
include/linux/vfio.h | 58 ++++
include/linux/vfio_pci_core.h | 2 +
kernel/liveupdate/luo_core.c | 1 +
kernel/liveupdate/luo_file.c | 2 +
tools/testing/selftests/liveupdate/.gitignore | 1 +
tools/testing/selftests/liveupdate/Makefile | 14 +-
.../include/libliveupdate.h} | 11 +-
.../selftests/liveupdate/lib/libliveupdate.mk | 20 ++
.../{luo_test_utils.c => lib/liveupdate.c} | 43 ++-
.../selftests/liveupdate/luo_kexec_simple.c | 2 +-
.../selftests/liveupdate/luo_multi_session.c | 2 +-
tools/testing/selftests/vfio/Makefile | 23 +-
.../vfio/lib/include/libvfio/iommu.h | 2 +
.../lib/include/libvfio/vfio_pci_device.h | 8 +
tools/testing/selftests/vfio/lib/iommu.c | 4 +-
.../selftests/vfio/lib/vfio_pci_device.c | 60 ++--
.../vfio/vfio_pci_liveupdate_kexec_test.c | 256 ++++++++++++++++++
.../vfio/vfio_pci_liveupdate_uapi_test.c | 93 +++++++
35 files changed, 1410 insertions(+), 88 deletions(-)
create mode 100644 drivers/pci/liveupdate.c
create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c
create mode 100644 include/linux/kho/abi/pci.h
create mode 100644 include/linux/kho/abi/vfio_pci.h
rename tools/testing/selftests/liveupdate/{luo_test_utils.h => lib/include/libliveupdate.h} (80%)
create mode 100644 tools/testing/selftests/liveupdate/lib/libliveupdate.mk
rename tools/testing/selftests/liveupdate/{luo_test_utils.c => lib/liveupdate.c} (89%)
create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c
create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c
base-commit: 9b7977f9e39b7768c70c2aa497f04e7569fd3e00
--
2.53.0.rc1.225.gd81095ad13-goog