Set skip_kexec_clear_master on live update prepare() so that the device
participating in live update can continue to perform DMA during kexec
phase.

Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
 drivers/vfio/pci/vfio_pci_liveupdate.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio_pci_liveupdate.c
index 8e0ee01127b3..789b52665e35 100644
--- a/drivers/vfio/pci/vfio_pci_liveupdate.c
+++ b/drivers/vfio/pci/vfio_pci_liveupdate.c
@@ -54,6 +54,7 @@ static int vfio_pci_liveupdate_prepare(struct liveupdate_file_handler *handler,
 		goto err_free_folio;
 
 	*data = virt_to_phys(ser);
+	vdev->pdev->skip_kexec_clear_master = true;
 
 	return 0;
 
@@ -67,7 +68,12 @@ static void vfio_pci_liveupdate_cancel(struct liveupdate_file_handler *handler,
 {
 	struct vfio_pci_core_device_ser *ser = phys_to_virt(data);
 	struct folio *folio = virt_to_folio(ser);
+	struct vfio_pci_core_device *vdev;
+	struct vfio_device *device;
 
+	device = vfio_device_from_file(file);
+	vdev = container_of(device, struct vfio_pci_core_device, vdev);
+	vdev->pdev->skip_kexec_clear_master = false;
 	WARN_ON_ONCE(kho_unpreserve_folio(folio));
 	folio_put(folio);
 }
-- 
2.51.0.858.gf9c4a03a3a-goog

On Fri, Oct 17, 2025 at 05:07:03PM -0700, Vipin Sharma wrote:
> Set skip_kexec_clear_master on live update prepare() so that the device
> participating in live update can continue to perform DMA during kexec
> phase.

Instead of introducing the skip_kexec_clear_master flag,
could you introduce a function to check whether a device
participates in live update and call that in pci_device_shutdown()?

I think that would be cleaner.  Otherwise someone reading
the code has to chase down the meaning of skip_kexec_clear_master,
i.e. search for places where the bit is set.

When the device is unbound from vfio-pci, don't you have to
clear the skip_kexec_clear_master flag?  I'm not seeing this
in your patches but maybe I'm missing something.  That problem
would solve itself if you follow the suggestion above.

Thanks,

Lukas

On 2025-10-18 09:09:06, Lukas Wunner wrote:
> On Fri, Oct 17, 2025 at 05:07:03PM -0700, Vipin Sharma wrote:
> > Set skip_kexec_clear_master on live update prepare() so that the device
> > participating in live update can continue to perform DMA during kexec
> > phase.
> 
> Instead of introducing the skip_kexec_clear_master flag,
> could you introduce a function to check whether a device
> participates in live update and call that in pci_device_shutdown()?
> 
> I think that would be cleaner.  Otherwise someone reading
> the code has to chase down the meaning of skip_kexec_clear_master,
> i.e. search for places where the bit is set.

That is one way to do it. In our internal implementation we have an API
which checks for the device participation in the live update, similar to
what you have suggested.

The PCI series posted by Chris [1] is providing a different way to know
the live update particpation of device. There pci_dev has a new struct
which contains particpation information.

In this VFIO series, my intention is to make minimal changes to PCI or
any other subsystem. I opted for a simple variable to check what device
should do during kexec reboot.

My hunch is that we will end up needing some state information in the
struct pci_dev{} which denotes device participation and whatever that
ends up being, we can use that here.

[1] https://lore.kernel.org/linux-pci/20250916-luo-pci-v2-0-c494053c3c08@kernel.org/
>
> When the device is unbound from vfio-pci, don't you have to
> clear the skip_kexec_clear_master flag?  I'm not seeing this
> in your patches but maybe I'm missing something.  That problem
> would solve itself if you follow the suggestion above.

VFIO subsystem blocks removal from vfio-pci if there is still a
reference to device (references are increased/decreased when device is
opened/closed, check vfio_unregister_group_dev()). LUO also do fget on
the VFIO FD which means we will not get closed callback on the VFIO FD
until that reference is dropped besides the opened file in userspace.

So, prior to kexec, luo will drop reference only if live update cancel
happens and that is the time we are resetting this flag in this patch
series.