[PATCH v2 10/22] vfio/pci: Skip reset of preserved device after Live Update

David Matlack posted 22 patches 1 week, 2 days ago
[PATCH v2 10/22] vfio/pci: Skip reset of preserved device after Live Update
Posted by David Matlack 1 week, 2 days ago
From: Vipin Sharma <vipinsh@google.com>

Do not reset the device when a Live Update preserved vfio-pci device is
retrieved and first enabled. vfio_pci_liveupdate_freeze() guarantees the
device is reset prior to Live Update, so there's no reason to reset it
again after Live Update.

Since VFIO normally uses the initial reset to detect if the device
supports function resets, pass that from the previous kernel via
struct vfio_pci_core_dev_ser.

Signed-off-by: Vipin Sharma <vipinsh@google.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/vfio/pci/vfio_pci_core.c       | 22 +++++++++++++++++-----
 drivers/vfio/pci/vfio_pci_liveupdate.c |  1 +
 include/linux/kho/abi/vfio_pci.h       |  2 ++
 include/linux/vfio_pci_core.h          |  1 +
 4 files changed, 21 insertions(+), 5 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index b01b94d81e28..c9f73f597797 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -515,12 +515,24 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 	if (ret)
 		goto out_power;
 
-	/* If reset fails because of the device lock, fail this path entirely */
-	ret = pci_try_reset_function(pdev);
-	if (ret == -EAGAIN)
-		goto out_disable_device;
+	if (vdev->liveupdate_incoming_state) {
+		/*
+		 * This device was preserved by the previous kernel across a
+		 * Live Update, so it does not need to be reset.
+		 */
+		vdev->reset_works = vdev->liveupdate_incoming_state->reset_works;
+	} else {
+		/*
+		 * If reset fails because of the device lock, fail this path
+		 * entirely.
+		 */
+		ret = pci_try_reset_function(pdev);
+		if (ret == -EAGAIN)
+			goto out_disable_device;
+
+		vdev->reset_works = !ret;
+	}
 
-	vdev->reset_works = !ret;
 	pci_save_state(pdev);
 	vdev->pci_saved_state = pci_store_saved_state(pdev);
 	if (!vdev->pci_saved_state)
diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c b/drivers/vfio/pci/vfio_pci_liveupdate.c
index 1ad7379c70c4..c52d6bdb455f 100644
--- a/drivers/vfio/pci/vfio_pci_liveupdate.c
+++ b/drivers/vfio/pci/vfio_pci_liveupdate.c
@@ -57,6 +57,7 @@ static int vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *args)
 
 	ser->bdf = pci_dev_id(pdev);
 	ser->domain = pci_domain_nr(pdev->bus);
+	ser->reset_works = vdev->reset_works;
 
 	args->serialized_data = virt_to_phys(ser);
 	return 0;
diff --git a/include/linux/kho/abi/vfio_pci.h b/include/linux/kho/abi/vfio_pci.h
index 9bf58a2f3820..6c3d3c6dfc09 100644
--- a/include/linux/kho/abi/vfio_pci.h
+++ b/include/linux/kho/abi/vfio_pci.h
@@ -34,10 +34,12 @@
  *
  * @bdf: The device's PCI bus, device, and function number.
  * @domain: The device's PCI domain number (segment).
+ * @reset_works: Non-zero if the device supports function resets.
  */
 struct vfio_pci_core_device_ser {
 	u16 bdf;
 	u16 domain;
+	u8 reset_works;
 } __packed;
 
 #endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 350c30f84a13..95835298e29e 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -16,6 +16,7 @@
 #include <linux/types.h>
 #include <linux/uuid.h>
 #include <linux/notifier.h>
+#include <linux/kho/abi/vfio_pci.h>
 
 #ifndef VFIO_PCI_CORE_H
 #define VFIO_PCI_CORE_H
-- 
2.53.0.rc1.225.gd81095ad13-goog
Re: [PATCH v2 10/22] vfio/pci: Skip reset of preserved device after Live Update
Posted by Jacob Pan 1 week, 2 days ago
Hi David,

On Thu, 29 Jan 2026 21:24:57 +0000
David Matlack <dmatlack@google.com> wrote:

> From: Vipin Sharma <vipinsh@google.com>
> 
> Do not reset the device when a Live Update preserved vfio-pci device
> is retrieved and first enabled. vfio_pci_liveupdate_freeze()
> guarantees the device is reset prior to Live Update, so there's no
> reason to reset it again after Live Update.
> 
> Since VFIO normally uses the initial reset to detect if the device
> supports function resets, pass that from the previous kernel via
> struct vfio_pci_core_dev_ser.
> 
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> Signed-off-by: David Matlack <dmatlack@google.com>
> ---
>  drivers/vfio/pci/vfio_pci_core.c       | 22 +++++++++++++++++-----
>  drivers/vfio/pci/vfio_pci_liveupdate.c |  1 +
>  include/linux/kho/abi/vfio_pci.h       |  2 ++
>  include/linux/vfio_pci_core.h          |  1 +
>  4 files changed, 21 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/vfio/pci/vfio_pci_core.c
> b/drivers/vfio/pci/vfio_pci_core.c index b01b94d81e28..c9f73f597797
> 100644 --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -515,12 +515,24 @@ int vfio_pci_core_enable(struct
> vfio_pci_core_device *vdev) if (ret)
>  		goto out_power;
>  
> -	/* If reset fails because of the device lock, fail this path
> entirely */
> -	ret = pci_try_reset_function(pdev);
> -	if (ret == -EAGAIN)
> -		goto out_disable_device;
> +	if (vdev->liveupdate_incoming_state) {
> +		/*
> +		 * This device was preserved by the previous kernel
> across a
> +		 * Live Update, so it does not need to be reset.
> +		 */
> +		vdev->reset_works =
> vdev->liveupdate_incoming_state->reset_works;
Just wondering what happened to skipping the bus master clearing. I
understand this version does not preserve the device itself yet; I’m
just curious whether there were specific difficulties that led to
dropping the earlier patch which skipped clearing bus master.

> +	} else {
> +		/*
> +		 * If reset fails because of the device lock, fail
> this path
> +		 * entirely.
> +		 */
> +		ret = pci_try_reset_function(pdev);
> +		if (ret == -EAGAIN)
> +			goto out_disable_device;
> +
> +		vdev->reset_works = !ret;
> +	}
>  
> -	vdev->reset_works = !ret;
>  	pci_save_state(pdev);
>  	vdev->pci_saved_state = pci_store_saved_state(pdev);
>  	if (!vdev->pci_saved_state)
> diff --git a/drivers/vfio/pci/vfio_pci_liveupdate.c
> b/drivers/vfio/pci/vfio_pci_liveupdate.c index
> 1ad7379c70c4..c52d6bdb455f 100644 ---
> a/drivers/vfio/pci/vfio_pci_liveupdate.c +++
> b/drivers/vfio/pci/vfio_pci_liveupdate.c @@ -57,6 +57,7 @@ static int
> vfio_pci_liveupdate_preserve(struct liveupdate_file_op_args *args) 
>  	ser->bdf = pci_dev_id(pdev);
>  	ser->domain = pci_domain_nr(pdev->bus);
> +	ser->reset_works = vdev->reset_works;
>  
>  	args->serialized_data = virt_to_phys(ser);
>  	return 0;
> diff --git a/include/linux/kho/abi/vfio_pci.h
> b/include/linux/kho/abi/vfio_pci.h index 9bf58a2f3820..6c3d3c6dfc09
> 100644 --- a/include/linux/kho/abi/vfio_pci.h
> +++ b/include/linux/kho/abi/vfio_pci.h
> @@ -34,10 +34,12 @@
>   *
>   * @bdf: The device's PCI bus, device, and function number.
>   * @domain: The device's PCI domain number (segment).
> + * @reset_works: Non-zero if the device supports function resets.
>   */
>  struct vfio_pci_core_device_ser {
>  	u16 bdf;
>  	u16 domain;
> +	u8 reset_works;
>  } __packed;
>  
>  #endif /* _LINUX_LIVEUPDATE_ABI_VFIO_PCI_H */
> diff --git a/include/linux/vfio_pci_core.h
> b/include/linux/vfio_pci_core.h index 350c30f84a13..95835298e29e
> 100644 --- a/include/linux/vfio_pci_core.h
> +++ b/include/linux/vfio_pci_core.h
> @@ -16,6 +16,7 @@
>  #include <linux/types.h>
>  #include <linux/uuid.h>
>  #include <linux/notifier.h>
> +#include <linux/kho/abi/vfio_pci.h>
>  
>  #ifndef VFIO_PCI_CORE_H
>  #define VFIO_PCI_CORE_H
Re: [PATCH v2 10/22] vfio/pci: Skip reset of preserved device after Live Update
Posted by David Matlack 1 week, 2 days ago
On 2026-01-29 02:21 PM, Jacob Pan wrote:
> On Thu, 29 Jan 2026 21:24:57 +0000 David Matlack <dmatlack@google.com> wrote:

> > diff --git a/drivers/vfio/pci/vfio_pci_core.c
> > b/drivers/vfio/pci/vfio_pci_core.c index b01b94d81e28..c9f73f597797
> > 100644 --- a/drivers/vfio/pci/vfio_pci_core.c
> > +++ b/drivers/vfio/pci/vfio_pci_core.c
> > @@ -515,12 +515,24 @@ int vfio_pci_core_enable(struct
> > vfio_pci_core_device *vdev) if (ret)
> >  		goto out_power;
> >  
> > -	/* If reset fails because of the device lock, fail this path
> > entirely */
> > -	ret = pci_try_reset_function(pdev);
> > -	if (ret == -EAGAIN)
> > -		goto out_disable_device;
> > +	if (vdev->liveupdate_incoming_state) {
> > +		/*
> > +		 * This device was preserved by the previous kernel
> > across a
> > +		 * Live Update, so it does not need to be reset.
> > +		 */
> > +		vdev->reset_works =
> > vdev->liveupdate_incoming_state->reset_works;
>
> Just wondering what happened to skipping the bus master clearing. I
> understand this version does not preserve the device itself yet; I’m
> just curious whether there were specific difficulties that led to
> dropping the earlier patch which skipped clearing bus master.

Hi Jacob,

There's several places where bus master gets cleared that we need to
eventually eliminate to fully preserve the device.

 1. vfio_pci_liveupdate_freeze() clears it during shutdown when it
    restores vdev->pci_saved_state.
 2. pci_device_shutdown() clears it during shutdown.
 3. vfio_pci_core_enable() clears it when the preserved device file
    is bound to an iommufd after the Live Update (in
    vfio_pci_core_enable()).

I think it would be safe to skip (3) in this series, since that's very
similar to how this series skips resets during vfio_pci_core_enable()
for preserved devices.

But I don't think it would be safe to skip (1) or (2) until the attached
iommufd is fully preserved.

If you are just asking about (3) then I agree it could be skipped and I
can include that in the next version.
Re: [PATCH v2 10/22] vfio/pci: Skip reset of preserved device after Live Update
Posted by Jacob Pan 1 week, 2 days ago
Hi David,

On Thu, 29 Jan 2026 22:33:13 +0000
David Matlack <dmatlack@google.com> wrote:

> On 2026-01-29 02:21 PM, Jacob Pan wrote:
> > On Thu, 29 Jan 2026 21:24:57 +0000 David Matlack
> > <dmatlack@google.com> wrote:  
> 
> > > diff --git a/drivers/vfio/pci/vfio_pci_core.c
> > > b/drivers/vfio/pci/vfio_pci_core.c index
> > > b01b94d81e28..c9f73f597797 100644 ---
> > > a/drivers/vfio/pci/vfio_pci_core.c +++
> > > b/drivers/vfio/pci/vfio_pci_core.c @@ -515,12 +515,24 @@ int
> > > vfio_pci_core_enable(struct vfio_pci_core_device *vdev) if (ret)
> > >  		goto out_power;
> > >  
> > > -	/* If reset fails because of the device lock, fail this
> > > path entirely */
> > > -	ret = pci_try_reset_function(pdev);
> > > -	if (ret == -EAGAIN)
> > > -		goto out_disable_device;
> > > +	if (vdev->liveupdate_incoming_state) {
> > > +		/*
> > > +		 * This device was preserved by the previous
> > > kernel across a
> > > +		 * Live Update, so it does not need to be reset.
> > > +		 */
> > > +		vdev->reset_works =
> > > vdev->liveupdate_incoming_state->reset_works;  
> >
> > Just wondering what happened to skipping the bus master clearing. I
> > understand this version does not preserve the device itself yet; I’m
> > just curious whether there were specific difficulties that led to
> > dropping the earlier patch which skipped clearing bus master.  
> 
> Hi Jacob,
> 
> There's several places where bus master gets cleared that we need to
> eventually eliminate to fully preserve the device.
> 
>  1. vfio_pci_liveupdate_freeze() clears it during shutdown when it
>     restores vdev->pci_saved_state.
>  2. pci_device_shutdown() clears it during shutdown.
>  3. vfio_pci_core_enable() clears it when the preserved device file
>     is bound to an iommufd after the Live Update (in
>     vfio_pci_core_enable()).
> 
> I think it would be safe to skip (3) in this series, since that's very
> similar to how this series skips resets during vfio_pci_core_enable()
> for preserved devices.
> 
> But I don't think it would be safe to skip (1) or (2) until the
> attached iommufd is fully preserved.
> 
> If you are just asking about (3) then I agree it could be skipped and
> I can include that in the next version.

I was just asking about (3) and trying to understand the asymmetric
handling compared to reset. I don’t have a strong preference since this
is temporary—thanks for the explanation.

I’ve been testing my noiommu cdev patches on top of yours, and so far
they behave the same as with a real IOMMU. As you noted, however, final
device preservation still depends on iommufd.