[PATCH v5 0/3] vhost: fix the IO error after live migration

Haoqian He posted 3 patches 3 weeks, 4 days ago
backends/vhost-user.c                  | 20 +++++++--------
hw/block/vhost-user-blk.c              | 27 +++++++++++---------
hw/block/virtio-blk.c                  |  7 +++---
hw/char/virtio-serial-bus.c            |  3 ++-
hw/core/vm-change-state-handler.c      | 18 ++++++++-----
hw/display/vhost-user-gpu.c            | 12 ++++++---
hw/input/virtio-input.c                |  3 ++-
hw/net/virtio-net.c                    |  3 ++-
hw/scsi/scsi-bus.c                     |  2 +-
hw/scsi/vhost-scsi-common.c            | 13 +++++-----
hw/scsi/vhost-scsi.c                   |  5 ++--
hw/scsi/vhost-user-scsi.c              | 18 +++++++------
hw/vfio/migration.c                    |  2 +-
hw/virtio/vdpa-dev.c                   |  5 ++--
hw/virtio/vhost-user-base.c            | 23 ++++++++++-------
hw/virtio/vhost-user-fs.c              | 23 ++++++++++-------
hw/virtio/vhost-user-scmi.c            | 27 ++++++++++++--------
hw/virtio/vhost-user-vsock.c           | 15 +++++++----
hw/virtio/vhost-vsock-common.c         | 12 ++++-----
hw/virtio/vhost-vsock.c                | 11 ++++----
hw/virtio/vhost.c                      | 23 +++++++++--------
hw/virtio/virtio-balloon.c             |  3 ++-
hw/virtio/virtio-crypto.c              |  3 ++-
hw/virtio/virtio-iommu.c               |  3 ++-
hw/virtio/virtio-rng.c                 |  5 ++--
hw/virtio/virtio.c                     | 22 ++++++++++------
include/hw/virtio/vhost-scsi-common.h  |  2 +-
include/hw/virtio/vhost-vsock-common.h |  2 +-
include/hw/virtio/vhost.h              |  8 +++---
include/hw/virtio/virtio.h             |  2 +-
include/system/runstate.h              | 13 +++++++---
include/system/vhost-user-backend.h    |  2 +-
system/cpus.c                          |  8 ++++--
system/runstate.c                      | 35 ++++++++++++++++++++++----
34 files changed, 239 insertions(+), 141 deletions(-)
[PATCH v5 0/3] vhost: fix the IO error after live migration
Posted by Haoqian He 3 weeks, 4 days ago
At the end of the VM live migration, the vhost device will be stopped.
Currently, if the vhost-user backend crashes, vhost device's set_status()
would not return failure, live migration won't perceive the disconnection
with the backend. After the live migration is successful, the stale inflight
IO will be submitted to the migration target host, which may be leading to
the IO error.

The following patch series fixes the issue by making the live migration
aware of the loss of connection with the vhost-user backend and aborting
the live migration.

---
  v1 ... v2
    1. Fix some grammar issues in commit message.
    2. Remove assert in vhost_scsi_common_stop and return error upwards.

  v2 ... v3
    1. Added more detailed comments and commit message.
    2. Change the newly added type name and parameter name.
    3. Remove set_status_ext, change the return type of set_status to int.

  v3 ... v4
    1. Call set_status() only if the function pointer is not NULL in the
       3rd patch.
    2. Add the more detailed commit messages for the 3rd patch.

  v4 ... v5
    1. Fix code style issues in patch v4.
    2. Rebased on master.

Haoqian He (3):
  system/runstate: add VM state change cb with return value
  vhost: return failure if stop virtqueue failed in vhost_dev_stop
  vhost-user: return failure if backend crash when live migration

 backends/vhost-user.c                  | 20 +++++++--------
 hw/block/vhost-user-blk.c              | 27 +++++++++++---------
 hw/block/virtio-blk.c                  |  7 +++---
 hw/char/virtio-serial-bus.c            |  3 ++-
 hw/core/vm-change-state-handler.c      | 18 ++++++++-----
 hw/display/vhost-user-gpu.c            | 12 ++++++---
 hw/input/virtio-input.c                |  3 ++-
 hw/net/virtio-net.c                    |  3 ++-
 hw/scsi/scsi-bus.c                     |  2 +-
 hw/scsi/vhost-scsi-common.c            | 13 +++++-----
 hw/scsi/vhost-scsi.c                   |  5 ++--
 hw/scsi/vhost-user-scsi.c              | 18 +++++++------
 hw/vfio/migration.c                    |  2 +-
 hw/virtio/vdpa-dev.c                   |  5 ++--
 hw/virtio/vhost-user-base.c            | 23 ++++++++++-------
 hw/virtio/vhost-user-fs.c              | 23 ++++++++++-------
 hw/virtio/vhost-user-scmi.c            | 27 ++++++++++++--------
 hw/virtio/vhost-user-vsock.c           | 15 +++++++----
 hw/virtio/vhost-vsock-common.c         | 12 ++++-----
 hw/virtio/vhost-vsock.c                | 11 ++++----
 hw/virtio/vhost.c                      | 23 +++++++++--------
 hw/virtio/virtio-balloon.c             |  3 ++-
 hw/virtio/virtio-crypto.c              |  3 ++-
 hw/virtio/virtio-iommu.c               |  3 ++-
 hw/virtio/virtio-rng.c                 |  5 ++--
 hw/virtio/virtio.c                     | 22 ++++++++++------
 include/hw/virtio/vhost-scsi-common.h  |  2 +-
 include/hw/virtio/vhost-vsock-common.h |  2 +-
 include/hw/virtio/vhost.h              |  8 +++---
 include/hw/virtio/virtio.h             |  2 +-
 include/system/runstate.h              | 13 +++++++---
 include/system/vhost-user-backend.h    |  2 +-
 system/cpus.c                          |  8 ++++--
 system/runstate.c                      | 35 ++++++++++++++++++++++----
 34 files changed, 239 insertions(+), 141 deletions(-)

-- 
2.44.0
Re: [PATCH v5 0/3] vhost: fix the IO error after live migration
Posted by Lei Yang 3 weeks, 3 days ago
I tested this series of patches v5 with virtio-net regression tests,
everything works fine.

Tested-by: Lei Yang <leiyang@redhat.com>

On Wed, Apr 16, 2025 at 10:49 AM Haoqian He <haoqian.he@smartx.com> wrote:
>
> At the end of the VM live migration, the vhost device will be stopped.
> Currently, if the vhost-user backend crashes, vhost device's set_status()
> would not return failure, live migration won't perceive the disconnection
> with the backend. After the live migration is successful, the stale inflight
> IO will be submitted to the migration target host, which may be leading to
> the IO error.
>
> The following patch series fixes the issue by making the live migration
> aware of the loss of connection with the vhost-user backend and aborting
> the live migration.
>
> ---
>   v1 ... v2
>     1. Fix some grammar issues in commit message.
>     2. Remove assert in vhost_scsi_common_stop and return error upwards.
>
>   v2 ... v3
>     1. Added more detailed comments and commit message.
>     2. Change the newly added type name and parameter name.
>     3. Remove set_status_ext, change the return type of set_status to int.
>
>   v3 ... v4
>     1. Call set_status() only if the function pointer is not NULL in the
>        3rd patch.
>     2. Add the more detailed commit messages for the 3rd patch.
>
>   v4 ... v5
>     1. Fix code style issues in patch v4.
>     2. Rebased on master.
>
> Haoqian He (3):
>   system/runstate: add VM state change cb with return value
>   vhost: return failure if stop virtqueue failed in vhost_dev_stop
>   vhost-user: return failure if backend crash when live migration
>
>  backends/vhost-user.c                  | 20 +++++++--------
>  hw/block/vhost-user-blk.c              | 27 +++++++++++---------
>  hw/block/virtio-blk.c                  |  7 +++---
>  hw/char/virtio-serial-bus.c            |  3 ++-
>  hw/core/vm-change-state-handler.c      | 18 ++++++++-----
>  hw/display/vhost-user-gpu.c            | 12 ++++++---
>  hw/input/virtio-input.c                |  3 ++-
>  hw/net/virtio-net.c                    |  3 ++-
>  hw/scsi/scsi-bus.c                     |  2 +-
>  hw/scsi/vhost-scsi-common.c            | 13 +++++-----
>  hw/scsi/vhost-scsi.c                   |  5 ++--
>  hw/scsi/vhost-user-scsi.c              | 18 +++++++------
>  hw/vfio/migration.c                    |  2 +-
>  hw/virtio/vdpa-dev.c                   |  5 ++--
>  hw/virtio/vhost-user-base.c            | 23 ++++++++++-------
>  hw/virtio/vhost-user-fs.c              | 23 ++++++++++-------
>  hw/virtio/vhost-user-scmi.c            | 27 ++++++++++++--------
>  hw/virtio/vhost-user-vsock.c           | 15 +++++++----
>  hw/virtio/vhost-vsock-common.c         | 12 ++++-----
>  hw/virtio/vhost-vsock.c                | 11 ++++----
>  hw/virtio/vhost.c                      | 23 +++++++++--------
>  hw/virtio/virtio-balloon.c             |  3 ++-
>  hw/virtio/virtio-crypto.c              |  3 ++-
>  hw/virtio/virtio-iommu.c               |  3 ++-
>  hw/virtio/virtio-rng.c                 |  5 ++--
>  hw/virtio/virtio.c                     | 22 ++++++++++------
>  include/hw/virtio/vhost-scsi-common.h  |  2 +-
>  include/hw/virtio/vhost-vsock-common.h |  2 +-
>  include/hw/virtio/vhost.h              |  8 +++---
>  include/hw/virtio/virtio.h             |  2 +-
>  include/system/runstate.h              | 13 +++++++---
>  include/system/vhost-user-backend.h    |  2 +-
>  system/cpus.c                          |  8 ++++--
>  system/runstate.c                      | 35 ++++++++++++++++++++++----
>  34 files changed, 239 insertions(+), 141 deletions(-)
>
> --
> 2.44.0
>
>