[PATCH 2/2] vhost-user-blk: support inflight migration

Alexandr Moshkov posted 2 patches 3 weeks, 4 days ago
Maintainers: Raphael Norwitz <raphael@enfabrica.net>, "Michael S. Tsirkin" <mst@redhat.com>, Stefano Garzarella <sgarzare@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Peter Xu <peterx@redhat.com>, Fabiano Rosas <farosas@suse.de>, Eric Blake <eblake@redhat.com>, Markus Armbruster <armbru@redhat.com>
There is a newer version of this series
[PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Alexandr Moshkov 3 weeks, 4 days ago
In vhost_user_blk_stop() on incoming migration make force_stop = true,
so GET_VRING_BASE will not be executed.

Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>
---
 hw/block/vhost-user-blk.c | 52 +++++++++++++++++++++++++++++++++++++++
 migration/options.c       |  7 ++++++
 migration/options.h       |  1 +
 qapi/migration.json       |  9 +++++--
 4 files changed, 67 insertions(+), 2 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index c0cc5f6942..49f67d0451 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -31,6 +31,7 @@
 #include "hw/virtio/virtio-access.h"
 #include "system/system.h"
 #include "system/runstate.h"
+#include "migration/options.h"
 
 static const int user_feature_bits[] = {
     VIRTIO_BLK_F_SIZE_MAX,
@@ -224,6 +225,11 @@ static int vhost_user_blk_stop(VirtIODevice *vdev)
     force_stop = s->skip_get_vring_base_on_force_shutdown &&
                  qemu_force_shutdown_requested();
 
+    if (migrate_inflight_vhost_user_blk() &&
+        runstate_check(RUN_STATE_FINISH_MIGRATE)) {
+        force_stop = true;
+    }
+
     ret = force_stop ? vhost_dev_force_stop(&s->dev, vdev, true) :
                        vhost_dev_stop(&s->dev, vdev, true);
 
@@ -568,12 +574,58 @@ static struct vhost_dev *vhost_user_blk_get_vhost(VirtIODevice *vdev)
     return &s->dev;
 }
 
+static int vhost_user_blk_save(QEMUFile *f, void *pv, size_t size,
+                               const VMStateField *field, JSONWriter *vmdesc)
+{
+    VirtIODevice *vdev = pv;
+    VHostUserBlk *s = VHOST_USER_BLK(vdev);
+
+    if (!migrate_inflight_vhost_user_blk()) {
+        return 0;
+    }
+
+    vhost_dev_save_inflight(s->inflight, f);
+
+    return 0;
+}
+
+static int vhost_user_blk_load(QEMUFile *f, void *pv, size_t size,
+                               const VMStateField *field)
+{
+    VirtIODevice *vdev = pv;
+    VHostUserBlk *s = VHOST_USER_BLK(vdev);
+    int ret;
+
+    if (!migrate_inflight_vhost_user_blk()) {
+        return 0;
+    }
+
+    ret = vhost_dev_load_inflight(s->inflight, f);
+    if (ret < 0) {
+        g_autofree char *path = object_get_canonical_path(OBJECT(vdev));
+        error_report("%s [%s]: can't load in-flight requests",
+                     path, TYPE_VHOST_USER_BLK);
+        return ret;
+    }
+
+    return 0;
+}
+
 static const VMStateDescription vmstate_vhost_user_blk = {
     .name = "vhost-user-blk",
     .minimum_version_id = 1,
     .version_id = 1,
     .fields = (const VMStateField[]) {
         VMSTATE_VIRTIO_DEVICE,
+        {
+            .name = "backend state",
+            .info = &(const VMStateInfo) {
+                .name = "vhost-user-blk backend state",
+                .get = vhost_user_blk_load,
+                .put = vhost_user_blk_save,
+            },
+            .flags = VMS_SINGLE,
+        },
         VMSTATE_END_OF_LIST()
     },
 };
diff --git a/migration/options.c b/migration/options.c
index 5183112775..fcae2b4559 100644
--- a/migration/options.c
+++ b/migration/options.c
@@ -262,6 +262,13 @@ bool migrate_mapped_ram(void)
     return s->capabilities[MIGRATION_CAPABILITY_MAPPED_RAM];
 }
 
+bool migrate_inflight_vhost_user_blk(void)
+{
+    MigrationState *s = migrate_get_current();
+
+    return s->capabilities[MIGRATION_CAPABILITY_INFLIGHT_VHOST_USER_BLK];
+}
+
 bool migrate_ignore_shared(void)
 {
     MigrationState *s = migrate_get_current();
diff --git a/migration/options.h b/migration/options.h
index 82d839709e..eab1485d1a 100644
--- a/migration/options.h
+++ b/migration/options.h
@@ -30,6 +30,7 @@ bool migrate_colo(void);
 bool migrate_dirty_bitmaps(void);
 bool migrate_events(void);
 bool migrate_mapped_ram(void);
+bool migrate_inflight_vhost_user_blk(void);
 bool migrate_ignore_shared(void);
 bool migrate_late_block_activate(void);
 bool migrate_multifd(void);
diff --git a/qapi/migration.json b/qapi/migration.json
index be0f3fcc12..c9fea59515 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -517,9 +517,13 @@
 #     each RAM page.  Requires a migration URI that supports seeking,
 #     such as a file.  (since 9.0)
 #
+# @inflight-vhost-user-blk: If enabled, QEMU will migrate inflight
+#    region for vhost-user-blk.  (since 10.2)
+#
 # Features:
 #
-# @unstable: Members @x-colo and @x-ignore-shared are experimental.
+# @unstable: Members @x-colo and @x-ignore-shared,
+#     @inflight-vhost-user-blk are experimental.
 # @deprecated: Member @zero-blocks is deprecated as being part of
 #     block migration which was already removed.
 #
@@ -536,7 +540,8 @@
            { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
            'validate-uuid', 'background-snapshot',
            'zero-copy-send', 'postcopy-preempt', 'switchover-ack',
-           'dirty-limit', 'mapped-ram'] }
+           'dirty-limit', 'mapped-ram',
+           { 'name': 'inflight-vhost-user-blk', 'features': [ 'unstable' ] } ] }
 
 ##
 # @MigrationCapabilityStatus:
-- 
2.34.1
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Lei Yang 3 weeks, 1 day ago
Hi Alexandr

According to my test result, this series of patches introduce issues,
it prints the following error messages when compiling the process
after applying your patch.
The test based on this commit:
commit 3a2d5612a7422732b648b46d4b934e2e54622fd6 (origin/master, origin/HEAD)
Author: Peter Maydell <peter.maydell@linaro.org>
Date:   Fri Oct 17 14:31:56 2025 +0100

Error messages:
[1849/2964] Compiling C object
libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o
FAILED: libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o
cc -m64 -Ilibqemu-x86_64-softmmu.a.p -I. -I.. -Itarget/i386
-I../target/i386 -Isubprojects/dtc/libfdt -I../subprojects/dtc/libfdt
-Isubprojects/libvduse -I../subprojects/libvduse -Iqapi -Itrace -Iui
-Iui/shader -I/usr/include/pixman-1 -I/usr/include/glib-2.0
-I/usr/lib64/glib-2.0/include -I/usr/include/libmount
-I/usr/include/blkid -I/usr/include/sysprof-6
-I/usr/include/gio-unix-2.0 -I/usr/include/slirp
-fdiagnostics-color=auto -Wall -Winvalid-pch -Werror -std=gnu11 -O0 -g
-fstack-protector-strong -Wempty-body -Wendif-labels
-Wexpansion-to-defined -Wformat-security -Wformat-y2k
-Wignored-qualifiers -Wimplicit-fallthrough=2 -Winit-self
-Wmissing-format-attribute -Wmissing-prototypes -Wnested-externs
-Wold-style-declaration -Wold-style-definition -Wredundant-decls
-Wshadow=local -Wstrict-prototypes -Wtype-limits -Wundef -Wvla
-Wwrite-strings -Wno-missing-include-dirs -Wno-psabi
-Wno-shift-negative-value -isystem
/mnt/tests/distribution/command/qemu/linux-headers -isystem
linux-headers -iquote . -iquote /mnt/tests/distribution/command/qemu
-iquote /mnt/tests/distribution/command/qemu/include -iquote
/mnt/tests/distribution/command/qemu/host/include/x86_64 -iquote
/mnt/tests/distribution/command/qemu/host/include/generic -iquote
/mnt/tests/distribution/command/qemu/tcg/i386 -pthread -mcx16 -msse2
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-fno-strict-aliasing -fno-common -fwrapv -ftrivial-auto-var-init=zero
-fzero-call-used-regs=used-gpr -fPIE -DWITH_GZFILEOP
-isystem../linux-headers -isystemlinux-headers -DCOMPILING_PER_TARGET
'-DCONFIG_TARGET="x86_64-softmmu-config-target.h"'
'-DCONFIG_DEVICES="x86_64-softmmu-config-devices.h"' -MD -MQ
libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o -MF
libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o.d -o
libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o -c
../hw/block/vhost-user-blk.c
In file included from
/mnt/tests/distribution/command/qemu/migration/options.h:19,
                 from ../hw/block/vhost-user-blk.c:34:
/mnt/tests/distribution/command/qemu/include/migration/client-options.h:26:1:
error: unknown type name ‘MigMode’
   26 | MigMode migrate_mode(void);
      | ^~~~~~~
/mnt/tests/distribution/command/qemu/migration/options.h:66:7: error:
unknown type name ‘BitmapMigrationNodeAliasList’
   66 | const BitmapMigrationNodeAliasList *migrate_block_bitmap_mapping(void);
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mnt/tests/distribution/command/qemu/migration/options.h:80:1: error:
unknown type name ‘MultiFDCompression’
   80 | MultiFDCompression migrate_multifd_compression(void);
      | ^~~~~~~~~~~~~~~~~~
/mnt/tests/distribution/command/qemu/migration/options.h:89:1: error:
unknown type name ‘ZeroPageDetection’
   89 | ZeroPageDetection migrate_zero_page_detection(void);
      | ^~~~~~~~~~~~~~~~~
/mnt/tests/distribution/command/qemu/migration/options.h:93:27: error:
unknown type name ‘MigrationParameters’; did you mean
‘MigrationState’?
   93 | bool migrate_params_check(MigrationParameters *params, Error **errp);
      |                           ^~~~~~~~~~~~~~~~~~~
      |                           MigrationState
/mnt/tests/distribution/command/qemu/migration/options.h:94:26: error:
unknown type name ‘MigrationParameters’; did you mean
‘MigrationState’?
   94 | void migrate_params_init(MigrationParameters *params);
      |                          ^~~~~~~~~~~~~~~~~~~
      |                          MigrationState
ninja: build stopped: subcommand failed.
make[1]: *** [Makefile:168: run-ninja] Error 1

Thanks
Lei


On Mon, Oct 20, 2025 at 1:47 PM Alexandr Moshkov
<dtalexundeer@yandex-team.ru> wrote:
>
> In vhost_user_blk_stop() on incoming migration make force_stop = true,
> so GET_VRING_BASE will not be executed.
>
> Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c | 52 +++++++++++++++++++++++++++++++++++++++
>  migration/options.c       |  7 ++++++
>  migration/options.h       |  1 +
>  qapi/migration.json       |  9 +++++--
>  4 files changed, 67 insertions(+), 2 deletions(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index c0cc5f6942..49f67d0451 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -31,6 +31,7 @@
>  #include "hw/virtio/virtio-access.h"
>  #include "system/system.h"
>  #include "system/runstate.h"
> +#include "migration/options.h"
>
>  static const int user_feature_bits[] = {
>      VIRTIO_BLK_F_SIZE_MAX,
> @@ -224,6 +225,11 @@ static int vhost_user_blk_stop(VirtIODevice *vdev)
>      force_stop = s->skip_get_vring_base_on_force_shutdown &&
>                   qemu_force_shutdown_requested();
>
> +    if (migrate_inflight_vhost_user_blk() &&
> +        runstate_check(RUN_STATE_FINISH_MIGRATE)) {
> +        force_stop = true;
> +    }
> +
>      ret = force_stop ? vhost_dev_force_stop(&s->dev, vdev, true) :
>                         vhost_dev_stop(&s->dev, vdev, true);
>
> @@ -568,12 +574,58 @@ static struct vhost_dev *vhost_user_blk_get_vhost(VirtIODevice *vdev)
>      return &s->dev;
>  }
>
> +static int vhost_user_blk_save(QEMUFile *f, void *pv, size_t size,
> +                               const VMStateField *field, JSONWriter *vmdesc)
> +{
> +    VirtIODevice *vdev = pv;
> +    VHostUserBlk *s = VHOST_USER_BLK(vdev);
> +
> +    if (!migrate_inflight_vhost_user_blk()) {
> +        return 0;
> +    }
> +
> +    vhost_dev_save_inflight(s->inflight, f);
> +
> +    return 0;
> +}
> +
> +static int vhost_user_blk_load(QEMUFile *f, void *pv, size_t size,
> +                               const VMStateField *field)
> +{
> +    VirtIODevice *vdev = pv;
> +    VHostUserBlk *s = VHOST_USER_BLK(vdev);
> +    int ret;
> +
> +    if (!migrate_inflight_vhost_user_blk()) {
> +        return 0;
> +    }
> +
> +    ret = vhost_dev_load_inflight(s->inflight, f);
> +    if (ret < 0) {
> +        g_autofree char *path = object_get_canonical_path(OBJECT(vdev));
> +        error_report("%s [%s]: can't load in-flight requests",
> +                     path, TYPE_VHOST_USER_BLK);
> +        return ret;
> +    }
> +
> +    return 0;
> +}
> +
>  static const VMStateDescription vmstate_vhost_user_blk = {
>      .name = "vhost-user-blk",
>      .minimum_version_id = 1,
>      .version_id = 1,
>      .fields = (const VMStateField[]) {
>          VMSTATE_VIRTIO_DEVICE,
> +        {
> +            .name = "backend state",
> +            .info = &(const VMStateInfo) {
> +                .name = "vhost-user-blk backend state",
> +                .get = vhost_user_blk_load,
> +                .put = vhost_user_blk_save,
> +            },
> +            .flags = VMS_SINGLE,
> +        },
>          VMSTATE_END_OF_LIST()
>      },
>  };
> diff --git a/migration/options.c b/migration/options.c
> index 5183112775..fcae2b4559 100644
> --- a/migration/options.c
> +++ b/migration/options.c
> @@ -262,6 +262,13 @@ bool migrate_mapped_ram(void)
>      return s->capabilities[MIGRATION_CAPABILITY_MAPPED_RAM];
>  }
>
> +bool migrate_inflight_vhost_user_blk(void)
> +{
> +    MigrationState *s = migrate_get_current();
> +
> +    return s->capabilities[MIGRATION_CAPABILITY_INFLIGHT_VHOST_USER_BLK];
> +}
> +
>  bool migrate_ignore_shared(void)
>  {
>      MigrationState *s = migrate_get_current();
> diff --git a/migration/options.h b/migration/options.h
> index 82d839709e..eab1485d1a 100644
> --- a/migration/options.h
> +++ b/migration/options.h
> @@ -30,6 +30,7 @@ bool migrate_colo(void);
>  bool migrate_dirty_bitmaps(void);
>  bool migrate_events(void);
>  bool migrate_mapped_ram(void);
> +bool migrate_inflight_vhost_user_blk(void);
>  bool migrate_ignore_shared(void);
>  bool migrate_late_block_activate(void);
>  bool migrate_multifd(void);
> diff --git a/qapi/migration.json b/qapi/migration.json
> index be0f3fcc12..c9fea59515 100644
> --- a/qapi/migration.json
> +++ b/qapi/migration.json
> @@ -517,9 +517,13 @@
>  #     each RAM page.  Requires a migration URI that supports seeking,
>  #     such as a file.  (since 9.0)
>  #
> +# @inflight-vhost-user-blk: If enabled, QEMU will migrate inflight
> +#    region for vhost-user-blk.  (since 10.2)
> +#
>  # Features:
>  #
> -# @unstable: Members @x-colo and @x-ignore-shared are experimental.
> +# @unstable: Members @x-colo and @x-ignore-shared,
> +#     @inflight-vhost-user-blk are experimental.
>  # @deprecated: Member @zero-blocks is deprecated as being part of
>  #     block migration which was already removed.
>  #
> @@ -536,7 +540,8 @@
>             { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
>             'validate-uuid', 'background-snapshot',
>             'zero-copy-send', 'postcopy-preempt', 'switchover-ack',
> -           'dirty-limit', 'mapped-ram'] }
> +           'dirty-limit', 'mapped-ram',
> +           { 'name': 'inflight-vhost-user-blk', 'features': [ 'unstable' ] } ] }
>
>  ##
>  # @MigrationCapabilityStatus:
> --
> 2.34.1
>
>
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Alexandr Moshkov 3 weeks ago
Hi, thanks for testing!

On 10/23/25 19:29, Lei Yang wrote:
> Hi Alexandr
>
> According to my test result, this series of patches introduce issues,
> it prints the following error messages when compiling the process
> after applying your patch.
> The test based on this commit:
> commit 3a2d5612a7422732b648b46d4b934e2e54622fd6 (origin/master, origin/HEAD)
> Author: Peter Maydell<peter.maydell@linaro.org>
> Date:   Fri Oct 17 14:31:56 2025 +0100
>
> Error messages:
> [1849/2964] Compiling C object
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o
> FAILED: libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o
> cc -m64 -Ilibqemu-x86_64-softmmu.a.p -I. -I.. -Itarget/i386
> -I../target/i386 -Isubprojects/dtc/libfdt -I../subprojects/dtc/libfdt
> -Isubprojects/libvduse -I../subprojects/libvduse -Iqapi -Itrace -Iui
> -Iui/shader -I/usr/include/pixman-1 -I/usr/include/glib-2.0
> -I/usr/lib64/glib-2.0/include -I/usr/include/libmount
> -I/usr/include/blkid -I/usr/include/sysprof-6
> -I/usr/include/gio-unix-2.0 -I/usr/include/slirp
> -fdiagnostics-color=auto -Wall -Winvalid-pch -Werror -std=gnu11 -O0 -g
> -fstack-protector-strong -Wempty-body -Wendif-labels
> -Wexpansion-to-defined -Wformat-security -Wformat-y2k
> -Wignored-qualifiers -Wimplicit-fallthrough=2 -Winit-self
> -Wmissing-format-attribute -Wmissing-prototypes -Wnested-externs
> -Wold-style-declaration -Wold-style-definition -Wredundant-decls
> -Wshadow=local -Wstrict-prototypes -Wtype-limits -Wundef -Wvla
> -Wwrite-strings -Wno-missing-include-dirs -Wno-psabi
> -Wno-shift-negative-value -isystem
> /mnt/tests/distribution/command/qemu/linux-headers -isystem
> linux-headers -iquote . -iquote /mnt/tests/distribution/command/qemu
> -iquote /mnt/tests/distribution/command/qemu/include -iquote
> /mnt/tests/distribution/command/qemu/host/include/x86_64 -iquote
> /mnt/tests/distribution/command/qemu/host/include/generic -iquote
> /mnt/tests/distribution/command/qemu/tcg/i386 -pthread -mcx16 -msse2
> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
> -fno-strict-aliasing -fno-common -fwrapv -ftrivial-auto-var-init=zero
> -fzero-call-used-regs=used-gpr -fPIE -DWITH_GZFILEOP
> -isystem../linux-headers -isystemlinux-headers -DCOMPILING_PER_TARGET
> '-DCONFIG_TARGET="x86_64-softmmu-config-target.h"'
> '-DCONFIG_DEVICES="x86_64-softmmu-config-devices.h"' -MD -MQ
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o -MF
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o.d -o
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o -c
> ../hw/block/vhost-user-blk.c
> In file included from
> /mnt/tests/distribution/command/qemu/migration/options.h:19,
>                   from ../hw/block/vhost-user-blk.c:34:
> /mnt/tests/distribution/command/qemu/include/migration/client-options.h:26:1:
> error: unknown type name ‘MigMode’
>     26 | MigMode migrate_mode(void);
>        | ^~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:66:7: error:
> unknown type name ‘BitmapMigrationNodeAliasList’
>     66 | const BitmapMigrationNodeAliasList *migrate_block_bitmap_mapping(void);
>        |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:80:1: error:
> unknown type name ‘MultiFDCompression’
>     80 | MultiFDCompression migrate_multifd_compression(void);
>        | ^~~~~~~~~~~~~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:89:1: error:
> unknown type name ‘ZeroPageDetection’
>     89 | ZeroPageDetection migrate_zero_page_detection(void);
>        | ^~~~~~~~~~~~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:93:27: error:
> unknown type name ‘MigrationParameters’; did you mean
> ‘MigrationState’?
>     93 | bool migrate_params_check(MigrationParameters *params, Error **errp);
>        |                           ^~~~~~~~~~~~~~~~~~~
>        |                           MigrationState
> /mnt/tests/distribution/command/qemu/migration/options.h:94:26: error:
> unknown type name ‘MigrationParameters’; did you mean
> ‘MigrationState’?
>     94 | void migrate_params_init(MigrationParameters *params);
>        |                          ^~~~~~~~~~~~~~~~~~~
>        |                          MigrationState
> ninja: build stopped: subcommand failed.
> make[1]: *** [Makefile:168: run-ninja] Error 1
>
> Thanks
> Lei

I have the same issue on my machine...

In file included from 
/home/dtalexundeer/code/qemu-upstream/migration/options.h:19,
                  from ../hw/block/vhost-user-blk.c:34:
/home/dtalexundeer/code/qemu-upstream/include/migration/client-options.h:26:1: 
error: unknown type name ‘MigMode’
    26 | MigMode migrate_mode(void);
       | ^~~~~~~
In file included from ../hw/block/vhost-user-blk.c:34:
/home/dtalexundeer/code/qemu-upstream/migration/options.h:65:7: error: 
unknown type name ‘BitmapMigrationNodeAliasList’
    65 | const BitmapMigrationNodeAliasList 
*migrate_block_bitmap_mapping(void);
       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/dtalexundeer/code/qemu-upstream/migration/options.h:79:1: error: 
unknown type name ‘MultiFDCompression’
    79 | MultiFDCompression migrate_multifd_compression(void);
       | ^~~~~~~~~~~~~~~~~~
/home/dtalexundeer/code/qemu-upstream/migration/options.h:88:1: error: 
unknown type name ‘ZeroPageDetection’
    88 | ZeroPageDetection migrate_zero_page_detection(void);
       | ^~~~~~~~~~~~~~~~~
/home/dtalexundeer/code/qemu-upstream/migration/options.h:92:27: error: 
unknown type name ‘MigrationParameters’; did you mean ‘MigrationState’?
    92 | bool migrate_params_check(MigrationParameters *params, Error 
**errp);
       |                           ^~~~~~~~~~~~~~~~~~~
       |                           MigrationState
/home/dtalexundeer/code/qemu-upstream/migration/options.h:93:26: error: 
unknown type name ‘MigrationParameters’; did you mean ‘MigrationState’?
    93 | void migrate_params_init(MigrationParameters *params);
       |                          ^~~~~~~~~~~~~~~~~~~
       |                          MigrationState

When I send a patch, did you know, is any CI performed (like compilation)?

I was thinking that there was some issue in my environment (or some 
other code) because it even reproduces on the latest master if I add 
this include to the vhost-user-blk.c file:

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 
c0cc5f6942..70235737f0 100644 --- a/hw/block/vhost-user-blk.c +++ 
b/hw/block/vhost-user-blk.c @@ -31,6 +31,7 @@ #include 
"hw/virtio/virtio-access.h" #include "system/system.h" #include 
"system/runstate.h" +#include "migration/options.h"

So, it looks like there is a problem with the client-options.h file 
(that needs to include the qapi file) or in qapi generation process. If 
I apply this patch:

diff --git a/include/migration/client-options.h 
b/include/migration/client-options.h index 289c9d7762..38cf53388d 100644 
--- a/include/migration/client-options.h +++ 
b/include/migration/client-options.h @@ -10,6 +10,7 @@ #ifndef 
QEMU_MIGRATION_CLIENT_OPTIONS_H #define QEMU_MIGRATION_CLIENT_OPTIONS_H 
+#include "qapi/qapi-types-migration.h" /* properties */

the problem goes away..
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Lei Yang 2 weeks, 6 days ago
On Fri, Oct 24, 2025 at 4:38 PM Alexandr Moshkov
<dtalexundeer@yandex-team.ru> wrote:
>
> Hi, thanks for testing!
>
> On 10/23/25 19:29, Lei Yang wrote:
>
> Hi Alexandr
>
> According to my test result, this series of patches introduce issues,
> it prints the following error messages when compiling the process
> after applying your patch.
> The test based on this commit:
> commit 3a2d5612a7422732b648b46d4b934e2e54622fd6 (origin/master, origin/HEAD)
> Author: Peter Maydell <peter.maydell@linaro.org>
> Date:   Fri Oct 17 14:31:56 2025 +0100
>
> Error messages:
> [1849/2964] Compiling C object
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o
> FAILED: libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o
> cc -m64 -Ilibqemu-x86_64-softmmu.a.p -I. -I.. -Itarget/i386
> -I../target/i386 -Isubprojects/dtc/libfdt -I../subprojects/dtc/libfdt
> -Isubprojects/libvduse -I../subprojects/libvduse -Iqapi -Itrace -Iui
> -Iui/shader -I/usr/include/pixman-1 -I/usr/include/glib-2.0
> -I/usr/lib64/glib-2.0/include -I/usr/include/libmount
> -I/usr/include/blkid -I/usr/include/sysprof-6
> -I/usr/include/gio-unix-2.0 -I/usr/include/slirp
> -fdiagnostics-color=auto -Wall -Winvalid-pch -Werror -std=gnu11 -O0 -g
> -fstack-protector-strong -Wempty-body -Wendif-labels
> -Wexpansion-to-defined -Wformat-security -Wformat-y2k
> -Wignored-qualifiers -Wimplicit-fallthrough=2 -Winit-self
> -Wmissing-format-attribute -Wmissing-prototypes -Wnested-externs
> -Wold-style-declaration -Wold-style-definition -Wredundant-decls
> -Wshadow=local -Wstrict-prototypes -Wtype-limits -Wundef -Wvla
> -Wwrite-strings -Wno-missing-include-dirs -Wno-psabi
> -Wno-shift-negative-value -isystem
> /mnt/tests/distribution/command/qemu/linux-headers -isystem
> linux-headers -iquote . -iquote /mnt/tests/distribution/command/qemu
> -iquote /mnt/tests/distribution/command/qemu/include -iquote
> /mnt/tests/distribution/command/qemu/host/include/x86_64 -iquote
> /mnt/tests/distribution/command/qemu/host/include/generic -iquote
> /mnt/tests/distribution/command/qemu/tcg/i386 -pthread -mcx16 -msse2
> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
> -fno-strict-aliasing -fno-common -fwrapv -ftrivial-auto-var-init=zero
> -fzero-call-used-regs=used-gpr -fPIE -DWITH_GZFILEOP
> -isystem../linux-headers -isystemlinux-headers -DCOMPILING_PER_TARGET
> '-DCONFIG_TARGET="x86_64-softmmu-config-target.h"'
> '-DCONFIG_DEVICES="x86_64-softmmu-config-devices.h"' -MD -MQ
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o -MF
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o.d -o
> libqemu-x86_64-softmmu.a.p/hw_block_vhost-user-blk.c.o -c
> ../hw/block/vhost-user-blk.c
> In file included from
> /mnt/tests/distribution/command/qemu/migration/options.h:19,
>                  from ../hw/block/vhost-user-blk.c:34:
> /mnt/tests/distribution/command/qemu/include/migration/client-options.h:26:1:
> error: unknown type name ‘MigMode’
>    26 | MigMode migrate_mode(void);
>       | ^~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:66:7: error:
> unknown type name ‘BitmapMigrationNodeAliasList’
>    66 | const BitmapMigrationNodeAliasList *migrate_block_bitmap_mapping(void);
>       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:80:1: error:
> unknown type name ‘MultiFDCompression’
>    80 | MultiFDCompression migrate_multifd_compression(void);
>       | ^~~~~~~~~~~~~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:89:1: error:
> unknown type name ‘ZeroPageDetection’
>    89 | ZeroPageDetection migrate_zero_page_detection(void);
>       | ^~~~~~~~~~~~~~~~~
> /mnt/tests/distribution/command/qemu/migration/options.h:93:27: error:
> unknown type name ‘MigrationParameters’; did you mean
> ‘MigrationState’?
>    93 | bool migrate_params_check(MigrationParameters *params, Error **errp);
>       |                           ^~~~~~~~~~~~~~~~~~~
>       |                           MigrationState
> /mnt/tests/distribution/command/qemu/migration/options.h:94:26: error:
> unknown type name ‘MigrationParameters’; did you mean
> ‘MigrationState’?
>    94 | void migrate_params_init(MigrationParameters *params);
>       |                          ^~~~~~~~~~~~~~~~~~~
>       |                          MigrationState
> ninja: build stopped: subcommand failed.
> make[1]: *** [Makefile:168: run-ninja] Error 1
>
> Thanks
> Lei
>
> I have the same issue on my machine...
>
> In file included from /home/dtalexundeer/code/qemu-upstream/migration/options.h:19,
>                  from ../hw/block/vhost-user-blk.c:34:
> /home/dtalexundeer/code/qemu-upstream/include/migration/client-options.h:26:1: error: unknown type name ‘MigMode’
>    26 | MigMode migrate_mode(void);
>       | ^~~~~~~
> In file included from ../hw/block/vhost-user-blk.c:34:
> /home/dtalexundeer/code/qemu-upstream/migration/options.h:65:7: error: unknown type name ‘BitmapMigrationNodeAliasList’
>    65 | const BitmapMigrationNodeAliasList *migrate_block_bitmap_mapping(void);
>       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /home/dtalexundeer/code/qemu-upstream/migration/options.h:79:1: error: unknown type name ‘MultiFDCompression’
>    79 | MultiFDCompression migrate_multifd_compression(void);
>       | ^~~~~~~~~~~~~~~~~~
> /home/dtalexundeer/code/qemu-upstream/migration/options.h:88:1: error: unknown type name ‘ZeroPageDetection’
>    88 | ZeroPageDetection migrate_zero_page_detection(void);
>       | ^~~~~~~~~~~~~~~~~
> /home/dtalexundeer/code/qemu-upstream/migration/options.h:92:27: error: unknown type name ‘MigrationParameters’; did you mean ‘MigrationState’?
>    92 | bool migrate_params_check(MigrationParameters *params, Error **errp);
>       |                           ^~~~~~~~~~~~~~~~~~~
>       |                           MigrationState
> /home/dtalexundeer/code/qemu-upstream/migration/options.h:93:26: error: unknown type name ‘MigrationParameters’; did you mean ‘MigrationState’?
>    93 | void migrate_params_init(MigrationParameters *params);
>       |                          ^~~~~~~~~~~~~~~~~~~
>       |                          MigrationState
>

Hi Alexandr

> When I send a patch, did you know, is any CI performed (like compilation)?

To be honest, I'm not sure if there's such a check in place upstream.
I discovered this issue because my local CI tool caught the files
modified by your patch, found the ones I was interested in, triggered
the tests, and only discovered the issue when I looked at the results.

Thanks
Lei
>
> I was thinking that there was some issue in my environment (or some other code) because it even reproduces on the latest master if I add this include to the vhost-user-blk.c file:
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index c0cc5f6942..70235737f0 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -31,6 +31,7 @@ #include "hw/virtio/virtio-access.h" #include "system/system.h" #include "system/runstate.h" +#include "migration/options.h"
>
> So, it looks like there is a problem with the client-options.h file (that needs to include the qapi file) or in qapi generation process. If I apply this patch:
>
> diff --git a/include/migration/client-options.h b/include/migration/client-options.h index 289c9d7762..38cf53388d 100644 --- a/include/migration/client-options.h +++ b/include/migration/client-options.h @@ -10,6 +10,7 @@ #ifndef QEMU_MIGRATION_CLIENT_OPTIONS_H #define QEMU_MIGRATION_CLIENT_OPTIONS_H +#include "qapi/qapi-types-migration.h" /* properties */
>
> the problem goes away..
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Markus Armbruster 3 weeks, 4 days ago
Alexandr Moshkov <dtalexundeer@yandex-team.ru> writes:

> In vhost_user_blk_stop() on incoming migration make force_stop = true,
> so GET_VRING_BASE will not be executed.
>
> Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>

Your cover letter explains why this is useful.  Please work it into your
commit message.

[...]

> diff --git a/qapi/migration.json b/qapi/migration.json
> index be0f3fcc12..c9fea59515 100644
> --- a/qapi/migration.json
> +++ b/qapi/migration.json
> @@ -517,9 +517,13 @@
>  #     each RAM page.  Requires a migration URI that supports seeking,
>  #     such as a file.  (since 9.0)
>  #
> +# @inflight-vhost-user-blk: If enabled, QEMU will migrate inflight
> +#    region for vhost-user-blk.  (since 10.2)
> +#

Any guidance why and when users would want to enable it?

Is it a good idea to have device-specific capabilities?

>  # Features:
>  #
> -# @unstable: Members @x-colo and @x-ignore-shared are experimental.
> +# @unstable: Members @x-colo and @x-ignore-shared,
> +#     @inflight-vhost-user-blk are experimental.

"and" is misplaced now.  Fix:

   # @unstable: Members @x-colo, @x-ignore-shared, and
   #     @inflight-vhost-user-blk are experimental.

Use the opportunity and insert a blank line here.

>  # @deprecated: Member @zero-blocks is deprecated as being part of
>  #     block migration which was already removed.
>  #
> @@ -536,7 +540,8 @@
>             { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
>             'validate-uuid', 'background-snapshot',
>             'zero-copy-send', 'postcopy-preempt', 'switchover-ack',
> -           'dirty-limit', 'mapped-ram'] }
> +           'dirty-limit', 'mapped-ram',
> +           { 'name': 'inflight-vhost-user-blk', 'features': [ 'unstable' ] } ] }

Long line.  Obvious line break:

              { 'name': 'inflight-vhost-user-blk',
                'features': [ 'unstable' ] } ] }

>  
>  ##
>  # @MigrationCapabilityStatus:
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Alexandr Moshkov 3 weeks, 4 days ago
Thanks for review!

On 10/20/25 14:55, Markus Armbruster wrote:
> Alexandr Moshkov<dtalexundeer@yandex-team.ru> writes:
>
>> In vhost_user_blk_stop() on incoming migration make force_stop = true,
>> so GET_VRING_BASE will not be executed.
>>
>> Signed-off-by: Alexandr Moshkov<dtalexundeer@yandex-team.ru>
> Your cover letter explains why this is useful.  Please work it into your
> commit message.

Ok

> [...]
>
>> diff --git a/qapi/migration.json b/qapi/migration.json
>> index be0f3fcc12..c9fea59515 100644
>> --- a/qapi/migration.json
>> +++ b/qapi/migration.json
>> @@ -517,9 +517,13 @@
>>   #     each RAM page.  Requires a migration URI that supports seeking,
>>   #     such as a file.  (since 9.0)
>>   #
>> +# @inflight-vhost-user-blk: If enabled, QEMU will migrate inflight
>> +#    region for vhost-user-blk.  (since 10.2)
>> +#
> Any guidance why and when users would want to enable it?
>
> Is it a good idea to have device-specific capabilities?

Hmm, maybe it's better way to make a parameter for the vhost-user-blk 
instead of migration capability?

What do you think?

>>   # Features:
>>   #
>> -# @unstable: Members @x-colo and @x-ignore-shared are experimental.
>> +# @unstable: Members @x-colo and @x-ignore-shared,
>> +#     @inflight-vhost-user-blk are experimental.
> "and" is misplaced now.  Fix:
>
>     # @unstable: Members @x-colo, @x-ignore-shared, and
>     #     @inflight-vhost-user-blk are experimental.
>
> Use the opportunity and insert a blank line here.
>
>>   # @deprecated: Member @zero-blocks is deprecated as being part of
>>   #     block migration which was already removed.
>>   #
>> @@ -536,7 +540,8 @@
>>              { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
>>              'validate-uuid', 'background-snapshot',
>>              'zero-copy-send', 'postcopy-preempt', 'switchover-ack',
>> -           'dirty-limit', 'mapped-ram'] }
>> +           'dirty-limit', 'mapped-ram',
>> +           { 'name': 'inflight-vhost-user-blk', 'features': [ 'unstable' ] } ] }
> Long line.  Obvious line break:
>
>                { 'name': 'inflight-vhost-user-blk',
>                  'features': [ 'unstable' ] } ] }
>
>>   
>>   ##
>>   # @MigrationCapabilityStatus:
Wiil be fixed, thank!
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Markus Armbruster 3 weeks, 4 days ago
Alexandr Moshkov <dtalexundeer@yandex-team.ru> writes:

> Thanks for review!
>
> On 10/20/25 14:55, Markus Armbruster wrote:
>> Alexandr Moshkov<dtalexundeer@yandex-team.ru> writes:
>>
>>> In vhost_user_blk_stop() on incoming migration make force_stop = true,
>>> so GET_VRING_BASE will not be executed.
>>>
>>> Signed-off-by: Alexandr Moshkov<dtalexundeer@yandex-team.ru>
>> Your cover letter explains why this is useful.  Please work it into your
>> commit message.
>
> Ok
>
>> [...]
>>
>>> diff --git a/qapi/migration.json b/qapi/migration.json
>>> index be0f3fcc12..c9fea59515 100644
>>> --- a/qapi/migration.json
>>> +++ b/qapi/migration.json
>>> @@ -517,9 +517,13 @@
>>>  #     each RAM page.  Requires a migration URI that supports seeking,
>>>  #     such as a file.  (since 9.0)
>>>  #
>>> +# @inflight-vhost-user-blk: If enabled, QEMU will migrate inflight
>>> +#    region for vhost-user-blk.  (since 10.2)
>>> +#
>> Any guidance why and when users would want to enable it?
>>
>> Is it a good idea to have device-specific capabilities?
>
> Hmm, maybe it's better way to make a parameter for the vhost-user-blk instead of migration capability?
>
> What do you think?

I think this is a question for the migration maintainers :)

[...]
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Peter Xu 3 weeks ago
On Mon, Oct 20, 2025 at 01:47:12PM +0200, Markus Armbruster wrote:
> Alexandr Moshkov <dtalexundeer@yandex-team.ru> writes:
> 
> > Thanks for review!
> >
> > On 10/20/25 14:55, Markus Armbruster wrote:
> >> Alexandr Moshkov<dtalexundeer@yandex-team.ru> writes:
> >>
> >>> In vhost_user_blk_stop() on incoming migration make force_stop = true,
> >>> so GET_VRING_BASE will not be executed.
> >>>
> >>> Signed-off-by: Alexandr Moshkov<dtalexundeer@yandex-team.ru>
> >> Your cover letter explains why this is useful.  Please work it into your
> >> commit message.
> >
> > Ok
> >
> >> [...]
> >>
> >>> diff --git a/qapi/migration.json b/qapi/migration.json
> >>> index be0f3fcc12..c9fea59515 100644
> >>> --- a/qapi/migration.json
> >>> +++ b/qapi/migration.json
> >>> @@ -517,9 +517,13 @@
> >>>  #     each RAM page.  Requires a migration URI that supports seeking,
> >>>  #     such as a file.  (since 9.0)
> >>>  #
> >>> +# @inflight-vhost-user-blk: If enabled, QEMU will migrate inflight
> >>> +#    region for vhost-user-blk.  (since 10.2)
> >>> +#
> >> Any guidance why and when users would want to enable it?
> >>
> >> Is it a good idea to have device-specific capabilities?
> >
> > Hmm, maybe it's better way to make a parameter for the vhost-user-blk instead of migration capability?
> >
> > What do you think?
> 
> I think this is a question for the migration maintainers :)

Oops, I missed this email previously..

We discussed similar things with Vladimir on virtio-net.  Unless extremely
necessary, we should avoid adding any cap into migration that is relevant
to a specific device.  Yes, per-device is better..

Thanks,

-- 
Peter Xu
Re: [PATCH 2/2] vhost-user-blk: support inflight migration
Posted by Alexandr Moshkov 3 weeks ago
On 10/24/25 00:24, Peter Xu wrote:
> On Mon, Oct 20, 2025 at 01:47:12PM +0200, Markus Armbruster wrote:
>> Alexandr Moshkov <dtalexundeer@yandex-team.ru> writes:
>>>> [...]
>>>>
>>>>> diff --git a/qapi/migration.json b/qapi/migration.json
>>>>> index be0f3fcc12..c9fea59515 100644
>>>>> --- a/qapi/migration.json
>>>>> +++ b/qapi/migration.json
>>>>> @@ -517,9 +517,13 @@
>>>>>   #     each RAM page.  Requires a migration URI that supports seeking,
>>>>>   #     such as a file.  (since 9.0)
>>>>>   #
>>>>> +# @inflight-vhost-user-blk: If enabled, QEMU will migrate inflight
>>>>> +#    region for vhost-user-blk.  (since 10.2)
>>>>> +#
>>>> Any guidance why and when users would want to enable it?
>>>>
>>>> Is it a good idea to have device-specific capabilities?
>>> Hmm, maybe it's better way to make a parameter for the vhost-user-blk instead of migration capability?
>>>
>>> What do you think?
>> I think this is a question for the migration maintainers :)
> Oops, I missed this email previously..
>
> We discussed similar things with Vladimir on virtio-net.  Unless extremely
> necessary, we should avoid adding any cap into migration that is relevant
> to a specific device.  Yes, per-device is better..
>
> Thanks,
Arlight, thanks!