[PATCH v6 2/5] vhost-user: introduce protocol feature for skip drain on GET_VRING_BASE

Alexandr Moshkov posted 5 patches 4 weeks ago
There is a newer version of this series
[PATCH v6 2/5] vhost-user: introduce protocol feature for skip drain on GET_VRING_BASE
Posted by Alexandr Moshkov 4 weeks ago
Add vhost-user protocol feature
VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT

Now on GET_VRING_BASE this feature can control whether to wait for
in-flight requests to complete or not.
Also we have to validate that this feature will be enabled only when
qemu and back-end supports in-flight buffer and in-flight migration

It will be helpfull in future for in-flight requests migration in
vhost-user devices.

Update docs, add ref to label for inflight-io-tracking

Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>
---
 docs/interop/vhost-user.rst    | 59 +++++++++++++++++++++-------------
 hw/virtio/vhost-user.c         |  5 +++
 include/hw/virtio/vhost-user.h |  2 ++
 3 files changed, 44 insertions(+), 22 deletions(-)

diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
index 02908b48fa..dcf79de0c0 100644
--- a/docs/interop/vhost-user.rst
+++ b/docs/interop/vhost-user.rst
@@ -736,6 +736,8 @@ negotiated, back-end can send file descriptors (at most 8 descriptors in
 each message) to front-end via ancillary data using this fd communication
 channel.
 
+.. _inflight_io_tracking:
+
 Inflight I/O tracking
 ---------------------
 
@@ -1033,26 +1035,27 @@ Protocol features
 
 .. code:: c
 
-  #define VHOST_USER_PROTOCOL_F_MQ                    0
-  #define VHOST_USER_PROTOCOL_F_LOG_SHMFD             1
-  #define VHOST_USER_PROTOCOL_F_RARP                  2
-  #define VHOST_USER_PROTOCOL_F_REPLY_ACK             3
-  #define VHOST_USER_PROTOCOL_F_MTU                   4
-  #define VHOST_USER_PROTOCOL_F_BACKEND_REQ           5
-  #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN          6
-  #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION        7
-  #define VHOST_USER_PROTOCOL_F_PAGEFAULT             8
-  #define VHOST_USER_PROTOCOL_F_CONFIG                9
-  #define VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD      10
-  #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER        11
-  #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD       12
-  #define VHOST_USER_PROTOCOL_F_RESET_DEVICE         13
-  #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
-  #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS  15
-  #define VHOST_USER_PROTOCOL_F_STATUS               16
-  #define VHOST_USER_PROTOCOL_F_XEN_MMAP             17
-  #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT        18
-  #define VHOST_USER_PROTOCOL_F_DEVICE_STATE         19
+  #define VHOST_USER_PROTOCOL_F_MQ                       0
+  #define VHOST_USER_PROTOCOL_F_LOG_SHMFD                1
+  #define VHOST_USER_PROTOCOL_F_RARP                     2
+  #define VHOST_USER_PROTOCOL_F_REPLY_ACK                3
+  #define VHOST_USER_PROTOCOL_F_MTU                      4
+  #define VHOST_USER_PROTOCOL_F_BACKEND_REQ              5
+  #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN             6
+  #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION           7
+  #define VHOST_USER_PROTOCOL_F_PAGEFAULT                8
+  #define VHOST_USER_PROTOCOL_F_CONFIG                   9
+  #define VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD         10
+  #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER           11
+  #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD          12
+  #define VHOST_USER_PROTOCOL_F_RESET_DEVICE            13
+  #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS    14
+  #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS     15
+  #define VHOST_USER_PROTOCOL_F_STATUS                  16
+  #define VHOST_USER_PROTOCOL_F_XEN_MMAP                17
+  #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT           18
+  #define VHOST_USER_PROTOCOL_F_DEVICE_STATE            19
+  #define VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT 20
 
 Front-end message types
 -----------------------
@@ -1243,12 +1246,24 @@ Front-end message types
 
   When and as long as all of a device's vrings are stopped, it is
   *suspended*, see :ref:`Suspended device state
-  <suspended_device_state>`. The back-end must complete all inflight I/O
-  requests for the specified vring before stopping it.
+  <suspended_device_state>`.
 
   The request payload's *num* field is currently reserved and must be
   set to 0.
 
+  By default, the back-end must complete all inflight I/O requests for the
+  specified vring before stopping it.
+
+  If the ``VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT`` protocol
+  feature has been negotiated, the back-end may suspend in-flight I/O
+  requests and record them as described in :ref:`Inflight I/O tracking
+  <inflight_io_tracking>` instead of completing them before stopping the vring.
+  How to suspend an in-flight request depends on the implementation of the back-end
+  but it typically can be done by aborting or cancelling the underlying I/O
+  request. The ``VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT``
+  protocol feature must only be neogotiated if
+  ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` is also negotiated.
+
 ``VHOST_USER_SET_VRING_KICK``
   :id: 12
   :equivalent ioctl: ``VHOST_SET_VRING_KICK``
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index a820214188..793d1b36d8 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -2320,6 +2320,11 @@ static int vhost_user_backend_connect(struct vhost_dev *dev, Error **errp)
             }
         }
 
+        if (!u->user->supports_inflight_migration ||
+            !virtio_has_feature(VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD)) {
+            protocol_features &= ~(1ULL <<
+                               VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT);
+        }
+
         /* final set of protocol features */
         u->protocol_features = protocol_features;
         err = vhost_user_set_protocol_features(dev, u->protocol_features);
diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
index fb89268de2..f30c1792e4 100644
--- a/include/hw/virtio/vhost-user.h
+++ b/include/hw/virtio/vhost-user.h
@@ -33,6 +33,7 @@ enum VhostUserProtocolFeature {
     /* Feature 17 reserved for VHOST_USER_PROTOCOL_F_XEN_MMAP. */
     VHOST_USER_PROTOCOL_F_SHARED_OBJECT = 18,
     VHOST_USER_PROTOCOL_F_DEVICE_STATE = 19,
+    VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT = 20,
     VHOST_USER_PROTOCOL_F_MAX
 };
 
@@ -69,6 +70,7 @@ typedef struct VhostUserState {
     GPtrArray *notifiers;
     int memory_slots;
     bool supports_config;
+    bool supports_inflight_migration;
 } VhostUserState;
 
 /**
-- 
2.34.1
Re: [PATCH v6 2/5] vhost-user: introduce protocol feature for skip drain on GET_VRING_BASE
Posted by Stefan Hajnoczi 3 weeks, 6 days ago
On Tue, Jan 13, 2026 at 02:58:13PM +0500, Alexandr Moshkov wrote:
> Add vhost-user protocol feature
> VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT
> 
> Now on GET_VRING_BASE this feature can control whether to wait for
> in-flight requests to complete or not.
> Also we have to validate that this feature will be enabled only when
> qemu and back-end supports in-flight buffer and in-flight migration
> 
> It will be helpfull in future for in-flight requests migration in
> vhost-user devices.
> 
> Update docs, add ref to label for inflight-io-tracking
> 
> Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>
> ---
>  docs/interop/vhost-user.rst    | 59 +++++++++++++++++++++-------------
>  hw/virtio/vhost-user.c         |  5 +++
>  include/hw/virtio/vhost-user.h |  2 ++
>  3 files changed, 44 insertions(+), 22 deletions(-)
> 
> diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
> index 02908b48fa..dcf79de0c0 100644
> --- a/docs/interop/vhost-user.rst
> +++ b/docs/interop/vhost-user.rst
> @@ -736,6 +736,8 @@ negotiated, back-end can send file descriptors (at most 8 descriptors in
>  each message) to front-end via ancillary data using this fd communication
>  channel.
>  
> +.. _inflight_io_tracking:
> +
>  Inflight I/O tracking
>  ---------------------
>  
> @@ -1033,26 +1035,27 @@ Protocol features
>  
>  .. code:: c
>  
> -  #define VHOST_USER_PROTOCOL_F_MQ                    0
> -  #define VHOST_USER_PROTOCOL_F_LOG_SHMFD             1
> -  #define VHOST_USER_PROTOCOL_F_RARP                  2
> -  #define VHOST_USER_PROTOCOL_F_REPLY_ACK             3
> -  #define VHOST_USER_PROTOCOL_F_MTU                   4
> -  #define VHOST_USER_PROTOCOL_F_BACKEND_REQ           5
> -  #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN          6
> -  #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION        7
> -  #define VHOST_USER_PROTOCOL_F_PAGEFAULT             8
> -  #define VHOST_USER_PROTOCOL_F_CONFIG                9
> -  #define VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD      10
> -  #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER        11
> -  #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD       12
> -  #define VHOST_USER_PROTOCOL_F_RESET_DEVICE         13
> -  #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
> -  #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS  15
> -  #define VHOST_USER_PROTOCOL_F_STATUS               16
> -  #define VHOST_USER_PROTOCOL_F_XEN_MMAP             17
> -  #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT        18
> -  #define VHOST_USER_PROTOCOL_F_DEVICE_STATE         19
> +  #define VHOST_USER_PROTOCOL_F_MQ                       0
> +  #define VHOST_USER_PROTOCOL_F_LOG_SHMFD                1
> +  #define VHOST_USER_PROTOCOL_F_RARP                     2
> +  #define VHOST_USER_PROTOCOL_F_REPLY_ACK                3
> +  #define VHOST_USER_PROTOCOL_F_MTU                      4
> +  #define VHOST_USER_PROTOCOL_F_BACKEND_REQ              5
> +  #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN             6
> +  #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION           7
> +  #define VHOST_USER_PROTOCOL_F_PAGEFAULT                8
> +  #define VHOST_USER_PROTOCOL_F_CONFIG                   9
> +  #define VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD         10
> +  #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER           11
> +  #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD          12
> +  #define VHOST_USER_PROTOCOL_F_RESET_DEVICE            13
> +  #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS    14
> +  #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS     15
> +  #define VHOST_USER_PROTOCOL_F_STATUS                  16
> +  #define VHOST_USER_PROTOCOL_F_XEN_MMAP                17
> +  #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT           18
> +  #define VHOST_USER_PROTOCOL_F_DEVICE_STATE            19
> +  #define VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT 20
>  
>  Front-end message types
>  -----------------------
> @@ -1243,12 +1246,24 @@ Front-end message types
>  
>    When and as long as all of a device's vrings are stopped, it is
>    *suspended*, see :ref:`Suspended device state
> -  <suspended_device_state>`. The back-end must complete all inflight I/O
> -  requests for the specified vring before stopping it.
> +  <suspended_device_state>`.
>  
>    The request payload's *num* field is currently reserved and must be
>    set to 0.
>  
> +  By default, the back-end must complete all inflight I/O requests for the
> +  specified vring before stopping it.
> +
> +  If the ``VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT`` protocol
> +  feature has been negotiated, the back-end may suspend in-flight I/O
> +  requests and record them as described in :ref:`Inflight I/O tracking
> +  <inflight_io_tracking>` instead of completing them before stopping the vring.
> +  How to suspend an in-flight request depends on the implementation of the back-end
> +  but it typically can be done by aborting or cancelling the underlying I/O
> +  request. The ``VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT``
> +  protocol feature must only be neogotiated if
> +  ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` is also negotiated.
> +
>  ``VHOST_USER_SET_VRING_KICK``
>    :id: 12
>    :equivalent ioctl: ``VHOST_SET_VRING_KICK``
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index a820214188..793d1b36d8 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -2320,6 +2320,11 @@ static int vhost_user_backend_connect(struct vhost_dev *dev, Error **errp)
>              }
>          }
>  
> +        if (!u->user->supports_inflight_migration ||
> +            !virtio_has_feature(VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD)) {

This looks like it will not compile since virtio_has_features() takes
two arguments. I think it should be:

  !virtio_has_feature(dev->protocol_features,
                      VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD)

> +            protocol_features &= ~(1ULL <<
> +                               VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT);
> +        }
> +
>          /* final set of protocol features */
>          u->protocol_features = protocol_features;
>          err = vhost_user_set_protocol_features(dev, u->protocol_features);

I seem to have different version of hw/virtio/vhost-user.c. Is this
patch against qemu.git/master?

> diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
> index fb89268de2..f30c1792e4 100644
> --- a/include/hw/virtio/vhost-user.h
> +++ b/include/hw/virtio/vhost-user.h
> @@ -33,6 +33,7 @@ enum VhostUserProtocolFeature {
>      /* Feature 17 reserved for VHOST_USER_PROTOCOL_F_XEN_MMAP. */
>      VHOST_USER_PROTOCOL_F_SHARED_OBJECT = 18,
>      VHOST_USER_PROTOCOL_F_DEVICE_STATE = 19,
> +    VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT = 20,
>      VHOST_USER_PROTOCOL_F_MAX
>  };
>  
> @@ -69,6 +70,7 @@ typedef struct VhostUserState {
>      GPtrArray *notifiers;
>      int memory_slots;
>      bool supports_config;
> +    bool supports_inflight_migration;
>  } VhostUserState;
>  
>  /**
> -- 
> 2.34.1
> 
Re: [PATCH v6 2/5] vhost-user: introduce protocol feature for skip drain on GET_VRING_BASE
Posted by Alexandr Moshkov 3 weeks, 6 days ago
On 1/13/26 23:00, Stefan Hajnoczi wrote:
> On Tue, Jan 13, 2026 at 02:58:13PM +0500, Alexandr Moshkov wrote:
>> Add vhost-user protocol feature
>> VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT
>>
>> Now on GET_VRING_BASE this feature can control whether to wait for
>> in-flight requests to complete or not.
>> Also we have to validate that this feature will be enabled only when
>> qemu and back-end supports in-flight buffer and in-flight migration
>>
>> It will be helpfull in future for in-flight requests migration in
>> vhost-user devices.
>>
>> Update docs, add ref to label for inflight-io-tracking
>>
>> Signed-off-by: Alexandr Moshkov<dtalexundeer@yandex-team.ru>
>> ---
>>   docs/interop/vhost-user.rst    | 59 +++++++++++++++++++++-------------
>>   hw/virtio/vhost-user.c         |  5 +++
>>   include/hw/virtio/vhost-user.h |  2 ++
>>   3 files changed, 44 insertions(+), 22 deletions(-)
>>
>> diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
>> index 02908b48fa..dcf79de0c0 100644
>> --- a/docs/interop/vhost-user.rst
>> +++ b/docs/interop/vhost-user.rst
>> @@ -736,6 +736,8 @@ negotiated, back-end can send file descriptors (at most 8 descriptors in
>>   each message) to front-end via ancillary data using this fd communication
>>   channel.
>>   
>> +.. _inflight_io_tracking:
>> +
>>   Inflight I/O tracking
>>   ---------------------
>>   
>> @@ -1033,26 +1035,27 @@ Protocol features
>>   
>>   .. code:: c
>>   
>> -  #define VHOST_USER_PROTOCOL_F_MQ                    0
>> -  #define VHOST_USER_PROTOCOL_F_LOG_SHMFD             1
>> -  #define VHOST_USER_PROTOCOL_F_RARP                  2
>> -  #define VHOST_USER_PROTOCOL_F_REPLY_ACK             3
>> -  #define VHOST_USER_PROTOCOL_F_MTU                   4
>> -  #define VHOST_USER_PROTOCOL_F_BACKEND_REQ           5
>> -  #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN          6
>> -  #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION        7
>> -  #define VHOST_USER_PROTOCOL_F_PAGEFAULT             8
>> -  #define VHOST_USER_PROTOCOL_F_CONFIG                9
>> -  #define VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD      10
>> -  #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER        11
>> -  #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD       12
>> -  #define VHOST_USER_PROTOCOL_F_RESET_DEVICE         13
>> -  #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
>> -  #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS  15
>> -  #define VHOST_USER_PROTOCOL_F_STATUS               16
>> -  #define VHOST_USER_PROTOCOL_F_XEN_MMAP             17
>> -  #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT        18
>> -  #define VHOST_USER_PROTOCOL_F_DEVICE_STATE         19
>> +  #define VHOST_USER_PROTOCOL_F_MQ                       0
>> +  #define VHOST_USER_PROTOCOL_F_LOG_SHMFD                1
>> +  #define VHOST_USER_PROTOCOL_F_RARP                     2
>> +  #define VHOST_USER_PROTOCOL_F_REPLY_ACK                3
>> +  #define VHOST_USER_PROTOCOL_F_MTU                      4
>> +  #define VHOST_USER_PROTOCOL_F_BACKEND_REQ              5
>> +  #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN             6
>> +  #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION           7
>> +  #define VHOST_USER_PROTOCOL_F_PAGEFAULT                8
>> +  #define VHOST_USER_PROTOCOL_F_CONFIG                   9
>> +  #define VHOST_USER_PROTOCOL_F_BACKEND_SEND_FD         10
>> +  #define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER           11
>> +  #define VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD          12
>> +  #define VHOST_USER_PROTOCOL_F_RESET_DEVICE            13
>> +  #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS    14
>> +  #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS     15
>> +  #define VHOST_USER_PROTOCOL_F_STATUS                  16
>> +  #define VHOST_USER_PROTOCOL_F_XEN_MMAP                17
>> +  #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT           18
>> +  #define VHOST_USER_PROTOCOL_F_DEVICE_STATE            19
>> +  #define VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT 20
>>   
>>   Front-end message types
>>   -----------------------
>> @@ -1243,12 +1246,24 @@ Front-end message types
>>   
>>     When and as long as all of a device's vrings are stopped, it is
>>     *suspended*, see :ref:`Suspended device state
>> -  <suspended_device_state>`. The back-end must complete all inflight I/O
>> -  requests for the specified vring before stopping it.
>> +  <suspended_device_state>`.
>>   
>>     The request payload's *num* field is currently reserved and must be
>>     set to 0.
>>   
>> +  By default, the back-end must complete all inflight I/O requests for the
>> +  specified vring before stopping it.
>> +
>> +  If the ``VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT`` protocol
>> +  feature has been negotiated, the back-end may suspend in-flight I/O
>> +  requests and record them as described in :ref:`Inflight I/O tracking
>> +  <inflight_io_tracking>` instead of completing them before stopping the vring.
>> +  How to suspend an in-flight request depends on the implementation of the back-end
>> +  but it typically can be done by aborting or cancelling the underlying I/O
>> +  request. The ``VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT``
>> +  protocol feature must only be neogotiated if
>> +  ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD`` is also negotiated.
>> +
>>   ``VHOST_USER_SET_VRING_KICK``
>>     :id: 12
>>     :equivalent ioctl: ``VHOST_SET_VRING_KICK``
>> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
>> index a820214188..793d1b36d8 100644
>> --- a/hw/virtio/vhost-user.c
>> +++ b/hw/virtio/vhost-user.c
>> @@ -2320,6 +2320,11 @@ static int vhost_user_backend_connect(struct vhost_dev *dev, Error **errp)
>>               }
>>           }
>>   
>> +        if (!u->user->supports_inflight_migration ||
>> +            !virtio_has_feature(VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD)) {
> This looks like it will not compile since virtio_has_features() takes
> two arguments. I think it should be:
>
>    !virtio_has_feature(dev->protocol_features,
>                        VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD)
>
Oh, how did I miss that...

I'll fix it! Thanks

>> +            protocol_features &= ~(1ULL <<
>> +                               VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT);
>> +        }
>> +
>>           /* final set of protocol features */
>>           u->protocol_features = protocol_features;
>>           err = vhost_user_set_protocol_features(dev, u->protocol_features);
> I seem to have different version of hw/virtio/vhost-user.c. Is this
> patch against qemu.git/master?

This patch is based on Vladimir's patch, but I now thinks that I can 
separate my patch from his.

At first I thought there would be a lot of intersections with his patch.

In the next version, I will rebase this patch on the master

>> diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
>> index fb89268de2..f30c1792e4 100644
>> --- a/include/hw/virtio/vhost-user.h
>> +++ b/include/hw/virtio/vhost-user.h
>> @@ -33,6 +33,7 @@ enum VhostUserProtocolFeature {
>>       /* Feature 17 reserved for VHOST_USER_PROTOCOL_F_XEN_MMAP. */
>>       VHOST_USER_PROTOCOL_F_SHARED_OBJECT = 18,
>>       VHOST_USER_PROTOCOL_F_DEVICE_STATE = 19,
>> +    VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT = 20,
>>       VHOST_USER_PROTOCOL_F_MAX
>>   };
>>   
>> @@ -69,6 +70,7 @@ typedef struct VhostUserState {
>>       GPtrArray *notifiers;
>>       int memory_slots;
>>       bool supports_config;
>> +    bool supports_inflight_migration;
>>   } VhostUserState;
>>   
>>   /**
>> -- 
>> 2.34.1
>>