[v6] vhost-user-blk: Add support for backend reconnecting

[Qemu-devel] [PATCH v6 0/7] vhost-user-blk: Add support for backend reconnecting

Posted by elohimes@gmail.com 5 years, 2 months ago

From: Xie Yongji <xieyongji@baidu.com>

This patchset is aimed at supporting qemu to reconnect
vhost-user-blk backend after vhost-user-blk backend crash or
restart.

The patch 1 introduces two new messages VHOST_USER_GET_INFLIGHT_FD
and VHOST_USER_SET_INFLIGHT_FD to support transferring shared
buffer between qemu and backend.

The patch 2 deletes some redundant check in contrib/libvhost-user.c.

The patch 3,4 are the corresponding libvhost-user patches of
patch 1. Make libvhost-user support VHOST_USER_GET_INFLIGHT_FD
and VHOST_USER_SET_INFLIGHT_FD.

The patch 5 allows vhost-user-blk to use the two new messages
to get/set inflight buffer from/to backend.

The patch 6 supports vhost-user-blk to reconnect backend when
connection closed.

The patch 7 introduces VHOST_USER_PROTOCOL_F_SLAVE_SHMFD
to vhost-user-blk backend which is used to tell qemu that
we support reconnecting now.

To use it, we could start qemu with:

qemu-system-x86_64 \
        -chardev socket,id=char0,path=/path/vhost.socket,reconnect=1, \
        -device vhost-user-blk-pci,chardev=char0 \

and start vhost-user-blk backend with:

vhost-user-blk -b /path/file -s /path/vhost.socket

Then we can restart vhost-user-blk at any time during VM running.

V5 to V6:
- Document the layout in inflight buffer for packed virtqueue
- Rework the layout in inflight buffer for split virtqueue
- Remove version field in VhostUserInflight
- Add a patch to remove some redundant check in
  contrib/libvhost-user.c
- Document more details in vhost-user.txt

V4 to V5:
- Drop patch that enables "nowait" option on client sockets
- Support resubmitting inflight I/O in order
- Make inflight I/O tracking more robust
- Remove align field and add queue size field in VhostUserInflight
- Document more details in vhost-user.txt

V3 to V4:
- Drop messages VHOST_USER_GET_SHM_SIZE and VHOST_USER_SET_SHM_FD
- Introduce two new messages VHOST_USER_GET_INFLIGHT_FD
  and VHOST_USER_SET_INFLIGHT_FD
- Allocate inflight buffer in backend rather than in qemu
- Document a recommended format for inflight buffer

V2 to V3:
- Using exisiting wait/nowait options to control connection on
  client sockets instead of introducing "disconnected" option.
- Support the case that vhost-user backend restart during initialzation
  of vhost-user-blk device.

V1 to V2:
- Introduce "disconnected" option for chardev instead of reuse "wait"
  option
- Support the case that QEMU starts before vhost-user backend
- Drop message VHOST_USER_SET_VRING_INFLIGHT
- Introduce two new messages VHOST_USER_GET_SHM_SIZE
  and VHOST_USER_SET_SHM_FD

Xie Yongji (7):
  vhost-user: Support transferring inflight buffer between qemu and
    backend
  libvhost-user: Remove unnecessary FD flag check for event file
    descriptors
  libvhost-user: Introduce vu_queue_map_desc()
  libvhost-user: Support tracking inflight I/O in shared memory
  vhost-user-blk: Add support to get/set inflight buffer
  vhost-user-blk: Add support to reconnect backend
  contrib/vhost-user-blk: enable inflight I/O tracking

 Makefile                                |   2 +-
 contrib/libvhost-user/libvhost-user.c   | 400 ++++++++++++++++++++----
 contrib/libvhost-user/libvhost-user.h   |  58 ++++
 contrib/vhost-user-blk/vhost-user-blk.c |   3 +-
 docs/interop/vhost-user.txt             | 264 ++++++++++++++++
 hw/block/vhost-user-blk.c               | 229 +++++++++++---
 hw/virtio/vhost-user.c                  | 107 +++++++
 hw/virtio/vhost.c                       |  96 ++++++
 include/hw/virtio/vhost-backend.h       |  10 +
 include/hw/virtio/vhost-user-blk.h      |   5 +
 include/hw/virtio/vhost.h               |  18 ++
 11 files changed, 1084 insertions(+), 108 deletions(-)

-- 
2.17.1

Re: [Qemu-devel] [PATCH v6 0/7] vhost-user-blk: Add support for backend reconnecting

Posted by Michael S. Tsirkin 5 years, 2 months ago

On Mon, Feb 18, 2019 at 06:27:41PM +0800, elohimes@gmail.com wrote:
> From: Xie Yongji <xieyongji@baidu.com>
> 
> This patchset is aimed at supporting qemu to reconnect
> vhost-user-blk backend after vhost-user-blk backend crash or
> restart.
> 
> The patch 1 introduces two new messages VHOST_USER_GET_INFLIGHT_FD
> and VHOST_USER_SET_INFLIGHT_FD to support transferring shared
> buffer between qemu and backend.
> 
> The patch 2 deletes some redundant check in contrib/libvhost-user.c.
> 
> The patch 3,4 are the corresponding libvhost-user patches of
> patch 1. Make libvhost-user support VHOST_USER_GET_INFLIGHT_FD
> and VHOST_USER_SET_INFLIGHT_FD.
> 
> The patch 5 allows vhost-user-blk to use the two new messages
> to get/set inflight buffer from/to backend.
> 
> The patch 6 supports vhost-user-blk to reconnect backend when
> connection closed.
> 
> The patch 7 introduces VHOST_USER_PROTOCOL_F_SLAVE_SHMFD
> to vhost-user-blk backend which is used to tell qemu that
> we support reconnecting now.
> 
> To use it, we could start qemu with:
> 
> qemu-system-x86_64 \
>         -chardev socket,id=char0,path=/path/vhost.socket,reconnect=1, \
>         -device vhost-user-blk-pci,chardev=char0 \
> 
> and start vhost-user-blk backend with:
> 
> vhost-user-blk -b /path/file -s /path/vhost.socket
> 
> Then we can restart vhost-user-blk at any time during VM running.

Sorry is elohimes@gmail.com also an address that belongs to
Xie Yongji?

If not we need a signed off by from that address'
owner as well.

Thanks!



> V5 to V6:
> - Document the layout in inflight buffer for packed virtqueue
> - Rework the layout in inflight buffer for split virtqueue
> - Remove version field in VhostUserInflight
> - Add a patch to remove some redundant check in
>   contrib/libvhost-user.c
> - Document more details in vhost-user.txt
> 
> V4 to V5:
> - Drop patch that enables "nowait" option on client sockets
> - Support resubmitting inflight I/O in order
> - Make inflight I/O tracking more robust
> - Remove align field and add queue size field in VhostUserInflight
> - Document more details in vhost-user.txt
> 
> V3 to V4:
> - Drop messages VHOST_USER_GET_SHM_SIZE and VHOST_USER_SET_SHM_FD
> - Introduce two new messages VHOST_USER_GET_INFLIGHT_FD
>   and VHOST_USER_SET_INFLIGHT_FD
> - Allocate inflight buffer in backend rather than in qemu
> - Document a recommended format for inflight buffer
> 
> V2 to V3:
> - Using exisiting wait/nowait options to control connection on
>   client sockets instead of introducing "disconnected" option.
> - Support the case that vhost-user backend restart during initialzation
>   of vhost-user-blk device.
> 
> V1 to V2:
> - Introduce "disconnected" option for chardev instead of reuse "wait"
>   option
> - Support the case that QEMU starts before vhost-user backend
> - Drop message VHOST_USER_SET_VRING_INFLIGHT
> - Introduce two new messages VHOST_USER_GET_SHM_SIZE
>   and VHOST_USER_SET_SHM_FD
> 
> Xie Yongji (7):
>   vhost-user: Support transferring inflight buffer between qemu and
>     backend
>   libvhost-user: Remove unnecessary FD flag check for event file
>     descriptors
>   libvhost-user: Introduce vu_queue_map_desc()
>   libvhost-user: Support tracking inflight I/O in shared memory
>   vhost-user-blk: Add support to get/set inflight buffer
>   vhost-user-blk: Add support to reconnect backend
>   contrib/vhost-user-blk: enable inflight I/O tracking
> 
>  Makefile                                |   2 +-
>  contrib/libvhost-user/libvhost-user.c   | 400 ++++++++++++++++++++----
>  contrib/libvhost-user/libvhost-user.h   |  58 ++++
>  contrib/vhost-user-blk/vhost-user-blk.c |   3 +-
>  docs/interop/vhost-user.txt             | 264 ++++++++++++++++
>  hw/block/vhost-user-blk.c               | 229 +++++++++++---
>  hw/virtio/vhost-user.c                  | 107 +++++++
>  hw/virtio/vhost.c                       |  96 ++++++
>  include/hw/virtio/vhost-backend.h       |  10 +
>  include/hw/virtio/vhost-user-blk.h      |   5 +
>  include/hw/virtio/vhost.h               |  18 ++
>  11 files changed, 1084 insertions(+), 108 deletions(-)
> 
> -- 
> 2.17.1

Re: [Qemu-devel] [PATCH v6 0/7] vhost-user-blk: Add support for backend reconnecting

Posted by Yongji Xie 5 years, 2 months ago

On Thu, 21 Feb 2019 at 04:00, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Mon, Feb 18, 2019 at 06:27:41PM +0800, elohimes@gmail.com wrote:
> > From: Xie Yongji <xieyongji@baidu.com>
> >
> > This patchset is aimed at supporting qemu to reconnect
> > vhost-user-blk backend after vhost-user-blk backend crash or
> > restart.
> >
> > The patch 1 introduces two new messages VHOST_USER_GET_INFLIGHT_FD
> > and VHOST_USER_SET_INFLIGHT_FD to support transferring shared
> > buffer between qemu and backend.
> >
> > The patch 2 deletes some redundant check in contrib/libvhost-user.c.
> >
> > The patch 3,4 are the corresponding libvhost-user patches of
> > patch 1. Make libvhost-user support VHOST_USER_GET_INFLIGHT_FD
> > and VHOST_USER_SET_INFLIGHT_FD.
> >
> > The patch 5 allows vhost-user-blk to use the two new messages
> > to get/set inflight buffer from/to backend.
> >
> > The patch 6 supports vhost-user-blk to reconnect backend when
> > connection closed.
> >
> > The patch 7 introduces VHOST_USER_PROTOCOL_F_SLAVE_SHMFD
> > to vhost-user-blk backend which is used to tell qemu that
> > we support reconnecting now.
> >
> > To use it, we could start qemu with:
> >
> > qemu-system-x86_64 \
> >         -chardev socket,id=char0,path=/path/vhost.socket,reconnect=1, \
> >         -device vhost-user-blk-pci,chardev=char0 \
> >
> > and start vhost-user-blk backend with:
> >
> > vhost-user-blk -b /path/file -s /path/vhost.socket
> >
> > Then we can restart vhost-user-blk at any time during VM running.
>
> Sorry is elohimes@gmail.com also an address that belongs to
> Xie Yongji?
>

Yes, that's also my email address.

Thanks,
Yongji