[v5] Support exporting BDSs via VDUSE

[PATCH v5 0/8] Support exporting BDSs via VDUSE

Xie Yongji posted 8 patches 1 year, 11 months ago

Diff against v1 v2 v3 v4 v6
Download series mbox

Failed in applying to current master (apply log)

Maintainers: Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Xie Yongji <xieyongji@bytedance.com>, Coiby Xu <Coiby.Xu@gmail.com>, "Michael S. Tsirkin" <mst@redhat.com>, Cornelia Huck <cohuck@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Eric Blake <eblake@redhat.com>, Markus Armbruster <armbru@redhat.com>

There is a newer version of this series

MAINTAINERS                                 |    9 +
block/block-backend.c                       |    8 +-
block/export/export.c                       |    6 +
block/export/meson.build                    |    7 +-
block/export/vduse-blk.c                    |  346 +++++
block/export/vduse-blk.h                    |   20 +
block/export/vhost-user-blk-server.c        |  249 +---
block/export/virtio-blk-handler.c           |  237 ++++
block/export/virtio-blk-handler.h           |   33 +
include/sysemu/block-backend-io.h           |    1 +
linux-headers/linux/vduse.h                 |  306 ++++
meson.build                                 |   28 +
meson_options.txt                           |    4 +
qapi/block-export.json                      |   25 +-
scripts/meson-buildoptions.sh               |    7 +
scripts/update-linux-headers.sh             |    2 +-
subprojects/libvduse/include/atomic.h       |    1 +
subprojects/libvduse/include/compiler.h     |    1 +
subprojects/libvduse/libvduse.c             | 1386 +++++++++++++++++++
subprojects/libvduse/libvduse.h             |  247 ++++
subprojects/libvduse/linux-headers/linux    |    1 +
subprojects/libvduse/meson.build            |   10 +
subprojects/libvduse/standard-headers/linux |    1 +
23 files changed, 2695 insertions(+), 240 deletions(-)
create mode 100644 block/export/vduse-blk.c
create mode 100644 block/export/vduse-blk.h
create mode 100644 block/export/virtio-blk-handler.c
create mode 100644 block/export/virtio-blk-handler.h
create mode 100644 linux-headers/linux/vduse.h
create mode 120000 subprojects/libvduse/include/atomic.h
create mode 120000 subprojects/libvduse/include/compiler.h
create mode 100644 subprojects/libvduse/libvduse.c
create mode 100644 subprojects/libvduse/libvduse.h
create mode 120000 subprojects/libvduse/linux-headers/linux
create mode 100644 subprojects/libvduse/meson.build
create mode 120000 subprojects/libvduse/standard-headers/linux

Expand all Fold all

[PATCH v5 0/8] Support exporting BDSs via VDUSE

Posted by Xie Yongji 1 year, 11 months ago

Hi all,

Last few months ago, VDUSE (vDPA Device in Userspace) [1] has
been merged into Linux kernel as a framework that make it
possible to emulate a vDPA device in userspace. This series
aimed at implementing a VDUSE block backend based on the
qemu-storage-daemon infrastructure.

To support that, we firstly introduce a VDUSE library as a
subproject (like what libvhost-user does) to help implementing
VDUSE backends in QEMU. Then a VDUSE block export is implemented
based on this library. At last, we add resize and reconnect support
to the VDUSE block export and VDUSE library.

Since we don't support vdpa-blk in QEMU currently, the VM case is
tested with my previous patchset [2].

[1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
[2] https://www.mail-archive.com/qemu-devel@nongnu.org/msg797569.html

Please review, thanks!

V4 to V5:
- Abstract out the logic for virito-blk I/O process from
  vhost-user-blk-server and reuse it [Kevin]
- Fix missing VIRTIO_BLK_F_FLUSH [Kevin]
- Support discard and write_zeroes [Kevin]
- Rebase to the newest tree

V3 to V4:
- Fix some comments on QAPI [Eric]

V2 to V3:
- Introduce vduse_get_virtio_features() [Stefan]
- Update MAINTAINERS file [Stefan]
- Fix handler of VIRTIO_BLK_T_GET_ID request [Stefan]
- Add barrier for vduse_queue_inflight_get() [Stefan]

V1 to V2:
- Move vduse header to linux-headers [Stefan]
- Add two new API to support creating device from /dev/vduse/$NAME or
  file descriptor [Stefan]
- Check VIRTIO_F_VERSION_1 during intialization [Stefan]
- Replace malloc() + memset to calloc() [Stefan]
- Increase default queue size to 256 for vduse-blk [Stefan]
- Zero-initialize virtio-blk config space [Stefan]
- Add a patch to support reset blk->dev_ops
- Validate vq->log->inflight fields [Stefan]
- Add vduse_set_reconnect_log_file() API to support specifing the
  reconnect log file
- Fix some bugs [Stefan]

Xie Yongji (8):
  block: Support passing NULL ops to blk_set_dev_ops()
  block-backend: Introduce blk_get_guest_block_size()
  block/export: Abstract out the logic of virtio-blk I/O process
  linux-headers: Add vduse.h
  libvduse: Add VDUSE (vDPA Device in Userspace) library
  vduse-blk: Implement vduse-blk export
  vduse-blk: Add vduse-blk resize support
  libvduse: Add support for reconnecting

 MAINTAINERS                                 |    9 +
 block/block-backend.c                       |    8 +-
 block/export/export.c                       |    6 +
 block/export/meson.build                    |    7 +-
 block/export/vduse-blk.c                    |  346 +++++
 block/export/vduse-blk.h                    |   20 +
 block/export/vhost-user-blk-server.c        |  249 +---
 block/export/virtio-blk-handler.c           |  237 ++++
 block/export/virtio-blk-handler.h           |   33 +
 include/sysemu/block-backend-io.h           |    1 +
 linux-headers/linux/vduse.h                 |  306 ++++
 meson.build                                 |   28 +
 meson_options.txt                           |    4 +
 qapi/block-export.json                      |   25 +-
 scripts/meson-buildoptions.sh               |    7 +
 scripts/update-linux-headers.sh             |    2 +-
 subprojects/libvduse/include/atomic.h       |    1 +
 subprojects/libvduse/include/compiler.h     |    1 +
 subprojects/libvduse/libvduse.c             | 1386 +++++++++++++++++++
 subprojects/libvduse/libvduse.h             |  247 ++++
 subprojects/libvduse/linux-headers/linux    |    1 +
 subprojects/libvduse/meson.build            |   10 +
 subprojects/libvduse/standard-headers/linux |    1 +
 23 files changed, 2695 insertions(+), 240 deletions(-)
 create mode 100644 block/export/vduse-blk.c
 create mode 100644 block/export/vduse-blk.h
 create mode 100644 block/export/virtio-blk-handler.c
 create mode 100644 block/export/virtio-blk-handler.h
 create mode 100644 linux-headers/linux/vduse.h
 create mode 120000 subprojects/libvduse/include/atomic.h
 create mode 120000 subprojects/libvduse/include/compiler.h
 create mode 100644 subprojects/libvduse/libvduse.c
 create mode 100644 subprojects/libvduse/libvduse.h
 create mode 120000 subprojects/libvduse/linux-headers/linux
 create mode 100644 subprojects/libvduse/meson.build
 create mode 120000 subprojects/libvduse/standard-headers/linux

-- 
2.20.1

Re: [PATCH v5 0/8] Support exporting BDSs via VDUSE

Posted by Michael S. Tsirkin 1 year, 11 months ago

On Wed, May 04, 2022 at 03:40:43PM +0800, Xie Yongji wrote:
> Hi all,
> 
> Last few months ago, VDUSE (vDPA Device in Userspace) [1] has
> been merged into Linux kernel as a framework that make it
> possible to emulate a vDPA device in userspace. This series
> aimed at implementing a VDUSE block backend based on the
> qemu-storage-daemon infrastructure.
> 
> To support that, we firstly introduce a VDUSE library as a
> subproject (like what libvhost-user does) to help implementing
> VDUSE backends in QEMU. Then a VDUSE block export is implemented
> based on this library. At last, we add resize and reconnect support
> to the VDUSE block export and VDUSE library.
> 
> Since we don't support vdpa-blk in QEMU currently, the VM case is
> tested with my previous patchset [2].
> 
> [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
> [2] https://www.mail-archive.com/qemu-devel@nongnu.org/msg797569.html
> 
> Please review, thanks!

This needs review by storage maintainers.

> V4 to V5:
> - Abstract out the logic for virito-blk I/O process from
>   vhost-user-blk-server and reuse it [Kevin]
> - Fix missing VIRTIO_BLK_F_FLUSH [Kevin]
> - Support discard and write_zeroes [Kevin]
> - Rebase to the newest tree
> 
> V3 to V4:
> - Fix some comments on QAPI [Eric]
> 
> V2 to V3:
> - Introduce vduse_get_virtio_features() [Stefan]
> - Update MAINTAINERS file [Stefan]
> - Fix handler of VIRTIO_BLK_T_GET_ID request [Stefan]
> - Add barrier for vduse_queue_inflight_get() [Stefan]
> 
> V1 to V2:
> - Move vduse header to linux-headers [Stefan]
> - Add two new API to support creating device from /dev/vduse/$NAME or
>   file descriptor [Stefan]
> - Check VIRTIO_F_VERSION_1 during intialization [Stefan]
> - Replace malloc() + memset to calloc() [Stefan]
> - Increase default queue size to 256 for vduse-blk [Stefan]
> - Zero-initialize virtio-blk config space [Stefan]
> - Add a patch to support reset blk->dev_ops
> - Validate vq->log->inflight fields [Stefan]
> - Add vduse_set_reconnect_log_file() API to support specifing the
>   reconnect log file
> - Fix some bugs [Stefan]
> 
> Xie Yongji (8):
>   block: Support passing NULL ops to blk_set_dev_ops()
>   block-backend: Introduce blk_get_guest_block_size()
>   block/export: Abstract out the logic of virtio-blk I/O process
>   linux-headers: Add vduse.h
>   libvduse: Add VDUSE (vDPA Device in Userspace) library
>   vduse-blk: Implement vduse-blk export
>   vduse-blk: Add vduse-blk resize support
>   libvduse: Add support for reconnecting
> 
>  MAINTAINERS                                 |    9 +
>  block/block-backend.c                       |    8 +-
>  block/export/export.c                       |    6 +
>  block/export/meson.build                    |    7 +-
>  block/export/vduse-blk.c                    |  346 +++++
>  block/export/vduse-blk.h                    |   20 +
>  block/export/vhost-user-blk-server.c        |  249 +---
>  block/export/virtio-blk-handler.c           |  237 ++++
>  block/export/virtio-blk-handler.h           |   33 +
>  include/sysemu/block-backend-io.h           |    1 +
>  linux-headers/linux/vduse.h                 |  306 ++++
>  meson.build                                 |   28 +
>  meson_options.txt                           |    4 +
>  qapi/block-export.json                      |   25 +-
>  scripts/meson-buildoptions.sh               |    7 +
>  scripts/update-linux-headers.sh             |    2 +-
>  subprojects/libvduse/include/atomic.h       |    1 +
>  subprojects/libvduse/include/compiler.h     |    1 +
>  subprojects/libvduse/libvduse.c             | 1386 +++++++++++++++++++
>  subprojects/libvduse/libvduse.h             |  247 ++++
>  subprojects/libvduse/linux-headers/linux    |    1 +
>  subprojects/libvduse/meson.build            |   10 +
>  subprojects/libvduse/standard-headers/linux |    1 +
>  23 files changed, 2695 insertions(+), 240 deletions(-)
>  create mode 100644 block/export/vduse-blk.c
>  create mode 100644 block/export/vduse-blk.h
>  create mode 100644 block/export/virtio-blk-handler.c
>  create mode 100644 block/export/virtio-blk-handler.h
>  create mode 100644 linux-headers/linux/vduse.h
>  create mode 120000 subprojects/libvduse/include/atomic.h
>  create mode 120000 subprojects/libvduse/include/compiler.h
>  create mode 100644 subprojects/libvduse/libvduse.c
>  create mode 100644 subprojects/libvduse/libvduse.h
>  create mode 120000 subprojects/libvduse/linux-headers/linux
>  create mode 100644 subprojects/libvduse/meson.build
>  create mode 120000 subprojects/libvduse/standard-headers/linux
> 
> -- 
> 2.20.1

Re: [PATCH v5 0/8] Support exporting BDSs via VDUSE

Posted by Yongji Xie 1 year, 11 months ago

On Fri, May 13, 2022 at 6:03 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, May 04, 2022 at 03:40:43PM +0800, Xie Yongji wrote:
> > Hi all,
> >
> > Last few months ago, VDUSE (vDPA Device in Userspace) [1] has
> > been merged into Linux kernel as a framework that make it
> > possible to emulate a vDPA device in userspace. This series
> > aimed at implementing a VDUSE block backend based on the
> > qemu-storage-daemon infrastructure.
> >
> > To support that, we firstly introduce a VDUSE library as a
> > subproject (like what libvhost-user does) to help implementing
> > VDUSE backends in QEMU. Then a VDUSE block export is implemented
> > based on this library. At last, we add resize and reconnect support
> > to the VDUSE block export and VDUSE library.
> >
> > Since we don't support vdpa-blk in QEMU currently, the VM case is
> > tested with my previous patchset [2].
> >
> > [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html
> > [2] https://www.mail-archive.com/qemu-devel@nongnu.org/msg797569.html
> >
> > Please review, thanks!
>
> This needs review by storage maintainers.
>

Hi Stefan and Kevin, could you take a look?

Thanks,
Yongji