[RFC 0/8] blkio: add libblkio BlockDriver

Stefan Hajnoczi posted 8 patches 2 years, 1 month ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20220323111727.1100209-1-stefanha@redhat.com
Test checkpatch failed
Maintainers: Stefan Hajnoczi <stefanha@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Fam Zheng <fam@euphon.net>, John Snow <jsnow@redhat.com>, Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Yanan Wang <wangyanan55@huawei.com>, Markus Armbruster <armbru@redhat.com>, Eric Blake <eblake@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Thomas Huth <thuth@redhat.com>, Laurent Vivier <lvivier@redhat.com>
There is a newer version of this series
MAINTAINERS                                 |   7 +
meson_options.txt                           |   2 +
qapi/block-core.json                        |  18 +-
meson.build                                 |   9 +
include/block/block-common.h                |   9 +
include/block/block-global-state.h          |   5 +-
include/block/block_int-common.h            |   2 +-
include/hw/virtio/virtio-blk.h              |   2 +
include/sysemu/block-backend-global-state.h |   2 +-
include/sysemu/block-ram-registrar.h        |  30 ++
block/blkio.c                               | 560 ++++++++++++++++++++
block/blkverify.c                           |   4 +-
block/block-backend.c                       |   4 +-
block/block-ram-registrar.c                 |  39 ++
block/crypto.c                              |   2 +
block/io.c                                  |  36 +-
block/mirror.c                              |   2 +
block/nvme.c                                |   2 +-
block/raw-format.c                          |   2 +
hw/block/virtio-blk.c                       |  13 +-
hw/core/numa.c                              |  17 +
qemu-img.c                                  |   4 +-
stubs/memory.c                              |  13 +
tests/qtest/modules-test.c                  |   3 +
util/vfio-helpers.c                         |   5 +-
block/meson.build                           |   2 +
scripts/meson-buildoptions.sh               |   3 +
stubs/meson.build                           |   1 +
28 files changed, 772 insertions(+), 26 deletions(-)
create mode 100644 include/sysemu/block-ram-registrar.h
create mode 100644 block/blkio.c
create mode 100644 block/block-ram-registrar.c
create mode 100644 stubs/memory.c
[RFC 0/8] blkio: add libblkio BlockDriver
Posted by Stefan Hajnoczi 2 years, 1 month ago
This patch series adds a QEMU BlockDriver for libblkio
(https://gitlab.com/libblkio/libblkio/), a library for high-performance block
device I/O. Currently libblkio has basic io_uring support with additional
drivers in development.

The first patch adds the core BlockDriver and most of the libblkio API usage.
The remainder of the patch series reworks the existing QEMU bdrv_register_buf()
API so virtio-blk emulation efficiently map guest RAM for libblkio - some
libblkio drivers require that I/O buffer memory is pre-registered (think VFIO,
vhost, etc).

This block driver is incomplete because bdrv_refresh_limits() and several other
APIs are not yet implemented. You can already boot a guest though. Once the
missing gaps have been filled in I will send a non-RFC patch series.

Regarding the design: each libblkio driver is a separately named BlockDriver.
That means there is an "io_uring" BlockDriver and not a generic "libblkio"
BlockDriver. In the future there will be additional BlockDrivers, all defined
in block/blkio.c. This way QAPI and open parameters are type-safe and mandatory
parameters can be checked by QEMU.

Stefan Hajnoczi (8):
  blkio: add io_uring block driver using libblkio
  numa: call ->ram_block_removed() in ram_block_notifer_remove()
  block: pass size to bdrv_unregister_buf()
  block: add BDRV_REQ_REGISTERED_BUF request flag
  block: add BlockRAMRegistrar
  stubs: add memory_region_from_host() and memory_region_get_fd()
  blkio: implement BDRV_REQ_REGISTERED_BUF optimization
  virtio-blk: use BDRV_REQ_REGISTERED_BUF optimization hint

 MAINTAINERS                                 |   7 +
 meson_options.txt                           |   2 +
 qapi/block-core.json                        |  18 +-
 meson.build                                 |   9 +
 include/block/block-common.h                |   9 +
 include/block/block-global-state.h          |   5 +-
 include/block/block_int-common.h            |   2 +-
 include/hw/virtio/virtio-blk.h              |   2 +
 include/sysemu/block-backend-global-state.h |   2 +-
 include/sysemu/block-ram-registrar.h        |  30 ++
 block/blkio.c                               | 560 ++++++++++++++++++++
 block/blkverify.c                           |   4 +-
 block/block-backend.c                       |   4 +-
 block/block-ram-registrar.c                 |  39 ++
 block/crypto.c                              |   2 +
 block/io.c                                  |  36 +-
 block/mirror.c                              |   2 +
 block/nvme.c                                |   2 +-
 block/raw-format.c                          |   2 +
 hw/block/virtio-blk.c                       |  13 +-
 hw/core/numa.c                              |  17 +
 qemu-img.c                                  |   4 +-
 stubs/memory.c                              |  13 +
 tests/qtest/modules-test.c                  |   3 +
 util/vfio-helpers.c                         |   5 +-
 block/meson.build                           |   2 +
 scripts/meson-buildoptions.sh               |   3 +
 stubs/meson.build                           |   1 +
 28 files changed, 772 insertions(+), 26 deletions(-)
 create mode 100644 include/sysemu/block-ram-registrar.h
 create mode 100644 block/blkio.c
 create mode 100644 block/block-ram-registrar.c
 create mode 100644 stubs/memory.c

-- 
2.35.1

Re: [RFC 0/8] blkio: add libblkio BlockDriver
Posted by Stefano Garzarella 2 years, 1 month ago
On Wed, Mar 23, 2022 at 11:17:19AM +0000, Stefan Hajnoczi wrote:
>This patch series adds a QEMU BlockDriver for libblkio
>(https://gitlab.com/libblkio/libblkio/), a library for high-performance block
>device I/O. Currently libblkio has basic io_uring support with additional
>drivers in development.
>
>The first patch adds the core BlockDriver and most of the libblkio API usage.
>The remainder of the patch series reworks the existing QEMU bdrv_register_buf()
>API so virtio-blk emulation efficiently map guest RAM for libblkio - some
>libblkio drivers require that I/O buffer memory is pre-registered (think VFIO,
>vhost, etc).
>
>This block driver is incomplete because bdrv_refresh_limits() and several other
>APIs are not yet implemented. You can already boot a guest though. Once the
>missing gaps have been filled in I will send a non-RFC patch series.
>
>Regarding the design: each libblkio driver is a separately named BlockDriver.
>That means there is an "io_uring" BlockDriver and not a generic "libblkio"
>BlockDriver. In the future there will be additional BlockDrivers, all defined
>in block/blkio.c. This way QAPI and open parameters are type-safe and mandatory
>parameters can be checked by QEMU.

Indeed it was a great design. Adding the "vhost_vdpa" driver was quick 
and easy.

I've been reviewing and testing it. Aside from the problem in patch 7, 
the rest already looks good to me.

Thanks,
Stefano