[v1] vhost-user block device backend implementation

[PATCH v1 0/5] vhost-user block device backend implementation

Posted by Coiby Xu 4 years, 3 months ago

This patch series is an implementation of vhost-user-backend server, thanks to Stefan's guidance after reviewing the draft version and the instructions on https://wiki.qemu.org/Google_Summer_of_Code_2019#vhost-user-blk_device_backend,

Vhost-user-backend server is a UserCreatable object and can be started using object_add,

 (qemu) object_add vhost-user-server,id=ID,unix_socket=/tmp/vhost-user-blk_vhost.socket,name=DRIVE_NAME,writable=off
 (qemu) object_del ID

or appending the "-object" option when starting QEMU,

  $ -object vhost-user-server,id=disk,unix_socket=/tmp/vhost-user-blk_vhost.socket,name=disk,writable=off

Then vhost-user client can connect to the server backend. For example, QEMU could act as a client,
  $ -m 256 -object memory-backend-memfd,id=mem,size=256M,share=on -numa node,memdev=mem -chardev socket,id=char1,path=/tmp/vhost-user-blk_vhost.socket -device vhost-user-blk-pci,id=blk0,chardev=char1

And guest OS could access this vhost-user block device after mouting it.

patches are against commit 035eed4c0d257c905a556fa0f4865a0c077b4e7f.

Coiby Xu (5):
  vhost-user block device backend
  extend libvhost to support IOThread
  a standone-alone tool to directly share disk image file via vhost-user
    protocol
  new qTest case for the vhost-user-blk device backend
  building configuration files changes

 Makefile                              |    1 +
 Makefile.objs                         |    2 +-
 Makefile.target                       |    1 +
 blockdev-vu.c                         | 1008 +++++++++++++++++++++++++
 configure                             |    2 +-
 contrib/libvhost-user/libvhost-user.c |   64 +-
 contrib/libvhost-user/libvhost-user.h |   36 +-
 include/block/vhost-user.h            |   46 ++
 qemu-vu.c                             |  264 +++++++
 tests/Makefile.include                |    5 +-
 tests/libqos/vhost-user-blk.c         |  125 +++
 tests/libqos/vhost-user-blk.h         |   44 ++
 tests/vhost-user-blk-test.c           |  691 +++++++++++++++++
 vl.c                                  |    4 +
 14 files changed, 2277 insertions(+), 16 deletions(-)
 create mode 100644 blockdev-vu.c
 create mode 100644 include/block/vhost-user.h
 create mode 100644 qemu-vu.c
 create mode 100644 tests/libqos/vhost-user-blk.c
 create mode 100644 tests/libqos/vhost-user-blk.h
 create mode 100644 tests/vhost-user-blk-test.c

--
2.24.1

Re: [PATCH v1 0/5] vhost-user block device backend implementation

Posted by no-reply@patchew.org 4 years, 3 months ago

Patchew URL: https://patchew.org/QEMU/20200113045704.12318-1-coiby.xu@gmail.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      chardev/char-ringbuf.o
  CC      chardev/char-serial.o
  CC      chardev/char-socket.o
/tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.c:26:10: fatal error: sys/socket.h: No such file or directory
 #include <sys/socket.h>
          ^~~~~~~~~~~~~~
compilation terminated.
make: *** [/tmp/qemu-test/src/rules.mak:69: contrib/libvhost-user/libvhost-user.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from /tmp/qemu-test/src/include/block/vhost-user.h:3,
                 from /tmp/qemu-test/src/blockdev-vu.c:2:
/tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.h:20:10: fatal error: sys/poll.h: No such file or directory
 #include <sys/poll.h>
          ^~~~~~~~~~~~
compilation terminated.
make: *** [/tmp/qemu-test/src/rules.mak:69: blockdev-vu.o] Error 1
In file included from /tmp/qemu-test/src/include/block/vhost-user.h:3,
                 from /tmp/qemu-test/src/qemu-vu.c:22:
/tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.h:20:10: fatal error: sys/poll.h: No such file or directory
 #include <sys/poll.h>
          ^~~~~~~~~~~~
compilation terminated.
make: *** [/tmp/qemu-test/src/rules.mak:69: qemu-vu.o] Error 1
In file included from /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user-glib.h:19,
                 from /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user-glib.c:17:
/tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.h:20:10: fatal error: sys/poll.h: No such file or directory
 #include <sys/poll.h>
          ^~~~~~~~~~~~
compilation terminated.
make: *** [/tmp/qemu-test/src/rules.mak:69: contrib/libvhost-user/libvhost-user-glib.o] Error 1
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 662, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=5a27a9b1f81649c88588b26500a2460e', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-z0m6vw52/src/docker-src.2020-01-13-00.19.17.20483:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=5a27a9b1f81649c88588b26500a2460e
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-z0m6vw52/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    2m1.128s
user    0m6.706s


The full log is available at
http://patchew.org/logs/20200113045704.12318-1-coiby.xu@gmail.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PATCH v1 0/5] vhost-user block device backend implementation

Posted by Coiby Xu 4 years, 3 months ago

It fails MinGW because libvhost-user depends on eventfd which is a
Linux-specific feature. Now fixed in v2.

On Mon, Jan 13, 2020 at 1:21 PM <no-reply@patchew.org> wrote:

> Patchew URL:
> https://patchew.org/QEMU/20200113045704.12318-1-coiby.xu@gmail.com/
>
>
>
> Hi,
>
> This series failed the docker-mingw@fedora build test. Please find the
> testing commands and
> their output below. If you have Docker installed, you can probably
> reproduce it
> locally.
>
> === TEST SCRIPT BEGIN ===
> #! /bin/bash
> export ARCH=x86_64
> make docker-image-fedora V=1 NETWORK=1
> time make docker-test-mingw@fedora J=14 NETWORK=1
> === TEST SCRIPT END ===
>
>   CC      chardev/char-ringbuf.o
>   CC      chardev/char-serial.o
>   CC      chardev/char-socket.o
> /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.c:26:10: fatal
> error: sys/socket.h: No such file or directory
>  #include <sys/socket.h>
>           ^~~~~~~~~~~~~~
> compilation terminated.
> make: *** [/tmp/qemu-test/src/rules.mak:69:
> contrib/libvhost-user/libvhost-user.o] Error 1
> make: *** Waiting for unfinished jobs....
> In file included from /tmp/qemu-test/src/include/block/vhost-user.h:3,
>                  from /tmp/qemu-test/src/blockdev-vu.c:2:
> /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.h:20:10: fatal
> error: sys/poll.h: No such file or directory
>  #include <sys/poll.h>
>           ^~~~~~~~~~~~
> compilation terminated.
> make: *** [/tmp/qemu-test/src/rules.mak:69: blockdev-vu.o] Error 1
> In file included from /tmp/qemu-test/src/include/block/vhost-user.h:3,
>                  from /tmp/qemu-test/src/qemu-vu.c:22:
> /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.h:20:10: fatal
> error: sys/poll.h: No such file or directory
>  #include <sys/poll.h>
>           ^~~~~~~~~~~~
> compilation terminated.
> make: *** [/tmp/qemu-test/src/rules.mak:69: qemu-vu.o] Error 1
> In file included from
> /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user-glib.h:19,
>                  from
> /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user-glib.c:17:
> /tmp/qemu-test/src/contrib/libvhost-user/libvhost-user.h:20:10: fatal
> error: sys/poll.h: No such file or directory
>  #include <sys/poll.h>
>           ^~~~~~~~~~~~
> compilation terminated.
> make: *** [/tmp/qemu-test/src/rules.mak:69:
> contrib/libvhost-user/libvhost-user-glib.o] Error 1
> Traceback (most recent call last):
>   File "./tests/docker/docker.py", line 662, in <module>
>     sys.exit(main())
> ---
>     raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run',
> '--label', 'com.qemu.instance.uuid=5a27a9b1f81649c88588b26500a2460e', '-u',
> '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e',
> 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14',
> '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache',
> '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v',
> '/var/tmp/patchew-tester-tmp-z0m6vw52/src/docker-src.2020-01-13-00.19.17.20483:/var/tmp/qemu:z,ro',
> 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit
> status 2.
>
> filter=--filter=label=com.qemu.instance.uuid=5a27a9b1f81649c88588b26500a2460e
> make[1]: *** [docker-run] Error 1
> make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-z0m6vw52/src'
> make: *** [docker-run-test-mingw@fedora] Error 2
>
> real    2m1.128s
> user    0m6.706s
>
>
> The full log is available at
>
> http://patchew.org/logs/20200113045704.12318-1-coiby.xu@gmail.com/testing.docker-mingw@fedora/?type=message
> .
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-devel@redhat.com



-- 
*Best regards,*
*Coiby*