[PULL 00/17] pc,vhost: fixes, new test

Michael S. Tsirkin posted 17 patches 3 years, 5 months ago
Failed in applying to current master (apply log)
meson_options.txt                         |   2 +
configure                                 |  25 +-
contrib/libvhost-user/libvhost-user.h     |   2 +-
tests/qtest/libqos/libqtest.h             |  25 +
tests/qtest/libqos/vhost-user-blk.h       |  48 ++
block/export/vhost-user-blk-server.c      | 129 +++-
contrib/libvhost-user/libvhost-user.c     |  24 +-
contrib/vhost-user-blk/vhost-user-blk.c   |   2 +-
contrib/vhost-user-scsi/vhost-user-scsi.c |   2 +-
hw/i386/acpi-build.c                      |  45 +-
hw/virtio/vhost-user.c                    |   5 +-
tests/qtest/libqos/vhost-user-blk.c       | 129 ++++
tests/qtest/libqtest.c                    |  76 ++-
tests/qtest/vhost-user-blk-test.c         | 965 ++++++++++++++++++++++++++++++
block/export/meson.build                  |   5 +-
docs/interop/vhost-user.rst               |  21 +-
meson.build                               |  15 +
tests/qtest/libqos/meson.build            |   1 +
tests/qtest/meson.build                   |   2 +
19 files changed, 1419 insertions(+), 104 deletions(-)
create mode 100644 tests/qtest/libqos/vhost-user-blk.h
create mode 100644 tests/qtest/libqos/vhost-user-blk.c
create mode 100644 tests/qtest/vhost-user-blk-test.c
[PULL 00/17] pc,vhost: fixes, new test
Posted by Michael S. Tsirkin 3 years, 5 months ago
The following changes since commit c6f28ed5075df79fef39c500362a3f4089256c9c:

  Update version for v5.2.0-rc1 release (2020-11-10 22:29:57 +0000)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to fe8d9946228d4df6c020f2cb38b6ac08981727cf:

  vhost-user-blk/scsi: Fix broken error handling for socket call (2020-11-15 17:05:47 -0500)

----------------------------------------------------------------
pc,vhost: fixes, new test

Lots of fixes all over the place.
A new test case which seems like a good idea even at
this late stage: can't break things and will make
sure we don't introduce regressions.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

----------------------------------------------------------------
AlexChen (2):
      contrib/libvhost-user: Fix bad printf format specifiers
      vhost-user-blk/scsi: Fix broken error handling for socket call

Coiby Xu (1):
      test: new qTest case to test the vhost-user-blk-server

Philippe Mathieu-Daudé (1):
      hw/i386/acpi-build: Fix maybe-uninitialized error when ACPI hotplug off

Stefan Hajnoczi (13):
      vhost-user: fix VHOST_USER_ADD/REM_MEM_REG truncation
      meson: move vhost_user_blk_server to meson.build
      vhost-user-blk-server: depend on CONFIG_VHOST_USER
      configure: mark vhost-user Linux-only
      tests/qtest: add multi-queue test case to vhost-user-blk-test
      libqtest: add qtest_socket_server()
      vhost-user-blk-test: rename destroy_drive() to destroy_file()
      vhost-user-blk-test: close fork child file descriptors
      vhost-user-blk-test: drop unused return value
      vhost-user-blk-test: fix races by using fd passing
      block/export: port virtio-blk discard/write zeroes input validation
      vhost-user-blk-test: test discard/write zeroes invalid inputs
      block/export: port virtio-blk read/write range check

 meson_options.txt                         |   2 +
 configure                                 |  25 +-
 contrib/libvhost-user/libvhost-user.h     |   2 +-
 tests/qtest/libqos/libqtest.h             |  25 +
 tests/qtest/libqos/vhost-user-blk.h       |  48 ++
 block/export/vhost-user-blk-server.c      | 129 +++-
 contrib/libvhost-user/libvhost-user.c     |  24 +-
 contrib/vhost-user-blk/vhost-user-blk.c   |   2 +-
 contrib/vhost-user-scsi/vhost-user-scsi.c |   2 +-
 hw/i386/acpi-build.c                      |  45 +-
 hw/virtio/vhost-user.c                    |   5 +-
 tests/qtest/libqos/vhost-user-blk.c       | 129 ++++
 tests/qtest/libqtest.c                    |  76 ++-
 tests/qtest/vhost-user-blk-test.c         | 965 ++++++++++++++++++++++++++++++
 block/export/meson.build                  |   5 +-
 docs/interop/vhost-user.rst               |  21 +-
 meson.build                               |  15 +
 tests/qtest/libqos/meson.build            |   1 +
 tests/qtest/meson.build                   |   2 +
 19 files changed, 1419 insertions(+), 104 deletions(-)
 create mode 100644 tests/qtest/libqos/vhost-user-blk.h
 create mode 100644 tests/qtest/libqos/vhost-user-blk.c
 create mode 100644 tests/qtest/vhost-user-blk-test.c


Re: [PULL 00/17] pc,vhost: fixes, new test
Posted by Peter Maydell 3 years, 5 months ago
On Sun, 15 Nov 2020 at 22:27, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> The following changes since commit c6f28ed5075df79fef39c500362a3f4089256c9c:
>
>   Update version for v5.2.0-rc1 release (2020-11-10 22:29:57 +0000)
>
> are available in the Git repository at:
>
>   git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream
>
> for you to fetch changes up to fe8d9946228d4df6c020f2cb38b6ac08981727cf:
>
>   vhost-user-blk/scsi: Fix broken error handling for socket call (2020-11-15 17:05:47 -0500)
>
> ----------------------------------------------------------------
> pc,vhost: fixes, new test
>
> Lots of fixes all over the place.
> A new test case which seems like a good idea even at
> this late stage: can't break things and will make
> sure we don't introduce regressions.
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> ----------------------------------------------------------------

Something in this seems to cause hangs in 'make check' on
my x86-64 Linux box: sample 'ps wafux' output:

petmay01 30354  0.0  0.0  17392  9348 ?        S    13:40   0:00
                   \_ make --output-sync -C build/a
ll check V=1 -j8
petmay01  7093  0.0  0.0  13916  3608 ?        S    13:41   0:00
                       \_ bash -o pipefail -c echo
'MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-i386 tests/qtest/qos-test --tap -k' &&
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-i386 tests/qtest/qos-test --tap -k <
/dev/null | ./scripts/tap-driver.pl --test-name="qtest-i386/qos-test"
petmay01  7095  0.0  0.0  37764 11744 ?        S    13:41   0:00
                       |   \_ perl ./scripts/tap-driver.pl
--test-name=qtest-i386/qos-test
petmay01 14023  0.0  0.0  13916  3568 ?        S    13:41   0:00
                       \_ bash -o pipefail -c echo
'MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-ppc64 tests/qtest/qos-test --tap -k'
&& MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-ppc64 tests/qtest/qos-test --tap -k <
/dev/null | ./scripts/tap-driver.pl --test-name="qtest-ppc64/qos-test"
petmay01 14025  0.0  0.0  37828 11760 ?        S    13:41   0:00
                       |   \_ perl ./scripts/tap-driver.pl
--test-name=qtest-ppc64/qos-test
petmay01 22886  0.0  0.0  13916  3716 ?        S    13:42   0:00
                       \_ bash -o pipefail -c echo
'MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-x86_64 tests/qtest/qos-test --tap -k'
&& MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
QTEST_QEMU_IMG=./qemu-img
G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
QTEST_QEMU_BINARY=./qemu-system-x86_64 tests/qtest/qos-test --tap -k <
/dev/null | ./scripts/tap-driver.pl
--test-name="qtest-x86_64/qos-test"
petmay01 22888  0.0  0.0  37764 11836 ?        S    13:42   0:00
                           \_ perl ./scripts/tap-driver.pl
--test-name=qtest-x86_64/qos-test


Something somewhere is also apparently leaving a qemu-storage-daemon
process running on bigendian hosts only (?): I see this on my s390x
test box:

ubuntu   26700  0.0  0.5 330776 21552 ?        Sl   08:40   0:00
./storage-daemon/qemu-storage-daemon --chardev
socket,id=qmp,fd=11,server,nowait --monitor chardev=qmp --blockdev
driver=file,node-name=disk0,filename=qtest.V5gfPm --export
type=vhost-user-blk,id=disk0,addr.type=fd,addr.str=16,node-name=disk0,writable=on,num-queues=8
--blockdev driver=file,node-name=disk1,filename=qtest.JM24xB --export
type=vhost-user-blk,id=disk1,addr.type=fd,addr.str=17,node-name=disk1,writable=on,num-queues=8

and similarly on the ppc64be box (but not on other machines).
This seems to be associated with this test failure:

ERROR:../../tests/qtest/vhost-user-blk-test.c:738:multiqueue:
assertion failed (num_queues == 8): (2048 == 8)
ERROR qtest-i386/qos-test - Bail out!
ERROR:../../tests/qtest/vhost-user-blk-test.c:738:multiqueue:
assertion failed (num_queues == 8): (2048 == 8)
Makefile.mtest:1857: recipe for target 'run-test-230' failed

which looks suspiciously like an endianness bug somewhere.

Ideally if the test case starts external processes it should
make sure they're cleaned up even if the test fails.

thanks
-- PMM

Re: [PULL 00/17] pc,vhost: fixes, new test
Posted by Michael S. Tsirkin 3 years, 5 months ago
On Mon, Nov 16, 2020 at 02:19:11PM +0000, Peter Maydell wrote:
> On Sun, 15 Nov 2020 at 22:27, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > The following changes since commit c6f28ed5075df79fef39c500362a3f4089256c9c:
> >
> >   Update version for v5.2.0-rc1 release (2020-11-10 22:29:57 +0000)
> >
> > are available in the Git repository at:
> >
> >   git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream
> >
> > for you to fetch changes up to fe8d9946228d4df6c020f2cb38b6ac08981727cf:
> >
> >   vhost-user-blk/scsi: Fix broken error handling for socket call (2020-11-15 17:05:47 -0500)
> >
> > ----------------------------------------------------------------
> > pc,vhost: fixes, new test
> >
> > Lots of fixes all over the place.
> > A new test case which seems like a good idea even at
> > this late stage: can't break things and will make
> > sure we don't introduce regressions.
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> >
> > ----------------------------------------------------------------
> 
> Something in this seems to cause hangs in 'make check' on
> my x86-64 Linux box: sample 'ps wafux' output:
> 
> petmay01 30354  0.0  0.0  17392  9348 ?        S    13:40   0:00
>                    \_ make --output-sync -C build/a
> ll check V=1 -j8
> petmay01  7093  0.0  0.0  13916  3608 ?        S    13:41   0:00
>                        \_ bash -o pipefail -c echo
> 'MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img
> G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-i386 tests/qtest/qos-test --tap -k' &&
> MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img
> G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-i386 tests/qtest/qos-test --tap -k <
> /dev/null | ./scripts/tap-driver.pl --test-name="qtest-i386/qos-test"
> petmay01  7095  0.0  0.0  37764 11744 ?        S    13:41   0:00
>                        |   \_ perl ./scripts/tap-driver.pl
> --test-name=qtest-i386/qos-test
> petmay01 14023  0.0  0.0  13916  3568 ?        S    13:41   0:00
>                        \_ bash -o pipefail -c echo
> 'MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img
> G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-ppc64 tests/qtest/qos-test --tap -k'
> && MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img
> G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-ppc64 tests/qtest/qos-test --tap -k <
> /dev/null | ./scripts/tap-driver.pl --test-name="qtest-ppc64/qos-test"
> petmay01 14025  0.0  0.0  37828 11760 ?        S    13:41   0:00
>                        |   \_ perl ./scripts/tap-driver.pl
> --test-name=qtest-ppc64/qos-test
> petmay01 22886  0.0  0.0  13916  3716 ?        S    13:42   0:00
>                        \_ bash -o pipefail -c echo
> 'MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img
> G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-x86_64 tests/qtest/qos-test --tap -k'
> && MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}
> QTEST_QEMU_IMG=./qemu-img
> G_TEST_DBUS_DAEMON=/home/petmay01/linaro/qemu-for-merges/tests/dbus-vmstate-daemon.sh
> QTEST_QEMU_BINARY=./qemu-system-x86_64 tests/qtest/qos-test --tap -k <
> /dev/null | ./scripts/tap-driver.pl
> --test-name="qtest-x86_64/qos-test"
> petmay01 22888  0.0  0.0  37764 11836 ?        S    13:42   0:00
>                            \_ perl ./scripts/tap-driver.pl
> --test-name=qtest-x86_64/qos-test
> 
> 
> Something somewhere is also apparently leaving a qemu-storage-daemon
> process running on bigendian hosts only (?): I see this on my s390x
> test box:
> 
> ubuntu   26700  0.0  0.5 330776 21552 ?        Sl   08:40   0:00
> ./storage-daemon/qemu-storage-daemon --chardev
> socket,id=qmp,fd=11,server,nowait --monitor chardev=qmp --blockdev
> driver=file,node-name=disk0,filename=qtest.V5gfPm --export
> type=vhost-user-blk,id=disk0,addr.type=fd,addr.str=16,node-name=disk0,writable=on,num-queues=8
> --blockdev driver=file,node-name=disk1,filename=qtest.JM24xB --export
> type=vhost-user-blk,id=disk1,addr.type=fd,addr.str=17,node-name=disk1,writable=on,num-queues=8
> 
> and similarly on the ppc64be box (but not on other machines).
> This seems to be associated with this test failure:
> 
> ERROR:../../tests/qtest/vhost-user-blk-test.c:738:multiqueue:
> assertion failed (num_queues == 8): (2048 == 8)
> ERROR qtest-i386/qos-test - Bail out!
> ERROR:../../tests/qtest/vhost-user-blk-test.c:738:multiqueue:
> assertion failed (num_queues == 8): (2048 == 8)
> Makefile.mtest:1857: recipe for target 'run-test-230' failed
> 
> which looks suspiciously like an endianness bug somewhere.
> 
> Ideally if the test case starts external processes it should
> make sure they're cleaned up even if the test fails.
> 
> thanks
> -- PMM


Must be vhost user blk test patch exposing some more latent races ...
I dropped that for now, and sent pull v2.
Same tag.

-- 
MST