[PULL 00/31] Block layer patches

Kevin Wolf posted 31 patches 3 years, 1 month ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210305165454.356840-1-kwolf@redhat.com
Maintainers: John Snow <jsnow@redhat.com>, Raphael Norwitz <raphael.norwitz@nutanix.com>, Laurent Vivier <lvivier@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Coiby Xu <Coiby.Xu@gmail.com>, Thomas Huth <thuth@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Max Reitz <mreitz@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Eric Blake <eblake@redhat.com>, "Denis V. Lunev" <den@openvz.org>, Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>, Stefan Hajnoczi <stefanha@redhat.com>
There is a newer version of this series
docs/interop/parallels.txt                         |  28 +-
docs/tools/qemu-storage-daemon.rst                 |  68 +-
block/parallels.h                                  |   7 +-
include/block/dirty-bitmap.h                       |   2 +
tests/qtest/libqos/libqtest.h                      |  37 +
tests/qtest/libqos/vhost-user-blk.h                |  48 +
block/backup-top.c                                 |  10 +
block/backup.c                                     |   1 +
block/dirty-bitmap.c                               |  13 +
block/export/vhost-user-blk-server.c               | 150 +++-
block/parallels-ext.c                              | 300 +++++++
block/parallels.c                                  |  26 +-
block/qcow2-bitmap.c                               |  16 +-
blockjob.c                                         |  10 +-
hw/block/vhost-user-blk.c                          |   7 +-
storage-daemon/qemu-storage-daemon.c               |  56 +-
tests/qtest/libqos/vhost-user-blk.c                | 130 +++
tests/qtest/libqtest.c                             |  82 +-
tests/qtest/vhost-user-blk-test.c                  | 983 +++++++++++++++++++++
tests/qemu-iotests/iotests.py                      |  10 +
MAINTAINERS                                        |   5 +
block/meson.build                                  |   3 +-
tests/qemu-iotests/087                             |   8 +-
tests/qemu-iotests/184                             |  18 +-
tests/qemu-iotests/218                             |   2 +-
tests/qemu-iotests/235                             |   2 +-
tests/qemu-iotests/245                             |   4 +-
tests/qemu-iotests/258                             |   6 +-
tests/qemu-iotests/258.out                         |   4 +-
tests/qemu-iotests/283                             |  53 ++
tests/qemu-iotests/283.out                         |  15 +
tests/qemu-iotests/295                             |   2 +-
tests/qemu-iotests/296                             |   2 +-
tests/qemu-iotests/300                             |  10 +-
.../sample_images/parallels-with-bitmap.bz2        | Bin 0 -> 203 bytes
.../sample_images/parallels-with-bitmap.sh         |  51 ++
tests/qemu-iotests/tests/parallels-read-bitmap     |  55 ++
tests/qemu-iotests/tests/parallels-read-bitmap.out |   6 +
tests/qtest/libqos/meson.build                     |   1 +
tests/qtest/meson.build                            |   4 +
40 files changed, 2098 insertions(+), 137 deletions(-)
create mode 100644 tests/qtest/libqos/vhost-user-blk.h
create mode 100644 block/parallels-ext.c
create mode 100644 tests/qtest/libqos/vhost-user-blk.c
create mode 100644 tests/qtest/vhost-user-blk-test.c
create mode 100644 tests/qemu-iotests/sample_images/parallels-with-bitmap.bz2
create mode 100755 tests/qemu-iotests/sample_images/parallels-with-bitmap.sh
create mode 100755 tests/qemu-iotests/tests/parallels-read-bitmap
create mode 100644 tests/qemu-iotests/tests/parallels-read-bitmap.out
[PULL 00/31] Block layer patches
Posted by Kevin Wolf 3 years, 1 month ago
The following changes since commit 9a7beaad3dbba982f7a461d676b55a5c3851d312:

  Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20210304' into staging (2021-03-05 10:47:46 +0000)

are available in the Git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 67bedc3aed5c455b629c2cb5f523b536c46adff9:

  docs: qsd: Explain --export nbd,name=... default (2021-03-05 17:09:46 +0100)

----------------------------------------------------------------
Block layer patches:

- qemu-storage-daemon: add --pidfile option
- qemu-storage-daemon: CLI error messages include the option name now
- vhost-user-blk export: Misc fixes, added test cases
- docs: Improvements for qemu-storage-daemon documentation
- parallels: load bitmap extension
- backup-top: Don't crash on post-finalize accesses
- iotests improvements

----------------------------------------------------------------
Alberto Garcia (1):
      iotests: Drop deprecated 'props' from object-add

Coiby Xu (1):
      test: new qTest case to test the vhost-user-blk-server

Eric Blake (1):
      iotests: Fix up python style in 300

Kevin Wolf (1):
      docs: qsd: Explain --export nbd,name=... default

Max Reitz (3):
      backup: Remove nodes from job in .clean()
      backup-top: Refuse I/O in inactive state
      iotests/283: Check that finalize drops backup-top

Paolo Bonzini (2):
      storage-daemon: report unexpected arguments on the fly
      storage-daemon: include current command line option in the errors

Stefan Hajnoczi (14):
      qemu-storage-daemon: add --pidfile option
      docs: show how to spawn qemu-storage-daemon with fd passing
      docs: replace insecure /tmp examples in qsd docs
      vhost-user-blk: fix blkcfg->num_queues endianness
      libqtest: add qtest_socket_server()
      libqtest: add qtest_kill_qemu()
      libqtest: add qtest_remove_abrt_handler()
      tests/qtest: add multi-queue test case to vhost-user-blk-test
      block/export: fix blk_size double byteswap
      block/export: use VIRTIO_BLK_SECTOR_BITS
      block/export: fix vhost-user-blk export sector number calculation
      block/export: port virtio-blk discard/write zeroes input validation
      vhost-user-blk-test: test discard/write zeroes invalid inputs
      block/export: port virtio-blk read/write range check

Stefano Garzarella (1):
      blockjob: report a better error message

Vladimir Sementsov-Ogievskiy (7):
      qcow2-bitmap: make bytes_covered_by_bitmap_cluster() public
      parallels.txt: fix bitmap L1 table description
      block/parallels: BDRVParallelsState: add cluster_size field
      parallels: support bitmap extension for read-only mode
      iotests.py: add unarchive_sample_image() helper
      iotests: add parallels-read-bitmap test
      MAINTAINERS: update parallels block driver

 docs/interop/parallels.txt                         |  28 +-
 docs/tools/qemu-storage-daemon.rst                 |  68 +-
 block/parallels.h                                  |   7 +-
 include/block/dirty-bitmap.h                       |   2 +
 tests/qtest/libqos/libqtest.h                      |  37 +
 tests/qtest/libqos/vhost-user-blk.h                |  48 +
 block/backup-top.c                                 |  10 +
 block/backup.c                                     |   1 +
 block/dirty-bitmap.c                               |  13 +
 block/export/vhost-user-blk-server.c               | 150 +++-
 block/parallels-ext.c                              | 300 +++++++
 block/parallels.c                                  |  26 +-
 block/qcow2-bitmap.c                               |  16 +-
 blockjob.c                                         |  10 +-
 hw/block/vhost-user-blk.c                          |   7 +-
 storage-daemon/qemu-storage-daemon.c               |  56 +-
 tests/qtest/libqos/vhost-user-blk.c                | 130 +++
 tests/qtest/libqtest.c                             |  82 +-
 tests/qtest/vhost-user-blk-test.c                  | 983 +++++++++++++++++++++
 tests/qemu-iotests/iotests.py                      |  10 +
 MAINTAINERS                                        |   5 +
 block/meson.build                                  |   3 +-
 tests/qemu-iotests/087                             |   8 +-
 tests/qemu-iotests/184                             |  18 +-
 tests/qemu-iotests/218                             |   2 +-
 tests/qemu-iotests/235                             |   2 +-
 tests/qemu-iotests/245                             |   4 +-
 tests/qemu-iotests/258                             |   6 +-
 tests/qemu-iotests/258.out                         |   4 +-
 tests/qemu-iotests/283                             |  53 ++
 tests/qemu-iotests/283.out                         |  15 +
 tests/qemu-iotests/295                             |   2 +-
 tests/qemu-iotests/296                             |   2 +-
 tests/qemu-iotests/300                             |  10 +-
 .../sample_images/parallels-with-bitmap.bz2        | Bin 0 -> 203 bytes
 .../sample_images/parallels-with-bitmap.sh         |  51 ++
 tests/qemu-iotests/tests/parallels-read-bitmap     |  55 ++
 tests/qemu-iotests/tests/parallels-read-bitmap.out |   6 +
 tests/qtest/libqos/meson.build                     |   1 +
 tests/qtest/meson.build                            |   4 +
 40 files changed, 2098 insertions(+), 137 deletions(-)
 create mode 100644 tests/qtest/libqos/vhost-user-blk.h
 create mode 100644 block/parallels-ext.c
 create mode 100644 tests/qtest/libqos/vhost-user-blk.c
 create mode 100644 tests/qtest/vhost-user-blk-test.c
 create mode 100644 tests/qemu-iotests/sample_images/parallels-with-bitmap.bz2
 create mode 100755 tests/qemu-iotests/sample_images/parallels-with-bitmap.sh
 create mode 100755 tests/qemu-iotests/tests/parallels-read-bitmap
 create mode 100644 tests/qemu-iotests/tests/parallels-read-bitmap.out


Re: [PULL 00/31] Block layer patches
Posted by Peter Maydell 3 years, 1 month ago
On Fri, 5 Mar 2021 at 16:55, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit 9a7beaad3dbba982f7a461d676b55a5c3851d312:
>
>   Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20210304' into staging (2021-03-05 10:47:46 +0000)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 67bedc3aed5c455b629c2cb5f523b536c46adff9:
>
>   docs: qsd: Explain --export nbd,name=... default (2021-03-05 17:09:46 +0100)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - qemu-storage-daemon: add --pidfile option
> - qemu-storage-daemon: CLI error messages include the option name now
> - vhost-user-blk export: Misc fixes, added test cases
> - docs: Improvements for qemu-storage-daemon documentation
> - parallels: load bitmap extension
> - backup-top: Don't crash on post-finalize accesses
> - iotests improvements

This failed some of the gitlab CI jobs, like this:

https://gitlab.com/qemu-project/qemu/-/jobs/1077335781

Running test qtest-x86_64/test-hmp
Running test qtest-x86_64/qos-test
qemu-storage-daemon: vu_panic: Not implemented: memfd support is missing
qemu-storage-daemon: vu_panic: Failed to alloc vhost inflight area
qemu-system-x86_64: Failed to write msg. Wrote -1 instead of 20.
qemu-system-x86_64: vhost_set_features failed: Invalid argument (22)
qemu-system-x86_64: Error starting vhost: 22
qemu-system-x86_64: vhost-user-blk: vhost start failed: Invalid argument
**
ERROR:../tests/qtest/libqos/virtio.c:228:qvirtio_wait_used_elem:
assertion failed: (g_get_monotonic_time() - start_time <= timeout_us)
ERROR qtest-x86_64/qos-test - Bail out!
ERROR:../tests/qtest/libqos/virtio.c:228:qvirtio_wait_used_elem:
assertion failed: (g_get_monotonic_time() - start_time <= timeout_us)
make: *** [run-test-159] Error 1

I guess some test or other is assuming the presence of
a host feature that isn't guaranteed to be there ?

thanks
-- PMM

Re: [PULL 00/31] Block layer patches
Posted by Kevin Wolf 3 years, 1 month ago
Am 06.03.2021 um 12:22 hat Peter Maydell geschrieben:
> On Fri, 5 Mar 2021 at 16:55, Kevin Wolf <kwolf@redhat.com> wrote:
> >
> > The following changes since commit 9a7beaad3dbba982f7a461d676b55a5c3851d312:
> >
> >   Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20210304' into staging (2021-03-05 10:47:46 +0000)
> >
> > are available in the Git repository at:
> >
> >   git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 67bedc3aed5c455b629c2cb5f523b536c46adff9:
> >
> >   docs: qsd: Explain --export nbd,name=... default (2021-03-05 17:09:46 +0100)
> >
> > ----------------------------------------------------------------
> > Block layer patches:
> >
> > - qemu-storage-daemon: add --pidfile option
> > - qemu-storage-daemon: CLI error messages include the option name now
> > - vhost-user-blk export: Misc fixes, added test cases
> > - docs: Improvements for qemu-storage-daemon documentation
> > - parallels: load bitmap extension
> > - backup-top: Don't crash on post-finalize accesses
> > - iotests improvements
> 
> This failed some of the gitlab CI jobs, like this:
> 
> https://gitlab.com/qemu-project/qemu/-/jobs/1077335781
> 
> Running test qtest-x86_64/test-hmp
> Running test qtest-x86_64/qos-test
> qemu-storage-daemon: vu_panic: Not implemented: memfd support is missing
> qemu-storage-daemon: vu_panic: Failed to alloc vhost inflight area
> qemu-system-x86_64: Failed to write msg. Wrote -1 instead of 20.
> qemu-system-x86_64: vhost_set_features failed: Invalid argument (22)
> qemu-system-x86_64: Error starting vhost: 22
> qemu-system-x86_64: vhost-user-blk: vhost start failed: Invalid argument
> **
> ERROR:../tests/qtest/libqos/virtio.c:228:qvirtio_wait_used_elem:
> assertion failed: (g_get_monotonic_time() - start_time <= timeout_us)
> ERROR qtest-x86_64/qos-test - Bail out!
> ERROR:../tests/qtest/libqos/virtio.c:228:qvirtio_wait_used_elem:
> assertion failed: (g_get_monotonic_time() - start_time <= timeout_us)
> make: *** [run-test-159] Error 1
> 
> I guess some test or other is assuming the presence of
> a host feature that isn't guaranteed to be there ?

Stefan, can you have a look? This is from the new vhost-user-blk test
cases from your series.

If the fix isn't trivial, I'll resubmit v2 today with just the test case
dropped and then we can add it later.

Kevin