[PATCH v2 0/9] preallocate filter

Vladimir Sementsov-Ogievskiy posted 9 patches 3 years, 8 months ago
Test docker-quick@centos7 failed
Test docker-mingw@fedora failed
Test checkpatch passed
Test FreeBSD passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200814130348.20625-1-vsementsov@virtuozzo.com
Maintainers: Eric Blake <eblake@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Max Reitz <mreitz@redhat.com>, Fam Zheng <fam@euphon.net>, Stefan Hajnoczi <stefanha@redhat.com>, Markus Armbruster <armbru@redhat.com>
There is a newer version of this series
docs/system/qemu-block-drivers.rst.inc |  26 +++
qapi/block-core.json                   |  20 +-
include/block/block.h                  |  20 +-
include/block/block_int.h              |   3 +-
block/file-posix.c                     |   2 +-
block/io.c                             | 131 +++++++-----
block/preallocate.c                    | 264 +++++++++++++++++++++++++
block/Makefile.objs                    |   1 +
tests/qemu-iotests/298                 |  46 +++++
tests/qemu-iotests/298.out             |   5 +
tests/qemu-iotests/group               |   1 +
tests/qemu-iotests/iotests.py          |   6 +
12 files changed, 457 insertions(+), 68 deletions(-)
create mode 100644 block/preallocate.c
create mode 100644 tests/qemu-iotests/298
create mode 100644 tests/qemu-iotests/298.out
[PATCH v2 0/9] preallocate filter
Posted by Vladimir Sementsov-Ogievskiy 3 years, 8 months ago
Hi all!

Here is a filter, which does preallocation on write.

In Virtuozzo we have to deal with some custom distributed storage
solution, where allocation is relatively expensive operation. We have to
workaround it in Qemu, so here is a new filter.

For the details refer to original cover-letter
"[PATCH 0/5] preallocate filter"
https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06443.html

v2:
1-6 are new and substitutes bdrv_co_range_try_lock mechanism used in v1
07: add note to docs/system/qemu-block-drivers.rst.inc
    add open options
    rebase on new BDRV_REQ_NO_WAIT flag
    drop bs->file check in _co_flush()
08: new
09: use new iotests.verify_o_direct()

Vladimir Sementsov-Ogievskiy (9):
  block: simplify comment to BDRV_REQ_SERIALISING
  block/io.c: drop assertion on double waiting for request serialisation
  block/io: split out bdrv_find_conflicting_request
  block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg
  block: bdrv_mark_request_serialising: split non-waiting function
  block: introduce BDRV_REQ_NO_WAIT flag
  block: introduce preallocate filter
  iotests.py: add verify_o_direct helper
  iotests: add 298 to test new preallocate filter driver

 docs/system/qemu-block-drivers.rst.inc |  26 +++
 qapi/block-core.json                   |  20 +-
 include/block/block.h                  |  20 +-
 include/block/block_int.h              |   3 +-
 block/file-posix.c                     |   2 +-
 block/io.c                             | 131 +++++++-----
 block/preallocate.c                    | 264 +++++++++++++++++++++++++
 block/Makefile.objs                    |   1 +
 tests/qemu-iotests/298                 |  46 +++++
 tests/qemu-iotests/298.out             |   5 +
 tests/qemu-iotests/group               |   1 +
 tests/qemu-iotests/iotests.py          |   6 +
 12 files changed, 457 insertions(+), 68 deletions(-)
 create mode 100644 block/preallocate.c
 create mode 100644 tests/qemu-iotests/298
 create mode 100644 tests/qemu-iotests/298.out

-- 
2.21.3


Re: [PATCH v2 0/9] preallocate filter
Posted by no-reply@patchew.org 3 years, 8 months ago
Patchew URL: https://patchew.org/QEMU/20200814130348.20625-1-vsementsov@virtuozzo.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      nbd/trace.o
  CC      scsi/trace.o

Warning, treated as error:
../src/docs/system/qemu-block-drivers.rst.inc:980:Duplicate explicit target name: "cmdoption-preallocate-arg-prealloc-align".
  CC      audio/trace.o
  CC      chardev/trace.o
---
  CC      hw/ide/trace.o
  CC      hw/input/trace.o
  CC      hw/intc/trace.o
make: *** [Makefile:1111: .docs_system_qemu.1_docs_system_qemu-block-drivers.7_docs_system_qemu-cpu-models.7.sentinel.] Error 2
make: *** Deleting file '.docs_system_qemu.1_docs_system_qemu-block-drivers.7_docs_system_qemu-cpu-models.7.sentinel.'
make: *** Waiting for unfinished jobs....

Warning, treated as error:
../src/docs/system/qemu-block-drivers.rst.inc:980:Duplicate explicit target name: "cmdoption-preallocate-arg-prealloc-align".
make: *** [Makefile:1100: docs/system/index.html] Error 2
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 709, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=2b8332eeb0c84671bf5f795b37073245', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-msztbvd9/src/docker-src.2020-08-14-12.13.08.24140:/var/tmp/qemu:z,ro', 'qemu/fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=2b8332eeb0c84671bf5f795b37073245
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-msztbvd9/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    2m29.643s
user    0m7.966s


The full log is available at
http://patchew.org/logs/20200814130348.20625-1-vsementsov@virtuozzo.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v2 0/9] preallocate filter
Posted by no-reply@patchew.org 3 years, 8 months ago
Patchew URL: https://patchew.org/QEMU/20200814130348.20625-1-vsementsov@virtuozzo.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  TEST    check-unit: tests/test-char
Unexpected error in object_property_try_add() at /tmp/qemu-test/src/qom/object.c:1181:
attempt to add duplicate property 'serial-id' to object (type 'container')
ERROR test-char - too few tests run (expected 38, got 9)
make: *** [check-unit] Error 1
make: *** Waiting for unfinished jobs....
  TEST    iotest-qcow2: 029
  TEST    check-qtest-x86_64: tests/qtest/hd-geo-test
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=933e39134d33428cb91c95a7f1ece9e6', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-u8odh47c/src/docker-src.2020-08-14-12.16.21.29250:/var/tmp/qemu:z,ro', 'qemu/centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=933e39134d33428cb91c95a7f1ece9e6
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-u8odh47c/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    13m58.743s
user    0m8.567s


The full log is available at
http://patchew.org/logs/20200814130348.20625-1-vsementsov@virtuozzo.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
DROP Re: [PATCH v2 0/9] preallocate filter
Posted by Vladimir Sementsov-Ogievskiy 3 years, 8 months ago
v3 will come soon, don't look at this.

14.08.2020 16:03, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> Here is a filter, which does preallocation on write.
> 
> In Virtuozzo we have to deal with some custom distributed storage
> solution, where allocation is relatively expensive operation. We have to
> workaround it in Qemu, so here is a new filter.
> 
> For the details refer to original cover-letter
> "[PATCH 0/5] preallocate filter"
> https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06443.html
> 
> v2:
> 1-6 are new and substitutes bdrv_co_range_try_lock mechanism used in v1
> 07: add note to docs/system/qemu-block-drivers.rst.inc
>      add open options
>      rebase on new BDRV_REQ_NO_WAIT flag
>      drop bs->file check in _co_flush()
> 08: new
> 09: use new iotests.verify_o_direct()
> 
> Vladimir Sementsov-Ogievskiy (9):
>    block: simplify comment to BDRV_REQ_SERIALISING
>    block/io.c: drop assertion on double waiting for request serialisation
>    block/io: split out bdrv_find_conflicting_request
>    block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg
>    block: bdrv_mark_request_serialising: split non-waiting function
>    block: introduce BDRV_REQ_NO_WAIT flag
>    block: introduce preallocate filter
>    iotests.py: add verify_o_direct helper
>    iotests: add 298 to test new preallocate filter driver
> 
>   docs/system/qemu-block-drivers.rst.inc |  26 +++
>   qapi/block-core.json                   |  20 +-
>   include/block/block.h                  |  20 +-
>   include/block/block_int.h              |   3 +-
>   block/file-posix.c                     |   2 +-
>   block/io.c                             | 131 +++++++-----
>   block/preallocate.c                    | 264 +++++++++++++++++++++++++
>   block/Makefile.objs                    |   1 +
>   tests/qemu-iotests/298                 |  46 +++++
>   tests/qemu-iotests/298.out             |   5 +
>   tests/qemu-iotests/group               |   1 +
>   tests/qemu-iotests/iotests.py          |   6 +
>   12 files changed, 457 insertions(+), 68 deletions(-)
>   create mode 100644 block/preallocate.c
>   create mode 100644 tests/qemu-iotests/298
>   create mode 100644 tests/qemu-iotests/298.out
> 


-- 
Best regards,
Vladimir