[PATCH 0/5] preallocate filter

Vladimir Sementsov-Ogievskiy posted 5 patches 3 years, 11 months ago
Test FreeBSD passed
Test asan passed
Test docker-quick@centos7 passed
Test checkpatch passed
Test docker-mingw@fedora passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200620143649.225852-1-vsementsov@virtuozzo.com
Maintainers: Stefan Hajnoczi <stefanha@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Markus Armbruster <armbru@redhat.com>, Fam Zheng <fam@euphon.net>, Max Reitz <mreitz@redhat.com>, Eric Blake <eblake@redhat.com>
There is a newer version of this series
qapi/block-core.json          |   3 +-
include/block/block.h         |   9 ++
include/block/block_int.h     |  16 +++
include/qemu/typedefs.h       |   1 +
block/io.c                    | 142 ++++++++++++++++---
block/preallocate.c           | 255 ++++++++++++++++++++++++++++++++++
block/Makefile.objs           |   1 +
tests/qemu-iotests/298        |  45 ++++++
tests/qemu-iotests/298.out    |   5 +
tests/qemu-iotests/group      |   1 +
tests/qemu-iotests/iotests.py |   2 +-
11 files changed, 455 insertions(+), 25 deletions(-)
create mode 100644 block/preallocate.c
create mode 100644 tests/qemu-iotests/298
create mode 100644 tests/qemu-iotests/298.out
[PATCH 0/5] preallocate filter
Posted by Vladimir Sementsov-Ogievskiy 3 years, 11 months ago
Hi all!

Here is a filter, which does preallocation on write.

In Virtuozzo we have to deal with some custom distributed storage solution,
where allocation is relatively expensive operation. We have to workaround it
in Qemu, so here is a new filter.

Performance results with the following test are very significant:

    IMG=/megassd/z; FILE_OPTS=file.filename=$IMG; COUNT=15000; CHUNK=64K; \
    CLUSTER=1M; rm -f $IMG; \
    ./qemu-img create -f qcow2 -o cluster_size=$CLUSTER $IMG 1G; \
    ./qemu-img bench -c $COUNT -d 1 -s $CHUNK -w -t none --image-opts driver=qcow2,$FILE_OPTS;

 - qemu-img bench reports ~44.3s, and if I switch FILE_OPTS to be

    FILE_OPTS=file.driver=preallocate,file.file.filename=$IMG

 qemu-img bench reports ~4.3s, 90% benefit!

(/megassd is a mount point of distributed storage)

Unfortunately, I don't know another real-world case where preallocation
would so helpful. If someone have running ceph instance and can check
it I'd be very grateful.

Still, I hope, it may be useful at least for some testing purposes.
And, small example I have: small writes with small sectors on usual ext4 over ssd gives
the following for me:

    IMG=/ssd/z; FILE_OPTS=file.filename=$IMG; COUNT=15000; CHUNK=512; \
    CLUSTER=512; rm -f $IMG; \
    ./qemu-img create -f qcow2 -o cluster_size=$CLUSTER $IMG 1G; \
    ./qemu-img bench -c $COUNT -d 1 -s $CHUNK -w -t none --image-opts driver=qcow2,$FILE_OPTS;

 ~17.1s

 and, if I switch FILE_OPTS to use new filter, the result is ~14.9s, i.e ~13% better.

=====

The series also introduces bdrv-lock-region interface, which may be reused to
implement copy-on-read operation directly inside copy-on-read filter, instead of
handling the special flag in generic code.

Vladimir Sementsov-Ogievskiy (5):
  block/io: introduce bdrv_try_mark_request_serialising
  block/io: introduce bdrv_co_range_try_lock
  block: introduce preallocate filter
  iotests: QemuIoInteractive: use qemu_io_args_no_fmt
  iotests: add 298 to test new preallocate filter driver

 qapi/block-core.json          |   3 +-
 include/block/block.h         |   9 ++
 include/block/block_int.h     |  16 +++
 include/qemu/typedefs.h       |   1 +
 block/io.c                    | 142 ++++++++++++++++---
 block/preallocate.c           | 255 ++++++++++++++++++++++++++++++++++
 block/Makefile.objs           |   1 +
 tests/qemu-iotests/298        |  45 ++++++
 tests/qemu-iotests/298.out    |   5 +
 tests/qemu-iotests/group      |   1 +
 tests/qemu-iotests/iotests.py |   2 +-
 11 files changed, 455 insertions(+), 25 deletions(-)
 create mode 100644 block/preallocate.c
 create mode 100644 tests/qemu-iotests/298
 create mode 100644 tests/qemu-iotests/298.out

-- 
2.18.0