[PATCH 0/6] qcow2: advanced compression options

Andrey Shinkevich posted 6 patches 4 years, 6 months ago
Test checkpatch passed
Test docker-mingw@fedora failed
Test docker-quick@centos7 failed
Test docker-clang@ubuntu failed
Test asan failed
Test FreeBSD passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/1569958040-697220-1-git-send-email-andrey.shinkevich@virtuozzo.com
Maintainers: Fam Zheng <fam@euphon.net>, Max Reitz <mreitz@redhat.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Eric Blake <eblake@redhat.com>, John Snow <jsnow@redhat.com>, Markus Armbruster <armbru@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Stefan Hajnoczi <stefanha@redhat.com>
There is a newer version of this series
block/io.c                 |  21 +++++++--
block/qcow2.c              | 113 ++++++++++++++++++++++++++++++++++-----------
block/stream.c             |  19 +++++---
block/trace-events         |   2 +-
blockdev-nbd.c             |   2 +-
blockdev.c                 |  14 +++++-
hmp-commands.hx            |   4 +-
include/block/block_int.h  |   3 +-
include/block/nbd.h        |   7 ++-
monitor/hmp-cmds.c         |   5 +-
nbd/server.c               |   8 +++-
qapi/block-core.json       |   5 +-
qemu-nbd.c                 |  18 +++++++-
qemu-nbd.texi              |   2 +
tests/qemu-iotests/030     |  49 +++++++++++++++++++-
tests/qemu-iotests/030.out |   4 +-
tests/qemu-iotests/214     |   9 ++++
tests/qemu-iotests/214.out |   6 +++
18 files changed, 236 insertions(+), 55 deletions(-)
[PATCH 0/6] qcow2: advanced compression options
Posted by Andrey Shinkevich 4 years, 6 months ago
New enhancements of writing compressed data to QCOW2 image.

Based on: message ID <20190916175324.18478-1-vsementsov@virtuozzo.com>
          https://lists.nongnu.org/archive/html/qemu-block/2019-09/msg00751.html

Andrey Shinkevich (6):
  qcow2: multiple clusters write compressed
  tests/qemu-iotests: add case of writing compressed data to multiple
    clusters.
  block: support compressed write for copy-on-read
  qemu-nbd: add compression flag support
  block-stream: add compress option
  tests/qemu-iotests: add case for block-stream compress

 block/io.c                 |  21 +++++++--
 block/qcow2.c              | 113 ++++++++++++++++++++++++++++++++++-----------
 block/stream.c             |  19 +++++---
 block/trace-events         |   2 +-
 blockdev-nbd.c             |   2 +-
 blockdev.c                 |  14 +++++-
 hmp-commands.hx            |   4 +-
 include/block/block_int.h  |   3 +-
 include/block/nbd.h        |   7 ++-
 monitor/hmp-cmds.c         |   5 +-
 nbd/server.c               |   8 +++-
 qapi/block-core.json       |   5 +-
 qemu-nbd.c                 |  18 +++++++-
 qemu-nbd.texi              |   2 +
 tests/qemu-iotests/030     |  49 +++++++++++++++++++-
 tests/qemu-iotests/030.out |   4 +-
 tests/qemu-iotests/214     |   9 ++++
 tests/qemu-iotests/214.out |   6 +++
 18 files changed, 236 insertions(+), 55 deletions(-)

-- 
1.8.3.1


Re: [PATCH 0/6] qcow2: advanced compression options
Posted by no-reply@patchew.org 4 years, 6 months ago
Patchew URL: https://patchew.org/QEMU/1569958040-697220-1-git-send-email-andrey.shinkevich@virtuozzo.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      block/qed-l2-cache.o
  CC      block/qed-table.o
  CC      block/qed-cluster.o
/tmp/qemu-test/src/block/qcow2.c:4077:64: error: unknown type name 'AioTask'
 static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task)
                                                                ^
/tmp/qemu-test/src/block/qcow2.c: In function 'qcow2_co_pwritev_compressed_part':
/tmp/qemu-test/src/block/qcow2.c:4098:5: error: unknown type name 'AioTaskPool'
     AioTaskPool *aio = NULL;
     ^
/tmp/qemu-test/src/block/qcow2.c:4123:5: error: implicit declaration of function 'aio_task_pool_status' [-Werror=implicit-function-declaration]
     while (bytes && aio_task_pool_status(aio) == 0) {
     ^
/tmp/qemu-test/src/block/qcow2.c:4123:5: error: nested extern declaration of 'aio_task_pool_status' [-Werror=nested-externs]
/tmp/qemu-test/src/block/qcow2.c:4130:13: error: implicit declaration of function 'aio_task_pool_new' [-Werror=implicit-function-declaration]
             aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
             ^
/tmp/qemu-test/src/block/qcow2.c:4130:13: error: nested extern declaration of 'aio_task_pool_new' [-Werror=nested-externs]
/tmp/qemu-test/src/block/qcow2.c:4130:37: error: 'QCOW2_MAX_WORKERS' undeclared (first use in this function)
             aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
                                     ^
/tmp/qemu-test/src/block/qcow2.c:4130:37: note: each undeclared identifier is reported only once for each function it appears in
/tmp/qemu-test/src/block/qcow2.c:4133:9: error: implicit declaration of function 'qcow2_add_task' [-Werror=implicit-function-declaration]
         ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_entry,
         ^
/tmp/qemu-test/src/block/qcow2.c:4133:9: error: nested extern declaration of 'qcow2_add_task' [-Werror=nested-externs]
/tmp/qemu-test/src/block/qcow2.c:4133:39: error: 'qcow2_co_pwritev_compressed_task_entry' undeclared (first use in this function)
         ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_entry,
                                       ^
/tmp/qemu-test/src/block/qcow2.c:4145:9: error: implicit declaration of function 'aio_task_pool_wait_all' [-Werror=implicit-function-declaration]
         aio_task_pool_wait_all(aio);
         ^
/tmp/qemu-test/src/block/qcow2.c:4145:9: error: nested extern declaration of 'aio_task_pool_wait_all' [-Werror=nested-externs]
/tmp/qemu-test/src/block/qcow2.c: At top level:
/tmp/qemu-test/src/block/qcow2.c:4012:1: error: 'qcow2_co_pwritev_compressed_task' defined but not used [-Werror=unused-function]
 qcow2_co_pwritev_compressed_task(BlockDriverState *bs,
 ^
cc1: all warnings being treated as errors
make: *** [block/qcow2.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 662, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=c0ff6dd6fb3f46d1b9c07a106c9dd26a', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-8m_nebab/src/docker-src.2019-10-01-22.22.08.6455:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=c0ff6dd6fb3f46d1b9c07a106c9dd26a
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-8m_nebab/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    1m40.390s
user    0m8.966s


The full log is available at
http://patchew.org/logs/1569958040-697220-1-git-send-email-andrey.shinkevich@virtuozzo.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH 0/6] qcow2: advanced compression options
Posted by no-reply@patchew.org 4 years, 6 months ago
Patchew URL: https://patchew.org/QEMU/1569958040-697220-1-git-send-email-andrey.shinkevich@virtuozzo.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      block/qed-table.o
  CC      block/qed-cluster.o
  CC      block/qed-check.o
/tmp/qemu-test/src/block/qcow2.c:4077:64: error: unknown type name 'AioTask'; did you mean 'AioWait'?
 static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task)
                                                                ^~~~~~~
                                                                AioWait
/tmp/qemu-test/src/block/qcow2.c: In function 'qcow2_co_pwritev_compressed_part':
/tmp/qemu-test/src/block/qcow2.c:4098:5: error: unknown type name 'AioTaskPool'
     AioTaskPool *aio = NULL;
     ^~~~~~~~~~~
/tmp/qemu-test/src/block/qcow2.c:4123:21: error: implicit declaration of function 'aio_task_pool_status' [-Werror=implicit-function-declaration]
     while (bytes && aio_task_pool_status(aio) == 0) {
                     ^~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/block/qcow2.c:4123:21: error: nested extern declaration of 'aio_task_pool_status' [-Werror=nested-externs]
/tmp/qemu-test/src/block/qcow2.c:4130:19: error: implicit declaration of function 'aio_task_pool_new'; did you mean 'aio_timer_new'? [-Werror=implicit-function-declaration]
             aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
                   ^~~~~~~~~~~~~~~~~
                   aio_timer_new
/tmp/qemu-test/src/block/qcow2.c:4130:19: error: nested extern declaration of 'aio_task_pool_new' [-Werror=nested-externs]
/tmp/qemu-test/src/block/qcow2.c:4130:37: error: 'QCOW2_MAX_WORKERS' undeclared (first use in this function); did you mean 'QCOW2_MAX_THREADS'?
             aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
                                     ^~~~~~~~~~~~~~~~~
                                     QCOW2_MAX_THREADS
/tmp/qemu-test/src/block/qcow2.c:4130:37: note: each undeclared identifier is reported only once for each function it appears in
/tmp/qemu-test/src/block/qcow2.c:4133:15: error: implicit declaration of function 'qcow2_add_task'; did you mean 'qcow2_do_open'? [-Werror=implicit-function-declaration]
         ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_entry,
               ^~~~~~~~~~~~~~
               qcow2_do_open
/tmp/qemu-test/src/block/qcow2.c:4133:15: error: nested extern declaration of 'qcow2_add_task' [-Werror=nested-externs]
/tmp/qemu-test/src/block/qcow2.c:4133:39: error: 'qcow2_co_pwritev_compressed_task_entry' undeclared (first use in this function); did you mean 'qcow2_co_pwritev_compressed_task'?
         ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_entry,
                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                       qcow2_co_pwritev_compressed_task
/tmp/qemu-test/src/block/qcow2.c:4145:9: error: implicit declaration of function 'aio_task_pool_wait_all' [-Werror=implicit-function-declaration]
         aio_task_pool_wait_all(aio);
         ^~~~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/block/qcow2.c:4145:9: error: nested extern declaration of 'aio_task_pool_wait_all' [-Werror=nested-externs]
At top level:
/tmp/qemu-test/src/block/qcow2.c:4012:1: error: 'qcow2_co_pwritev_compressed_task' defined but not used [-Werror=unused-function]
 qcow2_co_pwritev_compressed_task(BlockDriverState *bs,
 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make: *** [/tmp/qemu-test/src/rules.mak:69: block/qcow2.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 662, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=5e43e6e4ab784c32a96f618a29b7ceef', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-yubxkwpj/src/docker-src.2019-10-01-22.23.28.10478:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=5e43e6e4ab784c32a96f618a29b7ceef
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-yubxkwpj/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    2m9.622s
user    0m7.777s


The full log is available at
http://patchew.org/logs/1569958040-697220-1-git-send-email-andrey.shinkevich@virtuozzo.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com