[PATCH v21 0/4] implement zstd cluster compression method

Denis Plotnikov posted 4 patches 4 years ago
Test docker-mingw@fedora passed
Test docker-quick@centos7 passed
Test checkpatch failed
Test FreeBSD passed
Test asan passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200428133407.10657-1-dplotnikov@virtuozzo.com
Maintainers: Eric Blake <eblake@redhat.com>, Max Reitz <mreitz@redhat.com>, Markus Armbruster <armbru@redhat.com>, Kevin Wolf <kwolf@redhat.com>
There is a newer version of this series
docs/interop/qcow2.txt           |   1 +
configure                        |   2 +-
qapi/block-core.json             |  23 ++-
block/qcow2.h                    |  20 ++-
include/block/block_int.h        |   1 +
block/qcow2-threads.c            | 238 +++++++++++++++++++++++++++++--
block/qcow2.c                    | 120 ++++++++++++++++
slirp                            |   2 +-
tests/qemu-iotests/031.out       |  14 +-
tests/qemu-iotests/036.out       |   4 +-
tests/qemu-iotests/049.out       | 102 ++++++-------
tests/qemu-iotests/060.out       |   1 +
tests/qemu-iotests/061.out       |  34 +++--
tests/qemu-iotests/065           |  28 ++--
tests/qemu-iotests/080           |   2 +-
tests/qemu-iotests/144.out       |   4 +-
tests/qemu-iotests/182.out       |   2 +-
tests/qemu-iotests/242.out       |   5 +
tests/qemu-iotests/255.out       |   8 +-
tests/qemu-iotests/287           | 152 ++++++++++++++++++++
tests/qemu-iotests/287.out       |  67 +++++++++
tests/qemu-iotests/common.filter |   3 +-
tests/qemu-iotests/group         |   1 +
23 files changed, 725 insertions(+), 109 deletions(-)
create mode 100755 tests/qemu-iotests/287
create mode 100644 tests/qemu-iotests/287.out
[PATCH v21 0/4] implement zstd cluster compression method
Posted by Denis Plotnikov 4 years ago
v21:
   03:
       * remove the loop on compression [Max]
       * use designated initializers [Max]
   04:
       * don't erase user's options [Max]
       * use _rm_test_img [Max]
       * add unsupported qcow2 options [Max]

v20:
   04: fix a number of flaws [Vladimir]
       * don't use $RAND_FILE passing to qemu-io,
         so check $TEST_DIR is redundant
       * re-arrage $RAND_FILE writing
       * fix a typo

v19:
   04: fix a number of flaws [Eric]
       * remove rudundant test case descriptions
       * fix stdout redirect
       * don't use (())
       * use peek_file_be instead of od
       * check $TEST_DIR for spaces and other before using
       * use $RAND_FILE safer

v18:
   * 04: add quotes to all file name variables [Vladimir] 
   * 04: add Vladimir's comment according to "qemu-io write -s"
         option issue.

v17:
   * 03: remove incorrect comment in zstd decompress [Vladimir]
   * 03: remove "paraniod" and rewrite the comment on decompress [Vladimir]
   * 03: fix dead assignment [Vladimir]
   * 04: add and remove quotes [Vladimir]
   * 04: replace long offset form with the short one [Vladimir]

v16:
   * 03: ssize_t for ret, size_t for zstd_ret [Vladimir]
   * 04: small fixes according to the comments [Vladimir] 

v15:
   * 01: aiming qemu 5.1 [Eric]
   * 03: change zstd_res definition place [Vladimir]
   * 04: add two new test cases [Eric]
         1. test adjacent cluster compression with zstd
         2. test incompressible cluster processing
   * 03, 04: many rewording and gramma fixing [Eric]

v14:
   * fix bug on compression - looping until compress == 0 [Me]
   * apply reworked Vladimir's suggestions:
      1. not mixing ssize_t with size_t
      2. safe check for ENOMEM in compression part - avoid overflow
      3. tolerate sanity check allow zstd to make progress only
         on one of the buffers
v13:
   * 03: add progress sanity check to decompression loop [Vladimir]
     03: add successful decompression check [Me]

v12:
   * 03: again, rework compression and decompression loops
         to make them more correct [Vladimir]
     03: move assert in compression to more appropriate place
             [Vladimir]
v11:
   * 03: the loops don't need "do{}while" form anymore and
         the they were buggy (missed "do" in the beginning)
         replace them with usual "while(){}" loops [Vladimir]
v10:
   * 03: fix zstd (de)compressed loops for multi-frame
         cases [Vladimir]
v9:
   * 01: fix error checking and reporting in qcow2_amend compression type part [Vladimir]
   * 03: replace asserts with -EIO in qcow2_zstd_decompression [Vladimir, Alberto]
   * 03: reword/amend/add comments, fix typos [Vladimir]

v8:
   * 03: switch zstd API from simple to stream [Eric]
         No need to state a special cluster layout for zstd
         compressed clusters.
v7:
   * use qapi_enum_parse instead of the open-coding [Eric]
   * fix wording, typos and spelling [Eric]

v6:
   * "block/qcow2-threads: fix qcow2_decompress" is removed from the series
      since it has been accepted by Max already
   * add compile time checking for Qcow2Header to be a multiple of 8 [Max, Alberto]
   * report error on qcow2 amending when the compression type is actually chnged [Max]
   * remove the extra space and the extra new line [Max]
   * re-arrange acks and signed-off-s [Vladimir]

v5:
   * replace -ENOTSUP with abort in qcow2_co_decompress [Vladimir]
   * set cluster size for all test cases in the beginning of the 287 test

v4:
   * the series is rebased on top of 01 "block/qcow2-threads: fix qcow2_decompress"
   * 01 is just a no-change resend to avoid extra dependencies. Still, it may be merged in separate

v3:
   * remove redundant max compression type value check [Vladimir, Eric]
     (the switch below checks everything)
   * prevent compression type changing on "qemu-img amend" [Vladimir]
   * remove zstd config setting, since it has been added already by
     "migration" patches [Vladimir]
   * change the compression type error message [Vladimir] 
   * fix alignment and 80-chars exceeding [Vladimir]

v2:
   * rework compression type setting [Vladimir]
   * squash iotest changes to the compression type introduction patch [Vladimir, Eric]
   * fix zstd availability checking in zstd iotest [Vladimir]
   * remove unnecessry casting [Eric]
   * remove rudundant checks [Eric]
   * fix compressed cluster layout in qcow2 spec [Vladimir]
   * fix wording [Eric, Vladimir]
   * fix compression type filtering in iotests [Eric]

v1:
   the initial series

Denis Plotnikov (4):
  qcow2: introduce compression type feature
  qcow2: rework the cluster compression routine
  qcow2: add zstd cluster compression
  iotests: 287: add qcow2 compression type test

 docs/interop/qcow2.txt           |   1 +
 configure                        |   2 +-
 qapi/block-core.json             |  23 ++-
 block/qcow2.h                    |  20 ++-
 include/block/block_int.h        |   1 +
 block/qcow2-threads.c            | 238 +++++++++++++++++++++++++++++--
 block/qcow2.c                    | 120 ++++++++++++++++
 slirp                            |   2 +-
 tests/qemu-iotests/031.out       |  14 +-
 tests/qemu-iotests/036.out       |   4 +-
 tests/qemu-iotests/049.out       | 102 ++++++-------
 tests/qemu-iotests/060.out       |   1 +
 tests/qemu-iotests/061.out       |  34 +++--
 tests/qemu-iotests/065           |  28 ++--
 tests/qemu-iotests/080           |   2 +-
 tests/qemu-iotests/144.out       |   4 +-
 tests/qemu-iotests/182.out       |   2 +-
 tests/qemu-iotests/242.out       |   5 +
 tests/qemu-iotests/255.out       |   8 +-
 tests/qemu-iotests/287           | 152 ++++++++++++++++++++
 tests/qemu-iotests/287.out       |  67 +++++++++
 tests/qemu-iotests/common.filter |   3 +-
 tests/qemu-iotests/group         |   1 +
 23 files changed, 725 insertions(+), 109 deletions(-)
 create mode 100755 tests/qemu-iotests/287
 create mode 100644 tests/qemu-iotests/287.out

-- 
2.17.0


Re: [PATCH v21 0/4] implement zstd cluster compression method
Posted by no-reply@patchew.org 4 years ago
Patchew URL: https://patchew.org/QEMU/20200428133407.10657-1-dplotnikov@virtuozzo.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [PATCH v21 0/4] implement zstd cluster compression method
Message-id: 20200428133407.10657-1-dplotnikov@virtuozzo.com
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
146ff2a iotests: 287: add qcow2 compression type test
694bbc7 qcow2: add zstd cluster compression
954ac0d qcow2: rework the cluster compression routine
3294c21 qcow2: introduce compression type feature

=== OUTPUT BEGIN ===
1/4 Checking commit 3294c21f8a66 (qcow2: introduce compression type feature)
2/4 Checking commit 954ac0d0a541 (qcow2: rework the cluster compression routine)
3/4 Checking commit 694bbc7bf00f (qcow2: add zstd cluster compression)
ERROR: do not use assignment in if condition
#115: FILE: block/qcow2-threads.c:225:
+    if ((zstd_ret = ZSTD_compressStream2(cctx, &output, &input, ZSTD_e_end))) {

total: 1 errors, 0 warnings, 238 lines checked

Patch 3/4 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/4 Checking commit 146ff2a078a9 (iotests: 287: add qcow2 compression type test)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#23: 
new file mode 100755

total: 0 errors, 1 warnings, 228 lines checked

Patch 4/4 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200428133407.10657-1-dplotnikov@virtuozzo.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH v21 0/4] implement zstd cluster compression method
Posted by no-reply@patchew.org 4 years ago
Patchew URL: https://patchew.org/QEMU/20200428133407.10657-1-dplotnikov@virtuozzo.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [PATCH v21 0/4] implement zstd cluster compression method
Message-id: 20200428133407.10657-1-dplotnikov@virtuozzo.com
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]      patchew/20200428132629.796753-1-mreitz@redhat.com -> patchew/20200428132629.796753-1-mreitz@redhat.com
Switched to a new branch 'test'
7201c7e iotests: 287: add qcow2 compression type test
7f44ce7 qcow2: add zstd cluster compression
0504310 qcow2: rework the cluster compression routine
01049b7 qcow2: introduce compression type feature

=== OUTPUT BEGIN ===
1/4 Checking commit 01049b7e28e9 (qcow2: introduce compression type feature)
2/4 Checking commit 0504310611bc (qcow2: rework the cluster compression routine)
3/4 Checking commit 7f44ce7d732b (qcow2: add zstd cluster compression)
ERROR: do not use assignment in if condition
#115: FILE: block/qcow2-threads.c:225:
+    if ((zstd_ret = ZSTD_compressStream2(cctx, &output, &input, ZSTD_e_end))) {

total: 1 errors, 0 warnings, 238 lines checked

Patch 3/4 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/4 Checking commit 7201c7e73ce4 (iotests: 287: add qcow2 compression type test)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#23: 
new file mode 100755

total: 0 errors, 1 warnings, 228 lines checked

Patch 4/4 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200428133407.10657-1-dplotnikov@virtuozzo.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com