Series comparison

-[PULL 00/18] Block patches
+[PULL 00/24] Block patches
-The following changes since commit bae31bfa48b9caecee25da3d5333901a126a06b4:
+The following changes since commit 804b30d25f8d70dc2dea951883ea92235274a50c:
-  Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200619-pull-request' into staging (2020-06-19 22:56:59 +0100)
+  Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220130' into staging (2022-01-31 11:10:08 +0000)
 are available in the Git repository at:
-  https://github.com/XanClic/qemu.git tags/pull-block-2020-06-22
+  https://gitlab.com/hreitz/qemu.git tags/pull-block-2022-02-01
-for you to fetch changes up to 74c55e4142a7bb835c38d3770c74210cbb1e4fab:
+for you to fetch changes up to 751486c18555169ca4baf59440275d5831140822:
-  iotests: don't test qcow2.py inside 291 (2020-06-22 16:05:23 +0200)
+  block.h: remove outdated comment (2022-02-01 13:28:53 +0100)
 ----------------------------------------------------------------
 Block patches:
-- Support modifying a LUKS-encrypted image's keyslots
+- Add support to the iotests to test qcow2's zstd compression mode
-- iotest fixes
+- Fix post-migration block node permissions
 - iotests fixes (051 and mirror-ready-cancel-error)
 - Remove an outdated comment
 ----------------------------------------------------------------
-Max Reitz (1):
+Emanuele Giuseppe Esposito (1):
-  iotests: Make _filter_img_create more active
+  block.h: remove outdated comment
-Maxim Levitsky (14):
+Hanna Reitz (3):
-  iotests: filter few more luks specific create options
+  iotests/MRCE: Write data to source
-  qcrypto/core: add generic infrastructure for crypto options amendment
+  block-backend: Retain permissions after migration
-  qcrypto/luks: implement encryption key management
+  iotests/migration-permissions: New test
   block/amend: add 'force' option
   block/amend: separate amend and create options for qemu-img
   block/amend: refactor qcow2 amend options
   block/crypto: rename two functions
   block/crypto: implement the encryption key management
   block/qcow2: extend qemu-img amend interface with crypto options
   iotests: qemu-img tests for luks key management
   block/core: add generic infrastructure for x-blockdev-amend qmp
     command
   block/crypto: implement blockdev-amend
   block/qcow2: implement blockdev-amend
   iotests: add tests for blockdev-amend
-Philippe Mathieu-Daudé (1):
+Thomas Huth (1):
-  iotests: Fix 051 output after qdev_init_nofail() removal
+  tests/qemu-iotests: Fix 051 for binaries without 'lsi53c895a'
-Vladimir Sementsov-Ogievskiy (2):
+Vladimir Sementsov-Ogievskiy (19):
-  block/block-copy: block_copy_dirty_clusters: fix failure check
+  iotests.py: img_info_log(): rename imgopts argument
-  iotests: don't test qcow2.py inside 291
+  iotests.py: implement unsupported_imgopts
   iotests: specify some unsupported_imgopts for python iotests
   iotests.py: qemu_img*("create"): support
     IMGOPTS='compression_type=zstd'
   iotests: drop qemu_img_verbose() helper
   iotests.py: rewrite default luks support in qemu_img
   iotest 303: explicit compression type
   iotest 065: explicit compression type
   iotests.py: filter out successful output of qemu-img create
   iotests.py: filter compression type out
   iotest 302: use img_info_log() helper
   qcow2: simple case support for downgrading of qcow2 images with zstd
   iotests/common.rc: introduce _qcow2_dump_header helper
   iotests: massive use _qcow2_dump_header
   iotest 39: use _qcow2_dump_header
   iotests: bash tests: filter compression type
   iotests 60: more accurate set dirty bit in qcow2 header
   iotest 214: explicit compression type
   iotests: declare lack of support for compresion_type in IMGOPTS
- docs/tools/qemu-img.rst          |   5 +-
+ include/block/block.h                         |   1 -
- qapi/block-core.json             |  68 +++++
+ block/block-backend.c                         |  11 ++
- qapi/crypto.json                 |  73 +++++-
+ block/qcow2.c                                 |  58 +++++++++-
- qapi/job.json                    |   4 +-
+ tests/qemu-iotests/031                        |  11 +-
- block/crypto.h                   |  37 +++
+ tests/qemu-iotests/036                        |   6 +-
- crypto/blockpriv.h               |   8 +
+ tests/qemu-iotests/039                        |  22 ++--
- include/block/block.h            |   1 +
+ tests/qemu-iotests/044                        |   8 +-
- include/block/block_int.h        |  24 +-
+ tests/qemu-iotests/044.out                    |   1 +
- include/crypto/block.h           |  22 ++
+ tests/qemu-iotests/051                        |   9 +-
- block.c                          |   4 +-
+ tests/qemu-iotests/060                        |  22 ++--
- block/amend.c                    | 113 +++++++++
+ tests/qemu-iotests/060.out                    |   2 +-
- block/block-copy.c               |   4 +-
+ tests/qemu-iotests/061                        |  42 ++++----
- block/crypto.c                   | 206 +++++++++++++--
+ tests/qemu-iotests/061.out                    |  12 +--
- block/qcow2.c                    | 332 +++++++++++++-----------
+ tests/qemu-iotests/065                        |  19 ++--
- crypto/block-luks.c              | 416 ++++++++++++++++++++++++++++++-
+ tests/qemu-iotests/082.out                    |  14 +--
- crypto/block.c                   |  29 +++
+ tests/qemu-iotests/112                        |   3 +-
- qemu-img.c                       |  44 +++-
+ tests/qemu-iotests/137                        |   2 +-
- block/Makefile.objs              |   2 +-
+ tests/qemu-iotests/149.out                    |  21 ----
- qemu-img-cmds.hx                 |   4 +-
+ tests/qemu-iotests/163                        |   3 +-
- tests/qemu-iotests/049.out       | 102 ++++----
+ tests/qemu-iotests/165                        |   3 +-
- tests/qemu-iotests/051.pc.out    |   4 +-
+ tests/qemu-iotests/196                        |   3 +-
- tests/qemu-iotests/061.out       |  12 +-
+ tests/qemu-iotests/198.out                    |   4 +-
- tests/qemu-iotests/082.out       | 185 ++++----------
+ tests/qemu-iotests/206.out                    |  10 +-
- tests/qemu-iotests/085.out       |  38 +--
+ tests/qemu-iotests/209                        |   7 +-
- tests/qemu-iotests/087.out       |   6 +-
+ tests/qemu-iotests/209.out                    |   2 +
- tests/qemu-iotests/112.out       |   2 +-
+ tests/qemu-iotests/210                        |   8 +-
- tests/qemu-iotests/134.out       |   2 +-
+ tests/qemu-iotests/214                        |   2 +-
- tests/qemu-iotests/141           |   2 +-
+ tests/qemu-iotests/237.out                    |   3 -
- tests/qemu-iotests/144.out       |   4 +-
+ tests/qemu-iotests/242                        |   3 +-
- tests/qemu-iotests/153           |   9 +-
+ tests/qemu-iotests/242.out                    |  10 +-
- tests/qemu-iotests/158.out       |   4 +-
+ tests/qemu-iotests/246                        |   3 +-
- tests/qemu-iotests/182.out       |   2 +-
+ tests/qemu-iotests/254                        |   3 +-
- tests/qemu-iotests/185.out       |   8 +-
+ tests/qemu-iotests/255.out                    |   4 -
- tests/qemu-iotests/188.out       |   2 +-
+ tests/qemu-iotests/260                        |   3 +-
- tests/qemu-iotests/189.out       |   4 +-
+ tests/qemu-iotests/274                        |   3 +-
- tests/qemu-iotests/198.out       |   4 +-
+ tests/qemu-iotests/274.out                    |  39 +------
- tests/qemu-iotests/255.out       |   8 +-
+ tests/qemu-iotests/280.out                    |   1 -
- tests/qemu-iotests/263.out       |   4 +-
+ tests/qemu-iotests/281                        |   3 +-
- tests/qemu-iotests/274.out       |  46 ++--
+ tests/qemu-iotests/287                        |   8 +-
- tests/qemu-iotests/280.out       |   2 +-
+ tests/qemu-iotests/290                        |   2 +-
- tests/qemu-iotests/284.out       |   6 +-
+ tests/qemu-iotests/296.out                    |  10 +-
- tests/qemu-iotests/291           |   4 -
+ tests/qemu-iotests/302                        |   4 +-
- tests/qemu-iotests/291.out       |  33 ---
+ tests/qemu-iotests/302.out                    |   7 +-
- tests/qemu-iotests/293           | 207 +++++++++++++++
+ tests/qemu-iotests/303                        |  26 +++--
- tests/qemu-iotests/293.out       |  99 ++++++++
+ tests/qemu-iotests/303.out                    |  30 +++++-
- tests/qemu-iotests/294           |  90 +++++++
+ tests/qemu-iotests/common.filter              |   8 ++
- tests/qemu-iotests/294.out       |  30 +++
+ tests/qemu-iotests/common.rc                  |  22 ++++
- tests/qemu-iotests/295           | 279 +++++++++++++++++++++
+ tests/qemu-iotests/iotests.py                 |  99 +++++++++++------
- tests/qemu-iotests/295.out       |  40 +++
+ .../tests/migrate-bitmaps-postcopy-test       |   3 +-
- tests/qemu-iotests/296           | 234 +++++++++++++++++
+ tests/qemu-iotests/tests/migrate-bitmaps-test |   3 +-
- tests/qemu-iotests/296.out       |  33 +++
+ .../qemu-iotests/tests/migration-permissions  | 101 ++++++++++++++++++
- tests/qemu-iotests/common.filter |  93 +++++--
+ .../tests/migration-permissions.out           |   5 +
- tests/qemu-iotests/group         |   4 +
+ .../tests/mirror-ready-cancel-error           |   7 +-
-files changed, 2482 insertions(+), 516 deletions(-)
+ .../tests/remove-bitmap-from-backing          |   3 +-
- create mode 100644 block/amend.c
+files changed, 483 insertions(+), 236 deletions(-)
- create mode 100755 tests/qemu-iotests/293
+ create mode 100755 tests/qemu-iotests/tests/migration-permissions
- create mode 100644 tests/qemu-iotests/293.out
+ create mode 100644 tests/qemu-iotests/tests/migration-permissions.out
  create mode 100755 tests/qemu-iotests/294
  create mode 100644 tests/qemu-iotests/294.out
  create mode 100755 tests/qemu-iotests/295
  create mode 100644 tests/qemu-iotests/295.out
  create mode 100755 tests/qemu-iotests/296
  create mode 100644 tests/qemu-iotests/296.out
 --
-.26.2
+.34.1

-New patch
+[PULL 01/24] tests/qemu-iotests: Fix 051 for binaries without 'lsi53c895a'
+From: Thomas Huth <thuth@redhat.com>
+The lsi53c895a SCSI adaptor might not be enabled in each and every
+x86 QEMU binary, e.g. it's disabled in the RHEL/CentOS build.
+Thus let's add a check to the 051 test so that it does not fail if
+this device is not available.
+Signed-off-by: Thomas Huth <thuth@redhat.com>
+Message-Id: <20211206143404.247032-1-thuth@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+---
+ tests/qemu-iotests/051 | 4 ++++
+file changed, 4 insertions(+)
+diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/051
++++ b/tests/qemu-iotests/051
+@@ -XXX,XX +XXX,XX @@ _supported_proto file
+ _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file
+ _require_drivers nbd
++if [ "$QEMU_DEFAULT_MACHINE" = "pc" ]; then
++    _require_devices lsi53c895a
++fi
++
+ do_run_qemu()
+ {
+     echo Testing: "$@"
+--
+.34.1

-New patch
+[PULL 02/24] iotests/MRCE: Write data to source
+This test assumes that mirror flushes the source when entering the READY
+state, and that the format level will pass that flush on to the protocol
+level (where we intercept it with blkdebug).
+However, apparently that does not happen when using a VMDK image with
+zeroed_grain=on, which actually is the default set by testenv.py.  Right
+now, Python tests ignore IMGOPTS, though, so this has no effect; but
+Vladimir has a series that will change this, so we need to fix this test
+before that series lands.
+We can fix it by writing data to the source before we start the mirror
+job; apparently that makes the (VMDK) format layer change its mind and
+pass on the pre-READY flush to the protocol level, so the test passes
+again.  (I presume, without any data written, mirror just does a 64M
+zero write on the target, which VMDK with zeroed_grain=on basically just
+ignores.)
+Without this, we do not get a flush, and so blkdebug only sees a single
+flush at the end of the job instead of two, and therefore does not
+inject an error, which makes the block job complete instead of raising
+an error.
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+Message-Id: <20211223165308.103793-1-hreitz@redhat.com>
+Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+---
+ tests/qemu-iotests/tests/mirror-ready-cancel-error | 7 ++++++-
+file changed, 6 insertions(+), 1 deletion(-)
+diff --git a/tests/qemu-iotests/tests/mirror-ready-cancel-error b/tests/qemu-iotests/tests/mirror-ready-cancel-error
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/tests/mirror-ready-cancel-error
++++ b/tests/qemu-iotests/tests/mirror-ready-cancel-error
+@@ -XXX,XX +XXX,XX @@ class TestMirrorReadyCancelError(iotests.QMPTestCase):
+         assert iotests.qemu_img_create('-f', iotests.imgfmt, target,
+                                        str(image_size)) == 0
++        # Ensure that mirror will copy something before READY so the
++        # target format layer will forward the pre-READY flush to its
++        # file child
++        assert iotests.qemu_io_silent('-c', 'write -P 1 0 64k', source) == 0
++
+         self.vm = iotests.VM()
+         self.vm.launch()
+@@ -XXX,XX +XXX,XX @@ class TestMirrorReadyCancelError(iotests.QMPTestCase):
+         # Write something so will not leave the job immediately, but
+         # flush first (which will fail, thanks to blkdebug)
+         res = self.vm.qmp('human-monitor-command',
+-                          command_line='qemu-io mirror-top "write 0 64k"')
++                          command_line='qemu-io mirror-top "write -P 2 0 64k"')
+         self.assert_qmp(res, 'return', '')
+         # Drain status change events
+--
+.34.1

-New patch
+[PULL 03/24] iotests.py: img_info_log(): rename imgopts argument
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+We are going to support IMGOPTS environment variable like in bash
+tests. Corresponding global variable in iotests.py should be called
+imgopts. So to not interfere with function argument, rename it in
+advance.
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Reviewed-by: Max Reitz <mreitz@redhat.com>
+Message-Id: <20211223160144.1097696-2-vsementsov@virtuozzo.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+---
+ tests/qemu-iotests/210        | 8 ++++----
+ tests/qemu-iotests/iotests.py | 5 +++--
+files changed, 7 insertions(+), 6 deletions(-)
+diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/210
++++ b/tests/qemu-iotests/210
+@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
+         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
+         filter_path=disk_path,
+         extra_args=['--object', 'secret,id=keysec0,data=foo'],
+-        imgopts=True)
++        use_image_opts=True)
+     #
+     # Successful image creation (with non-default options)
+@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
+         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
+         filter_path=disk_path,
+         extra_args=['--object', 'secret,id=keysec0,data=foo'],
+-        imgopts=True)
++        use_image_opts=True)
+     #
+     # Invalid BlockdevRef
+@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
+         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
+         filter_path=disk_path,
+         extra_args=['--object', 'secret,id=keysec0,data=foo'],
+-        imgopts=True)
++        use_image_opts=True)
+     #
+     # Invalid sizes
+@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
+         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
+         filter_path=disk_path,
+         extra_args=['--object', 'secret,id=keysec0,data=foo'],
+-        imgopts=True)
++        use_image_opts=True)
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/iotests.py
++++ b/tests/qemu-iotests/iotests.py
+@@ -XXX,XX +XXX,XX @@ def qemu_img_log(*args):
+     log(result, filters=[filter_testfiles])
+     return result
+-def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()):
++def img_info_log(filename, filter_path=None, use_image_opts=False,
++                 extra_args=()):
+     args = ['info']
+-    if imgopts:
++    if use_image_opts:
+         args.append('--image-opts')
+     else:
+         args += ['-f', imgfmt]
+--
+.34.1

-[PULL 10/18] block/crypto: implement the encryption key management
+[PULL 04/24] iotests.py: implement unsupported_imgopts
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-This implements the encryption key management using the generic code in
+We are going to support some addition IMGOPTS in python iotests like
-qcrypto layer and exposes it to the user via qemu-img
+in bash iotests. Similarly to bash iotests, we want a way to skip some
 tests which can't work with specific IMGOPTS.
-This code adds another 'write_func' because the initialization
+Globally for python iotests we will not support things like
-write_func works directly on the underlying file, and amend
+'data_file=$TEST_IMG.ext_data_file' in IMGOPTS, so, forbid this
-works on instance of luks device.
+globally in iotests.py.
-This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
+Suggested-by: Hanna Reitz <hreitz@redhat.com>
-made to make the driver both support write sharing (to avoid breaking the users),
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-and be safe against concurrent  metadata update (the keyslots)
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20211223160144.1097696-3-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/iotests.py | 15 ++++++++++++++-
 file changed, 14 insertions(+), 1 deletion(-)
-Eventually the write sharing for luks driver will be deprecated
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
 and removed together with this hack.
 The hack is that we ask (as a format driver) for BLK_PERM_CONSISTENT_READ
 and then when we want to update the keys, we unshare that permission.
 So if someone else has the image open, even readonly, encryption
 key update will fail gracefully.
 Also thanks to Daniel Berrange for the idea of
 unsharing read, rather that write permission which allows
 to avoid cases when the other user had opened the image read-only.
 Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
 Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Message-Id: <20200608094030.670121-8-mlevitsk@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
  block/crypto.h |  34 +++++++++++++
  block/crypto.c | 130 +++++++++++++++++++++++++++++++++++++++++++++++--
 files changed, 161 insertions(+), 3 deletions(-)
 diff --git a/block/crypto.h b/block/crypto.h
 index XXXXXXX..XXXXXXX 100644
---- a/block/crypto.h
+--- a/tests/qemu-iotests/iotests.py
-+++ b/block/crypto.h
++++ b/tests/qemu-iotests/iotests.py
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ def _verify_virtio_scsi_pci_or_ccw() -> None:
- #define BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG "ivgen-hash-alg"
+         notrun('Missing virtio-scsi-pci or virtio-scsi-ccw in QEMU binary')
- #define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg"
- #define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time"
-+#define BLOCK_CRYPTO_OPT_LUKS_KEYSLOT "keyslot"
++def _verify_imgopts(unsupported: Sequence[str] = ()) -> None:
-+#define BLOCK_CRYPTO_OPT_LUKS_STATE "state"
++    imgopts = os.environ.get('IMGOPTS')
-+#define BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET "old-secret"
++    # One of usage examples for IMGOPTS is "data_file=$TEST_IMG.ext_data_file"
-+#define BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET "new-secret"
++    # but it supported only for bash tests. We don't have a concept of global
-+
++    # TEST_IMG in iotests.py, not saying about somehow parsing $variables.
++    # So, for simplicity let's just not support any IMGOPTS with '$' inside.
- #define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(prefix)                    \
++    unsup = list(unsupported) + ['$']
-     BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix,                             \
++    if imgopts and any(x in imgopts for x in unsup):
-@@ -XXX,XX +XXX,XX @@
++        notrun(f'not suitable for this imgopts: {imgopts}')
          .help = "Time to spend in PBKDF in milliseconds",     \
      }
 +#define BLOCK_CRYPTO_OPT_DEF_LUKS_STATE(prefix)                           \
 +    {                                                                     \
 +        .name = prefix BLOCK_CRYPTO_OPT_LUKS_STATE,                       \
 +        .type = QEMU_OPT_STRING,                                          \
 +        .help = "Select new state of affected keyslots (active/inactive)",\
 +    }
 +
 +#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT(prefix)              \
 +    {                                                          \
 +        .name = prefix BLOCK_CRYPTO_OPT_LUKS_KEYSLOT,          \
 +        .type = QEMU_OPT_NUMBER,                               \
 +        .help = "Select a single keyslot to modify explicitly",\
 +    }
 +
 +#define BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET(prefix)            \
 +    {                                                           \
 +        .name = prefix BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET,        \
 +        .type = QEMU_OPT_STRING,                                \
 +        .help = "Select all keyslots that match this password", \
 +    }
 +
 +#define BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET(prefix)            \
 +    {                                                           \
 +        .name = prefix BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET,        \
 +        .type = QEMU_OPT_STRING,                                \
 +        .help = "New secret to set in the matching keyslots. "  \
 +                "Empty string to erase",                        \
 +    }
 +
  QCryptoBlockCreateOptions *
  block_crypto_create_opts_init(QDict *opts, Error **errp);
 diff --git a/block/crypto.c b/block/crypto.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/crypto.c
 +++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCrypto BlockCrypto;
  struct BlockCrypto {
      QCryptoBlock *block;
 +    bool updating_keys;
  };
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
      return ret;
  }
 +static ssize_t block_crypto_write_func(QCryptoBlock *block,
 +                                       size_t offset,
 +                                       const uint8_t *buf,
 +                                       size_t buflen,
 +                                       void *opaque,
 +                                       Error **errp)
 +{
 +    BlockDriverState *bs = opaque;
 +    ssize_t ret;
 +
 +    ret = bdrv_pwrite(bs->file, offset, buf, buflen);
 +    if (ret < 0) {
 +        error_setg_errno(errp, -ret, "Could not write encryption header");
 +        return ret;
 +    }
 +    return ret;
 +}
 +
  struct BlockCryptoCreateData {
      BlockBackend *blk;
@@ -XXX,XX +XXX,XX @@ static QemuOptsList block_crypto_create_opts_luks = {
  };
 +static QemuOptsList block_crypto_amend_opts_luks = {
 +    .name = "crypto",
 +    .head = QTAILQ_HEAD_INITIALIZER(block_crypto_create_opts_luks.head),
 +    .desc = {
 +        BLOCK_CRYPTO_OPT_DEF_LUKS_STATE(""),
 +        BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT(""),
 +        BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET(""),
 +        BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET(""),
 +        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME(""),
 +        { /* end of list */ }
 +    },
 +};
 +
  QCryptoBlockOpenOptions *
  block_crypto_open_opts_init(QDict *opts, Error **errp)
  {
@@ -XXX,XX +XXX,XX @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp)
      return spec_info;
  }
 +static int
 +block_crypto_amend_options_luks(BlockDriverState *bs,
 +                                QemuOpts *opts,
 +                                BlockDriverAmendStatusCB *status_cb,
 +                                void *cb_opaque,
 +                                bool force,
 +                                Error **errp)
 +{
 +    BlockCrypto *crypto = bs->opaque;
 +    QDict *cryptoopts = NULL;
 +    QCryptoBlockAmendOptions *amend_options = NULL;
 +    int ret;
 +
 +    assert(crypto);
 +    assert(crypto->block);
 +    crypto->updating_keys = true;
 +
 +    ret = bdrv_child_refresh_perms(bs, bs->file, errp);
 +    if (ret < 0) {
 +        goto cleanup;
 +    }
 +
 +    cryptoopts = qemu_opts_to_qdict(opts, NULL);
 +    qdict_put_str(cryptoopts, "format", "luks");
 +    amend_options = block_crypto_amend_opts_init(cryptoopts, errp);
 +    if (!amend_options) {
 +        ret = -EINVAL;
 +        goto cleanup;
 +    }
 +
 +    ret = qcrypto_block_amend_options(crypto->block,
 +                                      block_crypto_read_func,
 +                                      block_crypto_write_func,
 +                                      bs,
 +                                      amend_options,
 +                                      force,
 +                                      errp);
 +cleanup:
 +    crypto->updating_keys = false;
 +    bdrv_child_refresh_perms(bs, bs->file, errp);
 +    qapi_free_QCryptoBlockAmendOptions(amend_options);
 +    qobject_unref(cryptoopts);
 +    return ret;
 +}
 +
 +
-+static void
+ def supports_quorum():
-+block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
+     return 'quorum' in qemu_img_pipe('--help')
-+                         const BdrvChildRole role,
-+                         BlockReopenQueue *reopen_queue,
+@@ -XXX,XX +XXX,XX @@ def execute_setup_common(supported_fmts: Sequence[str] = (),
-+                         uint64_t perm, uint64_t shared,
+                          unsupported_fmts: Sequence[str] = (),
-+                         uint64_t *nperm, uint64_t *nshared)
+                          supported_protocols: Sequence[str] = (),
-+{
+                          unsupported_protocols: Sequence[str] = (),
-+
+-                         required_fmts: Sequence[str] = ()) -> bool:
-+    BlockCrypto *crypto = bs->opaque;
++                         required_fmts: Sequence[str] = (),
-+
++                         unsupported_imgopts: Sequence[str] = ()) -> bool:
-+    bdrv_default_perms(bs, c, role, reopen_queue, perm, shared, nperm, nshared);
+     """
-+
+     Perform necessary setup for either script-style or unittest-style tests.
-+    /*
-+     * For backward compatibility, manually share the write
+@@ -XXX,XX +XXX,XX @@ def execute_setup_common(supported_fmts: Sequence[str] = (),
-+     * and resize permission
+     _verify_aio_mode(supported_aio_modes)
-+     */
+     _verify_formats(required_fmts)
-+    *nshared |= (BLK_PERM_WRITE | BLK_PERM_RESIZE);
+     _verify_virtio_blk()
-+    /*
++    _verify_imgopts(unsupported_imgopts)
-+     * Since we are not fully a format driver, don't always request
-+     * the read/resize permission but only when explicitly
+     return debug
 +     * requested
 +     */
 +    *nperm &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
 +    *nperm |= perm & (BLK_PERM_WRITE | BLK_PERM_RESIZE);
 +
 +    /*
 +     * This driver doesn't modify LUKS metadata except
 +     * when updating the encryption slots.
 +     * Thus unlike a proper format driver we don't ask for
 +     * shared write/read permission. However we need it
 +     * when we are updating the keys, to ensure that only we
 +     * have access to the device.
 +     *
 +     * Encryption update will set the crypto->updating_keys
 +     * during that period and refresh permissions
 +     *
 +     */
 +    if (crypto->updating_keys) {
 +        /* need exclusive write access for header update */
 +        *nperm |= BLK_PERM_WRITE;
 +        /* unshare read and write permission */
 +        *nshared &= ~(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE);
 +    }
 +}
 +
 +
  static const char *const block_crypto_strong_runtime_opts[] = {
      BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET,
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_crypto_luks = {
      .bdrv_probe         = block_crypto_probe_luks,
      .bdrv_open          = block_crypto_open_luks,
      .bdrv_close         = block_crypto_close,
 -    /* This driver doesn't modify LUKS metadata except when creating image.
 -     * Allow share-rw=on as a special case. */
 -    .bdrv_child_perm    = bdrv_default_perms,
 +    .bdrv_child_perm    = block_crypto_child_perms,
      .bdrv_co_create     = block_crypto_co_create_luks,
      .bdrv_co_create_opts = block_crypto_co_create_opts_luks,
      .bdrv_co_truncate   = block_crypto_co_truncate,
      .create_opts        = &block_crypto_create_opts_luks,
 +    .amend_opts         = &block_crypto_amend_opts_luks,
      .bdrv_reopen_prepare = block_crypto_reopen_prepare,
      .bdrv_refresh_limits = block_crypto_refresh_limits,
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_crypto_luks = {
      .bdrv_measure       = block_crypto_measure,
      .bdrv_get_info      = block_crypto_get_info_luks,
      .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
 +    .bdrv_amend_options = block_crypto_amend_options_luks,
      .is_format          = true,
 --
-.26.2
+.34.1

-New patch
+[PULL 05/24] iotests: specify some unsupported_imgopts for python iotests
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 We are going to support IMGOPTS for python iotests. Still some iotests
 will not work with common IMGOPTS used with bash iotests like
 specifying refcount_bits and compat qcow2 options. So we
 should define corresponding unsupported_imgopts for now.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Message-Id: <20211223160144.1097696-4-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/044                                 | 3 ++-
  tests/qemu-iotests/065                                 | 3 ++-
  tests/qemu-iotests/163                                 | 3 ++-
  tests/qemu-iotests/165                                 | 3 ++-
  tests/qemu-iotests/196                                 | 3 ++-
  tests/qemu-iotests/242                                 | 3 ++-
  tests/qemu-iotests/246                                 | 3 ++-
  tests/qemu-iotests/254                                 | 3 ++-
  tests/qemu-iotests/260                                 | 3 ++-
  tests/qemu-iotests/274                                 | 3 ++-
  tests/qemu-iotests/281                                 | 3 ++-
  tests/qemu-iotests/303                                 | 3 ++-
  tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 ++-
  tests/qemu-iotests/tests/migrate-bitmaps-test          | 3 ++-
  tests/qemu-iotests/tests/remove-bitmap-from-backing    | 3 ++-
 files changed, 30 insertions(+), 15 deletions(-)
 diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/044
 +++ b/tests/qemu-iotests/044
@@ -XXX,XX +XXX,XX @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
  if __name__ == '__main__':
      iotests.main(supported_fmts=['qcow2'],
 -                 supported_protocols=['file'])
 +                 supported_protocols=['file'],
 +                 unsupported_imgopts=['refcount_bits'])
 diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/065
 +++ b/tests/qemu-iotests/065
@@ -XXX,XX +XXX,XX @@ TestQMP = None
  if __name__ == '__main__':
      iotests.main(supported_fmts=['qcow2'],
 -                 supported_protocols=['file'])
 +                 supported_protocols=['file'],
 +                 unsupported_imgopts=['refcount_bits'])
 diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/163
 +++ b/tests/qemu-iotests/163
@@ -XXX,XX +XXX,XX @@ ShrinkBaseClass = None
  if __name__ == '__main__':
      iotests.main(supported_fmts=['raw', 'qcow2'],
 -                 supported_protocols=['file'])
 +                 supported_protocols=['file'],
 +                 unsupported_imgopts=['compat'])
 diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/165
 +++ b/tests/qemu-iotests/165
@@ -XXX,XX +XXX,XX @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
  if __name__ == '__main__':
      iotests.main(supported_fmts=['qcow2'],
 -                 supported_protocols=['file'])
 +                 supported_protocols=['file'],
 +                 unsupported_imgopts=['compat'])
 diff --git a/tests/qemu-iotests/196 b/tests/qemu-iotests/196
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/196
 +++ b/tests/qemu-iotests/196
@@ -XXX,XX +XXX,XX @@ class TestInvalidateAutoclear(iotests.QMPTestCase):
  if __name__ == '__main__':
      iotests.main(supported_fmts=['qcow2'],
 -                 supported_protocols=['file'])
 +                 supported_protocols=['file'],
 +                 unsupported_imgopts=['compat'])
 diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/242
 +++ b/tests/qemu-iotests/242
@@ -XXX,XX +XXX,XX @@ from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \
      file_path, img_info_log, log, filter_qemu_io
  iotests.script_initialize(supported_fmts=['qcow2'],
 -                          supported_protocols=['file'])
 +                          supported_protocols=['file'],
 +                          unsupported_imgopts=['refcount_bits', 'compat'])
  disk = file_path('disk')
  chunk = 256 * 1024
 diff --git a/tests/qemu-iotests/246 b/tests/qemu-iotests/246
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/246
 +++ b/tests/qemu-iotests/246
@@ -XXX,XX +XXX,XX @@
  import iotests
  from iotests import log
 -iotests.script_initialize(supported_fmts=['qcow2'])
 +iotests.script_initialize(supported_fmts=['qcow2'],
 +                          unsupported_imgopts=['compat'])
  size = 64 * 1024 * 1024 * 1024
  gran_small = 32 * 1024
  gran_large = 128 * 1024
 diff --git a/tests/qemu-iotests/254 b/tests/qemu-iotests/254
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/254
 +++ b/tests/qemu-iotests/254
@@ -XXX,XX +XXX,XX @@
  import iotests
  from iotests import qemu_img_create, file_path, log
 -iotests.script_initialize(supported_fmts=['qcow2'])
 +iotests.script_initialize(supported_fmts=['qcow2'],
 +                          unsupported_imgopts=['compat'])
  disk, top = file_path('disk', 'top')
  size = 1024 * 1024
 diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/260
 +++ b/tests/qemu-iotests/260
@@ -XXX,XX +XXX,XX @@ import iotests
  from iotests import qemu_img_create, file_path, log, filter_qmp_event
  iotests.script_initialize(
 -    supported_fmts=['qcow2']
 +    supported_fmts=['qcow2'],
 +    unsupported_imgopts=['compat']
  )
  base, top = file_path('base', 'top')
 diff --git a/tests/qemu-iotests/274 b/tests/qemu-iotests/274
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/274
 +++ b/tests/qemu-iotests/274
@@ -XXX,XX +XXX,XX @@
  import iotests
  iotests.script_initialize(supported_fmts=['qcow2'],
 -                          supported_platforms=['linux'])
 +                          supported_platforms=['linux'],
 +                          unsupported_imgopts=['refcount_bits', 'compat'])
  size_short = 1 * 1024 * 1024
  size_long = 2 * 1024 * 1024
 diff --git a/tests/qemu-iotests/281 b/tests/qemu-iotests/281
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/281
 +++ b/tests/qemu-iotests/281
@@ -XXX,XX +XXX,XX @@ class TestBlockdevBackupAbort(iotests.QMPTestCase):
  if __name__ == '__main__':
      iotests.main(supported_fmts=['qcow2'],
 -                 supported_protocols=['file'])
 +                 supported_protocols=['file'],
 +                 unsupported_imgopts=['compat'])
 diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/303
 +++ b/tests/qemu-iotests/303
@@ -XXX,XX +XXX,XX @@ import iotests
  import subprocess
  from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_io
 -iotests.script_initialize(supported_fmts=['qcow2'])
 +iotests.script_initialize(supported_fmts=['qcow2'],
 +                          unsupported_imgopts=['refcount_bits', 'compat'])
  disk = file_path('disk')
  chunk = 1024 * 1024
 diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
 +++ b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
@@ -XXX,XX +XXX,XX @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
  if __name__ == '__main__':
 -    iotests.main(supported_fmts=['qcow2'])
 +    iotests.main(supported_fmts=['qcow2'],
 +                 unsupported_imgopts=['compat'])
 diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-test b/tests/qemu-iotests/tests/migrate-bitmaps-test
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/tests/migrate-bitmaps-test
 +++ b/tests/qemu-iotests/tests/migrate-bitmaps-test
@@ -XXX,XX +XXX,XX @@ def main() -> None:
      iotests.main(
          supported_fmts=['qcow2'],
 -        supported_protocols=['file']
 +        supported_protocols=['file'],
 +        unsupported_imgopts=['compat']
      )
 diff --git a/tests/qemu-iotests/tests/remove-bitmap-from-backing b/tests/qemu-iotests/tests/remove-bitmap-from-backing
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/tests/remove-bitmap-from-backing
 +++ b/tests/qemu-iotests/tests/remove-bitmap-from-backing
@@ -XXX,XX +XXX,XX @@
  import iotests
  from iotests import log, qemu_img_create, qemu_img, qemu_img_pipe
 -iotests.script_initialize(supported_fmts=['qcow2'])
 +iotests.script_initialize(supported_fmts=['qcow2'],
 +                          unsupported_imgopts=['compat'])
  top, base = iotests.file_path('top', 'base')
  size = '1M'
 --
 .34.1

-New patch
+[PULL 06/24] iotests.py: qemu_img*("create"): support IMGOPTS='compression_type=zstd'
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Adding support of IMGOPTS (like in bash tests) allows user to pass a
+lot of different options. Still, some may require additional logic.
+Now we want compression_type option, so add some smart logic around it:
+ignore compression_type=zstd in IMGOPTS, if test want qcow2 in
+compatibility mode. As well, ignore compression_type for non-qcow2
+formats.
+Note that we may instead add support only to qemu_img_create(), but
+that works bad:
+. We'll have to update a lot of tests to use qemu_img_create instead
+   of qemu_img('create'). (still, we may want do it anyway, but no
+   reason to create a dependancy between task of supporting IMGOPTS and
+   updating a lot of tests)
+. Some tests use qemu_img_pipe('create', ..) - even more work on
+   updating
+. Even if we update all tests to go through qemu_img_create, we'll
+   need a way to avoid creating new tests using qemu_img*('create') -
+   add assertions.. That doesn't seem good.
+So, let's add support of IMGOPTS to most generic
+qemu_img_pipe_and_status().
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
+Message-Id: <20211223160144.1097696-5-vsementsov@virtuozzo.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+---
+ tests/qemu-iotests/iotests.py | 27 ++++++++++++++++++++++++++-
+file changed, 26 insertions(+), 1 deletion(-)
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/iotests.py
++++ b/tests/qemu-iotests/iotests.py
+@@ -XXX,XX +XXX,XX @@
+ # along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ #
++import argparse
+ import atexit
+ import bz2
+ from collections import OrderedDict
+@@ -XXX,XX +XXX,XX @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
+                                {-subp.returncode}: {cmd}\n')
+         return (output, subp.returncode)
++def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
++    if not args or args[0] != 'create':
++        return list(args)
++    args = args[1:]
++
++    p = argparse.ArgumentParser(allow_abbrev=False)
++    p.add_argument('-f')
++    parsed, remaining = p.parse_known_args(args)
++
++    result = ['create']
++    if parsed.f is not None:
++        result += ['-f', parsed.f]
++
++    # IMGOPTS most probably contain options specific for the selected format,
++    # like extended_l2 or compression_type for qcow2. Test may want to create
++    # additional images in other formats that doesn't support these options.
++    # So, use IMGOPTS only for images created in imgfmt format.
++    if parsed.f == imgfmt and 'IMGOPTS' in os.environ:
++        result += ['-o', os.environ['IMGOPTS']]
++
++    result += remaining
++
++    return result
++
+ def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]:
+     """
+     Run qemu-img and return both its output and its exit code
+     """
+-    full_args = qemu_img_args + list(args)
++    full_args = qemu_img_args + qemu_img_create_prepare_args(list(args))
+     return qemu_tool_pipe_and_status('qemu-img', full_args)
+ def qemu_img(*args: str) -> int:
+--
+.34.1

-[PULL 13/18] block/core: add generic infrastructure for x-blockdev-amend qmp command
+[PULL 07/24] iotests: drop qemu_img_verbose() helper
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-blockdev-amend will be used similiar to blockdev-create
+qemu_img_verbose() has a drawback of not going through generic
-to allow on the fly changes of the structure of the format based block devices.
+qemu_img_pipe_and_status(). qemu_img_verbose() is not very popular, so
 update the only two users to qemu_img_log() and drop qemu_img_verbose()
 at all.
-Current plan is to first support encryption keyslot management for luks
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-based formats (raw and embedded in qcow2)
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20211223160144.1097696-6-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/044        | 5 +++--
  tests/qemu-iotests/044.out    | 1 +
  tests/qemu-iotests/209        | 7 ++++---
  tests/qemu-iotests/209.out    | 2 ++
  tests/qemu-iotests/iotests.py | 8 --------
 files changed, 10 insertions(+), 13 deletions(-)
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+index XXXXXXX..XXXXXXX 100755
-Message-Id: <20200608094030.670121-12-mlevitsk@redhat.com>
+--- a/tests/qemu-iotests/044
-Signed-off-by: Max Reitz <mreitz@redhat.com>
++++ b/tests/qemu-iotests/044
----
+@@ -XXX,XX +XXX,XX @@ import os
- qapi/block-core.json      |  42 ++++++++++++++
+ import qcow2
- qapi/job.json             |   4 +-
+ from qcow2 import QcowHeader
- include/block/block_int.h |  21 +++++--
+ import iotests
- block/amend.c             | 113 ++++++++++++++++++++++++++++++++++++++
+-from iotests import qemu_img, qemu_img_verbose, qemu_io
- block/Makefile.objs       |   2 +-
++from iotests import qemu_img, qemu_img_log, qemu_io
-files changed, 174 insertions(+), 8 deletions(-)
+ import struct
- create mode 100644 block/amend.c
+ import subprocess
+ import sys
-diff --git a/qapi/block-core.json b/qapi/block-core.json
+@@ -XXX,XX +XXX,XX @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
      def test_grow_refcount_table(self):
          qemu_io('-c', 'write 3800M 1M', test_img)
 -        qemu_img_verbose('check' , test_img)
 +        qemu_img_log('check' , test_img)
          pass
  if __name__ == '__main__':
 +    iotests.activate_logging()
      iotests.main(supported_fmts=['qcow2'],
                   supported_protocols=['file'],
                   unsupported_imgopts=['refcount_bits'])
 diff --git a/tests/qemu-iotests/044.out b/tests/qemu-iotests/044.out
 index XXXXXXX..XXXXXXX 100644
---- a/qapi/block-core.json
+--- a/tests/qemu-iotests/044.out
-+++ b/qapi/block-core.json
++++ b/tests/qemu-iotests/044.out
 @@ -XXX,XX +XXX,XX @@
-   'data': { 'job-id': 'str',
+ No errors were found on the image.
-             'options': 'BlockdevCreateOptions' } }
+ 7292415/33554432 = 21.73% allocated, 0.00% fragmented, 0.00% compressed clusters
+ Image end offset: 4296217088
 +##
 +# @BlockdevAmendOptions:
 +#
 +# Options for amending an image format
 +#
 +# @driver:          Block driver of the node to amend.
 +#
 +# Since: 5.1
 +##
 +{ 'union': 'BlockdevAmendOptions',
 +  'base': {
 +      'driver':         'BlockdevDriver' },
 +  'discriminator': 'driver',
 +  'data': {
 +  } }
 +
-+##
+ .
-+# @x-blockdev-amend:
+ ----------------------------------------------------------------------
-+#
+ Ran 1 tests
-+# Starts a job to amend format specific options of an existing open block device
+diff --git a/tests/qemu-iotests/209 b/tests/qemu-iotests/209
-+# The job is automatically finalized, but a manual job-dismiss is required.
+index XXXXXXX..XXXXXXX 100755
-+#
+--- a/tests/qemu-iotests/209
-+# @job-id:          Identifier for the newly created job.
++++ b/tests/qemu-iotests/209
 +#
 +# @node-name:       Name of the block node to work on
 +#
 +# @options:         Options (driver specific)
 +#
 +# @force:           Allow unsafe operations, format specific
 +#                   For luks that allows erase of the last active keyslot
 +#                   (permanent loss of data),
 +#                   and replacement of an active keyslot
 +#                   (possible loss of data if IO error happens)
 +#
 +# Since: 5.1
 +##
 +{ 'command': 'x-blockdev-amend',
 +  'data': { 'job-id': 'str',
 +            'node-name': 'str',
 +            'options': 'BlockdevAmendOptions',
 +            '*force': 'bool' } }
 +
  ##
  # @BlockErrorAction:
  #
 diff --git a/qapi/job.json b/qapi/job.json
 index XXXXXXX..XXXXXXX 100644
 --- a/qapi/job.json
 +++ b/qapi/job.json
 @@ -XXX,XX +XXX,XX @@
  #
- # @create: image creation job type, see "blockdev-create" (since 3.0)
- #
+ import iotests
-+# @amend: image options amend job type, see "x-blockdev-amend" (since 5.1)
+-from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \
-+#
+-                    file_path
- # Since: 1.7
++from iotests import qemu_img_create, qemu_io, qemu_img_log, qemu_nbd, \
- ##
++                    file_path, log
- { 'enum': 'JobType',
--  'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+ iotests.script_initialize(supported_fmts=['qcow2'])
-+  'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'amend'] }
+@@ -XXX,XX +XXX,XX @@ qemu_img_create('-f', iotests.imgfmt, disk, '1M')
- ##
+ qemu_io('-f', iotests.imgfmt, '-c', 'write 0 512K', disk)
- # @JobStatus:
-diff --git a/include/block/block_int.h b/include/block/block_int.h
+ qemu_nbd('-k', nbd_sock, '-x', 'exp', '-f', iotests.imgfmt, disk)
 -qemu_img_verbose('map', '-f', 'raw', '--output=json', nbd_uri)
 +qemu_img_log('map', '-f', 'raw', '--output=json', nbd_uri)
 +log('done.')  # avoid new line at the end of output file
 diff --git a/tests/qemu-iotests/209.out b/tests/qemu-iotests/209.out
 index XXXXXXX..XXXXXXX 100644
---- a/include/block/block_int.h
+--- a/tests/qemu-iotests/209.out
-+++ b/include/block/block_int.h
++++ b/tests/qemu-iotests/209.out
-@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
+@@ -XXX,XX +XXX,XX @@
-     int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
+ [{ "start": 0, "length": 524288, "depth": 0, "present": true, "zero": false, "data": true, "offset": 0},
-                           Error **errp);
+ { "start": 524288, "length": 524288, "depth": 0, "present": true, "zero": true, "data": false, "offset": 524288}]
      void (*bdrv_close)(BlockDriverState *bs);
 +
-+
++done.
-     int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
-                                        Error **errp);
+index XXXXXXX..XXXXXXX 100644
-     int coroutine_fn (*bdrv_co_create_opts)(BlockDriver *drv,
+--- a/tests/qemu-iotests/iotests.py
-                                             const char *filename,
++++ b/tests/qemu-iotests/iotests.py
-                                             QemuOpts *opts,
+@@ -XXX,XX +XXX,XX @@ def qemu_img_measure(*args):
-                                             Error **errp);
+ def qemu_img_check(*args):
-+
+     return json.loads(qemu_img_pipe("check", "--output", "json", *args))
-+    int coroutine_fn (*bdrv_co_amend)(BlockDriverState *bs,
-+                                      BlockdevAmendOptions *opts,
+-def qemu_img_verbose(*args):
-+                                      bool force,
+-    '''Run qemu-img without suppressing its output and return the exit code'''
-+                                      Error **errp);
+-    exitcode = subprocess.call(qemu_img_args + list(args))
-+
+-    if exitcode < 0:
-+    int (*bdrv_amend_options)(BlockDriverState *bs,
+-        sys.stderr.write('qemu-img received signal %i: %s\n'
-+                              QemuOpts *opts,
+-                         % (-exitcode, ' '.join(qemu_img_args + list(args))))
-+                              BlockDriverAmendStatusCB *status_cb,
+-    return exitcode
 +                              void *cb_opaque,
 +                              bool force,
 +                              Error **errp);
 +
      int (*bdrv_make_empty)(BlockDriverState *bs);
      /*
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
                                        BdrvCheckResult *result,
                                        BdrvCheckMode fix);
 -    int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
 -                              BlockDriverAmendStatusCB *status_cb,
 -                              void *cb_opaque,
 -                              bool force,
 -                              Error **errp);
 -
-     void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
+ def qemu_img_pipe(*args: str) -> str:
+     '''Run qemu-img and return its output'''
-     /* TODO Better pass a option string/QDict/QemuOpts to add any rule? */
+     return qemu_img_pipe_and_status(*args)[0]
 diff --git a/block/amend.c b/block/amend.c
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/block/amend.c
@@ -XXX,XX +XXX,XX @@
 +/*
 + * Block layer code related to image options amend
 + *
 + * Copyright (c) 2018 Kevin Wolf <kwolf@redhat.com>
 + * Copyright (c) 2020 Red Hat. Inc
 + *
 + * Heavily based on create.c
 + *
 + * Permission is hereby granted, free of charge, to any person obtaining a copy
 + * of this software and associated documentation files (the "Software"), to deal
 + * in the Software without restriction, including without limitation the rights
 + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 + * copies of the Software, and to permit persons to whom the Software is
 + * furnished to do so, subject to the following conditions:
 + *
 + * The above copyright notice and this permission notice shall be included in
 + * all copies or substantial portions of the Software.
 + *
 + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
 + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 + * THE SOFTWARE.
 + */
 +
 +#include "qemu/osdep.h"
 +#include "block/block_int.h"
 +#include "qemu/job.h"
 +#include "qemu/main-loop.h"
 +#include "qapi/qapi-commands-block-core.h"
 +#include "qapi/qapi-visit-block-core.h"
 +#include "qapi/clone-visitor.h"
 +#include "qapi/error.h"
 +
 +typedef struct BlockdevAmendJob {
 +    Job common;
 +    BlockdevAmendOptions *opts;
 +    BlockDriverState *bs;
 +    bool force;
 +} BlockdevAmendJob;
 +
 +static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
 +{
 +    BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
 +    int ret;
 +
 +    job_progress_set_remaining(&s->common, 1);
 +    ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
 +    job_progress_update(&s->common, 1);
 +    qapi_free_BlockdevAmendOptions(s->opts);
 +    return ret;
 +}
 +
 +static const JobDriver blockdev_amend_job_driver = {
 +    .instance_size = sizeof(BlockdevAmendJob),
 +    .job_type      = JOB_TYPE_AMEND,
 +    .run           = blockdev_amend_run,
 +};
 +
 +void qmp_x_blockdev_amend(const char *job_id,
 +                          const char *node_name,
 +                          BlockdevAmendOptions *options,
 +                          bool has_force,
 +                          bool force,
 +                          Error **errp)
 +{
 +    BlockdevAmendJob *s;
 +    const char *fmt = BlockdevDriver_str(options->driver);
 +    BlockDriver *drv = bdrv_find_format(fmt);
 +    BlockDriverState *bs = bdrv_find_node(node_name);
 +
 +
 +    if (!drv) {
 +        error_setg(errp, "Block driver '%s' not found or not supported", fmt);
 +        return;
 +    }
 +
 +    /*
 +     * If the driver is in the schema, we know that it exists. But it may not
 +     * be whitelisted.
 +     */
 +    if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
 +        error_setg(errp, "Driver is not whitelisted");
 +        return;
 +    }
 +
 +    if (bs->drv != drv) {
 +        error_setg(errp,
 +                   "x-blockdev-amend doesn't support changing the block driver");
 +        return;
 +    }
 +
 +    /* Error out if the driver doesn't support .bdrv_co_amend */
 +    if (!drv->bdrv_co_amend) {
 +        error_setg(errp, "Driver does not support x-blockdev-amend");
 +        return;
 +    }
 +
 +    /* Create the block job */
 +    s = job_create(job_id, &blockdev_amend_job_driver, NULL,
 +                   bdrv_get_aio_context(bs), JOB_DEFAULT | JOB_MANUAL_DISMISS,
 +                   NULL, NULL, errp);
 +    if (!s) {
 +        return;
 +    }
 +
 +    s->bs = bs,
 +    s->opts = QAPI_CLONE(BlockdevAmendOptions, options),
 +    s->force = has_force ? force : false;
 +    job_start(&s->common);
 +}
 diff --git a/block/Makefile.objs b/block/Makefile.objs
 index XXXXXXX..XXXXXXX 100644
 --- a/block/Makefile.objs
 +++ b/block/Makefile.objs
@@ -XXX,XX +XXX,XX @@ block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
  block-obj-$(CONFIG_POSIX) += file-posix.o
  block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
  block-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o
 -block-obj-y += null.o mirror.o commit.o io.o create.o
 +block-obj-y += null.o mirror.o commit.o io.o create.o amend.o
  block-obj-y += throttle-groups.o
  block-obj-$(CONFIG_LINUX) += nvme.o
 --
-.26.2
+.34.1

-[PULL 17/18] iotests: Fix 051 output after qdev_init_nofail() removal
+[PULL 08/24] iotests.py: rewrite default luks support in qemu_img
-From: Philippe Mathieu-Daudé <philmd@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Commit 96927c744 replaced qdev_init_nofail() call by
+Move the logic to more generic qemu_img_pipe_and_status(). Also behave
-isa_realize_and_unref() which has a different error
+better when we have several -o options. And reuse argument parser of
-message. Update the test output accordingly.
+course.
-Gitlab CI error after merging b77b5b3dc7:
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-https://gitlab.com/qemu-project/qemu/-/jobs/597414772#L4375
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20211223160144.1097696-7-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/iotests.py | 36 +++++++++++++++++------------------
 file changed, 17 insertions(+), 19 deletions(-)
-Reported-by: Thomas Huth <thuth@redhat.com>
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
 Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
 Message-Id: <20200616154949.6586-1-philmd@redhat.com>
 Reviewed-by: Thomas Huth <thuth@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
  tests/qemu-iotests/051.pc.out | 4 ++--
 file changed, 2 insertions(+), 2 deletions(-)
 diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
 index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/051.pc.out
+--- a/tests/qemu-iotests/iotests.py
-+++ b/tests/qemu-iotests/051.pc.out
++++ b/tests/qemu-iotests/iotests.py
-@@ -XXX,XX +XXX,XX @@ QEMU X.Y.Z monitor - type 'help' for more information
+@@ -XXX,XX +XXX,XX @@ def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
+     args = args[1:]
- Testing: -drive if=ide
- QEMU X.Y.Z monitor - type 'help' for more information
+     p = argparse.ArgumentParser(allow_abbrev=False)
--(qemu) QEMU_PROG: Initialization of device ide-hd failed: Device needs media, but drive is empty
++    # -o option may be specified several times
-+(qemu) QEMU_PROG: Device needs media, but drive is empty
++    p.add_argument('-o', action='append', default=[])
+     p.add_argument('-f')
- Testing: -drive if=virtio
+     parsed, remaining = p.parse_known_args(args)
- QEMU X.Y.Z monitor - type 'help' for more information
-@@ -XXX,XX +XXX,XX @@ QEMU X.Y.Z monitor - type 'help' for more information
++    opts_list = parsed.o
++
- Testing: -drive file=TEST_DIR/t.qcow2,if=ide,readonly=on
+     result = ['create']
- QEMU X.Y.Z monitor - type 'help' for more information
+     if parsed.f is not None:
--(qemu) QEMU_PROG: Initialization of device ide-hd failed: Block node is read-only
+         result += ['-f', parsed.f]
-+(qemu) QEMU_PROG: Block node is read-only
+@@ -XXX,XX +XXX,XX @@ def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
+     # like extended_l2 or compression_type for qcow2. Test may want to create
- Testing: -drive file=TEST_DIR/t.qcow2,if=virtio,readonly=on
+     # additional images in other formats that doesn't support these options.
- QEMU X.Y.Z monitor - type 'help' for more information
+     # So, use IMGOPTS only for images created in imgfmt format.
 -    if parsed.f == imgfmt and 'IMGOPTS' in os.environ:
 -        result += ['-o', os.environ['IMGOPTS']]
 +    imgopts = os.environ.get('IMGOPTS')
 +    if imgopts and parsed.f == imgfmt:
 +        opts_list.insert(0, imgopts)
 +
 +    # default luks support
 +    if parsed.f == 'luks' and \
 +            all('key-secret' not in opts for opts in opts_list):
 +        result += ['--object', luks_default_secret_object]
 +        opts_list.append(luks_default_key_secret_opt)
 +
 +    for opts in opts_list:
 +        result += ['-o', opts]
      result += remaining
@@ -XXX,XX +XXX,XX @@ def ordered_qmp(qmsg, conv_keys=True):
      return qmsg
  def qemu_img_create(*args):
 -    args = list(args)
 -
 -    # default luks support
 -    if '-f' in args and args[args.index('-f') + 1] == 'luks':
 -        if '-o' in args:
 -            i = args.index('-o')
 -            if 'key-secret' not in args[i + 1]:
 -                args[i + 1].append(luks_default_key_secret_opt)
 -                args.insert(i + 2, '--object')
 -                args.insert(i + 3, luks_default_secret_object)
 -        else:
 -            args = ['-o', luks_default_key_secret_opt,
 -                    '--object', luks_default_secret_object] + args
 -
 -    args.insert(0, 'create')
 -
 -    return qemu_img(*args)
 +    return qemu_img('create', *args)
  def qemu_img_measure(*args):
      return json.loads(qemu_img_pipe("measure", "--output", "json", *args))
 --
-.26.2
+.34.1

-[PULL 12/18] iotests: qemu-img tests for luks key management
+[PULL 09/24] iotest 303: explicit compression type
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-This commit adds two tests, which test the new amend interface
+The test prints qcow2 header fields which depends on chosen compression
-of both luks raw images and qcow2 luks encrypted images.
+type. So, let's be explicit in what compression type we want and
 independent of IMGOPTS. Test both existing compression types.
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+Reviewed-by: Max Reitz <mreitz@redhat.com>
-Message-Id: <20200608094030.670121-11-mlevitsk@redhat.com>
+Message-Id: <20211223160144.1097696-8-vsementsov@virtuozzo.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- tests/qemu-iotests/293     | 207 +++++++++++++++++++++++++++++++++++++
+ tests/qemu-iotests/303     | 25 ++++++++++++++++---------
- tests/qemu-iotests/293.out |  99 ++++++++++++++++++
+ tests/qemu-iotests/303.out | 30 +++++++++++++++++++++++++++++-
- tests/qemu-iotests/294     |  90 ++++++++++++++++
+files changed, 45 insertions(+), 10 deletions(-)
  tests/qemu-iotests/294.out |  30 ++++++
  tests/qemu-iotests/group   |   2 +
 files changed, 428 insertions(+)
  create mode 100755 tests/qemu-iotests/293
  create mode 100644 tests/qemu-iotests/293.out
  create mode 100755 tests/qemu-iotests/294
  create mode 100644 tests/qemu-iotests/294.out
-diff --git a/tests/qemu-iotests/293 b/tests/qemu-iotests/293
+diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
-new file mode 100755
+index XXXXXXX..XXXXXXX 100755
-index XXXXXXX..XXXXXXX
+--- a/tests/qemu-iotests/303
---- /dev/null
++++ b/tests/qemu-iotests/303
-+++ b/tests/qemu-iotests/293
+@@ -XXX,XX +XXX,XX @@ def add_bitmap(num, begin, end, disabled):
-@@ -XXX,XX +XXX,XX @@
+     log('')
-+#!/usr/bin/env bash
-+#
-+# Test encryption key management with luks
+-qemu_img_create('-f', iotests.imgfmt, disk, '10M')
-+# Based on 134
+-
-+#
+-add_bitmap(1, 0, 6, False)
-+# Copyright (C) 2019 Red Hat, Inc.
+-add_bitmap(2, 6, 8, True)
-+#
+-dump = ['./qcow2.py', disk, 'dump-header']
-+# This program is free software; you can redistribute it and/or modify
+-subprocess.run(dump)
-+# it under the terms of the GNU General Public License as published by
+-# Dump the metadata in JSON format
-+# the Free Software Foundation; either version 2 of the License, or
+-dump.append('-j')
-+# (at your option) any later version.
+-subprocess.run(dump)
-+#
++def test(compression_type: str, json_output: bool) -> None:
-+# This program is distributed in the hope that it will be useful,
++    qemu_img_create('-f', iotests.imgfmt,
-+# but WITHOUT ANY WARRANTY; without even the implied warranty of
++                    '-o', f'compression_type={compression_type}',
-+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
++                    disk, '10M')
-+# GNU General Public License for more details.
++    add_bitmap(1, 0, 6, False)
-+#
++    add_bitmap(2, 6, 8, True)
 +# You should have received a copy of the GNU General Public License
 +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 +#
 +
-+# creator
++    cmd = ['./qcow2.py', disk, 'dump-header']
-+owner=mlevitsk@redhat.com
++    if json_output:
 +        cmd.append('-j')
 +
-+seq=`basename $0`
++    subprocess.run(cmd)
 +echo "QA output created by $seq"
 +
 +status=1    # failure is the default!
 +
 +_cleanup()
 +{
 +    _cleanup_test_img
 +}
 +trap "_cleanup; exit \$status" 0 1 2 3 15
 +
 +# get standard environment, filters and checks
 +. ./common.rc
 +. ./common.filter
 +
 +_supported_fmt qcow2 luks
 +_supported_proto file #TODO
 +
 +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
 +
 +if [ "$IMGFMT" = "qcow2" ] ; then
 +    PR="encrypt."
 +    EXTRA_IMG_ARGS="-o encrypt.format=luks"
 +fi
 +
 +
-+# secrets: you are supposed to see the password as *******, see :-)
++test('zlib', False)
-+S0="--object secret,id=sec0,data=hunter0"
++test('zstd', True)
-+S1="--object secret,id=sec1,data=hunter1"
+diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out
-+S2="--object secret,id=sec2,data=hunter2"
+index XXXXXXX..XXXXXXX 100644
-+S3="--object secret,id=sec3,data=hunter3"
+--- a/tests/qemu-iotests/303.out
-+S4="--object secret,id=sec4,data=hunter4"
++++ b/tests/qemu-iotests/303.out
-+SECRETS="$S0 $S1 $S2 $S3 $S4"
+@@ -XXX,XX +XXX,XX @@ extra_data_size           0
  Bitmap table   type            size         offset
 all-zeroes      0            0
 +Add bitmap 1
 +wrote 1048576/1048576 bytes at offset 0
 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
-+# image with given secret
++wrote 1048576/1048576 bytes at offset 1048576
-+IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec0"
++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-+IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec1"
++
-+IMGS2="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec2"
++wrote 1048576/1048576 bytes at offset 2097152
-+IMGS3="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec3"
++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-+IMGS4="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec4"
++
 +wrote 1048576/1048576 bytes at offset 3145728
 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +wrote 1048576/1048576 bytes at offset 4194304
 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +wrote 1048576/1048576 bytes at offset 5242880
 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +
-+echo "== creating a test image =="
++Add bitmap 2
-+_make_test_img $S0 $EXTRA_IMG_ARGS -o ${PR}key-secret=sec0,${PR}iter-time=10 32M
++wrote 1048576/1048576 bytes at offset 6291456
 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
-+echo
++wrote 1048576/1048576 bytes at offset 7340032
-+echo "== test that key 0 opens the image =="
++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
 +
 +echo
 +echo "== adding a password to slot 4 =="
 +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec4,${PR}iter-time=10,${PR}keyslot=4
 +echo "== adding a password to slot 1 =="
 +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10
 +echo "== adding a password to slot 3 =="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10,${PR}keyslot=3
 +
 +echo "== adding a password to slot 2 =="
 +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10
 +
 +
-+echo "== erase slot 4 =="
+ {
-+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=4 | _filter_img_create
+     "magic": 1363560955,
-+
+     "version": 3,
-+
+@@ -XXX,XX +XXX,XX @@ Bitmap table   type            size         offset
-+echo
+     "refcount_table_clusters": 1,
-+echo "== all secrets should work =="
+     "nb_snapshots": 0,
-+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+     "snapshot_offset": 0,
-+    $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+-    "incompatible_features": 0,
-+done
++    "incompatible_features": 8,
-+
+     "compatible_features": 0,
-+echo
+     "autoclear_features": 1,
-+echo "== erase slot 0 and try it =="
+     "refcount_order": 4,
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 | _filter_img_create
 +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
 +
 +echo
 +echo "== erase slot 2 and try it =="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=2 | _filter_img_create
 +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS2 | _filter_qemu_io | _filter_testdir
 +
 +
 +# at this point slots 1 and 3 should be active
 +
 +echo
 +echo "== filling  4 slots with secret 2 =="
 +for i in $(seq 0 3) ; do
 +    $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10
 +done
 +
 +echo
 +echo "== adding secret 0 =="
 +    $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10
 +
 +echo
 +echo "== adding secret 3 (last slot) =="
 +    $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10
 +
 +echo
 +echo "== trying to add another slot (should fail) =="
 +$QEMU_IMG amend $SECRETS $IMGS2 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10
 +
 +echo
 +echo "== all secrets should work again =="
 +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
 +    $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
 +done
 +
 +
 +echo
 +
 +echo "== erase all keys of secret 2=="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec2
 +
 +echo "== erase all keys of secret 1=="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1
 +
 +echo "== erase all keys of secret 0=="
 +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec0
 +
 +echo "== erasing secret3 will fail now since it is the only secret (in 3 slots) =="
 +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=inactive,${PR}old-secret=sec3
 +
 +echo
 +echo "== only secret3 should work now  =="
 +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
 +    $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
 +done
 +
 +echo
 +echo "== add secret0  =="
 +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10
 +
 +echo "== erase secret3 =="
 +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec3
 +
 +echo
 +echo "== only secret0 should work now  =="
 +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
 +    $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
 +done
 +
 +echo
 +echo "== replace secret0 with secret1 (should fail)  =="
 +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}keyslot=0
 +
 +echo
 +echo "== replace secret0 with secret1 with force (should work)  =="
 +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10,${PR}keyslot=0 --force
 +
 +echo
 +echo "== only secret1 should work now  =="
 +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
 +    $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
 +done
 +
 +
 +echo
 +echo "== erase last secret (should fail)  =="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1
 +
 +
 +echo "== erase non existing secrets (should fail)  =="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec5 --force
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 --force
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=1 --force
 +
 +echo
 +echo "== erase last secret with force by slot (should work)  =="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0 --force
 +
 +echo
 +echo "== we have no secrets now, data is lost forever =="
 +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
 +    $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
 +done
 +
 +# success, all done
 +echo "*** done"
 +rm -f $seq.full
 +status=0
 +
 diff --git a/tests/qemu-iotests/293.out b/tests/qemu-iotests/293.out
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/tests/qemu-iotests/293.out
@@ -XXX,XX +XXX,XX @@
 +QA output created by 293
 +== creating a test image ==
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
 +
 +== test that key 0 opens the image ==
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== adding a password to slot 4 ==
 +== adding a password to slot 1 ==
 +== adding a password to slot 3 ==
 +== adding a password to slot 2 ==
 +== erase slot 4 ==
 +
 +== all secrets should work ==
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== erase slot 0 and try it ==
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +
 +== erase slot 2 and try it ==
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +
 +== filling  4 slots with secret 2 ==
 +
 +== adding secret 0 ==
 +
 +== adding secret 3 (last slot) ==
 +
 +== trying to add another slot (should fail) ==
 +qemu-img: Can't add a keyslot - all keyslots are in use
 +
 +== all secrets should work again ==
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== erase all keys of secret 2==
 +== erase all keys of secret 1==
 +== erase all keys of secret 0==
 +== erasing secret3 will fail now since it is the only secret (in 3 slots) ==
 +qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
 +
 +== only secret3 should work now  ==
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== add secret0  ==
 +== erase secret3 ==
 +
 +== only secret0 should work now  ==
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +
 +== replace secret0 with secret1 (should fail)  ==
 +qemu-img: Refusing to overwrite active keyslot 0 - please erase it first
 +
 +== replace secret0 with secret1 with force (should work)  ==
 +
 +== only secret1 should work now  ==
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +
 +== erase last secret (should fail)  ==
 +qemu-img: Attempt to erase the only active keyslot 0 which will erase all the data in the image irreversibly - refusing operation
 +qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
 +== erase non existing secrets (should fail)  ==
 +qemu-img: No secret with id 'sec5'
 +qemu-img: No keyslots match given (old) password for erase operation
 +
 +== erase last secret with force by slot (should work)  ==
 +
 +== we have no secrets now, data is lost forever ==
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +*** done
 diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294
 new file mode 100755
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/tests/qemu-iotests/294
@@ -XXX,XX +XXX,XX @@
 +#
 +# Copyright (C) 2019 Red Hat, Inc.
 +#
 +# This program is free software; you can redistribute it and/or modify
 +# it under the terms of the GNU General Public License as published by
 +# the Free Software Foundation; either version 2 of the License, or
 +# (at your option) any later version.
 +#
 +# This program is distributed in the hope that it will be useful,
 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 +# GNU General Public License for more details.
 +#
 +# You should have received a copy of the GNU General Public License
 +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 +#
 +
 +# creator
 +owner=mlevitsk@redhat.com
 +
 +seq=`basename $0`
 +echo "QA output created by $seq"
 +
 +status=1    # failure is the default!
 +
 +_cleanup()
 +{
 +    _cleanup_test_img
 +}
 +trap "_cleanup; exit \$status" 0 1 2 3 15
 +
 +# get standard environment, filters and checks
 +. ./common.rc
 +. ./common.filter
 +
 +_supported_fmt luks
 +_supported_proto file #TODO
 +
 +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
 +
 +# you are supposed to see the password as *******, see :-)
 +S0="--object secret,id=sec0,data=hunter0"
 +S1="--object secret,id=sec1,data=hunter1"
 +SECRETS="$S0 $S1"
 +
 +
 +IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec0"
 +IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec1"
 +
 +echo "== creating a test image =="
 +_make_test_img $S0 -o "key-secret=sec0,iter-time=10" 32M
 +
 +echo
 +echo "== test that key 0 opens the image =="
 +$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
 +
 +echo
 +echo "== adding a password to slot 1 =="
 +$QEMU_IMG amend $SECRETS $IMGS0 -o state=active,new-secret=sec1,keyslot=1,iter-time=10
 +
 +echo
 +echo "== 'backup' the image header =="
 +dd if=$TEST_IMG_FILE of=${TEST_IMG_FILE}.bk bs=4K skip=0 count=1
 +
 +echo
 +echo "== erase slot 0 =="
 +$QEMU_IMG amend $SECRETS $IMGS1 -o state=inactive,keyslot=0 | _filter_img_create
 +
 +echo
 +echo "== test that key 0 doesn't open the image =="
 +$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
 +
 +echo
 +echo "== 'restore' the image header =="
 +dd if=${TEST_IMG_FILE}.bk of=${TEST_IMG_FILE} bs=4K skip=0 count=1 conv=notrunc
 +
 +echo
 +echo "== test that key 0 still doesn't open the image (key material is erased) =="
 +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
 +
 +echo
 +echo "== test that key 1 still works =="
 +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS1 | _filter_qemu_io | _filter_testdir
 +
 +echo "*** done"
 +rm -f $seq.full
 +status=0
 +
 +
 +exit 0
 diff --git a/tests/qemu-iotests/294.out b/tests/qemu-iotests/294.out
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/tests/qemu-iotests/294.out
@@ -XXX,XX +XXX,XX @@
 +QA output created by 294
 +== creating a test image ==
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
 +
 +== test that key 0 opens the image ==
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== adding a password to slot 1 ==
 +
 +== 'backup' the image header ==
 +1+0 records in
 +1+0 records out
 +
 +== erase slot 0 ==
 +
 +== test that key 0 doesn't open the image ==
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +
 +== 'restore' the image header ==
 +1+0 records in
 +1+0 records out
 +
 +== test that key 0 still doesn't open the image (key material is erased) ==
 +qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +
 +== test that key 1 still works ==
 +read 4096/4096 bytes at offset 0
 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +*** done
 diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/group
 +++ b/tests/qemu-iotests/group
@@ -XXX,XX +XXX,XX @@
 rw auto quick
 rw quick
 rw auto quick
 +293 rw auto
 +294 rw auto quick
 meta
 --
-.26.2
+.34.1

-[PULL 05/18] qcrypto/luks: implement encryption key management
+[PULL 10/24] iotest 065: explicit compression type
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Next few patches will expose that functionality to the user.
+The test checks different options. It of course fails if set
 IMGOPTS='compression_type=zstd'. So, let's be explicit in what
 compression type we want and independent of IMGOPTS. Test both existing
 compression types.
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Message-Id: <20200608094030.670121-3-mlevitsk@redhat.com>
+Message-Id: <20211223160144.1097696-9-vsementsov@virtuozzo.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- qapi/crypto.json    |  59 ++++++-
+ tests/qemu-iotests/065 | 16 ++++++++--------
- crypto/block-luks.c | 416 +++++++++++++++++++++++++++++++++++++++++++-
+file changed, 8 insertions(+), 8 deletions(-)
 files changed, 469 insertions(+), 6 deletions(-)
-diff --git a/qapi/crypto.json b/qapi/crypto.json
+diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/qapi/crypto.json
+--- a/tests/qemu-iotests/065
-+++ b/qapi/crypto.json
++++ b/tests/qemu-iotests/065
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ class TestQMP(TestImageInfoSpecific):
-            'uuid': 'str',
-            'slots': [ 'QCryptoBlockInfoLUKSSlot' ] }}
+ class TestQCow2(TestQemuImgInfo):
+     '''Testing a qcow2 version 2 image'''
--
+-    img_options = 'compat=0.10'
- ##
++    img_options = 'compat=0.10,compression_type=zlib'
- # @QCryptoBlockInfo:
+     json_compare = { 'compat': '0.10', 'refcount-bits': 16,
- #
+                      'compression-type': 'zlib' }
-@@ -XXX,XX +XXX,XX @@
+     human_compare = [ 'compat: 0.10', 'compression type: zlib',
-   'discriminator': 'format',
+@@ -XXX,XX +XXX,XX @@ class TestQCow2(TestQemuImgInfo):
-   'data': { 'luks': 'QCryptoBlockInfoLUKS' } }
+ class TestQCow3NotLazy(TestQemuImgInfo):
-+##
+     '''Testing a qcow2 version 3 image with lazy refcounts disabled'''
-+# @QCryptoBlockLUKSKeyslotState:
+-    img_options = 'compat=1.1,lazy_refcounts=off'
-+#
++    img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zstd'
-+# Defines state of keyslots that are affected by the update
+     json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
-+#
+                      'refcount-bits': 16, 'corrupt': False,
-+# @active:    The slots contain the given password and marked as active
+-                     'compression-type': 'zlib', 'extended-l2': False }
-+# @inactive:  The slots are erased (contain garbage) and marked as inactive
+-    human_compare = [ 'compat: 1.1', 'compression type: zlib',
-+#
++                     'compression-type': 'zstd', 'extended-l2': False }
-+# Since: 5.1
++    human_compare = [ 'compat: 1.1', 'compression type: zstd',
-+##
+                       'lazy refcounts: false', 'refcount bits: 16',
-+{ 'enum': 'QCryptoBlockLUKSKeyslotState',
+                       'corrupt: false', 'extended l2: false' ]
-+  'data': [ 'active', 'inactive' ] }
-+
+ class TestQCow3Lazy(TestQemuImgInfo):
+     '''Testing a qcow2 version 3 image with lazy refcounts enabled'''
-+##
+-    img_options = 'compat=1.1,lazy_refcounts=on'
-+# @QCryptoBlockAmendOptionsLUKS:
++    img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zlib'
-+#
+     json_compare = { 'compat': '1.1', 'lazy-refcounts': True,
-+# This struct defines the update parameters that activate/de-activate set
+                      'refcount-bits': 16, 'corrupt': False,
-+# of keyslots
+                      'compression-type': 'zlib', 'extended-l2': False }
-+#
+@@ -XXX,XX +XXX,XX @@ class TestQCow3Lazy(TestQemuImgInfo):
-+# @state: the desired state of the keyslots
+ class TestQCow3NotLazyQMP(TestQMP):
-+#
+     '''Testing a qcow2 version 3 image with lazy refcounts disabled, opening
-+# @new-secret:    The ID of a QCryptoSecret object providing the password to be
+        with lazy refcounts enabled'''
-+#                 written into added active keyslots
+-    img_options = 'compat=1.1,lazy_refcounts=off'
-+#
++    img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zlib'
-+# @old-secret:    Optional (for deactivation only)
+     qemu_options = 'lazy-refcounts=on'
-+#                 If given will deactive all keyslots that
+     compare = { 'compat': '1.1', 'lazy-refcounts': False,
-+#                 match password located in QCryptoSecret with this ID
+                 'refcount-bits': 16, 'corrupt': False,
-+#
+@@ -XXX,XX +XXX,XX @@ class TestQCow3NotLazyQMP(TestQMP):
-+# @iter-time:     Optional (for activation only)
+ class TestQCow3LazyQMP(TestQMP):
-+#                 Number of milliseconds to spend in
+     '''Testing a qcow2 version 3 image with lazy refcounts enabled, opening
-+#                 PBKDF passphrase processing for the newly activated keyslot.
+        with lazy refcounts disabled'''
-+#                 Currently defaults to 2000.
+-    img_options = 'compat=1.1,lazy_refcounts=on'
-+#
++    img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zstd'
-+# @keyslot:       Optional. ID of the keyslot to activate/deactivate.
+     qemu_options = 'lazy-refcounts=off'
-+#                 For keyslot activation, keyslot should not be active already
+     compare = { 'compat': '1.1', 'lazy-refcounts': True,
-+#                 (this is unsafe to update an active keyslot),
+                 'refcount-bits': 16, 'corrupt': False,
-+#                 but possible if 'force' parameter is given.
+-                'compression-type': 'zlib', 'extended-l2': False }
-+#                 If keyslot is not given, first free keyslot will be written.
++                'compression-type': 'zstd', 'extended-l2': False }
-+#
-+#                 For keyslot deactivation, this parameter specifies the exact
+ TestImageInfoSpecific = None
-+#                 keyslot to deactivate
+ TestQemuImgInfo = None
 +#
 +# @secret:        Optional. The ID of a QCryptoSecret object providing the
 +#                 password to use to retrive current master key.
 +#                 Defaults to the same secret that was used to open the image
 +#
 +#
 +# Since 5.1
 +##
 +{ 'struct': 'QCryptoBlockAmendOptionsLUKS',
 +  'data': { 'state': 'QCryptoBlockLUKSKeyslotState',
 +            '*new-secret': 'str',
 +            '*old-secret': 'str',
 +            '*keyslot': 'int',
 +            '*iter-time': 'int',
 +            '*secret': 'str' } }
  ##
  # @QCryptoBlockAmendOptions:
@@ -XXX,XX +XXX,XX @@
    'base': 'QCryptoBlockOptionsBase',
    'discriminator': 'format',
    'data': {
 -            } }
 +          'luks': 'QCryptoBlockAmendOptionsLUKS' } }
 diff --git a/crypto/block-luks.c b/crypto/block-luks.c
 index XXXXXXX..XXXXXXX 100644
 --- a/crypto/block-luks.c
 +++ b/crypto/block-luks.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/uuid.h"
  #include "qemu/coroutine.h"
 +#include "qemu/bitmap.h"
  /*
   * Reference for the LUKS format implemented here is
@@ -XXX,XX +XXX,XX @@ typedef struct QCryptoBlockLUKSKeySlot QCryptoBlockLUKSKeySlot;
  #define QCRYPTO_BLOCK_LUKS_SECTOR_SIZE 512LL
 +#define QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS 2000
 +#define QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS 40
 +
  static const char qcrypto_block_luks_magic[QCRYPTO_BLOCK_LUKS_MAGIC_LEN] = {
      'L', 'U', 'K', 'S', 0xBA, 0xBE
  };
@@ -XXX,XX +XXX,XX @@ struct QCryptoBlockLUKS {
      /* Hash algorithm used in pbkdf2 function */
      QCryptoHashAlgorithm hash_alg;
 +
 +    /* Name of the secret that was used to open the image */
 +    char *secret;
  };
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_store_key(QCryptoBlock *block,
                               Error **errp)
  {
      QCryptoBlockLUKS *luks = block->opaque;
 -    QCryptoBlockLUKSKeySlot *slot = &luks->header.key_slots[slot_idx];
 +    QCryptoBlockLUKSKeySlot *slot;
      g_autofree uint8_t *splitkey = NULL;
      size_t splitkeylen;
      g_autofree uint8_t *slotkey = NULL;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_store_key(QCryptoBlock *block,
      uint64_t iters;
      int ret = -1;
 +    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
 +    slot = &luks->header.key_slots[slot_idx];
      if (qcrypto_random_bytes(slot->salt,
                               QCRYPTO_BLOCK_LUKS_SALT_LEN,
                               errp) < 0) {
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_load_key(QCryptoBlock *block,
                              Error **errp)
  {
      QCryptoBlockLUKS *luks = block->opaque;
 -    const QCryptoBlockLUKSKeySlot *slot = &luks->header.key_slots[slot_idx];
 +    const QCryptoBlockLUKSKeySlot *slot;
      g_autofree uint8_t *splitkey = NULL;
      size_t splitkeylen;
      g_autofree uint8_t *possiblekey = NULL;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_load_key(QCryptoBlock *block,
      g_autoptr(QCryptoIVGen) ivgen = NULL;
      size_t niv;
 +    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
 +    slot = &luks->header.key_slots[slot_idx];
      if (slot->active != QCRYPTO_BLOCK_LUKS_KEY_SLOT_ENABLED) {
          return 0;
      }
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_find_key(QCryptoBlock *block,
      return -1;
  }
 +/*
 + * Returns true if a slot i is marked as active
 + * (contains encrypted copy of the master key)
 + */
 +static bool
 +qcrypto_block_luks_slot_active(const QCryptoBlockLUKS *luks,
 +                               unsigned int slot_idx)
 +{
 +    uint32_t val;
 +
 +    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
 +    val = luks->header.key_slots[slot_idx].active;
 +    return val == QCRYPTO_BLOCK_LUKS_KEY_SLOT_ENABLED;
 +}
 +
 +/*
 + * Returns the number of slots that are marked as active
 + * (slots that contain encrypted copy of the master key)
 + */
 +static unsigned int
 +qcrypto_block_luks_count_active_slots(const QCryptoBlockLUKS *luks)
 +{
 +    size_t i = 0;
 +    unsigned int ret = 0;
 +
 +    for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
 +        if (qcrypto_block_luks_slot_active(luks, i)) {
 +            ret++;
 +        }
 +    }
 +    return ret;
 +}
 +
 +/*
 + * Finds first key slot which is not active
 + * Returns the key slot index, or -1 if it doesn't exist
 + */
 +static int
 +qcrypto_block_luks_find_free_keyslot(const QCryptoBlockLUKS *luks)
 +{
 +    size_t i;
 +
 +    for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
 +        if (!qcrypto_block_luks_slot_active(luks, i)) {
 +            return i;
 +        }
 +    }
 +    return -1;
 +}
 +
 +/*
 + * Erases an keyslot given its index
 + * Returns:
 + *    0 if the keyslot was erased successfully
 + *   -1 if a error occurred while erasing the keyslot
 + *
 + */
 +static int
 +qcrypto_block_luks_erase_key(QCryptoBlock *block,
 +                             unsigned int slot_idx,
 +                             QCryptoBlockWriteFunc writefunc,
 +                             void *opaque,
 +                             Error **errp)
 +{
 +    QCryptoBlockLUKS *luks = block->opaque;
 +    QCryptoBlockLUKSKeySlot *slot;
 +    g_autofree uint8_t *garbagesplitkey = NULL;
 +    size_t splitkeylen;
 +    size_t i;
 +    Error *local_err = NULL;
 +    int ret;
 +
 +    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
 +    slot = &luks->header.key_slots[slot_idx];
 +
 +    splitkeylen = luks->header.master_key_len * slot->stripes;
 +    assert(splitkeylen > 0);
 +
 +    garbagesplitkey = g_new0(uint8_t, splitkeylen);
 +
 +    /* Reset the key slot header */
 +    memset(slot->salt, 0, QCRYPTO_BLOCK_LUKS_SALT_LEN);
 +    slot->iterations = 0;
 +    slot->active = QCRYPTO_BLOCK_LUKS_KEY_SLOT_DISABLED;
 +
 +    ret = qcrypto_block_luks_store_header(block, writefunc,
 +                                          opaque, &local_err);
 +
 +    if (ret < 0) {
 +        error_propagate(errp, local_err);
 +    }
 +    /*
 +     * Now try to erase the key material, even if the header
 +     * update failed
 +     */
 +    for (i = 0; i < QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS; i++) {
 +        if (qcrypto_random_bytes(garbagesplitkey,
 +                                 splitkeylen, &local_err) < 0) {
 +            /*
 +             * If we failed to get the random data, still write
 +             * at least zeros to the key slot at least once
 +             */
 +            error_propagate(errp, local_err);
 +
 +            if (i > 0) {
 +                return -1;
 +            }
 +        }
 +        if (writefunc(block,
 +                      slot->key_offset_sector * QCRYPTO_BLOCK_LUKS_SECTOR_SIZE,
 +                      garbagesplitkey,
 +                      splitkeylen,
 +                      opaque,
 +                      &local_err) != splitkeylen) {
 +            error_propagate(errp, local_err);
 +            return -1;
 +        }
 +    }
 +    return ret;
 +}
  static int
  qcrypto_block_luks_open(QCryptoBlock *block,
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_open(QCryptoBlock *block,
      luks = g_new0(QCryptoBlockLUKS, 1);
      block->opaque = luks;
 +    luks->secret = g_strdup(options->u.luks.key_secret);
      if (qcrypto_block_luks_load_header(block, readfunc, opaque, errp) < 0) {
          goto fail;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_open(QCryptoBlock *block,
   fail:
      qcrypto_block_free_cipher(block);
      qcrypto_ivgen_free(block->ivgen);
 +    g_free(luks->secret);
      g_free(luks);
      return -1;
  }
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_create(QCryptoBlock *block,
      memcpy(&luks_opts, &options->u.luks, sizeof(luks_opts));
      if (!luks_opts.has_iter_time) {
 -        luks_opts.iter_time = 2000;
 +        luks_opts.iter_time = QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
      }
      if (!luks_opts.has_cipher_alg) {
          luks_opts.cipher_alg = QCRYPTO_CIPHER_ALG_AES_256;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_create(QCryptoBlock *block,
                     optprefix ? optprefix : "");
          goto error;
      }
 +    luks->secret = g_strdup(options->u.luks.key_secret);
 +
      password = qcrypto_secret_lookup_as_utf8(luks_opts.key_secret, errp);
      if (!password) {
          goto error;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_create(QCryptoBlock *block,
      qcrypto_block_free_cipher(block);
      qcrypto_ivgen_free(block->ivgen);
 +    g_free(luks->secret);
      g_free(luks);
      return -1;
  }
 +static int
 +qcrypto_block_luks_amend_add_keyslot(QCryptoBlock *block,
 +                                     QCryptoBlockReadFunc readfunc,
 +                                     QCryptoBlockWriteFunc writefunc,
 +                                     void *opaque,
 +                                     QCryptoBlockAmendOptionsLUKS *opts_luks,
 +                                     bool force,
 +                                     Error **errp)
 +{
 +    QCryptoBlockLUKS *luks = block->opaque;
 +    uint64_t iter_time = opts_luks->has_iter_time ?
 +                         opts_luks->iter_time :
 +                         QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
 +    int keyslot;
 +    g_autofree char *old_password = NULL;
 +    g_autofree char *new_password = NULL;
 +    g_autofree uint8_t *master_key = NULL;
 +
 +    char *secret = opts_luks->has_secret ? opts_luks->secret : luks->secret;
 +
 +    if (!opts_luks->has_new_secret) {
 +        error_setg(errp, "'new-secret' is required to activate a keyslot");
 +        return -1;
 +    }
 +    if (opts_luks->has_old_secret) {
 +        error_setg(errp,
 +                   "'old-secret' must not be given when activating keyslots");
 +        return -1;
 +    }
 +
 +    if (opts_luks->has_keyslot) {
 +        keyslot = opts_luks->keyslot;
 +        if (keyslot < 0 || keyslot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS) {
 +            error_setg(errp,
 +                       "Invalid keyslot %u specified, must be between 0 and %u",
 +                       keyslot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1);
 +            return -1;
 +        }
 +    } else {
 +        keyslot = qcrypto_block_luks_find_free_keyslot(luks);
 +        if (keyslot == -1) {
 +            error_setg(errp,
 +                       "Can't add a keyslot - all keyslots are in use");
 +            return -1;
 +        }
 +    }
 +
 +    if (!force && qcrypto_block_luks_slot_active(luks, keyslot)) {
 +        error_setg(errp,
 +                   "Refusing to overwrite active keyslot %i - "
 +                   "please erase it first",
 +                   keyslot);
 +        return -1;
 +    }
 +
 +    /* Locate the password that will be used to retrieve the master key */
 +    old_password = qcrypto_secret_lookup_as_utf8(secret, errp);
 +    if (!old_password) {
 +        return -1;
 +    }
 +
 +    /* Retrieve the master key */
 +    master_key = g_new0(uint8_t, luks->header.master_key_len);
 +
 +    if (qcrypto_block_luks_find_key(block, old_password, master_key,
 +                                    readfunc, opaque, errp) < 0) {
 +        error_append_hint(errp, "Failed to retrieve the master key");
 +        return -1;
 +    }
 +
 +    /* Locate the new password*/
 +    new_password = qcrypto_secret_lookup_as_utf8(opts_luks->new_secret, errp);
 +    if (!new_password) {
 +        return -1;
 +    }
 +
 +    /* Now set the new keyslots */
 +    if (qcrypto_block_luks_store_key(block, keyslot, new_password, master_key,
 +                                     iter_time, writefunc, opaque, errp)) {
 +        error_append_hint(errp, "Failed to write to keyslot %i", keyslot);
 +        return -1;
 +    }
 +    return 0;
 +}
 +
 +static int
 +qcrypto_block_luks_amend_erase_keyslots(QCryptoBlock *block,
 +                                        QCryptoBlockReadFunc readfunc,
 +                                        QCryptoBlockWriteFunc writefunc,
 +                                        void *opaque,
 +                                        QCryptoBlockAmendOptionsLUKS *opts_luks,
 +                                        bool force,
 +                                        Error **errp)
 +{
 +    QCryptoBlockLUKS *luks = block->opaque;
 +    g_autofree uint8_t *tmpkey = NULL;
 +    g_autofree char *old_password = NULL;
 +
 +    if (opts_luks->has_new_secret) {
 +        error_setg(errp,
 +                   "'new-secret' must not be given when erasing keyslots");
 +        return -1;
 +    }
 +    if (opts_luks->has_iter_time) {
 +        error_setg(errp,
 +                   "'iter-time' must not be given when erasing keyslots");
 +        return -1;
 +    }
 +    if (opts_luks->has_secret) {
 +        error_setg(errp,
 +                   "'secret' must not be given when erasing keyslots");
 +        return -1;
 +    }
 +
 +    /* Load the old password if given */
 +    if (opts_luks->has_old_secret) {
 +        old_password = qcrypto_secret_lookup_as_utf8(opts_luks->old_secret,
 +                                                     errp);
 +        if (!old_password) {
 +            return -1;
 +        }
 +
 +        /*
 +         * Allocate a temporary key buffer that we will need when
 +         * checking if slot matches the given old password
 +         */
 +        tmpkey = g_new0(uint8_t, luks->header.master_key_len);
 +    }
 +
 +    /* Erase an explicitly given keyslot */
 +    if (opts_luks->has_keyslot) {
 +        int keyslot = opts_luks->keyslot;
 +
 +        if (keyslot < 0 || keyslot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS) {
 +            error_setg(errp,
 +                       "Invalid keyslot %i specified, must be between 0 and %i",
 +                       keyslot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1);
 +            return -1;
 +        }
 +
 +        if (opts_luks->has_old_secret) {
 +            int rv = qcrypto_block_luks_load_key(block,
 +                                                 keyslot,
 +                                                 old_password,
 +                                                 tmpkey,
 +                                                 readfunc,
 +                                                 opaque,
 +                                                 errp);
 +            if (rv == -1) {
 +                return -1;
 +            } else if (rv == 0) {
 +                error_setg(errp,
 +                           "Given keyslot %i doesn't contain the given "
 +                           "old password for erase operation",
 +                           keyslot);
 +                return -1;
 +            }
 +        }
 +
 +        if (!force && !qcrypto_block_luks_slot_active(luks, keyslot)) {
 +            error_setg(errp,
 +                       "Given keyslot %i is already erased (inactive) ",
 +                       keyslot);
 +            return -1;
 +        }
 +
 +        if (!force && qcrypto_block_luks_count_active_slots(luks) == 1) {
 +            error_setg(errp,
 +                       "Attempt to erase the only active keyslot %i "
 +                       "which will erase all the data in the image "
 +                       "irreversibly - refusing operation",
 +                       keyslot);
 +            return -1;
 +        }
 +
 +        if (qcrypto_block_luks_erase_key(block, keyslot,
 +                                         writefunc, opaque, errp)) {
 +            error_append_hint(errp, "Failed to erase keyslot %i", keyslot);
 +            return -1;
 +        }
 +
 +    /* Erase all keyslots that match the given old password */
 +    } else if (opts_luks->has_old_secret) {
 +
 +        unsigned long slots_to_erase_bitmap = 0;
 +        size_t i;
 +        int slot_count;
 +
 +        assert(QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS <=
 +               sizeof(slots_to_erase_bitmap) * 8);
 +
 +        for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
 +            int rv = qcrypto_block_luks_load_key(block,
 +                                                 i,
 +                                                 old_password,
 +                                                 tmpkey,
 +                                                 readfunc,
 +                                                 opaque,
 +                                                 errp);
 +            if (rv == -1) {
 +                return -1;
 +            } else if (rv == 1) {
 +                bitmap_set(&slots_to_erase_bitmap, i, 1);
 +            }
 +        }
 +
 +        slot_count = bitmap_count_one(&slots_to_erase_bitmap,
 +                                      QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
 +        if (slot_count == 0) {
 +            error_setg(errp,
 +                       "No keyslots match given (old) password for erase operation");
 +            return -1;
 +        }
 +
 +        if (!force &&
 +            slot_count == qcrypto_block_luks_count_active_slots(luks)) {
 +            error_setg(errp,
 +                       "All the active keyslots match the (old) password that "
 +                       "was given and erasing them will erase all the data in "
 +                       "the image irreversibly - refusing operation");
 +            return -1;
 +        }
 +
 +        /* Now apply the update */
 +        for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
 +            if (!test_bit(i, &slots_to_erase_bitmap)) {
 +                continue;
 +            }
 +            if (qcrypto_block_luks_erase_key(block, i, writefunc,
 +                opaque, errp)) {
 +                error_append_hint(errp, "Failed to erase keyslot %zu", i);
 +                return -1;
 +            }
 +        }
 +    } else {
 +        error_setg(errp,
 +                   "To erase keyslot(s), either explicit keyslot index "
 +                   "or the password currently contained in them must be given");
 +        return -1;
 +    }
 +    return 0;
 +}
 +
 +static int
 +qcrypto_block_luks_amend_options(QCryptoBlock *block,
 +                                 QCryptoBlockReadFunc readfunc,
 +                                 QCryptoBlockWriteFunc writefunc,
 +                                 void *opaque,
 +                                 QCryptoBlockAmendOptions *options,
 +                                 bool force,
 +                                 Error **errp)
 +{
 +    QCryptoBlockAmendOptionsLUKS *opts_luks = &options->u.luks;
 +
 +    switch (opts_luks->state) {
 +    case Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE_ACTIVE:
 +        return qcrypto_block_luks_amend_add_keyslot(block, readfunc,
 +                                                    writefunc, opaque,
 +                                                    opts_luks, force, errp);
 +    case Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE_INACTIVE:
 +        return qcrypto_block_luks_amend_erase_keyslots(block, readfunc,
 +                                                       writefunc, opaque,
 +                                                       opts_luks, force, errp);
 +    default:
 +        g_assert_not_reached();
 +    }
 +}
  static int qcrypto_block_luks_get_info(QCryptoBlock *block,
                                         QCryptoBlockInfo *info,
@@ -XXX,XX +XXX,XX @@ static int qcrypto_block_luks_get_info(QCryptoBlock *block,
  static void qcrypto_block_luks_cleanup(QCryptoBlock *block)
  {
 -    g_free(block->opaque);
 +    QCryptoBlockLUKS *luks = block->opaque;
 +    if (luks) {
 +        g_free(luks->secret);
 +        g_free(luks);
 +    }
  }
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_encrypt(QCryptoBlock *block,
  const QCryptoBlockDriver qcrypto_block_driver_luks = {
      .open = qcrypto_block_luks_open,
      .create = qcrypto_block_luks_create,
 +    .amend = qcrypto_block_luks_amend_options,
      .get_info = qcrypto_block_luks_get_info,
      .cleanup = qcrypto_block_luks_cleanup,
      .decrypt = qcrypto_block_luks_decrypt,
 --
-.26.2
+.34.1

-[PULL 08/18] block/amend: refactor qcow2 amend options
+[PULL 11/24] iotests.py: filter out successful output of qemu-img create
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Some qcow2 create options can't be used for amend.
+The only "feature" of this "Formatting ..." line is that we have to
-Remove them from the qcow2 create options and add generic logic to detect
+update it every time we add new option. Let's drop it.
 such options in qemu-img
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Message-Id: <20200608094030.670121-6-mlevitsk@redhat.com>
+Message-Id: <20211223160144.1097696-10-vsementsov@virtuozzo.com>
-[mreitz: Dropped some iotests reference output hunks that became
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
          unnecessary thanks to
          "iotests: Make _filter_img_create more active"]
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/qcow2.c              | 138 +++++++++-----------------------
+ tests/qemu-iotests/149.out    | 21 ---------------------
- qemu-img.c                 |  18 ++++-
+ tests/qemu-iotests/237.out    |  3 ---
- tests/qemu-iotests/049.out | 102 ++++++++++++------------
+ tests/qemu-iotests/255.out    |  4 ----
- tests/qemu-iotests/061.out |  12 ++-
+ tests/qemu-iotests/274.out    | 29 -----------------------------
- tests/qemu-iotests/082.out | 158 ++++---------------------------------
+ tests/qemu-iotests/280.out    |  1 -
- tests/qemu-iotests/085.out |  38 ++++-----
+ tests/qemu-iotests/296.out    | 10 +++-------
- tests/qemu-iotests/144.out |   4 +-
+ tests/qemu-iotests/iotests.py | 10 ++++++++--
- tests/qemu-iotests/182.out |   2 +-
+files changed, 11 insertions(+), 67 deletions(-)
  tests/qemu-iotests/185.out |   8 +-
  tests/qemu-iotests/255.out |   8 +-
  tests/qemu-iotests/274.out |  46 +++++------
  tests/qemu-iotests/280.out |   2 +-
 files changed, 183 insertions(+), 353 deletions(-)
-diff --git a/block/qcow2.c b/block/qcow2.c
+diff --git a/tests/qemu-iotests/149.out b/tests/qemu-iotests/149.out
 index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2.c
+--- a/tests/qemu-iotests/149.out
-+++ b/block/qcow2.c
++++ b/tests/qemu-iotests/149.out
-@@ -XXX,XX +XXX,XX @@ static int qcow2_change_backing_file(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha1.img
-     return qcow2_update_header(bs);
+ # ================= qemu-img aes-256-xts-plain64-sha1 =================
- }
+ # Create image
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain64-sha1.img 4194304M
--static int qcow2_crypt_method_from_format(const char *encryptfmt)
+-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
--{
--    if (g_str_equal(encryptfmt, "luks")) {
+ # Open dev
--        return QCOW_CRYPT_LUKS;
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1
--    } else if (g_str_equal(encryptfmt, "aes")) {
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-twofish-256-xts-plain64-sha1.img
--        return QCOW_CRYPT_AES;
+ # ================= qemu-img twofish-256-xts-plain64-sha1 =================
--    } else {
+ # Create image
--        return -EINVAL;
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-256-xts-plain64-sha1.img 4194304M
--    }
+-Formatting 'TEST_DIR/luks-twofish-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
--}
--
+ # Open dev
- static int qcow2_set_up_encryption(BlockDriverState *bs,
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1
-                                    QCryptoBlockCreateOptions *cryptoopts,
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-256-xts-plain64-sha1.img
-                                    Error **errp)
+ # ================= qemu-img serpent-256-xts-plain64-sha1 =================
-@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
+ # Create image
-     bool lazy_refcounts = s->use_lazy_refcounts;
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-256-xts-plain64-sha1.img 4194304M
-     bool data_file_raw = data_file_is_raw(bs);
+-Formatting 'TEST_DIR/luks-serpent-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
-     const char *compat = NULL;
--    uint64_t cluster_size = s->cluster_size;
+ # Open dev
--    bool encrypt;
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1
--    int encformat;
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img
-     int refcount_bits = s->refcount_bits;
+ # ================= qemu-img cast5-128-cbc-plain64-sha1 =================
-     int ret;
+ # Create image
-     QemuOptDesc *desc = opts->list->desc;
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=cast5-128,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img 4194304M
-@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
+-Formatting 'TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=cast5-128 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10
-                 error_setg(errp, "Unknown compatibility level %s", compat);
-                 return -EINVAL;
+ # Open dev
-             }
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1
--        } else if (!strcmp(desc->name, BLOCK_OPT_PREALLOC)) {
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain-sha1.img
--            error_setg(errp, "Cannot change preallocation mode");
+ # ================= qemu-img aes-256-cbc-plain-sha1 =================
--            return -ENOTSUP;
+ # Create image
-         } else if (!strcmp(desc->name, BLOCK_OPT_SIZE)) {
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain-sha1.img 4194304M
-             new_size = qemu_opt_get_size(opts, BLOCK_OPT_SIZE, 0);
+-Formatting 'TEST_DIR/luks-aes-256-cbc-plain-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain hash-alg=sha1 iter-time=10
-         } else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FILE)) {
-             backing_file = qemu_opt_get(opts, BLOCK_OPT_BACKING_FILE);
+ # Open dev
-         } else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FMT)) {
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1
-             backing_format = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT);
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha1.img
--        } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT)) {
+ # ================= qemu-img aes-256-cbc-plain64-sha1 =================
--            encrypt = qemu_opt_get_bool(opts, BLOCK_OPT_ENCRYPT,
+ # Create image
--                                        !!s->crypto);
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain64-sha1.img 4194304M
--
+-Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10
--            if (encrypt != !!s->crypto) {
--                error_setg(errp,
+ # Open dev
--                           "Changing the encryption flag is not supported");
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1
--                return -ENOTSUP;
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img
--            }
+ # ================= qemu-img aes-256-cbc-essiv-sha256-sha1 =================
--        } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT_FORMAT)) {
+ # Create image
--            encformat = qcow2_crypt_method_from_format(
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img 4194304M
--                qemu_opt_get(opts, BLOCK_OPT_ENCRYPT_FORMAT));
+-Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
--
--            if (encformat != s->crypt_method_header) {
+ # Open dev
--                error_setg(errp,
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1
--                           "Changing the encryption format is not supported");
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img
--                return -ENOTSUP;
+ # ================= qemu-img aes-256-xts-essiv-sha256-sha1 =================
--            }
+ # Create image
--        } else if (g_str_has_prefix(desc->name, "encrypt.")) {
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img 4194304M
--            error_setg(errp,
+-Formatting 'TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
--                       "Changing the encryption parameters is not supported");
--            return -ENOTSUP;
+ # Open dev
--        } else if (!strcmp(desc->name, BLOCK_OPT_CLUSTER_SIZE)) {
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1
--            cluster_size = qemu_opt_get_size(opts, BLOCK_OPT_CLUSTER_SIZE,
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img
--                                             cluster_size);
+ # ================= qemu-img aes-128-xts-plain64-sha256-sha1 =================
--            if (cluster_size != s->cluster_size) {
+ # Create image
--                error_setg(errp, "Changing the cluster size is not supported");
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img 4194304M
--                return -ENOTSUP;
+-Formatting 'TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
--            }
-         } else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) {
+ # Open dev
-             lazy_refcounts = qemu_opt_get_bool(opts, BLOCK_OPT_LAZY_REFCOUNTS,
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1
-                                                lazy_refcounts);
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img
-@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
+ # ================= qemu-img aes-192-xts-plain64-sha256-sha1 =================
-                                  "images");
+ # Create image
-                 return -EINVAL;
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img 4194304M
-             }
+-Formatting 'TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
--        } else if (!strcmp(desc->name, BLOCK_OPT_COMPRESSION_TYPE)) {
--            const char *ct_name =
+ # Open dev
--                qemu_opt_get(opts, BLOCK_OPT_COMPRESSION_TYPE);
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1
--            int compression_type =
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-twofish-128-xts-plain64-sha1.img
--                qapi_enum_parse(&Qcow2CompressionType_lookup, ct_name, -1,
+ # ================= qemu-img twofish-128-xts-plain64-sha1 =================
--                                NULL);
+ # Create image
--            if (compression_type == -1) {
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-128-xts-plain64-sha1.img 4194304M
--                error_setg(errp, "Unknown compression type: %s", ct_name);
+-Formatting 'TEST_DIR/luks-twofish-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
--                return -ENOTSUP;
--            }
+ # Open dev
--
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1
--            if (compression_type != s->compression_type) {
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-128-xts-plain64-sha1.img
--                error_setg(errp, "Changing the compression type "
+ # ================= qemu-img serpent-128-xts-plain64-sha1 =================
--                                 "is not supported");
+ # Create image
--                return -ENOTSUP;
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-128-xts-plain64-sha1.img 4194304M
--            }
+-Formatting 'TEST_DIR/luks-serpent-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
-         } else {
-             /* if this point is reached, this probably means a new option was
+ # Open dev
-              * added without having it covered here */
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1
-@@ -XXX,XX +XXX,XX @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-192-xts-plain64-sha1.img
-         .help = "The external data file must stay valid "           \
+ # ================= qemu-img serpent-192-xts-plain64-sha1 =================
-                 "as a raw image"                                    \
+ # Create image
-     },                                                              \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-192-xts-plain64-sha1.img 4194304M
--    {                                                               \
+-Formatting 'TEST_DIR/luks-serpent-192-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
--        .name = BLOCK_OPT_ENCRYPT,                                  \
--        .type = QEMU_OPT_BOOL,                                      \
+ # Open dev
--        .help = "Encrypt the image with format 'aes'. (Deprecated " \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1
--                "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",    \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha224.img
--    },                                                              \
+ # ================= qemu-img aes-256-xts-plain64-sha224 =================
--    {                                                               \
+ # Create image
--        .name = BLOCK_OPT_ENCRYPT_FORMAT,                           \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha224 TEST_DIR/luks-aes-256-xts-plain64-sha224.img 4194304M
--        .type = QEMU_OPT_STRING,                                    \
+-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha224.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha224 iter-time=10
--        .help = "Encrypt the image, format choices: 'aes', 'luks'", \
--    },                                                              \
+ # Open dev
--    BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",                     \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224
--        "ID of secret providing qcow AES key or LUKS passphrase"),  \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha256.img
--    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),               \
+ # ================= qemu-img aes-256-xts-plain64-sha256 =================
--    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),              \
+ # Create image
--    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),                \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha256 TEST_DIR/luks-aes-256-xts-plain64-sha256.img 4194304M
--    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),           \
+-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha256.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha256 iter-time=10
--    BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),                 \
--    BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),                \
+ # Open dev
--    {                                                               \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256
--        .name = BLOCK_OPT_CLUSTER_SIZE,                             \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha384.img
--        .type = QEMU_OPT_SIZE,                                      \
+ # ================= qemu-img aes-256-xts-plain64-sha384 =================
--        .help = "qcow2 cluster size",                               \
+ # Create image
--        .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)            \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha384 TEST_DIR/luks-aes-256-xts-plain64-sha384.img 4194304M
--    },                                                              \
+-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha384.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha384 iter-time=10
--    {                                                               \
--        .name = BLOCK_OPT_PREALLOC,                                 \
+ # Open dev
--        .type = QEMU_OPT_STRING,                                    \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384
--        .help = "Preallocation mode (allowed values: off, "         \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha512.img
--                "metadata, falloc, full)"                           \
+ # ================= qemu-img aes-256-xts-plain64-sha512 =================
--    },                                                              \
+ # Create image
-     {                                                               \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha512 TEST_DIR/luks-aes-256-xts-plain64-sha512.img 4194304M
-         .name = BLOCK_OPT_LAZY_REFCOUNTS,                           \
+-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha512.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha512 iter-time=10
-         .type = QEMU_OPT_BOOL,                                      \
-@@ -XXX,XX +XXX,XX @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
+ # Open dev
-         .type = QEMU_OPT_NUMBER,                                    \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512
-         .help = "Width of a reference count entry in bits",         \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img
-         .def_value_str = "16"                                       \
+ # ================= qemu-img aes-256-xts-plain64-ripemd160 =================
--    },                                                              \
+ # Create image
--    {                                                               \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=ripemd160 TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img 4194304M
--        .name = BLOCK_OPT_COMPRESSION_TYPE,                         \
+-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=ripemd160 iter-time=10
--        .type = QEMU_OPT_STRING,                                    \
--        .help = "Compression method used for image cluster "        \
+ # Open dev
--                "compression",                                      \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160
--        .def_value_str = "zlib"                                     \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img
-     }
+ # ================= qemu-img aes-256-xts-plain-sha1-pwallslots =================
+ # Create image
- static QemuOptsList qcow2_create_opts = {
+ qemu-img create -f luks --object secret,id=sec0,data=c2xvdDE=,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img 4194304M
-     .name = "qcow2-create-opts",
+-Formatting 'TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain hash-alg=sha1 iter-time=10
-     .head = QTAILQ_HEAD_INITIALIZER(qcow2_create_opts.head),
-     .desc = {
+ # Open dev
-+        {                                                               \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots
-+            .name = BLOCK_OPT_ENCRYPT,                                  \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img
-+            .type = QEMU_OPT_BOOL,                                      \
+ # ================= qemu-img aes-256-cbc-essiv-auto-sha1 =================
-+            .help = "Encrypt the image with format 'aes'. (Deprecated " \
+ # Create image
-+                    "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",    \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img 4194304M
-+        },                                                              \
+-Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv hash-alg=sha1 iter-time=10
-+        {                                                               \
-+            .name = BLOCK_OPT_ENCRYPT_FORMAT,                           \
+ # Open dev
-+            .type = QEMU_OPT_STRING,                                    \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1
-+            .help = "Encrypt the image, format choices: 'aes', 'luks'", \
+@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img
-+        },                                                              \
+ # ================= qemu-img aes-256-cbc-plain64-sha256-sha1 =================
-+        BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",                     \
+ # Create image
-+            "ID of secret providing qcow AES key or LUKS passphrase"),  \
+ qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img 4194304M
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),               \
+-Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),              \
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),                \
+ # Open dev
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),           \
+ sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),                 \
+diff --git a/tests/qemu-iotests/237.out b/tests/qemu-iotests/237.out
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),                \
+index XXXXXXX..XXXXXXX 100644
-+        {                                                               \
+--- a/tests/qemu-iotests/237.out
-+            .name = BLOCK_OPT_CLUSTER_SIZE,                             \
++++ b/tests/qemu-iotests/237.out
-+            .type = QEMU_OPT_SIZE,                                      \
+@@ -XXX,XX +XXX,XX @@ Job failed: Cannot find device='this doesn't exist' nor node-name='this doesn't
-+            .help = "qcow2 cluster size",                               \
-+            .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)            \
+ === Other subformats ===
-+        },                                                              \
-+        {                                                               \
+-Formatting 'TEST_DIR/PID-t.vmdk.1', fmt=vmdk size=0 compat6=off hwversion=undefined
-+            .name = BLOCK_OPT_PREALLOC,                                 \
-+            .type = QEMU_OPT_STRING,                                    \
+-Formatting 'TEST_DIR/PID-t.vmdk.2', fmt=vmdk size=0 compat6=off hwversion=undefined
-+            .help = "Preallocation mode (allowed values: off, "         \
-+                    "metadata, falloc, full)"                           \
+-Formatting 'TEST_DIR/PID-t.vmdk.3', fmt=vmdk size=0 compat6=off hwversion=undefined
-+        },                                                              \
-+        {                                                               \
+ == Missing extent ==
-+            .name = BLOCK_OPT_COMPRESSION_TYPE,                         \
 +            .type = QEMU_OPT_STRING,                                    \
 +            .help = "Compression method used for image cluster "        \
 +                    "compression",                                      \
 +            .def_value_str = "zlib"                                     \
 +        },
          QCOW_COMMON_OPTIONS,
          { /* end of list */ }
      }
 diff --git a/qemu-img.c b/qemu-img.c
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-img.c
 +++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int print_amend_option_help(const char *format)
      /* Every driver supporting amendment must have amend_opts */
      assert(drv->amend_opts);
 -    printf("Creation options for '%s':\n", format);
 +    printf("Amend options for '%s':\n", format);
      qemu_opts_print_help(drv->amend_opts, false);
 -    printf("\nNote that not all of these options may be amendable.\n");
      return 0;
  }
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
      amend_opts = qemu_opts_append(amend_opts, bs->drv->amend_opts);
      opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
      qemu_opts_do_parse(opts, options, NULL, &err);
 +
      if (err) {
 +        /* Try to parse options using the create options */
 +        Error *err1 = NULL;
 +        amend_opts = qemu_opts_append(amend_opts, bs->drv->create_opts);
 +        qemu_opts_del(opts);
 +        opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
 +        qemu_opts_do_parse(opts, options, NULL, &err1);
 +
 +        if (!err1) {
 +            error_append_hint(&err,
 +                              "This option is only supported for image creation\n");
 +        } else {
 +            error_free(err1);
 +        }
 +
          error_report_err(err);
          ret = -1;
          goto out;
 diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/049.out
 +++ b/tests/qemu-iotests/049.out
@@ -XXX,XX +XXX,XX @@ QA output created by 049
  == 1. Traditional size parameter ==
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1k
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1K
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1G
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1T
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0b
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5k
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5K
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5G
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5T
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
  == 2. Specifying size via -o ==
  qemu-img create -f qcow2 -o size=1024 TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1024b TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1k TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1K TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1M TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1G TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1T TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1024.0 TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1024.0b TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1.5k TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1.5K TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1.5M TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1.5G TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o size=1.5T TEST_DIR/t.qcow2
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
  == 3. Invalid sizes ==
@@ -XXX,XX +XXX,XX @@ qemu-img: TEST_DIR/t.qcow2: The image size must be specified only once
  == Check correct interpretation of suffixes for cluster size ==
  qemu-img create -f qcow2 -o cluster_size=1024 TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=1024b TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=1k TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=1K TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=1M TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1048576 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1048576 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=1024.0 TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=1024.0b TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=0.5k TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=0.5K TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o cluster_size=0.5M TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=524288 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=524288 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  == Check compat level option ==
  qemu-img create -f qcow2 -o compat=0.10 TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o compat=1.1 TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o compat=0.42 TEST_DIR/t.qcow2 64M
  qemu-img: TEST_DIR/t.qcow2: Invalid parameter '0.42'
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.42 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.42 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o compat=foobar TEST_DIR/t.qcow2 64M
  qemu-img: TEST_DIR/t.qcow2: Invalid parameter 'foobar'
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=foobar cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=foobar lazy_refcounts=off refcount_bits=16
  == Check preallocation option ==
  qemu-img create -f qcow2 -o preallocation=off TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o preallocation=metadata TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=metadata compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o preallocation=1234 TEST_DIR/t.qcow2 64M
  qemu-img: TEST_DIR/t.qcow2: Invalid parameter '1234'
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=1234 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=1234 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  == Check encryption option ==
  qemu-img create -f qcow2 -o encryption=off TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=off cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=on encrypt.key-secret=sec0 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  == Check lazy_refcounts option (only with v3) ==
  qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=off TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=on refcount_bits=16
  qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=off TEST_DIR/t.qcow2 64M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
  qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=on TEST_DIR/t.qcow2 64M
  qemu-img: TEST_DIR/t.qcow2: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater)
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=on refcount_bits=16
  *** done
 diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/061.out
 +++ b/tests/qemu-iotests/061.out
@@ -XXX,XX +XXX,XX @@ qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (
  qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater)
  qemu-img: Unknown compatibility level 0.42
  qemu-img: Invalid parameter 'foo'
 -qemu-img: Changing the cluster size is not supported
 -qemu-img: Changing the encryption flag is not supported
 -qemu-img: Cannot change preallocation mode
 +qemu-img: Invalid parameter 'cluster_size'
 +This option is only supported for image creation
 +qemu-img: Invalid parameter 'encryption'
 +This option is only supported for image creation
 +qemu-img: Invalid parameter 'preallocation'
 +This option is only supported for image creation
  === Testing correct handling of unset value ===
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  Should work:
  Should not work:
 -qemu-img: Changing the cluster size is not supported
 +qemu-img: Invalid parameter 'cluster_size'
 +This option is only supported for image creation
  === Testing zero expansion on inactive clusters ===
 diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/082.out
 +++ b/tests/qemu-iotests/082.out
@@ -XXX,XX +XXX,XX @@ QA output created by 082
  === create: Options specified more than once ===
  Testing: create -f foo -f qcow2 TEST_DIR/t.qcow2 128M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
  image: TEST_DIR/t.IMGFMT
  file format: IMGFMT
  virtual size: 128 MiB (134217728 bytes)
  cluster_size: 65536
  Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on TEST_DIR/t.qcow2 128M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=4096 lazy_refcounts=on refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=4096 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
  image: TEST_DIR/t.IMGFMT
  file format: IMGFMT
  virtual size: 128 MiB (134217728 bytes)
@@ -XXX,XX +XXX,XX @@ Format specific information:
      corrupt: false
  Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 128M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=on refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
  image: TEST_DIR/t.IMGFMT
  file format: IMGFMT
  virtual size: 128 MiB (134217728 bytes)
@@ -XXX,XX +XXX,XX @@ Format specific information:
      corrupt: false
  Testing: create -f qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 128M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
  image: TEST_DIR/t.IMGFMT
  file format: IMGFMT
  virtual size: 128 MiB (134217728 bytes)
@@ -XXX,XX +XXX,XX @@ Supported options:
    size=<size>            - Virtual disk size
  Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 128M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,help cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,help lazy_refcounts=off refcount_bits=16
  Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 128M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,? cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,? lazy_refcounts=off refcount_bits=16
  Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2, -o help TEST_DIR/t.qcow2 128M
  qemu-img: Invalid option list: backing_file=TEST_DIR/t.qcow2,
@@ -XXX,XX +XXX,XX @@ qemu-img: Format driver 'bochs' does not support image creation
  === convert: Options specified more than once ===
  Testing: create -f qcow2 TEST_DIR/t.qcow2 128M
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
  Testing: convert -f foo -f qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base
  image: TEST_DIR/t.IMGFMT.base
@@ -XXX,XX +XXX,XX @@ cluster_size: 65536
  === amend: help for -o ===
  Testing: amend -f qcow2 -o help TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o ? TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o cluster_size=4k,help TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o cluster_size=4k,? TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o help,cluster_size=4k TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o ?,cluster_size=4k TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o cluster_size=4k -o help TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o cluster_size=4k -o ? TEST_DIR/t.qcow2
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2
  Testing: rebase -u -b  -f qcow2 TEST_DIR/t.qcow2
@@ -XXX,XX +XXX,XX @@ Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2 -o ,, -o help TEST_DIR/
  qemu-img: Invalid option list: ,,
  Testing: amend -f qcow2 -o help
 -Creation options for 'qcow2':
 +Amend options for 'qcow2':
    backing_file=<str>     - File name of a base image
    backing_fmt=<str>      - Image format of the base image
 -  cluster_size=<size>    - qcow2 cluster size
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
 -  compression_type=<str> - Compression method used for image cluster compression
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 -  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
 -  encrypt.cipher-mode=<str> - Name of encryption cipher mode
 -  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
 -  encrypt.hash-alg=<str> - Name of encryption hash algorithm
 -  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 -  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
 -  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
 -  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
 -  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
 -  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 -Note that not all of these options may be amendable.
 -
  Testing: amend -o help
  qemu-img: Expecting one image file name
 diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/085.out
 +++ b/tests/qemu-iotests/085.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728
  === Create a single snapshot on virtio0 ===
  { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/1-snapshot-v0.IMGFMT', 'format': 'IMGFMT' } }
 -Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  === Invalid command - missing device and nodename ===
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file
  === Create several transactional group snapshots ===
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/2-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/2-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/3-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/3-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/4-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/4-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/5-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/5-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/6-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/6-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/7-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/7-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/8-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/8-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/9-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/9-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/10-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/10-snapshot-v1.IMGFMT' } } ] } }
 -Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 -Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  === Create a couple of snapshots using blockdev-snapshot ===
 diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/144.out
 +++ b/tests/qemu-iotests/144.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912
  { 'execute': 'qmp_capabilities' }
  {"return": {}}
  { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp.IMGFMT', 'format': 'IMGFMT' } }
 -Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  === Performing block-commit on active layer ===
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/
  === Performing Live Snapshot 2 ===
  { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp2.IMGFMT', 'format': 'IMGFMT' } }
 -Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  *** done
 diff --git a/tests/qemu-iotests/182.out b/tests/qemu-iotests/182.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/182.out
 +++ b/tests/qemu-iotests/182.out
@@ -XXX,XX +XXX,XX @@ Is another process using the image [TEST_DIR/t.qcow2]?
  {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node0', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
  {"return": {}}
  {'execute': 'blockdev-snapshot-sync', 'arguments': { 'node-name': 'node0', 'snapshot-file': 'TEST_DIR/t.IMGFMT.overlay', 'snapshot-node-name': 'node1' } }
 -Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16
  {"return": {}}
  {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node1', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
  {"return": {}}
 diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/185.out
 +++ b/tests/qemu-iotests/185.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
  === Creating backing chain ===
  { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT.mid', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
 -Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  { 'execute': 'human-monitor-command', 'arguments': { 'command-line': 'qemu-io disk "write 0 4M"' } }
  wrote 4194304/4194304 bytes at offset 0
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  {"return": ""}
  { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
  {"return": {}}
  === Start commit job and exit qemu ===
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q
  { 'execute': 'qmp_capabilities' }
  {"return": {}}
  { 'execute': 'drive-mirror', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
 -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
  {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
  {"return": {}}
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l
  { 'execute': 'qmp_capabilities' }
  {"return": {}}
  { 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
 -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
  {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
  {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}}
 diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/255.out
 +++ b/tests/qemu-iotests/255.out
 @@ -XXX,XX +XXX,XX @@ Finishing a commit job with background reads
  === Create backing chain and start VM ===
--Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
  === Start background read requests ===
 @@ -XXX,XX +XXX,XX @@ Closing the VM while a job is being cancelled
  === Create images and start VM ===
--Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
  wrote 1048576/1048576 bytes at offset 0
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/274.out
 +++ b/tests/qemu-iotests/274.out
 @@ -XXX,XX +XXX,XX @@
  == Commit tests ==
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
--Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
  wrote 2097152/2097152 bytes at offset 0
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  === Testing HMP commit (top -> mid) ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
--Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
  wrote 2097152/2097152 bytes at offset 0
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  === Testing QMP active commit (top -> mid) ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
--Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
  wrote 2097152/2097152 bytes at offset 0
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
+MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing qemu-img commit (top -> base) ===
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+ wrote 2097152/2097152 bytes at offset 0
+MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
+MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing QMP active commit (top -> base) ===
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+ wrote 2097152/2097152 bytes at offset 0
+MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
  == Resize tests ==
  === preallocation=off ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=6442450944 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=1073741824 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
  wrote 65536/65536 bytes at offset 5368709120
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 5368709120
- { "start": 1073741824, "length": 7516192768, "depth": 0, "zero": true, "data": false}]
+ { "start": 1073741824, "length": 7516192768, "depth": 0, "present": true, "zero": true, "data": false}]
  === preallocation=metadata ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=32212254720 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
  wrote 65536/65536 bytes at offset 33285996544
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 33285996544
- { "start": 34896609280, "length": 536870912, "depth": 0, "zero": true, "data": false, "offset": 2685075456}]
+ { "start": 34896609280, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 2685075456}]
  === preallocation=falloc ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=5242880 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
  wrote 65536/65536 bytes at offset 9437184
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 9437184
- { "start": 5242880, "length": 10485760, "depth": 0, "zero": false, "data": true, "offset": 327680}]
+ { "start": 5242880, "length": 10485760, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}]
  === preallocation=full ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=16777216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=8388608 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
  wrote 65536/65536 bytes at offset 11534336
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 11534336
- { "start": 8388608, "length": 4194304, "depth": 0, "zero": false, "data": true, "offset": 327680}]
+ { "start": 8388608, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}]
  === preallocation=off ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=393216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=259072 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
  wrote 65536/65536 bytes at offset 259072
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 259072
- { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
+ { "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}]
  === preallocation=off ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=409600 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
  wrote 65536/65536 bytes at offset 344064
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 @@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 344064
- { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
+ { "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}]
  === preallocation=off ===
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=524288 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
-+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
+-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
 +Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
  wrote 65536/65536 bytes at offset 446464
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/280.out
 +++ b/tests/qemu-iotests/280.out
 @@ -XXX,XX +XXX,XX @@
--Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
  === Launch VM ===
  Enabling migration QMP events on VM...
+diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/296.out
++++ b/tests/qemu-iotests/296.out
+@@ -XXX,XX +XXX,XX @@
+-Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+ {"execute": "job-dismiss", "arguments": {"id": "job0"}}
+ {"return": {}}
+@@ -XXX,XX +XXX,XX @@ Job failed: Failed to get shared "consistent read" lock
+ qemu-img: Failed to get shared "consistent read" lock
+ Is another process using the image [TEST_DIR/test.img]?
+-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+-
++.
+ Job failed: Block node is read-only
+ {"execute": "job-dismiss", "arguments": {"id": "job0"}}
+ {"return": {}}
+@@ -XXX,XX +XXX,XX @@ Job failed: Failed to get shared "consistent read" lock
+ {"return": {}}
+ {"execute": "job-dismiss", "arguments": {"id": "job0"}}
+ {"return": {}}
+-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+-
++.
+ {"return": {}}
+ {"error": {"class": "GenericError", "desc": "Failed to get \"write\" lock"}}
+-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+-
++.
+ {"return": {}}
+ {"return": {}}
+ .
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/iotests.py
++++ b/tests/qemu-iotests/iotests.py
+@@ -XXX,XX +XXX,XX @@ def qemu_tool_popen(args: Sequence[str],
+ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
+-                              connect_stderr: bool = True) -> Tuple[str, int]:
++                              connect_stderr: bool = True,
++                              drop_successful_output: bool = False) \
++        -> Tuple[str, int]:
+     """
+     Run a tool and return both its output and its exit code
+     """
+@@ -XXX,XX +XXX,XX @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
+             cmd = ' '.join(args)
+             sys.stderr.write(f'{tool} received signal \
+                                {-subp.returncode}: {cmd}\n')
++        if drop_successful_output and subp.returncode == 0:
++            output = ''
+         return (output, subp.returncode)
+ def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
+@@ -XXX,XX +XXX,XX @@ def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]:
+     """
+     Run qemu-img and return both its output and its exit code
+     """
++    is_create = bool(args and args[0] == 'create')
+     full_args = qemu_img_args + qemu_img_create_prepare_args(list(args))
+-    return qemu_tool_pipe_and_status('qemu-img', full_args)
++    return qemu_tool_pipe_and_status('qemu-img', full_args,
++                                     drop_successful_output=is_create)
+ def qemu_img(*args: str) -> int:
+     '''Run qemu-img and return the exit code'''
 --
-.26.2
+.34.1

-[PULL 18/18] iotests: don't test qcow2.py inside 291
+[PULL 12/24] iotests.py: filter compression type out
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-c6bee534ec3b added testing of qcow2.py into 291, and it breaks 291
+We want iotests pass with both the default zlib compression and with
-with external data file. Actually, 291 is bad place for qcow2.py
+IMGOPTS='compression_type=zstd'.
 testing, better add a separate test.
-For now, drop qcow2.py testing from 291 to fix the regression.
+Actually the only test that is interested in real compression type in
 test output is 287 (test for qcow2 compression type) and it's in bash.
 So for now we can safely filter out compression type in all qcow2
 tests.
-Fixes: 820c6bee534ec3b
-Reported-by: Max Reitz <mreitz@redhat.com>
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-Id: <20200618154052.8629-1-vsementsov@virtuozzo.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Message-Id: <20211223160144.1097696-11-vsementsov@virtuozzo.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- tests/qemu-iotests/291     |  4 ----
+ tests/qemu-iotests/206.out    | 10 +++++-----
- tests/qemu-iotests/291.out | 33 ---------------------------------
+ tests/qemu-iotests/242.out    | 10 +++++-----
-files changed, 37 deletions(-)
+ tests/qemu-iotests/274.out    | 10 +++++-----
  tests/qemu-iotests/iotests.py |  2 ++
 files changed, 17 insertions(+), 15 deletions(-)
-diff --git a/tests/qemu-iotests/291 b/tests/qemu-iotests/291
+diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/291
 +++ b/tests/qemu-iotests/291
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c 'w 1M 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io
  $QEMU_IMG bitmap --disable -f $IMGFMT "$TEST_IMG" b1
  $QEMU_IMG bitmap --enable -f $IMGFMT "$TEST_IMG" b2
  $QEMU_IO -c 'w 2M 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io
 -echo "Check resulting qcow2 header extensions:"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header-exts
  echo
  echo "=== Bitmap preservation not possible to non-qcow2 ==="
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG bitmap --merge tmp -f $IMGFMT "$TEST_IMG" b0
  $QEMU_IMG bitmap --remove --image-opts \
      driver=$IMGFMT,file.driver=file,file.filename="$TEST_IMG" tmp
  _img_info --format-specific
 -echo "Check resulting qcow2 header extensions:"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header-exts
  echo
  echo "=== Check bitmap contents ==="
 diff --git a/tests/qemu-iotests/291.out b/tests/qemu-iotests/291.out
 index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/291.out
+--- a/tests/qemu-iotests/206.out
-+++ b/tests/qemu-iotests/291.out
++++ b/tests/qemu-iotests/206.out
-@@ -XXX,XX +XXX,XX @@ wrote 1048576/1048576 bytes at offset 1048576
+@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ cluster_size: 65536
- wrote 1048576/1048576 bytes at offset 2097152
+ Format specific information:
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+     compat: 1.1
--Check resulting qcow2 header extensions:
+-    compression type: zlib
--Header extension:
++    compression type: COMPRESSION_TYPE
--magic                     0xe2792aca (Backing format)
+     lazy refcounts: false
 -length                    5
 -data                      'qcow2'
 -
 -Header extension:
 -magic                     0x6803f857 (Feature table)
 -length                    336
 -data                      <binary>
 -
 -Header extension:
 -magic                     0x23852875 (Bitmaps)
 -length                    24
 -nb_bitmaps                2
 -reserved32                0
 -bitmap_directory_size     0x40
 -bitmap_directory_offset   0x510000
 -
  === Bitmap preservation not possible to non-qcow2 ===
@@ -XXX,XX +XXX,XX @@ Format specific information:
              granularity: 65536
      refcount bits: 16
      corrupt: false
--Check resulting qcow2 header extensions:
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
--Header extension:
+ cluster_size: 65536
--magic                     0x6803f857 (Feature table)
+ Format specific information:
--length                    336
+     compat: 1.1
--data                      <binary>
+-    compression type: zlib
--
++    compression type: COMPRESSION_TYPE
--Header extension:
+     lazy refcounts: false
--magic                     0x23852875 (Bitmaps)
+     refcount bits: 16
--length                    24
+     corrupt: false
--nb_bitmaps                3
+@@ -XXX,XX +XXX,XX @@ virtual size: 32 MiB (33554432 bytes)
--reserved32                0
+ cluster_size: 2097152
--bitmap_directory_size     0x60
+ Format specific information:
--bitmap_directory_offset   0x520000
+     compat: 1.1
--
+-    compression type: zlib
++    compression type: COMPRESSION_TYPE
- === Check bitmap contents ===
+     lazy refcounts: true
      refcount bits: 1
      corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_IMG.base
  backing file format: IMGFMT
  Format specific information:
      compat: 0.10
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      refcount bits: 16
  === Successful image creation (encrypted) ===
@@ -XXX,XX +XXX,XX @@ encrypted: yes
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      encrypt:
 diff --git a/tests/qemu-iotests/242.out b/tests/qemu-iotests/242.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/242.out
 +++ b/tests/qemu-iotests/242.out
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      bitmaps:
          [0]:
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      bitmaps:
          [0]:
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      bitmaps:
          [0]:
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      bitmaps:
          [0]:
 diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/274.out
 +++ b/tests/qemu-iotests/274.out
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
  backing file format: IMGFMT
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
  backing file format: IMGFMT
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
  backing file format: IMGFMT
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 2 MiB (2097152 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
  backing file format: IMGFMT
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
 diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/iotests.py
 +++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@ def filter_img_info(output, filename):
                        'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX',
                        line)
          line = re.sub('cid: [0-9]+', 'cid: XXXXXXXXXX', line)
 +        line = re.sub('(compression type: )(zlib|zstd)', r'\1COMPRESSION_TYPE',
 +                      line)
          lines.append(line)
      return '\n'.join(lines)
 --
-.26.2
+.34.1

-[PULL 03/18] block/block-copy: block_copy_dirty_clusters: fix failure check
+[PULL 13/24] iotest 302: use img_info_log() helper
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-ret may be > 0 on success path at this point. Fix assertion, which may
+Instead of qemu_img_log("info", ..) use generic helper img_info_log().
 crash currently.
-Fixes: 4ce5dd3e9b5ee0fac18625860eb3727399ee965e
+img_info_log() has smarter logic. For example it use filter_img_info()
 to filter output, which in turns filter a compression type. So it will
 help us in future when we implement a possibility to use zstd
 compression by default (with help of some runtime config file or maybe
 build option). For now to test you should recompile qemu with a small
 addition into block/qcow2.c before
 "if (qcow2_opts->has_compression_type":
     if (!qcow2_opts->has_compression_type && version >= 3) {
         qcow2_opts->has_compression_type = true;
         qcow2_opts->compression_type = QCOW2_COMPRESSION_TYPE_ZSTD;
     }
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-Id: <20200526181347.489557-1-vsementsov@virtuozzo.com>
+Reviewed-by: Max Reitz <mreitz@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Message-Id: <20211223160144.1097696-12-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- block/block-copy.c | 4 +++-
+ tests/qemu-iotests/302     | 4 +++-
-file changed, 3 insertions(+), 1 deletion(-)
+ tests/qemu-iotests/302.out | 7 +++----
 files changed, 6 insertions(+), 5 deletions(-)
-diff --git a/block/block-copy.c b/block/block-copy.c
+diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/302
 +++ b/tests/qemu-iotests/302
@@ -XXX,XX +XXX,XX @@ from iotests import (
      qemu_img_measure,
      qemu_io,
      qemu_nbd_popen,
 +    img_info_log,
  )
  iotests.script_initialize(supported_fmts=["qcow2"])
@@ -XXX,XX +XXX,XX @@ with tarfile.open(tar_file, "w") as tar:
              tar_file):
          iotests.log("=== Target image info ===")
 +        # Not img_info_log as it enforces imgfmt, but now we print info on raw
          qemu_img_log("info", nbd_uri)
          qemu_img(
@@ -XXX,XX +XXX,XX @@ with tarfile.open(tar_file, "w") as tar:
              nbd_uri)
          iotests.log("=== Converted image info ===")
 -        qemu_img_log("info", nbd_uri)
 +        img_info_log(nbd_uri)
          iotests.log("=== Converted image check ===")
          qemu_img_log("check", nbd_uri)
 diff --git a/tests/qemu-iotests/302.out b/tests/qemu-iotests/302.out
 index XXXXXXX..XXXXXXX 100644
---- a/block/block-copy.c
+--- a/tests/qemu-iotests/302.out
-+++ b/block/block-copy.c
++++ b/tests/qemu-iotests/302.out
-@@ -XXX,XX +XXX,XX @@ out:
+@@ -XXX,XX +XXX,XX @@ virtual size: 448 KiB (458752 bytes)
-          * block_copy_task_run. If it fails, it means some task already failed
+ disk size: unavailable
-          * for real reason, let's return first failure.
-          * Still, assert that we don't rewrite failure by success.
+ === Converted image info ===
-+         *
+-image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock
-+         * Note: ret may be positive here because of block-status result.
+-file format: qcow2
-          */
++image: TEST_IMG
--        assert(ret == 0 || aio_task_pool_status(aio) < 0);
++file format: IMGFMT
-+        assert(ret >= 0 || aio_task_pool_status(aio) < 0);
+ virtual size: 1 GiB (1073741824 bytes)
-         ret = aio_task_pool_status(aio);
+-disk size: unavailable
+ cluster_size: 65536
-         aio_task_pool_free(aio);
+ Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
 --
-.26.2
+.34.1

-[PULL 11/18] block/qcow2: extend qemu-img amend interface with crypto options
+[PULL 14/24] qcow2: simple case support for downgrading of qcow2 images with zstd
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Now that we have all the infrastructure in place,
+If image doesn't have any compressed cluster we can easily switch to
-wire it in the qcow2 driver and expose this to the user.
+zlib compression, which may allow to downgrade the image.
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+That's mostly needed to support IMGOPTS='compression_type=zstd' in some
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+iotests which do qcow2 downgrade.
 While being here also fix checkpatch complain against '#' in printf
 formatting.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
-Message-Id: <20200608094030.670121-9-mlevitsk@redhat.com>
+Message-Id: <20211223160144.1097696-13-vsementsov@virtuozzo.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- block/qcow2.c              | 71 +++++++++++++++++++++++++++++++++-----
+ block/qcow2.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++--
- tests/qemu-iotests/082.out | 45 ++++++++++++++++++++++++
+file changed, 56 insertions(+), 2 deletions(-)
 files changed, 107 insertions(+), 9 deletions(-)
 diff --git a/block/qcow2.c b/block/qcow2.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qcow2.c
 +++ b/block/qcow2.c
-@@ -XXX,XX +XXX,XX @@ static ssize_t qcow2_crypto_hdr_write_func(QCryptoBlock *block, size_t offset,
+@@ -XXX,XX +XXX,XX @@ static int qcow2_load_vmstate(BlockDriverState *bs, QEMUIOVector *qiov,
-     return ret;
+     return bs->drv->bdrv_co_preadv_part(bs, offset, qiov->size, qiov, 0, 0);
  }
-+static QDict*
++static int qcow2_has_compressed_clusters(BlockDriverState *bs)
 +qcow2_extract_crypto_opts(QemuOpts *opts, const char *fmt, Error **errp)
 +{
-+    QDict *cryptoopts_qdict;
++    int64_t offset = 0;
-+    QDict *opts_qdict;
++    int64_t bytes = bdrv_getlength(bs);
 +
-+    /* Extract "encrypt." options into a qdict */
++    if (bytes < 0) {
-+    opts_qdict = qemu_opts_to_qdict(opts, NULL);
++        return bytes;
-+    qdict_extract_subqdict(opts_qdict, &cryptoopts_qdict, "encrypt.");
++    }
 +    qobject_unref(opts_qdict);
 +    qdict_put_str(cryptoopts_qdict, "format", fmt);
 +    return cryptoopts_qdict;
 +}
  /*
   * read qcow2 extension and fill bs
@@ -XXX,XX +XXX,XX @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
      if (has_luks) {
          g_autoptr(QCryptoBlockCreateOptions) create_opts = NULL;
 -        QDict *opts_qdict;
 -        QDict *cryptoopts;
 +        QDict *cryptoopts = qcow2_extract_crypto_opts(opts, "luks", errp);
          size_t headerlen;
 -        opts_qdict = qemu_opts_to_qdict(opts, NULL);
 -        qdict_extract_subqdict(opts_qdict, &cryptoopts, "encrypt.");
 -        qobject_unref(opts_qdict);
 -
 -        qdict_put_str(cryptoopts, "format", "luks");
 -
          create_opts = block_crypto_create_opts_init(cryptoopts, errp);
          qobject_unref(cryptoopts);
          if (!create_opts) {
@@ -XXX,XX +XXX,XX @@ typedef enum Qcow2AmendOperation {
      QCOW2_NO_OPERATION = 0,
      QCOW2_UPGRADING,
 +    QCOW2_UPDATING_ENCRYPTION,
      QCOW2_CHANGING_REFCOUNT_ORDER,
      QCOW2_DOWNGRADING,
  } Qcow2AmendOperation;
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
      int ret;
      QemuOptDesc *desc = opts->list->desc;
      Qcow2AmendHelperCBInfo helper_cb_info;
 +    bool encryption_update = false;
      while (desc && desc->name) {
          if (!qemu_opt_find(opts, desc->name)) {
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
              backing_file = qemu_opt_get(opts, BLOCK_OPT_BACKING_FILE);
          } else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FMT)) {
              backing_format = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT);
 +        } else if (g_str_has_prefix(desc->name, "encrypt.")) {
 +            if (!s->crypto) {
 +                error_setg(errp,
 +                           "Can't amend encryption options - encryption not present");
 +                return -EINVAL;
 +            }
 +            if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
 +                error_setg(errp,
 +                           "Only LUKS encryption options can be amended");
 +                return -ENOTSUP;
 +            }
 +            encryption_update = true;
          } else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) {
              lazy_refcounts = qemu_opt_get_bool(opts, BLOCK_OPT_LAZY_REFCOUNTS,
                                                 lazy_refcounts);
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
          .original_status_cb = status_cb,
          .original_cb_opaque = cb_opaque,
          .total_operations = (new_version != old_version)
 -                          + (s->refcount_bits != refcount_bits)
 +                          + (s->refcount_bits != refcount_bits) +
 +                            (encryption_update == true)
      };
      /* Upgrade first (some features may require compat=1.1) */
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
          }
      }
 +    if (encryption_update) {
 +        QDict *amend_opts_dict;
 +        QCryptoBlockAmendOptions *amend_opts;
 +
-+        helper_cb_info.current_operation = QCOW2_UPDATING_ENCRYPTION;
++    while (bytes != 0) {
-+        amend_opts_dict = qcow2_extract_crypto_opts(opts, "luks", errp);
++        int ret;
-+        if (!amend_opts_dict) {
++        QCow2SubclusterType type;
-+            return -EINVAL;
++        unsigned int cur_bytes = MIN(INT_MAX, bytes);
-+        }
++        uint64_t host_offset;
-+        amend_opts = block_crypto_amend_opts_init(amend_opts_dict, errp);
++
-+        qobject_unref(amend_opts_dict);
++        ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset,
-+        if (!amend_opts) {
++                                    &type);
 +            return -EINVAL;
 +        }
 +        ret = qcrypto_block_amend_options(s->crypto,
 +                                          qcow2_crypto_hdr_read_func,
 +                                          qcow2_crypto_hdr_write_func,
 +                                          bs,
 +                                          amend_opts,
 +                                          force,
 +                                          errp);
 +        qapi_free_QCryptoBlockAmendOptions(amend_opts);
 +        if (ret < 0) {
 +            return ret;
 +        }
++
++        if (type == QCOW2_SUBCLUSTER_COMPRESSED) {
++            return 1;
++        }
++
++        offset += cur_bytes;
++        bytes -= cur_bytes;
 +    }
 +
-     if (s->refcount_bits != refcount_bits) {
++    return 0;
-         int refcount_order = ctz32(refcount_bits);
++}
++
-@@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_amend_opts = {
+ /*
-     .name = "qcow2-amend-opts",
+  * Downgrades an image's version. To achieve this, any incompatible features
-     .head = QTAILQ_HEAD_INITIALIZER(qcow2_amend_opts.head),
+  * have to be removed.
-     .desc = {
+@@ -XXX,XX +XXX,XX @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_STATE("encrypt."),
+      * the first place; if that happens nonetheless, returning -ENOTSUP is the
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT("encrypt."),
+      * best thing to do anyway */
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET("encrypt."),
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET("encrypt."),
+-    if (s->incompatible_features) {
-+        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),
++    if (s->incompatible_features & ~QCOW2_INCOMPAT_COMPRESSION) {
-         QCOW_COMMON_OPTIONS,
+         error_setg(errp, "Cannot downgrade an image with incompatible features "
-         { /* end of list */ }
+-                   "%#" PRIx64 " set", s->incompatible_features);
 +                   "0x%" PRIx64 " set",
 +                   s->incompatible_features & ~QCOW2_INCOMPAT_COMPRESSION);
          return -ENOTSUP;
      }
-diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
-index XXXXXXX..XXXXXXX 100644
+@@ -XXX,XX +XXX,XX @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
---- a/tests/qemu-iotests/082.out
+         return ret;
-+++ b/tests/qemu-iotests/082.out
+     }
-@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
-   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
++    if (s->incompatible_features & QCOW2_INCOMPAT_COMPRESSION) {
-   data_file=<str>        - File name of an external data file
++        ret = qcow2_has_compressed_clusters(bs);
-   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
++        if (ret < 0) {
-+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
++            error_setg(errp, "Failed to check block status");
-+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
++            return -EINVAL;
-+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
++        }
-+  encrypt.old-secret=<str> - Select all keyslots that match this password
++        if (ret) {
-+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
++            error_setg(errp, "Cannot downgrade an image with zstd compression "
-   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
++                       "type and existing compressed clusters");
-   refcount_bits=<num>    - Width of a reference count entry in bits
++            return -ENOTSUP;
-   size=<size>            - Virtual disk size
++        }
-@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
++        /*
-   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
++         * No compressed clusters for now, so just chose default zlib
-   data_file=<str>        - File name of an external data file
++         * compression.
-   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
++         */
-+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
++        s->incompatible_features &= ~QCOW2_INCOMPAT_COMPRESSION;
-+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
++        s->compression_type = QCOW2_COMPRESSION_TYPE_ZLIB;
-+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
++    }
-+  encrypt.old-secret=<str> - Select all keyslots that match this password
++
-+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
++    assert(s->incompatible_features == 0);
-   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
++
-   refcount_bits=<num>    - Width of a reference count entry in bits
+     s->qcow_version = target_version;
-   size=<size>            - Virtual disk size
+     ret = qcow2_update_header(bs);
-@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
+     if (ret < 0) {
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 +  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 +  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
 +  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
 +  encrypt.old-secret=<str> - Select all keyslots that match this password
 +  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 +  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 +  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
 +  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
 +  encrypt.old-secret=<str> - Select all keyslots that match this password
 +  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 +  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 +  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
 +  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
 +  encrypt.old-secret=<str> - Select all keyslots that match this password
 +  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 +  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 +  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
 +  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
 +  encrypt.old-secret=<str> - Select all keyslots that match this password
 +  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 +  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 +  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
 +  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
 +  encrypt.old-secret=<str> - Select all keyslots that match this password
 +  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 +  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 +  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
 +  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
 +  encrypt.old-secret=<str> - Select all keyslots that match this password
 +  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
    compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
    data_file=<str>        - File name of an external data file
    data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
 +  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
 +  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
 +  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
 +  encrypt.old-secret=<str> - Select all keyslots that match this password
 +  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
    lazy_refcounts=<bool (on/off)> - Postpone refcount updates
    refcount_bits=<num>    - Width of a reference count entry in bits
    size=<size>            - Virtual disk size
 --
-.26.2
+.34.1

-[PULL 15/18] block/qcow2: implement blockdev-amend
+[PULL 15/24] iotests/common.rc: introduce _qcow2_dump_header helper
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Currently the implementation only supports amending the encryption
+We'll use it in tests instead of explicit qcow2.py. Then we are going
-options, unlike the qemu-img version
+to add some filtering in _qcow2_dump_header.
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
-Message-Id: <20200608094030.670121-14-mlevitsk@redhat.com>
+Message-Id: <20211223160144.1097696-14-vsementsov@virtuozzo.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- qapi/block-core.json | 16 +++++++++++++++-
+ tests/qemu-iotests/common.rc | 10 ++++++++++
- block/qcow2.c        | 39 +++++++++++++++++++++++++++++++++++++++
+file changed, 10 insertions(+)
 files changed, 54 insertions(+), 1 deletion(-)
-diff --git a/qapi/block-core.json b/qapi/block-core.json
+diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
 index XXXXXXX..XXXXXXX 100644
---- a/qapi/block-core.json
+--- a/tests/qemu-iotests/common.rc
-+++ b/qapi/block-core.json
++++ b/tests/qemu-iotests/common.rc
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ _require_one_device_of()
-   'data': { }
+     _notrun "$* not available"
  }
-+##
++_qcow2_dump_header()
-+# @BlockdevAmendOptionsQcow2:
++{
-+#
++    img="$1"
-+# Driver specific image amend options for qcow2.
++    if [ -z "$img" ]; then
-+# For now, only encryption options can be amended
++        img="$TEST_IMG"
-+#
++    fi
 +# @encrypt          Encryption options to be amended
 +#
 +# Since: 5.1
 +##
 +{ 'struct': 'BlockdevAmendOptionsQcow2',
 +  'data': { '*encrypt':         'QCryptoBlockAmendOptions' } }
 +
- ##
++    $PYTHON qcow2.py "$img" dump-header
  # @BlockdevAmendOptions:
  #
@@ -XXX,XX +XXX,XX @@
        'driver':         'BlockdevDriver' },
    'discriminator': 'driver',
    'data': {
 -      'luks':           'BlockdevAmendOptionsLUKS' } }
 +      'luks':           'BlockdevAmendOptionsLUKS',
 +      'qcow2':          'BlockdevAmendOptionsQcow2' } }
  ##
  # @x-blockdev-amend:
 diff --git a/block/qcow2.c b/block/qcow2.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qcow2.c
 +++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
      return 0;
  }
 +static int coroutine_fn qcow2_co_amend(BlockDriverState *bs,
 +                                       BlockdevAmendOptions *opts,
 +                                       bool force,
 +                                       Error **errp)
 +{
 +    BlockdevAmendOptionsQcow2 *qopts = &opts->u.qcow2;
 +    BDRVQcow2State *s = bs->opaque;
 +    int ret = 0;
 +
 +    if (qopts->has_encrypt) {
 +        if (!s->crypto) {
 +            error_setg(errp, "image is not encrypted, can't amend");
 +            return -EOPNOTSUPP;
 +        }
 +
 +        if (qopts->encrypt->format != Q_CRYPTO_BLOCK_FORMAT_LUKS) {
 +            error_setg(errp,
 +                       "Amend can't be used to change the qcow2 encryption format");
 +            return -EOPNOTSUPP;
 +        }
 +
 +        if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
 +            error_setg(errp,
 +                       "Only LUKS encryption options can be amended for qcow2 with blockdev-amend");
 +            return -EOPNOTSUPP;
 +        }
 +
 +        ret = qcrypto_block_amend_options(s->crypto,
 +                                          qcow2_crypto_hdr_read_func,
 +                                          qcow2_crypto_hdr_write_func,
 +                                          bs,
 +                                          qopts->encrypt,
 +                                          force,
 +                                          errp);
 +    }
 +    return ret;
 +}
 +
- /*
+ # make sure this script returns success
-  * If offset or size are negative, respectively, they will not be included in
+ true
   * the BLOCK_IMAGE_CORRUPTED event emitted.
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
      .mutable_opts        = mutable_opts,
      .bdrv_co_check       = qcow2_co_check,
      .bdrv_amend_options  = qcow2_amend_options,
 +    .bdrv_co_amend       = qcow2_co_amend,
      .bdrv_detach_aio_context  = qcow2_detach_aio_context,
      .bdrv_attach_aio_context  = qcow2_attach_aio_context,
 --
-.26.2
+.34.1

-New patch
+[PULL 16/24] iotests: massive use _qcow2_dump_header
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 We are going to add filtering in _qcow2_dump_header and want all tests
 use it.
 The patch is generated by commands:
   cd tests/qemu-iotests
   sed -ie 's/$PYTHON qcow2.py "$TEST_IMG" dump-header\($\| \)/_qcow2_dump_header\1/' ??? tests/*
 (the difficulty is to avoid converting dump-header-exts)
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Message-Id: <20211223160144.1097696-15-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/031 |  6 +++---
  tests/qemu-iotests/036 |  6 +++---
  tests/qemu-iotests/039 | 20 ++++++++++----------
  tests/qemu-iotests/060 | 20 ++++++++++----------
  tests/qemu-iotests/061 | 36 ++++++++++++++++++------------------
  tests/qemu-iotests/137 |  2 +-
  tests/qemu-iotests/287 |  8 ++++----
 files changed, 49 insertions(+), 49 deletions(-)
 diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/031
 +++ b/tests/qemu-iotests/031
@@ -XXX,XX +XXX,XX @@ for compat in "compat=0.10" "compat=1.1"; do
      echo
      _make_test_img -o $compat 64M
      $PYTHON qcow2.py "$TEST_IMG" add-header-ext 0x12345678 "This is a test header extension"
 -    $PYTHON qcow2.py "$TEST_IMG" dump-header
 +    _qcow2_dump_header
      _check_test_img
      echo
      echo === Rewrite header with no backing file ===
      echo
      $QEMU_IMG rebase -u -b "" "$TEST_IMG"
 -    $PYTHON qcow2.py "$TEST_IMG" dump-header
 +    _qcow2_dump_header
      _check_test_img
      echo
      echo === Add a backing file and format ===
      echo
      $QEMU_IMG rebase -u -b "/some/backing/file/path" -F host_device "$TEST_IMG"
 -    $PYTHON qcow2.py "$TEST_IMG" dump-header
 +    _qcow2_dump_header
  done
  # success, all done
 diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/036
 +++ b/tests/qemu-iotests/036
@@ -XXX,XX +XXX,XX @@ $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 63
  # Without feature table
  $PYTHON qcow2.py "$TEST_IMG" del-header-ext 0x6803f857
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
 +_qcow2_dump_header | grep features
  $PYTHON qcow2.py "$TEST_IMG" dump-header-exts
  _img_info
@@ -XXX,XX +XXX,XX @@ echo === Create image with unknown autoclear feature bit ===
  echo
  _make_test_img 64M
  $PYTHON qcow2.py "$TEST_IMG" set-feature-bit autoclear 63
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
 +_qcow2_dump_header | grep features
  $PYTHON qcow2.py "$TEST_IMG" dump-header-exts
  echo
@@ -XXX,XX +XXX,XX @@ echo === Repair image ===
  echo
  _check_test_img -r all
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
 +_qcow2_dump_header | grep features
  $PYTHON qcow2.py "$TEST_IMG" dump-header-exts
  # success, all done
 diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/039
 +++ b/tests/qemu-iotests/039
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" $size
  $QEMU_IO -c "write -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io
  # The dirty bit must not be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  _check_test_img
  echo
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x5a 0 512" \
      | _filter_qemu_io
  # The dirty bit must be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  _check_test_img
  echo
@@ -XXX,XX +XXX,XX @@ echo "== Read-only access must still work =="
  $QEMU_IO -r -c "read -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io
  # The dirty bit must be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  echo
  echo "== Repairing the image file must succeed =="
@@ -XXX,XX +XXX,XX @@ echo "== Repairing the image file must succeed =="
  _check_test_img -r all
  # The dirty bit must not be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  echo
  echo "== Data should still be accessible after repair =="
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x5a 0 512" \
      | _filter_qemu_io
  # The dirty bit must be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  $QEMU_IO -c "write 0 512" "$TEST_IMG" | _filter_qemu_io
  # The dirty bit must not be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  echo
  echo "== Creating an image file with lazy_refcounts=off =="
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x5a 0 512" \
      | _filter_qemu_io
  # The dirty bit must not be set since lazy_refcounts=off
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  _check_test_img
  echo
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write 0 512" "$TEST_IMG" | _filter_qemu_io
  $QEMU_IMG commit "$TEST_IMG"
  # The dirty bit must not be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  $PYTHON qcow2.py "$TEST_IMG".base dump-header | grep incompatible_features
  _check_test_img
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "reopen -o lazy-refcounts=on" \
      | _filter_qemu_io
  # The dirty bit must be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  _check_test_img
  _make_test_img -o "compat=1.1,lazy_refcounts=on" $size
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "reopen -o lazy-refcounts=off" \
      | _filter_qemu_io
  # The dirty bit must not be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  _check_test_img
 diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/060
 +++ b/tests/qemu-iotests/060
@@ -XXX,XX +XXX,XX @@ poke_file "$TEST_IMG" "$l1_offset" "\x80\x00\x00\x00\x00\x03\x00\x00"
  _check_test_img
  # The corrupt bit should not be set anyway
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  # Try to write something, thereby forcing the corrupt bit to be set
  $QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
  # The corrupt bit must now be set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  # This information should be available through qemu-img info
  _img_info --format-specific
@@ -XXX,XX +XXX,XX @@ poke_file "$TEST_IMG" "$(($rb_offset+8))" "\x00\x01"
  # Redirect new data cluster onto refcount block
  poke_file "$TEST_IMG" "$l2_offset" "\x80\x00\x00\x00\x00\x02\x00\x00"
  _check_test_img
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  $QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  # Try to fix it
  _check_test_img -r all
  # The corrupt bit should be cleared
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  # Look if it's really really fixed
  $QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  echo
  echo "=== Testing cluster data reference into inactive L2 table ==="
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "$OPEN_RW" -c "write -P 2 0 512" | _filter_qemu_io
  poke_file "$TEST_IMG" "$l2_offset_after_snapshot" \
                        "\x80\x00\x00\x00\x00\x04\x00\x00"
  _check_test_img
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  $QEMU_IO -c "$OPEN_RW" -c "write -P 3 0 512" | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  _check_test_img -r all
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  $QEMU_IO -c "$OPEN_RW" -c "write -P 4 0 512" | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  # Check data
  $QEMU_IO -c "$OPEN_RO" -c "read -P 4 0 512" | _filter_qemu_io
 diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/061
 +++ b/tests/qemu-iotests/061
@@ -XXX,XX +XXX,XX @@ echo "=== Testing version downgrade with zero expansion ==="
  echo
  _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
  $QEMU_IO -c "write -z 0 128k" "$TEST_IMG" | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IO -c "read -P 0 0 128k" "$TEST_IMG" | _filter_qemu_io
  _check_test_img
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
  $QEMU_IO -c "write -z 0 128k" "$TEST_IMG" | _filter_qemu_io
  $QEMU_IO -c "write -z 32M 128k" "$TEST_IMG" | _filter_qemu_io
  $QEMU_IO -c map "$TEST_IMG" | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IMG amend -o "compat=0.10" --image-opts \
            driver=qcow2,file.filename=$TEST_IMG,l2-cache-entry-size=4096
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IO -c "read -P 0 0 128k" "$TEST_IMG" | _filter_qemu_io
  $QEMU_IO -c "read -P 0 32M 128k" "$TEST_IMG" | _filter_qemu_io
  $QEMU_IO -c map "$TEST_IMG" | _filter_qemu_io
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
  _NO_VALGRIND \
  $QEMU_IO -c "write -P 0x2a 0 128k" -c flush \
           -c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IO -c "read -P 0x2a 0 128k" "$TEST_IMG" | _filter_qemu_io
  _check_test_img
@@ -XXX,XX +XXX,XX @@ echo
  _make_test_img -o "compat=1.1" 64M
  $PYTHON qcow2.py "$TEST_IMG" set-feature-bit compatible 42
  $PYTHON qcow2.py "$TEST_IMG" set-feature-bit autoclear 42
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  _check_test_img
  echo
@@ -XXX,XX +XXX,XX @@ echo "=== Testing version upgrade and resize ==="
  echo
  _make_test_img -o "compat=0.10" 64M
  $QEMU_IO -c "write -P 0x2a 42M 64k" "$TEST_IMG" | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IMG amend -o "compat=1.1,lazy_refcounts=on,size=128M" "$TEST_IMG"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IO -c "read -P 0x2a 42M 64k" "$TEST_IMG" | _filter_qemu_io
  _check_test_img
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x2a 24M 64k" "$TEST_IMG" | _filter_qemu_io
  $QEMU_IMG snapshot -c foo "$TEST_IMG"
  $QEMU_IMG resize "$TEST_IMG" 64M &&
      echo "unexpected pass"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
 +_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
  $QEMU_IMG amend -o "compat=1.1,size=128M" "$TEST_IMG" ||
      echo "unexpected fail"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
 +_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
  $QEMU_IMG snapshot -c bar "$TEST_IMG"
  $QEMU_IMG resize --shrink "$TEST_IMG" 64M ||
      echo "unexpected fail"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
 +_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
  $QEMU_IMG amend -o "compat=0.10,size=32M" "$TEST_IMG" &&
      echo "unexpected pass"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
 +_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
  $QEMU_IMG snapshot -a bar "$TEST_IMG" ||
      echo "unexpected fail"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
 +_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
  $QEMU_IMG snapshot -d bar "$TEST_IMG"
  $QEMU_IMG amend -o "compat=0.10,size=32M" "$TEST_IMG" ||
      echo "unexpected fail"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^\(version\|size\|nb_snap\)'
 +_qcow2_dump_header | grep '^\(version\|size\|nb_snap\)'
  _check_test_img
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
  _NO_VALGRIND \
  $QEMU_IO -c "write -P 0x2a 0 128k" -c flush \
           -c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 | _filter_qemu_io
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IMG amend -o "lazy_refcounts=off" "$TEST_IMG"
 -$PYTHON qcow2.py "$TEST_IMG" dump-header
 +_qcow2_dump_header
  $QEMU_IO -c "read -P 0x2a 0 128k" "$TEST_IMG" | _filter_qemu_io
  _check_test_img
 diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/137
 +++ b/tests/qemu-iotests/137
@@ -XXX,XX +XXX,XX @@ $QEMU_IO \
  # The dirty bit must not be set
  # (Filter the external data file bit)
 -if $PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features \
 +if _qcow2_dump_header | grep incompatible_features \
      | grep -q '\<0\>'
  then
      echo 'ERROR: Dirty bit set'
 diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/287
 +++ b/tests/qemu-iotests/287
@@ -XXX,XX +XXX,XX @@ echo
  echo "=== Testing compression type incompatible bit setting for zlib ==="
  echo
  _make_test_img -o compression_type=zlib 64M
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  echo
  echo "=== Testing compression type incompatible bit setting for zstd ==="
  echo
  _make_test_img -o compression_type=zstd 64M
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  echo
  echo "=== Testing zlib with incompatible bit set ==="
@@ -XXX,XX +XXX,XX @@ echo
  _make_test_img -o compression_type=zlib 64M
  $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 3
  # to make sure the bit was actually set
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
      echo "Error: The image opened successfully. The image must not be opened."
@@ -XXX,XX +XXX,XX @@ echo
  _make_test_img -o compression_type=zstd 64M
  $PYTHON qcow2.py "$TEST_IMG" set-header incompatible_features 0
  # to make sure the bit was actually unset
 -$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
 +_qcow2_dump_header | grep incompatible_features
  if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
      echo "Error: The image opened successfully. The image must not be opened."
 --
 .34.1

-[PULL 07/18] block/amend: separate amend and create options for qemu-img
+[PULL 17/24] iotest 39: use _qcow2_dump_header
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Some options are only useful for creation
+_qcow2_dump_header has filter for compression type, so this change
-(or hard to be amended, like cluster size for qcow2), while some other
+makes test pass with IMGOPTS='compression_type=zstd'.
 options are only useful for amend, like upcoming keyslot management
 options for luks
-Since currently only qcow2 supports amend, move all its options
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-to a common macro and then include it in each action option list.
+Reviewed-by: Max Reitz <mreitz@redhat.com>
 Message-Id: <20211223160144.1097696-16-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/039 | 2 +-
 file changed, 1 insertion(+), 1 deletion(-)
-In future it might be useful to remove some options which are
+diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
-not supported anyway from amend list, which currently
+index XXXXXXX..XXXXXXX 100755
-cause an error message if amended.
+--- a/tests/qemu-iotests/039
++++ b/tests/qemu-iotests/039
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+@@ -XXX,XX +XXX,XX @@ $QEMU_IMG commit "$TEST_IMG"
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
-Reviewed-by: Max Reitz <mreitz@redhat.com>
+ # The dirty bit must not be set
-Message-Id: <20200608094030.670121-5-mlevitsk@redhat.com>
+ _qcow2_dump_header | grep incompatible_features
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+-$PYTHON qcow2.py "$TEST_IMG".base dump-header | grep incompatible_features
----
++_qcow2_dump_header "$TEST_IMG".base | grep incompatible_features
- include/block/block_int.h |   4 +
- block/qcow2.c             | 173 +++++++++++++++++++++-----------------
+ _check_test_img
- qemu-img.c                |  18 ++--
+ TEST_IMG="$TEST_IMG".base _check_test_img
 files changed, 107 insertions(+), 88 deletions(-)
 diff --git a/include/block/block_int.h b/include/block/block_int.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block_int.h
 +++ b/include/block/block_int.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
      /* List of options for creating images, terminated by name == NULL */
      QemuOptsList *create_opts;
 +
 +    /* List of options for image amend */
 +    QemuOptsList *amend_opts;
 +
      /*
       * If this driver supports reopening images this contains a
       * NULL-terminated list of the runtime options that can be
 diff --git a/block/qcow2.c b/block/qcow2.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qcow2.c
 +++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
      s->signaled_corruption = true;
  }
 +#define QCOW_COMMON_OPTIONS                                         \
 +    {                                                               \
 +        .name = BLOCK_OPT_SIZE,                                     \
 +        .type = QEMU_OPT_SIZE,                                      \
 +        .help = "Virtual disk size"                                 \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_COMPAT_LEVEL,                             \
 +        .type = QEMU_OPT_STRING,                                    \
 +        .help = "Compatibility level (v2 [0.10] or v3 [1.1])"       \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_BACKING_FILE,                             \
 +        .type = QEMU_OPT_STRING,                                    \
 +        .help = "File name of a base image"                         \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_BACKING_FMT,                              \
 +        .type = QEMU_OPT_STRING,                                    \
 +        .help = "Image format of the base image"                    \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_DATA_FILE,                                \
 +        .type = QEMU_OPT_STRING,                                    \
 +        .help = "File name of an external data file"                \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_DATA_FILE_RAW,                            \
 +        .type = QEMU_OPT_BOOL,                                      \
 +        .help = "The external data file must stay valid "           \
 +                "as a raw image"                                    \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_ENCRYPT,                                  \
 +        .type = QEMU_OPT_BOOL,                                      \
 +        .help = "Encrypt the image with format 'aes'. (Deprecated " \
 +                "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",    \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_ENCRYPT_FORMAT,                           \
 +        .type = QEMU_OPT_STRING,                                    \
 +        .help = "Encrypt the image, format choices: 'aes', 'luks'", \
 +    },                                                              \
 +    BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",                     \
 +        "ID of secret providing qcow AES key or LUKS passphrase"),  \
 +    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),               \
 +    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),              \
 +    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),                \
 +    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),           \
 +    BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),                 \
 +    BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),                \
 +    {                                                               \
 +        .name = BLOCK_OPT_CLUSTER_SIZE,                             \
 +        .type = QEMU_OPT_SIZE,                                      \
 +        .help = "qcow2 cluster size",                               \
 +        .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)            \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_PREALLOC,                                 \
 +        .type = QEMU_OPT_STRING,                                    \
 +        .help = "Preallocation mode (allowed values: off, "         \
 +                "metadata, falloc, full)"                           \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_LAZY_REFCOUNTS,                           \
 +        .type = QEMU_OPT_BOOL,                                      \
 +        .help = "Postpone refcount updates",                        \
 +        .def_value_str = "off"                                      \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_REFCOUNT_BITS,                            \
 +        .type = QEMU_OPT_NUMBER,                                    \
 +        .help = "Width of a reference count entry in bits",         \
 +        .def_value_str = "16"                                       \
 +    },                                                              \
 +    {                                                               \
 +        .name = BLOCK_OPT_COMPRESSION_TYPE,                         \
 +        .type = QEMU_OPT_STRING,                                    \
 +        .help = "Compression method used for image cluster "        \
 +                "compression",                                      \
 +        .def_value_str = "zlib"                                     \
 +    }
 +
  static QemuOptsList qcow2_create_opts = {
      .name = "qcow2-create-opts",
      .head = QTAILQ_HEAD_INITIALIZER(qcow2_create_opts.head),
      .desc = {
 -        {
 -            .name = BLOCK_OPT_SIZE,
 -            .type = QEMU_OPT_SIZE,
 -            .help = "Virtual disk size"
 -        },
 -        {
 -            .name = BLOCK_OPT_COMPAT_LEVEL,
 -            .type = QEMU_OPT_STRING,
 -            .help = "Compatibility level (v2 [0.10] or v3 [1.1])"
 -        },
 -        {
 -            .name = BLOCK_OPT_BACKING_FILE,
 -            .type = QEMU_OPT_STRING,
 -            .help = "File name of a base image"
 -        },
 -        {
 -            .name = BLOCK_OPT_BACKING_FMT,
 -            .type = QEMU_OPT_STRING,
 -            .help = "Image format of the base image"
 -        },
 -        {
 -            .name = BLOCK_OPT_DATA_FILE,
 -            .type = QEMU_OPT_STRING,
 -            .help = "File name of an external data file"
 -        },
 -        {
 -            .name = BLOCK_OPT_DATA_FILE_RAW,
 -            .type = QEMU_OPT_BOOL,
 -            .help = "The external data file must stay valid as a raw image"
 -        },
 -        {
 -            .name = BLOCK_OPT_ENCRYPT,
 -            .type = QEMU_OPT_BOOL,
 -            .help = "Encrypt the image with format 'aes'. (Deprecated "
 -                    "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",
 -        },
 -        {
 -            .name = BLOCK_OPT_ENCRYPT_FORMAT,
 -            .type = QEMU_OPT_STRING,
 -            .help = "Encrypt the image, format choices: 'aes', 'luks'",
 -        },
 -        BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",
 -            "ID of secret providing qcow AES key or LUKS passphrase"),
 -        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),
 -        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),
 -        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),
 -        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),
 -        BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),
 -        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),
 -        {
 -            .name = BLOCK_OPT_CLUSTER_SIZE,
 -            .type = QEMU_OPT_SIZE,
 -            .help = "qcow2 cluster size",
 -            .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)
 -        },
 -        {
 -            .name = BLOCK_OPT_PREALLOC,
 -            .type = QEMU_OPT_STRING,
 -            .help = "Preallocation mode (allowed values: off, metadata, "
 -                    "falloc, full)"
 -        },
 -        {
 -            .name = BLOCK_OPT_LAZY_REFCOUNTS,
 -            .type = QEMU_OPT_BOOL,
 -            .help = "Postpone refcount updates",
 -            .def_value_str = "off"
 -        },
 -        {
 -            .name = BLOCK_OPT_REFCOUNT_BITS,
 -            .type = QEMU_OPT_NUMBER,
 -            .help = "Width of a reference count entry in bits",
 -            .def_value_str = "16"
 -        },
 -        {
 -            .name = BLOCK_OPT_COMPRESSION_TYPE,
 -            .type = QEMU_OPT_STRING,
 -            .help = "Compression method used for image cluster compression",
 -            .def_value_str = "zlib"
 -        },
 +        QCOW_COMMON_OPTIONS,
 +        { /* end of list */ }
 +    }
 +};
 +
 +static QemuOptsList qcow2_amend_opts = {
 +    .name = "qcow2-amend-opts",
 +    .head = QTAILQ_HEAD_INITIALIZER(qcow2_amend_opts.head),
 +    .desc = {
 +        QCOW_COMMON_OPTIONS,
          { /* end of list */ }
      }
  };
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
      .bdrv_inactivate            = qcow2_inactivate,
      .create_opts         = &qcow2_create_opts,
 +    .amend_opts          = &qcow2_amend_opts,
      .strong_runtime_opts = qcow2_strong_runtime_opts,
      .mutable_opts        = mutable_opts,
      .bdrv_co_check       = qcow2_co_check,
 diff --git a/qemu-img.c b/qemu-img.c
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-img.c
 +++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int print_amend_option_help(const char *format)
          return 1;
      }
 -    /* Every driver supporting amendment must have create_opts */
 -    assert(drv->create_opts);
 +    /* Every driver supporting amendment must have amend_opts */
 +    assert(drv->amend_opts);
      printf("Creation options for '%s':\n", format);
 -    qemu_opts_print_help(drv->create_opts, false);
 +    qemu_opts_print_help(drv->amend_opts, false);
      printf("\nNote that not all of these options may be amendable.\n");
      return 0;
  }
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
      Error *err = NULL;
      int c, ret = 0;
      char *options = NULL;
 -    QemuOptsList *create_opts = NULL;
 +    QemuOptsList *amend_opts = NULL;
      QemuOpts *opts = NULL;
      const char *fmt = NULL, *filename, *cache;
      int flags;
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
          goto out;
      }
 -    /* Every driver supporting amendment must have create_opts */
 -    assert(bs->drv->create_opts);
 +    /* Every driver supporting amendment must have amend_opts */
 +    assert(bs->drv->amend_opts);
 -    create_opts = qemu_opts_append(create_opts, bs->drv->create_opts);
 -    opts = qemu_opts_create(create_opts, NULL, 0, &error_abort);
 +    amend_opts = qemu_opts_append(amend_opts, bs->drv->amend_opts);
 +    opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
      qemu_opts_do_parse(opts, options, NULL, &err);
      if (err) {
          error_report_err(err);
@@ -XXX,XX +XXX,XX @@ out:
  out_no_progress:
      blk_unref(blk);
      qemu_opts_del(opts);
 -    qemu_opts_free(create_opts);
 +    qemu_opts_free(amend_opts);
      g_free(options);
      if (ret) {
 --
-.26.2
+.34.1

-[PULL 02/18] iotests: filter few more luks specific create options
+[PULL 18/24] iotests: bash tests: filter compression type
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-This allows more tests to be able to have same output on both qcow2 luks encrypted images
+We want iotests pass with both the default zlib compression and with
-and raw luks images
+IMGOPTS='compression_type=zstd'.
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Actually the only test that is interested in real compression type in
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+test output is 287 (test for qcow2 compression type), so implement
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+specific option for it.
-Message-Id: <20200618150628.2169239-3-mreitz@redhat.com>
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20211223160144.1097696-17-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- tests/qemu-iotests/087.out       | 6 +++---
+ tests/qemu-iotests/060.out       |  2 +-
- tests/qemu-iotests/134.out       | 2 +-
+ tests/qemu-iotests/061.out       | 12 ++++++------
- tests/qemu-iotests/158.out       | 4 ++--
+ tests/qemu-iotests/082.out       | 14 +++++++-------
- tests/qemu-iotests/188.out       | 2 +-
+ tests/qemu-iotests/198.out       |  4 ++--
- tests/qemu-iotests/189.out       | 4 ++--
+ tests/qemu-iotests/287           |  8 ++++----
- tests/qemu-iotests/198.out       | 4 ++--
+ tests/qemu-iotests/common.filter |  8 ++++++++
- tests/qemu-iotests/263.out       | 4 ++--
+ tests/qemu-iotests/common.rc     | 14 +++++++++++++-
- tests/qemu-iotests/284.out       | 6 +++---
+files changed, 41 insertions(+), 21 deletions(-)
- tests/qemu-iotests/common.filter | 5 +----
-files changed, 17 insertions(+), 20 deletions(-)
+diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out
+index XXXXXXX..XXXXXXX 100644
-diff --git a/tests/qemu-iotests/087.out b/tests/qemu-iotests/087.out
+--- a/tests/qemu-iotests/060.out
-index XXXXXXX..XXXXXXX 100644
++++ b/tests/qemu-iotests/060.out
---- a/tests/qemu-iotests/087.out
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
-+++ b/tests/qemu-iotests/087.out
+ cluster_size: 65536
-@@ -XXX,XX +XXX,XX @@ QMP_VERSION
+ Format specific information:
+     compat: 1.1
- === Encrypted image QCow ===
+-    compression type: zlib
++    compression type: COMPRESSION_TYPE
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+     lazy refcounts: false
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
+     refcount bits: 16
- Testing:
+     corrupt: true
- QMP_VERSION
+diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
- {"return": {}}
+index XXXXXXX..XXXXXXX 100644
-@@ -XXX,XX +XXX,XX @@ QMP_VERSION
+--- a/tests/qemu-iotests/061.out
++++ b/tests/qemu-iotests/061.out
- === Encrypted image LUKS ===
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
+ cluster_size: 65536
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0
+ Format specific information:
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
+     compat: 1.1
- Testing:
+-    compression type: zlib
- QMP_VERSION
++    compression type: COMPRESSION_TYPE
- {"return": {}}
+     lazy refcounts: false
-@@ -XXX,XX +XXX,XX @@ QMP_VERSION
+     refcount bits: 16
+     data file: TEST_DIR/t.IMGFMT.data
- === Missing driver ===
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
+ cluster_size: 65536
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+ Format specific information:
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
+     compat: 1.1
- Testing: -S
+-    compression type: zlib
- QMP_VERSION
++    compression type: COMPRESSION_TYPE
- {"return": {}}
+     lazy refcounts: false
-diff --git a/tests/qemu-iotests/134.out b/tests/qemu-iotests/134.out
+     refcount bits: 16
-index XXXXXXX..XXXXXXX 100644
+     data file: foo
---- a/tests/qemu-iotests/134.out
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
-+++ b/tests/qemu-iotests/134.out
+ cluster_size: 65536
-@@ -XXX,XX +XXX,XX @@
+ Format specific information:
- QA output created by 134
+     compat: 1.1
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+-    compression type: zlib
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
++    compression type: COMPRESSION_TYPE
+     lazy refcounts: false
- == reading whole image ==
+     refcount bits: 16
- read 134217728/134217728 bytes at offset 0
+     data file raw: false
-diff --git a/tests/qemu-iotests/158.out b/tests/qemu-iotests/158.out
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
-index XXXXXXX..XXXXXXX 100644
+ cluster_size: 65536
---- a/tests/qemu-iotests/158.out
+ Format specific information:
-+++ b/tests/qemu-iotests/158.out
+     compat: 1.1
-@@ -XXX,XX +XXX,XX @@
+-    compression type: zlib
- QA output created by 158
++    compression type: COMPRESSION_TYPE
- == create base ==
+     lazy refcounts: false
--Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+     refcount bits: 16
-+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on
+     data file: TEST_DIR/t.IMGFMT.data
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
- == writing whole image ==
+ cluster_size: 65536
- wrote 134217728/134217728 bytes at offset 0
+ Format specific information:
-@@ -XXX,XX +XXX,XX @@ wrote 134217728/134217728 bytes at offset 0
+     compat: 1.1
- read 134217728/134217728 bytes at offset 0
+-    compression type: zlib
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
++    compression type: COMPRESSION_TYPE
- == create overlay ==
+     lazy refcounts: false
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on encrypt.key-secret=sec0
+     refcount bits: 16
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on
+     data file: TEST_DIR/t.IMGFMT.data
+@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
- == writing part of a cluster ==
+ cluster_size: 65536
- wrote 1024/1024 bytes at offset 0
+ Format specific information:
-diff --git a/tests/qemu-iotests/188.out b/tests/qemu-iotests/188.out
+     compat: 1.1
-index XXXXXXX..XXXXXXX 100644
+-    compression type: zlib
---- a/tests/qemu-iotests/188.out
++    compression type: COMPRESSION_TYPE
-+++ b/tests/qemu-iotests/188.out
+     lazy refcounts: false
-@@ -XXX,XX +XXX,XX @@
+     refcount bits: 16
- QA output created by 188
+     data file: TEST_DIR/t.IMGFMT.data
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/082.out
- == reading whole image ==
++++ b/tests/qemu-iotests/082.out
- read 16777216/16777216 bytes at offset 0
+@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
-diff --git a/tests/qemu-iotests/189.out b/tests/qemu-iotests/189.out
+ cluster_size: 4096
-index XXXXXXX..XXXXXXX 100644
+ Format specific information:
---- a/tests/qemu-iotests/189.out
+     compat: 1.1
-+++ b/tests/qemu-iotests/189.out
+-    compression type: zlib
-@@ -XXX,XX +XXX,XX @@
++    compression type: COMPRESSION_TYPE
- QA output created by 189
+     lazy refcounts: true
- == create base ==
+     refcount bits: 16
--Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+     corrupt: false
-+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
+@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
+ cluster_size: 8192
- == writing whole image ==
+ Format specific information:
- wrote 16777216/16777216 bytes at offset 0
+     compat: 1.1
-@@ -XXX,XX +XXX,XX @@ wrote 16777216/16777216 bytes at offset 0
+-    compression type: zlib
- read 16777216/16777216 bytes at offset 0
++    compression type: COMPRESSION_TYPE
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+     lazy refcounts: true
- == create overlay ==
+     refcount bits: 16
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+     corrupt: false
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
+@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
+ cluster_size: 4096
- == writing part of a cluster ==
+ Format specific information:
- wrote 1024/1024 bytes at offset 0
+     compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: true
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
  cluster_size: 8192
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: true
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: true
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 130 MiB (136314880 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 132 MiB (138412032 bytes)
  cluster_size: 65536
  Format specific information:
      compat: 1.1
 -    compression type: zlib
 +    compression type: COMPRESSION_TYPE
      lazy refcounts: true
      refcount bits: 16
      corrupt: false
 diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/198.out
 +++ b/tests/qemu-iotests/198.out
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ image: json:{ /* filtered */ }
- QA output created by 198
+ file format: IMGFMT
- == create base ==
+ virtual size: 16 MiB (16777216 bytes)
--Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+ Format specific information:
-+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
+-    compression type: zlib
++    compression type: COMPRESSION_TYPE
- == writing whole image base ==
+     encrypt:
- wrote 16777216/16777216 bytes at offset 0
+         ivgen alg: plain64
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+         hash alg: sha256
- == create overlay ==
+@@ -XXX,XX +XXX,XX @@ virtual size: 16 MiB (16777216 bytes)
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+ backing file: TEST_DIR/t.IMGFMT.base
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
+ backing file format: IMGFMT
+ Format specific information:
- == writing whole image layer ==
+-    compression type: zlib
- wrote 16777216/16777216 bytes at offset 0
++    compression type: COMPRESSION_TYPE
-diff --git a/tests/qemu-iotests/263.out b/tests/qemu-iotests/263.out
+     encrypt:
-index XXXXXXX..XXXXXXX 100644
+         ivgen alg: plain64
---- a/tests/qemu-iotests/263.out
+         hash alg: sha256
-+++ b/tests/qemu-iotests/263.out
+diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
-@@ -XXX,XX +XXX,XX @@ QA output created by 263
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/287
- testing LUKS qcow2 encryption
++++ b/tests/qemu-iotests/287
+@@ -XXX,XX +XXX,XX @@ echo
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+ echo "=== Testing compression type incompatible bit setting for zlib ==="
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
+ echo
- == reading the whole image ==
+ _make_test_img -o compression_type=zlib 64M
- read 1048576/1048576 bytes at offset 0
+-_qcow2_dump_header | grep incompatible_features
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
++_qcow2_dump_header --no-filter-compression | grep incompatible_features
-@@ -XXX,XX +XXX,XX @@ read 982528/982528 bytes at offset 66048
+ echo
- testing legacy AES qcow2 encryption
+ echo "=== Testing compression type incompatible bit setting for zstd ==="
+ echo
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=aes encrypt.key-secret=sec0
+ _make_test_img -o compression_type=zstd 64M
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
+-_qcow2_dump_header | grep incompatible_features
- == reading the whole image ==
++_qcow2_dump_header --no-filter-compression | grep incompatible_features
- read 1048576/1048576 bytes at offset 0
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ echo
-diff --git a/tests/qemu-iotests/284.out b/tests/qemu-iotests/284.out
+ echo "=== Testing zlib with incompatible bit set ==="
-index XXXXXXX..XXXXXXX 100644
+@@ -XXX,XX +XXX,XX @@ echo
---- a/tests/qemu-iotests/284.out
+ _make_test_img -o compression_type=zlib 64M
-+++ b/tests/qemu-iotests/284.out
+ $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 3
-@@ -XXX,XX +XXX,XX @@ QA output created by 284
+ # to make sure the bit was actually set
+-_qcow2_dump_header | grep incompatible_features
- testing LUKS qcow2 encryption
++_qcow2_dump_header --no-filter-compression | grep incompatible_features
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+ if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
+     echo "Error: The image opened successfully. The image must not be opened."
+@@ -XXX,XX +XXX,XX @@ echo
- == cluster size 512
+ _make_test_img -o compression_type=zstd 64M
- == checking image refcounts ==
+ $PYTHON qcow2.py "$TEST_IMG" set-header incompatible_features 0
-@@ -XXX,XX +XXX,XX @@ wrote 1/1 bytes at offset 512
+ # to make sure the bit was actually unset
+-_qcow2_dump_header | grep incompatible_features
- == rechecking image refcounts ==
++_qcow2_dump_header --no-filter-compression | grep incompatible_features
- No errors were found on the image.
--Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+ if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
+     echo "Error: The image opened successfully. The image must not be opened."
  == cluster size 2048
  == checking image refcounts ==
@@ -XXX,XX +XXX,XX @@ wrote 1/1 bytes at offset 2048
  == rechecking image refcounts ==
  No errors were found on the image.
 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
  == cluster size 32768
  == checking image refcounts ==
 diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/common.filter
 +++ b/tests/qemu-iotests/common.filter
-@@ -XXX,XX +XXX,XX @@ _filter_img_create()
+@@ -XXX,XX +XXX,XX @@ _filter_img_info()
-         echo "$options" \
+         -e "/block_state_zero: \\(on\\|off\\)/d" \
-         | tr '\n' '\0' \
+         -e "/log_size: [0-9]\\+/d" \
-         | $SED -e 's/\x0$//' -e 's/ \([a-z0-9_.-]*\)=/\n\1=/g' \
+         -e "s/iters: [0-9]\\+/iters: 1024/" \
--        | grep -ae "^\(fmt\\|size\\|backing\\|preallocation\\|encrypt$grep_data_file\\)" \
++        -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \
-+        | grep -ae "^\(fmt\\|size\\|backing\\|preallocation\\|encryption$grep_data_file\\)" \
+         -e "s/uuid: [-a-f0-9]\\+/uuid: 00000000-0000-0000-0000-000000000000/" | \
-         | $SED "${filename_filters[@]}" \
+     while IFS='' read -r line; do
-             -e 's/^\(fmt\)/0-\1/' \
+         if [[ $format_specific == 1 ]]; then
-             -e 's/^\(size\)/1-\1/' \
+@@ -XXX,XX +XXX,XX @@ _filter_authz_check_tls()
-             -e 's/^\(backing\)/2-\1/' \
+     $SED -e 's/TLS x509 authz check for .* is denied/TLS x509 authz check for DISTINGUISHED-NAME is denied/'
-             -e 's/^\(data_file\)/3-\1/' \
+ }
-             -e 's/^\(encryption\)/4-\1/' \
--            -e 's/^\(encrypt\.format\)/5-\1/' \
++_filter_qcow2_compression_type_bit()
--            -e 's/^\(encrypt\.key-secret\)/6-\1/' \
++{
--            -e 's/^\(encrypt\.iter-time\)/7-\1/' \
++    $SED -e 's/\(incompatible_features\s\+\)\[3\(, \)\?/\1[/' \
-             -e 's/^\(preallocation\)/8-\1/' \
++         -e 's/\(incompatible_features.*\), 3\]/\1]/' \
-         | sort \
++         -e 's/\(incompatible_features.*\), 3\(,.*\)/\1\2/'
-         | $SED -e 's/^[0-9]-//' \
++}
 +
  # make sure this script returns success
  true
 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/common.rc
 +++ b/tests/qemu-iotests/common.rc
@@ -XXX,XX +XXX,XX @@ _img_info()
              -e "s#$TEST_DIR#TEST_DIR#g" \
              -e "s#$SOCK_DIR/fuse-#TEST_DIR/#g" \
              -e "s#$IMGFMT#IMGFMT#g" \
 +            -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \
              -e "/^disk size:/ D" \
              -e "/actual-size/ D" | \
          while IFS='' read -r line; do
@@ -XXX,XX +XXX,XX @@ _require_one_device_of()
  _qcow2_dump_header()
  {
 +    if [[ "$1" == "--no-filter-compression" ]]; then
 +        local filter_compression=0
 +        shift
 +    else
 +        local filter_compression=1
 +    fi
 +
      img="$1"
      if [ -z "$img" ]; then
          img="$TEST_IMG"
      fi
 -    $PYTHON qcow2.py "$img" dump-header
 +    if [[ $filter_compression == 0 ]]; then
 +        $PYTHON qcow2.py "$img" dump-header
 +    else
 +        $PYTHON qcow2.py "$img" dump-header | _filter_qcow2_compression_type_bit
 +    fi
  }
  # make sure this script returns success
 --
-.26.2
+.34.1

-[PULL 14/18] block/crypto: implement blockdev-amend
+[PULL 19/24] iotests 60: more accurate set dirty bit in qcow2 header
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Don't touch other incompatible bits, like compression-type. This makes
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+the test pass with IMGOPTS='compression_type=zstd'.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
-Message-Id: <20200608094030.670121-13-mlevitsk@redhat.com>
+Message-Id: <20211223160144.1097696-18-vsementsov@virtuozzo.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- qapi/block-core.json | 14 ++++++++-
+ tests/qemu-iotests/060 | 2 +-
- block/crypto.c       | 72 ++++++++++++++++++++++++++++++++------------
+file changed, 1 insertion(+), 1 deletion(-)
 files changed, 66 insertions(+), 20 deletions(-)
-diff --git a/qapi/block-core.json b/qapi/block-core.json
+diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/qapi/block-core.json
+--- a/tests/qemu-iotests/060
-+++ b/qapi/block-core.json
++++ b/tests/qemu-iotests/060
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ _make_test_img 64M
-   'data': { 'job-id': 'str',
+ # Let the refblock appear unaligned
-             'options': 'BlockdevCreateOptions' } }
+ poke_file "$TEST_IMG" "$rt_offset"        "\x00\x00\x00\x00\xff\xff\x2a\x00"
+ # Mark the image dirty, thus forcing an automatic check when opening it
-+##
+-poke_file "$TEST_IMG" 72 "\x00\x00\x00\x00\x00\x00\x00\x01"
-+# @BlockdevAmendOptionsLUKS:
++$PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 0
-+#
+ # Open the image (qemu should refuse to do so)
-+# Driver specific image amend options for LUKS.
+ $QEMU_IO -c close "$TEST_IMG" 2>&1 | _filter_testdir | _filter_imgfmt
 +#
 +# Since: 5.1
 +##
 +{ 'struct': 'BlockdevAmendOptionsLUKS',
 +  'base': 'QCryptoBlockAmendOptionsLUKS',
 +  'data': { }
 +}
 +
  ##
  # @BlockdevAmendOptions:
  #
@@ -XXX,XX +XXX,XX @@
        'driver':         'BlockdevDriver' },
    'discriminator': 'driver',
    'data': {
 -  } }
 +      'luks':           'BlockdevAmendOptionsLUKS' } }
  ##
  # @x-blockdev-amend:
 diff --git a/block/crypto.c b/block/crypto.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/crypto.c
 +++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp)
  }
  static int
 -block_crypto_amend_options_luks(BlockDriverState *bs,
 -                                QemuOpts *opts,
 -                                BlockDriverAmendStatusCB *status_cb,
 -                                void *cb_opaque,
 -                                bool force,
 -                                Error **errp)
 +block_crypto_amend_options_generic_luks(BlockDriverState *bs,
 +                                        QCryptoBlockAmendOptions *amend_options,
 +                                        bool force,
 +                                        Error **errp)
  {
      BlockCrypto *crypto = bs->opaque;
 -    QDict *cryptoopts = NULL;
 -    QCryptoBlockAmendOptions *amend_options = NULL;
      int ret;
      assert(crypto);
      assert(crypto->block);
 -    crypto->updating_keys = true;
 +    /* apply for exclusive read/write permissions to the underlying file*/
 +    crypto->updating_keys = true;
      ret = bdrv_child_refresh_perms(bs, bs->file, errp);
 -    if (ret < 0) {
 -        goto cleanup;
 -    }
 -
 -    cryptoopts = qemu_opts_to_qdict(opts, NULL);
 -    qdict_put_str(cryptoopts, "format", "luks");
 -    amend_options = block_crypto_amend_opts_init(cryptoopts, errp);
 -    if (!amend_options) {
 -        ret = -EINVAL;
 +    if (ret) {
          goto cleanup;
      }
@@ -XXX,XX +XXX,XX @@ block_crypto_amend_options_luks(BlockDriverState *bs,
                                        force,
                                        errp);
  cleanup:
 +    /* release exclusive read/write permissions to the underlying file*/
      crypto->updating_keys = false;
      bdrv_child_refresh_perms(bs, bs->file, errp);
 -    qapi_free_QCryptoBlockAmendOptions(amend_options);
 +    return ret;
 +}
 +
 +static int
 +block_crypto_amend_options_luks(BlockDriverState *bs,
 +                                QemuOpts *opts,
 +                                BlockDriverAmendStatusCB *status_cb,
 +                                void *cb_opaque,
 +                                bool force,
 +                                Error **errp)
 +{
 +    BlockCrypto *crypto = bs->opaque;
 +    QDict *cryptoopts = NULL;
 +    QCryptoBlockAmendOptions *amend_options = NULL;
 +    int ret = -EINVAL;
 +
 +    assert(crypto);
 +    assert(crypto->block);
 +
 +    cryptoopts = qemu_opts_to_qdict(opts, NULL);
 +    qdict_put_str(cryptoopts, "format", "luks");
 +    amend_options = block_crypto_amend_opts_init(cryptoopts, errp);
      qobject_unref(cryptoopts);
 +    if (!amend_options) {
 +        goto cleanup;
 +    }
 +    ret = block_crypto_amend_options_generic_luks(bs, amend_options,
 +                                                  force, errp);
 +cleanup:
 +    qapi_free_QCryptoBlockAmendOptions(amend_options);
      return ret;
  }
 +static int
 +coroutine_fn block_crypto_co_amend_luks(BlockDriverState *bs,
 +                                        BlockdevAmendOptions *opts,
 +                                        bool force,
 +                                        Error **errp)
 +{
 +    QCryptoBlockAmendOptions amend_opts;
 +
 +    amend_opts = (QCryptoBlockAmendOptions) {
 +        .format = Q_CRYPTO_BLOCK_FORMAT_LUKS,
 +        .u.luks = *qapi_BlockdevAmendOptionsLUKS_base(&opts->u.luks),
 +    };
 +    return block_crypto_amend_options_generic_luks(bs, &amend_opts,
 +                                                   force, errp);
 +}
  static void
  block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_crypto_luks = {
      .bdrv_get_info      = block_crypto_get_info_luks,
      .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
      .bdrv_amend_options = block_crypto_amend_options_luks,
 +    .bdrv_co_amend      = block_crypto_co_amend_luks,
      .is_format          = true,
 --
-.26.2
+.34.1

-[PULL 09/18] block/crypto: rename two functions
+[PULL 20/24] iotest 214: explicit compression type
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-rename the write_func to create_write_func, and init_func to create_init_func.
+The test-case "Corrupted size field in compressed cluster descriptor"
-This is preparation for other write_func that will be used to update the encryption keys.
+heavily depends on zlib compression type. So, make it explicit. This
 way test passes with IMGOPTS='compression_type=zstd'.
-No functional changes
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Message-Id: <20211223160144.1097696-19-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/214 | 2 +-
 file changed, 1 insertion(+), 1 deletion(-)
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+index XXXXXXX..XXXXXXX 100755
-Message-Id: <20200608094030.670121-7-mlevitsk@redhat.com>
+--- a/tests/qemu-iotests/214
-Signed-off-by: Max Reitz <mreitz@redhat.com>
++++ b/tests/qemu-iotests/214
----
+@@ -XXX,XX +XXX,XX @@ echo
- block/crypto.c | 25 ++++++++++++-------------
+ # The L2 entries of the two compressed clusters are located at
-file changed, 12 insertions(+), 13 deletions(-)
+ # 0x800000 and 0x800008, their original values are 0x4008000000a00000
+ # and 0x4008000000a00802 (5 sectors for compressed data each).
-diff --git a/block/crypto.c b/block/crypto.c
+-_make_test_img 8M -o cluster_size=2M
-index XXXXXXX..XXXXXXX 100644
++_make_test_img 8M -o cluster_size=2M,compression_type=zlib
---- a/block/crypto.c
+ $QEMU_IO -c "write -c -P 0x11 0 2M" -c "write -c -P 0x11 2M 2M" "$TEST_IMG" \
-+++ b/block/crypto.c
+>&1 | _filter_qemu_io | _filter_testdir
@@ -XXX,XX +XXX,XX @@ struct BlockCryptoCreateData {
  };
 -static ssize_t block_crypto_write_func(QCryptoBlock *block,
 -                                       size_t offset,
 -                                       const uint8_t *buf,
 -                                       size_t buflen,
 -                                       void *opaque,
 -                                       Error **errp)
 +static ssize_t block_crypto_create_write_func(QCryptoBlock *block,
 +                                              size_t offset,
 +                                              const uint8_t *buf,
 +                                              size_t buflen,
 +                                              void *opaque,
 +                                              Error **errp)
  {
      struct BlockCryptoCreateData *data = opaque;
      ssize_t ret;
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_write_func(QCryptoBlock *block,
      return ret;
  }
 -
 -static ssize_t block_crypto_init_func(QCryptoBlock *block,
 -                                      size_t headerlen,
 -                                      void *opaque,
 -                                      Error **errp)
 +static ssize_t block_crypto_create_init_func(QCryptoBlock *block,
 +                                             size_t headerlen,
 +                                             void *opaque,
 +                                             Error **errp)
  {
      struct BlockCryptoCreateData *data = opaque;
      Error *local_error = NULL;
@@ -XXX,XX +XXX,XX @@ static int block_crypto_co_create_generic(BlockDriverState *bs,
      };
      crypto = qcrypto_block_create(opts, NULL,
 -                                  block_crypto_init_func,
 -                                  block_crypto_write_func,
 +                                  block_crypto_create_init_func,
 +                                  block_crypto_create_write_func,
                                    &data,
                                    errp);
 --
-.26.2
+.34.1

-[PULL 01/18] iotests: Make _filter_img_create more active
+[PULL 21/24] iotests: declare lack of support for compresion_type in IMGOPTS
-Right now, _filter_img_create just filters out everything that looks
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 format-dependent, and applies some filename filters.  That means that we
 have to add another filter line every time some format gets a new
 creation option.  This can be avoided by instead discarding everything
 and just keeping what we know is format-independent (format, size,
 backing file, encryption information[1], preallocation) or just
 interesting to have in the reference output (external data file path).
-Furthermore, we probably want to sort these options.  Format drivers are
+compression_type can't be used if we want to create image with
-not required to define them in any specific order, so the output is
+compat=0.10. So, skip these tests, not many of them.
 effectively random (although this has never bothered us until now).  We
 need a specific order for our reference outputs, though.  Unfortunately,
 just using a plain "sort" would change a lot of existing reference
 outputs, so we have to pre-filter the option keys to keep our existing
 order (fmt, size, backing*, data, encryption info, preallocation).
-Finally, this makes it difficult for _filter_img_create to automagically
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-work for QMP output.  Thus, this patch adds a separate
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-_filter_img_create_for_qmp function that echos every line verbatim that
+Message-Id: <20211223160144.1097696-20-vsementsov@virtuozzo.com>
-does not start with "Formatting", and pipes those "Formatting" lines to
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
-_filter_img_create.
+---
  tests/qemu-iotests/031 | 5 +++--
  tests/qemu-iotests/051 | 5 +++--
  tests/qemu-iotests/061 | 6 +++++-
  tests/qemu-iotests/112 | 3 ++-
  tests/qemu-iotests/290 | 2 +-
 files changed, 14 insertions(+), 7 deletions(-)
-[1] Actually, the only thing that is really important is whether
+diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
     encryption is enabled or not.  A patch by Maxim thus removes all
     other "encrypt.*" options from the output:
     https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00339.html
     But that patch needs to come later so we can get away with changing
     as few reference outputs in this patch here as possible.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 Message-Id: <20200618150628.2169239-2-mreitz@redhat.com>
 ---
  tests/qemu-iotests/112.out       |  2 +-
  tests/qemu-iotests/141           |  2 +-
  tests/qemu-iotests/153           |  9 ++-
  tests/qemu-iotests/common.filter | 96 ++++++++++++++++++++++++--------
 files changed, 78 insertions(+), 31 deletions(-)
 diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/112.out
 +++ b/tests/qemu-iotests/112.out
@@ -XXX,XX +XXX,XX @@ QA output created by 112
  qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 refcount_bits=-1
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
 diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
 index XXXXXXX..XXXXXXX 100755
---- a/tests/qemu-iotests/141
+--- a/tests/qemu-iotests/031
-+++ b/tests/qemu-iotests/141
++++ b/tests/qemu-iotests/031
-@@ -XXX,XX +XXX,XX @@ test_blockjob()
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-     _send_qemu_cmd $QEMU_HANDLE \
+ _supported_fmt qcow2
-         "$1" \
+ _supported_proto file fuse
-         "$2" \
+ # We want to test compat=0.10, which does not support external data
--        | _filter_img_create | _filter_qmp_empty_return
+-# files or refcount widths other than 16
-+        | _filter_img_create_in_qmp | _filter_qmp_empty_return
+-_unsupported_imgopts data_file 'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
++# files or refcount widths other than 16 or compression type
-     # We want this to return an error because the block job is still running
++_unsupported_imgopts data_file compression_type \
-     _send_qemu_cmd $QEMU_HANDLE \
++    'refcount_bits=\([^1]\|.\([^6]\|$\)\)'
-diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
  CLUSTER_SIZE=65536
 diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051
 index XXXXXXX..XXXXXXX 100755
---- a/tests/qemu-iotests/153
+--- a/tests/qemu-iotests/051
-+++ b/tests/qemu-iotests/153
++++ b/tests/qemu-iotests/051
-@@ -XXX,XX +XXX,XX @@ done
+@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow2
  _supported_proto file
  # A compat=0.10 image is created in this test which does not support anything
  # other than refcount_bits=16;
 -# it also will not support an external data file
 -_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file
 +# it also will not support an external data file and compression type
 +_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file \
 +    compression_type
  _require_drivers nbd
  if [ "$QEMU_DEFAULT_MACHINE" = "pc" ]; then
 diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/061
 +++ b/tests/qemu-iotests/061
@@ -XXX,XX +XXX,XX @@ _supported_os Linux
  # not work with it;
  # we have explicit tests for various cluster sizes, the remaining tests
  # require the default 64k cluster
 -_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file cluster_size
 +# we don't have explicit tests for zstd qcow2 compression type, as zstd may be
 +# not compiled in. And we can't create compat images with comression type
 +# extension
 +_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' data_file \
 +    cluster_size compression_type
  echo
- echo "== Creating ${TEST_IMG}.[abc] ==" | _filter_testdir
+ echo "=== Testing version downgrade with zero expansion ==="
--(
+diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
--    $QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}"
+index XXXXXXX..XXXXXXX 100755
--    $QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}"
+--- a/tests/qemu-iotests/112
--    $QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b"
++++ b/tests/qemu-iotests/112
--) | _filter_img_create
+@@ -XXX,XX +XXX,XX @@ _supported_proto file fuse
-+$QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" | _filter_img_create
+ # This test will set refcount_bits on its own which would conflict with the
-+$QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" | _filter_img_create
+ # manual setting; compat will be overridden as well;
-+$QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" \
+ # and external data files do not work well with our refcount testing
-+    | _filter_img_create
+-_unsupported_imgopts refcount_bits 'compat=0.10' data_file
 +# also, compression type is not supported with compat=0.10 used in test
 +_unsupported_imgopts refcount_bits 'compat=0.10' data_file compression_type
  print_refcount_bits()
  {
 diff --git a/tests/qemu-iotests/290 b/tests/qemu-iotests/290
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/290
 +++ b/tests/qemu-iotests/290
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow2
  _supported_proto file fuse
  _supported_os Linux
 -_unsupported_imgopts 'compat=0.10' refcount_bits data_file
 +_unsupported_imgopts 'compat=0.10' refcount_bits data_file compression_type
  echo
- echo "== Two devices sharing the same file in backing chain =="
+ echo "### Test 'qemu-io -c discard' on a QCOW2 image without a backing file"
 diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/common.filter
 +++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_actual_image_size()
  # replace driver-specific options in the "Formatting..." line
  _filter_img_create()
  {
 -    data_file_filter=()
 -    if data_file=$(_get_data_file "$TEST_IMG"); then
 -        data_file_filter=(-e "s# data_file=$data_file##")
 +    # Split the line into the pre-options part ($filename_part, which
 +    # precedes ", fmt=") and the options part ($options, which starts
 +    # with "fmt=")
 +    readarray -td '' formatting_line < <(sed -e 's/, fmt=/\x0/')
 +
 +    filename_part=${formatting_line[0]}
 +    if [ -n "${formatting_line[1]}" ]; then
 +        options="fmt=${formatting_line[1]}"
 +    else
 +        options=''
 +    fi
 +
 +    # Set grep_data_file to '\|data_file' to keep it; make it empty
 +    # to drop it.
 +    # We want to drop it if it is part of the global $IMGOPTS, and we
 +    # want to keep it otherwise (if the test specifically wants to
 +    # test data files).
 +    grep_data_file='\|data_file'
 +    if _get_data_file "$TEST_IMG" > /dev/null; then
 +        grep_data_file=''
      fi
 -    $SED "${data_file_filter[@]}" \
 +    filename_filters=(
          -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \
          -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
          -e "s#$TEST_DIR#TEST_DIR#g" \
          -e "s#$SOCK_DIR#SOCK_DIR#g" \
          -e "s#$IMGFMT#IMGFMT#g" \
          -e 's#nbd:127.0.0.1:[0-9]\\+#TEST_DIR/t.IMGFMT#g' \
 -        -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' \
 -        -e "s# encryption=off##g" \
 -        -e "s# cluster_size=[0-9]\\+##g" \
 -        -e "s# table_size=[0-9]\\+##g" \
 -        -e "s# compat=[^ ]*##g" \
 -        -e "s# compat6=\\(on\\|off\\)##g" \
 -        -e "s# static=\\(on\\|off\\)##g" \
 -        -e "s# zeroed_grain=\\(on\\|off\\)##g" \
 -        -e "s# subformat=[^ ]*##g" \
 -        -e "s# adapter_type=[^ ]*##g" \
 -        -e "s# hwversion=[^ ]*##g" \
 -        -e "s# lazy_refcounts=\\(on\\|off\\)##g" \
 -        -e "s# block_size=[0-9]\\+##g" \
 -        -e "s# block_state_zero=\\(on\\|off\\)##g" \
 -        -e "s# log_size=[0-9]\\+##g" \
 -        -e "s# refcount_bits=[0-9]\\+##g" \
 -        -e "s# key-secret=[a-zA-Z0-9]\\+##g" \
 -        -e "s# iter-time=[0-9]\\+##g" \
 -        -e "s# force_size=\\(on\\|off\\)##g" \
 -        -e "s# compression_type=[a-zA-Z0-9]\\+##g"
 +        -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g'
 +    )
 +
 +    filename_part=$(echo "$filename_part" | $SED "${filename_filters[@]}")
 +
 +    # Break the option line before each option (preserving pre-existing
 +    # line breaks by replacing them by \0 and restoring them at the end),
 +    # then filter out the options we want to keep and sort them according
 +    # to some order that all block drivers used at the time of writing
 +    # this function.
 +    options=$(
 +        echo "$options" \
 +        | tr '\n' '\0' \
 +        | $SED -e 's/\x0$//' -e 's/ \([a-z0-9_.-]*\)=/\n\1=/g' \
 +        | grep -ae "^\(fmt\\|size\\|backing\\|preallocation\\|encrypt$grep_data_file\\)" \
 +        | $SED "${filename_filters[@]}" \
 +            -e 's/^\(fmt\)/0-\1/' \
 +            -e 's/^\(size\)/1-\1/' \
 +            -e 's/^\(backing\)/2-\1/' \
 +            -e 's/^\(data_file\)/3-\1/' \
 +            -e 's/^\(encryption\)/4-\1/' \
 +            -e 's/^\(encrypt\.format\)/5-\1/' \
 +            -e 's/^\(encrypt\.key-secret\)/6-\1/' \
 +            -e 's/^\(encrypt\.iter-time\)/7-\1/' \
 +            -e 's/^\(preallocation\)/8-\1/' \
 +        | sort \
 +        | $SED -e 's/^[0-9]-//' \
 +        | tr '\n\0' ' \n' \
 +        | $SED -e 's/^ *$//' -e 's/ *$//'
 +    )
 +
 +    if [ -n "$options" ]; then
 +        echo "$filename_part, $options"
 +    elif [ -n "$filename_part" ]; then
 +        echo "$filename_part"
 +    fi
 +}
 +
 +# Filter the "Formatting..." line in QMP output (leaving the QMP output
 +# untouched)
 +# (In contrast to _filter_img_create(), this function does not support
 +# multi-line Formatting output)
 +_filter_img_create_in_qmp()
 +{
 +    while read -r line; do
 +        if echo "$line" | grep -q '^Formatting'; then
 +            echo "$line" | _filter_img_create
 +        else
 +            echo "$line"
 +        fi
 +    done
  }
  _filter_img_create_size()
 --
-.26.2
+.34.1

-[PULL 04/18] qcrypto/core: add generic infrastructure for crypto options amendment
+[PULL 22/24] block-backend: Retain permissions after migration
-From: Maxim Levitsky <mlevitsk@redhat.com>
+After migration, the permissions the guest device wants to impose on its
 BlockBackend are stored in blk->perm and blk->shared_perm.  In
 blk_root_activate(), we take our permissions, but keep all shared
 permissions open by calling `blk_set_perm(blk->perm, BLK_PERM_ALL)`.
-This will be used first to implement luks keyslot management.
+Only afterwards (immediately or later, depending on the runstate) do we
 restrict the shared permissions by calling
 `blk_set_perm(blk->perm, blk->shared_perm)`.  Unfortunately, our first
 call with shared_perm=BLK_PERM_ALL has overwritten blk->shared_perm to
 be BLK_PERM_ALL, so this is a no-op and the set of shared permissions is
 not restricted.
-block_crypto_amend_opts_init will be used to convert
+Fix this bug by saving the set of shared permissions before invoking
-qemu-img cmdline to QCryptoBlockAmendOptions
+blk_set_perm() with BLK_PERM_ALL and restoring it afterwards.
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Fixes: 5f7772c4d0cf32f4e779fcd5a69ae4dae24aeebf
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
+       ("block-backend: Defer shared_perm tightening migration
-Message-Id: <20200608094030.670121-2-mlevitsk@redhat.com>
+       completion")
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Reported-by: Peng Liang <liangpeng10@huawei.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20211125135317.186576-2-hreitz@redhat.com>
 Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
 Tested-by: Peng Liang <liangpeng10@huawei.com>
 ---
- qapi/crypto.json       | 16 ++++++++++++++++
+ block/block-backend.c | 11 +++++++++++
- block/crypto.h         |  3 +++
+file changed, 11 insertions(+)
  crypto/blockpriv.h     |  8 ++++++++
  include/crypto/block.h | 22 ++++++++++++++++++++++
  block/crypto.c         | 17 +++++++++++++++++
  crypto/block.c         | 29 +++++++++++++++++++++++++++++
 files changed, 95 insertions(+)
-diff --git a/qapi/crypto.json b/qapi/crypto.json
+diff --git a/block/block-backend.c b/block/block-backend.c
 index XXXXXXX..XXXXXXX 100644
---- a/qapi/crypto.json
+--- a/block/block-backend.c
-+++ b/qapi/crypto.json
++++ b/block/block-backend.c
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static void blk_root_activate(BdrvChild *child, Error **errp)
-   'base': 'QCryptoBlockInfoBase',
+ {
-   'discriminator': 'format',
+     BlockBackend *blk = child->opaque;
-   'data': { 'luks': 'QCryptoBlockInfoLUKS' } }
+     Error *local_err = NULL;
 +    uint64_t saved_shared_perm;
      if (!blk->disable_perm) {
          return;
@@ -XXX,XX +XXX,XX @@ static void blk_root_activate(BdrvChild *child, Error **errp)
      blk->disable_perm = false;
 +    /*
 +     * blk->shared_perm contains the permissions we want to share once
 +     * migration is really completely done.  For now, we need to share
 +     * all; but we also need to retain blk->shared_perm, which is
 +     * overwritten by a successful blk_set_perm() call.  Save it and
 +     * restore it below.
 +     */
 +    saved_shared_perm = blk->shared_perm;
 +
-+
+     blk_set_perm(blk, blk->perm, BLK_PERM_ALL, &local_err);
-+
+     if (local_err) {
-+##
+         error_propagate(errp, local_err);
-+# @QCryptoBlockAmendOptions:
+         blk->disable_perm = true;
-+#
+         return;
-+# The options that are available for all encryption formats
+     }
-+# when amending encryption settings
++    blk->shared_perm = saved_shared_perm;
-+#
-+# Since: 5.1
+     if (runstate_check(RUN_STATE_INMIGRATE)) {
-+##
+         /* Activation can happen when migration process is still active, for
 +{ 'union': 'QCryptoBlockAmendOptions',
 +  'base': 'QCryptoBlockOptionsBase',
 +  'discriminator': 'format',
 +  'data': {
 +            } }
 diff --git a/block/crypto.h b/block/crypto.h
 index XXXXXXX..XXXXXXX 100644
 --- a/block/crypto.h
 +++ b/block/crypto.h
@@ -XXX,XX +XXX,XX @@
  QCryptoBlockCreateOptions *
  block_crypto_create_opts_init(QDict *opts, Error **errp);
 +QCryptoBlockAmendOptions *
 +block_crypto_amend_opts_init(QDict *opts, Error **errp);
 +
  QCryptoBlockOpenOptions *
  block_crypto_open_opts_init(QDict *opts, Error **errp);
 diff --git a/crypto/blockpriv.h b/crypto/blockpriv.h
 index XXXXXXX..XXXXXXX 100644
 --- a/crypto/blockpriv.h
 +++ b/crypto/blockpriv.h
@@ -XXX,XX +XXX,XX @@ struct QCryptoBlockDriver {
                    void *opaque,
                    Error **errp);
 +    int (*amend)(QCryptoBlock *block,
 +                 QCryptoBlockReadFunc readfunc,
 +                 QCryptoBlockWriteFunc writefunc,
 +                 void *opaque,
 +                 QCryptoBlockAmendOptions *options,
 +                 bool force,
 +                 Error **errp);
 +
      int (*get_info)(QCryptoBlock *block,
                      QCryptoBlockInfo *info,
                      Error **errp);
 diff --git a/include/crypto/block.h b/include/crypto/block.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/crypto/block.h
 +++ b/include/crypto/block.h
@@ -XXX,XX +XXX,XX @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,
                                     void *opaque,
                                     Error **errp);
 +/**
 + * qcrypto_block_amend_options:
 + * @block: the block encryption object
 + *
 + * @readfunc: callback for reading data from the volume header
 + * @writefunc: callback for writing data to the volume header
 + * @opaque: data to pass to @readfunc and @writefunc
 + * @options: the new/amended encryption options
 + * @force: hint for the driver to allow unsafe operation
 + * @errp: error pointer
 + *
 + * Changes the crypto options of the encryption format
 + *
 + */
 +int qcrypto_block_amend_options(QCryptoBlock *block,
 +                                QCryptoBlockReadFunc readfunc,
 +                                QCryptoBlockWriteFunc writefunc,
 +                                void *opaque,
 +                                QCryptoBlockAmendOptions *options,
 +                                bool force,
 +                                Error **errp);
 +
  /**
   * qcrypto_block_calculate_payload_offset:
 diff --git a/block/crypto.c b/block/crypto.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/crypto.c
 +++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ block_crypto_create_opts_init(QDict *opts, Error **errp)
      return ret;
  }
 +QCryptoBlockAmendOptions *
 +block_crypto_amend_opts_init(QDict *opts, Error **errp)
 +{
 +    Visitor *v;
 +    QCryptoBlockAmendOptions *ret;
 +
 +    v = qobject_input_visitor_new_flat_confused(opts, errp);
 +    if (!v) {
 +        return NULL;
 +    }
 +
 +    visit_type_QCryptoBlockAmendOptions(v, NULL, &ret, errp);
 +
 +    visit_free(v);
 +    return ret;
 +}
 +
  static int block_crypto_open_generic(QCryptoBlockFormat format,
                                       QemuOptsList *opts_spec,
 diff --git a/crypto/block.c b/crypto/block.c
 index XXXXXXX..XXXXXXX 100644
 --- a/crypto/block.c
 +++ b/crypto/block.c
@@ -XXX,XX +XXX,XX @@ qcrypto_block_calculate_payload_offset(QCryptoBlockCreateOptions *create_opts,
      return crypto != NULL;
  }
 +int qcrypto_block_amend_options(QCryptoBlock *block,
 +                                QCryptoBlockReadFunc readfunc,
 +                                QCryptoBlockWriteFunc writefunc,
 +                                void *opaque,
 +                                QCryptoBlockAmendOptions *options,
 +                                bool force,
 +                                Error **errp)
 +{
 +    if (options->format != block->format) {
 +        error_setg(errp,
 +                   "Cannot amend encryption format");
 +        return -1;
 +    }
 +
 +    if (!block->driver->amend) {
 +        error_setg(errp,
 +                   "Crypto format %s doesn't support format options amendment",
 +                   QCryptoBlockFormat_str(block->format));
 +        return -1;
 +    }
 +
 +    return block->driver->amend(block,
 +                                readfunc,
 +                                writefunc,
 +                                opaque,
 +                                options,
 +                                force,
 +                                errp);
 +}
  QCryptoBlockInfo *qcrypto_block_get_info(QCryptoBlock *block,
                                           Error **errp)
 --
-.26.2
+.34.1

-[PULL 16/18] iotests: add tests for blockdev-amend
+[PULL 23/24] iotests/migration-permissions: New test
-From: Maxim Levitsky <mlevitsk@redhat.com>
+This test checks that a raw image in use by a virtio-blk device does not
 share the WRITE permission both before and after migration.
-This commit adds two tests that cover the
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
-new blockdev-amend functionality of luks and qcow2 driver
+---
  .../qemu-iotests/tests/migration-permissions  | 101 ++++++++++++++++++
  .../tests/migration-permissions.out           |   5 +
 files changed, 106 insertions(+)
  create mode 100755 tests/qemu-iotests/tests/migration-permissions
  create mode 100644 tests/qemu-iotests/tests/migration-permissions.out
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+diff --git a/tests/qemu-iotests/tests/migration-permissions b/tests/qemu-iotests/tests/migration-permissions
 Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
 Message-Id: <20200608094030.670121-15-mlevitsk@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
  tests/qemu-iotests/295     | 279 +++++++++++++++++++++++++++++++++++++
  tests/qemu-iotests/295.out |  40 ++++++
  tests/qemu-iotests/296     | 234 +++++++++++++++++++++++++++++++
  tests/qemu-iotests/296.out |  33 +++++
  tests/qemu-iotests/group   |   2 +
 files changed, 588 insertions(+)
  create mode 100755 tests/qemu-iotests/295
  create mode 100644 tests/qemu-iotests/295.out
  create mode 100755 tests/qemu-iotests/296
  create mode 100644 tests/qemu-iotests/296.out
 diff --git a/tests/qemu-iotests/295 b/tests/qemu-iotests/295
 new file mode 100755
 index XXXXXXX..XXXXXXX
 --- /dev/null
-+++ b/tests/qemu-iotests/295
++++ b/tests/qemu-iotests/tests/migration-permissions
 @@ -XXX,XX +XXX,XX @@
 +#!/usr/bin/env python3
++# group: migration
 +#
-+# Test case QMP's encrypted key management
++# Copyright (C) 2021 Red Hat, Inc.
 +#
 +# Copyright (C) 2019 Red Hat, Inc.
 +#
 +# This program is free software; you can redistribute it and/or modify
 +# it under the terms of the GNU General Public License as published by
 +# the Free Software Foundation; either version 2 of the License, or
 +# (at your option) any later version.
 ...
 +#
 +# You should have received a copy of the GNU General Public License
 +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 +#
 +
++import os
 +import iotests
-+import os
++from iotests import imgfmt, qemu_img_create, qemu_io
-+import time
++
 +import json
 +
 +test_img = os.path.join(iotests.test_dir, 'test.img')
++mig_sock = os.path.join(iotests.sock_dir, 'mig.sock')
 +
-+class Secret:
-+    def __init__(self, index):
-+        self._id = "keysec" + str(index)
-+        # you are not supposed to see the password...
-+        self._secret = "hunter" + str(index)
 +
-+    def id(self):
++class TestMigrationPermissions(iotests.QMPTestCase):
-+        return self._id
++    def setUp(self):
 +        qemu_img_create('-f', imgfmt, test_img, '1M')
 +
-+    def secret(self):
++        # Set up two VMs (source and destination) accessing the same raw
-+        return self._secret
++        # image file with a virtio-blk device; prepare the destination for
 +        # migration with .add_incoming() and enable migration events
 +        vms = [None, None]
 +        for i in range(2):
 +            vms[i] = iotests.VM(path_suffix=f'{i}')
 +            vms[i].add_blockdev(f'file,node-name=prot,filename={test_img}')
 +            vms[i].add_blockdev(f'{imgfmt},node-name=fmt,file=prot')
 +            vms[i].add_device('virtio-blk,drive=fmt')
 +
-+    def to_cmdline_object(self):
++            if i == 1:
-+        return  [ "secret,id=" + self._id + ",data=" + self._secret]
++                vms[i].add_incoming(f'unix:{mig_sock}')
 +
-+    def to_qmp_object(self):
++            vms[i].launch()
 +        return { "qom_type" : "secret", "id": self.id(),
 +                 "props": { "data": self.secret() } }
 +
-+################################################################################
++            result = vms[i].qmp('migrate-set-capabilities',
-+class EncryptionSetupTestCase(iotests.QMPTestCase):
++                                capabilities=[
-+
++                                    {'capability': 'events', 'state': True}
-+    # test case startup
++                                ])
 +    def setUp(self):
 +        # start the VM
 +        self.vm = iotests.VM()
 +        self.vm.launch()
 +
 +        # create the secrets and load 'em into the VM
 +        self.secrets = [ Secret(i) for i in range(0, 6) ]
 +        for secret in self.secrets:
 +            result = self.vm.qmp("object-add", **secret.to_qmp_object())
 +            self.assert_qmp(result, 'return', {})
 +
-+        if iotests.imgfmt == "qcow2":
++        self.vm_s = vms[0]
-+            self.pfx = "encrypt."
++        self.vm_d = vms[1]
 +            self.img_opts = [ '-o', "encrypt.format=luks" ]
 +        else:
 +            self.pfx = ""
 +            self.img_opts = []
 +
-+    # test case shutdown
 +    def tearDown(self):
-+        # stop the VM
++        self.vm_s.shutdown()
-+        self.vm.shutdown()
++        self.vm_d.shutdown()
-+
++        try:
-+    ###########################################################################
++            os.remove(mig_sock)
-+    # create the encrypted block device
++        except FileNotFoundError:
-+    def createImg(self, file, secret):
++            pass
 +
 +        iotests.qemu_img(
 +            'create',
 +            '--object', *secret.to_cmdline_object(),
 +            '-f', iotests.imgfmt,
 +            '-o', self.pfx + 'key-secret=' + secret.id(),
 +            '-o', self.pfx + 'iter-time=10',
 +            *self.img_opts,
 +            file,
 +            '1M')
 +
 +    ###########################################################################
 +    # open an encrypted block device
 +    def openImageQmp(self, id, file, secret, read_only = False):
 +
 +        encrypt_options = {
 +            'key-secret' : secret.id()
 +        }
 +
 +        if iotests.imgfmt == "qcow2":
 +            encrypt_options = {
 +                'encrypt': {
 +                    'format':'luks',
 +                    **encrypt_options
 +                }
 +            }
 +
 +        result = self.vm.qmp('blockdev-add', **
 +            {
 +                'driver': iotests.imgfmt,
 +                'node-name': id,
 +                'read-only': read_only,
 +
 +                **encrypt_options,
 +
 +                'file': {
 +                    'driver': 'file',
 +                    'filename': test_img,
 +                }
 +            }
 +        )
 +        self.assert_qmp(result, 'return', {})
 +
 +    # close the encrypted block device
 +    def closeImageQmp(self, id):
 +        result = self.vm.qmp('blockdev-del', **{ 'node-name': id })
 +        self.assert_qmp(result, 'return', {})
 +
 +    ###########################################################################
 +    # add a key to an encrypted block device
 +    def addKeyQmp(self, id, new_secret, secret = None,
 +                  slot = None, force = False):
 +
 +        crypt_options = {
 +            'state'      : 'active',
 +            'new-secret' : new_secret.id(),
 +            'iter-time' : 10
 +        }
 +
 +        if slot != None:
 +            crypt_options['keyslot'] = slot
 +
 +
 +        if secret != None:
 +            crypt_options['secret'] = secret.id()
 +
 +        if iotests.imgfmt == "qcow2":
 +            crypt_options['format'] = 'luks'
 +            crypt_options = {
 +                'encrypt': crypt_options
 +            }
 +
 +        args = {
 +            'node-name': id,
 +            'job-id' : 'job_add_key',
 +            'options' : {
 +                    'driver' : iotests.imgfmt,
 +                    **crypt_options
 +                },
 +        }
 +
 +        if force == True:
 +            args['force'] = True
 +
 +        #TODO: check what jobs return
 +        result = self.vm.qmp('x-blockdev-amend', **args)
 +        assert result['return'] == {}
 +        self.vm.run_job('job_add_key')
 +
 +    # erase a key from an encrypted block device
 +    def eraseKeyQmp(self, id, old_secret = None, slot = None, force = False):
 +
 +        crypt_options = {
 +            'state'      : 'inactive',
 +        }
 +
 +        if slot != None:
 +            crypt_options['keyslot'] = slot
 +        if old_secret != None:
 +            crypt_options['old-secret'] = old_secret.id()
 +
 +        if iotests.imgfmt == "qcow2":
 +            crypt_options['format'] = 'luks'
 +            crypt_options = {
 +                'encrypt': crypt_options
 +            }
 +
 +        args = {
 +            'node-name': id,
 +            'job-id' : 'job_erase_key',
 +            'options' : {
 +                    'driver' : iotests.imgfmt,
 +                    **crypt_options
 +                },
 +        }
 +
 +        if force == True:
 +            args['force'] = True
 +
 +        result = self.vm.qmp('x-blockdev-amend', **args)
 +        assert result['return'] == {}
 +        self.vm.run_job('job_erase_key')
 +
 +    ###########################################################################
 +    # create image, and change its key
 +    def testChangeKey(self):
 +
 +        # create the image with secret0 and open it
 +        self.createImg(test_img, self.secrets[0]);
 +        self.openImageQmp("testdev", test_img, self.secrets[0])
 +
 +        # add key to slot 1
 +        self.addKeyQmp("testdev", new_secret = self.secrets[1])
 +
 +        # add key to slot 5
 +        self.addKeyQmp("testdev", new_secret = self.secrets[2], slot=5)
 +
 +        # erase key from slot 0
 +        self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
 +
 +        #reopen the image with secret1
 +        self.closeImageQmp("testdev")
 +        self.openImageQmp("testdev", test_img, self.secrets[1])
 +
 +        # close and erase the image for good
 +        self.closeImageQmp("testdev")
 +        os.remove(test_img)
 +
-+    # test that if we erase the old password,
++    # Migrate an image in use by a virtio-blk device to another VM and
-+    # we can still change the encryption keys using 'old-secret'
++    # verify that the WRITE permission is unshared both before and after
-+    def testOldPassword(self):
++    # migration
 +    def test_post_migration_permissions(self):
 +        # Try to access the image R/W, which should fail because virtio-blk
 +        # has not been configured with share-rw=on
 +        log = qemu_io('-f', imgfmt, '-c', 'quit', test_img)
 +        if not log.strip():
 +            print('ERROR (pre-migration): qemu-io should not be able to '
 +                  'access this image, but it reported no error')
 +        else:
 +            # This is the expected output
 +            assert 'Is another process using the image' in log
 +
-+        # create the image with secret0 and open it
++        # Now migrate the VM
-+        self.createImg(test_img, self.secrets[0]);
++        self.vm_s.qmp('migrate', uri=f'unix:{mig_sock}')
-+        self.openImageQmp("testdev", test_img, self.secrets[0])
++        assert self.vm_s.wait_migration(None)
 +        assert self.vm_d.wait_migration(None)
 +
-+        # add key to slot 1
++        # Try the same qemu-io access again, verifying that the WRITE
-+        self.addKeyQmp("testdev", new_secret = self.secrets[1])
++        # permission remains unshared
-+
++        log = qemu_io('-f', imgfmt, '-c', 'quit', test_img)
-+        # erase key from slot 0
++        if not log.strip():
-+        self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
++            print('ERROR (post-migration): qemu-io should not be able to '
-+
++                  'access this image, but it reported no error')
-+        # this will fail as the old password is no longer valid
++        else:
-+        self.addKeyQmp("testdev", new_secret = self.secrets[2])
++            # This is the expected output
-+
++            assert 'Is another process using the image' in log
 +        # this will work
 +        self.addKeyQmp("testdev", new_secret = self.secrets[2], secret = self.secrets[1])
 +
 +        # close and erase the image for good
 +        self.closeImageQmp("testdev")
 +        os.remove(test_img)
 +
 +    def testUseForceLuke(self):
 +
 +        self.createImg(test_img, self.secrets[0]);
 +        self.openImageQmp("testdev", test_img, self.secrets[0])
 +
 +        # Add bunch of secrets
 +        self.addKeyQmp("testdev", new_secret = self.secrets[1], slot=4)
 +        self.addKeyQmp("testdev", new_secret = self.secrets[4], slot=2)
 +
 +        # overwrite an active secret
 +        self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2)
 +        self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2, force=True)
 +
 +        self.addKeyQmp("testdev", new_secret = self.secrets[0])
 +
 +        # Now erase all the secrets
 +        self.eraseKeyQmp("testdev", old_secret = self.secrets[5])
 +        self.eraseKeyQmp("testdev", slot=4)
 +
 +        # erase last keyslot
 +        self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
 +        self.eraseKeyQmp("testdev", old_secret = self.secrets[0], force=True)
 +
 +        self.closeImageQmp("testdev")
 +        os.remove(test_img)
 +
 +
 +if __name__ == '__main__':
-+    # Encrypted formats support
++    # Only works with raw images because we are testing the
-+    iotests.activate_logging()
++    # BlockBackend permissions; image format drivers may additionally
-+    iotests.main(supported_fmts = ['qcow2', 'luks'])
++    # unshare permissions and thus tamper with the result
-diff --git a/tests/qemu-iotests/295.out b/tests/qemu-iotests/295.out
++    iotests.main(supported_fmts=['raw'],
 +                 supported_protocols=['file'])
 diff --git a/tests/qemu-iotests/tests/migration-permissions.out b/tests/qemu-iotests/tests/migration-permissions.out
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
-+++ b/tests/qemu-iotests/295.out
++++ b/tests/qemu-iotests/tests/migration-permissions.out
 @@ -XXX,XX +XXX,XX @@
-+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
++.
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 +{"return": {}}
 +Job failed: Invalid password, cannot unlock any keyslot
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +Job failed: Refusing to overwrite active keyslot 2 - please erase it first
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 +{"return": {}}
 +Job failed: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
 +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 +{"return": {}}
 +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 +{"return": {}}
 +...
 +----------------------------------------------------------------------
-+Ran 3 tests
++Ran 1 tests
 +
 +OK
-diff --git a/tests/qemu-iotests/296 b/tests/qemu-iotests/296
-new file mode 100755
-index XXXXXXX..XXXXXXX
---- /dev/null
-+++ b/tests/qemu-iotests/296
-@@ -XXX,XX +XXX,XX @@
-+#!/usr/bin/env python3
-+#
-+# Test case for encryption key management versus image sharing
-+#
-+# Copyright (C) 2019 Red Hat, Inc.
-+#
-+# This program is free software; you can redistribute it and/or modify
-+# it under the terms of the GNU General Public License as published by
-+# the Free Software Foundation; either version 2 of the License, or
-+# (at your option) any later version.
-+#
-+# This program is distributed in the hope that it will be useful,
-+# but WITHOUT ANY WARRANTY; without even the implied warranty of
-+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-+# GNU General Public License for more details.
-+#
-+# You should have received a copy of the GNU General Public License
-+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-+#
-+
-+import iotests
-+import os
-+import time
-+import json
-+
-+test_img = os.path.join(iotests.test_dir, 'test.img')
-+
-+class Secret:
-+    def __init__(self, index):
-+        self._id = "keysec" + str(index)
-+        # you are not supposed to see the password...
-+        self._secret = "hunter" + str(index)
-+
-+    def id(self):
-+        return self._id
-+
-+    def secret(self):
-+        return self._secret
-+
-+    def to_cmdline_object(self):
-+        return  [ "secret,id=" + self._id + ",data=" + self._secret]
-+
-+    def to_qmp_object(self):
-+        return { "qom_type" : "secret", "id": self.id(),
-+                 "props": { "data": self.secret() } }
-+
-+################################################################################
-+
-+class EncryptionSetupTestCase(iotests.QMPTestCase):
-+
-+    # test case startup
-+    def setUp(self):
-+
-+        # start the VMs
-+        self.vm1 = iotests.VM(path_suffix = 'VM1')
-+        self.vm2 = iotests.VM(path_suffix = 'VM2')
-+        self.vm1.launch()
-+        self.vm2.launch()
-+
-+        # create the secrets and load 'em into the VMs
-+        self.secrets = [ Secret(i) for i in range(0, 4) ]
-+        for secret in self.secrets:
-+            result = self.vm1.qmp("object-add", **secret.to_qmp_object())
-+            self.assert_qmp(result, 'return', {})
-+            result = self.vm2.qmp("object-add", **secret.to_qmp_object())
-+            self.assert_qmp(result, 'return', {})
-+
-+    # test case shutdown
-+    def tearDown(self):
-+        # stop the VM
-+        self.vm1.shutdown()
-+        self.vm2.shutdown()
-+
-+    ###########################################################################
-+    # create the encrypted block device using qemu-img
-+    def createImg(self, file, secret):
-+
-+        output = iotests.qemu_img_pipe(
-+            'create',
-+            '--object', *secret.to_cmdline_object(),
-+            '-f', iotests.imgfmt,
-+            '-o', 'key-secret=' + secret.id(),
-+            '-o', 'iter-time=10',
-+            file,
-+            '1M')
-+
-+        iotests.log(output, filters=[iotests.filter_test_dir])
-+
-+    # attempts to add a key using qemu-img
-+    def addKey(self, file, secret, new_secret):
-+
-+        image_options = {
-+            'key-secret' : secret.id(),
-+            'driver' : iotests.imgfmt,
-+            'file' : {
-+                'driver':'file',
-+                'filename': file,
-+                }
-+            }
-+
-+        output = iotests.qemu_img_pipe(
-+            'amend',
-+            '--object', *secret.to_cmdline_object(),
-+            '--object', *new_secret.to_cmdline_object(),
-+
-+            '-o', 'state=active',
-+            '-o', 'new-secret=' + new_secret.id(),
-+            '-o', 'iter-time=10',
-+
-+            "json:" + json.dumps(image_options)
-+            )
-+
-+        iotests.log(output, filters=[iotests.filter_test_dir])
-+
-+    ###########################################################################
-+    # open an encrypted block device
-+    def openImageQmp(self, vm, id, file, secret,
-+                     readOnly = False, reOpen = False):
-+
-+        command = 'x-blockdev-reopen' if reOpen else 'blockdev-add'
-+
-+        result = vm.qmp(command, **
-+            {
-+                'driver': iotests.imgfmt,
-+                'node-name': id,
-+                'read-only': readOnly,
-+                'key-secret' : secret.id(),
-+                'file': {
-+                    'driver': 'file',
-+                    'filename': test_img,
-+                }
-+            }
-+        )
-+        self.assert_qmp(result, 'return', {})
-+
-+    # close the encrypted block device
-+    def closeImageQmp(self, vm, id):
-+        result = vm.qmp('blockdev-del', **{ 'node-name': id })
-+        self.assert_qmp(result, 'return', {})
-+
-+    ###########################################################################
-+
-+    # add a key to an encrypted block device
-+    def addKeyQmp(self, vm, id, new_secret):
-+
-+        args = {
-+            'node-name': id,
-+            'job-id' : 'job0',
-+            'options' : {
-+                'state'     : 'active',
-+                'driver'    : iotests.imgfmt,
-+                'new-secret': new_secret.id(),
-+                'iter-time' : 10
-+            },
-+        }
-+
-+        result = vm.qmp('x-blockdev-amend', **args)
-+        assert result['return'] == {}
-+        vm.run_job('job0')
-+
-+    # test that when the image opened by two qemu processes,
-+    # neither of them can update the image
-+    def test1(self):
-+        self.createImg(test_img, self.secrets[0]);
-+
-+        # VM1 opens the image and adds a key
-+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0])
-+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[1])
-+
-+
-+        # VM2 opens the image
-+        self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
-+
-+
-+        # neither VMs now should be able to add a key
-+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
-+        self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
-+
-+
-+        # VM 1 closes the image
-+        self.closeImageQmp(self.vm1, "testdev")
-+
-+
-+        # now VM2 can add the key
-+        self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
-+
-+
-+        # qemu-img should also not be able to add a key
-+        self.addKey(test_img, self.secrets[0], self.secrets[2])
-+
-+        # cleanup
-+        self.closeImageQmp(self.vm2, "testdev")
-+        os.remove(test_img)
-+
-+
-+    def test2(self):
-+        self.createImg(test_img, self.secrets[0]);
-+
-+        # VM1 opens the image readonly
-+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0],
-+                          readOnly = True)
-+
-+        # VM2 opens the image
-+        self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
-+
-+        # VM1 can't add a key since image is readonly
-+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
-+
-+        # VM2 can't add a key since VM is has the image opened
-+        self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
-+
-+
-+        #VM1 reopens the image read-write
-+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0],
-+                          reOpen = True, readOnly = False)
-+
-+        # VM1 still can't add the key
-+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
-+
-+        # VM2 gets away
-+        self.closeImageQmp(self.vm2, "testdev")
-+
-+        # VM1 now can add the key
-+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
-+
-+        self.closeImageQmp(self.vm1, "testdev")
-+        os.remove(test_img)
-+
-+
-+if __name__ == '__main__':
-+    # support only raw luks since luks encrypted qcow2 is a proper
-+    # format driver which doesn't allow any sharing
-+    iotests.activate_logging()
-+    iotests.main(supported_fmts = ['luks'])
-diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
-new file mode 100644
-index XXXXXXX..XXXXXXX
---- /dev/null
-+++ b/tests/qemu-iotests/296.out
-@@ -XXX,XX +XXX,XX @@
-+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-+
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+Job failed: Failed to get shared "consistent read" lock
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+Job failed: Failed to get shared "consistent read" lock
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+qemu-img: Failed to get shared "consistent read" lock
-+Is another process using the image [TEST_DIR/test.img]?
-+
-+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-+
-+Job failed: Block node is read-only
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+Job failed: Failed to get shared "consistent read" lock
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+Job failed: Failed to get shared "consistent read" lock
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
-+{"return": {}}
-+..
-+----------------------------------------------------------------------
-+Ran 2 tests
-+
-+OK
-diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
-index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/group
-+++ b/tests/qemu-iotests/group
-@@ -XXX,XX +XXX,XX @@
-rw auto quick
-rw auto
-rw auto quick
-+295 rw auto
-+296 rw auto
-meta
 --
-.26.2
+.34.1

-[PULL 06/18] block/amend: add 'force' option
+[PULL 24/24] block.h: remove outdated comment
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Emanuele Giuseppe Esposito <eesposit@redhat.com>
-'force' option will be used for some unsafe amend operations.
+The comment "disk I/O throttling" doesn't make any sense at all
 any more. It was added in commit 0563e191516 to describe
 bdrv_io_limits_enable()/disable(), which were removed in commit
 97148076, so the comment is just a forgotten leftover.
-This includes things like erasing last keyslot in luks based formats
+Suggested-by: Kevin Wolf <kwolf@redhat.com>
-(which destroys the data, unless the master key is backed up
+Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
-by external means), but that _might_ be desired result.
+Message-Id: <20220131125615.74612-1-eesposit@redhat.com>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  include/block/block.h | 1 -
 file changed, 1 deletion(-)
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
-Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
-Reviewed-by: Max Reitz <mreitz@redhat.com>
-Message-Id: <20200608094030.670121-4-mlevitsk@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- docs/tools/qemu-img.rst   | 5 ++++-
- include/block/block.h     | 1 +
- include/block/block_int.h | 1 +
- block.c                   | 4 +++-
- block/qcow2.c             | 1 +
- qemu-img.c                | 8 +++++++-
- qemu-img-cmds.hx          | 4 ++--
-files changed, 19 insertions(+), 5 deletions(-)
-diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
-index XXXXXXX..XXXXXXX 100644
---- a/docs/tools/qemu-img.rst
-+++ b/docs/tools/qemu-img.rst
-@@ -XXX,XX +XXX,XX @@ Command description:
- .. program:: qemu-img-commands
--.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME
-+.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME
-   Amends the image format specific *OPTIONS* for the image file
-   *FILENAME*. Not all file formats support this operation.
-+  --force allows some unsafe operations. Currently for -f luks, it allows to
-+  erase the last encryption key, and to overwrite an active encryption key.
-+
- .. option:: bench [-c COUNT] [-d DEPTH] [-f FMT] [--flush-interval=FLUSH_INTERVAL] [-i AIO] [-n] [--no-drain] [-o OFFSET] [--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t CACHE] [-w] [-U] FILENAME
-   Run a simple sequential I/O benchmark on the specified image. If ``-w`` is
 diff --git a/include/block/block.h b/include/block/block.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block.h
 +++ b/include/block/block.h
-@@ -XXX,XX +XXX,XX @@ typedef void BlockDriverAmendStatusCB(BlockDriverState *bs, int64_t offset,
+@@ -XXX,XX +XXX,XX @@ typedef unsigned int BdrvChildRole;
-                                       int64_t total_work_size, void *opaque);
+ char *bdrv_perm_names(uint64_t perm);
- int bdrv_amend_options(BlockDriverState *bs_new, QemuOpts *opts,
+ uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm);
-                        BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
-+                       bool force,
+-/* disk I/O throttling */
-                        Error **errp);
+ void bdrv_init(void);
+ void bdrv_init_with_whitelist(void);
- /* check if a named node can be replaced when doing drive-mirror */
+ bool bdrv_uses_whitelist(void);
 diff --git a/include/block/block_int.h b/include/block/block_int.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block_int.h
 +++ b/include/block/block_int.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
      int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
                                BlockDriverAmendStatusCB *status_cb,
                                void *cb_opaque,
 +                              bool force,
                                Error **errp);
      void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
 diff --git a/block.c b/block.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block.c
 +++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_remove_aio_context_notifier(BlockDriverState *bs,
  int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
                         BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
 +                       bool force,
                         Error **errp)
  {
      if (!bs->drv) {
@@ -XXX,XX +XXX,XX @@ int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
                     bs->drv->format_name);
          return -ENOTSUP;
      }
 -    return bs->drv->bdrv_amend_options(bs, opts, status_cb, cb_opaque, errp);
 +    return bs->drv->bdrv_amend_options(bs, opts, status_cb,
 +                                       cb_opaque, force, errp);
  }
  /*
 diff --git a/block/qcow2.c b/block/qcow2.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qcow2.c
 +++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static void qcow2_amend_helper_cb(BlockDriverState *bs,
  static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
                                 BlockDriverAmendStatusCB *status_cb,
                                 void *cb_opaque,
 +                               bool force,
                                 Error **errp)
  {
      BDRVQcow2State *s = bs->opaque;
 diff --git a/qemu-img.c b/qemu-img.c
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-img.c
 +++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ enum {
      OPTION_DISABLE = 273,
      OPTION_MERGE = 274,
      OPTION_BITMAPS = 275,
 +    OPTION_FORCE = 276,
  };
  typedef enum OutputFormat {
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
      BlockBackend *blk = NULL;
      BlockDriverState *bs = NULL;
      bool image_opts = false;
 +    bool force = false;
      cache = BDRV_DEFAULT_CACHE;
      for (;;) {
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
              {"help", no_argument, 0, 'h'},
              {"object", required_argument, 0, OPTION_OBJECT},
              {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
 +            {"force", no_argument, 0, OPTION_FORCE},
              {0, 0, 0, 0}
          };
          c = getopt_long(argc, argv, ":ho:f:t:pq",
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
          case OPTION_IMAGE_OPTS:
              image_opts = true;
              break;
 +        case OPTION_FORCE:
 +            force = true;
 +            break;
          }
      }
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
      /* In case the driver does not call amend_status_cb() */
      qemu_progress_print(0.f, 0);
 -    ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, &err);
 +    ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, force, &err);
      qemu_progress_print(100.f, 0);
      if (ret < 0) {
          error_report_err(err);
 diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-img-cmds.hx
 +++ b/qemu-img-cmds.hx
@@ -XXX,XX +XXX,XX @@ HXCOMM When amending the rST sections, please remember to copy the usage
  HXCOMM over to the per-command sections in docs/tools/qemu-img.rst.
  DEF("amend", img_amend,
 -    "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] -o options filename")
 +    "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] [--force] -o options filename")
  SRST
 -.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME
 +.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME
  ERST
  DEF("bench", img_bench,
 --
-.26.2
+.34.1

The following changes since commit bae31bfa48b9caecee25da3d5333901a126a06b4:

Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200619-pull-request' into staging (2020-06-19 22:56:59 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2020-06-22

for you to fetch changes up to 74c55e4142a7bb835c38d3770c74210cbb1e4fab:

iotests: don't test qcow2.py inside 291 (2020-06-22 16:05:23 +0200)

----------------------------------------------------------------
Block patches:
- Support modifying a LUKS-encrypted image's keyslots
- iotest fixes

----------------------------------------------------------------
Max Reitz (1):
  iotests: Make _filter_img_create more active

Maxim Levitsky (14):
  iotests: filter few more luks specific create options
  qcrypto/core: add generic infrastructure for crypto options amendment
  qcrypto/luks: implement encryption key management
  block/amend: add 'force' option
  block/amend: separate amend and create options for qemu-img
  block/amend: refactor qcow2 amend options
  block/crypto: rename two functions
  block/crypto: implement the encryption key management
  block/qcow2: extend qemu-img amend interface with crypto options
  iotests: qemu-img tests for luks key management
  block/core: add generic infrastructure for x-blockdev-amend qmp
    command
  block/crypto: implement blockdev-amend
  block/qcow2: implement blockdev-amend
  iotests: add tests for blockdev-amend

Philippe Mathieu-Daudé (1):
  iotests: Fix 051 output after qdev_init_nofail() removal

Vladimir Sementsov-Ogievskiy (2):
  block/block-copy: block_copy_dirty_clusters: fix failure check
  iotests: don't test qcow2.py inside 291

-- 
2.26.2

Right now, _filter_img_create just filters out everything that looks
format-dependent, and applies some filename filters.  That means that we
have to add another filter line every time some format gets a new
creation option.  This can be avoided by instead discarding everything
and just keeping what we know is format-independent (format, size,
backing file, encryption information[1], preallocation) or just
interesting to have in the reference output (external data file path).

Furthermore, we probably want to sort these options.  Format drivers are
not required to define them in any specific order, so the output is
effectively random (although this has never bothered us until now).  We
need a specific order for our reference outputs, though.  Unfortunately,
just using a plain "sort" would change a lot of existing reference
outputs, so we have to pre-filter the option keys to keep our existing
order (fmt, size, backing*, data, encryption info, preallocation).

Finally, this makes it difficult for _filter_img_create to automagically
work for QMP output.  Thus, this patch adds a separate
_filter_img_create_for_qmp function that echos every line verbatim that
does not start with "Formatting", and pipes those "Formatting" lines to
_filter_img_create.

[1] Actually, the only thing that is really important is whether
    encryption is enabled or not.  A patch by Maxim thus removes all
    other "encrypt.*" options from the output:
    https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00339.html
    But that patch needs to come later so we can get away with changing
    as few reference outputs in this patch here as possible.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200618150628.2169239-2-mreitz@redhat.com>
---
 tests/qemu-iotests/112.out       |  2 +-
 tests/qemu-iotests/141           |  2 +-
 tests/qemu-iotests/153           |  9 ++-
 tests/qemu-iotests/common.filter | 96 ++++++++++++++++++++++++--------
 4 files changed, 78 insertions(+), 31 deletions(-)

diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/112.out
+++ b/tests/qemu-iotests/112.out
@@ -XXX,XX +XXX,XX @@ QA output created by 112
 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 refcount_bits=-1
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/141
+++ b/tests/qemu-iotests/141
@@ -XXX,XX +XXX,XX @@ test_blockjob()
     _send_qemu_cmd $QEMU_HANDLE \
         "$1" \
         "$2" \
-        | _filter_img_create | _filter_qmp_empty_return
+        | _filter_img_create_in_qmp | _filter_qmp_empty_return
 
     # We want this to return an error because the block job is still running
     _send_qemu_cmd $QEMU_HANDLE \
diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/153
+++ b/tests/qemu-iotests/153
@@ -XXX,XX +XXX,XX @@ done
 
 echo
 echo "== Creating ${TEST_IMG}.[abc] ==" | _filter_testdir
-(
-    $QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}"
-    $QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}"
-    $QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b"
-) | _filter_img_create
+$QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" | _filter_img_create
+$QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" | _filter_img_create
+$QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" \
+    | _filter_img_create
 
 echo
 echo "== Two devices sharing the same file in backing chain =="
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_actual_image_size()
 # replace driver-specific options in the "Formatting..." line
 _filter_img_create()
 {
-    data_file_filter=()
-    if data_file=$(_get_data_file "$TEST_IMG"); then
-        data_file_filter=(-e "s# data_file=$data_file##")
+    # Split the line into the pre-options part ($filename_part, which
+    # precedes ", fmt=") and the options part ($options, which starts
+    # with "fmt=")
+    readarray -td '' formatting_line < <(sed -e 's/, fmt=/\x0/')
+
+    filename_part=${formatting_line[0]}
+    if [ -n "${formatting_line[1]}" ]; then
+        options="fmt=${formatting_line[1]}"
+    else
+        options=''
+    fi
+
+    # Set grep_data_file to '\|data_file' to keep it; make it empty
+    # to drop it.
+    # We want to drop it if it is part of the global $IMGOPTS, and we
+    # want to keep it otherwise (if the test specifically wants to
+    # test data files).
+    grep_data_file='\|data_file'
+    if _get_data_file "$TEST_IMG" > /dev/null; then
+        grep_data_file=''
     fi
 
-    $SED "${data_file_filter[@]}" \
+    filename_filters=(
         -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \
         -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
         -e "s#$TEST_DIR#TEST_DIR#g" \
         -e "s#$SOCK_DIR#SOCK_DIR#g" \
         -e "s#$IMGFMT#IMGFMT#g" \
         -e 's#nbd:127.0.0.1:[0-9]\\+#TEST_DIR/t.IMGFMT#g' \
-        -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' \
-        -e "s# encryption=off##g" \
-        -e "s# cluster_size=[0-9]\\+##g" \
-        -e "s# table_size=[0-9]\\+##g" \
-        -e "s# compat=[^ ]*##g" \
-        -e "s# compat6=\$on\\|off\$##g" \
-        -e "s# static=\$on\\|off\$##g" \
-        -e "s# zeroed_grain=\$on\\|off\$##g" \
-        -e "s# subformat=[^ ]*##g" \
-        -e "s# adapter_type=[^ ]*##g" \
-        -e "s# hwversion=[^ ]*##g" \
-        -e "s# lazy_refcounts=\$on\\|off\$##g" \
-        -e "s# block_size=[0-9]\\+##g" \
-        -e "s# block_state_zero=\$on\\|off\$##g" \
-        -e "s# log_size=[0-9]\\+##g" \
-        -e "s# refcount_bits=[0-9]\\+##g" \
-        -e "s# key-secret=[a-zA-Z0-9]\\+##g" \
-        -e "s# iter-time=[0-9]\\+##g" \
-        -e "s# force_size=\$on\\|off\$##g" \
-        -e "s# compression_type=[a-zA-Z0-9]\\+##g"
+        -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g'
+    )
+
+    filename_part=$(echo "$filename_part" | $SED "${filename_filters[@]}")
+
+    # Break the option line before each option (preserving pre-existing
+    # line breaks by replacing them by \0 and restoring them at the end),
+    # then filter out the options we want to keep and sort them according
+    # to some order that all block drivers used at the time of writing
+    # this function.
+    options=$(
+        echo "$options" \
+        | tr '\n' '\0' \
+        | $SED -e 's/\x0$//' -e 's/ $[a-z0-9_.-]*$=/\n\1=/g' \
+        | grep -ae "^$fmt\\|size\\|backing\\|preallocation\\|encrypt$grep_data_file\$" \
+        | $SED "${filename_filters[@]}" \
+            -e 's/^$fmt$/0-\1/' \
+            -e 's/^$size$/1-\1/' \
+            -e 's/^$backing$/2-\1/' \
+            -e 's/^$data_file$/3-\1/' \
+            -e 's/^$encryption$/4-\1/' \
+            -e 's/^$encrypt\.format$/5-\1/' \
+            -e 's/^$encrypt\.key-secret$/6-\1/' \
+            -e 's/^$encrypt\.iter-time$/7-\1/' \
+            -e 's/^$preallocation$/8-\1/' \
+        | sort \
+        | $SED -e 's/^[0-9]-//' \
+        | tr '\n\0' ' \n' \
+        | $SED -e 's/^ *$//' -e 's/ *$//'
+    )
+
+    if [ -n "$options" ]; then
+        echo "$filename_part, $options"
+    elif [ -n "$filename_part" ]; then
+        echo "$filename_part"
+    fi
+}
+
+# Filter the "Formatting..." line in QMP output (leaving the QMP output
+# untouched)
+# (In contrast to _filter_img_create(), this function does not support
+# multi-line Formatting output)
+_filter_img_create_in_qmp()
+{
+    while read -r line; do
+        if echo "$line" | grep -q '^Formatting'; then
+            echo "$line" | _filter_img_create
+        else
+            echo "$line"
+        fi
+    done
 }
 
 _filter_img_create_size()
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

This allows more tests to be able to have same output on both qcow2 luks encrypted images
and raw luks images

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20200618150628.2169239-3-mreitz@redhat.com>
---
 tests/qemu-iotests/087.out       | 6 +++---
 tests/qemu-iotests/134.out       | 2 +-
 tests/qemu-iotests/158.out       | 4 ++--
 tests/qemu-iotests/188.out       | 2 +-
 tests/qemu-iotests/189.out       | 4 ++--
 tests/qemu-iotests/198.out       | 4 ++--
 tests/qemu-iotests/263.out       | 4 ++--
 tests/qemu-iotests/284.out       | 6 +++---
 tests/qemu-iotests/common.filter | 5 +----
 9 files changed, 17 insertions(+), 20 deletions(-)

diff --git a/tests/qemu-iotests/087.out b/tests/qemu-iotests/087.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/087.out
+++ b/tests/qemu-iotests/087.out
@@ -XXX,XX +XXX,XX @@ QMP_VERSION
 
 === Encrypted image QCow ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 Testing:
 QMP_VERSION
 {"return": {}}
@@ -XXX,XX +XXX,XX @@ QMP_VERSION
 
 === Encrypted image LUKS ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 Testing:
 QMP_VERSION
 {"return": {}}
@@ -XXX,XX +XXX,XX @@ QMP_VERSION
 
 === Missing driver ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 Testing: -S
 QMP_VERSION
 {"return": {}}
diff --git a/tests/qemu-iotests/134.out b/tests/qemu-iotests/134.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/134.out
+++ b/tests/qemu-iotests/134.out
@@ -XXX,XX +XXX,XX @@
 QA output created by 134
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on
 
 == reading whole image ==
 read 134217728/134217728 bytes at offset 0
diff --git a/tests/qemu-iotests/158.out b/tests/qemu-iotests/158.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/158.out
+++ b/tests/qemu-iotests/158.out
@@ -XXX,XX +XXX,XX @@
 QA output created by 158
 == create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on
 
 == writing whole image ==
 wrote 134217728/134217728 bytes at offset 0
@@ -XXX,XX +XXX,XX @@ wrote 134217728/134217728 bytes at offset 0
 read 134217728/134217728 bytes at offset 0
 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 == create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on
 
 == writing part of a cluster ==
 wrote 1024/1024 bytes at offset 0
diff --git a/tests/qemu-iotests/188.out b/tests/qemu-iotests/188.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/188.out
+++ b/tests/qemu-iotests/188.out
@@ -XXX,XX +XXX,XX @@
 QA output created by 188
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216
 
 == reading whole image ==
 read 16777216/16777216 bytes at offset 0
diff --git a/tests/qemu-iotests/189.out b/tests/qemu-iotests/189.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/189.out
+++ b/tests/qemu-iotests/189.out
@@ -XXX,XX +XXX,XX @@
 QA output created by 189
 == create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
 
 == writing whole image ==
 wrote 16777216/16777216 bytes at offset 0
@@ -XXX,XX +XXX,XX @@ wrote 16777216/16777216 bytes at offset 0
 read 16777216/16777216 bytes at offset 0
 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 == create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
 
 == writing part of a cluster ==
 wrote 1024/1024 bytes at offset 0
diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/198.out
+++ b/tests/qemu-iotests/198.out
@@ -XXX,XX +XXX,XX @@
 QA output created by 198
 == create base ==
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216
 
 == writing whole image base ==
 wrote 16777216/16777216 bytes at offset 0
 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 == create overlay ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base
 
 == writing whole image layer ==
 wrote 16777216/16777216 bytes at offset 0
diff --git a/tests/qemu-iotests/263.out b/tests/qemu-iotests/263.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/263.out
+++ b/tests/qemu-iotests/263.out
@@ -XXX,XX +XXX,XX @@ QA output created by 263
 
 testing LUKS qcow2 encryption
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 == reading the whole image ==
 read 1048576/1048576 bytes at offset 0
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 982528/982528 bytes at offset 66048
 
 testing legacy AES qcow2 encryption
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=aes encrypt.key-secret=sec0
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 == reading the whole image ==
 read 1048576/1048576 bytes at offset 0
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/284.out b/tests/qemu-iotests/284.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/284.out
+++ b/tests/qemu-iotests/284.out
@@ -XXX,XX +XXX,XX @@ QA output created by 284
 
 testing LUKS qcow2 encryption
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 
 == cluster size 512
 == checking image refcounts ==
@@ -XXX,XX +XXX,XX @@ wrote 1/1 bytes at offset 512
 
 == rechecking image refcounts ==
 No errors were found on the image.
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 
 == cluster size 2048
 == checking image refcounts ==
@@ -XXX,XX +XXX,XX @@ wrote 1/1 bytes at offset 2048
 
 == rechecking image refcounts ==
 No errors were found on the image.
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 
 == cluster size 32768
 == checking image refcounts ==
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_img_create()
         echo "$options" \
         | tr '\n' '\0' \
         | $SED -e 's/\x0$//' -e 's/ $[a-z0-9_.-]*$=/\n\1=/g' \
-        | grep -ae "^$fmt\\|size\\|backing\\|preallocation\\|encrypt$grep_data_file\$" \
+        | grep -ae "^$fmt\\|size\\|backing\\|preallocation\\|encryption$grep_data_file\$" \
         | $SED "${filename_filters[@]}" \
             -e 's/^$fmt$/0-\1/' \
             -e 's/^$size$/1-\1/' \
             -e 's/^$backing$/2-\1/' \
             -e 's/^$data_file$/3-\1/' \
             -e 's/^$encryption$/4-\1/' \
-            -e 's/^$encrypt\.format$/5-\1/' \
-            -e 's/^$encrypt\.key-secret$/6-\1/' \
-            -e 's/^$encrypt\.iter-time$/7-\1/' \
             -e 's/^$preallocation$/8-\1/' \
         | sort \
         | $SED -e 's/^[0-9]-//' \
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

ret may be > 0 on success path at this point. Fix assertion, which may
crash currently.

Fixes: 4ce5dd3e9b5ee0fac18625860eb3727399ee965e
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200526181347.489557-1-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/block-copy.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/block/block-copy.c b/block/block-copy.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -XXX,XX +XXX,XX @@ out:
          * block_copy_task_run. If it fails, it means some task already failed
          * for real reason, let's return first failure.
          * Still, assert that we don't rewrite failure by success.
+         *
+         * Note: ret may be positive here because of block-status result.
          */
-        assert(ret == 0 || aio_task_pool_status(aio) < 0);
+        assert(ret >= 0 || aio_task_pool_status(aio) < 0);
         ret = aio_task_pool_status(aio);
 
         aio_task_pool_free(aio);
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

This will be used first to implement luks keyslot management.

block_crypto_amend_opts_init will be used to convert
qemu-img cmdline to QCryptoBlockAmendOptions

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-2-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qapi/crypto.json       | 16 ++++++++++++++++
 block/crypto.h         |  3 +++
 crypto/blockpriv.h     |  8 ++++++++
 include/crypto/block.h | 22 ++++++++++++++++++++++
 block/crypto.c         | 17 +++++++++++++++++
 crypto/block.c         | 29 +++++++++++++++++++++++++++++
 6 files changed, 95 insertions(+)

diff --git a/qapi/crypto.json b/qapi/crypto.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -XXX,XX +XXX,XX @@
   'base': 'QCryptoBlockInfoBase',
   'discriminator': 'format',
   'data': { 'luks': 'QCryptoBlockInfoLUKS' } }
+
+
+
+##
+# @QCryptoBlockAmendOptions:
+#
+# The options that are available for all encryption formats
+# when amending encryption settings
+#
+# Since: 5.1
+##
+{ 'union': 'QCryptoBlockAmendOptions',
+  'base': 'QCryptoBlockOptionsBase',
+  'discriminator': 'format',
+  'data': {
+            } }
diff --git a/block/crypto.h b/block/crypto.h
index XXXXXXX..XXXXXXX 100644
--- a/block/crypto.h
+++ b/block/crypto.h
@@ -XXX,XX +XXX,XX @@
 QCryptoBlockCreateOptions *
 block_crypto_create_opts_init(QDict *opts, Error **errp);
 
+QCryptoBlockAmendOptions *
+block_crypto_amend_opts_init(QDict *opts, Error **errp);
+
 QCryptoBlockOpenOptions *
 block_crypto_open_opts_init(QDict *opts, Error **errp);
 
diff --git a/crypto/blockpriv.h b/crypto/blockpriv.h
index XXXXXXX..XXXXXXX 100644
--- a/crypto/blockpriv.h
+++ b/crypto/blockpriv.h
@@ -XXX,XX +XXX,XX @@ struct QCryptoBlockDriver {
                   void *opaque,
                   Error **errp);
 
+    int (*amend)(QCryptoBlock *block,
+                 QCryptoBlockReadFunc readfunc,
+                 QCryptoBlockWriteFunc writefunc,
+                 void *opaque,
+                 QCryptoBlockAmendOptions *options,
+                 bool force,
+                 Error **errp);
+
     int (*get_info)(QCryptoBlock *block,
                     QCryptoBlockInfo *info,
                     Error **errp);
diff --git a/include/crypto/block.h b/include/crypto/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/crypto/block.h
+++ b/include/crypto/block.h
@@ -XXX,XX +XXX,XX @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,
                                    void *opaque,
                                    Error **errp);
 
+/**
+ * qcrypto_block_amend_options:
+ * @block: the block encryption object
+ *
+ * @readfunc: callback for reading data from the volume header
+ * @writefunc: callback for writing data to the volume header
+ * @opaque: data to pass to @readfunc and @writefunc
+ * @options: the new/amended encryption options
+ * @force: hint for the driver to allow unsafe operation
+ * @errp: error pointer
+ *
+ * Changes the crypto options of the encryption format
+ *
+ */
+int qcrypto_block_amend_options(QCryptoBlock *block,
+                                QCryptoBlockReadFunc readfunc,
+                                QCryptoBlockWriteFunc writefunc,
+                                void *opaque,
+                                QCryptoBlockAmendOptions *options,
+                                bool force,
+                                Error **errp);
+
 
 /**
  * qcrypto_block_calculate_payload_offset:
diff --git a/block/crypto.c b/block/crypto.c
index XXXXXXX..XXXXXXX 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ block_crypto_create_opts_init(QDict *opts, Error **errp)
     return ret;
 }
 
+QCryptoBlockAmendOptions *
+block_crypto_amend_opts_init(QDict *opts, Error **errp)
+{
+    Visitor *v;
+    QCryptoBlockAmendOptions *ret;
+
+    v = qobject_input_visitor_new_flat_confused(opts, errp);
+    if (!v) {
+        return NULL;
+    }
+
+    visit_type_QCryptoBlockAmendOptions(v, NULL, &ret, errp);
+
+    visit_free(v);
+    return ret;
+}
+
 
 static int block_crypto_open_generic(QCryptoBlockFormat format,
                                      QemuOptsList *opts_spec,
diff --git a/crypto/block.c b/crypto/block.c
index XXXXXXX..XXXXXXX 100644
--- a/crypto/block.c
+++ b/crypto/block.c
@@ -XXX,XX +XXX,XX @@ qcrypto_block_calculate_payload_offset(QCryptoBlockCreateOptions *create_opts,
     return crypto != NULL;
 }
 
+int qcrypto_block_amend_options(QCryptoBlock *block,
+                                QCryptoBlockReadFunc readfunc,
+                                QCryptoBlockWriteFunc writefunc,
+                                void *opaque,
+                                QCryptoBlockAmendOptions *options,
+                                bool force,
+                                Error **errp)
+{
+    if (options->format != block->format) {
+        error_setg(errp,
+                   "Cannot amend encryption format");
+        return -1;
+    }
+
+    if (!block->driver->amend) {
+        error_setg(errp,
+                   "Crypto format %s doesn't support format options amendment",
+                   QCryptoBlockFormat_str(block->format));
+        return -1;
+    }
+
+    return block->driver->amend(block,
+                                readfunc,
+                                writefunc,
+                                opaque,
+                                options,
+                                force,
+                                errp);
+}
 
 QCryptoBlockInfo *qcrypto_block_get_info(QCryptoBlock *block,
                                          Error **errp)
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

Next few patches will expose that functionality to the user.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-3-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qapi/crypto.json    |  59 ++++++-
 crypto/block-luks.c | 416 +++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 469 insertions(+), 6 deletions(-)

diff --git a/qapi/crypto.json b/qapi/crypto.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/crypto.json
+++ b/qapi/crypto.json
@@ -XXX,XX +XXX,XX @@
            'uuid': 'str',
            'slots': [ 'QCryptoBlockInfoLUKSSlot' ] }}
 
-
 ##
 # @QCryptoBlockInfo:
 #
@@ -XXX,XX +XXX,XX @@
   'discriminator': 'format',
   'data': { 'luks': 'QCryptoBlockInfoLUKS' } }
 
+##
+# @QCryptoBlockLUKSKeyslotState:
+#
+# Defines state of keyslots that are affected by the update
+#
+# @active:    The slots contain the given password and marked as active
+# @inactive:  The slots are erased (contain garbage) and marked as inactive
+#
+# Since: 5.1
+##
+{ 'enum': 'QCryptoBlockLUKSKeyslotState',
+  'data': [ 'active', 'inactive' ] }
+
 
+##
+# @QCryptoBlockAmendOptionsLUKS:
+#
+# This struct defines the update parameters that activate/de-activate set
+# of keyslots
+#
+# @state: the desired state of the keyslots
+#
+# @new-secret:    The ID of a QCryptoSecret object providing the password to be
+#                 written into added active keyslots
+#
+# @old-secret:    Optional (for deactivation only)
+#                 If given will deactive all keyslots that
+#                 match password located in QCryptoSecret with this ID
+#
+# @iter-time:     Optional (for activation only)
+#                 Number of milliseconds to spend in
+#                 PBKDF passphrase processing for the newly activated keyslot.
+#                 Currently defaults to 2000.
+#
+# @keyslot:       Optional. ID of the keyslot to activate/deactivate.
+#                 For keyslot activation, keyslot should not be active already
+#                 (this is unsafe to update an active keyslot),
+#                 but possible if 'force' parameter is given.
+#                 If keyslot is not given, first free keyslot will be written.
+#
+#                 For keyslot deactivation, this parameter specifies the exact
+#                 keyslot to deactivate
+#
+# @secret:        Optional. The ID of a QCryptoSecret object providing the
+#                 password to use to retrive current master key.
+#                 Defaults to the same secret that was used to open the image
+#
+#
+# Since 5.1
+##
+{ 'struct': 'QCryptoBlockAmendOptionsLUKS',
+  'data': { 'state': 'QCryptoBlockLUKSKeyslotState',
+            '*new-secret': 'str',
+            '*old-secret': 'str',
+            '*keyslot': 'int',
+            '*iter-time': 'int',
+            '*secret': 'str' } }
 
 ##
 # @QCryptoBlockAmendOptions:
@@ -XXX,XX +XXX,XX @@
   'base': 'QCryptoBlockOptionsBase',
   'discriminator': 'format',
   'data': {
-            } }
+          'luks': 'QCryptoBlockAmendOptionsLUKS' } }
diff --git a/crypto/block-luks.c b/crypto/block-luks.c
index XXXXXXX..XXXXXXX 100644
--- a/crypto/block-luks.c
+++ b/crypto/block-luks.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/uuid.h"
 
 #include "qemu/coroutine.h"
+#include "qemu/bitmap.h"
 
 /*
  * Reference for the LUKS format implemented here is
@@ -XXX,XX +XXX,XX @@ typedef struct QCryptoBlockLUKSKeySlot QCryptoBlockLUKSKeySlot;
 
 #define QCRYPTO_BLOCK_LUKS_SECTOR_SIZE 512LL
 
+#define QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS 2000
+#define QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS 40
+
 static const char qcrypto_block_luks_magic[QCRYPTO_BLOCK_LUKS_MAGIC_LEN] = {
     'L', 'U', 'K', 'S', 0xBA, 0xBE
 };
@@ -XXX,XX +XXX,XX @@ struct QCryptoBlockLUKS {
 
     /* Hash algorithm used in pbkdf2 function */
     QCryptoHashAlgorithm hash_alg;
+
+    /* Name of the secret that was used to open the image */
+    char *secret;
 };
 
 
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_store_key(QCryptoBlock *block,
                              Error **errp)
 {
     QCryptoBlockLUKS *luks = block->opaque;
-    QCryptoBlockLUKSKeySlot *slot = &luks->header.key_slots[slot_idx];
+    QCryptoBlockLUKSKeySlot *slot;
     g_autofree uint8_t *splitkey = NULL;
     size_t splitkeylen;
     g_autofree uint8_t *slotkey = NULL;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_store_key(QCryptoBlock *block,
     uint64_t iters;
     int ret = -1;
 
+    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+    slot = &luks->header.key_slots[slot_idx];
     if (qcrypto_random_bytes(slot->salt,
                              QCRYPTO_BLOCK_LUKS_SALT_LEN,
                              errp) < 0) {
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_load_key(QCryptoBlock *block,
                             Error **errp)
 {
     QCryptoBlockLUKS *luks = block->opaque;
-    const QCryptoBlockLUKSKeySlot *slot = &luks->header.key_slots[slot_idx];
+    const QCryptoBlockLUKSKeySlot *slot;
     g_autofree uint8_t *splitkey = NULL;
     size_t splitkeylen;
     g_autofree uint8_t *possiblekey = NULL;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_load_key(QCryptoBlock *block,
     g_autoptr(QCryptoIVGen) ivgen = NULL;
     size_t niv;
 
+    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+    slot = &luks->header.key_slots[slot_idx];
     if (slot->active != QCRYPTO_BLOCK_LUKS_KEY_SLOT_ENABLED) {
         return 0;
     }
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_find_key(QCryptoBlock *block,
     return -1;
 }
 
+/*
+ * Returns true if a slot i is marked as active
+ * (contains encrypted copy of the master key)
+ */
+static bool
+qcrypto_block_luks_slot_active(const QCryptoBlockLUKS *luks,
+                               unsigned int slot_idx)
+{
+    uint32_t val;
+
+    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+    val = luks->header.key_slots[slot_idx].active;
+    return val == QCRYPTO_BLOCK_LUKS_KEY_SLOT_ENABLED;
+}
+
+/*
+ * Returns the number of slots that are marked as active
+ * (slots that contain encrypted copy of the master key)
+ */
+static unsigned int
+qcrypto_block_luks_count_active_slots(const QCryptoBlockLUKS *luks)
+{
+    size_t i = 0;
+    unsigned int ret = 0;
+
+    for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+        if (qcrypto_block_luks_slot_active(luks, i)) {
+            ret++;
+        }
+    }
+    return ret;
+}
+
+/*
+ * Finds first key slot which is not active
+ * Returns the key slot index, or -1 if it doesn't exist
+ */
+static int
+qcrypto_block_luks_find_free_keyslot(const QCryptoBlockLUKS *luks)
+{
+    size_t i;
+
+    for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+        if (!qcrypto_block_luks_slot_active(luks, i)) {
+            return i;
+        }
+    }
+    return -1;
+}
+
+/*
+ * Erases an keyslot given its index
+ * Returns:
+ *    0 if the keyslot was erased successfully
+ *   -1 if a error occurred while erasing the keyslot
+ *
+ */
+static int
+qcrypto_block_luks_erase_key(QCryptoBlock *block,
+                             unsigned int slot_idx,
+                             QCryptoBlockWriteFunc writefunc,
+                             void *opaque,
+                             Error **errp)
+{
+    QCryptoBlockLUKS *luks = block->opaque;
+    QCryptoBlockLUKSKeySlot *slot;
+    g_autofree uint8_t *garbagesplitkey = NULL;
+    size_t splitkeylen;
+    size_t i;
+    Error *local_err = NULL;
+    int ret;
+
+    assert(slot_idx < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+    slot = &luks->header.key_slots[slot_idx];
+
+    splitkeylen = luks->header.master_key_len * slot->stripes;
+    assert(splitkeylen > 0);
+
+    garbagesplitkey = g_new0(uint8_t, splitkeylen);
+
+    /* Reset the key slot header */
+    memset(slot->salt, 0, QCRYPTO_BLOCK_LUKS_SALT_LEN);
+    slot->iterations = 0;
+    slot->active = QCRYPTO_BLOCK_LUKS_KEY_SLOT_DISABLED;
+
+    ret = qcrypto_block_luks_store_header(block, writefunc,
+                                          opaque, &local_err);
+
+    if (ret < 0) {
+        error_propagate(errp, local_err);
+    }
+    /*
+     * Now try to erase the key material, even if the header
+     * update failed
+     */
+    for (i = 0; i < QCRYPTO_BLOCK_LUKS_ERASE_ITERATIONS; i++) {
+        if (qcrypto_random_bytes(garbagesplitkey,
+                                 splitkeylen, &local_err) < 0) {
+            /*
+             * If we failed to get the random data, still write
+             * at least zeros to the key slot at least once
+             */
+            error_propagate(errp, local_err);
+
+            if (i > 0) {
+                return -1;
+            }
+        }
+        if (writefunc(block,
+                      slot->key_offset_sector * QCRYPTO_BLOCK_LUKS_SECTOR_SIZE,
+                      garbagesplitkey,
+                      splitkeylen,
+                      opaque,
+                      &local_err) != splitkeylen) {
+            error_propagate(errp, local_err);
+            return -1;
+        }
+    }
+    return ret;
+}
 
 static int
 qcrypto_block_luks_open(QCryptoBlock *block,
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_open(QCryptoBlock *block,
 
     luks = g_new0(QCryptoBlockLUKS, 1);
     block->opaque = luks;
+    luks->secret = g_strdup(options->u.luks.key_secret);
 
     if (qcrypto_block_luks_load_header(block, readfunc, opaque, errp) < 0) {
         goto fail;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_open(QCryptoBlock *block,
  fail:
     qcrypto_block_free_cipher(block);
     qcrypto_ivgen_free(block->ivgen);
+    g_free(luks->secret);
     g_free(luks);
     return -1;
 }
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_create(QCryptoBlock *block,
 
     memcpy(&luks_opts, &options->u.luks, sizeof(luks_opts));
     if (!luks_opts.has_iter_time) {
-        luks_opts.iter_time = 2000;
+        luks_opts.iter_time = QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
     }
     if (!luks_opts.has_cipher_alg) {
         luks_opts.cipher_alg = QCRYPTO_CIPHER_ALG_AES_256;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_create(QCryptoBlock *block,
                    optprefix ? optprefix : "");
         goto error;
     }
+    luks->secret = g_strdup(options->u.luks.key_secret);
+
     password = qcrypto_secret_lookup_as_utf8(luks_opts.key_secret, errp);
     if (!password) {
         goto error;
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_create(QCryptoBlock *block,
     qcrypto_block_free_cipher(block);
     qcrypto_ivgen_free(block->ivgen);
 
+    g_free(luks->secret);
     g_free(luks);
     return -1;
 }
 
+static int
+qcrypto_block_luks_amend_add_keyslot(QCryptoBlock *block,
+                                     QCryptoBlockReadFunc readfunc,
+                                     QCryptoBlockWriteFunc writefunc,
+                                     void *opaque,
+                                     QCryptoBlockAmendOptionsLUKS *opts_luks,
+                                     bool force,
+                                     Error **errp)
+{
+    QCryptoBlockLUKS *luks = block->opaque;
+    uint64_t iter_time = opts_luks->has_iter_time ?
+                         opts_luks->iter_time :
+                         QCRYPTO_BLOCK_LUKS_DEFAULT_ITER_TIME_MS;
+    int keyslot;
+    g_autofree char *old_password = NULL;
+    g_autofree char *new_password = NULL;
+    g_autofree uint8_t *master_key = NULL;
+
+    char *secret = opts_luks->has_secret ? opts_luks->secret : luks->secret;
+
+    if (!opts_luks->has_new_secret) {
+        error_setg(errp, "'new-secret' is required to activate a keyslot");
+        return -1;
+    }
+    if (opts_luks->has_old_secret) {
+        error_setg(errp,
+                   "'old-secret' must not be given when activating keyslots");
+        return -1;
+    }
+
+    if (opts_luks->has_keyslot) {
+        keyslot = opts_luks->keyslot;
+        if (keyslot < 0 || keyslot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS) {
+            error_setg(errp,
+                       "Invalid keyslot %u specified, must be between 0 and %u",
+                       keyslot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1);
+            return -1;
+        }
+    } else {
+        keyslot = qcrypto_block_luks_find_free_keyslot(luks);
+        if (keyslot == -1) {
+            error_setg(errp,
+                       "Can't add a keyslot - all keyslots are in use");
+            return -1;
+        }
+    }
+
+    if (!force && qcrypto_block_luks_slot_active(luks, keyslot)) {
+        error_setg(errp,
+                   "Refusing to overwrite active keyslot %i - "
+                   "please erase it first",
+                   keyslot);
+        return -1;
+    }
+
+    /* Locate the password that will be used to retrieve the master key */
+    old_password = qcrypto_secret_lookup_as_utf8(secret, errp);
+    if (!old_password) {
+        return -1;
+    }
+
+    /* Retrieve the master key */
+    master_key = g_new0(uint8_t, luks->header.master_key_len);
+
+    if (qcrypto_block_luks_find_key(block, old_password, master_key,
+                                    readfunc, opaque, errp) < 0) {
+        error_append_hint(errp, "Failed to retrieve the master key");
+        return -1;
+    }
+
+    /* Locate the new password*/
+    new_password = qcrypto_secret_lookup_as_utf8(opts_luks->new_secret, errp);
+    if (!new_password) {
+        return -1;
+    }
+
+    /* Now set the new keyslots */
+    if (qcrypto_block_luks_store_key(block, keyslot, new_password, master_key,
+                                     iter_time, writefunc, opaque, errp)) {
+        error_append_hint(errp, "Failed to write to keyslot %i", keyslot);
+        return -1;
+    }
+    return 0;
+}
+
+static int
+qcrypto_block_luks_amend_erase_keyslots(QCryptoBlock *block,
+                                        QCryptoBlockReadFunc readfunc,
+                                        QCryptoBlockWriteFunc writefunc,
+                                        void *opaque,
+                                        QCryptoBlockAmendOptionsLUKS *opts_luks,
+                                        bool force,
+                                        Error **errp)
+{
+    QCryptoBlockLUKS *luks = block->opaque;
+    g_autofree uint8_t *tmpkey = NULL;
+    g_autofree char *old_password = NULL;
+
+    if (opts_luks->has_new_secret) {
+        error_setg(errp,
+                   "'new-secret' must not be given when erasing keyslots");
+        return -1;
+    }
+    if (opts_luks->has_iter_time) {
+        error_setg(errp,
+                   "'iter-time' must not be given when erasing keyslots");
+        return -1;
+    }
+    if (opts_luks->has_secret) {
+        error_setg(errp,
+                   "'secret' must not be given when erasing keyslots");
+        return -1;
+    }
+
+    /* Load the old password if given */
+    if (opts_luks->has_old_secret) {
+        old_password = qcrypto_secret_lookup_as_utf8(opts_luks->old_secret,
+                                                     errp);
+        if (!old_password) {
+            return -1;
+        }
+
+        /*
+         * Allocate a temporary key buffer that we will need when
+         * checking if slot matches the given old password
+         */
+        tmpkey = g_new0(uint8_t, luks->header.master_key_len);
+    }
+
+    /* Erase an explicitly given keyslot */
+    if (opts_luks->has_keyslot) {
+        int keyslot = opts_luks->keyslot;
+
+        if (keyslot < 0 || keyslot >= QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS) {
+            error_setg(errp,
+                       "Invalid keyslot %i specified, must be between 0 and %i",
+                       keyslot, QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS - 1);
+            return -1;
+        }
+
+        if (opts_luks->has_old_secret) {
+            int rv = qcrypto_block_luks_load_key(block,
+                                                 keyslot,
+                                                 old_password,
+                                                 tmpkey,
+                                                 readfunc,
+                                                 opaque,
+                                                 errp);
+            if (rv == -1) {
+                return -1;
+            } else if (rv == 0) {
+                error_setg(errp,
+                           "Given keyslot %i doesn't contain the given "
+                           "old password for erase operation",
+                           keyslot);
+                return -1;
+            }
+        }
+
+        if (!force && !qcrypto_block_luks_slot_active(luks, keyslot)) {
+            error_setg(errp,
+                       "Given keyslot %i is already erased (inactive) ",
+                       keyslot);
+            return -1;
+        }
+
+        if (!force && qcrypto_block_luks_count_active_slots(luks) == 1) {
+            error_setg(errp,
+                       "Attempt to erase the only active keyslot %i "
+                       "which will erase all the data in the image "
+                       "irreversibly - refusing operation",
+                       keyslot);
+            return -1;
+        }
+
+        if (qcrypto_block_luks_erase_key(block, keyslot,
+                                         writefunc, opaque, errp)) {
+            error_append_hint(errp, "Failed to erase keyslot %i", keyslot);
+            return -1;
+        }
+
+    /* Erase all keyslots that match the given old password */
+    } else if (opts_luks->has_old_secret) {
+
+        unsigned long slots_to_erase_bitmap = 0;
+        size_t i;
+        int slot_count;
+
+        assert(QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS <=
+               sizeof(slots_to_erase_bitmap) * 8);
+
+        for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+            int rv = qcrypto_block_luks_load_key(block,
+                                                 i,
+                                                 old_password,
+                                                 tmpkey,
+                                                 readfunc,
+                                                 opaque,
+                                                 errp);
+            if (rv == -1) {
+                return -1;
+            } else if (rv == 1) {
+                bitmap_set(&slots_to_erase_bitmap, i, 1);
+            }
+        }
+
+        slot_count = bitmap_count_one(&slots_to_erase_bitmap,
+                                      QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS);
+        if (slot_count == 0) {
+            error_setg(errp,
+                       "No keyslots match given (old) password for erase operation");
+            return -1;
+        }
+
+        if (!force &&
+            slot_count == qcrypto_block_luks_count_active_slots(luks)) {
+            error_setg(errp,
+                       "All the active keyslots match the (old) password that "
+                       "was given and erasing them will erase all the data in "
+                       "the image irreversibly - refusing operation");
+            return -1;
+        }
+
+        /* Now apply the update */
+        for (i = 0; i < QCRYPTO_BLOCK_LUKS_NUM_KEY_SLOTS; i++) {
+            if (!test_bit(i, &slots_to_erase_bitmap)) {
+                continue;
+            }
+            if (qcrypto_block_luks_erase_key(block, i, writefunc,
+                opaque, errp)) {
+                error_append_hint(errp, "Failed to erase keyslot %zu", i);
+                return -1;
+            }
+        }
+    } else {
+        error_setg(errp,
+                   "To erase keyslot(s), either explicit keyslot index "
+                   "or the password currently contained in them must be given");
+        return -1;
+    }
+    return 0;
+}
+
+static int
+qcrypto_block_luks_amend_options(QCryptoBlock *block,
+                                 QCryptoBlockReadFunc readfunc,
+                                 QCryptoBlockWriteFunc writefunc,
+                                 void *opaque,
+                                 QCryptoBlockAmendOptions *options,
+                                 bool force,
+                                 Error **errp)
+{
+    QCryptoBlockAmendOptionsLUKS *opts_luks = &options->u.luks;
+
+    switch (opts_luks->state) {
+    case Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE_ACTIVE:
+        return qcrypto_block_luks_amend_add_keyslot(block, readfunc,
+                                                    writefunc, opaque,
+                                                    opts_luks, force, errp);
+    case Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE_INACTIVE:
+        return qcrypto_block_luks_amend_erase_keyslots(block, readfunc,
+                                                       writefunc, opaque,
+                                                       opts_luks, force, errp);
+    default:
+        g_assert_not_reached();
+    }
+}
 
 static int qcrypto_block_luks_get_info(QCryptoBlock *block,
                                        QCryptoBlockInfo *info,
@@ -XXX,XX +XXX,XX @@ static int qcrypto_block_luks_get_info(QCryptoBlock *block,
 
 static void qcrypto_block_luks_cleanup(QCryptoBlock *block)
 {
-    g_free(block->opaque);
+    QCryptoBlockLUKS *luks = block->opaque;
+    if (luks) {
+        g_free(luks->secret);
+        g_free(luks);
+    }
 }
 
 
@@ -XXX,XX +XXX,XX @@ qcrypto_block_luks_encrypt(QCryptoBlock *block,
 const QCryptoBlockDriver qcrypto_block_driver_luks = {
     .open = qcrypto_block_luks_open,
     .create = qcrypto_block_luks_create,
+    .amend = qcrypto_block_luks_amend_options,
     .get_info = qcrypto_block_luks_get_info,
     .cleanup = qcrypto_block_luks_cleanup,
     .decrypt = qcrypto_block_luks_decrypt,
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

'force' option will be used for some unsafe amend operations.

This includes things like erasing last keyslot in luks based formats
(which destroys the data, unless the master key is backed up
by external means), but that _might_ be desired result.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-4-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 docs/tools/qemu-img.rst   | 5 ++++-
 include/block/block.h     | 1 +
 include/block/block_int.h | 1 +
 block.c                   | 4 +++-
 block/qcow2.c             | 1 +
 qemu-img.c                | 8 +++++++-
 qemu-img-cmds.hx          | 4 ++--
 7 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index XXXXXXX..XXXXXXX 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -XXX,XX +XXX,XX @@ Command description:
 
 .. program:: qemu-img-commands
 
-.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME
+.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME
 
   Amends the image format specific *OPTIONS* for the image file
   *FILENAME*. Not all file formats support this operation.
 
+  --force allows some unsafe operations. Currently for -f luks, it allows to
+  erase the last encryption key, and to overwrite an active encryption key.
+
 .. option:: bench [-c COUNT] [-d DEPTH] [-f FMT] [--flush-interval=FLUSH_INTERVAL] [-i AIO] [-n] [--no-drain] [-o OFFSET] [--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t CACHE] [-w] [-U] FILENAME
 
   Run a simple sequential I/O benchmark on the specified image. If ``-w`` is
diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ typedef void BlockDriverAmendStatusCB(BlockDriverState *bs, int64_t offset,
                                       int64_t total_work_size, void *opaque);
 int bdrv_amend_options(BlockDriverState *bs_new, QemuOpts *opts,
                        BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
+                       bool force,
                        Error **errp);
 
 /* check if a named node can be replaced when doing drive-mirror */
diff --git a/include/block/block_int.h b/include/block/block_int.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
     int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
                               BlockDriverAmendStatusCB *status_cb,
                               void *cb_opaque,
+                              bool force,
                               Error **errp);
 
     void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_remove_aio_context_notifier(BlockDriverState *bs,
 
 int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
                        BlockDriverAmendStatusCB *status_cb, void *cb_opaque,
+                       bool force,
                        Error **errp)
 {
     if (!bs->drv) {
@@ -XXX,XX +XXX,XX @@ int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
                    bs->drv->format_name);
         return -ENOTSUP;
     }
-    return bs->drv->bdrv_amend_options(bs, opts, status_cb, cb_opaque, errp);
+    return bs->drv->bdrv_amend_options(bs, opts, status_cb,
+                                       cb_opaque, force, errp);
 }
 
 /*
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static void qcow2_amend_helper_cb(BlockDriverState *bs,
 static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
                                BlockDriverAmendStatusCB *status_cb,
                                void *cb_opaque,
+                               bool force,
                                Error **errp)
 {
     BDRVQcow2State *s = bs->opaque;
diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ enum {
     OPTION_DISABLE = 273,
     OPTION_MERGE = 274,
     OPTION_BITMAPS = 275,
+    OPTION_FORCE = 276,
 };
 
 typedef enum OutputFormat {
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
     BlockBackend *blk = NULL;
     BlockDriverState *bs = NULL;
     bool image_opts = false;
+    bool force = false;
 
     cache = BDRV_DEFAULT_CACHE;
     for (;;) {
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
             {"help", no_argument, 0, 'h'},
             {"object", required_argument, 0, OPTION_OBJECT},
             {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
+            {"force", no_argument, 0, OPTION_FORCE},
             {0, 0, 0, 0}
         };
         c = getopt_long(argc, argv, ":ho:f:t:pq",
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
         case OPTION_IMAGE_OPTS:
             image_opts = true;
             break;
+        case OPTION_FORCE:
+            force = true;
+            break;
         }
     }
 
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
 
     /* In case the driver does not call amend_status_cb() */
     qemu_progress_print(0.f, 0);
-    ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, &err);
+    ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, force, &err);
     qemu_progress_print(100.f, 0);
     if (ret < 0) {
         error_report_err(err);
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -XXX,XX +XXX,XX @@ HXCOMM When amending the rST sections, please remember to copy the usage
 HXCOMM over to the per-command sections in docs/tools/qemu-img.rst.
 
 DEF("amend", img_amend,
-    "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] -o options filename")
+    "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] [--force] -o options filename")
 SRST
-.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME
+.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME
 ERST
 
 DEF("bench", img_bench,
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

Some options are only useful for creation
(or hard to be amended, like cluster size for qcow2), while some other
options are only useful for amend, like upcoming keyslot management
options for luks

Since currently only qcow2 supports amend, move all its options
to a common macro and then include it in each action option list.

In future it might be useful to remove some options which are
not supported anyway from amend list, which currently
cause an error message if amended.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-5-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 include/block/block_int.h |   4 +
 block/qcow2.c             | 173 +++++++++++++++++++++-----------------
 qemu-img.c                |  18 ++--
 3 files changed, 107 insertions(+), 88 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
 
     /* List of options for creating images, terminated by name == NULL */
     QemuOptsList *create_opts;
+
+    /* List of options for image amend */
+    QemuOptsList *amend_opts;
+
     /*
      * If this driver supports reopening images this contains a
      * NULL-terminated list of the runtime options that can be
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
     s->signaled_corruption = true;
 }
 
+#define QCOW_COMMON_OPTIONS                                         \
+    {                                                               \
+        .name = BLOCK_OPT_SIZE,                                     \
+        .type = QEMU_OPT_SIZE,                                      \
+        .help = "Virtual disk size"                                 \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_COMPAT_LEVEL,                             \
+        .type = QEMU_OPT_STRING,                                    \
+        .help = "Compatibility level (v2 [0.10] or v3 [1.1])"       \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_BACKING_FILE,                             \
+        .type = QEMU_OPT_STRING,                                    \
+        .help = "File name of a base image"                         \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_BACKING_FMT,                              \
+        .type = QEMU_OPT_STRING,                                    \
+        .help = "Image format of the base image"                    \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_DATA_FILE,                                \
+        .type = QEMU_OPT_STRING,                                    \
+        .help = "File name of an external data file"                \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_DATA_FILE_RAW,                            \
+        .type = QEMU_OPT_BOOL,                                      \
+        .help = "The external data file must stay valid "           \
+                "as a raw image"                                    \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_ENCRYPT,                                  \
+        .type = QEMU_OPT_BOOL,                                      \
+        .help = "Encrypt the image with format 'aes'. (Deprecated " \
+                "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",    \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_ENCRYPT_FORMAT,                           \
+        .type = QEMU_OPT_STRING,                                    \
+        .help = "Encrypt the image, format choices: 'aes', 'luks'", \
+    },                                                              \
+    BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",                     \
+        "ID of secret providing qcow AES key or LUKS passphrase"),  \
+    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),               \
+    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),              \
+    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),                \
+    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),           \
+    BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),                 \
+    BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),                \
+    {                                                               \
+        .name = BLOCK_OPT_CLUSTER_SIZE,                             \
+        .type = QEMU_OPT_SIZE,                                      \
+        .help = "qcow2 cluster size",                               \
+        .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)            \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_PREALLOC,                                 \
+        .type = QEMU_OPT_STRING,                                    \
+        .help = "Preallocation mode (allowed values: off, "         \
+                "metadata, falloc, full)"                           \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_LAZY_REFCOUNTS,                           \
+        .type = QEMU_OPT_BOOL,                                      \
+        .help = "Postpone refcount updates",                        \
+        .def_value_str = "off"                                      \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_REFCOUNT_BITS,                            \
+        .type = QEMU_OPT_NUMBER,                                    \
+        .help = "Width of a reference count entry in bits",         \
+        .def_value_str = "16"                                       \
+    },                                                              \
+    {                                                               \
+        .name = BLOCK_OPT_COMPRESSION_TYPE,                         \
+        .type = QEMU_OPT_STRING,                                    \
+        .help = "Compression method used for image cluster "        \
+                "compression",                                      \
+        .def_value_str = "zlib"                                     \
+    }
+
 static QemuOptsList qcow2_create_opts = {
     .name = "qcow2-create-opts",
     .head = QTAILQ_HEAD_INITIALIZER(qcow2_create_opts.head),
     .desc = {
-        {
-            .name = BLOCK_OPT_SIZE,
-            .type = QEMU_OPT_SIZE,
-            .help = "Virtual disk size"
-        },
-        {
-            .name = BLOCK_OPT_COMPAT_LEVEL,
-            .type = QEMU_OPT_STRING,
-            .help = "Compatibility level (v2 [0.10] or v3 [1.1])"
-        },
-        {
-            .name = BLOCK_OPT_BACKING_FILE,
-            .type = QEMU_OPT_STRING,
-            .help = "File name of a base image"
-        },
-        {
-            .name = BLOCK_OPT_BACKING_FMT,
-            .type = QEMU_OPT_STRING,
-            .help = "Image format of the base image"
-        },
-        {
-            .name = BLOCK_OPT_DATA_FILE,
-            .type = QEMU_OPT_STRING,
-            .help = "File name of an external data file"
-        },
-        {
-            .name = BLOCK_OPT_DATA_FILE_RAW,
-            .type = QEMU_OPT_BOOL,
-            .help = "The external data file must stay valid as a raw image"
-        },
-        {
-            .name = BLOCK_OPT_ENCRYPT,
-            .type = QEMU_OPT_BOOL,
-            .help = "Encrypt the image with format 'aes'. (Deprecated "
-                    "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",
-        },
-        {
-            .name = BLOCK_OPT_ENCRYPT_FORMAT,
-            .type = QEMU_OPT_STRING,
-            .help = "Encrypt the image, format choices: 'aes', 'luks'",
-        },
-        BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",
-            "ID of secret providing qcow AES key or LUKS passphrase"),
-        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),
-        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),
-        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),
-        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),
-        BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),
-        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),
-        {
-            .name = BLOCK_OPT_CLUSTER_SIZE,
-            .type = QEMU_OPT_SIZE,
-            .help = "qcow2 cluster size",
-            .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)
-        },
-        {
-            .name = BLOCK_OPT_PREALLOC,
-            .type = QEMU_OPT_STRING,
-            .help = "Preallocation mode (allowed values: off, metadata, "
-                    "falloc, full)"
-        },
-        {
-            .name = BLOCK_OPT_LAZY_REFCOUNTS,
-            .type = QEMU_OPT_BOOL,
-            .help = "Postpone refcount updates",
-            .def_value_str = "off"
-        },
-        {
-            .name = BLOCK_OPT_REFCOUNT_BITS,
-            .type = QEMU_OPT_NUMBER,
-            .help = "Width of a reference count entry in bits",
-            .def_value_str = "16"
-        },
-        {
-            .name = BLOCK_OPT_COMPRESSION_TYPE,
-            .type = QEMU_OPT_STRING,
-            .help = "Compression method used for image cluster compression",
-            .def_value_str = "zlib"
-        },
+        QCOW_COMMON_OPTIONS,
+        { /* end of list */ }
+    }
+};
+
+static QemuOptsList qcow2_amend_opts = {
+    .name = "qcow2-amend-opts",
+    .head = QTAILQ_HEAD_INITIALIZER(qcow2_amend_opts.head),
+    .desc = {
+        QCOW_COMMON_OPTIONS,
         { /* end of list */ }
     }
 };
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
     .bdrv_inactivate            = qcow2_inactivate,
 
     .create_opts         = &qcow2_create_opts,
+    .amend_opts          = &qcow2_amend_opts,
     .strong_runtime_opts = qcow2_strong_runtime_opts,
     .mutable_opts        = mutable_opts,
     .bdrv_co_check       = qcow2_co_check,
diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int print_amend_option_help(const char *format)
         return 1;
     }
 
-    /* Every driver supporting amendment must have create_opts */
-    assert(drv->create_opts);
+    /* Every driver supporting amendment must have amend_opts */
+    assert(drv->amend_opts);
 
     printf("Creation options for '%s':\n", format);
-    qemu_opts_print_help(drv->create_opts, false);
+    qemu_opts_print_help(drv->amend_opts, false);
     printf("\nNote that not all of these options may be amendable.\n");
     return 0;
 }
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
     Error *err = NULL;
     int c, ret = 0;
     char *options = NULL;
-    QemuOptsList *create_opts = NULL;
+    QemuOptsList *amend_opts = NULL;
     QemuOpts *opts = NULL;
     const char *fmt = NULL, *filename, *cache;
     int flags;
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
         goto out;
     }
 
-    /* Every driver supporting amendment must have create_opts */
-    assert(bs->drv->create_opts);
+    /* Every driver supporting amendment must have amend_opts */
+    assert(bs->drv->amend_opts);
 
-    create_opts = qemu_opts_append(create_opts, bs->drv->create_opts);
-    opts = qemu_opts_create(create_opts, NULL, 0, &error_abort);
+    amend_opts = qemu_opts_append(amend_opts, bs->drv->amend_opts);
+    opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
     qemu_opts_do_parse(opts, options, NULL, &err);
     if (err) {
         error_report_err(err);
@@ -XXX,XX +XXX,XX @@ out:
 out_no_progress:
     blk_unref(blk);
     qemu_opts_del(opts);
-    qemu_opts_free(create_opts);
+    qemu_opts_free(amend_opts);
     g_free(options);
 
     if (ret) {
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

Some qcow2 create options can't be used for amend.
Remove them from the qcow2 create options and add generic logic to detect
such options in qemu-img

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-6-mlevitsk@redhat.com>
[mreitz: Dropped some iotests reference output hunks that became
         unnecessary thanks to
         "iotests: Make _filter_img_create more active"]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qcow2.c              | 138 +++++++++-----------------------
 qemu-img.c                 |  18 ++++-
 tests/qemu-iotests/049.out | 102 ++++++++++++------------
 tests/qemu-iotests/061.out |  12 ++-
 tests/qemu-iotests/082.out | 158 ++++---------------------------------
 tests/qemu-iotests/085.out |  38 ++++-----
 tests/qemu-iotests/144.out |   4 +-
 tests/qemu-iotests/182.out |   2 +-
 tests/qemu-iotests/185.out |   8 +-
 tests/qemu-iotests/255.out |   8 +-
 tests/qemu-iotests/274.out |  46 +++++------
 tests/qemu-iotests/280.out |   2 +-
 12 files changed, 183 insertions(+), 353 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static int qcow2_change_backing_file(BlockDriverState *bs,
     return qcow2_update_header(bs);
 }
 
-static int qcow2_crypt_method_from_format(const char *encryptfmt)
-{
-    if (g_str_equal(encryptfmt, "luks")) {
-        return QCOW_CRYPT_LUKS;
-    } else if (g_str_equal(encryptfmt, "aes")) {
-        return QCOW_CRYPT_AES;
-    } else {
-        return -EINVAL;
-    }
-}
-
 static int qcow2_set_up_encryption(BlockDriverState *bs,
                                    QCryptoBlockCreateOptions *cryptoopts,
                                    Error **errp)
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     bool lazy_refcounts = s->use_lazy_refcounts;
     bool data_file_raw = data_file_is_raw(bs);
     const char *compat = NULL;
-    uint64_t cluster_size = s->cluster_size;
-    bool encrypt;
-    int encformat;
     int refcount_bits = s->refcount_bits;
     int ret;
     QemuOptDesc *desc = opts->list->desc;
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
                 error_setg(errp, "Unknown compatibility level %s", compat);
                 return -EINVAL;
             }
-        } else if (!strcmp(desc->name, BLOCK_OPT_PREALLOC)) {
-            error_setg(errp, "Cannot change preallocation mode");
-            return -ENOTSUP;
         } else if (!strcmp(desc->name, BLOCK_OPT_SIZE)) {
             new_size = qemu_opt_get_size(opts, BLOCK_OPT_SIZE, 0);
         } else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FILE)) {
             backing_file = qemu_opt_get(opts, BLOCK_OPT_BACKING_FILE);
         } else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FMT)) {
             backing_format = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT);
-        } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT)) {
-            encrypt = qemu_opt_get_bool(opts, BLOCK_OPT_ENCRYPT,
-                                        !!s->crypto);
-
-            if (encrypt != !!s->crypto) {
-                error_setg(errp,
-                           "Changing the encryption flag is not supported");
-                return -ENOTSUP;
-            }
-        } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT_FORMAT)) {
-            encformat = qcow2_crypt_method_from_format(
-                qemu_opt_get(opts, BLOCK_OPT_ENCRYPT_FORMAT));
-
-            if (encformat != s->crypt_method_header) {
-                error_setg(errp,
-                           "Changing the encryption format is not supported");
-                return -ENOTSUP;
-            }
-        } else if (g_str_has_prefix(desc->name, "encrypt.")) {
-            error_setg(errp,
-                       "Changing the encryption parameters is not supported");
-            return -ENOTSUP;
-        } else if (!strcmp(desc->name, BLOCK_OPT_CLUSTER_SIZE)) {
-            cluster_size = qemu_opt_get_size(opts, BLOCK_OPT_CLUSTER_SIZE,
-                                             cluster_size);
-            if (cluster_size != s->cluster_size) {
-                error_setg(errp, "Changing the cluster size is not supported");
-                return -ENOTSUP;
-            }
         } else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) {
             lazy_refcounts = qemu_opt_get_bool(opts, BLOCK_OPT_LAZY_REFCOUNTS,
                                                lazy_refcounts);
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
                                  "images");
                 return -EINVAL;
             }
-        } else if (!strcmp(desc->name, BLOCK_OPT_COMPRESSION_TYPE)) {
-            const char *ct_name =
-                qemu_opt_get(opts, BLOCK_OPT_COMPRESSION_TYPE);
-            int compression_type =
-                qapi_enum_parse(&Qcow2CompressionType_lookup, ct_name, -1,
-                                NULL);
-            if (compression_type == -1) {
-                error_setg(errp, "Unknown compression type: %s", ct_name);
-                return -ENOTSUP;
-            }
-
-            if (compression_type != s->compression_type) {
-                error_setg(errp, "Changing the compression type "
-                                 "is not supported");
-                return -ENOTSUP;
-            }
         } else {
             /* if this point is reached, this probably means a new option was
              * added without having it covered here */
@@ -XXX,XX +XXX,XX @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
         .help = "The external data file must stay valid "           \
                 "as a raw image"                                    \
     },                                                              \
-    {                                                               \
-        .name = BLOCK_OPT_ENCRYPT,                                  \
-        .type = QEMU_OPT_BOOL,                                      \
-        .help = "Encrypt the image with format 'aes'. (Deprecated " \
-                "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",    \
-    },                                                              \
-    {                                                               \
-        .name = BLOCK_OPT_ENCRYPT_FORMAT,                           \
-        .type = QEMU_OPT_STRING,                                    \
-        .help = "Encrypt the image, format choices: 'aes', 'luks'", \
-    },                                                              \
-    BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",                     \
-        "ID of secret providing qcow AES key or LUKS passphrase"),  \
-    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),               \
-    BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),              \
-    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),                \
-    BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),           \
-    BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),                 \
-    BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),                \
-    {                                                               \
-        .name = BLOCK_OPT_CLUSTER_SIZE,                             \
-        .type = QEMU_OPT_SIZE,                                      \
-        .help = "qcow2 cluster size",                               \
-        .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)            \
-    },                                                              \
-    {                                                               \
-        .name = BLOCK_OPT_PREALLOC,                                 \
-        .type = QEMU_OPT_STRING,                                    \
-        .help = "Preallocation mode (allowed values: off, "         \
-                "metadata, falloc, full)"                           \
-    },                                                              \
     {                                                               \
         .name = BLOCK_OPT_LAZY_REFCOUNTS,                           \
         .type = QEMU_OPT_BOOL,                                      \
@@ -XXX,XX +XXX,XX @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset,
         .type = QEMU_OPT_NUMBER,                                    \
         .help = "Width of a reference count entry in bits",         \
         .def_value_str = "16"                                       \
-    },                                                              \
-    {                                                               \
-        .name = BLOCK_OPT_COMPRESSION_TYPE,                         \
-        .type = QEMU_OPT_STRING,                                    \
-        .help = "Compression method used for image cluster "        \
-                "compression",                                      \
-        .def_value_str = "zlib"                                     \
     }
 
 static QemuOptsList qcow2_create_opts = {
     .name = "qcow2-create-opts",
     .head = QTAILQ_HEAD_INITIALIZER(qcow2_create_opts.head),
     .desc = {
+        {                                                               \
+            .name = BLOCK_OPT_ENCRYPT,                                  \
+            .type = QEMU_OPT_BOOL,                                      \
+            .help = "Encrypt the image with format 'aes'. (Deprecated " \
+                    "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)",    \
+        },                                                              \
+        {                                                               \
+            .name = BLOCK_OPT_ENCRYPT_FORMAT,                           \
+            .type = QEMU_OPT_STRING,                                    \
+            .help = "Encrypt the image, format choices: 'aes', 'luks'", \
+        },                                                              \
+        BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.",                     \
+            "ID of secret providing qcow AES key or LUKS passphrase"),  \
+        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."),               \
+        BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."),              \
+        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."),                \
+        BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."),           \
+        BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."),                 \
+        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),                \
+        {                                                               \
+            .name = BLOCK_OPT_CLUSTER_SIZE,                             \
+            .type = QEMU_OPT_SIZE,                                      \
+            .help = "qcow2 cluster size",                               \
+            .def_value_str = stringify(DEFAULT_CLUSTER_SIZE)            \
+        },                                                              \
+        {                                                               \
+            .name = BLOCK_OPT_PREALLOC,                                 \
+            .type = QEMU_OPT_STRING,                                    \
+            .help = "Preallocation mode (allowed values: off, "         \
+                    "metadata, falloc, full)"                           \
+        },                                                              \
+        {                                                               \
+            .name = BLOCK_OPT_COMPRESSION_TYPE,                         \
+            .type = QEMU_OPT_STRING,                                    \
+            .help = "Compression method used for image cluster "        \
+                    "compression",                                      \
+            .def_value_str = "zlib"                                     \
+        },
         QCOW_COMMON_OPTIONS,
         { /* end of list */ }
     }
diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int print_amend_option_help(const char *format)
     /* Every driver supporting amendment must have amend_opts */
     assert(drv->amend_opts);
 
-    printf("Creation options for '%s':\n", format);
+    printf("Amend options for '%s':\n", format);
     qemu_opts_print_help(drv->amend_opts, false);
-    printf("\nNote that not all of these options may be amendable.\n");
     return 0;
 }
 
@@ -XXX,XX +XXX,XX @@ static int img_amend(int argc, char **argv)
     amend_opts = qemu_opts_append(amend_opts, bs->drv->amend_opts);
     opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
     qemu_opts_do_parse(opts, options, NULL, &err);
+
     if (err) {
+        /* Try to parse options using the create options */
+        Error *err1 = NULL;
+        amend_opts = qemu_opts_append(amend_opts, bs->drv->create_opts);
+        qemu_opts_del(opts);
+        opts = qemu_opts_create(amend_opts, NULL, 0, &error_abort);
+        qemu_opts_do_parse(opts, options, NULL, &err1);
+
+        if (!err1) {
+            error_append_hint(&err,
+                              "This option is only supported for image creation\n");
+        } else {
+            error_free(err1);
+        }
+
         error_report_err(err);
         ret = -1;
         goto out;
diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/049.out
+++ b/tests/qemu-iotests/049.out
@@ -XXX,XX +XXX,XX @@ QA output created by 049
 == 1. Traditional size parameter ==
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1k
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1K
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1G
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1T
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0b
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5k
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5K
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5G
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5T
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
 
 == 2. Specifying size via -o ==
 
 qemu-img create -f qcow2 -o size=1024 TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1024b TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1k TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1K TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1M TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1G TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1T TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1024.0 TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1024.0b TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1024 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1.5k TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1.5K TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1536 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1.5M TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1572864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1.5G TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1610612736 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o size=1.5T TEST_DIR/t.qcow2
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1649267441664 lazy_refcounts=off refcount_bits=16
 
 == 3. Invalid sizes ==
 
@@ -XXX,XX +XXX,XX @@ qemu-img: TEST_DIR/t.qcow2: The image size must be specified only once
 == Check correct interpretation of suffixes for cluster size ==
 
 qemu-img create -f qcow2 -o cluster_size=1024 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=1024b TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=1k TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=1K TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=1M TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1048576 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1048576 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=1024.0 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=1024.0b TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=1024 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=0.5k TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=0.5K TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=512 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o cluster_size=0.5M TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=524288 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=524288 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 == Check compat level option ==
 
 qemu-img create -f qcow2 -o compat=0.10 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o compat=1.1 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o compat=0.42 TEST_DIR/t.qcow2 64M
 qemu-img: TEST_DIR/t.qcow2: Invalid parameter '0.42'
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.42 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.42 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o compat=foobar TEST_DIR/t.qcow2 64M
 qemu-img: TEST_DIR/t.qcow2: Invalid parameter 'foobar'
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=foobar cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=foobar lazy_refcounts=off refcount_bits=16
 
 == Check preallocation option ==
 
 qemu-img create -f qcow2 -o preallocation=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o preallocation=metadata TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=metadata compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o preallocation=1234 TEST_DIR/t.qcow2 64M
 qemu-img: TEST_DIR/t.qcow2: Invalid parameter '1234'
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=1234 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 preallocation=1234 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 == Check encryption option ==
 
 qemu-img create -f qcow2 -o encryption=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=off cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 encryption=on encrypt.key-secret=sec0 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 == Check lazy_refcounts option (only with v3) ==
 
 qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=1.1 lazy_refcounts=on refcount_bits=16
 
 qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=off TEST_DIR/t.qcow2 64M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=off refcount_bits=16
 
 qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=on TEST_DIR/t.qcow2 64M
 qemu-img: TEST_DIR/t.qcow2: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater)
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 compat=0.10 lazy_refcounts=on refcount_bits=16
 
 *** done
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/061.out
+++ b/tests/qemu-iotests/061.out
@@ -XXX,XX +XXX,XX @@ qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (
 qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater)
 qemu-img: Unknown compatibility level 0.42
 qemu-img: Invalid parameter 'foo'
-qemu-img: Changing the cluster size is not supported
-qemu-img: Changing the encryption flag is not supported
-qemu-img: Cannot change preallocation mode
+qemu-img: Invalid parameter 'cluster_size'
+This option is only supported for image creation
+qemu-img: Invalid parameter 'encryption'
+This option is only supported for image creation
+qemu-img: Invalid parameter 'preallocation'
+This option is only supported for image creation
 
 === Testing correct handling of unset value ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 Should work:
 Should not work:
-qemu-img: Changing the cluster size is not supported
+qemu-img: Invalid parameter 'cluster_size'
+This option is only supported for image creation
 
 === Testing zero expansion on inactive clusters ===
 
diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/082.out
+++ b/tests/qemu-iotests/082.out
@@ -XXX,XX +XXX,XX @@ QA output created by 082
 === create: Options specified more than once ===
 
 Testing: create -f foo -f qcow2 TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 128 MiB (134217728 bytes)
 cluster_size: 65536
 
 Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=4096 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=4096 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 128 MiB (134217728 bytes)
@@ -XXX,XX +XXX,XX @@ Format specific information:
     corrupt: false
 
 Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=on refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=on refcount_bits=16
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 128 MiB (134217728 bytes)
@@ -XXX,XX +XXX,XX @@ Format specific information:
     corrupt: false
 
 Testing: create -f qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=8192 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 128 MiB (134217728 bytes)
@@ -XXX,XX +XXX,XX @@ Supported options:
   size=<size>            - Virtual disk size
 
 Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,help cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,help lazy_refcounts=off refcount_bits=16
 
 Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,? cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2,,? lazy_refcounts=off refcount_bits=16
 
 Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2, -o help TEST_DIR/t.qcow2 128M
 qemu-img: Invalid option list: backing_file=TEST_DIR/t.qcow2,
@@ -XXX,XX +XXX,XX @@ qemu-img: Format driver 'bochs' does not support image creation
 === convert: Options specified more than once ===
 
 Testing: create -f qcow2 TEST_DIR/t.qcow2 128M
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
 Testing: convert -f foo -f qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base
 image: TEST_DIR/t.IMGFMT.base
@@ -XXX,XX +XXX,XX @@ cluster_size: 65536
 === amend: help for -o ===
 
 Testing: amend -f qcow2 -o help TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o ? TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o cluster_size=4k,help TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o cluster_size=4k,? TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o help,cluster_size=4k TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o ?,cluster_size=4k TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o cluster_size=4k -o help TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o cluster_size=4k -o ? TEST_DIR/t.qcow2
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2
 
 Testing: rebase -u -b  -f qcow2 TEST_DIR/t.qcow2
@@ -XXX,XX +XXX,XX @@ Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2 -o ,, -o help TEST_DIR/
 qemu-img: Invalid option list: ,,
 
 Testing: amend -f qcow2 -o help
-Creation options for 'qcow2':
+Amend options for 'qcow2':
   backing_file=<str>     - File name of a base image
   backing_fmt=<str>      - Image format of the base image
-  cluster_size=<size>    - qcow2 cluster size
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
-  compression_type=<str> - Compression method used for image cluster compression
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
-  encrypt.cipher-alg=<str> - Name of encryption cipher algorithm
-  encrypt.cipher-mode=<str> - Name of encryption cipher mode
-  encrypt.format=<str>   - Encrypt the image, format choices: 'aes', 'luks'
-  encrypt.hash-alg=<str> - Name of encryption hash algorithm
-  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
-  encrypt.ivgen-alg=<str> - Name of IV generator algorithm
-  encrypt.ivgen-hash-alg=<str> - Name of IV generator hash algorithm
-  encrypt.key-secret=<str> - ID of secret providing qcow AES key or LUKS passphrase
-  encryption=<bool (on/off)> - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
-  preallocation=<str>    - Preallocation mode (allowed values: off, metadata, falloc, full)
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
 
-Note that not all of these options may be amendable.
-
 Testing: amend -o help
 qemu-img: Expecting one image file name
 
diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/085.out
+++ b/tests/qemu-iotests/085.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728
 === Create a single snapshot on virtio0 ===
 
 { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/1-snapshot-v0.IMGFMT', 'format': 'IMGFMT' } }
-Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 
 === Invalid command - missing device and nodename ===
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file
 === Create several transactional group snapshots ===
 
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/2-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/2-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/3-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/3-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/4-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/4-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/5-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/5-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/6-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/6-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/7-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/7-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/8-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/8-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/9-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/9-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/10-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/10-snapshot-v1.IMGFMT' } } ] } }
-Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
-Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 
 === Create a couple of snapshots using blockdev-snapshot ===
diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/144.out
+++ b/tests/qemu-iotests/144.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912
 { 'execute': 'qmp_capabilities' }
 {"return": {}}
 { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp.IMGFMT', 'format': 'IMGFMT' } }
-Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 
 === Performing block-commit on active layer ===
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/
 === Performing Live Snapshot 2 ===
 
 { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp2.IMGFMT', 'format': 'IMGFMT' } }
-Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 *** done
diff --git a/tests/qemu-iotests/182.out b/tests/qemu-iotests/182.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/182.out
+++ b/tests/qemu-iotests/182.out
@@ -XXX,XX +XXX,XX @@ Is another process using the image [TEST_DIR/t.qcow2]?
 {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node0', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
 {"return": {}}
 {'execute': 'blockdev-snapshot-sync', 'arguments': { 'node-name': 'node0', 'snapshot-file': 'TEST_DIR/t.IMGFMT.overlay', 'snapshot-node-name': 'node1' } }
-Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16
 {"return": {}}
 {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node1', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } }
 {"return": {}}
diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/185.out
+++ b/tests/qemu-iotests/185.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
 === Creating backing chain ===
 
 { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT.mid', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
-Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 { 'execute': 'human-monitor-command', 'arguments': { 'command-line': 'qemu-io disk "write 0 4M"' } }
 wrote 4194304/4194304 bytes at offset 0
 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 {"return": ""}
 { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'absolute-paths' } }
-Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 {"return": {}}
 
 === Start commit job and exit qemu ===
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q
 { 'execute': 'qmp_capabilities' }
 {"return": {}}
 { 'execute': 'drive-mirror', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
-Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
 {"return": {}}
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l
 { 'execute': 'qmp_capabilities' }
 {"return": {}}
 { 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
-Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}}
diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/255.out
+++ b/tests/qemu-iotests/255.out
@@ -XXX,XX +XXX,XX @@ Finishing a commit job with background reads
 
 === Create backing chain and start VM ===
 
-Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
 === Start background read requests ===
 
@@ -XXX,XX +XXX,XX @@ Closing the VM while a job is being cancelled
 
 === Create images and start VM ===
 
-Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
 wrote 1048576/1048576 bytes at offset 0
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/274.out
+++ b/tests/qemu-iotests/274.out
@@ -XXX,XX +XXX,XX @@
 == Commit tests ==
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing HMP commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing QMP active commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 
 == Resize tests ==
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=6442450944 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=1073741824 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 5368709120
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 5368709120
 { "start": 1073741824, "length": 7516192768, "depth": 0, "zero": true, "data": false}]
 
 === preallocation=metadata ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=32212254720 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 33285996544
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 33285996544
 { "start": 34896609280, "length": 536870912, "depth": 0, "zero": true, "data": false, "offset": 2685075456}]
 
 === preallocation=falloc ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=5242880 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 9437184
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 9437184
 { "start": 5242880, "length": 10485760, "depth": 0, "zero": false, "data": true, "offset": 327680}]
 
 === preallocation=full ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=16777216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=8388608 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 11534336
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 11534336
 { "start": 8388608, "length": 4194304, "depth": 0, "zero": false, "data": true, "offset": 327680}]
 
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=393216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=259072 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 259072
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 259072
 { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
 
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=409600 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 344064
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 344064
 { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}]
 
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=524288 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 446464
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/280.out
+++ b/tests/qemu-iotests/280.out
@@ -XXX,XX +XXX,XX @@
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 === Launch VM ===
 Enabling migration QMP events on VM...
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

rename the write_func to create_write_func, and init_func to create_init_func.
This is preparation for other write_func that will be used to update the encryption keys.

No functional changes

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-7-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/crypto.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/block/crypto.c b/block/crypto.c
index XXXXXXX..XXXXXXX 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ struct BlockCryptoCreateData {
 };
 
 
-static ssize_t block_crypto_write_func(QCryptoBlock *block,
-                                       size_t offset,
-                                       const uint8_t *buf,
-                                       size_t buflen,
-                                       void *opaque,
-                                       Error **errp)
+static ssize_t block_crypto_create_write_func(QCryptoBlock *block,
+                                              size_t offset,
+                                              const uint8_t *buf,
+                                              size_t buflen,
+                                              void *opaque,
+                                              Error **errp)
 {
     struct BlockCryptoCreateData *data = opaque;
     ssize_t ret;
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_write_func(QCryptoBlock *block,
     return ret;
 }
 
-
-static ssize_t block_crypto_init_func(QCryptoBlock *block,
-                                      size_t headerlen,
-                                      void *opaque,
-                                      Error **errp)
+static ssize_t block_crypto_create_init_func(QCryptoBlock *block,
+                                             size_t headerlen,
+                                             void *opaque,
+                                             Error **errp)
 {
     struct BlockCryptoCreateData *data = opaque;
     Error *local_error = NULL;
@@ -XXX,XX +XXX,XX @@ static int block_crypto_co_create_generic(BlockDriverState *bs,
     };
 
     crypto = qcrypto_block_create(opts, NULL,
-                                  block_crypto_init_func,
-                                  block_crypto_write_func,
+                                  block_crypto_create_init_func,
+                                  block_crypto_create_write_func,
                                   &data,
                                   errp);
 
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

This implements the encryption key management using the generic code in
qcrypto layer and exposes it to the user via qemu-img

This code adds another 'write_func' because the initialization
write_func works directly on the underlying file, and amend
works on instance of luks device.

This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks)
made to make the driver both support write sharing (to avoid breaking the users),
and be safe against concurrent  metadata update (the keyslots)

Eventually the write sharing for luks driver will be deprecated
and removed together with this hack.

The hack is that we ask (as a format driver) for BLK_PERM_CONSISTENT_READ
and then when we want to update the keys, we unshare that permission.
So if someone else has the image open, even readonly, encryption
key update will fail gracefully.

Also thanks to Daniel Berrange for the idea of
unsharing read, rather that write permission which allows
to avoid cases when the other user had opened the image read-only.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-8-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/crypto.h |  34 +++++++++++++
 block/crypto.c | 130 +++++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 161 insertions(+), 3 deletions(-)

diff --git a/block/crypto.h b/block/crypto.h
index XXXXXXX..XXXXXXX 100644
--- a/block/crypto.h
+++ b/block/crypto.h
@@ -XXX,XX +XXX,XX @@
 #define BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG "ivgen-hash-alg"
 #define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg"
 #define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time"
+#define BLOCK_CRYPTO_OPT_LUKS_KEYSLOT "keyslot"
+#define BLOCK_CRYPTO_OPT_LUKS_STATE "state"
+#define BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET "old-secret"
+#define BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET "new-secret"
+
 
 #define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET(prefix)                    \
     BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix,                             \
@@ -XXX,XX +XXX,XX @@
         .help = "Time to spend in PBKDF in milliseconds",     \
     }
 
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_STATE(prefix)                           \
+    {                                                                     \
+        .name = prefix BLOCK_CRYPTO_OPT_LUKS_STATE,                       \
+        .type = QEMU_OPT_STRING,                                          \
+        .help = "Select new state of affected keyslots (active/inactive)",\
+    }
+
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT(prefix)              \
+    {                                                          \
+        .name = prefix BLOCK_CRYPTO_OPT_LUKS_KEYSLOT,          \
+        .type = QEMU_OPT_NUMBER,                               \
+        .help = "Select a single keyslot to modify explicitly",\
+    }
+
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET(prefix)            \
+    {                                                           \
+        .name = prefix BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET,        \
+        .type = QEMU_OPT_STRING,                                \
+        .help = "Select all keyslots that match this password", \
+    }
+
+#define BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET(prefix)            \
+    {                                                           \
+        .name = prefix BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET,        \
+        .type = QEMU_OPT_STRING,                                \
+        .help = "New secret to set in the matching keyslots. "  \
+                "Empty string to erase",                        \
+    }
+
 QCryptoBlockCreateOptions *
 block_crypto_create_opts_init(QDict *opts, Error **errp);
 
diff --git a/block/crypto.c b/block/crypto.c
index XXXXXXX..XXXXXXX 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCrypto BlockCrypto;
 
 struct BlockCrypto {
     QCryptoBlock *block;
+    bool updating_keys;
 };
 
 
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
     return ret;
 }
 
+static ssize_t block_crypto_write_func(QCryptoBlock *block,
+                                       size_t offset,
+                                       const uint8_t *buf,
+                                       size_t buflen,
+                                       void *opaque,
+                                       Error **errp)
+{
+    BlockDriverState *bs = opaque;
+    ssize_t ret;
+
+    ret = bdrv_pwrite(bs->file, offset, buf, buflen);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Could not write encryption header");
+        return ret;
+    }
+    return ret;
+}
+
 
 struct BlockCryptoCreateData {
     BlockBackend *blk;
@@ -XXX,XX +XXX,XX @@ static QemuOptsList block_crypto_create_opts_luks = {
 };
 
 
+static QemuOptsList block_crypto_amend_opts_luks = {
+    .name = "crypto",
+    .head = QTAILQ_HEAD_INITIALIZER(block_crypto_create_opts_luks.head),
+    .desc = {
+        BLOCK_CRYPTO_OPT_DEF_LUKS_STATE(""),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT(""),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET(""),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET(""),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME(""),
+        { /* end of list */ }
+    },
+};
+
 QCryptoBlockOpenOptions *
 block_crypto_open_opts_init(QDict *opts, Error **errp)
 {
@@ -XXX,XX +XXX,XX @@ block_crypto_get_specific_info_luks(BlockDriverState *bs, Error **errp)
     return spec_info;
 }
 
+static int
+block_crypto_amend_options_luks(BlockDriverState *bs,
+                                QemuOpts *opts,
+                                BlockDriverAmendStatusCB *status_cb,
+                                void *cb_opaque,
+                                bool force,
+                                Error **errp)
+{
+    BlockCrypto *crypto = bs->opaque;
+    QDict *cryptoopts = NULL;
+    QCryptoBlockAmendOptions *amend_options = NULL;
+    int ret;
+
+    assert(crypto);
+    assert(crypto->block);
+    crypto->updating_keys = true;
+
+    ret = bdrv_child_refresh_perms(bs, bs->file, errp);
+    if (ret < 0) {
+        goto cleanup;
+    }
+
+    cryptoopts = qemu_opts_to_qdict(opts, NULL);
+    qdict_put_str(cryptoopts, "format", "luks");
+    amend_options = block_crypto_amend_opts_init(cryptoopts, errp);
+    if (!amend_options) {
+        ret = -EINVAL;
+        goto cleanup;
+    }
+
+    ret = qcrypto_block_amend_options(crypto->block,
+                                      block_crypto_read_func,
+                                      block_crypto_write_func,
+                                      bs,
+                                      amend_options,
+                                      force,
+                                      errp);
+cleanup:
+    crypto->updating_keys = false;
+    bdrv_child_refresh_perms(bs, bs->file, errp);
+    qapi_free_QCryptoBlockAmendOptions(amend_options);
+    qobject_unref(cryptoopts);
+    return ret;
+}
+
+
+static void
+block_crypto_child_perms(BlockDriverState *bs, BdrvChild *c,
+                         const BdrvChildRole role,
+                         BlockReopenQueue *reopen_queue,
+                         uint64_t perm, uint64_t shared,
+                         uint64_t *nperm, uint64_t *nshared)
+{
+
+    BlockCrypto *crypto = bs->opaque;
+
+    bdrv_default_perms(bs, c, role, reopen_queue, perm, shared, nperm, nshared);
+
+    /*
+     * For backward compatibility, manually share the write
+     * and resize permission
+     */
+    *nshared |= (BLK_PERM_WRITE | BLK_PERM_RESIZE);
+    /*
+     * Since we are not fully a format driver, don't always request
+     * the read/resize permission but only when explicitly
+     * requested
+     */
+    *nperm &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
+    *nperm |= perm & (BLK_PERM_WRITE | BLK_PERM_RESIZE);
+
+    /*
+     * This driver doesn't modify LUKS metadata except
+     * when updating the encryption slots.
+     * Thus unlike a proper format driver we don't ask for
+     * shared write/read permission. However we need it
+     * when we are updating the keys, to ensure that only we
+     * have access to the device.
+     *
+     * Encryption update will set the crypto->updating_keys
+     * during that period and refresh permissions
+     *
+     */
+    if (crypto->updating_keys) {
+        /* need exclusive write access for header update */
+        *nperm |= BLK_PERM_WRITE;
+        /* unshare read and write permission */
+        *nshared &= ~(BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE);
+    }
+}
+
+
 static const char *const block_crypto_strong_runtime_opts[] = {
     BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET,
 
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_crypto_luks = {
     .bdrv_probe         = block_crypto_probe_luks,
     .bdrv_open          = block_crypto_open_luks,
     .bdrv_close         = block_crypto_close,
-    /* This driver doesn't modify LUKS metadata except when creating image.
-     * Allow share-rw=on as a special case. */
-    .bdrv_child_perm    = bdrv_default_perms,
+    .bdrv_child_perm    = block_crypto_child_perms,
     .bdrv_co_create     = block_crypto_co_create_luks,
     .bdrv_co_create_opts = block_crypto_co_create_opts_luks,
     .bdrv_co_truncate   = block_crypto_co_truncate,
     .create_opts        = &block_crypto_create_opts_luks,
+    .amend_opts         = &block_crypto_amend_opts_luks,
 
     .bdrv_reopen_prepare = block_crypto_reopen_prepare,
     .bdrv_refresh_limits = block_crypto_refresh_limits,
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_crypto_luks = {
     .bdrv_measure       = block_crypto_measure,
     .bdrv_get_info      = block_crypto_get_info_luks,
     .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
+    .bdrv_amend_options = block_crypto_amend_options_luks,
 
     .is_format          = true,
 
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

Now that we have all the infrastructure in place,
wire it in the qcow2 driver and expose this to the user.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-9-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qcow2.c              | 71 +++++++++++++++++++++++++++++++++-----
 tests/qemu-iotests/082.out | 45 ++++++++++++++++++++++++
 2 files changed, 107 insertions(+), 9 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static ssize_t qcow2_crypto_hdr_write_func(QCryptoBlock *block, size_t offset,
     return ret;
 }
 
+static QDict*
+qcow2_extract_crypto_opts(QemuOpts *opts, const char *fmt, Error **errp)
+{
+    QDict *cryptoopts_qdict;
+    QDict *opts_qdict;
+
+    /* Extract "encrypt." options into a qdict */
+    opts_qdict = qemu_opts_to_qdict(opts, NULL);
+    qdict_extract_subqdict(opts_qdict, &cryptoopts_qdict, "encrypt.");
+    qobject_unref(opts_qdict);
+    qdict_put_str(cryptoopts_qdict, "format", fmt);
+    return cryptoopts_qdict;
+}
 
 /*
  * read qcow2 extension and fill bs
@@ -XXX,XX +XXX,XX @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,
 
     if (has_luks) {
         g_autoptr(QCryptoBlockCreateOptions) create_opts = NULL;
-        QDict *opts_qdict;
-        QDict *cryptoopts;
+        QDict *cryptoopts = qcow2_extract_crypto_opts(opts, "luks", errp);
         size_t headerlen;
 
-        opts_qdict = qemu_opts_to_qdict(opts, NULL);
-        qdict_extract_subqdict(opts_qdict, &cryptoopts, "encrypt.");
-        qobject_unref(opts_qdict);
-
-        qdict_put_str(cryptoopts, "format", "luks");
-
         create_opts = block_crypto_create_opts_init(cryptoopts, errp);
         qobject_unref(cryptoopts);
         if (!create_opts) {
@@ -XXX,XX +XXX,XX @@ typedef enum Qcow2AmendOperation {
     QCOW2_NO_OPERATION = 0,
 
     QCOW2_UPGRADING,
+    QCOW2_UPDATING_ENCRYPTION,
     QCOW2_CHANGING_REFCOUNT_ORDER,
     QCOW2_DOWNGRADING,
 } Qcow2AmendOperation;
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     int ret;
     QemuOptDesc *desc = opts->list->desc;
     Qcow2AmendHelperCBInfo helper_cb_info;
+    bool encryption_update = false;
 
     while (desc && desc->name) {
         if (!qemu_opt_find(opts, desc->name)) {
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
             backing_file = qemu_opt_get(opts, BLOCK_OPT_BACKING_FILE);
         } else if (!strcmp(desc->name, BLOCK_OPT_BACKING_FMT)) {
             backing_format = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT);
+        } else if (g_str_has_prefix(desc->name, "encrypt.")) {
+            if (!s->crypto) {
+                error_setg(errp,
+                           "Can't amend encryption options - encryption not present");
+                return -EINVAL;
+            }
+            if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
+                error_setg(errp,
+                           "Only LUKS encryption options can be amended");
+                return -ENOTSUP;
+            }
+            encryption_update = true;
         } else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) {
             lazy_refcounts = qemu_opt_get_bool(opts, BLOCK_OPT_LAZY_REFCOUNTS,
                                                lazy_refcounts);
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
         .original_status_cb = status_cb,
         .original_cb_opaque = cb_opaque,
         .total_operations = (new_version != old_version)
-                          + (s->refcount_bits != refcount_bits)
+                          + (s->refcount_bits != refcount_bits) +
+                            (encryption_update == true)
     };
 
     /* Upgrade first (some features may require compat=1.1) */
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
         }
     }
 
+    if (encryption_update) {
+        QDict *amend_opts_dict;
+        QCryptoBlockAmendOptions *amend_opts;
+
+        helper_cb_info.current_operation = QCOW2_UPDATING_ENCRYPTION;
+        amend_opts_dict = qcow2_extract_crypto_opts(opts, "luks", errp);
+        if (!amend_opts_dict) {
+            return -EINVAL;
+        }
+        amend_opts = block_crypto_amend_opts_init(amend_opts_dict, errp);
+        qobject_unref(amend_opts_dict);
+        if (!amend_opts) {
+            return -EINVAL;
+        }
+        ret = qcrypto_block_amend_options(s->crypto,
+                                          qcow2_crypto_hdr_read_func,
+                                          qcow2_crypto_hdr_write_func,
+                                          bs,
+                                          amend_opts,
+                                          force,
+                                          errp);
+        qapi_free_QCryptoBlockAmendOptions(amend_opts);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
     if (s->refcount_bits != refcount_bits) {
         int refcount_order = ctz32(refcount_bits);
 
@@ -XXX,XX +XXX,XX @@ static QemuOptsList qcow2_amend_opts = {
     .name = "qcow2-amend-opts",
     .head = QTAILQ_HEAD_INITIALIZER(qcow2_amend_opts.head),
     .desc = {
+        BLOCK_CRYPTO_OPT_DEF_LUKS_STATE("encrypt."),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT("encrypt."),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET("encrypt."),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET("encrypt."),
+        BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."),
         QCOW_COMMON_OPTIONS,
         { /* end of list */ }
     }
diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/082.out
+++ b/tests/qemu-iotests/082.out
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
@@ -XXX,XX +XXX,XX @@ Amend options for 'qcow2':
   compat=<str>           - Compatibility level (v2 [0.10] or v3 [1.1])
   data_file=<str>        - File name of an external data file
   data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image
+  encrypt.iter-time=<num> - Time to spend in PBKDF in milliseconds
+  encrypt.keyslot=<num>  - Select a single keyslot to modify explicitly
+  encrypt.new-secret=<str> - New secret to set in the matching keyslots. Empty string to erase
+  encrypt.old-secret=<str> - Select all keyslots that match this password
+  encrypt.state=<str>    - Select new state of affected keyslots (active/inactive)
   lazy_refcounts=<bool (on/off)> - Postpone refcount updates
   refcount_bits=<num>    - Width of a reference count entry in bits
   size=<size>            - Virtual disk size
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

This commit adds two tests, which test the new amend interface
of both luks raw images and qcow2 luks encrypted images.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-11-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/293     | 207 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/293.out |  99 ++++++++++++++++++
 tests/qemu-iotests/294     |  90 ++++++++++++++++
 tests/qemu-iotests/294.out |  30 ++++++
 tests/qemu-iotests/group   |   2 +
 5 files changed, 428 insertions(+)
 create mode 100755 tests/qemu-iotests/293
 create mode 100644 tests/qemu-iotests/293.out
 create mode 100755 tests/qemu-iotests/294
 create mode 100644 tests/qemu-iotests/294.out

diff --git a/tests/qemu-iotests/293 b/tests/qemu-iotests/293
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/293
@@ -XXX,XX +XXX,XX @@
+#!/usr/bin/env bash
+#
+# Test encryption key management with luks
+# Based on 134
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mlevitsk@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2 luks
+_supported_proto file #TODO
+
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+if [ "$IMGFMT" = "qcow2" ] ; then
+	PR="encrypt."
+	EXTRA_IMG_ARGS="-o encrypt.format=luks"
+fi
+
+
+# secrets: you are supposed to see the password as *******, see :-)
+S0="--object secret,id=sec0,data=hunter0"
+S1="--object secret,id=sec1,data=hunter1"
+S2="--object secret,id=sec2,data=hunter2"
+S3="--object secret,id=sec3,data=hunter3"
+S4="--object secret,id=sec4,data=hunter4"
+SECRETS="$S0 $S1 $S2 $S3 $S4"
+
+# image with given secret
+IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec0"
+IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec1"
+IMGS2="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec2"
+IMGS3="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec3"
+IMGS4="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec4"
+
+
+echo "== creating a test image =="
+_make_test_img $S0 $EXTRA_IMG_ARGS -o ${PR}key-secret=sec0,${PR}iter-time=10 32M
+
+echo
+echo "== test that key 0 opens the image =="
+$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== adding a password to slot 4 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec4,${PR}iter-time=10,${PR}keyslot=4
+echo "== adding a password to slot 1 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10
+echo "== adding a password to slot 3 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10,${PR}keyslot=3
+
+echo "== adding a password to slot 2 =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10
+
+
+echo "== erase slot 4 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=4 | _filter_img_create
+
+
+echo
+echo "== all secrets should work =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== erase slot 0 and try it =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 | _filter_img_create
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== erase slot 2 and try it =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=2 | _filter_img_create
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS2 | _filter_qemu_io | _filter_testdir
+
+
+# at this point slots 1 and 3 should be active
+
+echo
+echo "== filling  4 slots with secret 2 =="
+for i in $(seq 0 3) ; do
+	$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10
+done
+
+echo
+echo "== adding secret 0 =="
+	$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10
+
+echo
+echo "== adding secret 3 (last slot) =="
+	$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10
+
+echo
+echo "== trying to add another slot (should fail) =="
+$QEMU_IMG amend $SECRETS $IMGS2 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10
+
+echo
+echo "== all secrets should work again =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+
+echo
+
+echo "== erase all keys of secret 2=="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec2
+
+echo "== erase all keys of secret 1=="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1
+
+echo "== erase all keys of secret 0=="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec0
+
+echo "== erasing secret3 will fail now since it is the only secret (in 3 slots) =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=inactive,${PR}old-secret=sec3
+
+echo
+echo "== only secret3 should work now  =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== add secret0  =="
+$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10
+
+echo "== erase secret3 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec3
+
+echo
+echo "== only secret0 should work now  =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+echo
+echo "== replace secret0 with secret1 (should fail)  =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}keyslot=0
+
+echo
+echo "== replace secret0 with secret1 with force (should work)  =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10,${PR}keyslot=0 --force
+
+echo
+echo "== only secret1 should work now  =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+
+echo
+echo "== erase last secret (should fail)  =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1
+
+
+echo "== erase non existing secrets (should fail)  =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec5 --force
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 --force
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=1 --force
+
+echo
+echo "== erase last secret with force by slot (should work)  =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0 --force
+
+echo
+echo "== we have no secrets now, data is lost forever =="
+for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do
+	$QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir
+done
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
+
diff --git a/tests/qemu-iotests/293.out b/tests/qemu-iotests/293.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/293.out
@@ -XXX,XX +XXX,XX @@
+QA output created by 293
+== creating a test image ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
+
+== test that key 0 opens the image ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== adding a password to slot 4 ==
+== adding a password to slot 1 ==
+== adding a password to slot 3 ==
+== adding a password to slot 2 ==
+== erase slot 4 ==
+
+== all secrets should work ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== erase slot 0 and try it ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== erase slot 2 and try it ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== filling  4 slots with secret 2 ==
+
+== adding secret 0 ==
+
+== adding secret 3 (last slot) ==
+
+== trying to add another slot (should fail) ==
+qemu-img: Can't add a keyslot - all keyslots are in use
+
+== all secrets should work again ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== erase all keys of secret 2==
+== erase all keys of secret 1==
+== erase all keys of secret 0==
+== erasing secret3 will fail now since it is the only secret (in 3 slots) ==
+qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
+
+== only secret3 should work now  ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== add secret0  ==
+== erase secret3 ==
+
+== only secret0 should work now  ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== replace secret0 with secret1 (should fail)  ==
+qemu-img: Refusing to overwrite active keyslot 0 - please erase it first
+
+== replace secret0 with secret1 with force (should work)  ==
+
+== only secret1 should work now  ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== erase last secret (should fail)  ==
+qemu-img: Attempt to erase the only active keyslot 0 which will erase all the data in the image irreversibly - refusing operation
+qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
+== erase non existing secrets (should fail)  ==
+qemu-img: No secret with id 'sec5'
+qemu-img: No keyslots match given (old) password for erase operation
+
+== erase last secret with force by slot (should work)  ==
+
+== we have no secrets now, data is lost forever ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+*** done
diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/294
@@ -XXX,XX +XXX,XX @@
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mlevitsk@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt luks
+_supported_proto file #TODO
+
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+# you are supposed to see the password as *******, see :-)
+S0="--object secret,id=sec0,data=hunter0"
+S1="--object secret,id=sec1,data=hunter1"
+SECRETS="$S0 $S1"
+
+
+IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec0"
+IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec1"
+
+echo "== creating a test image =="
+_make_test_img $S0 -o "key-secret=sec0,iter-time=10" 32M
+
+echo
+echo "== test that key 0 opens the image =="
+$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== adding a password to slot 1 =="
+$QEMU_IMG amend $SECRETS $IMGS0 -o state=active,new-secret=sec1,keyslot=1,iter-time=10
+
+echo
+echo "== 'backup' the image header =="
+dd if=$TEST_IMG_FILE of=${TEST_IMG_FILE}.bk bs=4K skip=0 count=1
+
+echo
+echo "== erase slot 0 =="
+$QEMU_IMG amend $SECRETS $IMGS1 -o state=inactive,keyslot=0 | _filter_img_create
+
+echo
+echo "== test that key 0 doesn't open the image =="
+$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== 'restore' the image header =="
+dd if=${TEST_IMG_FILE}.bk of=${TEST_IMG_FILE} bs=4K skip=0 count=1 conv=notrunc
+
+echo
+echo "== test that key 0 still doesn't open the image (key material is erased) =="
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== test that key 1 still works =="
+$QEMU_IO $SECRETS -c "read 0 4096" $IMGS1 | _filter_qemu_io | _filter_testdir
+
+echo "*** done"
+rm -f $seq.full
+status=0
+
+
+exit 0
diff --git a/tests/qemu-iotests/294.out b/tests/qemu-iotests/294.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/294.out
@@ -XXX,XX +XXX,XX @@
+QA output created by 294
+== creating a test image ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
+
+== test that key 0 opens the image ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== adding a password to slot 1 ==
+
+== 'backup' the image header ==
+1+0 records in
+1+0 records out
+
+== erase slot 0 ==
+
+== test that key 0 doesn't open the image ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== 'restore' the image header ==
+1+0 records in
+1+0 records out
+
+== test that key 0 still doesn't open the image (key material is erased) ==
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== test that key 1 still works ==
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -XXX,XX +XXX,XX @@
 290 rw auto quick
 291 rw quick
 292 rw auto quick
+293 rw auto
+294 rw auto quick
 297 meta
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

blockdev-amend will be used similiar to blockdev-create
to allow on the fly changes of the structure of the format based block devices.

Current plan is to first support encryption keyslot management for luks
based formats (raw and embedded in qcow2)

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-12-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qapi/block-core.json      |  42 ++++++++++++++
 qapi/job.json             |   4 +-
 include/block/block_int.h |  21 +++++--
 block/amend.c             | 113 ++++++++++++++++++++++++++++++++++++++
 block/Makefile.objs       |   2 +-
 5 files changed, 174 insertions(+), 8 deletions(-)
 create mode 100644 block/amend.c

diff --git a/qapi/block-core.json b/qapi/block-core.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
   'data': { 'job-id': 'str',
             'options': 'BlockdevCreateOptions' } }
 
+##
+# @BlockdevAmendOptions:
+#
+# Options for amending an image format
+#
+# @driver:          Block driver of the node to amend.
+#
+# Since: 5.1
+##
+{ 'union': 'BlockdevAmendOptions',
+  'base': {
+      'driver':         'BlockdevDriver' },
+  'discriminator': 'driver',
+  'data': {
+  } }
+
+##
+# @x-blockdev-amend:
+#
+# Starts a job to amend format specific options of an existing open block device
+# The job is automatically finalized, but a manual job-dismiss is required.
+#
+# @job-id:          Identifier for the newly created job.
+#
+# @node-name:       Name of the block node to work on
+#
+# @options:         Options (driver specific)
+#
+# @force:           Allow unsafe operations, format specific
+#                   For luks that allows erase of the last active keyslot
+#                   (permanent loss of data),
+#                   and replacement of an active keyslot
+#                   (possible loss of data if IO error happens)
+#
+# Since: 5.1
+##
+{ 'command': 'x-blockdev-amend',
+  'data': { 'job-id': 'str',
+            'node-name': 'str',
+            'options': 'BlockdevAmendOptions',
+            '*force': 'bool' } }
+
 ##
 # @BlockErrorAction:
 #
diff --git a/qapi/job.json b/qapi/job.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -XXX,XX +XXX,XX @@
 #
 # @create: image creation job type, see "blockdev-create" (since 3.0)
 #
+# @amend: image options amend job type, see "x-blockdev-amend" (since 5.1)
+#
 # Since: 1.7
 ##
 { 'enum': 'JobType',
-  'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+  'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'amend'] }
 
 ##
 # @JobStatus:
diff --git a/include/block/block_int.h b/include/block/block_int.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
     int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
                           Error **errp);
     void (*bdrv_close)(BlockDriverState *bs);
+
+
     int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
                                        Error **errp);
     int coroutine_fn (*bdrv_co_create_opts)(BlockDriver *drv,
                                             const char *filename,
                                             QemuOpts *opts,
                                             Error **errp);
+
+    int coroutine_fn (*bdrv_co_amend)(BlockDriverState *bs,
+                                      BlockdevAmendOptions *opts,
+                                      bool force,
+                                      Error **errp);
+
+    int (*bdrv_amend_options)(BlockDriverState *bs,
+                              QemuOpts *opts,
+                              BlockDriverAmendStatusCB *status_cb,
+                              void *cb_opaque,
+                              bool force,
+                              Error **errp);
+
     int (*bdrv_make_empty)(BlockDriverState *bs);
 
     /*
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
                                       BdrvCheckResult *result,
                                       BdrvCheckMode fix);
 
-    int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
-                              BlockDriverAmendStatusCB *status_cb,
-                              void *cb_opaque,
-                              bool force,
-                              Error **errp);
-
     void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event);
 
     /* TODO Better pass a option string/QDict/QemuOpts to add any rule? */
diff --git a/block/amend.c b/block/amend.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/block/amend.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * Block layer code related to image options amend
+ *
+ * Copyright (c) 2018 Kevin Wolf <kwolf@redhat.com>
+ * Copyright (c) 2020 Red Hat. Inc
+ *
+ * Heavily based on create.c
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qemu/job.h"
+#include "qemu/main-loop.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-visit-block-core.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/error.h"
+
+typedef struct BlockdevAmendJob {
+    Job common;
+    BlockdevAmendOptions *opts;
+    BlockDriverState *bs;
+    bool force;
+} BlockdevAmendJob;
+
+static int coroutine_fn blockdev_amend_run(Job *job, Error **errp)
+{
+    BlockdevAmendJob *s = container_of(job, BlockdevAmendJob, common);
+    int ret;
+
+    job_progress_set_remaining(&s->common, 1);
+    ret = s->bs->drv->bdrv_co_amend(s->bs, s->opts, s->force, errp);
+    job_progress_update(&s->common, 1);
+    qapi_free_BlockdevAmendOptions(s->opts);
+    return ret;
+}
+
+static const JobDriver blockdev_amend_job_driver = {
+    .instance_size = sizeof(BlockdevAmendJob),
+    .job_type      = JOB_TYPE_AMEND,
+    .run           = blockdev_amend_run,
+};
+
+void qmp_x_blockdev_amend(const char *job_id,
+                          const char *node_name,
+                          BlockdevAmendOptions *options,
+                          bool has_force,
+                          bool force,
+                          Error **errp)
+{
+    BlockdevAmendJob *s;
+    const char *fmt = BlockdevDriver_str(options->driver);
+    BlockDriver *drv = bdrv_find_format(fmt);
+    BlockDriverState *bs = bdrv_find_node(node_name);
+
+
+    if (!drv) {
+        error_setg(errp, "Block driver '%s' not found or not supported", fmt);
+        return;
+    }
+
+    /*
+     * If the driver is in the schema, we know that it exists. But it may not
+     * be whitelisted.
+     */
+    if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
+        error_setg(errp, "Driver is not whitelisted");
+        return;
+    }
+
+    if (bs->drv != drv) {
+        error_setg(errp,
+                   "x-blockdev-amend doesn't support changing the block driver");
+        return;
+    }
+
+    /* Error out if the driver doesn't support .bdrv_co_amend */
+    if (!drv->bdrv_co_amend) {
+        error_setg(errp, "Driver does not support x-blockdev-amend");
+        return;
+    }
+
+    /* Create the block job */
+    s = job_create(job_id, &blockdev_amend_job_driver, NULL,
+                   bdrv_get_aio_context(bs), JOB_DEFAULT | JOB_MANUAL_DISMISS,
+                   NULL, NULL, errp);
+    if (!s) {
+        return;
+    }
+
+    s->bs = bs,
+    s->opts = QAPI_CLONE(BlockdevAmendOptions, options),
+    s->force = has_force ? force : false;
+    job_start(&s->common);
+}
diff --git a/block/Makefile.objs b/block/Makefile.objs
index XXXXXXX..XXXXXXX 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -XXX,XX +XXX,XX @@ block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
 block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
 block-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o
-block-obj-y += null.o mirror.o commit.o io.o create.o
+block-obj-y += null.o mirror.o commit.o io.o create.o amend.o
 block-obj-y += throttle-groups.o
 block-obj-$(CONFIG_LINUX) += nvme.o
 
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-13-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qapi/block-core.json | 14 ++++++++-
 block/crypto.c       | 72 ++++++++++++++++++++++++++++++++------------
 2 files changed, 66 insertions(+), 20 deletions(-)

From: Maxim Levitsky <mlevitsk@redhat.com>

Currently the implementation only supports amending the encryption
options, unlike the qemu-img version

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200608094030.670121-14-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qapi/block-core.json | 16 +++++++++++++++-
 block/qcow2.c        | 39 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
   'data': { }
 }
 
+##
+# @BlockdevAmendOptionsQcow2:
+#
+# Driver specific image amend options for qcow2.
+# For now, only encryption options can be amended
+#
+# @encrypt          Encryption options to be amended
+#
+# Since: 5.1
+##
+{ 'struct': 'BlockdevAmendOptionsQcow2',
+  'data': { '*encrypt':         'QCryptoBlockAmendOptions' } }
+
 ##
 # @BlockdevAmendOptions:
 #
@@ -XXX,XX +XXX,XX @@
       'driver':         'BlockdevDriver' },
   'discriminator': 'driver',
   'data': {
-      'luks':           'BlockdevAmendOptionsLUKS' } }
+      'luks':           'BlockdevAmendOptionsLUKS',
+      'qcow2':          'BlockdevAmendOptionsQcow2' } }
 
 ##
 # @x-blockdev-amend:
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
     return 0;
 }
 
+static int coroutine_fn qcow2_co_amend(BlockDriverState *bs,
+                                       BlockdevAmendOptions *opts,
+                                       bool force,
+                                       Error **errp)
+{
+    BlockdevAmendOptionsQcow2 *qopts = &opts->u.qcow2;
+    BDRVQcow2State *s = bs->opaque;
+    int ret = 0;
+
+    if (qopts->has_encrypt) {
+        if (!s->crypto) {
+            error_setg(errp, "image is not encrypted, can't amend");
+            return -EOPNOTSUPP;
+        }
+
+        if (qopts->encrypt->format != Q_CRYPTO_BLOCK_FORMAT_LUKS) {
+            error_setg(errp,
+                       "Amend can't be used to change the qcow2 encryption format");
+            return -EOPNOTSUPP;
+        }
+
+        if (s->crypt_method_header != QCOW_CRYPT_LUKS) {
+            error_setg(errp,
+                       "Only LUKS encryption options can be amended for qcow2 with blockdev-amend");
+            return -EOPNOTSUPP;
+        }
+
+        ret = qcrypto_block_amend_options(s->crypto,
+                                          qcow2_crypto_hdr_read_func,
+                                          qcow2_crypto_hdr_write_func,
+                                          bs,
+                                          qopts->encrypt,
+                                          force,
+                                          errp);
+    }
+    return ret;
+}
+
 /*
  * If offset or size are negative, respectively, they will not be included in
  * the BLOCK_IMAGE_CORRUPTED event emitted.
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
     .mutable_opts        = mutable_opts,
     .bdrv_co_check       = qcow2_co_check,
     .bdrv_amend_options  = qcow2_amend_options,
+    .bdrv_co_amend       = qcow2_co_amend,
 
     .bdrv_detach_aio_context  = qcow2_detach_aio_context,
     .bdrv_attach_aio_context  = qcow2_attach_aio_context,
-- 
2.26.2

From: Maxim Levitsky <mlevitsk@redhat.com>

This commit adds two tests that cover the
new blockdev-amend functionality of luks and qcow2 driver

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200608094030.670121-15-mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/295     | 279 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/295.out |  40 ++++++
 tests/qemu-iotests/296     | 234 +++++++++++++++++++++++++++++++
 tests/qemu-iotests/296.out |  33 +++++
 tests/qemu-iotests/group   |   2 +
 5 files changed, 588 insertions(+)
 create mode 100755 tests/qemu-iotests/295
 create mode 100644 tests/qemu-iotests/295.out
 create mode 100755 tests/qemu-iotests/296
 create mode 100644 tests/qemu-iotests/296.out

diff --git a/tests/qemu-iotests/295 b/tests/qemu-iotests/295
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/295
@@ -XXX,XX +XXX,XX @@
+#!/usr/bin/env python3
+#
+# Test case QMP's encrypted key management
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+import os
+import time
+import json
+
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+class Secret:
+    def __init__(self, index):
+        self._id = "keysec" + str(index)
+        # you are not supposed to see the password...
+        self._secret = "hunter" + str(index)
+
+    def id(self):
+        return self._id
+
+    def secret(self):
+        return self._secret
+
+    def to_cmdline_object(self):
+        return  [ "secret,id=" + self._id + ",data=" + self._secret]
+
+    def to_qmp_object(self):
+        return { "qom_type" : "secret", "id": self.id(),
+                 "props": { "data": self.secret() } }
+
+################################################################################
+class EncryptionSetupTestCase(iotests.QMPTestCase):
+
+    # test case startup
+    def setUp(self):
+        # start the VM
+        self.vm = iotests.VM()
+        self.vm.launch()
+
+        # create the secrets and load 'em into the VM
+        self.secrets = [ Secret(i) for i in range(0, 6) ]
+        for secret in self.secrets:
+            result = self.vm.qmp("object-add", **secret.to_qmp_object())
+            self.assert_qmp(result, 'return', {})
+
+        if iotests.imgfmt == "qcow2":
+            self.pfx = "encrypt."
+            self.img_opts = [ '-o', "encrypt.format=luks" ]
+        else:
+            self.pfx = ""
+            self.img_opts = []
+
+    # test case shutdown
+    def tearDown(self):
+        # stop the VM
+        self.vm.shutdown()
+
+    ###########################################################################
+    # create the encrypted block device
+    def createImg(self, file, secret):
+
+        iotests.qemu_img(
+            'create',
+            '--object', *secret.to_cmdline_object(),
+            '-f', iotests.imgfmt,
+            '-o', self.pfx + 'key-secret=' + secret.id(),
+            '-o', self.pfx + 'iter-time=10',
+            *self.img_opts,
+            file,
+            '1M')
+
+    ###########################################################################
+    # open an encrypted block device
+    def openImageQmp(self, id, file, secret, read_only = False):
+
+        encrypt_options = {
+            'key-secret' : secret.id()
+        }
+
+        if iotests.imgfmt == "qcow2":
+            encrypt_options = {
+                'encrypt': {
+                    'format':'luks',
+                    **encrypt_options
+                }
+            }
+
+        result = self.vm.qmp('blockdev-add', **
+            {
+                'driver': iotests.imgfmt,
+                'node-name': id,
+                'read-only': read_only,
+
+                **encrypt_options,
+
+                'file': {
+                    'driver': 'file',
+                    'filename': test_img,
+                }
+            }
+        )
+        self.assert_qmp(result, 'return', {})
+
+    # close the encrypted block device
+    def closeImageQmp(self, id):
+        result = self.vm.qmp('blockdev-del', **{ 'node-name': id })
+        self.assert_qmp(result, 'return', {})
+
+    ###########################################################################
+    # add a key to an encrypted block device
+    def addKeyQmp(self, id, new_secret, secret = None,
+                  slot = None, force = False):
+
+        crypt_options = {
+            'state'      : 'active',
+            'new-secret' : new_secret.id(),
+            'iter-time' : 10
+        }
+
+        if slot != None:
+            crypt_options['keyslot'] = slot
+
+
+        if secret != None:
+            crypt_options['secret'] = secret.id()
+
+        if iotests.imgfmt == "qcow2":
+            crypt_options['format'] = 'luks'
+            crypt_options = {
+                'encrypt': crypt_options
+            }
+
+        args = {
+            'node-name': id,
+            'job-id' : 'job_add_key',
+            'options' : {
+                    'driver' : iotests.imgfmt,
+                    **crypt_options
+                },
+        }
+
+        if force == True:
+            args['force'] = True
+
+        #TODO: check what jobs return
+        result = self.vm.qmp('x-blockdev-amend', **args)
+        assert result['return'] == {}
+        self.vm.run_job('job_add_key')
+
+    # erase a key from an encrypted block device
+    def eraseKeyQmp(self, id, old_secret = None, slot = None, force = False):
+
+        crypt_options = {
+            'state'      : 'inactive',
+        }
+
+        if slot != None:
+            crypt_options['keyslot'] = slot
+        if old_secret != None:
+            crypt_options['old-secret'] = old_secret.id()
+
+        if iotests.imgfmt == "qcow2":
+            crypt_options['format'] = 'luks'
+            crypt_options = {
+                'encrypt': crypt_options
+            }
+
+        args = {
+            'node-name': id,
+            'job-id' : 'job_erase_key',
+            'options' : {
+                    'driver' : iotests.imgfmt,
+                    **crypt_options
+                },
+        }
+
+        if force == True:
+            args['force'] = True
+
+        result = self.vm.qmp('x-blockdev-amend', **args)
+        assert result['return'] == {}
+        self.vm.run_job('job_erase_key')
+
+    ###########################################################################
+    # create image, and change its key
+    def testChangeKey(self):
+
+        # create the image with secret0 and open it
+        self.createImg(test_img, self.secrets[0]);
+        self.openImageQmp("testdev", test_img, self.secrets[0])
+
+        # add key to slot 1
+        self.addKeyQmp("testdev", new_secret = self.secrets[1])
+
+        # add key to slot 5
+        self.addKeyQmp("testdev", new_secret = self.secrets[2], slot=5)
+
+        # erase key from slot 0
+        self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
+
+        #reopen the image with secret1
+        self.closeImageQmp("testdev")
+        self.openImageQmp("testdev", test_img, self.secrets[1])
+
+        # close and erase the image for good
+        self.closeImageQmp("testdev")
+        os.remove(test_img)
+
+    # test that if we erase the old password,
+    # we can still change the encryption keys using 'old-secret'
+    def testOldPassword(self):
+
+        # create the image with secret0 and open it
+        self.createImg(test_img, self.secrets[0]);
+        self.openImageQmp("testdev", test_img, self.secrets[0])
+
+        # add key to slot 1
+        self.addKeyQmp("testdev", new_secret = self.secrets[1])
+
+        # erase key from slot 0
+        self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
+
+        # this will fail as the old password is no longer valid
+        self.addKeyQmp("testdev", new_secret = self.secrets[2])
+
+        # this will work
+        self.addKeyQmp("testdev", new_secret = self.secrets[2], secret = self.secrets[1])
+
+        # close and erase the image for good
+        self.closeImageQmp("testdev")
+        os.remove(test_img)
+
+    def testUseForceLuke(self):
+
+        self.createImg(test_img, self.secrets[0]);
+        self.openImageQmp("testdev", test_img, self.secrets[0])
+
+        # Add bunch of secrets
+        self.addKeyQmp("testdev", new_secret = self.secrets[1], slot=4)
+        self.addKeyQmp("testdev", new_secret = self.secrets[4], slot=2)
+
+        # overwrite an active secret
+        self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2)
+        self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2, force=True)
+
+        self.addKeyQmp("testdev", new_secret = self.secrets[0])
+
+        # Now erase all the secrets
+        self.eraseKeyQmp("testdev", old_secret = self.secrets[5])
+        self.eraseKeyQmp("testdev", slot=4)
+
+        # erase last keyslot
+        self.eraseKeyQmp("testdev", old_secret = self.secrets[0])
+        self.eraseKeyQmp("testdev", old_secret = self.secrets[0], force=True)
+
+        self.closeImageQmp("testdev")
+        os.remove(test_img)
+
+
+if __name__ == '__main__':
+    # Encrypted formats support
+    iotests.activate_logging()
+    iotests.main(supported_fmts = ['qcow2', 'luks'])
diff --git a/tests/qemu-iotests/295.out b/tests/qemu-iotests/295.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/295.out
@@ -XXX,XX +XXX,XX @@
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+Job failed: Invalid password, cannot unlock any keyslot
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+Job failed: Refusing to overwrite active keyslot 2 - please erase it first
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+Job failed: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
+{"return": {}}
+...
+----------------------------------------------------------------------
+Ran 3 tests
+
+OK
diff --git a/tests/qemu-iotests/296 b/tests/qemu-iotests/296
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/296
@@ -XXX,XX +XXX,XX @@
+#!/usr/bin/env python3
+#
+# Test case for encryption key management versus image sharing
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+import os
+import time
+import json
+
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+class Secret:
+    def __init__(self, index):
+        self._id = "keysec" + str(index)
+        # you are not supposed to see the password...
+        self._secret = "hunter" + str(index)
+
+    def id(self):
+        return self._id
+
+    def secret(self):
+        return self._secret
+
+    def to_cmdline_object(self):
+        return  [ "secret,id=" + self._id + ",data=" + self._secret]
+
+    def to_qmp_object(self):
+        return { "qom_type" : "secret", "id": self.id(),
+                 "props": { "data": self.secret() } }
+
+################################################################################
+
+class EncryptionSetupTestCase(iotests.QMPTestCase):
+
+    # test case startup
+    def setUp(self):
+
+        # start the VMs
+        self.vm1 = iotests.VM(path_suffix = 'VM1')
+        self.vm2 = iotests.VM(path_suffix = 'VM2')
+        self.vm1.launch()
+        self.vm2.launch()
+
+        # create the secrets and load 'em into the VMs
+        self.secrets = [ Secret(i) for i in range(0, 4) ]
+        for secret in self.secrets:
+            result = self.vm1.qmp("object-add", **secret.to_qmp_object())
+            self.assert_qmp(result, 'return', {})
+            result = self.vm2.qmp("object-add", **secret.to_qmp_object())
+            self.assert_qmp(result, 'return', {})
+
+    # test case shutdown
+    def tearDown(self):
+        # stop the VM
+        self.vm1.shutdown()
+        self.vm2.shutdown()
+
+    ###########################################################################
+    # create the encrypted block device using qemu-img
+    def createImg(self, file, secret):
+
+        output = iotests.qemu_img_pipe(
+            'create',
+            '--object', *secret.to_cmdline_object(),
+            '-f', iotests.imgfmt,
+            '-o', 'key-secret=' + secret.id(),
+            '-o', 'iter-time=10',
+            file,
+            '1M')
+
+        iotests.log(output, filters=[iotests.filter_test_dir])
+
+    # attempts to add a key using qemu-img
+    def addKey(self, file, secret, new_secret):
+
+        image_options = {
+            'key-secret' : secret.id(),
+            'driver' : iotests.imgfmt,
+            'file' : {
+                'driver':'file',
+                'filename': file,
+                }
+            }
+
+        output = iotests.qemu_img_pipe(
+            'amend',
+            '--object', *secret.to_cmdline_object(),
+            '--object', *new_secret.to_cmdline_object(),
+
+            '-o', 'state=active',
+            '-o', 'new-secret=' + new_secret.id(),
+            '-o', 'iter-time=10',
+
+            "json:" + json.dumps(image_options)
+            )
+
+        iotests.log(output, filters=[iotests.filter_test_dir])
+
+    ###########################################################################
+    # open an encrypted block device
+    def openImageQmp(self, vm, id, file, secret,
+                     readOnly = False, reOpen = False):
+
+        command = 'x-blockdev-reopen' if reOpen else 'blockdev-add'
+
+        result = vm.qmp(command, **
+            {
+                'driver': iotests.imgfmt,
+                'node-name': id,
+                'read-only': readOnly,
+                'key-secret' : secret.id(),
+                'file': {
+                    'driver': 'file',
+                    'filename': test_img,
+                }
+            }
+        )
+        self.assert_qmp(result, 'return', {})
+
+    # close the encrypted block device
+    def closeImageQmp(self, vm, id):
+        result = vm.qmp('blockdev-del', **{ 'node-name': id })
+        self.assert_qmp(result, 'return', {})
+
+    ###########################################################################
+
+    # add a key to an encrypted block device
+    def addKeyQmp(self, vm, id, new_secret):
+
+        args = {
+            'node-name': id,
+            'job-id' : 'job0',
+            'options' : {
+                'state'     : 'active',
+                'driver'    : iotests.imgfmt,
+                'new-secret': new_secret.id(),
+                'iter-time' : 10
+            },
+        }
+
+        result = vm.qmp('x-blockdev-amend', **args)
+        assert result['return'] == {}
+        vm.run_job('job0')
+
+    # test that when the image opened by two qemu processes,
+    # neither of them can update the image
+    def test1(self):
+        self.createImg(test_img, self.secrets[0]);
+
+        # VM1 opens the image and adds a key
+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0])
+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[1])
+
+
+        # VM2 opens the image
+        self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
+
+
+        # neither VMs now should be able to add a key
+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+        self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
+
+
+        # VM 1 closes the image
+        self.closeImageQmp(self.vm1, "testdev")
+
+
+        # now VM2 can add the key
+        self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
+
+
+        # qemu-img should also not be able to add a key
+        self.addKey(test_img, self.secrets[0], self.secrets[2])
+
+        # cleanup
+        self.closeImageQmp(self.vm2, "testdev")
+        os.remove(test_img)
+
+
+    def test2(self):
+        self.createImg(test_img, self.secrets[0]);
+
+        # VM1 opens the image readonly
+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0],
+                          readOnly = True)
+
+        # VM2 opens the image
+        self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0])
+
+        # VM1 can't add a key since image is readonly
+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+
+        # VM2 can't add a key since VM is has the image opened
+        self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2])
+
+
+        #VM1 reopens the image read-write
+        self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0],
+                          reOpen = True, readOnly = False)
+
+        # VM1 still can't add the key
+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+
+        # VM2 gets away
+        self.closeImageQmp(self.vm2, "testdev")
+
+        # VM1 now can add the key
+        self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2])
+
+        self.closeImageQmp(self.vm1, "testdev")
+        os.remove(test_img)
+
+
+if __name__ == '__main__':
+    # support only raw luks since luks encrypted qcow2 is a proper
+    # format driver which doesn't allow any sharing
+    iotests.activate_logging()
+    iotests.main(supported_fmts = ['luks'])
diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/296.out
@@ -XXX,XX +XXX,XX @@
+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+qemu-img: Failed to get shared "consistent read" lock
+Is another process using the image [TEST_DIR/test.img]?
+
+Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
+
+Job failed: Block node is read-only
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+Job failed: Failed to get shared "consistent read" lock
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+..
+----------------------------------------------------------------------
+Ran 2 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -XXX,XX +XXX,XX @@
 292 rw auto quick
 293 rw auto
 294 rw auto quick
+295 rw auto
+296 rw auto
 297 meta
-- 
2.26.2

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Commit 96927c744 replaced qdev_init_nofail() call by
isa_realize_and_unref() which has a different error
message. Update the test output accordingly.

Gitlab CI error after merging b77b5b3dc7:
https://gitlab.com/qemu-project/qemu/-/jobs/597414772#L4375

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200616154949.6586-1-philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/051.pc.out | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/051.pc.out
+++ b/tests/qemu-iotests/051.pc.out
@@ -XXX,XX +XXX,XX @@ QEMU X.Y.Z monitor - type 'help' for more information
 
 Testing: -drive if=ide
 QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: Initialization of device ide-hd failed: Device needs media, but drive is empty
+(qemu) QEMU_PROG: Device needs media, but drive is empty
 
 Testing: -drive if=virtio
 QEMU X.Y.Z monitor - type 'help' for more information
@@ -XXX,XX +XXX,XX @@ QEMU X.Y.Z monitor - type 'help' for more information
 
 Testing: -drive file=TEST_DIR/t.qcow2,if=ide,readonly=on
 QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: Initialization of device ide-hd failed: Block node is read-only
+(qemu) QEMU_PROG: Block node is read-only
 
 Testing: -drive file=TEST_DIR/t.qcow2,if=virtio,readonly=on
 QEMU X.Y.Z monitor - type 'help' for more information
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

820c6bee534ec3b added testing of qcow2.py into 291, and it breaks 291
with external data file. Actually, 291 is bad place for qcow2.py
testing, better add a separate test.

For now, drop qcow2.py testing from 291 to fix the regression.

Fixes: 820c6bee534ec3b
Reported-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200618154052.8629-1-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/291     |  4 ----
 tests/qemu-iotests/291.out | 33 ---------------------------------
 2 files changed, 37 deletions(-)

diff --git a/tests/qemu-iotests/291 b/tests/qemu-iotests/291
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/291
+++ b/tests/qemu-iotests/291
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c 'w 1M 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io
 $QEMU_IMG bitmap --disable -f $IMGFMT "$TEST_IMG" b1
 $QEMU_IMG bitmap --enable -f $IMGFMT "$TEST_IMG" b2
 $QEMU_IO -c 'w 2M 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io
-echo "Check resulting qcow2 header extensions:"
-$PYTHON qcow2.py "$TEST_IMG" dump-header-exts
 
 echo
 echo "=== Bitmap preservation not possible to non-qcow2 ==="
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG bitmap --merge tmp -f $IMGFMT "$TEST_IMG" b0
 $QEMU_IMG bitmap --remove --image-opts \
     driver=$IMGFMT,file.driver=file,file.filename="$TEST_IMG" tmp
 _img_info --format-specific
-echo "Check resulting qcow2 header extensions:"
-$PYTHON qcow2.py "$TEST_IMG" dump-header-exts
 
 echo
 echo "=== Check bitmap contents ==="
diff --git a/tests/qemu-iotests/291.out b/tests/qemu-iotests/291.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/291.out
+++ b/tests/qemu-iotests/291.out
@@ -XXX,XX +XXX,XX @@ wrote 1048576/1048576 bytes at offset 1048576
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 1048576/1048576 bytes at offset 2097152
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Check resulting qcow2 header extensions:
-Header extension:
-magic                     0xe2792aca (Backing format)
-length                    5
-data                      'qcow2'
-
-Header extension:
-magic                     0x6803f857 (Feature table)
-length                    336
-data                      <binary>
-
-Header extension:
-magic                     0x23852875 (Bitmaps)
-length                    24
-nb_bitmaps                2
-reserved32                0
-bitmap_directory_size     0x40
-bitmap_directory_offset   0x510000
-
 
 === Bitmap preservation not possible to non-qcow2 ===
 
@@ -XXX,XX +XXX,XX @@ Format specific information:
             granularity: 65536
     refcount bits: 16
     corrupt: false
-Check resulting qcow2 header extensions:
-Header extension:
-magic                     0x6803f857 (Feature table)
-length                    336
-data                      <binary>
-
-Header extension:
-magic                     0x23852875 (Bitmaps)
-length                    24
-nb_bitmaps                3
-reserved32                0
-bitmap_directory_size     0x60
-bitmap_directory_offset   0x520000
-
 
 === Check bitmap contents ===
 
-- 
2.26.2

The following changes since commit 804b30d25f8d70dc2dea951883ea92235274a50c:

Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220130' into staging (2022-01-31 11:10:08 +0000)

are available in the Git repository at:

https://gitlab.com/hreitz/qemu.git tags/pull-block-2022-02-01

for you to fetch changes up to 751486c18555169ca4baf59440275d5831140822:

block.h: remove outdated comment (2022-02-01 13:28:53 +0100)

----------------------------------------------------------------
Block patches:
- Add support to the iotests to test qcow2's zstd compression mode
- Fix post-migration block node permissions
- iotests fixes (051 and mirror-ready-cancel-error)
- Remove an outdated comment

----------------------------------------------------------------
Emanuele Giuseppe Esposito (1):
  block.h: remove outdated comment

Hanna Reitz (3):
  iotests/MRCE: Write data to source
  block-backend: Retain permissions after migration
  iotests/migration-permissions: New test

Thomas Huth (1):
  tests/qemu-iotests: Fix 051 for binaries without 'lsi53c895a'

Vladimir Sementsov-Ogievskiy (19):
  iotests.py: img_info_log(): rename imgopts argument
  iotests.py: implement unsupported_imgopts
  iotests: specify some unsupported_imgopts for python iotests
  iotests.py: qemu_img*("create"): support
    IMGOPTS='compression_type=zstd'
  iotests: drop qemu_img_verbose() helper
  iotests.py: rewrite default luks support in qemu_img
  iotest 303: explicit compression type
  iotest 065: explicit compression type
  iotests.py: filter out successful output of qemu-img create
  iotests.py: filter compression type out
  iotest 302: use img_info_log() helper
  qcow2: simple case support for downgrading of qcow2 images with zstd
  iotests/common.rc: introduce _qcow2_dump_header helper
  iotests: massive use _qcow2_dump_header
  iotest 39: use _qcow2_dump_header
  iotests: bash tests: filter compression type
  iotests 60: more accurate set dirty bit in qcow2 header
  iotest 214: explicit compression type
  iotests: declare lack of support for compresion_type in IMGOPTS

-- 
2.34.1

This test assumes that mirror flushes the source when entering the READY
state, and that the format level will pass that flush on to the protocol
level (where we intercept it with blkdebug).

However, apparently that does not happen when using a VMDK image with
zeroed_grain=on, which actually is the default set by testenv.py.  Right
now, Python tests ignore IMGOPTS, though, so this has no effect; but
Vladimir has a series that will change this, so we need to fix this test
before that series lands.

We can fix it by writing data to the source before we start the mirror
job; apparently that makes the (VMDK) format layer change its mind and
pass on the pre-READY flush to the protocol level, so the test passes
again.  (I presume, without any data written, mirror just does a 64M
zero write on the target, which VMDK with zeroed_grain=on basically just
ignores.)

Without this, we do not get a flush, and so blkdebug only sees a single
flush at the end of the job instead of two, and therefore does not
inject an error, which makes the block job complete instead of raising
an error.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223165308.103793-1-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/tests/mirror-ready-cancel-error | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/tests/mirror-ready-cancel-error b/tests/qemu-iotests/tests/mirror-ready-cancel-error
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/tests/mirror-ready-cancel-error
+++ b/tests/qemu-iotests/tests/mirror-ready-cancel-error
@@ -XXX,XX +XXX,XX @@ class TestMirrorReadyCancelError(iotests.QMPTestCase):
         assert iotests.qemu_img_create('-f', iotests.imgfmt, target,
                                        str(image_size)) == 0
 
+        # Ensure that mirror will copy something before READY so the
+        # target format layer will forward the pre-READY flush to its
+        # file child
+        assert iotests.qemu_io_silent('-c', 'write -P 1 0 64k', source) == 0
+
         self.vm = iotests.VM()
         self.vm.launch()
 
@@ -XXX,XX +XXX,XX @@ class TestMirrorReadyCancelError(iotests.QMPTestCase):
         # Write something so will not leave the job immediately, but
         # flush first (which will fail, thanks to blkdebug)
         res = self.vm.qmp('human-monitor-command',
-                          command_line='qemu-io mirror-top "write 0 64k"')
+                          command_line='qemu-io mirror-top "write -P 2 0 64k"')
         self.assert_qmp(res, 'return', '')
 
         # Drain status change events
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We are going to support IMGOPTS environment variable like in bash
tests. Corresponding global variable in iotests.py should be called
imgopts. So to not interfere with function argument, rename it in
advance.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-2-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/210        | 8 ++++----
 tests/qemu-iotests/iotests.py | 5 +++--
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/210
+++ b/tests/qemu-iotests/210
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
         filter_path=disk_path,
         extra_args=['--object', 'secret,id=keysec0,data=foo'],
-        imgopts=True)
+        use_image_opts=True)
 
     #
     # Successful image creation (with non-default options)
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
         filter_path=disk_path,
         extra_args=['--object', 'secret,id=keysec0,data=foo'],
-        imgopts=True)
+        use_image_opts=True)
 
     #
     # Invalid BlockdevRef
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
         filter_path=disk_path,
         extra_args=['--object', 'secret,id=keysec0,data=foo'],
-        imgopts=True)
+        use_image_opts=True)
 
     #
     # Invalid sizes
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.luks') as disk_path, \
         'driver=luks,file.driver=file,file.filename=%s,key-secret=keysec0' % (disk_path),
         filter_path=disk_path,
         extra_args=['--object', 'secret,id=keysec0,data=foo'],
-        imgopts=True)
+        use_image_opts=True)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@ def qemu_img_log(*args):
     log(result, filters=[filter_testfiles])
     return result
 
-def img_info_log(filename, filter_path=None, imgopts=False, extra_args=()):
+def img_info_log(filename, filter_path=None, use_image_opts=False,
+                 extra_args=()):
     args = ['info']
-    if imgopts:
+    if use_image_opts:
         args.append('--image-opts')
     else:
         args += ['-f', imgfmt]
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We are going to support some addition IMGOPTS in python iotests like
in bash iotests. Similarly to bash iotests, we want a way to skip some
tests which can't work with specific IMGOPTS.

Globally for python iotests we will not support things like
'data_file=$TEST_IMG.ext_data_file' in IMGOPTS, so, forbid this
globally in iotests.py.

Suggested-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/iotests.py | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@ def _verify_virtio_scsi_pci_or_ccw() -> None:
         notrun('Missing virtio-scsi-pci or virtio-scsi-ccw in QEMU binary')
 
 
+def _verify_imgopts(unsupported: Sequence[str] = ()) -> None:
+    imgopts = os.environ.get('IMGOPTS')
+    # One of usage examples for IMGOPTS is "data_file=$TEST_IMG.ext_data_file"
+    # but it supported only for bash tests. We don't have a concept of global
+    # TEST_IMG in iotests.py, not saying about somehow parsing $variables.
+    # So, for simplicity let's just not support any IMGOPTS with '$' inside.
+    unsup = list(unsupported) + ['$']
+    if imgopts and any(x in imgopts for x in unsup):
+        notrun(f'not suitable for this imgopts: {imgopts}')
+
+
 def supports_quorum():
     return 'quorum' in qemu_img_pipe('--help')
 
@@ -XXX,XX +XXX,XX @@ def execute_setup_common(supported_fmts: Sequence[str] = (),
                          unsupported_fmts: Sequence[str] = (),
                          supported_protocols: Sequence[str] = (),
                          unsupported_protocols: Sequence[str] = (),
-                         required_fmts: Sequence[str] = ()) -> bool:
+                         required_fmts: Sequence[str] = (),
+                         unsupported_imgopts: Sequence[str] = ()) -> bool:
     """
     Perform necessary setup for either script-style or unittest-style tests.
 
@@ -XXX,XX +XXX,XX @@ def execute_setup_common(supported_fmts: Sequence[str] = (),
     _verify_aio_mode(supported_aio_modes)
     _verify_formats(required_fmts)
     _verify_virtio_blk()
+    _verify_imgopts(unsupported_imgopts)
 
     return debug
 
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We are going to support IMGOPTS for python iotests. Still some iotests
will not work with common IMGOPTS used with bash iotests like
specifying refcount_bits and compat qcow2 options. So we
should define corresponding unsupported_imgopts for now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20211223160144.1097696-4-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/044                                 | 3 ++-
 tests/qemu-iotests/065                                 | 3 ++-
 tests/qemu-iotests/163                                 | 3 ++-
 tests/qemu-iotests/165                                 | 3 ++-
 tests/qemu-iotests/196                                 | 3 ++-
 tests/qemu-iotests/242                                 | 3 ++-
 tests/qemu-iotests/246                                 | 3 ++-
 tests/qemu-iotests/254                                 | 3 ++-
 tests/qemu-iotests/260                                 | 3 ++-
 tests/qemu-iotests/274                                 | 3 ++-
 tests/qemu-iotests/281                                 | 3 ++-
 tests/qemu-iotests/303                                 | 3 ++-
 tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 ++-
 tests/qemu-iotests/tests/migrate-bitmaps-test          | 3 ++-
 tests/qemu-iotests/tests/remove-bitmap-from-backing    | 3 ++-
 15 files changed, 30 insertions(+), 15 deletions(-)

diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/044
+++ b/tests/qemu-iotests/044
@@ -XXX,XX +XXX,XX @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 unsupported_imgopts=['refcount_bits'])
diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/065
+++ b/tests/qemu-iotests/065
@@ -XXX,XX +XXX,XX @@ TestQMP = None
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 unsupported_imgopts=['refcount_bits'])
diff --git a/tests/qemu-iotests/163 b/tests/qemu-iotests/163
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/163
+++ b/tests/qemu-iotests/163
@@ -XXX,XX +XXX,XX @@ ShrinkBaseClass = None
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['raw', 'qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/165
+++ b/tests/qemu-iotests/165
@@ -XXX,XX +XXX,XX @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/196 b/tests/qemu-iotests/196
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/196
+++ b/tests/qemu-iotests/196
@@ -XXX,XX +XXX,XX @@ class TestInvalidateAutoclear(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/242
+++ b/tests/qemu-iotests/242
@@ -XXX,XX +XXX,XX @@ from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \
     file_path, img_info_log, log, filter_qemu_io
 
 iotests.script_initialize(supported_fmts=['qcow2'],
-                          supported_protocols=['file'])
+                          supported_protocols=['file'],
+                          unsupported_imgopts=['refcount_bits', 'compat'])
 
 disk = file_path('disk')
 chunk = 256 * 1024
diff --git a/tests/qemu-iotests/246 b/tests/qemu-iotests/246
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/246
+++ b/tests/qemu-iotests/246
@@ -XXX,XX +XXX,XX @@
 import iotests
 from iotests import log
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          unsupported_imgopts=['compat'])
 size = 64 * 1024 * 1024 * 1024
 gran_small = 32 * 1024
 gran_large = 128 * 1024
diff --git a/tests/qemu-iotests/254 b/tests/qemu-iotests/254
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/254
+++ b/tests/qemu-iotests/254
@@ -XXX,XX +XXX,XX @@
 import iotests
 from iotests import qemu_img_create, file_path, log
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          unsupported_imgopts=['compat'])
 
 disk, top = file_path('disk', 'top')
 size = 1024 * 1024
diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/260
+++ b/tests/qemu-iotests/260
@@ -XXX,XX +XXX,XX @@ import iotests
 from iotests import qemu_img_create, file_path, log, filter_qmp_event
 
 iotests.script_initialize(
-    supported_fmts=['qcow2']
+    supported_fmts=['qcow2'],
+    unsupported_imgopts=['compat']
 )
 
 base, top = file_path('base', 'top')
diff --git a/tests/qemu-iotests/274 b/tests/qemu-iotests/274
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/274
+++ b/tests/qemu-iotests/274
@@ -XXX,XX +XXX,XX @@
 import iotests
 
 iotests.script_initialize(supported_fmts=['qcow2'],
-                          supported_platforms=['linux'])
+                          supported_platforms=['linux'],
+                          unsupported_imgopts=['refcount_bits', 'compat'])
 
 size_short = 1 * 1024 * 1024
 size_long = 2 * 1024 * 1024
diff --git a/tests/qemu-iotests/281 b/tests/qemu-iotests/281
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/281
+++ b/tests/qemu-iotests/281
@@ -XXX,XX +XXX,XX @@ class TestBlockdevBackupAbort(iotests.QMPTestCase):
 
 if __name__ == '__main__':
     iotests.main(supported_fmts=['qcow2'],
-                 supported_protocols=['file'])
+                 supported_protocols=['file'],
+                 unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/303
+++ b/tests/qemu-iotests/303
@@ -XXX,XX +XXX,XX @@ import iotests
 import subprocess
 from iotests import qemu_img_create, qemu_io, file_path, log, filter_qemu_io
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          unsupported_imgopts=['refcount_bits', 'compat'])
 
 disk = file_path('disk')
 chunk = 1024 * 1024
diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
+++ b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
@@ -XXX,XX +XXX,XX @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
 
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 unsupported_imgopts=['compat'])
diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-test b/tests/qemu-iotests/tests/migrate-bitmaps-test
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/tests/migrate-bitmaps-test
+++ b/tests/qemu-iotests/tests/migrate-bitmaps-test
@@ -XXX,XX +XXX,XX @@ def main() -> None:
 
     iotests.main(
         supported_fmts=['qcow2'],
-        supported_protocols=['file']
+        supported_protocols=['file'],
+        unsupported_imgopts=['compat']
     )
 
 
diff --git a/tests/qemu-iotests/tests/remove-bitmap-from-backing b/tests/qemu-iotests/tests/remove-bitmap-from-backing
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/tests/remove-bitmap-from-backing
+++ b/tests/qemu-iotests/tests/remove-bitmap-from-backing
@@ -XXX,XX +XXX,XX @@
 import iotests
 from iotests import log, qemu_img_create, qemu_img, qemu_img_pipe
 
-iotests.script_initialize(supported_fmts=['qcow2'])
+iotests.script_initialize(supported_fmts=['qcow2'],
+                          unsupported_imgopts=['compat'])
 
 top, base = iotests.file_path('top', 'base')
 size = '1M'
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Adding support of IMGOPTS (like in bash tests) allows user to pass a
lot of different options. Still, some may require additional logic.

Now we want compression_type option, so add some smart logic around it:
ignore compression_type=zstd in IMGOPTS, if test want qcow2 in
compatibility mode. As well, ignore compression_type for non-qcow2
formats.

Note that we may instead add support only to qemu_img_create(), but
that works bad:

1. We'll have to update a lot of tests to use qemu_img_create instead
   of qemu_img('create'). (still, we may want do it anyway, but no
   reason to create a dependancy between task of supporting IMGOPTS and
   updating a lot of tests)

2. Some tests use qemu_img_pipe('create', ..) - even more work on
   updating

3. Even if we update all tests to go through qemu_img_create, we'll
   need a way to avoid creating new tests using qemu_img*('create') -
   add assertions.. That doesn't seem good.

So, let's add support of IMGOPTS to most generic
qemu_img_pipe_and_status().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-5-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/iotests.py | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 
+import argparse
 import atexit
 import bz2
 from collections import OrderedDict
@@ -XXX,XX +XXX,XX @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
                                {-subp.returncode}: {cmd}\n')
         return (output, subp.returncode)
 
+def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
+    if not args or args[0] != 'create':
+        return list(args)
+    args = args[1:]
+
+    p = argparse.ArgumentParser(allow_abbrev=False)
+    p.add_argument('-f')
+    parsed, remaining = p.parse_known_args(args)
+
+    result = ['create']
+    if parsed.f is not None:
+        result += ['-f', parsed.f]
+
+    # IMGOPTS most probably contain options specific for the selected format,
+    # like extended_l2 or compression_type for qcow2. Test may want to create
+    # additional images in other formats that doesn't support these options.
+    # So, use IMGOPTS only for images created in imgfmt format.
+    if parsed.f == imgfmt and 'IMGOPTS' in os.environ:
+        result += ['-o', os.environ['IMGOPTS']]
+
+    result += remaining
+
+    return result
+
 def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]:
     """
     Run qemu-img and return both its output and its exit code
     """
-    full_args = qemu_img_args + list(args)
+    full_args = qemu_img_args + qemu_img_create_prepare_args(list(args))
     return qemu_tool_pipe_and_status('qemu-img', full_args)
 
 def qemu_img(*args: str) -> int:
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

qemu_img_verbose() has a drawback of not going through generic
qemu_img_pipe_and_status(). qemu_img_verbose() is not very popular, so
update the only two users to qemu_img_log() and drop qemu_img_verbose()
at all.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-6-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/044        | 5 +++--
 tests/qemu-iotests/044.out    | 1 +
 tests/qemu-iotests/209        | 7 ++++---
 tests/qemu-iotests/209.out    | 2 ++
 tests/qemu-iotests/iotests.py | 8 --------
 5 files changed, 10 insertions(+), 13 deletions(-)

diff --git a/tests/qemu-iotests/044 b/tests/qemu-iotests/044
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/044
+++ b/tests/qemu-iotests/044
@@ -XXX,XX +XXX,XX @@ import os
 import qcow2
 from qcow2 import QcowHeader
 import iotests
-from iotests import qemu_img, qemu_img_verbose, qemu_io
+from iotests import qemu_img, qemu_img_log, qemu_io
 import struct
 import subprocess
 import sys
@@ -XXX,XX +XXX,XX @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
 
     def test_grow_refcount_table(self):
         qemu_io('-c', 'write 3800M 1M', test_img)
-        qemu_img_verbose('check' , test_img)
+        qemu_img_log('check' , test_img)
         pass
 
 if __name__ == '__main__':
+    iotests.activate_logging()
     iotests.main(supported_fmts=['qcow2'],
                  supported_protocols=['file'],
                  unsupported_imgopts=['refcount_bits'])
diff --git a/tests/qemu-iotests/044.out b/tests/qemu-iotests/044.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/044.out
+++ b/tests/qemu-iotests/044.out
@@ -XXX,XX +XXX,XX @@
 No errors were found on the image.
 7292415/33554432 = 21.73% allocated, 0.00% fragmented, 0.00% compressed clusters
 Image end offset: 4296217088
+
 .
 ----------------------------------------------------------------------
 Ran 1 tests
diff --git a/tests/qemu-iotests/209 b/tests/qemu-iotests/209
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/209
+++ b/tests/qemu-iotests/209
@@ -XXX,XX +XXX,XX @@
 #
 
 import iotests
-from iotests import qemu_img_create, qemu_io, qemu_img_verbose, qemu_nbd, \
-                    file_path
+from iotests import qemu_img_create, qemu_io, qemu_img_log, qemu_nbd, \
+                    file_path, log
 
 iotests.script_initialize(supported_fmts=['qcow2'])
 
@@ -XXX,XX +XXX,XX @@ qemu_img_create('-f', iotests.imgfmt, disk, '1M')
 qemu_io('-f', iotests.imgfmt, '-c', 'write 0 512K', disk)
 
 qemu_nbd('-k', nbd_sock, '-x', 'exp', '-f', iotests.imgfmt, disk)
-qemu_img_verbose('map', '-f', 'raw', '--output=json', nbd_uri)
+qemu_img_log('map', '-f', 'raw', '--output=json', nbd_uri)
+log('done.')  # avoid new line at the end of output file
diff --git a/tests/qemu-iotests/209.out b/tests/qemu-iotests/209.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/209.out
+++ b/tests/qemu-iotests/209.out
@@ -XXX,XX +XXX,XX @@
 [{ "start": 0, "length": 524288, "depth": 0, "present": true, "zero": false, "data": true, "offset": 0},
 { "start": 524288, "length": 524288, "depth": 0, "present": true, "zero": true, "data": false, "offset": 524288}]
+
+done.
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@ def qemu_img_measure(*args):
 def qemu_img_check(*args):
     return json.loads(qemu_img_pipe("check", "--output", "json", *args))
 
-def qemu_img_verbose(*args):
-    '''Run qemu-img without suppressing its output and return the exit code'''
-    exitcode = subprocess.call(qemu_img_args + list(args))
-    if exitcode < 0:
-        sys.stderr.write('qemu-img received signal %i: %s\n'
-                         % (-exitcode, ' '.join(qemu_img_args + list(args))))
-    return exitcode
-
 def qemu_img_pipe(*args: str) -> str:
     '''Run qemu-img and return its output'''
     return qemu_img_pipe_and_status(*args)[0]
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Move the logic to more generic qemu_img_pipe_and_status(). Also behave
better when we have several -o options. And reuse argument parser of
course.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-7-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/iotests.py | 36 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 19 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@ def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
     args = args[1:]
 
     p = argparse.ArgumentParser(allow_abbrev=False)
+    # -o option may be specified several times
+    p.add_argument('-o', action='append', default=[])
     p.add_argument('-f')
     parsed, remaining = p.parse_known_args(args)
 
+    opts_list = parsed.o
+
     result = ['create']
     if parsed.f is not None:
         result += ['-f', parsed.f]
@@ -XXX,XX +XXX,XX @@ def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
     # like extended_l2 or compression_type for qcow2. Test may want to create
     # additional images in other formats that doesn't support these options.
     # So, use IMGOPTS only for images created in imgfmt format.
-    if parsed.f == imgfmt and 'IMGOPTS' in os.environ:
-        result += ['-o', os.environ['IMGOPTS']]
+    imgopts = os.environ.get('IMGOPTS')
+    if imgopts and parsed.f == imgfmt:
+        opts_list.insert(0, imgopts)
+
+    # default luks support
+    if parsed.f == 'luks' and \
+            all('key-secret' not in opts for opts in opts_list):
+        result += ['--object', luks_default_secret_object]
+        opts_list.append(luks_default_key_secret_opt)
+
+    for opts in opts_list:
+        result += ['-o', opts]
 
     result += remaining
 
@@ -XXX,XX +XXX,XX @@ def ordered_qmp(qmsg, conv_keys=True):
     return qmsg
 
 def qemu_img_create(*args):
-    args = list(args)
-
-    # default luks support
-    if '-f' in args and args[args.index('-f') + 1] == 'luks':
-        if '-o' in args:
-            i = args.index('-o')
-            if 'key-secret' not in args[i + 1]:
-                args[i + 1].append(luks_default_key_secret_opt)
-                args.insert(i + 2, '--object')
-                args.insert(i + 3, luks_default_secret_object)
-        else:
-            args = ['-o', luks_default_key_secret_opt,
-                    '--object', luks_default_secret_object] + args
-
-    args.insert(0, 'create')
-
-    return qemu_img(*args)
+    return qemu_img('create', *args)
 
 def qemu_img_measure(*args):
     return json.loads(qemu_img_pipe("measure", "--output", "json", *args))
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The test prints qcow2 header fields which depends on chosen compression
type. So, let's be explicit in what compression type we want and
independent of IMGOPTS. Test both existing compression types.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-8-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/303     | 25 ++++++++++++++++---------
 tests/qemu-iotests/303.out | 30 +++++++++++++++++++++++++++++-
 2 files changed, 45 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/303 b/tests/qemu-iotests/303
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/303
+++ b/tests/qemu-iotests/303
@@ -XXX,XX +XXX,XX @@ def add_bitmap(num, begin, end, disabled):
     log('')
 
 
-qemu_img_create('-f', iotests.imgfmt, disk, '10M')
-
-add_bitmap(1, 0, 6, False)
-add_bitmap(2, 6, 8, True)
-dump = ['./qcow2.py', disk, 'dump-header']
-subprocess.run(dump)
-# Dump the metadata in JSON format
-dump.append('-j')
-subprocess.run(dump)
+def test(compression_type: str, json_output: bool) -> None:
+    qemu_img_create('-f', iotests.imgfmt,
+                    '-o', f'compression_type={compression_type}',
+                    disk, '10M')
+    add_bitmap(1, 0, 6, False)
+    add_bitmap(2, 6, 8, True)
+
+    cmd = ['./qcow2.py', disk, 'dump-header']
+    if json_output:
+        cmd.append('-j')
+
+    subprocess.run(cmd)
+
+
+test('zlib', False)
+test('zstd', True)
diff --git a/tests/qemu-iotests/303.out b/tests/qemu-iotests/303.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/303.out
+++ b/tests/qemu-iotests/303.out
@@ -XXX,XX +XXX,XX @@ extra_data_size           0
 Bitmap table   type            size         offset
 0              all-zeroes      0            0
 
+Add bitmap 1
+wrote 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 1048576
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 2097152
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 3145728
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 4194304
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 5242880
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+
+Add bitmap 2
+wrote 1048576/1048576 bytes at offset 6291456
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+wrote 1048576/1048576 bytes at offset 7340032
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+
 {
     "magic": 1363560955,
     "version": 3,
@@ -XXX,XX +XXX,XX @@ Bitmap table   type            size         offset
     "refcount_table_clusters": 1,
     "nb_snapshots": 0,
     "snapshot_offset": 0,
-    "incompatible_features": 0,
+    "incompatible_features": 8,
     "compatible_features": 0,
     "autoclear_features": 1,
     "refcount_order": 4,
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The test checks different options. It of course fails if set
IMGOPTS='compression_type=zstd'. So, let's be explicit in what
compression type we want and independent of IMGOPTS. Test both existing
compression types.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-9-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/065 | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/065
+++ b/tests/qemu-iotests/065
@@ -XXX,XX +XXX,XX @@ class TestQMP(TestImageInfoSpecific):
 
 class TestQCow2(TestQemuImgInfo):
     '''Testing a qcow2 version 2 image'''
-    img_options = 'compat=0.10'
+    img_options = 'compat=0.10,compression_type=zlib'
     json_compare = { 'compat': '0.10', 'refcount-bits': 16,
                      'compression-type': 'zlib' }
     human_compare = [ 'compat: 0.10', 'compression type: zlib',
@@ -XXX,XX +XXX,XX @@ class TestQCow2(TestQemuImgInfo):
 
 class TestQCow3NotLazy(TestQemuImgInfo):
     '''Testing a qcow2 version 3 image with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=off'
+    img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zstd'
     json_compare = { 'compat': '1.1', 'lazy-refcounts': False,
                      'refcount-bits': 16, 'corrupt': False,
-                     'compression-type': 'zlib', 'extended-l2': False }
-    human_compare = [ 'compat: 1.1', 'compression type: zlib',
+                     'compression-type': 'zstd', 'extended-l2': False }
+    human_compare = [ 'compat: 1.1', 'compression type: zstd',
                       'lazy refcounts: false', 'refcount bits: 16',
                       'corrupt: false', 'extended l2: false' ]
 
 class TestQCow3Lazy(TestQemuImgInfo):
     '''Testing a qcow2 version 3 image with lazy refcounts enabled'''
-    img_options = 'compat=1.1,lazy_refcounts=on'
+    img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zlib'
     json_compare = { 'compat': '1.1', 'lazy-refcounts': True,
                      'refcount-bits': 16, 'corrupt': False,
                      'compression-type': 'zlib', 'extended-l2': False }
@@ -XXX,XX +XXX,XX @@ class TestQCow3Lazy(TestQemuImgInfo):
 class TestQCow3NotLazyQMP(TestQMP):
     '''Testing a qcow2 version 3 image with lazy refcounts disabled, opening
        with lazy refcounts enabled'''
-    img_options = 'compat=1.1,lazy_refcounts=off'
+    img_options = 'compat=1.1,lazy_refcounts=off,compression_type=zlib'
     qemu_options = 'lazy-refcounts=on'
     compare = { 'compat': '1.1', 'lazy-refcounts': False,
                 'refcount-bits': 16, 'corrupt': False,
@@ -XXX,XX +XXX,XX @@ class TestQCow3NotLazyQMP(TestQMP):
 class TestQCow3LazyQMP(TestQMP):
     '''Testing a qcow2 version 3 image with lazy refcounts enabled, opening
        with lazy refcounts disabled'''
-    img_options = 'compat=1.1,lazy_refcounts=on'
+    img_options = 'compat=1.1,lazy_refcounts=on,compression_type=zstd'
     qemu_options = 'lazy-refcounts=off'
     compare = { 'compat': '1.1', 'lazy-refcounts': True,
                 'refcount-bits': 16, 'corrupt': False,
-                'compression-type': 'zlib', 'extended-l2': False }
+                'compression-type': 'zstd', 'extended-l2': False }
 
 TestImageInfoSpecific = None
 TestQemuImgInfo = None
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The only "feature" of this "Formatting ..." line is that we have to
update it every time we add new option. Let's drop it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-10-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/149.out    | 21 ---------------------
 tests/qemu-iotests/237.out    |  3 ---
 tests/qemu-iotests/255.out    |  4 ----
 tests/qemu-iotests/274.out    | 29 -----------------------------
 tests/qemu-iotests/280.out    |  1 -
 tests/qemu-iotests/296.out    | 10 +++-------
 tests/qemu-iotests/iotests.py | 10 ++++++++--
 7 files changed, 11 insertions(+), 67 deletions(-)

diff --git a/tests/qemu-iotests/149.out b/tests/qemu-iotests/149.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/149.out
+++ b/tests/qemu-iotests/149.out
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha1.img
 # ================= qemu-img aes-256-xts-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha1.img qiotest-145-aes-256-xts-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-twofish-256-xts-plain64-sha1.img
 # ================= qemu-img twofish-256-xts-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-256-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-twofish-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-256-xts-plain64-sha1.img qiotest-145-twofish-256-xts-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-256-xts-plain64-sha1.img
 # ================= qemu-img serpent-256-xts-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-256-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-serpent-256-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-256-xts-plain64-sha1.img qiotest-145-serpent-256-xts-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img
 # ================= qemu-img cast5-128-cbc-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=cast5-128,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=cast5-128 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-cast5-128-cbc-plain64-sha1.img qiotest-145-cast5-128-cbc-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain-sha1.img
 # ================= qemu-img aes-256-cbc-plain-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-plain-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain-sha1.img qiotest-145-aes-256-cbc-plain-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha1.img
 # ================= qemu-img aes-256-cbc-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha1.img qiotest-145-aes-256-cbc-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img
 # ================= qemu-img aes-256-cbc-essiv-sha256-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-sha256-sha1.img qiotest-145-aes-256-cbc-essiv-sha256-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img
 # ================= qemu-img aes-256-xts-essiv-sha256-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=essiv,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=essiv ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-essiv-sha256-sha1.img qiotest-145-aes-256-xts-essiv-sha256-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img
 # ================= qemu-img aes-128-xts-plain64-sha256-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-128-xts-plain64-sha256-sha1.img qiotest-145-aes-128-xts-plain64-sha256-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img
 # ================= qemu-img aes-192-xts-plain64-sha256-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-192-xts-plain64-sha256-sha1.img qiotest-145-aes-192-xts-plain64-sha256-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-twofish-128-xts-plain64-sha1.img
 # ================= qemu-img twofish-128-xts-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=twofish-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-twofish-128-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-twofish-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=twofish-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-twofish-128-xts-plain64-sha1.img qiotest-145-twofish-128-xts-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-128-xts-plain64-sha1.img
 # ================= qemu-img serpent-128-xts-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-128,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-128-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-serpent-128-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-128 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-128-xts-plain64-sha1.img qiotest-145-serpent-128-xts-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-serpent-192-xts-plain64-sha1.img
 # ================= qemu-img serpent-192-xts-plain64-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=serpent-192,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha1 TEST_DIR/luks-serpent-192-xts-plain64-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-serpent-192-xts-plain64-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=serpent-192 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-serpent-192-xts-plain64-sha1.img qiotest-145-serpent-192-xts-plain64-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha224.img
 # ================= qemu-img aes-256-xts-plain64-sha224 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha224 TEST_DIR/luks-aes-256-xts-plain64-sha224.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha224.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha224 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha224.img qiotest-145-aes-256-xts-plain64-sha224
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha256.img
 # ================= qemu-img aes-256-xts-plain64-sha256 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha256 TEST_DIR/luks-aes-256-xts-plain64-sha256.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha256.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha256 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha256.img qiotest-145-aes-256-xts-plain64-sha256
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha384.img
 # ================= qemu-img aes-256-xts-plain64-sha384 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha384 TEST_DIR/luks-aes-256-xts-plain64-sha384.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha384.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha384 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha384.img qiotest-145-aes-256-xts-plain64-sha384
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-sha512.img
 # ================= qemu-img aes-256-xts-plain64-sha512 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=sha512 TEST_DIR/luks-aes-256-xts-plain64-sha512.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-sha512.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=sha512 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-sha512.img qiotest-145-aes-256-xts-plain64-sha512
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img
 # ================= qemu-img aes-256-xts-plain64-ripemd160 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain64,hash-alg=ripemd160 TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain64 hash-alg=ripemd160 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain64-ripemd160.img qiotest-145-aes-256-xts-plain64-ripemd160
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img
 # ================= qemu-img aes-256-xts-plain-sha1-pwallslots =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=c2xvdDE=,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=xts,ivgen-alg=plain,hash-alg=sha1 TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=xts ivgen-alg=plain hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-xts-plain-sha1-pwallslots.img qiotest-145-aes-256-xts-plain-sha1-pwallslots
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img
 # ================= qemu-img aes-256-cbc-essiv-auto-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=essiv,hash-alg=sha1 TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=essiv hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-essiv-auto-sha1.img qiotest-145-aes-256-cbc-essiv-auto-sha1
@@ -XXX,XX +XXX,XX @@ unlink TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img
 # ================= qemu-img aes-256-cbc-plain64-sha256-sha1 =================
 # Create image
 qemu-img create -f luks --object secret,id=sec0,data=MTIzNDU2,format=base64 -o key-secret=sec0,iter-time=10,cipher-alg=aes-256,cipher-mode=cbc,ivgen-alg=plain64,hash-alg=sha1,ivgen-hash-alg=sha256 TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img 4194304M
-Formatting 'TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img', fmt=luks size=4398046511104 key-secret=sec0 cipher-alg=aes-256 cipher-mode=cbc ivgen-alg=plain64 ivgen-hash-alg=sha256 hash-alg=sha1 iter-time=10
 
 # Open dev
 sudo cryptsetup -q -v luksOpen TEST_DIR/luks-aes-256-cbc-plain64-sha256-sha1.img qiotest-145-aes-256-cbc-plain64-sha256-sha1
diff --git a/tests/qemu-iotests/237.out b/tests/qemu-iotests/237.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/237.out
+++ b/tests/qemu-iotests/237.out
@@ -XXX,XX +XXX,XX @@ Job failed: Cannot find device='this doesn't exist' nor node-name='this doesn't
 
 === Other subformats ===
 
-Formatting 'TEST_DIR/PID-t.vmdk.1', fmt=vmdk size=0 compat6=off hwversion=undefined
 
-Formatting 'TEST_DIR/PID-t.vmdk.2', fmt=vmdk size=0 compat6=off hwversion=undefined
 
-Formatting 'TEST_DIR/PID-t.vmdk.3', fmt=vmdk size=0 compat6=off hwversion=undefined
 
 == Missing extent ==
 
diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/255.out
+++ b/tests/qemu-iotests/255.out
@@ -XXX,XX +XXX,XX @@ Finishing a commit job with background reads
 
 === Create backing chain and start VM ===
 
-Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
 === Start background read requests ===
 
@@ -XXX,XX +XXX,XX @@ Closing the VM while a job is being cancelled
 
 === Create images and start VM ===
 
-Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
 
 wrote 1048576/1048576 bytes at offset 0
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/274.out
+++ b/tests/qemu-iotests/274.out
@@ -XXX,XX +XXX,XX @@
 == Commit tests ==
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing HMP commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing QMP active commit (top -> mid) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing qemu-img commit (top -> base) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing QMP active commit (top -> base) ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 2097152/2097152 bytes at offset 0
 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 1048576/1048576 bytes at offset 1048576
 
 == Resize tests ==
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=6442450944 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1073741824 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 5368709120
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 5368709120
 { "start": 1073741824, "length": 7516192768, "depth": 0, "present": true, "zero": true, "data": false}]
 
 === preallocation=metadata ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=34359738368 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=32212254720 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 33285996544
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 33285996544
 { "start": 34896609280, "length": 536870912, "depth": 0, "present": true, "zero": true, "data": false, "offset": 2685075456}]
 
 === preallocation=falloc ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10485760 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=5242880 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 9437184
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 9437184
 { "start": 5242880, "length": 10485760, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}]
 
 === preallocation=full ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=16777216 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=8388608 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 11534336
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 11534336
 { "start": 8388608, "length": 4194304, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}]
 
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=393216 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=259072 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 259072
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 259072
 { "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}]
 
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=409600 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 344064
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 344064
 { "start": 262144, "length": 262144, "depth": 0, "present": true, "zero": true, "data": false}]
 
 === preallocation=off ===
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=524288 lazy_refcounts=off refcount_bits=16
 
-Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=262144 backing_file=TEST_DIR/PID-base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
 
 wrote 65536/65536 bytes at offset 446464
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/280.out
+++ b/tests/qemu-iotests/280.out
@@ -XXX,XX +XXX,XX @@
-Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 
 === Launch VM ===
 Enabling migration QMP events on VM...
diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/296.out
+++ b/tests/qemu-iotests/296.out
@@ -XXX,XX +XXX,XX @@
-Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
 
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
@@ -XXX,XX +XXX,XX @@ Job failed: Failed to get shared "consistent read" lock
 qemu-img: Failed to get shared "consistent read" lock
 Is another process using the image [TEST_DIR/test.img]?
 
-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-
+.
 Job failed: Block node is read-only
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
@@ -XXX,XX +XXX,XX @@ Job failed: Failed to get shared "consistent read" lock
 {"return": {}}
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-
+.
 {"return": {}}
 {"error": {"class": "GenericError", "desc": "Failed to get \"write\" lock"}}
-.Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10
-
+.
 {"return": {}}
 {"return": {}}
 .
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@ def qemu_tool_popen(args: Sequence[str],
 
 
 def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
-                              connect_stderr: bool = True) -> Tuple[str, int]:
+                              connect_stderr: bool = True,
+                              drop_successful_output: bool = False) \
+        -> Tuple[str, int]:
     """
     Run a tool and return both its output and its exit code
     """
@@ -XXX,XX +XXX,XX @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str],
             cmd = ' '.join(args)
             sys.stderr.write(f'{tool} received signal \
                                {-subp.returncode}: {cmd}\n')
+        if drop_successful_output and subp.returncode == 0:
+            output = ''
         return (output, subp.returncode)
 
 def qemu_img_create_prepare_args(args: List[str]) -> List[str]:
@@ -XXX,XX +XXX,XX @@ def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]:
     """
     Run qemu-img and return both its output and its exit code
     """
+    is_create = bool(args and args[0] == 'create')
     full_args = qemu_img_args + qemu_img_create_prepare_args(list(args))
-    return qemu_tool_pipe_and_status('qemu-img', full_args)
+    return qemu_tool_pipe_and_status('qemu-img', full_args,
+                                     drop_successful_output=is_create)
 
 def qemu_img(*args: str) -> int:
     '''Run qemu-img and return the exit code'''
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We want iotests pass with both the default zlib compression and with
IMGOPTS='compression_type=zstd'.

Actually the only test that is interested in real compression type in
test output is 287 (test for qcow2 compression type) and it's in bash.
So for now we can safely filter out compression type in all qcow2
tests.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-11-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/206.out    | 10 +++++-----
 tests/qemu-iotests/242.out    | 10 +++++-----
 tests/qemu-iotests/274.out    | 10 +++++-----
 tests/qemu-iotests/iotests.py |  2 ++
 4 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/206.out
+++ b/tests/qemu-iotests/206.out
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 32 MiB (33554432 bytes)
 cluster_size: 2097152
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: true
     refcount bits: 1
     corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_IMG.base
 backing file format: IMGFMT
 Format specific information:
     compat: 0.10
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     refcount bits: 16
 
 === Successful image creation (encrypted) ===
@@ -XXX,XX +XXX,XX @@ encrypted: yes
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     encrypt:
diff --git a/tests/qemu-iotests/242.out b/tests/qemu-iotests/242.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/242.out
+++ b/tests/qemu-iotests/242.out
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     bitmaps:
         [0]:
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     bitmaps:
         [0]:
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     bitmaps:
         [0]:
@@ -XXX,XX +XXX,XX @@ virtual size: 1 MiB (1048576 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     bitmaps:
         [0]:
diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/274.out
+++ b/tests/qemu-iotests/274.out
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
 backing file format: IMGFMT
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
 backing file format: IMGFMT
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
 backing file format: IMGFMT
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 2 MiB (2097152 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ backing file: TEST_DIR/PID-base
 backing file format: IMGFMT
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@ def filter_img_info(output, filename):
                       'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX',
                       line)
         line = re.sub('cid: [0-9]+', 'cid: XXXXXXXXXX', line)
+        line = re.sub('(compression type: )(zlib|zstd)', r'\1COMPRESSION_TYPE',
+                      line)
         lines.append(line)
     return '\n'.join(lines)
 
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Instead of qemu_img_log("info", ..) use generic helper img_info_log().

img_info_log() has smarter logic. For example it use filter_img_info()
to filter output, which in turns filter a compression type. So it will
help us in future when we implement a possibility to use zstd
compression by default (with help of some runtime config file or maybe
build option). For now to test you should recompile qemu with a small
addition into block/qcow2.c before
"if (qcow2_opts->has_compression_type":

if (!qcow2_opts->has_compression_type && version >= 3) {
        qcow2_opts->has_compression_type = true;
        qcow2_opts->compression_type = QCOW2_COMPRESSION_TYPE_ZSTD;
    }

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-12-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/302     | 4 +++-
 tests/qemu-iotests/302.out | 7 +++----
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/302 b/tests/qemu-iotests/302
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/302
+++ b/tests/qemu-iotests/302
@@ -XXX,XX +XXX,XX @@ from iotests import (
     qemu_img_measure,
     qemu_io,
     qemu_nbd_popen,
+    img_info_log,
 )
 
 iotests.script_initialize(supported_fmts=["qcow2"])
@@ -XXX,XX +XXX,XX @@ with tarfile.open(tar_file, "w") as tar:
             tar_file):
 
         iotests.log("=== Target image info ===")
+        # Not img_info_log as it enforces imgfmt, but now we print info on raw
         qemu_img_log("info", nbd_uri)
 
         qemu_img(
@@ -XXX,XX +XXX,XX @@ with tarfile.open(tar_file, "w") as tar:
             nbd_uri)
 
         iotests.log("=== Converted image info ===")
-        qemu_img_log("info", nbd_uri)
+        img_info_log(nbd_uri)
 
         iotests.log("=== Converted image check ===")
         qemu_img_log("check", nbd_uri)
diff --git a/tests/qemu-iotests/302.out b/tests/qemu-iotests/302.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/302.out
+++ b/tests/qemu-iotests/302.out
@@ -XXX,XX +XXX,XX @@ virtual size: 448 KiB (458752 bytes)
 disk size: unavailable
 
 === Converted image info ===
-image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock
-file format: qcow2
+image: TEST_IMG
+file format: IMGFMT
 virtual size: 1 GiB (1073741824 bytes)
-disk size: unavailable
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

If image doesn't have any compressed cluster we can easily switch to
zlib compression, which may allow to downgrade the image.

That's mostly needed to support IMGOPTS='compression_type=zstd' in some
iotests which do qcow2 downgrade.

While being here also fix checkpatch complain against '#' in printf
formatting.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-13-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 block/qcow2.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 56 insertions(+), 2 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static int qcow2_load_vmstate(BlockDriverState *bs, QEMUIOVector *qiov,
     return bs->drv->bdrv_co_preadv_part(bs, offset, qiov->size, qiov, 0, 0);
 }
 
+static int qcow2_has_compressed_clusters(BlockDriverState *bs)
+{
+    int64_t offset = 0;
+    int64_t bytes = bdrv_getlength(bs);
+
+    if (bytes < 0) {
+        return bytes;
+    }
+
+    while (bytes != 0) {
+        int ret;
+        QCow2SubclusterType type;
+        unsigned int cur_bytes = MIN(INT_MAX, bytes);
+        uint64_t host_offset;
+
+        ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset,
+                                    &type);
+        if (ret < 0) {
+            return ret;
+        }
+
+        if (type == QCOW2_SUBCLUSTER_COMPRESSED) {
+            return 1;
+        }
+
+        offset += cur_bytes;
+        bytes -= cur_bytes;
+    }
+
+    return 0;
+}
+
 /*
  * Downgrades an image's version. To achieve this, any incompatible features
  * have to be removed.
@@ -XXX,XX +XXX,XX @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
      * the first place; if that happens nonetheless, returning -ENOTSUP is the
      * best thing to do anyway */
 
-    if (s->incompatible_features) {
+    if (s->incompatible_features & ~QCOW2_INCOMPAT_COMPRESSION) {
         error_setg(errp, "Cannot downgrade an image with incompatible features "
-                   "%#" PRIx64 " set", s->incompatible_features);
+                   "0x%" PRIx64 " set",
+                   s->incompatible_features & ~QCOW2_INCOMPAT_COMPRESSION);
         return -ENOTSUP;
     }
 
@@ -XXX,XX +XXX,XX @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
         return ret;
     }
 
+    if (s->incompatible_features & QCOW2_INCOMPAT_COMPRESSION) {
+        ret = qcow2_has_compressed_clusters(bs);
+        if (ret < 0) {
+            error_setg(errp, "Failed to check block status");
+            return -EINVAL;
+        }
+        if (ret) {
+            error_setg(errp, "Cannot downgrade an image with zstd compression "
+                       "type and existing compressed clusters");
+            return -ENOTSUP;
+        }
+        /*
+         * No compressed clusters for now, so just chose default zlib
+         * compression.
+         */
+        s->incompatible_features &= ~QCOW2_INCOMPAT_COMPRESSION;
+        s->compression_type = QCOW2_COMPRESSION_TYPE_ZLIB;
+    }
+
+    assert(s->incompatible_features == 0);
+
     s->qcow_version = target_version;
     ret = qcow2_update_header(bs);
     if (ret < 0) {
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We are going to add filtering in _qcow2_dump_header and want all tests
use it.

The patch is generated by commands:
  cd tests/qemu-iotests
  sed -ie 's/$PYTHON qcow2.py "$TEST_IMG" dump-header$$\| $/_qcow2_dump_header\1/' ??? tests/*

(the difficulty is to avoid converting dump-header-exts)

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-15-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/031 |  6 +++---
 tests/qemu-iotests/036 |  6 +++---
 tests/qemu-iotests/039 | 20 ++++++++++----------
 tests/qemu-iotests/060 | 20 ++++++++++----------
 tests/qemu-iotests/061 | 36 ++++++++++++++++++------------------
 tests/qemu-iotests/137 |  2 +-
 tests/qemu-iotests/287 |  8 ++++----
 7 files changed, 49 insertions(+), 49 deletions(-)

diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -XXX,XX +XXX,XX @@ for compat in "compat=0.10" "compat=1.1"; do
     echo
     _make_test_img -o $compat 64M
     $PYTHON qcow2.py "$TEST_IMG" add-header-ext 0x12345678 "This is a test header extension"
-    $PYTHON qcow2.py "$TEST_IMG" dump-header
+    _qcow2_dump_header
     _check_test_img
 
     echo
     echo === Rewrite header with no backing file ===
     echo
     $QEMU_IMG rebase -u -b "" "$TEST_IMG"
-    $PYTHON qcow2.py "$TEST_IMG" dump-header
+    _qcow2_dump_header
     _check_test_img
 
     echo
     echo === Add a backing file and format ===
     echo
     $QEMU_IMG rebase -u -b "/some/backing/file/path" -F host_device "$TEST_IMG"
-    $PYTHON qcow2.py "$TEST_IMG" dump-header
+    _qcow2_dump_header
 done
 
 # success, all done
diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/036
+++ b/tests/qemu-iotests/036
@@ -XXX,XX +XXX,XX @@ $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 63
 
 # Without feature table
 $PYTHON qcow2.py "$TEST_IMG" del-header-ext 0x6803f857
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
+_qcow2_dump_header | grep features
 $PYTHON qcow2.py "$TEST_IMG" dump-header-exts
 _img_info
 
@@ -XXX,XX +XXX,XX @@ echo === Create image with unknown autoclear feature bit ===
 echo
 _make_test_img 64M
 $PYTHON qcow2.py "$TEST_IMG" set-feature-bit autoclear 63
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
+_qcow2_dump_header | grep features
 $PYTHON qcow2.py "$TEST_IMG" dump-header-exts
 
 echo
@@ -XXX,XX +XXX,XX @@ echo === Repair image ===
 echo
 _check_test_img -r all
 
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features
+_qcow2_dump_header | grep features
 $PYTHON qcow2.py "$TEST_IMG" dump-header-exts
 
 # success, all done
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" $size
 $QEMU_IO -c "write -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io
 
 # The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 _check_test_img
 
 echo
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x5a 0 512" \
     | _filter_qemu_io
 
 # The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 _check_test_img
 
 echo
@@ -XXX,XX +XXX,XX @@ echo "== Read-only access must still work =="
 $QEMU_IO -r -c "read -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io
 
 # The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 echo
 echo "== Repairing the image file must succeed =="
@@ -XXX,XX +XXX,XX @@ echo "== Repairing the image file must succeed =="
 _check_test_img -r all
 
 # The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 echo
 echo "== Data should still be accessible after repair =="
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x5a 0 512" \
     | _filter_qemu_io
 
 # The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 $QEMU_IO -c "write 0 512" "$TEST_IMG" | _filter_qemu_io
 
 # The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 echo
 echo "== Creating an image file with lazy_refcounts=off =="
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x5a 0 512" \
     | _filter_qemu_io
 
 # The dirty bit must not be set since lazy_refcounts=off
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 _check_test_img
 
 echo
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write 0 512" "$TEST_IMG" | _filter_qemu_io
 $QEMU_IMG commit "$TEST_IMG"
 
 # The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 $PYTHON qcow2.py "$TEST_IMG".base dump-header | grep incompatible_features
 
 _check_test_img
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "reopen -o lazy-refcounts=on" \
     | _filter_qemu_io
 
 # The dirty bit must be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 _check_test_img
 
 _make_test_img -o "compat=1.1,lazy_refcounts=on" $size
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "reopen -o lazy-refcounts=off" \
     | _filter_qemu_io
 
 # The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 _check_test_img
 
 
diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/060
+++ b/tests/qemu-iotests/060
@@ -XXX,XX +XXX,XX @@ poke_file "$TEST_IMG" "$l1_offset" "\x80\x00\x00\x00\x00\x03\x00\x00"
 _check_test_img
 
 # The corrupt bit should not be set anyway
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 # Try to write something, thereby forcing the corrupt bit to be set
 $QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
 
 # The corrupt bit must now be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 # This information should be available through qemu-img info
 _img_info --format-specific
@@ -XXX,XX +XXX,XX @@ poke_file "$TEST_IMG" "$(($rb_offset+8))" "\x00\x01"
 # Redirect new data cluster onto refcount block
 poke_file "$TEST_IMG" "$l2_offset" "\x80\x00\x00\x00\x00\x02\x00\x00"
 _check_test_img
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 $QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 # Try to fix it
 _check_test_img -r all
 
 # The corrupt bit should be cleared
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 # Look if it's really really fixed
 $QEMU_IO -c "$OPEN_RW" -c "write -P 0x2a 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 echo
 echo "=== Testing cluster data reference into inactive L2 table ==="
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "$OPEN_RW" -c "write -P 2 0 512" | _filter_qemu_io
 poke_file "$TEST_IMG" "$l2_offset_after_snapshot" \
                       "\x80\x00\x00\x00\x00\x04\x00\x00"
 _check_test_img
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 $QEMU_IO -c "$OPEN_RW" -c "write -P 3 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 _check_test_img -r all
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 $QEMU_IO -c "$OPEN_RW" -c "write -P 4 0 512" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 # Check data
 $QEMU_IO -c "$OPEN_RO" -c "read -P 4 0 512" | _filter_qemu_io
diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/061
+++ b/tests/qemu-iotests/061
@@ -XXX,XX +XXX,XX @@ echo "=== Testing version downgrade with zero expansion ==="
 echo
 _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
 $QEMU_IO -c "write -z 0 128k" "$TEST_IMG" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IO -c "read -P 0 0 128k" "$TEST_IMG" | _filter_qemu_io
 _check_test_img
 
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
 $QEMU_IO -c "write -z 0 128k" "$TEST_IMG" | _filter_qemu_io
 $QEMU_IO -c "write -z 32M 128k" "$TEST_IMG" | _filter_qemu_io
 $QEMU_IO -c map "$TEST_IMG" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IMG amend -o "compat=0.10" --image-opts \
           driver=qcow2,file.filename=$TEST_IMG,l2-cache-entry-size=4096
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IO -c "read -P 0 0 128k" "$TEST_IMG" | _filter_qemu_io
 $QEMU_IO -c "read -P 0 32M 128k" "$TEST_IMG" | _filter_qemu_io
 $QEMU_IO -c map "$TEST_IMG" | _filter_qemu_io
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
 _NO_VALGRIND \
 $QEMU_IO -c "write -P 0x2a 0 128k" -c flush \
          -c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IO -c "read -P 0x2a 0 128k" "$TEST_IMG" | _filter_qemu_io
 _check_test_img
 
@@ -XXX,XX +XXX,XX @@ echo
 _make_test_img -o "compat=1.1" 64M
 $PYTHON qcow2.py "$TEST_IMG" set-feature-bit compatible 42
 $PYTHON qcow2.py "$TEST_IMG" set-feature-bit autoclear 42
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 _check_test_img
 
 echo
@@ -XXX,XX +XXX,XX @@ echo "=== Testing version upgrade and resize ==="
 echo
 _make_test_img -o "compat=0.10" 64M
 $QEMU_IO -c "write -P 0x2a 42M 64k" "$TEST_IMG" | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IMG amend -o "compat=1.1,lazy_refcounts=on,size=128M" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IO -c "read -P 0x2a 42M 64k" "$TEST_IMG" | _filter_qemu_io
 _check_test_img
 
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "write -P 0x2a 24M 64k" "$TEST_IMG" | _filter_qemu_io
 $QEMU_IMG snapshot -c foo "$TEST_IMG"
 $QEMU_IMG resize "$TEST_IMG" 64M &&
     echo "unexpected pass"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^$version\|size\|nb_snap$'
+_qcow2_dump_header | grep '^$version\|size\|nb_snap$'
 
 $QEMU_IMG amend -o "compat=1.1,size=128M" "$TEST_IMG" ||
     echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^$version\|size\|nb_snap$'
+_qcow2_dump_header | grep '^$version\|size\|nb_snap$'
 
 $QEMU_IMG snapshot -c bar "$TEST_IMG"
 $QEMU_IMG resize --shrink "$TEST_IMG" 64M ||
     echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^$version\|size\|nb_snap$'
+_qcow2_dump_header | grep '^$version\|size\|nb_snap$'
 
 $QEMU_IMG amend -o "compat=0.10,size=32M" "$TEST_IMG" &&
     echo "unexpected pass"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^$version\|size\|nb_snap$'
+_qcow2_dump_header | grep '^$version\|size\|nb_snap$'
 
 $QEMU_IMG snapshot -a bar "$TEST_IMG" ||
     echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^$version\|size\|nb_snap$'
+_qcow2_dump_header | grep '^$version\|size\|nb_snap$'
 
 $QEMU_IMG snapshot -d bar "$TEST_IMG"
 $QEMU_IMG amend -o "compat=0.10,size=32M" "$TEST_IMG" ||
     echo "unexpected fail"
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep '^$version\|size\|nb_snap$'
+_qcow2_dump_header | grep '^$version\|size\|nb_snap$'
 
 _check_test_img
 
@@ -XXX,XX +XXX,XX @@ _make_test_img -o "compat=1.1,lazy_refcounts=on" 64M
 _NO_VALGRIND \
 $QEMU_IO -c "write -P 0x2a 0 128k" -c flush \
          -c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 | _filter_qemu_io
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IMG amend -o "lazy_refcounts=off" "$TEST_IMG"
-$PYTHON qcow2.py "$TEST_IMG" dump-header
+_qcow2_dump_header
 $QEMU_IO -c "read -P 0x2a 0 128k" "$TEST_IMG" | _filter_qemu_io
 _check_test_img
 
diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/137
+++ b/tests/qemu-iotests/137
@@ -XXX,XX +XXX,XX @@ $QEMU_IO \
 
 # The dirty bit must not be set
 # (Filter the external data file bit)
-if $PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features \
+if _qcow2_dump_header | grep incompatible_features \
     | grep -q '\<0\>'
 then
     echo 'ERROR: Dirty bit set'
diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/287
+++ b/tests/qemu-iotests/287
@@ -XXX,XX +XXX,XX @@ echo
 echo "=== Testing compression type incompatible bit setting for zlib ==="
 echo
 _make_test_img -o compression_type=zlib 64M
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 echo
 echo "=== Testing compression type incompatible bit setting for zstd ==="
 echo
 _make_test_img -o compression_type=zstd 64M
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 echo
 echo "=== Testing zlib with incompatible bit set ==="
@@ -XXX,XX +XXX,XX @@ echo
 _make_test_img -o compression_type=zlib 64M
 $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 3
 # to make sure the bit was actually set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
     echo "Error: The image opened successfully. The image must not be opened."
@@ -XXX,XX +XXX,XX @@ echo
 _make_test_img -o compression_type=zstd 64M
 $PYTHON qcow2.py "$TEST_IMG" set-header incompatible_features 0
 # to make sure the bit was actually unset
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_qcow2_dump_header | grep incompatible_features
 
 if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
     echo "Error: The image opened successfully. The image must not be opened."
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We want iotests pass with both the default zlib compression and with
IMGOPTS='compression_type=zstd'.

Actually the only test that is interested in real compression type in
test output is 287 (test for qcow2 compression type), so implement
specific option for it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-17-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/060.out       |  2 +-
 tests/qemu-iotests/061.out       | 12 ++++++------
 tests/qemu-iotests/082.out       | 14 +++++++-------
 tests/qemu-iotests/198.out       |  4 ++--
 tests/qemu-iotests/287           |  8 ++++----
 tests/qemu-iotests/common.filter |  8 ++++++++
 tests/qemu-iotests/common.rc     | 14 +++++++++++++-
 7 files changed, 41 insertions(+), 21 deletions(-)

diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/060.out
+++ b/tests/qemu-iotests/060.out
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: true
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/061.out
+++ b/tests/qemu-iotests/061.out
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     data file: TEST_DIR/t.IMGFMT.data
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     data file: foo
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     data file raw: false
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     data file: TEST_DIR/t.IMGFMT.data
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     data file: TEST_DIR/t.IMGFMT.data
@@ -XXX,XX +XXX,XX @@ virtual size: 64 MiB (67108864 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     data file: TEST_DIR/t.IMGFMT.data
diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/082.out
+++ b/tests/qemu-iotests/082.out
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
 cluster_size: 4096
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: true
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
 cluster_size: 8192
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: true
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
 cluster_size: 4096
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: true
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
 cluster_size: 8192
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: true
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 128 MiB (134217728 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: true
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 130 MiB (136314880 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: false
     refcount bits: 16
     corrupt: false
@@ -XXX,XX +XXX,XX @@ virtual size: 132 MiB (138412032 bytes)
 cluster_size: 65536
 Format specific information:
     compat: 1.1
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     lazy refcounts: true
     refcount bits: 16
     corrupt: false
diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/198.out
+++ b/tests/qemu-iotests/198.out
@@ -XXX,XX +XXX,XX @@ image: json:{ /* filtered */ }
 file format: IMGFMT
 virtual size: 16 MiB (16777216 bytes)
 Format specific information:
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     encrypt:
         ivgen alg: plain64
         hash alg: sha256
@@ -XXX,XX +XXX,XX @@ virtual size: 16 MiB (16777216 bytes)
 backing file: TEST_DIR/t.IMGFMT.base
 backing file format: IMGFMT
 Format specific information:
-    compression type: zlib
+    compression type: COMPRESSION_TYPE
     encrypt:
         ivgen alg: plain64
         hash alg: sha256
diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/287
+++ b/tests/qemu-iotests/287
@@ -XXX,XX +XXX,XX @@ echo
 echo "=== Testing compression type incompatible bit setting for zlib ==="
 echo
 _make_test_img -o compression_type=zlib 64M
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
 
 echo
 echo "=== Testing compression type incompatible bit setting for zstd ==="
 echo
 _make_test_img -o compression_type=zstd 64M
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
 
 echo
 echo "=== Testing zlib with incompatible bit set ==="
@@ -XXX,XX +XXX,XX @@ echo
 _make_test_img -o compression_type=zlib 64M
 $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 3
 # to make sure the bit was actually set
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
 
 if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
     echo "Error: The image opened successfully. The image must not be opened."
@@ -XXX,XX +XXX,XX @@ echo
 _make_test_img -o compression_type=zstd 64M
 $PYTHON qcow2.py "$TEST_IMG" set-header incompatible_features 0
 # to make sure the bit was actually unset
-_qcow2_dump_header | grep incompatible_features
+_qcow2_dump_header --no-filter-compression | grep incompatible_features
 
 if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then
     echo "Error: The image opened successfully. The image must not be opened."
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_img_info()
         -e "/block_state_zero: \$on\\|off\$/d" \
         -e "/log_size: [0-9]\\+/d" \
         -e "s/iters: [0-9]\\+/iters: 1024/" \
+        -e 's/$compression type: $$zlib\|zstd$/\1COMPRESSION_TYPE/' \
         -e "s/uuid: [-a-f0-9]\\+/uuid: 00000000-0000-0000-0000-000000000000/" | \
     while IFS='' read -r line; do
         if [[ $format_specific == 1 ]]; then
@@ -XXX,XX +XXX,XX @@ _filter_authz_check_tls()
     $SED -e 's/TLS x509 authz check for .* is denied/TLS x509 authz check for DISTINGUISHED-NAME is denied/'
 }
 
+_filter_qcow2_compression_type_bit()
+{
+    $SED -e 's/$incompatible_features\s\+$\[3$, $\?/\1[/' \
+         -e 's/$incompatible_features.*$, 3\]/\1]/' \
+         -e 's/$incompatible_features.*$, 3$,.*$/\1\2/'
+}
+
 # make sure this script returns success
 true
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -XXX,XX +XXX,XX @@ _img_info()
             -e "s#$TEST_DIR#TEST_DIR#g" \
             -e "s#$SOCK_DIR/fuse-#TEST_DIR/#g" \
             -e "s#$IMGFMT#IMGFMT#g" \
+            -e 's/$compression type: $$zlib\|zstd$/\1COMPRESSION_TYPE/' \
             -e "/^disk size:/ D" \
             -e "/actual-size/ D" | \
         while IFS='' read -r line; do
@@ -XXX,XX +XXX,XX @@ _require_one_device_of()
 
 _qcow2_dump_header()
 {
+    if [[ "$1" == "--no-filter-compression" ]]; then
+        local filter_compression=0
+        shift
+    else
+        local filter_compression=1
+    fi
+
     img="$1"
     if [ -z "$img" ]; then
         img="$TEST_IMG"
     fi
 
-    $PYTHON qcow2.py "$img" dump-header
+    if [[ $filter_compression == 0 ]]; then
+        $PYTHON qcow2.py "$img" dump-header
+    else
+        $PYTHON qcow2.py "$img" dump-header | _filter_qcow2_compression_type_bit
+    fi
 }
 
 # make sure this script returns success
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Don't touch other incompatible bits, like compression-type. This makes
the test pass with IMGOPTS='compression_type=zstd'.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-18-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/060 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/060
+++ b/tests/qemu-iotests/060
@@ -XXX,XX +XXX,XX @@ _make_test_img 64M
 # Let the refblock appear unaligned
 poke_file "$TEST_IMG" "$rt_offset"        "\x00\x00\x00\x00\xff\xff\x2a\x00"
 # Mark the image dirty, thus forcing an automatic check when opening it
-poke_file "$TEST_IMG" 72 "\x00\x00\x00\x00\x00\x00\x00\x01"
+$PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 0
 # Open the image (qemu should refuse to do so)
 $QEMU_IO -c close "$TEST_IMG" 2>&1 | _filter_testdir | _filter_imgfmt
 
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The test-case "Corrupted size field in compressed cluster descriptor"
heavily depends on zlib compression type. So, make it explicit. This
way test passes with IMGOPTS='compression_type=zstd'.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20211223160144.1097696-19-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/214 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/214
+++ b/tests/qemu-iotests/214
@@ -XXX,XX +XXX,XX @@ echo
 # The L2 entries of the two compressed clusters are located at
 # 0x800000 and 0x800008, their original values are 0x4008000000a00000
 # and 0x4008000000a00802 (5 sectors for compressed data each).
-_make_test_img 8M -o cluster_size=2M
+_make_test_img 8M -o cluster_size=2M,compression_type=zlib
 $QEMU_IO -c "write -c -P 0x11 0 2M" -c "write -c -P 0x11 2M 2M" "$TEST_IMG" \
          2>&1 | _filter_qemu_io | _filter_testdir
 
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

compression_type can't be used if we want to create image with
compat=0.10. So, skip these tests, not many of them.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211223160144.1097696-20-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/031 | 5 +++--
 tests/qemu-iotests/051 | 5 +++--
 tests/qemu-iotests/061 | 6 +++++-
 tests/qemu-iotests/112 | 3 ++-
 tests/qemu-iotests/290 | 2 +-
 5 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/031
+++ b/tests/qemu-iotests/031
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file fuse
 # We want to test compat=0.10, which does not support external data
-# files or refcount widths other than 16
-_unsupported_imgopts data_file 'refcount_bits=$[^1]\|.\([^6]\|$$\)'
+# files or refcount widths other than 16 or compression type
+_unsupported_imgopts data_file compression_type \
+    'refcount_bits=$[^1]\|.\([^6]\|$$\)'
 
 CLUSTER_SIZE=65536
 
diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/051
+++ b/tests/qemu-iotests/051
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow2
 _supported_proto file
 # A compat=0.10 image is created in this test which does not support anything
 # other than refcount_bits=16;
-# it also will not support an external data file
-_unsupported_imgopts 'refcount_bits=$[^1]\|.\([^6]\|$$\)' data_file
+# it also will not support an external data file and compression type
+_unsupported_imgopts 'refcount_bits=$[^1]\|.\([^6]\|$$\)' data_file \
+    compression_type
 _require_drivers nbd
 
 if [ "$QEMU_DEFAULT_MACHINE" = "pc" ]; then
diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/061
+++ b/tests/qemu-iotests/061
@@ -XXX,XX +XXX,XX @@ _supported_os Linux
 # not work with it;
 # we have explicit tests for various cluster sizes, the remaining tests
 # require the default 64k cluster
-_unsupported_imgopts 'refcount_bits=$[^1]\|.\([^6]\|$$\)' data_file cluster_size
+# we don't have explicit tests for zstd qcow2 compression type, as zstd may be
+# not compiled in. And we can't create compat images with comression type
+# extension
+_unsupported_imgopts 'refcount_bits=$[^1]\|.\([^6]\|$$\)' data_file \
+    cluster_size compression_type
 
 echo
 echo "=== Testing version downgrade with zero expansion ==="
diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -XXX,XX +XXX,XX @@ _supported_proto file fuse
 # This test will set refcount_bits on its own which would conflict with the
 # manual setting; compat will be overridden as well;
 # and external data files do not work well with our refcount testing
-_unsupported_imgopts refcount_bits 'compat=0.10' data_file
+# also, compression type is not supported with compat=0.10 used in test
+_unsupported_imgopts refcount_bits 'compat=0.10' data_file compression_type
 
 print_refcount_bits()
 {
diff --git a/tests/qemu-iotests/290 b/tests/qemu-iotests/290
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/290
+++ b/tests/qemu-iotests/290
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file fuse
 _supported_os Linux
-_unsupported_imgopts 'compat=0.10' refcount_bits data_file
+_unsupported_imgopts 'compat=0.10' refcount_bits data_file compression_type
 
 echo
 echo "### Test 'qemu-io -c discard' on a QCOW2 image without a backing file"
-- 
2.34.1

After migration, the permissions the guest device wants to impose on its
BlockBackend are stored in blk->perm and blk->shared_perm.  In
blk_root_activate(), we take our permissions, but keep all shared
permissions open by calling `blk_set_perm(blk->perm, BLK_PERM_ALL)`.

Only afterwards (immediately or later, depending on the runstate) do we
restrict the shared permissions by calling
`blk_set_perm(blk->perm, blk->shared_perm)`.  Unfortunately, our first
call with shared_perm=BLK_PERM_ALL has overwritten blk->shared_perm to
be BLK_PERM_ALL, so this is a no-op and the set of shared permissions is
not restricted.

Fix this bug by saving the set of shared permissions before invoking
blk_set_perm() with BLK_PERM_ALL and restoring it afterwards.

Fixes: 5f7772c4d0cf32f4e779fcd5a69ae4dae24aeebf
       ("block-backend: Defer shared_perm tightening migration
       completion")
Reported-by: Peng Liang <liangpeng10@huawei.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20211125135317.186576-2-hreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Peng Liang <liangpeng10@huawei.com>
---
 block/block-backend.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/block/block-backend.c b/block/block-backend.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -XXX,XX +XXX,XX @@ static void blk_root_activate(BdrvChild *child, Error **errp)
 {
     BlockBackend *blk = child->opaque;
     Error *local_err = NULL;
+    uint64_t saved_shared_perm;
 
     if (!blk->disable_perm) {
         return;
@@ -XXX,XX +XXX,XX @@ static void blk_root_activate(BdrvChild *child, Error **errp)
 
     blk->disable_perm = false;
 
+    /*
+     * blk->shared_perm contains the permissions we want to share once
+     * migration is really completely done.  For now, we need to share
+     * all; but we also need to retain blk->shared_perm, which is
+     * overwritten by a successful blk_set_perm() call.  Save it and
+     * restore it below.
+     */
+    saved_shared_perm = blk->shared_perm;
+
     blk_set_perm(blk, blk->perm, BLK_PERM_ALL, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
         blk->disable_perm = true;
         return;
     }
+    blk->shared_perm = saved_shared_perm;
 
     if (runstate_check(RUN_STATE_INMIGRATE)) {
         /* Activation can happen when migration process is still active, for
-- 
2.34.1

This test checks that a raw image in use by a virtio-blk device does not
share the WRITE permission both before and after migration.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 .../qemu-iotests/tests/migration-permissions  | 101 ++++++++++++++++++
 .../tests/migration-permissions.out           |   5 +
 2 files changed, 106 insertions(+)
 create mode 100755 tests/qemu-iotests/tests/migration-permissions
 create mode 100644 tests/qemu-iotests/tests/migration-permissions.out

diff --git a/tests/qemu-iotests/tests/migration-permissions b/tests/qemu-iotests/tests/migration-permissions
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/tests/migration-permissions
@@ -XXX,XX +XXX,XX @@
+#!/usr/bin/env python3
+# group: migration
+#
+# Copyright (C) 2021 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+from iotests import imgfmt, qemu_img_create, qemu_io
+
+
+test_img = os.path.join(iotests.test_dir, 'test.img')
+mig_sock = os.path.join(iotests.sock_dir, 'mig.sock')
+
+
+class TestMigrationPermissions(iotests.QMPTestCase):
+    def setUp(self):
+        qemu_img_create('-f', imgfmt, test_img, '1M')
+
+        # Set up two VMs (source and destination) accessing the same raw
+        # image file with a virtio-blk device; prepare the destination for
+        # migration with .add_incoming() and enable migration events
+        vms = [None, None]
+        for i in range(2):
+            vms[i] = iotests.VM(path_suffix=f'{i}')
+            vms[i].add_blockdev(f'file,node-name=prot,filename={test_img}')
+            vms[i].add_blockdev(f'{imgfmt},node-name=fmt,file=prot')
+            vms[i].add_device('virtio-blk,drive=fmt')
+
+            if i == 1:
+                vms[i].add_incoming(f'unix:{mig_sock}')
+
+            vms[i].launch()
+
+            result = vms[i].qmp('migrate-set-capabilities',
+                                capabilities=[
+                                    {'capability': 'events', 'state': True}
+                                ])
+            self.assert_qmp(result, 'return', {})
+
+        self.vm_s = vms[0]
+        self.vm_d = vms[1]
+
+    def tearDown(self):
+        self.vm_s.shutdown()
+        self.vm_d.shutdown()
+        try:
+            os.remove(mig_sock)
+        except FileNotFoundError:
+            pass
+        os.remove(test_img)
+
+    # Migrate an image in use by a virtio-blk device to another VM and
+    # verify that the WRITE permission is unshared both before and after
+    # migration
+    def test_post_migration_permissions(self):
+        # Try to access the image R/W, which should fail because virtio-blk
+        # has not been configured with share-rw=on
+        log = qemu_io('-f', imgfmt, '-c', 'quit', test_img)
+        if not log.strip():
+            print('ERROR (pre-migration): qemu-io should not be able to '
+                  'access this image, but it reported no error')
+        else:
+            # This is the expected output
+            assert 'Is another process using the image' in log
+
+        # Now migrate the VM
+        self.vm_s.qmp('migrate', uri=f'unix:{mig_sock}')
+        assert self.vm_s.wait_migration(None)
+        assert self.vm_d.wait_migration(None)
+
+        # Try the same qemu-io access again, verifying that the WRITE
+        # permission remains unshared
+        log = qemu_io('-f', imgfmt, '-c', 'quit', test_img)
+        if not log.strip():
+            print('ERROR (post-migration): qemu-io should not be able to '
+                  'access this image, but it reported no error')
+        else:
+            # This is the expected output
+            assert 'Is another process using the image' in log
+
+
+if __name__ == '__main__':
+    # Only works with raw images because we are testing the
+    # BlockBackend permissions; image format drivers may additionally
+    # unshare permissions and thus tamper with the result
+    iotests.main(supported_fmts=['raw'],
+                 supported_protocols=['file'])
diff --git a/tests/qemu-iotests/tests/migration-permissions.out b/tests/qemu-iotests/tests/migration-permissions.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/tests/migration-permissions.out
@@ -XXX,XX +XXX,XX @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
-- 
2.34.1

From: Emanuele Giuseppe Esposito <eesposit@redhat.com>

The comment "disk I/O throttling" doesn't make any sense at all
any more. It was added in commit 0563e191516 to describe
bdrv_io_limits_enable()/disable(), which were removed in commit
97148076, so the comment is just a forgotten leftover.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20220131125615.74612-1-eesposit@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/block.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ typedef unsigned int BdrvChildRole;
 char *bdrv_perm_names(uint64_t perm);
 uint64_t bdrv_qapi_perm_to_blk_perm(BlockPermission qapi_perm);
 
-/* disk I/O throttling */
 void bdrv_init(void);
 void bdrv_init_with_whitelist(void);
 bool bdrv_uses_whitelist(void);
-- 
2.34.1