Series comparison

-[Qemu-devel] [PULL 00/17] Block patches
+[Qemu-devel] [PULL 00/15] Block patches
-The following changes since commit 3fbd3405d2b0604ea530fc7a1828f19da1e95ff9:
+The following changes since commit 23919ddfd56135cad3cb468a8f54d5a595f024f4:
-  Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-08-17' into staging (2019-08-19 14:14:09 +0100)
+  Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20190827' into staging (2019-08-27 15:52:36 +0100)
 are available in the Git repository at:
-  https://github.com/XanClic/qemu.git tags/pull-block-2019-08-19
+  https://github.com/XanClic/qemu.git tags/pull-block-2019-08-27
-for you to fetch changes up to fa27c478102a6b5d1c6b02c005607ad9404b915f:
+for you to fetch changes up to bb043c056cffcc2f3ce88bfdaf2e76e455c09e2c:
-  doc: Preallocation does not require writing zeroes (2019-08-19 17:13:26 +0200)
+  iotests: Unify cache mode quoting (2019-08-27 19:48:44 +0200)
 ----------------------------------------------------------------
 Block patches:
-- preallocation=falloc/full support for LUKS
+- qemu-io now accepts a file to read a write pattern from
-- Various minor fixes
+- Ensure that raw files have their first block allocated so we can probe
   the O_DIRECT alignment if necessary
 - Various fixes
 ----------------------------------------------------------------
-Max Reitz (16):
+Denis Plotnikov (1):
-  qemu-img: Fix bdrv_has_zero_init() use in convert
+  qemu-io: add pattern file for write command
   mirror: Fix bdrv_has_zero_init() use
   block: Add bdrv_has_zero_init_truncate()
   block: Implement .bdrv_has_zero_init_truncate()
   block: Use bdrv_has_zero_init_truncate()
   qcow2: Fix .bdrv_has_zero_init()
   vdi: Fix .bdrv_has_zero_init()
   vhdx: Fix .bdrv_has_zero_init()
   iotests: Convert to preallocated encrypted qcow2
   iotests: Test convert -n to pre-filled image
   iotests: Full mirror to existing non-zero image
   vdi: Make block_status recurse for fixed images
   vmdk: Make block_status recurse for flat extents
   vpc: Do not return RAW from block_status
   iotests: Fix 141 when run with qed
   doc: Preallocation does not require writing zeroes
-Maxim Levitsky (1):
+Max Reitz (7):
-  LUKS: support preallocation
+  iotests: Fix _filter_img_create()
   vmdk: Use bdrv_dirname() for relative extent paths
   iotests: Keep testing broken relative extent paths
   vmdk: Reject invalid compressed writes
   iotests: Disable broken streamOptimized tests
   iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
   iotests: Disable 126 for flat vmdk subformats
- qapi/block-core.json             | 15 +++++---
+Nir Soffer (3):
- include/block/block.h            |  1 +
+  block: posix: Always allocate the first block
- include/block/block_int.h        |  9 +++++
+  iotests: Test allocate_first_block() with O_DIRECT
- block.c                          | 21 +++++++++++
+  iotests: Unify cache mode quoting
- block/crypto.c                   | 30 ++++++++++++++--
- block/file-posix.c               |  1 +
+Stefan Hajnoczi (1):
- block/file-win32.c               |  1 +
+  file-posix: fix request_alignment typo
- block/gluster.c                  |  4 +++
- block/mirror.c                   | 11 ++++--
+Thomas Huth (2):
- block/nfs.c                      |  1 +
+  iotests: Check for enabled drivers before testing them
- block/parallels.c                |  2 +-
+  tests/check-block: Skip iotests when sanitizers are enabled
- block/qcow2.c                    | 30 +++++++++++++++-
- block/qed.c                      |  1 +
+Vladimir Sementsov-Ogievskiy (1):
- block/raw-format.c               |  6 ++++
+  block: fix permission update in bdrv_replace_node
- block/rbd.c                      |  1 +
- block/sheepdog.c                 |  1 +
+ block.c                                       |  5 +-
- block/ssh.c                      |  1 +
+ block/file-posix.c                            | 53 +++++++++-
- block/vdi.c                      | 16 +++++++--
+ block/vmdk.c                                  | 64 ++++++++----
- block/vhdx.c                     | 28 +++++++++++++--
+ qemu-io-cmds.c                                | 99 +++++++++++++++++--
- block/vmdk.c                     |  3 ++
+ tests/check-block.sh                          |  5 +
- block/vpc.c                      |  2 +-
+ tests/qemu-iotests/002                        |  1 +
- blockdev.c                       | 16 +++++++--
+ tests/qemu-iotests/003                        |  1 +
- qemu-img.c                       | 11 ++++--
+ tests/qemu-iotests/005                        |  3 +-
- tests/test-block-iothread.c      |  2 +-
+ tests/qemu-iotests/009                        |  1 +
- docs/qemu-block-drivers.texi     |  4 +--
+ tests/qemu-iotests/010                        |  1 +
- qemu-img.texi                    |  4 +--
+ tests/qemu-iotests/011                        |  1 +
- tests/qemu-iotests/041           | 62 +++++++++++++++++++++++++++++---
+ tests/qemu-iotests/017                        |  3 +-
- tests/qemu-iotests/041.out       |  4 +--
+ tests/qemu-iotests/018                        |  3 +-
- tests/qemu-iotests/122           | 17 +++++++++
+ tests/qemu-iotests/019                        |  3 +-
- tests/qemu-iotests/122.out       |  8 +++++
+ tests/qemu-iotests/020                        |  3 +-
- tests/qemu-iotests/141           |  9 +++--
+ tests/qemu-iotests/026                        |  4 +-
- tests/qemu-iotests/141.out       |  5 ---
+ tests/qemu-iotests/027                        |  1 +
- tests/qemu-iotests/188           | 20 ++++++++++-
+ tests/qemu-iotests/032                        |  1 +
- tests/qemu-iotests/188.out       |  4 +++
+ tests/qemu-iotests/033                        |  1 +
- tests/qemu-iotests/common.filter |  5 +++
+ tests/qemu-iotests/034                        |  3 +-
-files changed, 313 insertions(+), 43 deletions(-)
+ tests/qemu-iotests/037                        |  3 +-
  tests/qemu-iotests/039                        |  4 +-
  tests/qemu-iotests/052                        |  2 +-
  tests/qemu-iotests/059                        | 34 ++++++-
  tests/qemu-iotests/059.out                    | 26 +++--
  tests/qemu-iotests/063                        |  3 +-
  tests/qemu-iotests/071                        |  1 +
  tests/qemu-iotests/072                        |  1 +
  tests/qemu-iotests/081                        |  4 +-
  tests/qemu-iotests/091                        |  4 +-
  tests/qemu-iotests/099                        |  1 +
  tests/qemu-iotests/105                        |  3 +-
  tests/qemu-iotests/110                        |  3 +-
  tests/qemu-iotests/120                        |  1 +
  tests/qemu-iotests/126                        |  2 +
  tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
  tests/qemu-iotests/150.out.raw                | 12 +++
  tests/qemu-iotests/162                        |  4 +-
  tests/qemu-iotests/175                        | 47 +++++++--
  tests/qemu-iotests/175.out                    | 16 ++-
  tests/qemu-iotests/178.out.qcow2              |  4 +-
  tests/qemu-iotests/184                        |  1 +
  tests/qemu-iotests/186                        |  1 +
  tests/qemu-iotests/197                        |  1 +
  tests/qemu-iotests/215                        |  1 +
  tests/qemu-iotests/221.out                    | 12 ++-
  tests/qemu-iotests/251                        |  1 +
  tests/qemu-iotests/253.out                    | 12 ++-
  tests/qemu-iotests/common.filter              |  4 +-
  tests/qemu-iotests/common.rc                  | 14 +++
 files changed, 391 insertions(+), 87 deletions(-)
  rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
  create mode 100644 tests/qemu-iotests/150.out.raw
 --
 .21.0

-[Qemu-devel] [PULL 01/17] LUKS: support preallocation
+[Qemu-devel] [PULL 01/15] qemu-io: add pattern file for write command
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Denis Plotnikov <dplotnikov@virtuozzo.com>
-preallocation=off and preallocation=metadata
+The patch allows to provide a pattern file for write
-both allocate luks header only, and preallocation=falloc/full
+command. There was no similar ability before.
 is passed to underlying file.
-Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951
+Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
+Message-id: 20190820164616.4072-1-dplotnikov@virtuozzo.com
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Reviewed-by: Eric Blake <eblake@redhat.com>
-Message-id: 20190716161901.1430-1-mlevitsk@redhat.com
+[mreitz: Keep optstring in alphabetical order]
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qapi/block-core.json |  6 +++++-
+ qemu-io-cmds.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++---
- block/crypto.c       | 30 +++++++++++++++++++++++++++---
+file changed, 93 insertions(+), 6 deletions(-)
 files changed, 32 insertions(+), 4 deletions(-)
-diff --git a/qapi/block-core.json b/qapi/block-core.json
+diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
 index XXXXXXX..XXXXXXX 100644
---- a/qapi/block-core.json
+--- a/qemu-io-cmds.c
-+++ b/qapi/block-core.json
++++ b/qemu-io-cmds.c
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static void qemu_io_free(void *p)
- #
+     qemu_vfree(p);
  # @file             Node to create the image format on
  # @size             Size of the virtual disk in bytes
 +# @preallocation    Preallocation mode for the new image
 +#                   (since: 4.2)
 +#                   (default: off; allowed values: off, metadata, falloc, full)
  #
  # Since: 2.12
  ##
  { 'struct': 'BlockdevCreateOptionsLUKS',
    'base': 'QCryptoBlockCreateOptionsLUKS',
    'data': { 'file':             'BlockdevRef',
 -            'size':             'size' } }
 +            'size':             'size',
 +            '*preallocation':   'PreallocMode' } }
  ##
  # @BlockdevCreateOptionsNfs:
 diff --git a/block/crypto.c b/block/crypto.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/crypto.c
 +++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
  struct BlockCryptoCreateData {
      BlockBackend *blk;
      uint64_t size;
 +    PreallocMode prealloc;
  };
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_init_func(QCryptoBlock *block,
       * available to the guest, so we must take account of that
       * which will be used by the crypto header
       */
 -    return blk_truncate(data->blk, data->size + headerlen, PREALLOC_MODE_OFF,
 +    return blk_truncate(data->blk, data->size + headerlen, data->prealloc,
                          errp);
  }
-@@ -XXX,XX +XXX,XX @@ static int block_crypto_open_generic(QCryptoBlockFormat format,
++/*
- static int block_crypto_co_create_generic(BlockDriverState *bs,
++ * qemu_io_alloc_from_file()
-                                           int64_t size,
++ *
-                                           QCryptoBlockCreateOptions *opts,
++ * Allocates the buffer and populates it with the content of the given file
-+                                          PreallocMode prealloc,
++ * up to @len bytes. If the file length is less than @len, then the buffer
-                                           Error **errp)
++ * is populated with the file content cyclically.
- {
++ *
-     int ret;
++ * @blk - the block backend where the buffer content is going to be written to
-@@ -XXX,XX +XXX,XX @@ static int block_crypto_co_create_generic(BlockDriverState *bs,
++ * @len - the buffer length
-         goto cleanup;
++ * @file_name - the file to read the content from
-     }
++ *
++ * Returns: the buffer pointer on success
-+    if (prealloc == PREALLOC_MODE_METADATA) {
++ *          NULL on error
-+        prealloc = PREALLOC_MODE_OFF;
++ */
 +static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
 +                                     const char *file_name)
 +{
 +    char *buf, *buf_origin;
 +    FILE *f = fopen(file_name, "r");
 +    int pattern_len;
 +
 +    if (!f) {
 +        perror(file_name);
 +        return NULL;
 +    }
 +
-     data = (struct BlockCryptoCreateData) {
++    if (qemuio_misalign) {
-         .blk = blk,
++        len += MISALIGN_OFFSET;
          .size = size,
 +        .prealloc = prealloc,
      };
      crypto = qcrypto_block_create(opts, NULL,
@@ -XXX,XX +XXX,XX @@ block_crypto_co_create_luks(BlockdevCreateOptions *create_options, Error **errp)
      BlockdevCreateOptionsLUKS *luks_opts;
      BlockDriverState *bs = NULL;
      QCryptoBlockCreateOptions create_opts;
 +    PreallocMode preallocation = PREALLOC_MODE_OFF;
      int ret;
      assert(create_options->driver == BLOCKDEV_DRIVER_LUKS);
@@ -XXX,XX +XXX,XX @@ block_crypto_co_create_luks(BlockdevCreateOptions *create_options, Error **errp)
          .u.luks = *qapi_BlockdevCreateOptionsLUKS_base(luks_opts),
      };
 +    if (luks_opts->has_preallocation) {
 +        preallocation = luks_opts->preallocation;
 +    }
 +
-     ret = block_crypto_co_create_generic(bs, luks_opts->size, &create_opts,
++    buf_origin = buf = blk_blockalign(blk, len);
--                                         errp);
++
-+                                         preallocation, errp);
++    if (qemuio_misalign) {
-     if (ret < 0) {
++        buf_origin += MISALIGN_OFFSET;
-         goto fail;
++        buf += MISALIGN_OFFSET;
-     }
++        len -= MISALIGN_OFFSET;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_crypto_co_create_opts_luks(const char *filename,
      QCryptoBlockCreateOptions *create_opts = NULL;
      BlockDriverState *bs = NULL;
      QDict *cryptoopts;
 +    PreallocMode prealloc;
 +    char *buf = NULL;
      int64_t size;
      int ret;
 +    Error *local_err = NULL;
      /* Parse options */
      size = qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0);
 +    buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
 +    prealloc = qapi_enum_parse(&PreallocMode_lookup, buf,
 +                               PREALLOC_MODE_OFF, &local_err);
 +    g_free(buf);
 +    if (local_err) {
 +        error_propagate(errp, local_err);
 +        return -EINVAL;
 +    }
 +
-     cryptoopts = qemu_opts_to_qdict_filtered(opts, NULL,
++    pattern_len = fread(buf_origin, 1, len, f);
-                                              &block_crypto_create_opts_luks,
++
-                                              true);
++    if (ferror(f)) {
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_crypto_co_create_opts_luks(const char *filename,
++        perror(file_name);
 +        goto error;
 +    }
 +
 +    if (pattern_len == 0) {
 +        fprintf(stderr, "%s: file is empty\n", file_name);
 +        goto error;
 +    }
 +
 +    fclose(f);
 +
 +    if (len > pattern_len) {
 +        len -= pattern_len;
 +        buf += pattern_len;
 +
 +        while (len > 0) {
 +            size_t len_to_copy = MIN(pattern_len, len);
 +
 +            memcpy(buf, buf_origin, len_to_copy);
 +
 +            len -= len_to_copy;
 +            buf += len_to_copy;
 +        }
 +    }
 +
 +    return buf_origin;
 +
 +error:
 +    qemu_io_free(buf_origin);
 +    return NULL;
 +}
 +
  static void dump_buffer(const void *buffer, int64_t offset, int64_t len)
  {
      uint64_t i;
@@ -XXX,XX +XXX,XX @@ static void write_help(void)
  " -n, -- with -z, don't allow slow fallback\n"
  " -p, -- ignored for backwards compatibility\n"
  " -P, -- use different pattern to fill file\n"
 +" -s, -- use a pattern file to fill the write buffer\n"
  " -C, -- report statistics in a machine parsable format\n"
  " -q, -- quiet mode, do not show I/O statistics\n"
  " -u, -- with -z, allow unmapping\n"
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t write_cmd = {
      .perm       = BLK_PERM_WRITE,
      .argmin     = 2,
      .argmax     = -1,
 -    .args       = "[-bcCfnquz] [-P pattern] off len",
 +    .args       = "[-bcCfnquz] [-P pattern | -s source_file] off len",
      .oneline    = "writes a number of bytes at a specified offset",
      .help       = write_help,
  };
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
  {
      struct timespec t1, t2;
      bool Cflag = false, qflag = false, bflag = false;
 -    bool Pflag = false, zflag = false, cflag = false;
 +    bool Pflag = false, zflag = false, cflag = false, sflag = false;
      int flags = 0;
      int c, cnt, ret;
      char *buf = NULL;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
      /* Some compilers get confused and warn if this is not initialized.  */
      int64_t total = 0;
      int pattern = 0xcd;
 +    const char *file_name = NULL;
 -    while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) {
 +    while ((c = getopt(argc, argv, "bcCfnpP:qs:uz")) != -1) {
          switch (c) {
          case 'b':
              bflag = true;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
          case 'q':
              qflag = true;
              break;
 +        case 's':
 +            sflag = true;
 +            file_name = optarg;
 +            break;
          case 'u':
              flags |= BDRV_REQ_MAY_UNMAP;
              break;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
          return -EINVAL;
      }
-     /* Create format layer */
+-    if (zflag && Pflag) {
--    ret = block_crypto_co_create_generic(bs, size, create_opts, errp);
+-        printf("-z and -P cannot be specified at the same time\n");
-+    ret = block_crypto_co_create_generic(bs, size, create_opts, prealloc, errp);
++    if (zflag + Pflag + sflag > 1) {
-     if (ret < 0) {
++        printf("Only one of -z, -P, and -s "
-         goto fail;
++               "can be specified at the same time\n");
          return -EINVAL;
      }
+@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
+     }
+     if (!zflag) {
+-        buf = qemu_io_alloc(blk, count, pattern);
++        if (sflag) {
++            buf = qemu_io_alloc_from_file(blk, count, file_name);
++            if (!buf) {
++                return -EINVAL;
++            }
++        } else {
++            buf = qemu_io_alloc(blk, count, pattern);
++        }
+     }
+     clock_gettime(CLOCK_MONOTONIC, &t1);
 --
 .21.0

-[Qemu-devel] [PULL 02/17] qemu-img: Fix bdrv_has_zero_init() use in convert
+Deleted patch
-bdrv_has_zero_init() only has meaning for newly created images or image
-areas.  If qemu-img convert did not create the image itself, it cannot
-rely on bdrv_has_zero_init()'s result to carry any meaning.
-Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190724171239.8764-2-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
-Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- qemu-img.c | 11 ++++++++---
-file changed, 8 insertions(+), 3 deletions(-)
-diff --git a/qemu-img.c b/qemu-img.c
-index XXXXXXX..XXXXXXX 100644
---- a/qemu-img.c
-+++ b/qemu-img.c
-@@ -XXX,XX +XXX,XX @@ typedef struct ImgConvertState {
-     bool has_zero_init;
-     bool compressed;
-     bool unallocated_blocks_are_zero;
-+    bool target_is_new;
-     bool target_has_backing;
-     int64_t target_backing_sectors; /* negative if unknown */
-     bool wr_in_order;
-@@ -XXX,XX +XXX,XX @@ static int convert_do_copy(ImgConvertState *s)
-     int64_t sector_num = 0;
-     /* Check whether we have zero initialisation or can get it efficiently */
--    s->has_zero_init = s->min_sparse && !s->target_has_backing
--                     ? bdrv_has_zero_init(blk_bs(s->target))
--                     : false;
-+    if (s->target_is_new && s->min_sparse && !s->target_has_backing) {
-+        s->has_zero_init = bdrv_has_zero_init(blk_bs(s->target));
-+    } else {
-+        s->has_zero_init = false;
-+    }
-     if (!s->has_zero_init && !s->target_has_backing &&
-         bdrv_can_write_zeroes_with_unmap(blk_bs(s->target)))
-@@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv)
-         }
-     }
-+    s.target_is_new = !skip_create;
-+
-     flags = s.min_sparse ? (BDRV_O_RDWR | BDRV_O_UNMAP) : BDRV_O_RDWR;
-     ret = bdrv_parse_cache_mode(cache, &flags, &writethrough);
-     if (ret < 0) {
---
-.21.0

-[Qemu-devel] [PULL 04/17] block: Add bdrv_has_zero_init_truncate()
+[Qemu-devel] [PULL 02/15] block: fix permission update in bdrv_replace_node
-No .bdrv_has_zero_init() implementation returns 1 if growing the file
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 would add non-zero areas (at least with PREALLOC_MODE_OFF), so using it
 in lieu of this new function was always safe.
-But on the other hand, it is possible that growing an image that is not
+It's wrong to OR shared permissions. It may lead to crash on further
-zero-initialized would still add a zero-initialized area, like when
+permission updates.
-using nonpreallocating truncation on a preallocated image.  For callers
+Also, no needs to consider previously calculated permissions, as at
-that care only about truncation, not about creation with potential
+this point we already bind all new parents and bdrv_get_cumulative_perm
-preallocation, this new function is useful.
+result is enough. So fix the bug by just set permissions by
 bdrv_get_cumulative_perm result.
-Alternatively, we could have added a PreallocMode parameter to
+Bug was introduced in long ago 234ac1a9025, in 2.9.
 bdrv_has_zero_init().  But the only user would have been qemu-img
 convert, which does not have a plain PreallocMode value right now -- it
 would have to parse the creation option to obtain it.  Therefore, the
 simpler solution is to let bdrv_has_zero_init() inquire the
 preallocation status and add the new bdrv_has_zero_init_truncate() that
 presupposes PREALLOC_MODE_OFF.
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-id: 20190724171239.8764-4-mreitz@redhat.com
+Message-id: 20190824100740.61635-1-vsementsov@virtuozzo.com
 Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
 Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- include/block/block.h     |  1 +
+ block.c | 5 ++---
- include/block/block_int.h |  7 +++++++
+file changed, 2 insertions(+), 3 deletions(-)
  block.c                   | 21 +++++++++++++++++++++
 files changed, 29 insertions(+)
-diff --git a/include/block/block.h b/include/block/block.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/block/block.h
-+++ b/include/block/block.h
-@@ -XXX,XX +XXX,XX @@ int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
- int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
- int bdrv_has_zero_init_1(BlockDriverState *bs);
- int bdrv_has_zero_init(BlockDriverState *bs);
-+int bdrv_has_zero_init_truncate(BlockDriverState *bs);
- bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs);
- bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
- int bdrv_block_status(BlockDriverState *bs, int64_t offset,
-diff --git a/include/block/block_int.h b/include/block/block_int.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/block/block_int.h
-+++ b/include/block/block_int.h
-@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
-     /*
-      * Returns 1 if newly created images are guaranteed to contain only
-      * zeros, 0 otherwise.
-+     * Must return 0 if .bdrv_has_zero_init_truncate() returns 0.
-      */
-     int (*bdrv_has_zero_init)(BlockDriverState *bs);
-+    /*
-+     * Returns 1 if new areas added by growing the image with
-+     * PREALLOC_MODE_OFF contain only zeros, 0 otherwise.
-+     */
-+    int (*bdrv_has_zero_init_truncate)(BlockDriverState *bs);
-+
-     /* Remove fd handlers, timers, and other event loop callbacks so the event
-      * loop is no longer in use.  Called with no in-flight requests and in
-      * depth-first traversal order with parents before child nodes.
 diff --git a/block.c b/block.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block.c
 +++ b/block.c
-@@ -XXX,XX +XXX,XX @@ int bdrv_has_zero_init(BlockDriverState *bs)
+@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
      return 0;
  }
 +int bdrv_has_zero_init_truncate(BlockDriverState *bs)
 +{
 +    if (!bs->drv) {
 +        return 0;
 +    }
 +
 +    if (bs->backing) {
 +        /* Depends on the backing image length, but better safe than sorry */
 +        return 0;
 +    }
 +    if (bs->drv->bdrv_has_zero_init_truncate) {
 +        return bs->drv->bdrv_has_zero_init_truncate(bs);
 +    }
 +    if (bs->file && bs->drv->is_filter) {
 +        return bdrv_has_zero_init_truncate(bs->file->bs);
 +    }
 +
 +    /* safe default */
 +    return 0;
 +}
 +
  bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs)
  {
-     BlockDriverInfo bdi;
+     BdrvChild *c, *next;
      GSList *list = NULL, *p;
 -    uint64_t old_perm, old_shared;
      uint64_t perm = 0, shared = BLK_PERM_ALL;
      int ret;
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
          bdrv_unref(from);
      }
 -    bdrv_get_cumulative_perm(to, &old_perm, &old_shared);
 -    bdrv_set_perm(to, old_perm | perm, old_shared | shared);
 +    bdrv_get_cumulative_perm(to, &perm, &shared);
 +    bdrv_set_perm(to, perm, shared);
  out:
      g_slist_free(list);
 --
 .21.0

-[Qemu-devel] [PULL 05/17] block: Implement .bdrv_has_zero_init_truncate()
+[Qemu-devel] [PULL 03/15] block: posix: Always allocate the first block
-We need to implement .bdrv_has_zero_init_truncate() for every block
+From: Nir Soffer <nirsof@gmail.com>
-driver that supports truncation and has a .bdrv_has_zero_init()
-implementation.
+When creating an image with preallocation "off" or "falloc", the first
+block of the image is typically not allocated. When using Gluster
-Implement it the same way each driver implements .bdrv_has_zero_init().
+storage backed by XFS filesystem, reading this block using direct I/O
-This is at least not any more unsafe than what we had before.
+succeeds regardless of request length, fooling alignment detection.
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+In this case we fallback to a safe value (4096) instead of the optimal
-Message-id: 20190724171239.8764-5-mreitz@redhat.com
+value (512), which may lead to unneeded data copying when aligning
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+requests.  Allocating the first block avoids the fallback.
-Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
 Since we allocate the first block even with preallocation=off, we no
 longer create images with zero disk size:
     $ ./qemu-img create -f raw test.raw 1g
     Formatting 'test.raw', fmt=raw size=1073741824
     $ ls -lhs test.raw
 .0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw
 And converting the image requires additional cluster:
     $ ./qemu-img measure -f raw -O qcow2 test.raw
     required size: 458752
     fully allocated size: 1074135040
 When using format like vmdk with multiple files per image, we allocate
 one block per file:
     $ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
     Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat
     $ ls -lhs test*.vmdk
 .0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
 .0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
 .0K -rw-r--r--. 1 nsoffer nsoffer  353 Aug 27 03:23 test.vmdk
 I did quick performance test for copying disks with qemu-img convert to
 new raw target image to Gluster storage with sector size of 512 bytes:
     for i in $(seq 10); do
         rm -f dst.raw
         sleep 10
         time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
     done
 Here is a table comparing the total time spent:
 Type    Before(s)   After(s)    Diff(%)
 ---------------------------------------
 real      530.028    469.123      -11.4
 user       17.204     10.768      -37.4
 sys        17.881      7.011      -60.7
 We can see very clear improvement in CPU usage.
 Signed-off-by: Nir Soffer <nsoffer@redhat.com>
 Message-id: 20190827010528.8818-2-nsoffer@redhat.com
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/file-posix.c | 1 +
+ block/file-posix.c                            | 51 +++++++++++++++++++
- block/file-win32.c | 1 +
+ tests/qemu-iotests/059.out                    |  2 +-
- block/gluster.c    | 4 ++++
+ tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
- block/nfs.c        | 1 +
+ tests/qemu-iotests/150.out.raw                | 12 +++++
- block/qcow2.c      | 1 +
+ tests/qemu-iotests/175                        | 19 ++++---
- block/qed.c        | 1 +
+ tests/qemu-iotests/175.out                    |  8 +--
- block/raw-format.c | 6 ++++++
+ tests/qemu-iotests/178.out.qcow2              |  4 +-
- block/rbd.c        | 1 +
+ tests/qemu-iotests/221.out                    | 12 +++--
- block/sheepdog.c   | 1 +
+ tests/qemu-iotests/253.out                    | 12 +++--
- block/ssh.c        | 1 +
+files changed, 99 insertions(+), 21 deletions(-)
-files changed, 18 insertions(+)
+ rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
  create mode 100644 tests/qemu-iotests/150.out.raw
 diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/file-posix.c
 +++ b/block/file-posix.c
-@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_file = {
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_discard(void *opaque)
-     .bdrv_co_create = raw_co_create,
+     return ret;
      .bdrv_co_create_opts = raw_co_create_opts,
      .bdrv_has_zero_init = bdrv_has_zero_init_1,
 +    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
      .bdrv_co_block_status = raw_co_block_status,
      .bdrv_co_invalidate_cache = raw_co_invalidate_cache,
      .bdrv_co_pwrite_zeroes = raw_co_pwrite_zeroes,
 diff --git a/block/file-win32.c b/block/file-win32.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/file-win32.c
 +++ b/block/file-win32.c
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_file = {
      .bdrv_close         = raw_close,
      .bdrv_co_create_opts = raw_co_create_opts,
      .bdrv_has_zero_init = bdrv_has_zero_init_1,
 +    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
      .bdrv_aio_preadv    = raw_aio_preadv,
      .bdrv_aio_pwritev   = raw_aio_pwritev,
 diff --git a/block/gluster.c b/block/gluster.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/gluster.c
 +++ b/block/gluster.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster = {
      .bdrv_co_writev               = qemu_gluster_co_writev,
      .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
      .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
 +    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
  #ifdef CONFIG_GLUSTERFS_DISCARD
      .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
  #endif
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster_tcp = {
      .bdrv_co_writev               = qemu_gluster_co_writev,
      .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
      .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
 +    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
  #ifdef CONFIG_GLUSTERFS_DISCARD
      .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
  #endif
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster_unix = {
      .bdrv_co_writev               = qemu_gluster_co_writev,
      .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
      .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
 +    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
  #ifdef CONFIG_GLUSTERFS_DISCARD
      .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
  #endif
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster_rdma = {
      .bdrv_co_writev               = qemu_gluster_co_writev,
      .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
      .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
 +    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
  #ifdef CONFIG_GLUSTERFS_DISCARD
      .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
  #endif
 diff --git a/block/nfs.c b/block/nfs.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/nfs.c
 +++ b/block/nfs.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_nfs = {
      .create_opts                    = &nfs_create_opts,
      .bdrv_has_zero_init             = nfs_has_zero_init,
 +    .bdrv_has_zero_init_truncate    = nfs_has_zero_init,
      .bdrv_get_allocated_file_size   = nfs_get_allocated_file_size,
      .bdrv_co_truncate               = nfs_file_co_truncate,
 diff --git a/block/qcow2.c b/block/qcow2.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qcow2.c
 +++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
      .bdrv_co_create_opts  = qcow2_co_create_opts,
      .bdrv_co_create       = qcow2_co_create,
      .bdrv_has_zero_init = bdrv_has_zero_init_1,
 +    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
      .bdrv_co_block_status = qcow2_co_block_status,
      .bdrv_co_preadv         = qcow2_co_preadv,
 diff --git a/block/qed.c b/block/qed.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qed.c
 +++ b/block/qed.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_qed = {
      .bdrv_co_create           = bdrv_qed_co_create,
      .bdrv_co_create_opts      = bdrv_qed_co_create_opts,
      .bdrv_has_zero_init       = bdrv_has_zero_init_1,
 +    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
      .bdrv_co_block_status     = bdrv_qed_co_block_status,
      .bdrv_co_readv            = bdrv_qed_co_readv,
      .bdrv_co_writev           = bdrv_qed_co_writev,
 diff --git a/block/raw-format.c b/block/raw-format.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/raw-format.c
 +++ b/block/raw-format.c
@@ -XXX,XX +XXX,XX @@ static int raw_has_zero_init(BlockDriverState *bs)
      return bdrv_has_zero_init(bs->file->bs);
  }
-+static int raw_has_zero_init_truncate(BlockDriverState *bs)
++/*
 + * Help alignment probing by allocating the first block.
 + *
 + * When reading with direct I/O from unallocated area on Gluster backed by XFS,
 + * reading succeeds regardless of request length. In this case we fallback to
 + * safe alignment which is not optimal. Allocating the first block avoids this
 + * fallback.
 + *
 + * fd may be opened with O_DIRECT, but we don't know the buffer alignment or
 + * request alignment, so we use safe values.
 + *
 + * Returns: 0 on success, -errno on failure. Since this is an optimization,
 + * caller may ignore failures.
 + */
 +static int allocate_first_block(int fd, size_t max_size)
 +{
-+    return bdrv_has_zero_init_truncate(bs->file->bs);
++    size_t write_size = (max_size < MAX_BLOCKSIZE)
 +        ? BDRV_SECTOR_SIZE
 +        : MAX_BLOCKSIZE;
 +    size_t max_align = MAX(MAX_BLOCKSIZE, getpagesize());
 +    void *buf;
 +    ssize_t n;
 +    int ret;
 +
 +    buf = qemu_memalign(max_align, write_size);
 +    memset(buf, 0, write_size);
 +
 +    do {
 +        n = pwrite(fd, buf, write_size, 0);
 +    } while (n == -1 && errno == EINTR);
 +
 +    ret = (n == -1) ? -errno : 0;
 +
 +    qemu_vfree(buf);
 +    return ret;
 +}
 +
- static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
+ static int handle_aiocb_truncate(void *opaque)
                                             Error **errp)
  {
-@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_raw = {
+     RawPosixAIOData *aiocb = opaque;
-     .bdrv_co_ioctl        = &raw_co_ioctl,
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
-     .create_opts          = &raw_create_opts,
+                 /* posix_fallocate() doesn't set errno. */
-     .bdrv_has_zero_init   = &raw_has_zero_init,
+                 error_setg_errno(errp, -result,
-+    .bdrv_has_zero_init_truncate = &raw_has_zero_init_truncate,
+                                  "Could not preallocate new data");
-     .strong_runtime_opts  = raw_strong_runtime_opts,
++            } else if (current_length == 0) {
-     .mutable_opts         = mutable_opts,
++                /*
- };
++                 * posix_fallocate() uses fallocate() if the filesystem
-diff --git a/block/rbd.c b/block/rbd.c
++                 * supports it, or fallback to manually writing zeroes. If
-index XXXXXXX..XXXXXXX 100644
++                 * fallocate() was used, unaligned reads from the fallocated
---- a/block/rbd.c
++                 * area in raw_probe_alignment() will succeed, hence we need to
-+++ b/block/rbd.c
++                 * allocate the first block.
-@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_rbd = {
++                 *
-     .bdrv_co_create         = qemu_rbd_co_create,
++                 * Optimize future alignment probing; ignore failures.
-     .bdrv_co_create_opts    = qemu_rbd_co_create_opts,
++                 */
-     .bdrv_has_zero_init     = bdrv_has_zero_init_1,
++                allocate_first_block(fd, offset);
-+    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
+             }
-     .bdrv_get_info          = qemu_rbd_getinfo,
+         } else {
-     .create_opts            = &qemu_rbd_create_opts,
+             result = 0;
-     .bdrv_getlength         = qemu_rbd_getlength,
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
-diff --git a/block/sheepdog.c b/block/sheepdog.c
+         if (ftruncate(fd, offset) != 0) {
-index XXXXXXX..XXXXXXX 100644
+             result = -errno;
---- a/block/sheepdog.c
+             error_setg_errno(errp, -result, "Could not resize file");
-+++ b/block/sheepdog.c
++        } else if (current_length == 0 && offset > current_length) {
-@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_sheepdog = {
++            /* Optimize future alignment probing; ignore failures. */
-     .bdrv_co_create               = sd_co_create,
++            allocate_first_block(fd, offset);
-     .bdrv_co_create_opts          = sd_co_create_opts,
+         }
-     .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+         return result;
-+    .bdrv_has_zero_init_truncate  = bdrv_has_zero_init_1,
+     default:
-     .bdrv_getlength               = sd_getlength,
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
-     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
+index XXXXXXX..XXXXXXX 100644
-     .bdrv_co_truncate             = sd_co_truncate,
+--- a/tests/qemu-iotests/059.out
-diff --git a/block/ssh.c b/block/ssh.c
++++ b/tests/qemu-iotests/059.out
-index XXXXXXX..XXXXXXX 100644
+@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMax
---- a/block/ssh.c
+ image: TEST_DIR/t.vmdk
-+++ b/block/ssh.c
+ file format: vmdk
-@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_ssh = {
+ virtual size: 0.977 TiB (1073741824000 bytes)
-     .bdrv_co_create_opts          = ssh_co_create_opts,
+-disk size: 16 KiB
-     .bdrv_close                   = ssh_close,
++disk size: 1.97 MiB
-     .bdrv_has_zero_init           = ssh_has_zero_init,
+ Format specific information:
-+    .bdrv_has_zero_init_truncate  = ssh_has_zero_init,
+     cid: XXXXXXXX
-     .bdrv_co_readv                = ssh_co_readv,
+     parent cid: XXXXXXXX
-     .bdrv_co_writev               = ssh_co_writev,
+diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out.qcow2
-     .bdrv_getlength               = ssh_getlength,
+similarity index 100%
 rename from tests/qemu-iotests/150.out
 rename to tests/qemu-iotests/150.out.qcow2
 diff --git a/tests/qemu-iotests/150.out.raw b/tests/qemu-iotests/150.out.raw
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/tests/qemu-iotests/150.out.raw
@@ -XXX,XX +XXX,XX @@
 +QA output created by 150
 +
 +=== Mapping sparse conversion ===
 +
 +Offset          Length          File
 +0               0x1000          TEST_DIR/t.IMGFMT
 +
 +=== Mapping non-sparse conversion ===
 +
 +Offset          Length          File
 +0               0x100000        TEST_DIR/t.IMGFMT
 +*** done
 diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/175
 +++ b/tests/qemu-iotests/175
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  # the file size.  This function hides the resulting difference in the
  # stat -c '%b' output.
  # Parameter 1: Number of blocks an empty file occupies
 -# Parameter 2: Image size in bytes
 +# Parameter 2: Minimal number of blocks in an image
 +# Parameter 3: Image size in bytes
  _filter_blocks()
  {
      extra_blocks=$1
 -    img_size=$2
 +    min_blocks=$2
 +    img_size=$3
 -    sed -e "s/blocks=$extra_blocks\\(\$\\|[^0-9]\\)/nothing allocated/" \
 -        -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/everything allocated/"
 +    sed -e "s/blocks=$min_blocks\\(\$\\|[^0-9]\\)/min allocation/" \
 +        -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/max allocation/"
  }
  # get standard environment, filters and checks
@@ -XXX,XX +XXX,XX @@ size=$((1 * 1024 * 1024))
  touch "$TEST_DIR/empty"
  extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 +# We always write the first byte; check how many blocks this filesystem
 +# allocates to match empty image alloation.
 +printf "\0" > "$TEST_DIR/empty"
 +min_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 +
  echo
  echo "== creating image with default preallocation =="
  _make_test_img $size | _filter_imgfmt
 -stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
 +stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
  for mode in off full falloc; do
      echo
      echo "== creating image with preallocation $mode =="
      IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
 -    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
 +    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
  done
  # success, all done
 diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/175.out
 +++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ QA output created by 175
  == creating image with default preallocation ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 -size=1048576, nothing allocated
 +size=1048576, min allocation
  == creating image with preallocation off ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
 -size=1048576, nothing allocated
 +size=1048576, min allocation
  == creating image with preallocation full ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
 -size=1048576, everything allocated
 +size=1048576, max allocation
  == creating image with preallocation falloc ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
 -size=1048576, everything allocated
 +size=1048576, max allocation
   *** done
 diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/178.out.qcow2
 +++ b/tests/qemu-iotests/178.out.qcow2
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
  == raw input image with data (human) ==
  Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
 -required size: 393216
 +required size: 458752
  fully allocated size: 1074135040
  wrote 512/512 bytes at offset 512
 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
  Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
  {
 -    "required": 393216,
 +    "required": 458752,
      "fully-allocated": 1074135040
  }
  wrote 512/512 bytes at offset 512
 diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/221.out
 +++ b/tests/qemu-iotests/221.out
@@ -XXX,XX +XXX,XX @@ QA output created by 221
  === Check mapping of unaligned raw image ===
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537
 -[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 -[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
  wrote 1/1 bytes at offset 65536
 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 -[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
  { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
  { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 -[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
  { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
  { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
  *** done
 diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/253.out
 +++ b/tests/qemu-iotests/253.out
@@ -XXX,XX +XXX,XX @@ QA output created by 253
  === Check mapping of unaligned raw image ===
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048575
 -[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 -[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
  wrote 65535/65535 bytes at offset 983040
 .999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 -[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
  { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
 -[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
  { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
  *** done
 --
 .21.0

-[Qemu-devel] [PULL 11/17] iotests: Test convert -n to pre-filled image
+[Qemu-devel] [PULL 04/15] iotests: Test allocate_first_block() with O_DIRECT
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+From: Nir Soffer <nirsof@gmail.com>
-Message-id: 20190724171239.8764-11-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+Using block_resize we can test allocate_first_block() with file
 descriptor opened with O_DIRECT, ensuring that it works for any size
 larger than 4096 bytes.
 Testing smaller sizes is tricky as the result depends on the filesystem
 used for testing. For example on NFS any size will work since O_DIRECT
 does not require any alignment.
 Signed-off-by: Nir Soffer <nsoffer@redhat.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Message-id: 20190827010528.8818-3-nsoffer@redhat.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/122     | 17 +++++++++++++++++
+ tests/qemu-iotests/175     | 28 ++++++++++++++++++++++++++++
- tests/qemu-iotests/122.out |  8 ++++++++
+ tests/qemu-iotests/175.out |  8 ++++++++
-files changed, 25 insertions(+)
+files changed, 36 insertions(+)
-diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
+diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
 index XXXXXXX..XXXXXXX 100755
---- a/tests/qemu-iotests/122
+--- a/tests/qemu-iotests/175
-+++ b/tests/qemu-iotests/122
++++ b/tests/qemu-iotests/175
-@@ -XXX,XX +XXX,XX @@ for min_sparse in 4k 8k; do
+@@ -XXX,XX +XXX,XX @@ _filter_blocks()
-     $QEMU_IMG map --output=json "$TEST_IMG".orig | _filter_qemu_img_map
+         -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/max allocation/"
  }
 +# Resize image using block_resize.
 +# Parameter 1: image path
 +# Parameter 2: new size
 +_block_resize()
 +{
 +    local path=$1
 +    local size=$2
 +
 +    $QEMU -qmp stdio -nographic -nodefaults \
 +        -blockdev file,node-name=file,filename=$path,cache.direct=on \
 +        <<EOF
 +{'execute': 'qmp_capabilities'}
 +{'execute': 'block_resize', 'arguments': {'node-name': 'file', 'size': $size}}
 +{'execute': 'quit'}
 +EOF
 +}
 +
  # get standard environment, filters and checks
  . ./common.rc
  . ./common.filter
@@ -XXX,XX +XXX,XX @@ _supported_fmt raw
  _supported_proto file
  _supported_os Linux
 +_default_cache_mode none
 +_supported_cache_modes none directsync
 +
  size=$((1 * 1024 * 1024))
  touch "$TEST_DIR/empty"
@@ -XXX,XX +XXX,XX @@ for mode in off full falloc; do
      stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
  done
-+
++for new_size in 4096 1048576; do
-+echo
++    echo
-+echo '=== -n to a non-zero image ==='
++    echo "== resize empty image with block_resize =="
-+echo
++    _make_test_img 0 | _filter_imgfmt
-+
++    _block_resize $TEST_IMG $new_size >/dev/null
-+# Keep source zero
++    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $new_size
-+_make_test_img 64M
++done
 +
 +# Output is not zero, but has bdrv_has_zero_init() == 1
 +TEST_IMG="$TEST_IMG".orig _make_test_img 64M
 +$QEMU_IO -c "write -P 42 0 64k" "$TEST_IMG".orig | _filter_qemu_io
 +
 +# Convert with -n, which should not assume that the target is zeroed
 +$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
 +
 +$QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
 +
  # success, all done
- echo '*** done'
+ echo "*** done"
  rm -f $seq.full
-diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out
+diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
 index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/122.out
+--- a/tests/qemu-iotests/175.out
-+++ b/tests/qemu-iotests/122.out
++++ b/tests/qemu-iotests/175.out
-@@ -XXX,XX +XXX,XX @@ convert -c -S 8k
+@@ -XXX,XX +XXX,XX @@ size=1048576, max allocation
- { "start": 9216, "length": 8192, "depth": 0, "zero": true, "data": false},
+ == creating image with preallocation falloc ==
- { "start": 17408, "length": 1024, "depth": 0, "zero": false, "data": true},
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
- { "start": 18432, "length": 67090432, "depth": 0, "zero": true, "data": false}]
+ size=1048576, max allocation
 +
-+=== -n to a non-zero image ===
++== resize empty image with block_resize ==
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
 +size=4096, min allocation
 +
-+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
++== resize empty image with block_resize ==
-+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
-+wrote 65536/65536 bytes at offset 0
++size=1048576, min allocation
-+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+  *** done
 +Images are identical.
  *** done
 --
 .21.0

-[Qemu-devel] [PULL 16/17] iotests: Fix 141 when run with qed
+[Qemu-devel] [PULL 05/15] iotests: Fix _filter_img_create()
-f47505ee has changed qcow2 in such a way that the commit job run in
+fe646693acc changed qemu-img create's output so that it no longer prints
-test 141 (and 144[1]) returns before it emits the READY event.  However,
+single quotes around parameter values.  The subformat and adapter_type
-also runs with qed, where the order is still the other way around.
+filters in _filter_img_create() have never been adapted to that change.
 Just filter out the {"return": {}} so the test passes for qed again.
-[1] 144 only runs with qcow2, so it is fine as it is.
+Fixes: fe646693acc13ac48b98435d14149ab04dc597bc
 Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190809185253.17535-1-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
-Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Message-id: 20190815153638.4600-2-mreitz@redhat.com
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/141           | 9 +++++++--
+ tests/qemu-iotests/059.out       | 16 ++++++++--------
- tests/qemu-iotests/141.out       | 5 -----
+ tests/qemu-iotests/common.filter |  4 ++--
- tests/qemu-iotests/common.filter | 5 +++++
+files changed, 10 insertions(+), 10 deletions(-)
 files changed, 12 insertions(+), 7 deletions(-)
-diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/141
 +++ b/tests/qemu-iotests/141
@@ -XXX,XX +XXX,XX @@ test_blockjob()
                }}}" \
          'return'
 +    # If "$2" is an event, we may or may not see it before the
 +    # {"return": {}}.  Therefore, filter the {"return": {}} out both
 +    # here and in the next command.  (Naturally, if we do not see it
 +    # here, we will see it before the next command can be executed,
 +    # so it will appear in the next _send_qemu_cmd's output.)
      _send_qemu_cmd $QEMU_HANDLE \
          "$1" \
          "$2" \
 -        | _filter_img_create
 +        | _filter_img_create | _filter_qmp_empty_return
      # We want this to return an error because the block job is still running
      _send_qemu_cmd $QEMU_HANDLE \
          "{'execute': 'blockdev-del',
            'arguments': {'node-name': 'drv0'}}" \
 -        'error' | _filter_generated_node_ids
 +        'error' | _filter_generated_node_ids | _filter_qmp_empty_return
      _send_qemu_cmd $QEMU_HANDLE \
          "{'execute': 'block-job-cancel',
 diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out
 index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/141.out
+--- a/tests/qemu-iotests/059.out
-+++ b/tests/qemu-iotests/141.out
++++ b/tests/qemu-iotests/059.out
-@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/m.
+@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
- Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT
+ qemu-io: can't open device TEST_DIR/t.vmdk: L1 size too big
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
+ === Testing monolithicFlat creation and opening ===
--{"return": {}}
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
- {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
- {"return": {}}
+ image: TEST_DIR/t.IMGFMT
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}}
+ file format: IMGFMT
-@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.
+ virtual size: 2 GiB (2147483648 bytes)
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}
+ === Testing monolithicFlat with zeroed_grain ===
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}}
+ qemu-img: TEST_DIR/t.IMGFMT: Flat image can't enable zeroed grain
--{"return": {}}
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
- {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: mirror"}}
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
- {"return": {}}
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}}
+ === Testing big twoGbMaxExtentFlat ===
-@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMaxExtentFlat
- {"return": {}}
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
+ image: TEST_DIR/t.vmdk
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
+ file format: vmdk
--{"return": {}}
+ virtual size: 0.977 TiB (1073741824000 bytes)
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}
+@@ -XXX,XX +XXX,XX @@ Format specific information:
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}
+ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Invalid extent line: RW 12582912 VMFS "dummy.IMGFMT" 1
- {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: commit"}}
-@@ -XXX,XX +XXX,XX @@ wrote 1048576/1048576 bytes at offset 0
+ === Testing truncated sparse ===
- {"return": {}}
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400 subformat=monolithicSparse
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
+ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': File truncated, expecting at least 13172736 bytes
--{"return": {}}
- {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
+ === Converting to streamOptimized from image with small cluster size===
- {"return": {}}
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}}
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-@@ -XXX,XX +XXX,XX @@ wrote 1048576/1048576 bytes at offset 0
- {"return": {}}
+ === Testing monolithicFlat with internally generated JSON file name ===
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
--{"return": {}}
+ qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
- {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
- {"return": {}}
+ === Testing version 3 ===
- {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}}
+@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 64931328
 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  === Testing 4TB monolithicFlat creation and IO ===
 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104 subformat=monolithicFlat
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104
  image: TEST_DIR/t.IMGFMT
  file format: IMGFMT
  virtual size: 4 TiB (4398046511104 bytes)
@@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 966367641600
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  === Testing qemu-img map on extents ===
 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=monolithicSparse
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
  wrote 1024/1024 bytes at offset 65024
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  wrote 1024/1024 bytes at offset 2147483136
@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
 0x20000         0x3f0000        TEST_DIR/t.vmdk
 x7fff0000      0x20000         0x410000        TEST_DIR/t.vmdk
 x140000000     0x10000         0x430000        TEST_DIR/t.vmdk
 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=twoGbMaxExtentSparse
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
  wrote 1024/1024 bytes at offset 65024
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  wrote 1024/1024 bytes at offset 2147483136
 diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/common.filter
 +++ b/tests/qemu-iotests/common.filter
-@@ -XXX,XX +XXX,XX @@ _filter_nbd()
+@@ -XXX,XX +XXX,XX @@ _filter_img_create()
-         -e 's#\(foo\|PORT/\?\|.sock\): Failed to .*$#\1#'
+         -e "s# compat6=\\(on\\|off\\)##g" \
- }
+         -e "s# static=\\(on\\|off\\)##g" \
+         -e "s# zeroed_grain=\\(on\\|off\\)##g" \
-+_filter_qmp_empty_return()
+-        -e "s# subformat='[^']*'##g" \
-+{
+-        -e "s# adapter_type='[^']*'##g" \
-+    grep -v '{"return": {}}'
++        -e "s# subformat=[^ ]*##g" \
-+}
++        -e "s# adapter_type=[^ ]*##g" \
-+
+         -e "s# hwversion=[^ ]*##g" \
- # make sure this script returns success
+         -e "s# lazy_refcounts=\\(on\\|off\\)##g" \
- true
+         -e "s# block_size=[0-9]\\+##g" \
 --
 .21.0

-[Qemu-devel] [PULL 17/17] doc: Preallocation does not require writing zeroes
+[Qemu-devel] [PULL 06/15] vmdk: Use bdrv_dirname() for relative extent paths
-When preallocating an encrypted qcow2 image, it just lets the protocol
+This makes iotest 033 pass with e.g. subformat=monolithicFlat.  It also
-driver write data and then does not mark the clusters as zero.
+turns a former error in 059 into success.
 Therefore, reading this image will yield effectively random data.
 As such, we have not fulfilled the promise of always writing zeroes when
 preallocating an image in a while.  It seems that nobody has really
 cared, so change the documentation to conform to qemu's actual behavior.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190711132935.13070-1-mreitz@redhat.com
+Message-id: 20190815153638.4600-3-mreitz@redhat.com
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
 Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
 Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qapi/block-core.json         | 9 +++++----
+ block/vmdk.c               | 54 ++++++++++++++++++++++++--------------
- docs/qemu-block-drivers.texi | 4 ++--
+ tests/qemu-iotests/059     |  7 +++--
- qemu-img.texi                | 4 ++--
+ tests/qemu-iotests/059.out |  4 ++-
-files changed, 9 insertions(+), 8 deletions(-)
+files changed, 42 insertions(+), 23 deletions(-)
-diff --git a/qapi/block-core.json b/qapi/block-core.json
+diff --git a/block/vmdk.c b/block/vmdk.c
 index XXXXXXX..XXXXXXX 100644
---- a/qapi/block-core.json
+--- a/block/vmdk.c
-+++ b/qapi/block-core.json
++++ b/block/vmdk.c
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static const char *next_line(const char *s)
- # @off: no preallocation
+ }
- # @metadata: preallocate only for metadata
- # @falloc: like @full preallocation but allocate disk space by
+ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
--#          posix_fallocate() rather than writing zeros.
+-                              const char *desc_file_path, QDict *options,
--# @full: preallocate all data by writing zeros to device to ensure disk
+-                              Error **errp)
--#        space is really available. @full preallocation also sets up
++                              QDict *options, Error **errp)
--#        metadata correctly.
+ {
-+#          posix_fallocate() rather than writing data.
+     int ret;
-+# @full: preallocate all data by writing it to the device to ensure
+     int matches;
-+#        disk space is really available. This data may or may not be
+@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
-+#        zero, depending on the image format and storage.
+     const char *p, *np;
-+#        @full preallocation also sets up metadata correctly.
+     int64_t sectors = 0;
- #
+     int64_t flat_offset;
- # Since: 2.2
++    char *desc_file_dir = NULL;
- ##
+     char *extent_path;
-diff --git a/docs/qemu-block-drivers.texi b/docs/qemu-block-drivers.texi
+     BdrvChild *extent_file;
      BDRVVmdkState *s = bs->opaque;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
              continue;
          }
 -        if (!path_is_absolute(fname) && !path_has_protocol(fname) &&
 -            !desc_file_path[0])
 -        {
 -            bdrv_refresh_filename(bs->file->bs);
 -            error_setg(errp, "Cannot use relative extent paths with VMDK "
 -                       "descriptor file '%s'", bs->file->bs->filename);
 -            return -EINVAL;
 -        }
 +        if (path_is_absolute(fname)) {
 +            extent_path = g_strdup(fname);
 +        } else {
 +            if (!desc_file_dir) {
 +                desc_file_dir = bdrv_dirname(bs->file->bs, errp);
 +                if (!desc_file_dir) {
 +                    bdrv_refresh_filename(bs->file->bs);
 +                    error_prepend(errp, "Cannot use relative paths with VMDK "
 +                                  "descriptor file '%s': ",
 +                                  bs->file->bs->filename);
 +                    ret = -EINVAL;
 +                    goto out;
 +                }
 +            }
 -        extent_path = path_combine(desc_file_path, fname);
 +            extent_path = g_strconcat(desc_file_dir, fname, NULL);
 +        }
          ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
          assert(ret < 32);
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
          g_free(extent_path);
          if (local_err) {
              error_propagate(errp, local_err);
 -            return -EINVAL;
 +            ret = -EINVAL;
 +            goto out;
          }
          /* save to extents array */
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
 , 0, 0, 0, 0, &extent, errp);
              if (ret < 0) {
                  bdrv_unref_child(bs, extent_file);
 -                return ret;
 +                goto out;
              }
              extent->flat_start_offset = flat_offset << 9;
          } else if (!strcmp(type, "SPARSE") || !strcmp(type, "VMFSSPARSE")) {
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
              g_free(buf);
              if (ret) {
                  bdrv_unref_child(bs, extent_file);
 -                return ret;
 +                goto out;
              }
              extent = &s->extents[s->num_extents - 1];
          } else if (!strcmp(type, "SESPARSE")) {
              ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
              if (ret) {
                  bdrv_unref_child(bs, extent_file);
 -                return ret;
 +                goto out;
              }
              extent = &s->extents[s->num_extents - 1];
          } else {
              error_setg(errp, "Unsupported extent type '%s'", type);
              bdrv_unref_child(bs, extent_file);
 -            return -ENOTSUP;
 +            ret = -ENOTSUP;
 +            goto out;
          }
          extent->type = g_strdup(type);
      }
 -    return 0;
 +
 +    ret = 0;
 +    goto out;
  invalid:
      np = next_line(p);
@@ -XXX,XX +XXX,XX @@ invalid:
          np--;
      }
      error_setg(errp, "Invalid extent line: %.*s", (int)(np - p), p);
 -    return -EINVAL;
 +    ret = -EINVAL;
 +
 +out:
 +    g_free(desc_file_dir);
 +    return ret;
  }
  static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
      }
      s->create_type = g_strdup(ct);
      s->desc_offset = 0;
 -    ret = vmdk_parse_extents(buf, bs, bs->file->bs->exact_filename, options,
 -                             errp);
 +    ret = vmdk_parse_extents(buf, bs, options, errp);
  exit:
      return ret;
  }
 diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/059
 +++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
  echo
  echo "=== Testing monolithicFlat with internally generated JSON file name ==="
 +# Should work, because bdrv_dirname() works fine with blkdebug
  IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
 -$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" 2>&1 \
 -    | _filter_testdir | _filter_imgfmt
 +$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
 +         -c info \
 +    2>&1 \
 +    | _filter_testdir | _filter_imgfmt | _filter_img_info
  _cleanup_test_img
  echo
 diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
 index XXXXXXX..XXXXXXX 100644
---- a/docs/qemu-block-drivers.texi
+--- a/tests/qemu-iotests/059.out
-+++ b/docs/qemu-block-drivers.texi
++++ b/tests/qemu-iotests/059.out
-@@ -XXX,XX +XXX,XX @@ Supported options:
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
- @item preallocation
- Preallocation mode (allowed values: @code{off}, @code{falloc}, @code{full}).
+ === Testing monolithicFlat with internally generated JSON file name ===
- @code{falloc} mode preallocates space for image by calling posix_fallocate().
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
--@code{full} mode preallocates space for image by writing zeros to underlying
+-qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
--storage.
++format name: IMGFMT
-+@code{full} mode preallocates space for image by writing data to underlying
++cluster size: 0 bytes
-+storage.  This data may or may not be zero, depending on the storage location.
++vm state offset: 0 bytes
- @end table
+ === Testing version 3 ===
- @item qcow2
+ image: TEST_DIR/iotest-version3.IMGFMT
 diff --git a/qemu-img.texi b/qemu-img.texi
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-img.texi
 +++ b/qemu-img.texi
@@ -XXX,XX +XXX,XX @@ Supported options:
  @item preallocation
  Preallocation mode (allowed values: @code{off}, @code{falloc}, @code{full}).
  @code{falloc} mode preallocates space for image by calling posix_fallocate().
 -@code{full} mode preallocates space for image by writing zeros to underlying
 -storage.
 +@code{full} mode preallocates space for image by writing data to underlying
 +storage.  This data may or may not be zero, depending on the storage location.
  @end table
  @item qcow2
 --
 .21.0

-[Qemu-devel] [PULL 12/17] iotests: Full mirror to existing non-zero image
+[Qemu-devel] [PULL 07/15] iotests: Keep testing broken relative extent paths
-The result of a sync=full mirror should always be the equal to the
+We had a test for a case where relative extent paths did not work, but
-input.  Therefore, existing images should be treated as potentially
+unfortunately we just fixed the underlying problem, so it works now.
-non-zero and thus should be explicitly initialized to be zero
+This patch adds a new test case that still fails.
 beforehand.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190724171239.8764-12-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
 Message-id: 20190815153638.4600-4-mreitz@redhat.com
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/041     | 62 +++++++++++++++++++++++++++++++++++---
+ tests/qemu-iotests/059     | 27 +++++++++++++++++++++++++++
- tests/qemu-iotests/041.out |  4 +--
+ tests/qemu-iotests/059.out |  4 ++++
-files changed, 60 insertions(+), 6 deletions(-)
+files changed, 31 insertions(+)
-diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
+diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
 index XXXXXXX..XXXXXXX 100755
---- a/tests/qemu-iotests/041
+--- a/tests/qemu-iotests/059
-+++ b/tests/qemu-iotests/041
++++ b/tests/qemu-iotests/059
-@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
+@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
-     def setUp(self):
-         qemu_img('create', '-f', iotests.imgfmt, test_img,
+ echo
-                  str(TestUnbackedSource.image_len))
+ echo "=== Testing monolithicFlat with internally generated JSON file name ==="
 -        self.vm = iotests.VM().add_drive(test_img)
 +        self.vm = iotests.VM()
          self.vm.launch()
 +        result = self.vm.qmp('blockdev-add', node_name='drive0',
 +                             driver=iotests.imgfmt,
 +                             file={
 +                                 'driver': 'file',
 +                                 'filename': test_img,
 +                             })
 +        self.assert_qmp(result, 'return', {})
      def tearDown(self):
          self.vm.shutdown()
@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
      def test_absolute_paths_full(self):
          self.assert_no_active_block_jobs()
 -        result = self.vm.qmp('drive-mirror', device='drive0',
 +        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
                               sync='full', target=target_img,
                               mode='absolute-paths')
          self.assert_qmp(result, 'return', {})
@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
      def test_absolute_paths_top(self):
          self.assert_no_active_block_jobs()
 -        result = self.vm.qmp('drive-mirror', device='drive0',
 +        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
                               sync='top', target=target_img,
                               mode='absolute-paths')
          self.assert_qmp(result, 'return', {})
@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
      def test_absolute_paths_none(self):
          self.assert_no_active_block_jobs()
 -        result = self.vm.qmp('drive-mirror', device='drive0',
 +        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
                               sync='none', target=target_img,
                               mode='absolute-paths')
          self.assert_qmp(result, 'return', {})
          self.complete_and_wait()
          self.assert_no_active_block_jobs()
 +    def test_existing_full(self):
 +        qemu_img('create', '-f', iotests.imgfmt, target_img,
 +                 str(self.image_len))
 +        qemu_io('-c', 'write -P 42 0 64k', target_img)
 +
-+        self.assert_no_active_block_jobs()
++echo '--- blkdebug ---'
-+        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
+ # Should work, because bdrv_dirname() works fine with blkdebug
-+                             sync='full', target=target_img, mode='existing')
+ IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
-+        self.assert_qmp(result, 'return', {})
+ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
-+        self.complete_and_wait()
+@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TE
-+        self.assert_no_active_block_jobs()
+     | _filter_testdir | _filter_imgfmt | _filter_img_info
  _cleanup_test_img
 +echo '--- quorum ---'
 +# Should not work, because bdrv_dirname() does not work with quorum
 +IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
 +cp "$TEST_IMG" "$TEST_IMG.orig"
 +
-+        result = self.vm.qmp('blockdev-del', node_name='drive0')
++filename="json:{
-+        self.assert_qmp(result, 'return', {})
++    \"driver\": \"$IMGFMT\",
 +    \"file\": {
 +        \"driver\": \"quorum\",
 +        \"children\": [ {
 +            \"driver\": \"file\",
 +            \"filename\": \"$TEST_IMG\"
 +        }, {
 +            \"driver\": \"file\",
 +            \"filename\": \"$TEST_IMG.orig\"
 +        } ],
 +        \"vote-threshold\": 1
 +    } }"
 +
-+        self.assertTrue(iotests.compare_images(test_img, target_img),
++filename=$(echo "$filename" | tr '\n' ' ' | sed -e 's/\s\+/ /g')
-+                        'target image does not match source after mirroring')
++$QEMU_IMG info "$filename" 2>&1 \
 +    | sed -e "s/'json:[^']*'/\$QUORUM_FILE/g" \
 +    | _filter_testdir | _filter_imgfmt | _filter_img_info
 +
-+    def test_blockdev_full(self):
-+        qemu_img('create', '-f', iotests.imgfmt, target_img,
-+                 str(self.image_len))
-+        qemu_io('-c', 'write -P 42 0 64k', target_img)
 +
-+        result = self.vm.qmp('blockdev-add', node_name='target',
+ echo
-+                             driver=iotests.imgfmt,
+ echo "=== Testing version 3 ==="
-+                             file={
+ _use_sample_img iotest-version3.vmdk.bz2
-+                                 'driver': 'file',
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
 +                                 'filename': target_img,
 +                             })
 +        self.assert_qmp(result, 'return', {})
 +
 +        self.assert_no_active_block_jobs()
 +        result = self.vm.qmp('blockdev-mirror', job_id='drive0', device='drive0',
 +                             sync='full', target='target')
 +        self.assert_qmp(result, 'return', {})
 +        self.complete_and_wait()
 +        self.assert_no_active_block_jobs()
 +
 +        result = self.vm.qmp('blockdev-del', node_name='drive0')
 +        self.assert_qmp(result, 'return', {})
 +
 +        result = self.vm.qmp('blockdev-del', node_name='target')
 +        self.assert_qmp(result, 'return', {})
 +
 +        self.assertTrue(iotests.compare_images(test_img, target_img),
 +                        'target image does not match source after mirroring')
 +
  class TestGranularity(iotests.QMPTestCase):
      image_len = 10 * 1024 * 1024 # MB
 diff --git a/tests/qemu-iotests/041.out b/tests/qemu-iotests/041.out
 index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/041.out
+--- a/tests/qemu-iotests/059.out
-+++ b/tests/qemu-iotests/041.out
++++ b/tests/qemu-iotests/059.out
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
--........................................................................................
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-+..........................................................................................
- ----------------------------------------------------------------------
+ === Testing monolithicFlat with internally generated JSON file name ===
--Ran 88 tests
++--- blkdebug ---
-+Ran 90 tests
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+ format name: IMGFMT
- OK
+ cluster size: 0 bytes
  vm state offset: 0 bytes
 +--- quorum ---
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 +qemu-img: Could not open $QUORUM_FILE: Cannot use relative paths with VMDK descriptor file $QUORUM_FILE: Cannot generate a base directory for quorum nodes
  === Testing version 3 ===
  image: TEST_DIR/iotest-version3.IMGFMT
 --
 .21.0

-[Qemu-devel] [PULL 14/17] vmdk: Make block_status recurse for flat extents
+[Qemu-devel] [PULL 08/15] vmdk: Reject invalid compressed writes
-Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
+Compressed writes generally have to write full clusters, not just in
 theory but also in practice when it comes to vmdk's streamOptimized
 subformat.  It currently is just silently broken for writes with
 non-zero in-cluster offsets:
 $ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
 $ qemu-io -c 'write 4k 4k' -c 'read 4k 4k' foo.vmdk
 wrote 4096/4096 bytes at offset 4096
 KiB, 1 ops; 00.01 sec (443.724 KiB/sec and 110.9309 ops/sec)
 read failed: Invalid argument
 (The technical reason is that vmdk_write_extent() just writes the
 incomplete compressed data actually to offset 4k.  When reading the
 data, vmdk_read_extent() looks at offset 0 and finds the compressed data
 size to be 0, because that is what it reads from there.  This yields an
 error.)
 For incomplete writes with zero in-cluster offsets, the error path when
 reading the rest of the cluster is a bit different, but the result is
 the same:
 $ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
 $ qemu-io -c 'write 0k 4k' -c 'read 4k 4k' foo.vmdk
 wrote 4096/4096 bytes at offset 0
 KiB, 1 ops; 00.01 sec (362.641 KiB/sec and 90.6603 ops/sec)
 read failed: Invalid argument
 (Here, vmdk_read_extent() finds the data and then sees that the
 uncompressed data is short.)
 It is better to reject invalid writes than to make the user believe they
 might have succeeded and then fail when trying to read it back.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190725155512.9827-3-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
-Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Message-id: 20190815153638.4600-5-mreitz@redhat.com
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/vmdk.c | 3 +++
+ block/vmdk.c | 10 ++++++++++
-file changed, 3 insertions(+)
+file changed, 10 insertions(+)
 diff --git a/block/vmdk.c b/block/vmdk.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/vmdk.c
 +++ b/block/vmdk.c
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vmdk_co_block_status(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset,
-         if (!extent->compressed) {
+     if (extent->compressed) {
-             ret |= BDRV_BLOCK_OFFSET_VALID;
+         void *compressed_data;
-             *map = cluster_offset + index_in_cluster;
-+            if (extent->flat) {
++        /* Only whole clusters */
-+                ret |= BDRV_BLOCK_RECURSE;
++        if (offset_in_cluster ||
-+            }
++            n_bytes > (extent->cluster_sectors * SECTOR_SIZE) ||
-         }
++            (n_bytes < (extent->cluster_sectors * SECTOR_SIZE) &&
-         *file = extent->file->bs;
++             offset + n_bytes != extent->end_sector * SECTOR_SIZE))
-         break;
++        {
 +            ret = -EINVAL;
 +            goto out;
 +        }
 +
          if (!extent->has_marker) {
              ret = -EINVAL;
              goto out;
 --
 .21.0

-[Qemu-devel] [PULL 06/17] block: Use bdrv_has_zero_init_truncate()
+[Qemu-devel] [PULL 09/15] iotests: Disable broken streamOptimized tests
-vhdx and parallels call bdrv_has_zero_init() when they do not really
+streamOptimized does not support writes that do not span exactly one
-care about an image's post-create state but only about what happens when
+cluster.  Furthermore, it cannot rewrite already allocated clusters.
-you grow an image.  That is a bit ugly, and also overly safe when
+As such, many iotests do not work with it.  Disable them.
 growing preallocated images without preallocating the new areas.
 Let them use bdrv_has_zero_init_truncate() instead.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190724171239.8764-6-mreitz@redhat.com
+Message-id: 20190815153638.4600-6-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
 Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
 [mreitz: Added commit message]
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/parallels.c | 2 +-
+ tests/qemu-iotests/002 | 1 +
- block/vhdx.c      | 2 +-
+ tests/qemu-iotests/003 | 1 +
-files changed, 2 insertions(+), 2 deletions(-)
+ tests/qemu-iotests/005 | 3 ++-
  tests/qemu-iotests/009 | 1 +
  tests/qemu-iotests/010 | 1 +
  tests/qemu-iotests/011 | 1 +
  tests/qemu-iotests/017 | 3 ++-
  tests/qemu-iotests/018 | 3 ++-
  tests/qemu-iotests/019 | 3 ++-
  tests/qemu-iotests/020 | 3 ++-
  tests/qemu-iotests/027 | 1 +
  tests/qemu-iotests/032 | 1 +
  tests/qemu-iotests/033 | 1 +
  tests/qemu-iotests/034 | 3 ++-
  tests/qemu-iotests/037 | 3 ++-
  tests/qemu-iotests/063 | 3 ++-
  tests/qemu-iotests/072 | 1 +
  tests/qemu-iotests/105 | 3 ++-
  tests/qemu-iotests/197 | 1 +
  tests/qemu-iotests/215 | 1 +
  tests/qemu-iotests/251 | 1 +
 files changed, 30 insertions(+), 9 deletions(-)
-diff --git a/block/parallels.c b/block/parallels.c
+diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/parallels.c
+--- a/tests/qemu-iotests/002
-+++ b/block/parallels.c
++++ b/tests/qemu-iotests/002
-@@ -XXX,XX +XXX,XX @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags,
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-         goto fail_options;
-     }
+ _supported_fmt generic
+ _supported_proto generic
--    if (!bdrv_has_zero_init(bs->file->bs)) {
++_unsupported_imgopts "subformat=streamOptimized"
-+    if (!bdrv_has_zero_init_truncate(bs->file->bs)) {
-         s->prealloc_mode = PRL_PREALLOC_MODE_FALLOCATE;
-     }
+ size=128M
+diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
-diff --git a/block/vhdx.c b/block/vhdx.c
+index XXXXXXX..XXXXXXX 100755
-index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/003
---- a/block/vhdx.c
++++ b/tests/qemu-iotests/003
-+++ b/block/vhdx.c
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-@@ -XXX,XX +XXX,XX @@ static coroutine_fn int vhdx_co_writev(BlockDriverState *bs, int64_t sector_num,
-                 /* Queue another write of zero buffers if the underlying file
+ _supported_fmt generic
-                  * does not zero-fill on file extension */
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
--                if (bdrv_has_zero_init(bs->file->bs) == 0) {
-+                if (bdrv_has_zero_init_truncate(bs->file->bs) == 0) {
+ size=128M
-                     use_zero_buffers = true;
+ offset=67M
+diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
-                     /* zero fill the front, if any */
+index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/005
 +++ b/tests/qemu-iotests/005
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
  _supported_proto generic
  _supported_os Linux
  _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  # vpc is limited to 127GB, so we can't test it here
  if [ "$IMGFMT" = "vpc" ]; then
 diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/009
 +++ b/tests/qemu-iotests/009
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=6G
 diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/010
 +++ b/tests/qemu-iotests/010
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=6G
 diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/011
 +++ b/tests/qemu-iotests/011
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=6G
 diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/017
 +++ b/tests/qemu-iotests/017
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow qcow2 vmdk qed
  _supported_proto generic
  _unsupported_proto vxhs
 -_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 +_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
 +                     "subformat=streamOptimized"
  TEST_OFFSETS="0 4294967296"
 diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/018
 +++ b/tests/qemu-iotests/018
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow qcow2 vmdk qed
  _supported_proto file
  _supported_os Linux
 -_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 +_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
 +                     "streamOptimized"
  TEST_OFFSETS="0 4294967296"
 diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/019
 +++ b/tests/qemu-iotests/019
@@ -XXX,XX +XXX,XX @@ _supported_proto file
  _supported_os Linux
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  TEST_OFFSETS="0 4294967296"
  CLUSTER_SIZE=65536
 diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/020
 +++ b/tests/qemu-iotests/020
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
  _supported_proto file
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  TEST_OFFSETS="0 4294967296"
 diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/027
 +++ b/tests/qemu-iotests/027
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt vmdk qcow qcow2 qed
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=128M
 diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/032
 +++ b/tests/qemu-iotests/032
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  # This works for any image format (though unlikely to segfault for raw)
  _supported_fmt generic
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  echo
  echo === Prepare image ===
 diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/033
 +++ b/tests/qemu-iotests/033
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=128M
 diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/034
 +++ b/tests/qemu-iotests/034
@@ -XXX,XX +XXX,XX @@ _supported_proto file
  _supported_os Linux
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  CLUSTER_SIZE=4k
  size=128M
 diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/037
 +++ b/tests/qemu-iotests/037
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
  _supported_proto file
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  CLUSTER_SIZE=4k
  size=128M
 diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/063
 +++ b/tests/qemu-iotests/063
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed raw
  _supported_proto file
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  _make_test_img 4M
 diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/072
 +++ b/tests/qemu-iotests/072
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
  _supported_proto file
 +_unsupported_imgopts "subformat=streamOptimized"
  IMG_SIZE=64M
 diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/105
 +++ b/tests/qemu-iotests/105
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow2 vmdk vhdx qed
  _supported_proto generic
  _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  echo
  echo "creating large image"
 diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/197
 +++ b/tests/qemu-iotests/197
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
  _supported_proto generic
  # LUKS support may be possible, but it complicates things.
  _unsupported_fmt luks
 +_unsupported_imgopts "subformat=streamOptimized"
  echo
  echo '=== Copy-on-read ==='
 diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/215
 +++ b/tests/qemu-iotests/215
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
  _supported_proto generic
  # LUKS support may be possible, but it complicates things.
  _unsupported_fmt luks
 +_unsupported_imgopts "subformat=streamOptimized"
  echo
  echo '=== Copy-on-read ==='
 diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/251
 +++ b/tests/qemu-iotests/251
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto file
  _supported_os Linux
 +_unsupported_imgopts "subformat=streamOptimized"
  if [ "$IMGOPTSSYNTAX" = "true" ]; then
      # We use json:{} filenames here, so we cannot work with additional options.
 --
 .21.0

-[Qemu-devel] [PULL 13/17] vdi: Make block_status recurse for fixed images
+[Qemu-devel] [PULL 10/15] iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
-Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+The error message for the test case where we have a quorum node for
-Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
+which no directory name can be generated is different: For
 twoGbMaxExtentSparse, it complains that it cannot open the extent file.
 For other (sub)formats, it just notes that it cannot determine the
 backing file path.  Both are fine, but just disable twoGbMaxExtentSparse
 for simplicity's sake.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190725155512.9827-2-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
-Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Message-id: 20190815153638.4600-7-mreitz@redhat.com
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/vdi.c | 3 ++-
+ tests/qemu-iotests/110 | 3 ++-
 file changed, 2 insertions(+), 1 deletion(-)
-diff --git a/block/vdi.c b/block/vdi.c
+diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/vdi.c
+--- a/tests/qemu-iotests/110
-+++ b/block/vdi.c
++++ b/tests/qemu-iotests/110
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vdi_co_block_status(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-     *map = s->header.offset_data + (uint64_t)bmap_entry * s->block_size +
+ # Any format supporting backing files
-         index_in_block;
+ _supported_fmt qed qcow qcow2 vmdk
-     *file = bs->file->bs;
+ _supported_proto file
--    return BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID;
+-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
-+    return BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID |
++_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
-+        (s->header.image_type == VDI_TYPE_STATIC ? BDRV_BLOCK_RECURSE : 0);
++                     "subformat=twoGbMaxExtentSparse"
- }
+ TEST_IMG_REL=$(basename "$TEST_IMG")
- static int coroutine_fn
 --
 .21.0

-[Qemu-devel] [PULL 15/17] vpc: Do not return RAW from block_status
+[Qemu-devel] [PULL 11/15] iotests: Disable 126 for flat vmdk subformats
-vpc is not really a passthrough driver, even when using the fixed
+iotest 126 requires backing file support, which flat vmdks cannot offer.
-subformat (where host and guest offsets are equal).  It should handle
+Skip this test for such subformats.
 preallocation like all other drivers do, namely by returning
 DATA | RECURSE instead of RAW.
 There is no tangible difference but the fact that bdrv_is_allocated() no
 longer falls through to the protocol layer.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190725155512.9827-4-mreitz@redhat.com
+Message-id: 20190815153638.4600-8-mreitz@redhat.com
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/vpc.c | 2 +-
+ tests/qemu-iotests/126 | 2 ++
-file changed, 1 insertion(+), 1 deletion(-)
+file changed, 2 insertions(+)
-diff --git a/block/vpc.c b/block/vpc.c
+diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/vpc.c
+--- a/tests/qemu-iotests/126
-+++ b/block/vpc.c
++++ b/tests/qemu-iotests/126
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_block_status(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ status=1    # failure is the default!
-         *pnum = bytes;
-         *map = offset;
+ # Needs backing file support
-         *file = bs->file->bs;
+ _supported_fmt qcow qcow2 qed vmdk
--        return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID;
++_unsupported_imgopts "subformat=monolithicFlat" \
-+        return BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_RECURSE;
++                     "subformat=twoGbMaxExtentFlat"
-     }
+ # This is the default protocol (and we want to test the difference between
+ # colons which separate a protocol prefix from the rest and colons which are
-     qemu_co_mutex_lock(&s->lock);
+ # just part of the filename, so we cannot test protocols which require a prefix)
 --
 .21.0

-[Qemu-devel] [PULL 03/17] mirror: Fix bdrv_has_zero_init() use
+[Qemu-devel] [PULL 12/15] file-posix: fix request_alignment typo
-bdrv_has_zero_init() only has meaning for newly created images or image
+From: Stefan Hajnoczi <stefanha@redhat.com>
 areas.  If the mirror job itself did not create the image, it cannot
 rely on bdrv_has_zero_init()'s result to carry any meaning.
-This is the case for drive-mirror with mode=existing and always for
+Fixes: a6b257a08e3d72219f03e461a52152672fec0612
-blockdev-mirror.
+       ("file-posix: Handle undetectable alignment")
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
-Note that we only have to zero-initialize the target with sync=full,
+Message-id: 20190827101328.4062-1-stefanha@redhat.com
-because other modes actually do not promise that the target will contain
+Reviewed-by: Thomas Huth <thuth@redhat.com>
 the same data as the source after the job -- sync=top only promises to
 copy anything allocated in the top layer, and sync=none will only copy
 new I/O.  (Which is how mirror has always handled it.)
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 Message-id: 20190724171239.8764-3-mreitz@redhat.com
 Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- include/block/block_int.h   |  2 ++
+ block/file-posix.c | 2 +-
- block/mirror.c              | 11 ++++++++---
+file changed, 1 insertion(+), 1 deletion(-)
  blockdev.c                  | 16 +++++++++++++---
  tests/test-block-iothread.c |  2 +-
 files changed, 24 insertions(+), 7 deletions(-)
-diff --git a/include/block/block_int.h b/include/block/block_int.h
+diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/block/block_int.h
+--- a/block/file-posix.c
-+++ b/include/block/block_int.h
++++ b/block/file-posix.c
-@@ -XXX,XX +XXX,XX @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
-  * @buf_size: The amount of data that can be in flight at one time.
+         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
-  * @mode: Whether to collapse all images in the chain to the target.
+             align = alignments[i];
-  * @backing_mode: How to establish the target's backing chain after completion.
+             if (raw_is_io_aligned(fd, buf + align, max_align)) {
-+ * @zero_target: Whether the target should be explicitly zero-initialized
+-                /* Fallback to request_aligment. */
-  * @on_source_error: The action to take upon error reading from the source.
++                /* Fallback to request_alignment. */
-  * @on_target_error: The action to take upon error writing to the target.
+                 s->buf_align = (align != 1) ? align : bs->bl.request_alignment;
-  * @unmap: Whether to unmap target where source sectors only contain zeroes.
+                 break;
-@@ -XXX,XX +XXX,XX @@ void mirror_start(const char *job_id, BlockDriverState *bs,
+             }
                    int creation_flags, int64_t speed,
                    uint32_t granularity, int64_t buf_size,
                    MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
 +                  bool zero_target,
                    BlockdevOnError on_source_error,
                    BlockdevOnError on_target_error,
                    bool unmap, const char *filter_node_name,
 diff --git a/block/mirror.c b/block/mirror.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/mirror.c
 +++ b/block/mirror.c
@@ -XXX,XX +XXX,XX @@ typedef struct MirrorBlockJob {
      Error *replace_blocker;
      bool is_none_mode;
      BlockMirrorBackingMode backing_mode;
 +    /* Whether the target image requires explicit zero-initialization */
 +    bool zero_target;
      MirrorCopyMode copy_mode;
      BlockdevOnError on_source_error, on_target_error;
      bool synced;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s)
      int ret;
      int64_t count;
 -    if (base == NULL && !bdrv_has_zero_init(target_bs)) {
 +    if (s->zero_target) {
          if (!bdrv_can_write_zeroes_with_unmap(target_bs)) {
              bdrv_set_dirty_bitmap(s->dirty_bitmap, 0, s->bdev_length);
              return 0;
@@ -XXX,XX +XXX,XX @@ static BlockJob *mirror_start_job(
                               const char *replaces, int64_t speed,
                               uint32_t granularity, int64_t buf_size,
                               BlockMirrorBackingMode backing_mode,
 +                             bool zero_target,
                               BlockdevOnError on_source_error,
                               BlockdevOnError on_target_error,
                               bool unmap,
@@ -XXX,XX +XXX,XX @@ static BlockJob *mirror_start_job(
      s->on_target_error = on_target_error;
      s->is_none_mode = is_none_mode;
      s->backing_mode = backing_mode;
 +    s->zero_target = zero_target;
      s->copy_mode = copy_mode;
      s->base = base;
      s->granularity = granularity;
@@ -XXX,XX +XXX,XX @@ void mirror_start(const char *job_id, BlockDriverState *bs,
                    int creation_flags, int64_t speed,
                    uint32_t granularity, int64_t buf_size,
                    MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
 +                  bool zero_target,
                    BlockdevOnError on_source_error,
                    BlockdevOnError on_target_error,
                    bool unmap, const char *filter_node_name,
@@ -XXX,XX +XXX,XX @@ void mirror_start(const char *job_id, BlockDriverState *bs,
      is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
      base = mode == MIRROR_SYNC_MODE_TOP ? backing_bs(bs) : NULL;
      mirror_start_job(job_id, bs, creation_flags, target, replaces,
 -                     speed, granularity, buf_size, backing_mode,
 +                     speed, granularity, buf_size, backing_mode, zero_target,
                       on_source_error, on_target_error, unmap, NULL, NULL,
                       &mirror_job_driver, is_none_mode, base, false,
                       filter_node_name, true, copy_mode, errp);
@@ -XXX,XX +XXX,XX @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
      ret = mirror_start_job(
                       job_id, bs, creation_flags, base, NULL, speed, 0, 0,
 -                     MIRROR_LEAVE_BACKING_CHAIN,
 +                     MIRROR_LEAVE_BACKING_CHAIN, false,
                       on_error, on_error, true, cb, opaque,
                       &commit_active_job_driver, false, base, auto_complete,
                       filter_node_name, false, MIRROR_COPY_MODE_BACKGROUND,
 diff --git a/blockdev.c b/blockdev.c
 index XXXXXXX..XXXXXXX 100644
 --- a/blockdev.c
 +++ b/blockdev.c
@@ -XXX,XX +XXX,XX @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
                                     bool has_replaces, const char *replaces,
                                     enum MirrorSyncMode sync,
                                     BlockMirrorBackingMode backing_mode,
 +                                   bool zero_target,
                                     bool has_speed, int64_t speed,
                                     bool has_granularity, uint32_t granularity,
                                     bool has_buf_size, int64_t buf_size,
@@ -XXX,XX +XXX,XX @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
       */
      mirror_start(job_id, bs, target,
                   has_replaces ? replaces : NULL, job_flags,
 -                 speed, granularity, buf_size, sync, backing_mode,
 +                 speed, granularity, buf_size, sync, backing_mode, zero_target,
                   on_source_error, on_target_error, unmap, filter_node_name,
                   copy_mode, errp);
  }
@@ -XXX,XX +XXX,XX @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
      int flags;
      int64_t size;
      const char *format = arg->format;
 +    bool zero_target;
      int ret;
      bs = qmp_get_root_bs(arg->device, errp);
@@ -XXX,XX +XXX,XX @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
          goto out;
      }
 +    zero_target = (arg->sync == MIRROR_SYNC_MODE_FULL &&
 +                   (arg->mode == NEW_IMAGE_MODE_EXISTING ||
 +                    !bdrv_has_zero_init(target_bs)));
 +
      ret = bdrv_try_set_aio_context(target_bs, aio_context, errp);
      if (ret < 0) {
          bdrv_unref(target_bs);
@@ -XXX,XX +XXX,XX @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
      blockdev_mirror_common(arg->has_job_id ? arg->job_id : NULL, bs, target_bs,
                             arg->has_replaces, arg->replaces, arg->sync,
 -                           backing_mode, arg->has_speed, arg->speed,
 +                           backing_mode, zero_target,
 +                           arg->has_speed, arg->speed,
                             arg->has_granularity, arg->granularity,
                             arg->has_buf_size, arg->buf_size,
                             arg->has_on_source_error, arg->on_source_error,
@@ -XXX,XX +XXX,XX @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
      AioContext *aio_context;
      BlockMirrorBackingMode backing_mode = MIRROR_LEAVE_BACKING_CHAIN;
      Error *local_err = NULL;
 +    bool zero_target;
      int ret;
      bs = qmp_get_root_bs(device, errp);
@@ -XXX,XX +XXX,XX @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
          return;
      }
 +    zero_target = (sync == MIRROR_SYNC_MODE_FULL);
 +
      aio_context = bdrv_get_aio_context(bs);
      aio_context_acquire(aio_context);
@@ -XXX,XX +XXX,XX @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
      blockdev_mirror_common(has_job_id ? job_id : NULL, bs, target_bs,
                             has_replaces, replaces, sync, backing_mode,
 -                           has_speed, speed,
 +                           zero_target, has_speed, speed,
                             has_granularity, granularity,
                             has_buf_size, buf_size,
                             has_on_source_error, on_source_error,
 diff --git a/tests/test-block-iothread.c b/tests/test-block-iothread.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/test-block-iothread.c
 +++ b/tests/test-block-iothread.c
@@ -XXX,XX +XXX,XX @@ static void test_propagate_mirror(void)
      /* Start a mirror job */
      mirror_start("job0", src, target, NULL, JOB_DEFAULT, 0, 0, 0,
 -                 MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN,
 +                 MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false,
                   BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
                   false, "filter_node", MIRROR_COPY_MODE_BACKGROUND,
                   &error_abort);
 --
 .21.0

-[Qemu-devel] [PULL 07/17] qcow2: Fix .bdrv_has_zero_init()
+Deleted patch
-If a qcow2 file is preallocated, it can no longer guarantee that it
-initially appears as filled with zeroes.
-So implement .bdrv_has_zero_init() by checking whether the file is
-preallocated; if so, forward the call to the underlying storage node,
-except for when it is encrypted: Encrypted preallocated images always
-return effectively random data, so .bdrv_has_zero_init() must always
-return 0 for them.
-.bdrv_has_zero_init_truncate() can remain bdrv_has_zero_init_1(),
-because it presupposes PREALLOC_MODE_OFF.
-Reported-by: Stefano Garzarella <sgarzare@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190724171239.8764-7-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- block/qcow2.c | 29 ++++++++++++++++++++++++++++-
-file changed, 28 insertions(+), 1 deletion(-)
-diff --git a/block/qcow2.c b/block/qcow2.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2.c
-+++ b/block/qcow2.c
-@@ -XXX,XX +XXX,XX @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs,
-     return spec_info;
- }
-+static int qcow2_has_zero_init(BlockDriverState *bs)
-+{
-+    BDRVQcow2State *s = bs->opaque;
-+    bool preallocated;
-+
-+    if (qemu_in_coroutine()) {
-+        qemu_co_mutex_lock(&s->lock);
-+    }
-+    /*
-+     * Check preallocation status: Preallocated images have all L2
-+     * tables allocated, nonpreallocated images have none.  It is
-+     * therefore enough to check the first one.
-+     */
-+    preallocated = s->l1_size > 0 && s->l1_table[0] != 0;
-+    if (qemu_in_coroutine()) {
-+        qemu_co_mutex_unlock(&s->lock);
-+    }
-+
-+    if (!preallocated) {
-+        return 1;
-+    } else if (bs->encrypted) {
-+        return 0;
-+    } else {
-+        return bdrv_has_zero_init(s->data_file->bs);
-+    }
-+}
-+
- static int qcow2_save_vmstate(BlockDriverState *bs, QEMUIOVector *qiov,
-                               int64_t pos)
- {
-@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
-     .bdrv_child_perm      = bdrv_format_default_perms,
-     .bdrv_co_create_opts  = qcow2_co_create_opts,
-     .bdrv_co_create       = qcow2_co_create,
--    .bdrv_has_zero_init = bdrv_has_zero_init_1,
-+    .bdrv_has_zero_init   = qcow2_has_zero_init,
-     .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
-     .bdrv_co_block_status = qcow2_co_block_status,
---
-.21.0

-[Qemu-devel] [PULL 09/17] vhdx: Fix .bdrv_has_zero_init()
+[Qemu-devel] [PULL 13/15] iotests: Check for enabled drivers before testing them
-Fixed VHDX images cannot guarantee to be zero-initialized.  If the image
+From: Thomas Huth <thuth@redhat.com>
 has the "fixed" subformat, forward the call to the underlying storage
 node.
-Reported-by: Stefano Garzarella <sgarzare@redhat.com>
+It is possible to enable only a subset of the block drivers with the
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+"--block-drv-rw-whitelist" option of the "configure" script. All other
-Message-id: 20190724171239.8764-9-mreitz@redhat.com
+drivers are marked as unusable (or only included as read-only with the
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+"--block-drv-ro-whitelist" option). If an iotest is now using such a
 disabled block driver, it is failing - which is bad, since at least the
 tests in the "auto" group should be able to deal with this situation.
 Thus let's introduce a "_require_drivers" function that can be used by
 the shell tests to check for the availability of certain drivers first,
 and marks the test as "not run" if one of the drivers is missing.
 This patch mainly targets the test in the "auto" group which should
 never fail in such a case, but also improves some of the other tests
 along the way. Note that we also assume that the "qcow2" and "file"
 drivers are always available - otherwise it does not make sense to
 run "make check-block" at all (which only tests with qcow2 by default).
 Signed-off-by: Thomas Huth <thuth@redhat.com>
 Message-id: 20190823133552.11680-1-thuth@redhat.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/vhdx.c | 26 +++++++++++++++++++++++++-
+ tests/qemu-iotests/071       |  1 +
-file changed, 25 insertions(+), 1 deletion(-)
+ tests/qemu-iotests/081       |  4 +---
  tests/qemu-iotests/099       |  1 +
  tests/qemu-iotests/120       |  1 +
  tests/qemu-iotests/162       |  4 +---
  tests/qemu-iotests/184       |  1 +
  tests/qemu-iotests/186       |  1 +
  tests/qemu-iotests/common.rc | 14 ++++++++++++++
 files changed, 21 insertions(+), 6 deletions(-)
-diff --git a/block/vhdx.c b/block/vhdx.c
+diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/071
 +++ b/tests/qemu-iotests/071
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow2
  _supported_proto file
 +_require_drivers blkdebug blkverify
  do_run_qemu()
  {
 diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/081
 +++ b/tests/qemu-iotests/081
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt raw
  _supported_proto file
  _supported_os Linux
 +_require_drivers quorum
  do_run_qemu()
  {
@@ -XXX,XX +XXX,XX @@ run_qemu()
                            | _filter_qemu_io | _filter_generated_node_ids
  }
 -test_quorum=$($QEMU_IMG --help|grep quorum)
 -[ "$test_quorum" = "" ] && _supported_fmt quorum
 -
  quorum="driver=raw,file.driver=quorum,file.vote-threshold=2"
  quorum="$quorum,file.children.0.file.filename=$TEST_DIR/1.raw"
  quorum="$quorum,file.children.1.file.filename=$TEST_DIR/2.raw"
 diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/099
 +++ b/tests/qemu-iotests/099
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow qcow2 qed vdi vhdx vmdk vpc
  _supported_proto file
  _supported_os Linux
 +_require_drivers blkdebug blkverify
  _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
      "subformat=twoGbMaxExtentSparse"
 diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/120
 +++ b/tests/qemu-iotests/120
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto file
  _unsupported_fmt luks
 +_require_drivers raw
  _make_test_img 64M
 diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/162
 +++ b/tests/qemu-iotests/162
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  . ./common.filter
  _supported_fmt generic
 -
 -test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh\( \|$\)')
 -[ "$test_ssh" = "" ] && _notrun "ssh support required"
 +_require_drivers ssh
  echo
  echo '=== NBD ==='
 diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/184
 +++ b/tests/qemu-iotests/184
@@ -XXX,XX +XXX,XX @@ trap "exit \$status" 0 1 2 3 15
  . ./common.filter
  _supported_os Linux
 +_require_drivers throttle
  do_run_qemu()
  {
 diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/186
 +++ b/tests/qemu-iotests/186
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow2
  _supported_proto file
 +_require_drivers null-co
  if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
      _notrun "Requires a PC machine"
 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
 index XXXXXXX..XXXXXXX 100644
---- a/block/vhdx.c
+--- a/tests/qemu-iotests/common.rc
-+++ b/block/vhdx.c
++++ b/tests/qemu-iotests/common.rc
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_check(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ _require_command()
-     return 0;
+     [ -x "$c" ] || _notrun "$1 utility required, skipped this test"
  }
-+static int vhdx_has_zero_init(BlockDriverState *bs)
++# Check that a set of drivers has been whitelisted in the QEMU binary
 +#
 +_require_drivers()
 +{
-+    BDRVVHDXState *s = bs->opaque;
++    available=$($QEMU -drive format=help | \
-+    int state;
++                sed -e '/Supported formats:/!d' -e 's/Supported formats://')
-+
++    for driver
-+    /*
++    do
-+     * Check the subformat: Fixed images have all BAT entries present,
++        if ! echo "$available" | grep -q " $driver\( \|$\)"; then
-+     * dynamic images have none (right after creation).  It is
++            _notrun "$driver not available"
-+     * therefore enough to check the first BAT entry.
++        fi
-+     */
++    done
 +    if (!s->bat_entries) {
 +        return 1;
 +    }
 +
 +    state = s->bat[0] & VHDX_BAT_STATE_BIT_MASK;
 +    if (state == PAYLOAD_BLOCK_FULLY_PRESENT) {
 +        /* Fixed subformat */
 +        return bdrv_has_zero_init(bs->file->bs);
 +    }
 +
 +    /* Dynamic subformat */
 +    return 1;
 +}
 +
- static QemuOptsList vhdx_create_opts = {
+ # make sure this script returns success
-     .name = "vhdx-create-opts",
+ true
      .head = QTAILQ_HEAD_INITIALIZER(vhdx_create_opts.head),
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_vhdx = {
      .bdrv_co_create_opts    = vhdx_co_create_opts,
      .bdrv_get_info          = vhdx_get_info,
      .bdrv_co_check          = vhdx_co_check,
 -    .bdrv_has_zero_init     = bdrv_has_zero_init_1,
 +    .bdrv_has_zero_init     = vhdx_has_zero_init,
      .create_opts            = &vhdx_create_opts,
  };
 --
 .21.0

-[Qemu-devel] [PULL 10/17] iotests: Convert to preallocated encrypted qcow2
+[Qemu-devel] [PULL 14/15] tests/check-block: Skip iotests when sanitizers are enabled
-Add a test case for converting an empty image (which only returns zeroes
+From: Thomas Huth <thuth@redhat.com>
 when read) to a preallocated encrypted qcow2 image.
 qcow2_has_zero_init() should return 0 then, thus forcing qemu-img
 convert to create zero clusters.
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+The sanitizers (especially the address sanitizer from Clang) are
-Acked-by: Stefano Garzarella <sgarzare@redhat.com>
+sometimes printing out warnings or false positives - this spoils
-Tested-by: Stefano Garzarella <sgarzare@redhat.com>
+the output of the iotests, causing some of the tests to fail.
-Message-id: 20190724171239.8764-10-mreitz@redhat.com
+Thus let's skip the automatic iotests during "make check" when the
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+user configured QEMU with --enable-sanitizers.
 Signed-off-by: Thomas Huth <thuth@redhat.com>
 Message-id: 20190823084203.29734-1-thuth@redhat.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/188     | 20 +++++++++++++++++++-
+ tests/check-block.sh | 5 +++++
- tests/qemu-iotests/188.out |  4 ++++
+file changed, 5 insertions(+)
 files changed, 23 insertions(+), 1 deletion(-)
-diff --git a/tests/qemu-iotests/188 b/tests/qemu-iotests/188
+diff --git a/tests/check-block.sh b/tests/check-block.sh
 index XXXXXXX..XXXXXXX 100755
---- a/tests/qemu-iotests/188
+--- a/tests/check-block.sh
-+++ b/tests/qemu-iotests/188
++++ b/tests/check-block.sh
-@@ -XXX,XX +XXX,XX @@ SECRETALT="secret,id=sec0,data=platypus"
+@@ -XXX,XX +XXX,XX @@ if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
+     exit 0
- _make_test_img --object $SECRET -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10" $size
+ fi
--IMGSPEC="driver=$IMGFMT,file.filename=$TEST_IMG,encrypt.key-secret=sec0"
++if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
-+IMGSPEC="driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
++    echo "Sanitizers are enabled ==> Not running the qemu-iotests."
++    exit 0
  QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
@@ -XXX,XX +XXX,XX @@ echo
  echo "== verify open failure with wrong password =="
  $QEMU_IO --object $SECRETALT -c "read -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
 +_cleanup_test_img
 +
 +echo
 +echo "== verify that has_zero_init returns false when preallocating =="
 +
 +# Empty source file
 +if [ -n "$TEST_IMG_FILE" ]; then
 +    TEST_IMG_FILE="${TEST_IMG_FILE}.orig" _make_test_img $size
 +else
 +    TEST_IMG="${TEST_IMG}.orig" _make_test_img $size
 +fi
 +
-+$QEMU_IMG convert -O "$IMGFMT" --object $SECRET \
+ if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
-+    -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10,preallocation=metadata" \
+     echo "No qemu-system binary available ==> Not running the qemu-iotests."
-+    "${TEST_IMG}.orig" "$TEST_IMG"
+     exit 0
 +
 +$QEMU_IMG compare --object $SECRET --image-opts "${IMGSPEC}.orig" "$IMGSPEC"
 +
  # success, all done
  echo "*** done"
 diff --git a/tests/qemu-iotests/188.out b/tests/qemu-iotests/188.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/188.out
 +++ b/tests/qemu-iotests/188.out
@@ -XXX,XX +XXX,XX @@ read 16777216/16777216 bytes at offset 0
  == verify open failure with wrong password ==
  qemu-io: can't open: Invalid password, cannot unlock any keyslot
 +
 +== verify that has_zero_init returns false when preallocating ==
 +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=16777216
 +Images are identical.
  *** done
 --
 .21.0

-[Qemu-devel] [PULL 08/17] vdi: Fix .bdrv_has_zero_init()
+[Qemu-devel] [PULL 15/15] iotests: Unify cache mode quoting
-Static VDI images cannot guarantee to be zero-initialized.  If the image
+From: Nir Soffer <nirsof@gmail.com>
 has been statically allocated, forward the call to the underlying
 storage node.
-Reported-by: Stefano Garzarella <sgarzare@redhat.com>
+Quoting cache mode is not needed, and most tests use unquoted values.
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Unify all test to use the same style.
-Reviewed-by: Stefan Weil <sw@weilnetz.de>
-Acked-by: Stefano Garzarella <sgarzare@redhat.com>
+Message-id: 20190827173432.7656-1-nsoffer@redhat.com
-Tested-by: Stefano Garzarella <sgarzare@redhat.com>
+Signed-off-by: Nir Soffer <nsoffer@redhat.com>
 Message-id: 20190724171239.8764-8-mreitz@redhat.com
 Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/vdi.c | 13 ++++++++++++-
+ tests/qemu-iotests/026 | 4 ++--
-file changed, 12 insertions(+), 1 deletion(-)
+ tests/qemu-iotests/039 | 4 ++--
  tests/qemu-iotests/052 | 2 +-
  tests/qemu-iotests/091 | 4 ++--
 files changed, 7 insertions(+), 7 deletions(-)
-diff --git a/block/vdi.c b/block/vdi.c
+diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/vdi.c
+--- a/tests/qemu-iotests/026
-+++ b/block/vdi.c
++++ b/tests/qemu-iotests/026
-@@ -XXX,XX +XXX,XX @@ static void vdi_close(BlockDriverState *bs)
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-     error_free(s->migration_blocker);
+ # Currently only qcow2 supports rebasing
- }
+ _supported_fmt qcow2
+ _supported_proto file
-+static int vdi_has_zero_init(BlockDriverState *bs)
+-_default_cache_mode "writethrough"
-+{
+-_supported_cache_modes "writethrough" "none"
-+    BDRVVdiState *s = bs->opaque;
++_default_cache_mode writethrough
-+
++_supported_cache_modes writethrough none
-+    if (s->header.image_type == VDI_TYPE_STATIC) {
+ # The refcount table tests expect a certain minimum width for refcount entries
-+        return bdrv_has_zero_init(bs->file->bs);
+ # (so that the refcount table actually needs to grow); that minimum is 16 bits,
-+    } else {
+ # being the default refcount entry width.
-+        return 1;
+diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
-+    }
+index XXXXXXX..XXXXXXX 100755
-+}
+--- a/tests/qemu-iotests/039
-+
++++ b/tests/qemu-iotests/039
- static QemuOptsList vdi_create_opts = {
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-     .name = "vdi-create-opts",
+ _supported_fmt qcow2
-     .head = QTAILQ_HEAD_INITIALIZER(vdi_create_opts.head),
+ _supported_proto file
-@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_vdi = {
+ _supported_os Linux
-     .bdrv_child_perm          = bdrv_format_default_perms,
+-_default_cache_mode "writethrough"
-     .bdrv_co_create      = vdi_co_create,
+-_supported_cache_modes "writethrough"
-     .bdrv_co_create_opts = vdi_co_create_opts,
++_default_cache_mode writethrough
--    .bdrv_has_zero_init = bdrv_has_zero_init_1,
++_supported_cache_modes writethrough
-+    .bdrv_has_zero_init  = vdi_has_zero_init,
-     .bdrv_co_block_status = vdi_co_block_status,
+ size=128M
-     .bdrv_make_empty = vdi_make_empty,
 diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/052
 +++ b/tests/qemu-iotests/052
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
  _supported_proto file
  # Don't do O_DIRECT on tmpfs
 -_supported_cache_modes "writeback" "writethrough" "unsafe"
 +_supported_cache_modes writeback writethrough unsafe
  size=128M
  _make_test_img $size
 diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/091
 +++ b/tests/qemu-iotests/091
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow2
  _supported_proto file
  _supported_os Linux
 -_default_cache_mode "none"
 -_supported_cache_modes "writethrough" "none" "writeback"
 +_default_cache_mode none
 +_supported_cache_modes writethrough none writeback
  size=1G
 --
 .21.0

The following changes since commit 3fbd3405d2b0604ea530fc7a1828f19da1e95ff9:

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-08-17' into staging (2019-08-19 14:14:09 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2019-08-19

for you to fetch changes up to fa27c478102a6b5d1c6b02c005607ad9404b915f:

doc: Preallocation does not require writing zeroes (2019-08-19 17:13:26 +0200)

----------------------------------------------------------------
Block patches:
- preallocation=falloc/full support for LUKS
- Various minor fixes

----------------------------------------------------------------
Max Reitz (16):
  qemu-img: Fix bdrv_has_zero_init() use in convert
  mirror: Fix bdrv_has_zero_init() use
  block: Add bdrv_has_zero_init_truncate()
  block: Implement .bdrv_has_zero_init_truncate()
  block: Use bdrv_has_zero_init_truncate()
  qcow2: Fix .bdrv_has_zero_init()
  vdi: Fix .bdrv_has_zero_init()
  vhdx: Fix .bdrv_has_zero_init()
  iotests: Convert to preallocated encrypted qcow2
  iotests: Test convert -n to pre-filled image
  iotests: Full mirror to existing non-zero image
  vdi: Make block_status recurse for fixed images
  vmdk: Make block_status recurse for flat extents
  vpc: Do not return RAW from block_status
  iotests: Fix 141 when run with qed
  doc: Preallocation does not require writing zeroes

Maxim Levitsky (1):
  LUKS: support preallocation

-- 
2.21.0

From: Maxim Levitsky <mlevitsk@redhat.com>

preallocation=off and preallocation=metadata
both allocate luks header only, and preallocation=falloc/full
is passed to underlying file.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20190716161901.1430-1-mlevitsk@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qapi/block-core.json |  6 +++++-
 block/crypto.c       | 30 +++++++++++++++++++++++++++---
 2 files changed, 32 insertions(+), 4 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
 #
 # @file             Node to create the image format on
 # @size             Size of the virtual disk in bytes
+# @preallocation    Preallocation mode for the new image
+#                   (since: 4.2)
+#                   (default: off; allowed values: off, metadata, falloc, full)
 #
 # Since: 2.12
 ##
 { 'struct': 'BlockdevCreateOptionsLUKS',
   'base': 'QCryptoBlockCreateOptionsLUKS',
   'data': { 'file':             'BlockdevRef',
-            'size':             'size' } }
+            'size':             'size',
+            '*preallocation':   'PreallocMode' } }
 
 ##
 # @BlockdevCreateOptionsNfs:
diff --git a/block/crypto.c b/block/crypto.c
index XXXXXXX..XXXXXXX 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_read_func(QCryptoBlock *block,
 struct BlockCryptoCreateData {
     BlockBackend *blk;
     uint64_t size;
+    PreallocMode prealloc;
 };
 
 
@@ -XXX,XX +XXX,XX @@ static ssize_t block_crypto_init_func(QCryptoBlock *block,
      * available to the guest, so we must take account of that
      * which will be used by the crypto header
      */
-    return blk_truncate(data->blk, data->size + headerlen, PREALLOC_MODE_OFF,
+    return blk_truncate(data->blk, data->size + headerlen, data->prealloc,
                         errp);
 }
 
@@ -XXX,XX +XXX,XX @@ static int block_crypto_open_generic(QCryptoBlockFormat format,
 static int block_crypto_co_create_generic(BlockDriverState *bs,
                                           int64_t size,
                                           QCryptoBlockCreateOptions *opts,
+                                          PreallocMode prealloc,
                                           Error **errp)
 {
     int ret;
@@ -XXX,XX +XXX,XX @@ static int block_crypto_co_create_generic(BlockDriverState *bs,
         goto cleanup;
     }
 
+    if (prealloc == PREALLOC_MODE_METADATA) {
+        prealloc = PREALLOC_MODE_OFF;
+    }
+
     data = (struct BlockCryptoCreateData) {
         .blk = blk,
         .size = size,
+        .prealloc = prealloc,
     };
 
     crypto = qcrypto_block_create(opts, NULL,
@@ -XXX,XX +XXX,XX @@ block_crypto_co_create_luks(BlockdevCreateOptions *create_options, Error **errp)
     BlockdevCreateOptionsLUKS *luks_opts;
     BlockDriverState *bs = NULL;
     QCryptoBlockCreateOptions create_opts;
+    PreallocMode preallocation = PREALLOC_MODE_OFF;
     int ret;
 
     assert(create_options->driver == BLOCKDEV_DRIVER_LUKS);
@@ -XXX,XX +XXX,XX @@ block_crypto_co_create_luks(BlockdevCreateOptions *create_options, Error **errp)
         .u.luks = *qapi_BlockdevCreateOptionsLUKS_base(luks_opts),
     };
 
+    if (luks_opts->has_preallocation) {
+        preallocation = luks_opts->preallocation;
+    }
+
     ret = block_crypto_co_create_generic(bs, luks_opts->size, &create_opts,
-                                         errp);
+                                         preallocation, errp);
     if (ret < 0) {
         goto fail;
     }
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_crypto_co_create_opts_luks(const char *filename,
     QCryptoBlockCreateOptions *create_opts = NULL;
     BlockDriverState *bs = NULL;
     QDict *cryptoopts;
+    PreallocMode prealloc;
+    char *buf = NULL;
     int64_t size;
     int ret;
+    Error *local_err = NULL;
 
     /* Parse options */
     size = qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0);
 
+    buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
+    prealloc = qapi_enum_parse(&PreallocMode_lookup, buf,
+                               PREALLOC_MODE_OFF, &local_err);
+    g_free(buf);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return -EINVAL;
+    }
+
     cryptoopts = qemu_opts_to_qdict_filtered(opts, NULL,
                                              &block_crypto_create_opts_luks,
                                              true);
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_crypto_co_create_opts_luks(const char *filename,
     }
 
     /* Create format layer */
-    ret = block_crypto_co_create_generic(bs, size, create_opts, errp);
+    ret = block_crypto_co_create_generic(bs, size, create_opts, prealloc, errp);
     if (ret < 0) {
         goto fail;
     }
-- 
2.21.0

bdrv_has_zero_init() only has meaning for newly created images or image
areas.  If qemu-img convert did not create the image itself, it cannot
rely on bdrv_has_zero_init()'s result to carry any meaning.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-2-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-img.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ typedef struct ImgConvertState {
     bool has_zero_init;
     bool compressed;
     bool unallocated_blocks_are_zero;
+    bool target_is_new;
     bool target_has_backing;
     int64_t target_backing_sectors; /* negative if unknown */
     bool wr_in_order;
@@ -XXX,XX +XXX,XX @@ static int convert_do_copy(ImgConvertState *s)
     int64_t sector_num = 0;
 
     /* Check whether we have zero initialisation or can get it efficiently */
-    s->has_zero_init = s->min_sparse && !s->target_has_backing
-                     ? bdrv_has_zero_init(blk_bs(s->target))
-                     : false;
+    if (s->target_is_new && s->min_sparse && !s->target_has_backing) {
+        s->has_zero_init = bdrv_has_zero_init(blk_bs(s->target));
+    } else {
+        s->has_zero_init = false;
+    }
 
     if (!s->has_zero_init && !s->target_has_backing &&
         bdrv_can_write_zeroes_with_unmap(blk_bs(s->target)))
@@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv)
         }
     }
 
+    s.target_is_new = !skip_create;
+
     flags = s.min_sparse ? (BDRV_O_RDWR | BDRV_O_UNMAP) : BDRV_O_RDWR;
     ret = bdrv_parse_cache_mode(cache, &flags, &writethrough);
     if (ret < 0) {
-- 
2.21.0

bdrv_has_zero_init() only has meaning for newly created images or image
areas.  If the mirror job itself did not create the image, it cannot
rely on bdrv_has_zero_init()'s result to carry any meaning.

This is the case for drive-mirror with mode=existing and always for
blockdev-mirror.

Note that we only have to zero-initialize the target with sync=full,
because other modes actually do not promise that the target will contain
the same data as the source after the job -- sync=top only promises to
copy anything allocated in the top layer, and sync=none will only copy
new I/O.  (Which is how mirror has always handled it.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-3-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 include/block/block_int.h   |  2 ++
 block/mirror.c              | 11 ++++++++---
 blockdev.c                  | 16 +++++++++++++---
 tests/test-block-iothread.c |  2 +-
 4 files changed, 24 insertions(+), 7 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -XXX,XX +XXX,XX @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
  * @buf_size: The amount of data that can be in flight at one time.
  * @mode: Whether to collapse all images in the chain to the target.
  * @backing_mode: How to establish the target's backing chain after completion.
+ * @zero_target: Whether the target should be explicitly zero-initialized
  * @on_source_error: The action to take upon error reading from the source.
  * @on_target_error: The action to take upon error writing to the target.
  * @unmap: Whether to unmap target where source sectors only contain zeroes.
@@ -XXX,XX +XXX,XX @@ void mirror_start(const char *job_id, BlockDriverState *bs,
                   int creation_flags, int64_t speed,
                   uint32_t granularity, int64_t buf_size,
                   MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
+                  bool zero_target,
                   BlockdevOnError on_source_error,
                   BlockdevOnError on_target_error,
                   bool unmap, const char *filter_node_name,
diff --git a/block/mirror.c b/block/mirror.c
index XXXXXXX..XXXXXXX 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -XXX,XX +XXX,XX @@ typedef struct MirrorBlockJob {
     Error *replace_blocker;
     bool is_none_mode;
     BlockMirrorBackingMode backing_mode;
+    /* Whether the target image requires explicit zero-initialization */
+    bool zero_target;
     MirrorCopyMode copy_mode;
     BlockdevOnError on_source_error, on_target_error;
     bool synced;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s)
     int ret;
     int64_t count;
 
-    if (base == NULL && !bdrv_has_zero_init(target_bs)) {
+    if (s->zero_target) {
         if (!bdrv_can_write_zeroes_with_unmap(target_bs)) {
             bdrv_set_dirty_bitmap(s->dirty_bitmap, 0, s->bdev_length);
             return 0;
@@ -XXX,XX +XXX,XX @@ static BlockJob *mirror_start_job(
                              const char *replaces, int64_t speed,
                              uint32_t granularity, int64_t buf_size,
                              BlockMirrorBackingMode backing_mode,
+                             bool zero_target,
                              BlockdevOnError on_source_error,
                              BlockdevOnError on_target_error,
                              bool unmap,
@@ -XXX,XX +XXX,XX @@ static BlockJob *mirror_start_job(
     s->on_target_error = on_target_error;
     s->is_none_mode = is_none_mode;
     s->backing_mode = backing_mode;
+    s->zero_target = zero_target;
     s->copy_mode = copy_mode;
     s->base = base;
     s->granularity = granularity;
@@ -XXX,XX +XXX,XX @@ void mirror_start(const char *job_id, BlockDriverState *bs,
                   int creation_flags, int64_t speed,
                   uint32_t granularity, int64_t buf_size,
                   MirrorSyncMode mode, BlockMirrorBackingMode backing_mode,
+                  bool zero_target,
                   BlockdevOnError on_source_error,
                   BlockdevOnError on_target_error,
                   bool unmap, const char *filter_node_name,
@@ -XXX,XX +XXX,XX @@ void mirror_start(const char *job_id, BlockDriverState *bs,
     is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
     base = mode == MIRROR_SYNC_MODE_TOP ? backing_bs(bs) : NULL;
     mirror_start_job(job_id, bs, creation_flags, target, replaces,
-                     speed, granularity, buf_size, backing_mode,
+                     speed, granularity, buf_size, backing_mode, zero_target,
                      on_source_error, on_target_error, unmap, NULL, NULL,
                      &mirror_job_driver, is_none_mode, base, false,
                      filter_node_name, true, copy_mode, errp);
@@ -XXX,XX +XXX,XX @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
 
     ret = mirror_start_job(
                      job_id, bs, creation_flags, base, NULL, speed, 0, 0,
-                     MIRROR_LEAVE_BACKING_CHAIN,
+                     MIRROR_LEAVE_BACKING_CHAIN, false,
                      on_error, on_error, true, cb, opaque,
                      &commit_active_job_driver, false, base, auto_complete,
                      filter_node_name, false, MIRROR_COPY_MODE_BACKGROUND,
diff --git a/blockdev.c b/blockdev.c
index XXXXXXX..XXXXXXX 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -XXX,XX +XXX,XX @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
                                    bool has_replaces, const char *replaces,
                                    enum MirrorSyncMode sync,
                                    BlockMirrorBackingMode backing_mode,
+                                   bool zero_target,
                                    bool has_speed, int64_t speed,
                                    bool has_granularity, uint32_t granularity,
                                    bool has_buf_size, int64_t buf_size,
@@ -XXX,XX +XXX,XX @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
      */
     mirror_start(job_id, bs, target,
                  has_replaces ? replaces : NULL, job_flags,
-                 speed, granularity, buf_size, sync, backing_mode,
+                 speed, granularity, buf_size, sync, backing_mode, zero_target,
                  on_source_error, on_target_error, unmap, filter_node_name,
                  copy_mode, errp);
 }
@@ -XXX,XX +XXX,XX @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
     int flags;
     int64_t size;
     const char *format = arg->format;
+    bool zero_target;
     int ret;
 
     bs = qmp_get_root_bs(arg->device, errp);
@@ -XXX,XX +XXX,XX @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
         goto out;
     }
 
+    zero_target = (arg->sync == MIRROR_SYNC_MODE_FULL &&
+                   (arg->mode == NEW_IMAGE_MODE_EXISTING ||
+                    !bdrv_has_zero_init(target_bs)));
+
     ret = bdrv_try_set_aio_context(target_bs, aio_context, errp);
     if (ret < 0) {
         bdrv_unref(target_bs);
@@ -XXX,XX +XXX,XX @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
 
     blockdev_mirror_common(arg->has_job_id ? arg->job_id : NULL, bs, target_bs,
                            arg->has_replaces, arg->replaces, arg->sync,
-                           backing_mode, arg->has_speed, arg->speed,
+                           backing_mode, zero_target,
+                           arg->has_speed, arg->speed,
                            arg->has_granularity, arg->granularity,
                            arg->has_buf_size, arg->buf_size,
                            arg->has_on_source_error, arg->on_source_error,
@@ -XXX,XX +XXX,XX @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
     AioContext *aio_context;
     BlockMirrorBackingMode backing_mode = MIRROR_LEAVE_BACKING_CHAIN;
     Error *local_err = NULL;
+    bool zero_target;
     int ret;
 
     bs = qmp_get_root_bs(device, errp);
@@ -XXX,XX +XXX,XX @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
         return;
     }
 
+    zero_target = (sync == MIRROR_SYNC_MODE_FULL);
+
     aio_context = bdrv_get_aio_context(bs);
     aio_context_acquire(aio_context);
 
@@ -XXX,XX +XXX,XX @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
 
     blockdev_mirror_common(has_job_id ? job_id : NULL, bs, target_bs,
                            has_replaces, replaces, sync, backing_mode,
-                           has_speed, speed,
+                           zero_target, has_speed, speed,
                            has_granularity, granularity,
                            has_buf_size, buf_size,
                            has_on_source_error, on_source_error,
diff --git a/tests/test-block-iothread.c b/tests/test-block-iothread.c
index XXXXXXX..XXXXXXX 100644
--- a/tests/test-block-iothread.c
+++ b/tests/test-block-iothread.c
@@ -XXX,XX +XXX,XX @@ static void test_propagate_mirror(void)
 
     /* Start a mirror job */
     mirror_start("job0", src, target, NULL, JOB_DEFAULT, 0, 0, 0,
-                 MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN,
+                 MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false,
                  BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
                  false, "filter_node", MIRROR_COPY_MODE_BACKGROUND,
                  &error_abort);
-- 
2.21.0

No .bdrv_has_zero_init() implementation returns 1 if growing the file
would add non-zero areas (at least with PREALLOC_MODE_OFF), so using it
in lieu of this new function was always safe.

But on the other hand, it is possible that growing an image that is not
zero-initialized would still add a zero-initialized area, like when
using nonpreallocating truncation on a preallocated image.  For callers
that care only about truncation, not about creation with potential
preallocation, this new function is useful.

Alternatively, we could have added a PreallocMode parameter to
bdrv_has_zero_init().  But the only user would have been qemu-img
convert, which does not have a plain PreallocMode value right now -- it
would have to parse the creation option to obtain it.  Therefore, the
simpler solution is to let bdrv_has_zero_init() inquire the
preallocation status and add the new bdrv_has_zero_init_truncate() that
presupposes PREALLOC_MODE_OFF.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-4-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 include/block/block.h     |  1 +
 include/block/block_int.h |  7 +++++++
 block.c                   | 21 +++++++++++++++++++++
 3 files changed, 29 insertions(+)

diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
 int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
 int bdrv_has_zero_init_1(BlockDriverState *bs);
 int bdrv_has_zero_init(BlockDriverState *bs);
+int bdrv_has_zero_init_truncate(BlockDriverState *bs);
 bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs);
 bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs);
 int bdrv_block_status(BlockDriverState *bs, int64_t offset,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
     /*
      * Returns 1 if newly created images are guaranteed to contain only
      * zeros, 0 otherwise.
+     * Must return 0 if .bdrv_has_zero_init_truncate() returns 0.
      */
     int (*bdrv_has_zero_init)(BlockDriverState *bs);
 
+    /*
+     * Returns 1 if new areas added by growing the image with
+     * PREALLOC_MODE_OFF contain only zeros, 0 otherwise.
+     */
+    int (*bdrv_has_zero_init_truncate)(BlockDriverState *bs);
+
     /* Remove fd handlers, timers, and other event loop callbacks so the event
      * loop is no longer in use.  Called with no in-flight requests and in
      * depth-first traversal order with parents before child nodes.
diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ int bdrv_has_zero_init(BlockDriverState *bs)
     return 0;
 }
 
+int bdrv_has_zero_init_truncate(BlockDriverState *bs)
+{
+    if (!bs->drv) {
+        return 0;
+    }
+
+    if (bs->backing) {
+        /* Depends on the backing image length, but better safe than sorry */
+        return 0;
+    }
+    if (bs->drv->bdrv_has_zero_init_truncate) {
+        return bs->drv->bdrv_has_zero_init_truncate(bs);
+    }
+    if (bs->file && bs->drv->is_filter) {
+        return bdrv_has_zero_init_truncate(bs->file->bs);
+    }
+
+    /* safe default */
+    return 0;
+}
+
 bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs)
 {
     BlockDriverInfo bdi;
-- 
2.21.0

We need to implement .bdrv_has_zero_init_truncate() for every block
driver that supports truncation and has a .bdrv_has_zero_init()
implementation.

Implement it the same way each driver implements .bdrv_has_zero_init().
This is at least not any more unsafe than what we had before.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-5-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/file-posix.c | 1 +
 block/file-win32.c | 1 +
 block/gluster.c    | 4 ++++
 block/nfs.c        | 1 +
 block/qcow2.c      | 1 +
 block/qed.c        | 1 +
 block/raw-format.c | 6 ++++++
 block/rbd.c        | 1 +
 block/sheepdog.c   | 1 +
 block/ssh.c        | 1 +
 10 files changed, 18 insertions(+)

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_file = {
     .bdrv_co_create = raw_co_create,
     .bdrv_co_create_opts = raw_co_create_opts,
     .bdrv_has_zero_init = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
     .bdrv_co_block_status = raw_co_block_status,
     .bdrv_co_invalidate_cache = raw_co_invalidate_cache,
     .bdrv_co_pwrite_zeroes = raw_co_pwrite_zeroes,
diff --git a/block/file-win32.c b/block/file-win32.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-win32.c
+++ b/block/file-win32.c
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_file = {
     .bdrv_close         = raw_close,
     .bdrv_co_create_opts = raw_co_create_opts,
     .bdrv_has_zero_init = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
 
     .bdrv_aio_preadv    = raw_aio_preadv,
     .bdrv_aio_pwritev   = raw_aio_pwritev,
diff --git a/block/gluster.c b/block/gluster.c
index XXXXXXX..XXXXXXX 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster = {
     .bdrv_co_writev               = qemu_gluster_co_writev,
     .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
     .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
+    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
 #ifdef CONFIG_GLUSTERFS_DISCARD
     .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
 #endif
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster_tcp = {
     .bdrv_co_writev               = qemu_gluster_co_writev,
     .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
     .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
+    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
 #ifdef CONFIG_GLUSTERFS_DISCARD
     .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
 #endif
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster_unix = {
     .bdrv_co_writev               = qemu_gluster_co_writev,
     .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
     .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
+    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
 #ifdef CONFIG_GLUSTERFS_DISCARD
     .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
 #endif
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_gluster_rdma = {
     .bdrv_co_writev               = qemu_gluster_co_writev,
     .bdrv_co_flush_to_disk        = qemu_gluster_co_flush_to_disk,
     .bdrv_has_zero_init           = qemu_gluster_has_zero_init,
+    .bdrv_has_zero_init_truncate  = qemu_gluster_has_zero_init,
 #ifdef CONFIG_GLUSTERFS_DISCARD
     .bdrv_co_pdiscard             = qemu_gluster_co_pdiscard,
 #endif
diff --git a/block/nfs.c b/block/nfs.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_nfs = {
     .create_opts                    = &nfs_create_opts,
 
     .bdrv_has_zero_init             = nfs_has_zero_init,
+    .bdrv_has_zero_init_truncate    = nfs_has_zero_init,
     .bdrv_get_allocated_file_size   = nfs_get_allocated_file_size,
     .bdrv_co_truncate               = nfs_file_co_truncate,
 
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
     .bdrv_co_create_opts  = qcow2_co_create_opts,
     .bdrv_co_create       = qcow2_co_create,
     .bdrv_has_zero_init = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
     .bdrv_co_block_status = qcow2_co_block_status,
 
     .bdrv_co_preadv         = qcow2_co_preadv,
diff --git a/block/qed.c b/block/qed.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_qed = {
     .bdrv_co_create           = bdrv_qed_co_create,
     .bdrv_co_create_opts      = bdrv_qed_co_create_opts,
     .bdrv_has_zero_init       = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
     .bdrv_co_block_status     = bdrv_qed_co_block_status,
     .bdrv_co_readv            = bdrv_qed_co_readv,
     .bdrv_co_writev           = bdrv_qed_co_writev,
diff --git a/block/raw-format.c b/block/raw-format.c
index XXXXXXX..XXXXXXX 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -XXX,XX +XXX,XX @@ static int raw_has_zero_init(BlockDriverState *bs)
     return bdrv_has_zero_init(bs->file->bs);
 }
 
+static int raw_has_zero_init_truncate(BlockDriverState *bs)
+{
+    return bdrv_has_zero_init_truncate(bs->file->bs);
+}
+
 static int coroutine_fn raw_co_create_opts(const char *filename, QemuOpts *opts,
                                            Error **errp)
 {
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_raw = {
     .bdrv_co_ioctl        = &raw_co_ioctl,
     .create_opts          = &raw_create_opts,
     .bdrv_has_zero_init   = &raw_has_zero_init,
+    .bdrv_has_zero_init_truncate = &raw_has_zero_init_truncate,
     .strong_runtime_opts  = raw_strong_runtime_opts,
     .mutable_opts         = mutable_opts,
 };
diff --git a/block/rbd.c b/block/rbd.c
index XXXXXXX..XXXXXXX 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_rbd = {
     .bdrv_co_create         = qemu_rbd_co_create,
     .bdrv_co_create_opts    = qemu_rbd_co_create_opts,
     .bdrv_has_zero_init     = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
     .bdrv_get_info          = qemu_rbd_getinfo,
     .create_opts            = &qemu_rbd_create_opts,
     .bdrv_getlength         = qemu_rbd_getlength,
diff --git a/block/sheepdog.c b/block/sheepdog.c
index XXXXXXX..XXXXXXX 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_sheepdog = {
     .bdrv_co_create               = sd_co_create,
     .bdrv_co_create_opts          = sd_co_create_opts,
     .bdrv_has_zero_init           = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init_truncate  = bdrv_has_zero_init_1,
     .bdrv_getlength               = sd_getlength,
     .bdrv_get_allocated_file_size = sd_get_allocated_file_size,
     .bdrv_co_truncate             = sd_co_truncate,
diff --git a/block/ssh.c b/block/ssh.c
index XXXXXXX..XXXXXXX 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_ssh = {
     .bdrv_co_create_opts          = ssh_co_create_opts,
     .bdrv_close                   = ssh_close,
     .bdrv_has_zero_init           = ssh_has_zero_init,
+    .bdrv_has_zero_init_truncate  = ssh_has_zero_init,
     .bdrv_co_readv                = ssh_co_readv,
     .bdrv_co_writev               = ssh_co_writev,
     .bdrv_getlength               = ssh_getlength,
-- 
2.21.0

vhdx and parallels call bdrv_has_zero_init() when they do not really
care about an image's post-create state but only about what happens when
you grow an image.  That is a bit ugly, and also overly safe when
growing preallocated images without preallocating the new areas.

Let them use bdrv_has_zero_init_truncate() instead.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-6-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
[mreitz: Added commit message]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/parallels.c | 2 +-
 block/vhdx.c      | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/parallels.c b/block/parallels.c
index XXXXXXX..XXXXXXX 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -XXX,XX +XXX,XX @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags,
         goto fail_options;
     }
 
-    if (!bdrv_has_zero_init(bs->file->bs)) {
+    if (!bdrv_has_zero_init_truncate(bs->file->bs)) {
         s->prealloc_mode = PRL_PREALLOC_MODE_FALLOCATE;
     }
 
diff --git a/block/vhdx.c b/block/vhdx.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int vhdx_co_writev(BlockDriverState *bs, int64_t sector_num,
                 /* Queue another write of zero buffers if the underlying file
                  * does not zero-fill on file extension */
 
-                if (bdrv_has_zero_init(bs->file->bs) == 0) {
+                if (bdrv_has_zero_init_truncate(bs->file->bs) == 0) {
                     use_zero_buffers = true;
 
                     /* zero fill the front, if any */
-- 
2.21.0

If a qcow2 file is preallocated, it can no longer guarantee that it
initially appears as filled with zeroes.

So implement .bdrv_has_zero_init() by checking whether the file is
preallocated; if so, forward the call to the underlying storage node,
except for when it is encrypted: Encrypted preallocated images always
return effectively random data, so .bdrv_has_zero_init() must always
return 0 for them.

.bdrv_has_zero_init_truncate() can remain bdrv_has_zero_init_1(),
because it presupposes PREALLOC_MODE_OFF.

Reported-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-7-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qcow2.c | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs,
     return spec_info;
 }
 
+static int qcow2_has_zero_init(BlockDriverState *bs)
+{
+    BDRVQcow2State *s = bs->opaque;
+    bool preallocated;
+
+    if (qemu_in_coroutine()) {
+        qemu_co_mutex_lock(&s->lock);
+    }
+    /*
+     * Check preallocation status: Preallocated images have all L2
+     * tables allocated, nonpreallocated images have none.  It is
+     * therefore enough to check the first one.
+     */
+    preallocated = s->l1_size > 0 && s->l1_table[0] != 0;
+    if (qemu_in_coroutine()) {
+        qemu_co_mutex_unlock(&s->lock);
+    }
+
+    if (!preallocated) {
+        return 1;
+    } else if (bs->encrypted) {
+        return 0;
+    } else {
+        return bdrv_has_zero_init(s->data_file->bs);
+    }
+}
+
 static int qcow2_save_vmstate(BlockDriverState *bs, QEMUIOVector *qiov,
                               int64_t pos)
 {
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_qcow2 = {
     .bdrv_child_perm      = bdrv_format_default_perms,
     .bdrv_co_create_opts  = qcow2_co_create_opts,
     .bdrv_co_create       = qcow2_co_create,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init   = qcow2_has_zero_init,
     .bdrv_has_zero_init_truncate = bdrv_has_zero_init_1,
     .bdrv_co_block_status = qcow2_co_block_status,
 
-- 
2.21.0

Static VDI images cannot guarantee to be zero-initialized.  If the image
has been statically allocated, forward the call to the underlying
storage node.

Reported-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Tested-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20190724171239.8764-8-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vdi.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/block/vdi.c b/block/vdi.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vdi.c
+++ b/block/vdi.c
@@ -XXX,XX +XXX,XX @@ static void vdi_close(BlockDriverState *bs)
     error_free(s->migration_blocker);
 }
 
+static int vdi_has_zero_init(BlockDriverState *bs)
+{
+    BDRVVdiState *s = bs->opaque;
+
+    if (s->header.image_type == VDI_TYPE_STATIC) {
+        return bdrv_has_zero_init(bs->file->bs);
+    } else {
+        return 1;
+    }
+}
+
 static QemuOptsList vdi_create_opts = {
     .name = "vdi-create-opts",
     .head = QTAILQ_HEAD_INITIALIZER(vdi_create_opts.head),
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_vdi = {
     .bdrv_child_perm          = bdrv_format_default_perms,
     .bdrv_co_create      = vdi_co_create,
     .bdrv_co_create_opts = vdi_co_create_opts,
-    .bdrv_has_zero_init = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init  = vdi_has_zero_init,
     .bdrv_co_block_status = vdi_co_block_status,
     .bdrv_make_empty = vdi_make_empty,
 
-- 
2.21.0

Fixed VHDX images cannot guarantee to be zero-initialized.  If the image
has the "fixed" subformat, forward the call to the underlying storage
node.

Reported-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-9-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vhdx.c | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/block/vhdx.c b/block/vhdx.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vhdx.c
+++ b/block/vhdx.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vhdx_co_check(BlockDriverState *bs,
     return 0;
 }
 
+static int vhdx_has_zero_init(BlockDriverState *bs)
+{
+    BDRVVHDXState *s = bs->opaque;
+    int state;
+
+    /*
+     * Check the subformat: Fixed images have all BAT entries present,
+     * dynamic images have none (right after creation).  It is
+     * therefore enough to check the first BAT entry.
+     */
+    if (!s->bat_entries) {
+        return 1;
+    }
+
+    state = s->bat[0] & VHDX_BAT_STATE_BIT_MASK;
+    if (state == PAYLOAD_BLOCK_FULLY_PRESENT) {
+        /* Fixed subformat */
+        return bdrv_has_zero_init(bs->file->bs);
+    }
+
+    /* Dynamic subformat */
+    return 1;
+}
+
 static QemuOptsList vhdx_create_opts = {
     .name = "vhdx-create-opts",
     .head = QTAILQ_HEAD_INITIALIZER(vhdx_create_opts.head),
@@ -XXX,XX +XXX,XX @@ static BlockDriver bdrv_vhdx = {
     .bdrv_co_create_opts    = vhdx_co_create_opts,
     .bdrv_get_info          = vhdx_get_info,
     .bdrv_co_check          = vhdx_co_check,
-    .bdrv_has_zero_init     = bdrv_has_zero_init_1,
+    .bdrv_has_zero_init     = vhdx_has_zero_init,
 
     .create_opts            = &vhdx_create_opts,
 };
-- 
2.21.0

Add a test case for converting an empty image (which only returns zeroes
when read) to a preallocated encrypted qcow2 image.
qcow2_has_zero_init() should return 0 then, thus forcing qemu-img
convert to create zero clusters.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Tested-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20190724171239.8764-10-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/188     | 20 +++++++++++++++++++-
 tests/qemu-iotests/188.out |  4 ++++
 2 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/188 b/tests/qemu-iotests/188
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/188
+++ b/tests/qemu-iotests/188
@@ -XXX,XX +XXX,XX @@ SECRETALT="secret,id=sec0,data=platypus"
 
 _make_test_img --object $SECRET -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10" $size
 
-IMGSPEC="driver=$IMGFMT,file.filename=$TEST_IMG,encrypt.key-secret=sec0"
+IMGSPEC="driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
 
 QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
 
@@ -XXX,XX +XXX,XX @@ echo
 echo "== verify open failure with wrong password =="
 $QEMU_IO --object $SECRETALT -c "read -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
 
+_cleanup_test_img
+
+echo
+echo "== verify that has_zero_init returns false when preallocating =="
+
+# Empty source file
+if [ -n "$TEST_IMG_FILE" ]; then
+    TEST_IMG_FILE="${TEST_IMG_FILE}.orig" _make_test_img $size
+else
+    TEST_IMG="${TEST_IMG}.orig" _make_test_img $size
+fi
+
+$QEMU_IMG convert -O "$IMGFMT" --object $SECRET \
+    -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10,preallocation=metadata" \
+    "${TEST_IMG}.orig" "$TEST_IMG"
+
+$QEMU_IMG compare --object $SECRET --image-opts "${IMGSPEC}.orig" "$IMGSPEC"
+
 
 # success, all done
 echo "*** done"
diff --git a/tests/qemu-iotests/188.out b/tests/qemu-iotests/188.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/188.out
+++ b/tests/qemu-iotests/188.out
@@ -XXX,XX +XXX,XX @@ read 16777216/16777216 bytes at offset 0
 
 == verify open failure with wrong password ==
 qemu-io: can't open: Invalid password, cannot unlock any keyslot
+
+== verify that has_zero_init returns false when preallocating ==
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=16777216
+Images are identical.
 *** done
-- 
2.21.0

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-11-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/122     | 17 +++++++++++++++++
 tests/qemu-iotests/122.out |  8 ++++++++
 2 files changed, 25 insertions(+)

diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/122
+++ b/tests/qemu-iotests/122
@@ -XXX,XX +XXX,XX @@ for min_sparse in 4k 8k; do
     $QEMU_IMG map --output=json "$TEST_IMG".orig | _filter_qemu_img_map
 done
 
+
+echo
+echo '=== -n to a non-zero image ==='
+echo
+
+# Keep source zero
+_make_test_img 64M
+
+# Output is not zero, but has bdrv_has_zero_init() == 1
+TEST_IMG="$TEST_IMG".orig _make_test_img 64M
+$QEMU_IO -c "write -P 42 0 64k" "$TEST_IMG".orig | _filter_qemu_io
+
+# Convert with -n, which should not assume that the target is zeroed
+$QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
+
+$QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
+
 # success, all done
 echo '*** done'
 rm -f $seq.full
diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/122.out
+++ b/tests/qemu-iotests/122.out
@@ -XXX,XX +XXX,XX @@ convert -c -S 8k
 { "start": 9216, "length": 8192, "depth": 0, "zero": true, "data": false},
 { "start": 17408, "length": 1024, "depth": 0, "zero": false, "data": true},
 { "start": 18432, "length": 67090432, "depth": 0, "zero": true, "data": false}]
+
+=== -n to a non-zero image ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
+wrote 65536/65536 bytes at offset 0
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Images are identical.
 *** done
-- 
2.21.0

The result of a sync=full mirror should always be the equal to the
input.  Therefore, existing images should be treated as potentially
non-zero and thus should be explicitly initialized to be zero
beforehand.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190724171239.8764-12-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/041     | 62 +++++++++++++++++++++++++++++++++++---
 tests/qemu-iotests/041.out |  4 +--
 2 files changed, 60 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/041
+++ b/tests/qemu-iotests/041
@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
     def setUp(self):
         qemu_img('create', '-f', iotests.imgfmt, test_img,
                  str(TestUnbackedSource.image_len))
-        self.vm = iotests.VM().add_drive(test_img)
+        self.vm = iotests.VM()
         self.vm.launch()
+        result = self.vm.qmp('blockdev-add', node_name='drive0',
+                             driver=iotests.imgfmt,
+                             file={
+                                 'driver': 'file',
+                                 'filename': test_img,
+                             })
+        self.assert_qmp(result, 'return', {})
 
     def tearDown(self):
         self.vm.shutdown()
@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
 
     def test_absolute_paths_full(self):
         self.assert_no_active_block_jobs()
-        result = self.vm.qmp('drive-mirror', device='drive0',
+        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
                              sync='full', target=target_img,
                              mode='absolute-paths')
         self.assert_qmp(result, 'return', {})
@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
 
     def test_absolute_paths_top(self):
         self.assert_no_active_block_jobs()
-        result = self.vm.qmp('drive-mirror', device='drive0',
+        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
                              sync='top', target=target_img,
                              mode='absolute-paths')
         self.assert_qmp(result, 'return', {})
@@ -XXX,XX +XXX,XX @@ class TestUnbackedSource(iotests.QMPTestCase):
 
     def test_absolute_paths_none(self):
         self.assert_no_active_block_jobs()
-        result = self.vm.qmp('drive-mirror', device='drive0',
+        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
                              sync='none', target=target_img,
                              mode='absolute-paths')
         self.assert_qmp(result, 'return', {})
         self.complete_and_wait()
         self.assert_no_active_block_jobs()
 
+    def test_existing_full(self):
+        qemu_img('create', '-f', iotests.imgfmt, target_img,
+                 str(self.image_len))
+        qemu_io('-c', 'write -P 42 0 64k', target_img)
+
+        self.assert_no_active_block_jobs()
+        result = self.vm.qmp('drive-mirror', job_id='drive0', device='drive0',
+                             sync='full', target=target_img, mode='existing')
+        self.assert_qmp(result, 'return', {})
+        self.complete_and_wait()
+        self.assert_no_active_block_jobs()
+
+        result = self.vm.qmp('blockdev-del', node_name='drive0')
+        self.assert_qmp(result, 'return', {})
+
+        self.assertTrue(iotests.compare_images(test_img, target_img),
+                        'target image does not match source after mirroring')
+
+    def test_blockdev_full(self):
+        qemu_img('create', '-f', iotests.imgfmt, target_img,
+                 str(self.image_len))
+        qemu_io('-c', 'write -P 42 0 64k', target_img)
+
+        result = self.vm.qmp('blockdev-add', node_name='target',
+                             driver=iotests.imgfmt,
+                             file={
+                                 'driver': 'file',
+                                 'filename': target_img,
+                             })
+        self.assert_qmp(result, 'return', {})
+
+        self.assert_no_active_block_jobs()
+        result = self.vm.qmp('blockdev-mirror', job_id='drive0', device='drive0',
+                             sync='full', target='target')
+        self.assert_qmp(result, 'return', {})
+        self.complete_and_wait()
+        self.assert_no_active_block_jobs()
+
+        result = self.vm.qmp('blockdev-del', node_name='drive0')
+        self.assert_qmp(result, 'return', {})
+
+        result = self.vm.qmp('blockdev-del', node_name='target')
+        self.assert_qmp(result, 'return', {})
+
+        self.assertTrue(iotests.compare_images(test_img, target_img),
+                        'target image does not match source after mirroring')
+
 class TestGranularity(iotests.QMPTestCase):
     image_len = 10 * 1024 * 1024 # MB
 
diff --git a/tests/qemu-iotests/041.out b/tests/qemu-iotests/041.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/041.out
+++ b/tests/qemu-iotests/041.out
@@ -XXX,XX +XXX,XX @@
-........................................................................................
+..........................................................................................
 ----------------------------------------------------------------------
-Ran 88 tests
+Ran 90 tests
 
 OK
-- 
2.21.0

vpc is not really a passthrough driver, even when using the fixed
subformat (where host and guest offsets are equal).  It should handle
preallocation like all other drivers do, namely by returning
DATA | RECURSE instead of RAW.

There is no tangible difference but the fact that bdrv_is_allocated() no
longer falls through to the protocol layer.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190725155512.9827-4-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vpc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/vpc.c b/block/vpc.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn vpc_co_block_status(BlockDriverState *bs,
         *pnum = bytes;
         *map = offset;
         *file = bs->file->bs;
-        return BDRV_BLOCK_RAW | BDRV_BLOCK_OFFSET_VALID;
+        return BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_RECURSE;
     }
 
     qemu_co_mutex_lock(&s->lock);
-- 
2.21.0

69f47505ee has changed qcow2 in such a way that the commit job run in
test 141 (and 144[1]) returns before it emits the READY event.  However,
141 also runs with qed, where the order is still the other way around.
Just filter out the {"return": {}} so the test passes for qed again.

[1] 144 only runs with qcow2, so it is fine as it is.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190809185253.17535-1-mreitz@redhat.com
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/141           | 9 +++++++--
 tests/qemu-iotests/141.out       | 5 -----
 tests/qemu-iotests/common.filter | 5 +++++
 3 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/141
+++ b/tests/qemu-iotests/141
@@ -XXX,XX +XXX,XX @@ test_blockjob()
               }}}" \
         'return'
 
+    # If "$2" is an event, we may or may not see it before the
+    # {"return": {}}.  Therefore, filter the {"return": {}} out both
+    # here and in the next command.  (Naturally, if we do not see it
+    # here, we will see it before the next command can be executed,
+    # so it will appear in the next _send_qemu_cmd's output.)
     _send_qemu_cmd $QEMU_HANDLE \
         "$1" \
         "$2" \
-        | _filter_img_create
+        | _filter_img_create | _filter_qmp_empty_return
 
     # We want this to return an error because the block job is still running
     _send_qemu_cmd $QEMU_HANDLE \
         "{'execute': 'blockdev-del',
           'arguments': {'node-name': 'drv0'}}" \
-        'error' | _filter_generated_node_ids
+        'error' | _filter_generated_node_ids | _filter_qmp_empty_return
 
     _send_qemu_cmd $QEMU_HANDLE \
         "{'execute': 'block-job-cancel',
diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/141.out
+++ b/tests/qemu-iotests/141.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/m.
 Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
-{"return": {}}
 {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}}
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}}
-{"return": {}}
 {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: mirror"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}}
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
-{"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}
 {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: block device is in use by block job: commit"}}
@@ -XXX,XX +XXX,XX @@ wrote 1048576/1048576 bytes at offset 0
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
-{"return": {}}
 {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}}
@@ -XXX,XX +XXX,XX @@ wrote 1048576/1048576 bytes at offset 0
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
-{"return": {}}
 {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}}
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_nbd()
         -e 's#$foo\|PORT/\?\|.sock$: Failed to .*$#\1#'
 }
 
+_filter_qmp_empty_return()
+{
+    grep -v '{"return": {}}'
+}
+
 # make sure this script returns success
 true
-- 
2.21.0

When preallocating an encrypted qcow2 image, it just lets the protocol
driver write data and then does not mark the clusters as zero.
Therefore, reading this image will yield effectively random data.

As such, we have not fulfilled the promise of always writing zeroes when
preallocating an image in a while.  It seems that nobody has really
cared, so change the documentation to conform to qemu's actual behavior.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190711132935.13070-1-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qapi/block-core.json         | 9 +++++----
 docs/qemu-block-drivers.texi | 4 ++--
 qemu-img.texi                | 4 ++--
 3 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
 # @off: no preallocation
 # @metadata: preallocate only for metadata
 # @falloc: like @full preallocation but allocate disk space by
-#          posix_fallocate() rather than writing zeros.
-# @full: preallocate all data by writing zeros to device to ensure disk
-#        space is really available. @full preallocation also sets up
-#        metadata correctly.
+#          posix_fallocate() rather than writing data.
+# @full: preallocate all data by writing it to the device to ensure
+#        disk space is really available. This data may or may not be
+#        zero, depending on the image format and storage.
+#        @full preallocation also sets up metadata correctly.
 #
 # Since: 2.2
 ##
diff --git a/docs/qemu-block-drivers.texi b/docs/qemu-block-drivers.texi
index XXXXXXX..XXXXXXX 100644
--- a/docs/qemu-block-drivers.texi
+++ b/docs/qemu-block-drivers.texi
@@ -XXX,XX +XXX,XX @@ Supported options:
 @item preallocation
 Preallocation mode (allowed values: @code{off}, @code{falloc}, @code{full}).
 @code{falloc} mode preallocates space for image by calling posix_fallocate().
-@code{full} mode preallocates space for image by writing zeros to underlying
-storage.
+@code{full} mode preallocates space for image by writing data to underlying
+storage.  This data may or may not be zero, depending on the storage location.
 @end table
 
 @item qcow2
diff --git a/qemu-img.texi b/qemu-img.texi
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -XXX,XX +XXX,XX @@ Supported options:
 @item preallocation
 Preallocation mode (allowed values: @code{off}, @code{falloc}, @code{full}).
 @code{falloc} mode preallocates space for image by calling posix_fallocate().
-@code{full} mode preallocates space for image by writing zeros to underlying
-storage.
+@code{full} mode preallocates space for image by writing data to underlying
+storage.  This data may or may not be zero, depending on the storage location.
 @end table
 
 @item qcow2
-- 
2.21.0

The following changes since commit 23919ddfd56135cad3cb468a8f54d5a595f024f4:

Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20190827' into staging (2019-08-27 15:52:36 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2019-08-27

for you to fetch changes up to bb043c056cffcc2f3ce88bfdaf2e76e455c09e2c:

iotests: Unify cache mode quoting (2019-08-27 19:48:44 +0200)

----------------------------------------------------------------
Block patches:
- qemu-io now accepts a file to read a write pattern from
- Ensure that raw files have their first block allocated so we can probe
  the O_DIRECT alignment if necessary
- Various fixes

----------------------------------------------------------------
Denis Plotnikov (1):
  qemu-io: add pattern file for write command

Max Reitz (7):
  iotests: Fix _filter_img_create()
  vmdk: Use bdrv_dirname() for relative extent paths
  iotests: Keep testing broken relative extent paths
  vmdk: Reject invalid compressed writes
  iotests: Disable broken streamOptimized tests
  iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
  iotests: Disable 126 for flat vmdk subformats

Nir Soffer (3):
  block: posix: Always allocate the first block
  iotests: Test allocate_first_block() with O_DIRECT
  iotests: Unify cache mode quoting

Stefan Hajnoczi (1):
  file-posix: fix request_alignment typo

Thomas Huth (2):
  iotests: Check for enabled drivers before testing them
  tests/check-block: Skip iotests when sanitizers are enabled

Vladimir Sementsov-Ogievskiy (1):
  block: fix permission update in bdrv_replace_node

block.c                                       |  5 +-
 block/file-posix.c                            | 53 +++++++++-
 block/vmdk.c                                  | 64 ++++++++----
 qemu-io-cmds.c                                | 99 +++++++++++++++++--
 tests/check-block.sh                          |  5 +
 tests/qemu-iotests/002                        |  1 +
 tests/qemu-iotests/003                        |  1 +
 tests/qemu-iotests/005                        |  3 +-
 tests/qemu-iotests/009                        |  1 +
 tests/qemu-iotests/010                        |  1 +
 tests/qemu-iotests/011                        |  1 +
 tests/qemu-iotests/017                        |  3 +-
 tests/qemu-iotests/018                        |  3 +-
 tests/qemu-iotests/019                        |  3 +-
 tests/qemu-iotests/020                        |  3 +-
 tests/qemu-iotests/026                        |  4 +-
 tests/qemu-iotests/027                        |  1 +
 tests/qemu-iotests/032                        |  1 +
 tests/qemu-iotests/033                        |  1 +
 tests/qemu-iotests/034                        |  3 +-
 tests/qemu-iotests/037                        |  3 +-
 tests/qemu-iotests/039                        |  4 +-
 tests/qemu-iotests/052                        |  2 +-
 tests/qemu-iotests/059                        | 34 ++++++-
 tests/qemu-iotests/059.out                    | 26 +++--
 tests/qemu-iotests/063                        |  3 +-
 tests/qemu-iotests/071                        |  1 +
 tests/qemu-iotests/072                        |  1 +
 tests/qemu-iotests/081                        |  4 +-
 tests/qemu-iotests/091                        |  4 +-
 tests/qemu-iotests/099                        |  1 +
 tests/qemu-iotests/105                        |  3 +-
 tests/qemu-iotests/110                        |  3 +-
 tests/qemu-iotests/120                        |  1 +
 tests/qemu-iotests/126                        |  2 +
 tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
 tests/qemu-iotests/150.out.raw                | 12 +++
 tests/qemu-iotests/162                        |  4 +-
 tests/qemu-iotests/175                        | 47 +++++++--
 tests/qemu-iotests/175.out                    | 16 ++-
 tests/qemu-iotests/178.out.qcow2              |  4 +-
 tests/qemu-iotests/184                        |  1 +
 tests/qemu-iotests/186                        |  1 +
 tests/qemu-iotests/197                        |  1 +
 tests/qemu-iotests/215                        |  1 +
 tests/qemu-iotests/221.out                    | 12 ++-
 tests/qemu-iotests/251                        |  1 +
 tests/qemu-iotests/253.out                    | 12 ++-
 tests/qemu-iotests/common.filter              |  4 +-
 tests/qemu-iotests/common.rc                  | 14 +++
 50 files changed, 391 insertions(+), 87 deletions(-)
 rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
 create mode 100644 tests/qemu-iotests/150.out.raw

-- 
2.21.0

From: Denis Plotnikov <dplotnikov@virtuozzo.com>

The patch allows to provide a pattern file for write
command. There was no similar ability before.

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Message-id: 20190820164616.4072-1-dplotnikov@virtuozzo.com
Reviewed-by: Eric Blake <eblake@redhat.com>
[mreitz: Keep optstring in alphabetical order]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-io-cmds.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 93 insertions(+), 6 deletions(-)

diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -XXX,XX +XXX,XX @@ static void qemu_io_free(void *p)
     qemu_vfree(p);
 }
 
+/*
+ * qemu_io_alloc_from_file()
+ *
+ * Allocates the buffer and populates it with the content of the given file
+ * up to @len bytes. If the file length is less than @len, then the buffer
+ * is populated with the file content cyclically.
+ *
+ * @blk - the block backend where the buffer content is going to be written to
+ * @len - the buffer length
+ * @file_name - the file to read the content from
+ *
+ * Returns: the buffer pointer on success
+ *          NULL on error
+ */
+static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
+                                     const char *file_name)
+{
+    char *buf, *buf_origin;
+    FILE *f = fopen(file_name, "r");
+    int pattern_len;
+
+    if (!f) {
+        perror(file_name);
+        return NULL;
+    }
+
+    if (qemuio_misalign) {
+        len += MISALIGN_OFFSET;
+    }
+
+    buf_origin = buf = blk_blockalign(blk, len);
+
+    if (qemuio_misalign) {
+        buf_origin += MISALIGN_OFFSET;
+        buf += MISALIGN_OFFSET;
+        len -= MISALIGN_OFFSET;
+    }
+
+    pattern_len = fread(buf_origin, 1, len, f);
+
+    if (ferror(f)) {
+        perror(file_name);
+        goto error;
+    }
+
+    if (pattern_len == 0) {
+        fprintf(stderr, "%s: file is empty\n", file_name);
+        goto error;
+    }
+
+    fclose(f);
+
+    if (len > pattern_len) {
+        len -= pattern_len;
+        buf += pattern_len;
+
+        while (len > 0) {
+            size_t len_to_copy = MIN(pattern_len, len);
+
+            memcpy(buf, buf_origin, len_to_copy);
+
+            len -= len_to_copy;
+            buf += len_to_copy;
+        }
+    }
+
+    return buf_origin;
+
+error:
+    qemu_io_free(buf_origin);
+    return NULL;
+}
+
 static void dump_buffer(const void *buffer, int64_t offset, int64_t len)
 {
     uint64_t i;
@@ -XXX,XX +XXX,XX @@ static void write_help(void)
 " -n, -- with -z, don't allow slow fallback\n"
 " -p, -- ignored for backwards compatibility\n"
 " -P, -- use different pattern to fill file\n"
+" -s, -- use a pattern file to fill the write buffer\n"
 " -C, -- report statistics in a machine parsable format\n"
 " -q, -- quiet mode, do not show I/O statistics\n"
 " -u, -- with -z, allow unmapping\n"
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t write_cmd = {
     .perm       = BLK_PERM_WRITE,
     .argmin     = 2,
     .argmax     = -1,
-    .args       = "[-bcCfnquz] [-P pattern] off len",
+    .args       = "[-bcCfnquz] [-P pattern | -s source_file] off len",
     .oneline    = "writes a number of bytes at a specified offset",
     .help       = write_help,
 };
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
 {
     struct timespec t1, t2;
     bool Cflag = false, qflag = false, bflag = false;
-    bool Pflag = false, zflag = false, cflag = false;
+    bool Pflag = false, zflag = false, cflag = false, sflag = false;
     int flags = 0;
     int c, cnt, ret;
     char *buf = NULL;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
     /* Some compilers get confused and warn if this is not initialized.  */
     int64_t total = 0;
     int pattern = 0xcd;
+    const char *file_name = NULL;
 
-    while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) {
+    while ((c = getopt(argc, argv, "bcCfnpP:qs:uz")) != -1) {
         switch (c) {
         case 'b':
             bflag = true;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
         case 'q':
             qflag = true;
             break;
+        case 's':
+            sflag = true;
+            file_name = optarg;
+            break;
         case 'u':
             flags |= BDRV_REQ_MAY_UNMAP;
             break;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
         return -EINVAL;
     }
 
-    if (zflag && Pflag) {
-        printf("-z and -P cannot be specified at the same time\n");
+    if (zflag + Pflag + sflag > 1) {
+        printf("Only one of -z, -P, and -s "
+               "can be specified at the same time\n");
         return -EINVAL;
     }
 
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
     }
 
     if (!zflag) {
-        buf = qemu_io_alloc(blk, count, pattern);
+        if (sflag) {
+            buf = qemu_io_alloc_from_file(blk, count, file_name);
+            if (!buf) {
+                return -EINVAL;
+            }
+        } else {
+            buf = qemu_io_alloc(blk, count, pattern);
+        }
     }
 
     clock_gettime(CLOCK_MONOTONIC, &t1);
-- 
2.21.0

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

It's wrong to OR shared permissions. It may lead to crash on further
permission updates.
Also, no needs to consider previously calculated permissions, as at
this point we already bind all new parents and bdrv_get_cumulative_perm
result is enough. So fix the bug by just set permissions by
bdrv_get_cumulative_perm result.

Bug was introduced in long ago 234ac1a9025, in 2.9.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190824100740.61635-1-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
 {
     BdrvChild *c, *next;
     GSList *list = NULL, *p;
-    uint64_t old_perm, old_shared;
     uint64_t perm = 0, shared = BLK_PERM_ALL;
     int ret;
 
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
         bdrv_unref(from);
     }
 
-    bdrv_get_cumulative_perm(to, &old_perm, &old_shared);
-    bdrv_set_perm(to, old_perm | perm, old_shared | shared);
+    bdrv_get_cumulative_perm(to, &perm, &shared);
+    bdrv_set_perm(to, perm, shared);
 
 out:
     g_slist_free(list);
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster
storage backed by XFS filesystem, reading this block using direct I/O
succeeds regardless of request length, fooling alignment detection.

In this case we fallback to a safe value (4096) instead of the optimal
value (512), which may lead to unneeded data copying when aligning
requests.  Allocating the first block avoids the fallback.

Since we allocate the first block even with preallocation=off, we no
longer create images with zero disk size:

$ ./qemu-img create -f raw test.raw 1g
    Formatting 'test.raw', fmt=raw size=1073741824

$ ls -lhs test.raw
    4.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw

And converting the image requires additional cluster:

$ ./qemu-img measure -f raw -O qcow2 test.raw
    required size: 458752
    fully allocated size: 1074135040

When using format like vmdk with multiple files per image, we allocate
one block per file:

$ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
    Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

$ ls -lhs test*.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer  353 Aug 27 03:23 test.vmdk

I did quick performance test for copying disks with qemu-img convert to
new raw target image to Gluster storage with sector size of 512 bytes:

for i in $(seq 10); do
        rm -f dst.raw
        sleep 10
        time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
    done

Here is a table comparing the total time spent:

Type    Before(s)   After(s)    Diff(%)
---------------------------------------
real      530.028    469.123      -11.4
user       17.204     10.768      -37.4
sys        17.881      7.011      -60.7

We can see very clear improvement in CPU usage.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827010528.8818-2-nsoffer@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/file-posix.c                            | 51 +++++++++++++++++++
 tests/qemu-iotests/059.out                    |  2 +-
 tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
 tests/qemu-iotests/150.out.raw                | 12 +++++
 tests/qemu-iotests/175                        | 19 ++++---
 tests/qemu-iotests/175.out                    |  8 +--
 tests/qemu-iotests/178.out.qcow2              |  4 +-
 tests/qemu-iotests/221.out                    | 12 +++--
 tests/qemu-iotests/253.out                    | 12 +++--
 9 files changed, 99 insertions(+), 21 deletions(-)
 rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
 create mode 100644 tests/qemu-iotests/150.out.raw

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_discard(void *opaque)
     return ret;
 }
 
+/*
+ * Help alignment probing by allocating the first block.
+ *
+ * When reading with direct I/O from unallocated area on Gluster backed by XFS,
+ * reading succeeds regardless of request length. In this case we fallback to
+ * safe alignment which is not optimal. Allocating the first block avoids this
+ * fallback.
+ *
+ * fd may be opened with O_DIRECT, but we don't know the buffer alignment or
+ * request alignment, so we use safe values.
+ *
+ * Returns: 0 on success, -errno on failure. Since this is an optimization,
+ * caller may ignore failures.
+ */
+static int allocate_first_block(int fd, size_t max_size)
+{
+    size_t write_size = (max_size < MAX_BLOCKSIZE)
+        ? BDRV_SECTOR_SIZE
+        : MAX_BLOCKSIZE;
+    size_t max_align = MAX(MAX_BLOCKSIZE, getpagesize());
+    void *buf;
+    ssize_t n;
+    int ret;
+
+    buf = qemu_memalign(max_align, write_size);
+    memset(buf, 0, write_size);
+
+    do {
+        n = pwrite(fd, buf, write_size, 0);
+    } while (n == -1 && errno == EINTR);
+
+    ret = (n == -1) ? -errno : 0;
+
+    qemu_vfree(buf);
+    return ret;
+}
+
 static int handle_aiocb_truncate(void *opaque)
 {
     RawPosixAIOData *aiocb = opaque;
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
                 /* posix_fallocate() doesn't set errno. */
                 error_setg_errno(errp, -result,
                                  "Could not preallocate new data");
+            } else if (current_length == 0) {
+                /*
+                 * posix_fallocate() uses fallocate() if the filesystem
+                 * supports it, or fallback to manually writing zeroes. If
+                 * fallocate() was used, unaligned reads from the fallocated
+                 * area in raw_probe_alignment() will succeed, hence we need to
+                 * allocate the first block.
+                 *
+                 * Optimize future alignment probing; ignore failures.
+                 */
+                allocate_first_block(fd, offset);
             }
         } else {
             result = 0;
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
         if (ftruncate(fd, offset) != 0) {
             result = -errno;
             error_setg_errno(errp, -result, "Could not resize file");
+        } else if (current_length == 0 && offset > current_length) {
+            /* Optimize future alignment probing; ignore failures. */
+            allocate_first_block(fd, offset);
         }
         return result;
     default:
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMax
 image: TEST_DIR/t.vmdk
 file format: vmdk
 virtual size: 0.977 TiB (1073741824000 bytes)
-disk size: 16 KiB
+disk size: 1.97 MiB
 Format specific information:
     cid: XXXXXXXX
     parent cid: XXXXXXXX
diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out.qcow2
similarity index 100%
rename from tests/qemu-iotests/150.out
rename to tests/qemu-iotests/150.out.qcow2
diff --git a/tests/qemu-iotests/150.out.raw b/tests/qemu-iotests/150.out.raw
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/150.out.raw
@@ -XXX,XX +XXX,XX @@
+QA output created by 150
+
+=== Mapping sparse conversion ===
+
+Offset          Length          File
+0               0x1000          TEST_DIR/t.IMGFMT
+
+=== Mapping non-sparse conversion ===
+
+Offset          Length          File
+0               0x100000        TEST_DIR/t.IMGFMT
+*** done
diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # the file size.  This function hides the resulting difference in the
 # stat -c '%b' output.
 # Parameter 1: Number of blocks an empty file occupies
-# Parameter 2: Image size in bytes
+# Parameter 2: Minimal number of blocks in an image
+# Parameter 3: Image size in bytes
 _filter_blocks()
 {
     extra_blocks=$1
-    img_size=$2
+    min_blocks=$2
+    img_size=$3
 
-    sed -e "s/blocks=$extra_blocks\$\$\\|[^0-9]\$/nothing allocated/" \
-        -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/everything allocated/"
+    sed -e "s/blocks=$min_blocks\$\$\\|[^0-9]\$/min allocation/" \
+        -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/max allocation/"
 }
 
 # get standard environment, filters and checks
@@ -XXX,XX +XXX,XX @@ size=$((1 * 1024 * 1024))
 touch "$TEST_DIR/empty"
 extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 
+# We always write the first byte; check how many blocks this filesystem
+# allocates to match empty image alloation.
+printf "\0" > "$TEST_DIR/empty"
+min_blocks=$(stat -c '%b' "$TEST_DIR/empty")
+
 echo
 echo "== creating image with default preallocation =="
 _make_test_img $size | _filter_imgfmt
-stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
+stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 
 for mode in off full falloc; do
     echo
     echo "== creating image with preallocation $mode =="
     IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
-    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
+    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 done
 
 # success, all done
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ QA output created by 175
 
 == creating image with default preallocation ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
-size=1048576, nothing allocated
+size=1048576, min allocation
 
 == creating image with preallocation off ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
-size=1048576, nothing allocated
+size=1048576, min allocation
 
 == creating image with preallocation full ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
-size=1048576, everything allocated
+size=1048576, max allocation
 
 == creating image with preallocation falloc ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
-size=1048576, everything allocated
+size=1048576, max allocation
  *** done
diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/178.out.qcow2
+++ b/tests/qemu-iotests/178.out.qcow2
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
 == raw input image with data (human) ==
 
 Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
-required size: 393216
+required size: 458752
 fully allocated size: 1074135040
 wrote 512/512 bytes at offset 512
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
 
 Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
 {
-    "required": 393216,
+    "required": 458752,
     "fully-allocated": 1074135040
 }
 wrote 512/512 bytes at offset 512
diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/221.out
+++ b/tests/qemu-iotests/221.out
@@ -XXX,XX +XXX,XX @@ QA output created by 221
 === Check mapping of unaligned raw image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537
-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 wrote 1/1 bytes at offset 65536
 1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 *** done
diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/253.out
+++ b/tests/qemu-iotests/253.out
@@ -XXX,XX +XXX,XX @@ QA output created by 253
 === Check mapping of unaligned raw image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048575
-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 wrote 65535/65535 bytes at offset 983040
 63.999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
 *** done
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Using block_resize we can test allocate_first_block() with file
descriptor opened with O_DIRECT, ensuring that it works for any size
larger than 4096 bytes.

Testing smaller sizes is tricky as the result depends on the filesystem
used for testing. For example on NFS any size will work since O_DIRECT
does not require any alignment.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190827010528.8818-3-nsoffer@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/175     | 28 ++++++++++++++++++++++++++++
 tests/qemu-iotests/175.out |  8 ++++++++
 2 files changed, 36 insertions(+)

diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -XXX,XX +XXX,XX @@ _filter_blocks()
         -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/max allocation/"
 }
 
+# Resize image using block_resize.
+# Parameter 1: image path
+# Parameter 2: new size
+_block_resize()
+{
+    local path=$1
+    local size=$2
+
+    $QEMU -qmp stdio -nographic -nodefaults \
+        -blockdev file,node-name=file,filename=$path,cache.direct=on \
+        <<EOF
+{'execute': 'qmp_capabilities'}
+{'execute': 'block_resize', 'arguments': {'node-name': 'file', 'size': $size}}
+{'execute': 'quit'}
+EOF
+}
+
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
@@ -XXX,XX +XXX,XX @@ _supported_fmt raw
 _supported_proto file
 _supported_os Linux
 
+_default_cache_mode none
+_supported_cache_modes none directsync
+
 size=$((1 * 1024 * 1024))
 
 touch "$TEST_DIR/empty"
@@ -XXX,XX +XXX,XX @@ for mode in off full falloc; do
     stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 done
 
+for new_size in 4096 1048576; do
+    echo
+    echo "== resize empty image with block_resize =="
+    _make_test_img 0 | _filter_imgfmt
+    _block_resize $TEST_IMG $new_size >/dev/null
+    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $new_size
+done
+
 # success, all done
 echo "*** done"
 rm -f $seq.full
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ size=1048576, max allocation
 == creating image with preallocation falloc ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
 size=1048576, max allocation
+
+== resize empty image with block_resize ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
+size=4096, min allocation
+
+== resize empty image with block_resize ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
+size=1048576, min allocation
  *** done
-- 
2.21.0

fe646693acc changed qemu-img create's output so that it no longer prints
single quotes around parameter values.  The subformat and adapter_type
filters in _filter_img_create() have never been adapted to that change.

Fixes: fe646693acc13ac48b98435d14149ab04dc597bc
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-2-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/059.out       | 16 ++++++++--------
 tests/qemu-iotests/common.filter |  4 ++--
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-io: can't open device TEST_DIR/t.vmdk: L1 size too big
 
 === Testing monolithicFlat creation and opening ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 2 GiB (2147483648 bytes)
 
 === Testing monolithicFlat with zeroed_grain ===
 qemu-img: TEST_DIR/t.IMGFMT: Flat image can't enable zeroed grain
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
 
 === Testing big twoGbMaxExtentFlat ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMaxExtentFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000
 image: TEST_DIR/t.vmdk
 file format: vmdk
 virtual size: 0.977 TiB (1073741824000 bytes)
@@ -XXX,XX +XXX,XX @@ Format specific information:
 qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Invalid extent line: RW 12582912 VMFS "dummy.IMGFMT" 1
 
 === Testing truncated sparse ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400
 qemu-img: Could not open 'TEST_DIR/t.IMGFMT': File truncated, expecting at least 13172736 bytes
 
 === Converting to streamOptimized from image with small cluster size===
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing monolithicFlat with internally generated JSON file name ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
 
 === Testing version 3 ===
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 64931328
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing 4TB monolithicFlat creation and IO ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 4 TiB (4398046511104 bytes)
@@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 966367641600
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing qemu-img map on extents ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
 wrote 1024/1024 bytes at offset 65024
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 1024/1024 bytes at offset 2147483136
@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
 0               0x20000         0x3f0000        TEST_DIR/t.vmdk
 0x7fff0000      0x20000         0x410000        TEST_DIR/t.vmdk
 0x140000000     0x10000         0x430000        TEST_DIR/t.vmdk
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=twoGbMaxExtentSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
 wrote 1024/1024 bytes at offset 65024
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 1024/1024 bytes at offset 2147483136
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_img_create()
         -e "s# compat6=\$on\\|off\$##g" \
         -e "s# static=\$on\\|off\$##g" \
         -e "s# zeroed_grain=\$on\\|off\$##g" \
-        -e "s# subformat='[^']*'##g" \
-        -e "s# adapter_type='[^']*'##g" \
+        -e "s# subformat=[^ ]*##g" \
+        -e "s# adapter_type=[^ ]*##g" \
         -e "s# hwversion=[^ ]*##g" \
         -e "s# lazy_refcounts=\$on\\|off\$##g" \
         -e "s# block_size=[0-9]\\+##g" \
-- 
2.21.0

This makes iotest 033 pass with e.g. subformat=monolithicFlat.  It also
turns a former error in 059 into success.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190815153638.4600-3-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c               | 54 ++++++++++++++++++++++++--------------
 tests/qemu-iotests/059     |  7 +++--
 tests/qemu-iotests/059.out |  4 ++-
 3 files changed, 42 insertions(+), 23 deletions(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static const char *next_line(const char *s)
 }
 
 static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
-                              const char *desc_file_path, QDict *options,
-                              Error **errp)
+                              QDict *options, Error **errp)
 {
     int ret;
     int matches;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
     const char *p, *np;
     int64_t sectors = 0;
     int64_t flat_offset;
+    char *desc_file_dir = NULL;
     char *extent_path;
     BdrvChild *extent_file;
     BDRVVmdkState *s = bs->opaque;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
             continue;
         }
 
-        if (!path_is_absolute(fname) && !path_has_protocol(fname) &&
-            !desc_file_path[0])
-        {
-            bdrv_refresh_filename(bs->file->bs);
-            error_setg(errp, "Cannot use relative extent paths with VMDK "
-                       "descriptor file '%s'", bs->file->bs->filename);
-            return -EINVAL;
-        }
+        if (path_is_absolute(fname)) {
+            extent_path = g_strdup(fname);
+        } else {
+            if (!desc_file_dir) {
+                desc_file_dir = bdrv_dirname(bs->file->bs, errp);
+                if (!desc_file_dir) {
+                    bdrv_refresh_filename(bs->file->bs);
+                    error_prepend(errp, "Cannot use relative paths with VMDK "
+                                  "descriptor file '%s': ",
+                                  bs->file->bs->filename);
+                    ret = -EINVAL;
+                    goto out;
+                }
+            }
 
-        extent_path = path_combine(desc_file_path, fname);
+            extent_path = g_strconcat(desc_file_dir, fname, NULL);
+        }
 
         ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
         assert(ret < 32);
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
         g_free(extent_path);
         if (local_err) {
             error_propagate(errp, local_err);
-            return -EINVAL;
+            ret = -EINVAL;
+            goto out;
         }
 
         /* save to extents array */
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
                             0, 0, 0, 0, 0, &extent, errp);
             if (ret < 0) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent->flat_start_offset = flat_offset << 9;
         } else if (!strcmp(type, "SPARSE") || !strcmp(type, "VMFSSPARSE")) {
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
             g_free(buf);
             if (ret) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent = &s->extents[s->num_extents - 1];
         } else if (!strcmp(type, "SESPARSE")) {
             ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
             if (ret) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent = &s->extents[s->num_extents - 1];
         } else {
             error_setg(errp, "Unsupported extent type '%s'", type);
             bdrv_unref_child(bs, extent_file);
-            return -ENOTSUP;
+            ret = -ENOTSUP;
+            goto out;
         }
         extent->type = g_strdup(type);
     }
-    return 0;
+
+    ret = 0;
+    goto out;
 
 invalid:
     np = next_line(p);
@@ -XXX,XX +XXX,XX @@ invalid:
         np--;
     }
     error_setg(errp, "Invalid extent line: %.*s", (int)(np - p), p);
-    return -EINVAL;
+    ret = -EINVAL;
+
+out:
+    g_free(desc_file_dir);
+    return ret;
 }
 
 static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
     }
     s->create_type = g_strdup(ct);
     s->desc_offset = 0;
-    ret = vmdk_parse_extents(buf, bs, bs->file->bs->exact_filename, options,
-                             errp);
+    ret = vmdk_parse_extents(buf, bs, options, errp);
 exit:
     return ret;
 }
diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
 
 echo
 echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+# Should work, because bdrv_dirname() works fine with blkdebug
 IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
-$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" 2>&1 \
-    | _filter_testdir | _filter_imgfmt
+$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
+         -c info \
+    2>&1 \
+    | _filter_testdir | _filter_imgfmt | _filter_img_info
 _cleanup_test_img
 
 echo
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 
 === Testing monolithicFlat with internally generated JSON file name ===
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
+format name: IMGFMT
+cluster size: 0 bytes
+vm state offset: 0 bytes
 
 === Testing version 3 ===
 image: TEST_DIR/iotest-version3.IMGFMT
-- 
2.21.0

We had a test for a case where relative extent paths did not work, but
unfortunately we just fixed the underlying problem, so it works now.
This patch adds a new test case that still fails.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-4-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/059     | 27 +++++++++++++++++++++++++++
 tests/qemu-iotests/059.out |  4 ++++
 2 files changed, 31 insertions(+)

diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
 
 echo
 echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+
+echo '--- blkdebug ---'
 # Should work, because bdrv_dirname() works fine with blkdebug
 IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
 $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TE
     | _filter_testdir | _filter_imgfmt | _filter_img_info
 _cleanup_test_img
 
+echo '--- quorum ---'
+# Should not work, because bdrv_dirname() does not work with quorum
+IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
+cp "$TEST_IMG" "$TEST_IMG.orig"
+
+filename="json:{
+    \"driver\": \"$IMGFMT\",
+    \"file\": {
+        \"driver\": \"quorum\",
+        \"children\": [ {
+            \"driver\": \"file\",
+            \"filename\": \"$TEST_IMG\"
+        }, {
+            \"driver\": \"file\",
+            \"filename\": \"$TEST_IMG.orig\"
+        } ],
+        \"vote-threshold\": 1
+    } }"
+
+filename=$(echo "$filename" | tr '\n' ' ' | sed -e 's/\s\+/ /g')
+$QEMU_IMG info "$filename" 2>&1 \
+    | sed -e "s/'json:[^']*'/\$QUORUM_FILE/g" \
+    | _filter_testdir | _filter_imgfmt | _filter_img_info
+
+
 echo
 echo "=== Testing version 3 ==="
 _use_sample_img iotest-version3.vmdk.bz2
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing monolithicFlat with internally generated JSON file name ===
+--- blkdebug ---
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 format name: IMGFMT
 cluster size: 0 bytes
 vm state offset: 0 bytes
+--- quorum ---
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+qemu-img: Could not open $QUORUM_FILE: Cannot use relative paths with VMDK descriptor file $QUORUM_FILE: Cannot generate a base directory for quorum nodes
 
 === Testing version 3 ===
 image: TEST_DIR/iotest-version3.IMGFMT
-- 
2.21.0

Compressed writes generally have to write full clusters, not just in
theory but also in practice when it comes to vmdk's streamOptimized
subformat.  It currently is just silently broken for writes with
non-zero in-cluster offsets:

$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 4k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 4096
4 KiB, 1 ops; 00.01 sec (443.724 KiB/sec and 110.9309 ops/sec)
read failed: Invalid argument

(The technical reason is that vmdk_write_extent() just writes the
incomplete compressed data actually to offset 4k.  When reading the
data, vmdk_read_extent() looks at offset 0 and finds the compressed data
size to be 0, because that is what it reads from there.  This yields an
error.)

For incomplete writes with zero in-cluster offsets, the error path when
reading the rest of the cluster is a bit different, but the result is
the same:

$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 0k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 0
4 KiB, 1 ops; 00.01 sec (362.641 KiB/sec and 90.6603 ops/sec)
read failed: Invalid argument

(Here, vmdk_read_extent() finds the data and then sees that the
uncompressed data is short.)

It is better to reject invalid writes than to make the user believe they
might have succeeded and then fail when trying to read it back.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-5-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset,
     if (extent->compressed) {
         void *compressed_data;
 
+        /* Only whole clusters */
+        if (offset_in_cluster ||
+            n_bytes > (extent->cluster_sectors * SECTOR_SIZE) ||
+            (n_bytes < (extent->cluster_sectors * SECTOR_SIZE) &&
+             offset + n_bytes != extent->end_sector * SECTOR_SIZE))
+        {
+            ret = -EINVAL;
+            goto out;
+        }
+
         if (!extent->has_marker) {
             ret = -EINVAL;
             goto out;
-- 
2.21.0

streamOptimized does not support writes that do not span exactly one
cluster.  Furthermore, it cannot rewrite already allocated clusters.
As such, many iotests do not work with it.  Disable them.

diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/002
+++ b/tests/qemu-iotests/002
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/003
+++ b/tests/qemu-iotests/003
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 size=128M
 offset=67M
diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 _supported_os Linux
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 # vpc is limited to 127GB, so we can't test it here
 if [ "$IMGFMT" = "vpc" ]; then
diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/009
+++ b/tests/qemu-iotests/009
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/010
+++ b/tests/qemu-iotests/010
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/011
+++ b/tests/qemu-iotests/011
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/017
+++ b/tests/qemu-iotests/017
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto generic
 _unsupported_proto vxhs
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/018
+++ b/tests/qemu-iotests/018
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _supported_os Linux
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/019
+++ b/tests/qemu-iotests/019
@@ -XXX,XX +XXX,XX @@ _supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 CLUSTER_SIZE=65536
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/027
+++ b/tests/qemu-iotests/027
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vmdk qcow qcow2 qed
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/032
+++ b/tests/qemu-iotests/032
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This works for any image format (though unlikely to segfault for raw)
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo === Prepare image ===
diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/034
+++ b/tests/qemu-iotests/034
@@ -XXX,XX +XXX,XX @@ _supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 CLUSTER_SIZE=4k
 size=128M
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 CLUSTER_SIZE=4k
 size=128M
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed raw
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 _make_test_img 4M
 
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
 _supported_proto file
+_unsupported_imgopts "subformat=streamOptimized"
 
 IMG_SIZE=64M
 
diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/105
+++ b/tests/qemu-iotests/105
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2 vmdk vhdx qed
 _supported_proto generic
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 echo
 echo "creating large image"
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/251
+++ b/tests/qemu-iotests/251
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt generic
 _supported_proto file
 _supported_os Linux
+_unsupported_imgopts "subformat=streamOptimized"
 
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
     # We use json:{} filenames here, so we cannot work with additional options.
-- 
2.21.0

The error message for the test case where we have a quorum node for
which no directory name can be generated is different: For
twoGbMaxExtentSparse, it complains that it cannot open the extent file.
For other (sub)formats, it just notes that it cannot determine the
backing file path.  Both are fine, but just disable twoGbMaxExtentSparse
for simplicity's sake.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-7-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/110 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "subformat=twoGbMaxExtentSparse"
 
 TEST_IMG_REL=$(basename "$TEST_IMG")
 
-- 
2.21.0

From: Stefan Hajnoczi <stefanha@redhat.com>

Fixes: a6b257a08e3d72219f03e461a52152672fec0612
       ("file-posix: Handle undetectable alignment")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20190827101328.4062-1-stefanha@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/file-posix.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
             align = alignments[i];
             if (raw_is_io_aligned(fd, buf + align, max_align)) {
-                /* Fallback to request_aligment. */
+                /* Fallback to request_alignment. */
                 s->buf_align = (align != 1) ? align : bs->bl.request_alignment;
                 break;
             }
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

It is possible to enable only a subset of the block drivers with the
"--block-drv-rw-whitelist" option of the "configure" script. All other
drivers are marked as unusable (or only included as read-only with the
"--block-drv-ro-whitelist" option). If an iotest is now using such a
disabled block driver, it is failing - which is bad, since at least the
tests in the "auto" group should be able to deal with this situation.
Thus let's introduce a "_require_drivers" function that can be used by
the shell tests to check for the availability of certain drivers first,
and marks the test as "not run" if one of the drivers is missing.

This patch mainly targets the test in the "auto" group which should
never fail in such a case, but also improves some of the other tests
along the way. Note that we also assume that the "qcow2" and "file"
drivers are always available - otherwise it does not make sense to
run "make check-block" at all (which only tests with qcow2 by default).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190823133552.11680-1-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/071       |  1 +
 tests/qemu-iotests/081       |  4 +---
 tests/qemu-iotests/099       |  1 +
 tests/qemu-iotests/120       |  1 +
 tests/qemu-iotests/162       |  4 +---
 tests/qemu-iotests/184       |  1 +
 tests/qemu-iotests/186       |  1 +
 tests/qemu-iotests/common.rc | 14 ++++++++++++++
 8 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
+_require_drivers blkdebug blkverify
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/081
+++ b/tests/qemu-iotests/081
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt raw
 _supported_proto file
 _supported_os Linux
+_require_drivers quorum
 
 do_run_qemu()
 {
@@ -XXX,XX +XXX,XX @@ run_qemu()
                           | _filter_qemu_io | _filter_generated_node_ids
 }
 
-test_quorum=$($QEMU_IMG --help|grep quorum)
-[ "$test_quorum" = "" ] && _supported_fmt quorum
-
 quorum="driver=raw,file.driver=quorum,file.vote-threshold=2"
 quorum="$quorum,file.children.0.file.filename=$TEST_DIR/1.raw"
 quorum="$quorum,file.children.1.file.filename=$TEST_DIR/2.raw"
diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/099
+++ b/tests/qemu-iotests/099
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 qed vdi vhdx vmdk vpc
 _supported_proto file
 _supported_os Linux
+_require_drivers blkdebug blkverify
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
     "subformat=twoGbMaxExtentSparse"
 
diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/120
+++ b/tests/qemu-iotests/120
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt generic
 _supported_proto file
 _unsupported_fmt luks
+_require_drivers raw
 
 _make_test_img 64M
 
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt generic
-
-test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh$ \|$$')
-[ "$test_ssh" = "" ] && _notrun "ssh support required"
+_require_drivers ssh
 
 echo
 echo '=== NBD ==='
diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/184
+++ b/tests/qemu-iotests/184
@@ -XXX,XX +XXX,XX @@ trap "exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_os Linux
+_require_drivers throttle
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/186
+++ b/tests/qemu-iotests/186
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
+_require_drivers null-co
 
 if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
     _notrun "Requires a PC machine"
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -XXX,XX +XXX,XX @@ _require_command()
     [ -x "$c" ] || _notrun "$1 utility required, skipped this test"
 }
 
+# Check that a set of drivers has been whitelisted in the QEMU binary
+#
+_require_drivers()
+{
+    available=$($QEMU -drive format=help | \
+                sed -e '/Supported formats:/!d' -e 's/Supported formats://')
+    for driver
+    do
+        if ! echo "$available" | grep -q " $driver$ \|$$"; then
+            _notrun "$driver not available"
+        fi
+    done
+}
+
 # make sure this script returns success
 true
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

The sanitizers (especially the address sanitizer from Clang) are
sometimes printing out warnings or false positives - this spoils
the output of the iotests, causing some of the tests to fail.
Thus let's skip the automatic iotests during "make check" when the
user configured QEMU with --enable-sanitizers.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190823084203.29734-1-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/check-block.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tests/check-block.sh b/tests/check-block.sh
index XXXXXXX..XXXXXXX 100755
--- a/tests/check-block.sh
+++ b/tests/check-block.sh
@@ -XXX,XX +XXX,XX @@ if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
     exit 0
 fi
 
+if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
+    echo "Sanitizers are enabled ==> Not running the qemu-iotests."
+    exit 0
+fi
+
 if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
     echo "No qemu-system binary available ==> Not running the qemu-iotests."
     exit 0
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Quoting cache mode is not needed, and most tests use unquoted values.
Unify all test to use the same style.

Message-id: 20190827173432.7656-1-nsoffer@redhat.com
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/026 | 4 ++--
 tests/qemu-iotests/039 | 4 ++--
 tests/qemu-iotests/052 | 2 +-
 tests/qemu-iotests/091 | 4 ++--
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/026
+++ b/tests/qemu-iotests/026
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Currently only qcow2 supports rebasing
 _supported_fmt qcow2
 _supported_proto file
-_default_cache_mode "writethrough"
-_supported_cache_modes "writethrough" "none"
+_default_cache_mode writethrough
+_supported_cache_modes writethrough none
 # The refcount table tests expect a certain minimum width for refcount entries
 # (so that the refcount table actually needs to grow); that minimum is 16 bits,
 # being the default refcount entry width.
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
-_default_cache_mode "writethrough"
-_supported_cache_modes "writethrough"
+_default_cache_mode writethrough
+_supported_cache_modes writethrough
 
 size=128M
 
diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/052
+++ b/tests/qemu-iotests/052
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto file
 
 # Don't do O_DIRECT on tmpfs
-_supported_cache_modes "writeback" "writethrough" "unsafe"
+_supported_cache_modes writeback writethrough unsafe
 
 size=128M
 _make_test_img $size
diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/091
+++ b/tests/qemu-iotests/091
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
-_default_cache_mode "none"
-_supported_cache_modes "writethrough" "none" "writeback"
+_default_cache_mode none
+_supported_cache_modes writethrough none writeback
 
 size=1G
 
-- 
2.21.0