Series comparison

-[Qemu-devel] [PULL 00/21] Block patches
+[Qemu-devel] [PULL 00/15] Block patches
-The following changes since commit 98b2faeaee96ab084d0b1669918688d8895c155f:
+The following changes since commit 23919ddfd56135cad3cb468a8f54d5a595f024f4:
-  Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging (2017-02-10 18:07:02 +0000)
+  Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20190827' into staging (2019-08-27 15:52:36 +0100)
-are available in the git repository at:
+are available in the Git repository at:
-  git://github.com/XanClic/qemu.git tags/pull-block-2017-02-12
+  https://github.com/XanClic/qemu.git tags/pull-block-2019-08-27
-for you to fetch changes up to 10d6eda1926804a09aa0710ca62933087813de0b:
+for you to fetch changes up to bb043c056cffcc2f3ce88bfdaf2e76e455c09e2c:
-  qemu-img: Avoid setting ret to unused value in img_convert() (2017-02-12 00:56:32 +0100)
+  iotests: Unify cache mode quoting (2019-08-27 19:48:44 +0200)
 ----------------------------------------------------------------
-Block patches
+Block patches:
 - qemu-io now accepts a file to read a write pattern from
 - Ensure that raw files have their first block allocated so we can probe
   the O_DIRECT alignment if necessary
 - Various fixes
 ----------------------------------------------------------------
-Alberto Garcia (2):
+Denis Plotnikov (1):
-      qcow2: Optimize the refcount-block overlap check
+  qemu-io: add pattern file for write command
       qemu-io: don't allow I/O operations larger than BDRV_REQUEST_MAX_BYTES
-Daniel P. Berrange (1):
+Max Reitz (7):
-      iotests: record separate timings per format,protocol pair
+  iotests: Fix _filter_img_create()
+  vmdk: Use bdrv_dirname() for relative extent paths
-Dou Liyang (2):
+  iotests: Keep testing broken relative extent paths
-      block/qapi: reduce the coupling between the bdrv_query_stats and bdrv_query_bds_stats
+  vmdk: Reject invalid compressed writes
-      block/qapi: reduce the execution time of qmp_query_blockstats
+  iotests: Disable broken streamOptimized tests
+  iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
-Fam Zheng (2):
+  iotests: Disable 126 for flat vmdk subformats
       qapi: Tweak error message of bdrv_query_image_info
       iotests: Fix reference output for 059
 Jeff Cody (3):
       block: check full backing filename when searching protocol filenames
       qemu-iotests: Don't create fifos / pidfiles with protocol paths
       qemu-iotest: test to lookup protocol-based image with relative backing
 Max Reitz (1):
       qemu-img: Improve commit invalid base message
 Nir Soffer (3):
-      qemu-io: Return non-zero exit code on failure
+  block: posix: Always allocate the first block
-      qemu-iotests: Add _unsupported_fmt helper
+  iotests: Test allocate_first_block() with O_DIRECT
-      qemu-io: Add failure regression tests
+  iotests: Unify cache mode quoting
-Peter Lieven (2):
+Stefan Hajnoczi (1):
-      block/nfs: fix NULL pointer dereference in URI parsing
+  file-posix: fix request_alignment typo
       block/nfs: fix naming of runtime opts
-Peter Maydell (2):
+Thomas Huth (2):
-      qemu-img: Use qemu_strtoul() rather than raw strtoul()
+  iotests: Check for enabled drivers before testing them
-      qemu-img: Avoid setting ret to unused value in img_convert()
+  tests/check-block: Skip iotests when sanitizers are enabled
 QingFeng Hao (2):
       iotests: Fix a problem in common.filter
       block/vmdk: Fix the endian problem of buf_len and lba
 Vladimir Sementsov-Ogievskiy (1):
-      block: bdrv_invalidate_cache: invalidate children first
+  block: fix permission update in bdrv_replace_node
- tests/qemu-iotests/Makefile      |  2 +-
+ block.c                                       |  5 +-
- block/qcow2.h                    |  1 +
+ block/file-posix.c                            | 53 +++++++++-
- block.c                          | 24 +++++++---
+ block/vmdk.c                                  | 64 ++++++++----
- block/nfs.c                      | 49 ++++++++++----------
+ qemu-io-cmds.c                                | 99 +++++++++++++++++--
- block/qapi.c                     | 99 +++++++++++++++++-----------------------
+ tests/check-block.sh                          |  5 +
- block/qcow2-refcount.c           | 24 +++++++++-
+ tests/qemu-iotests/002                        |  1 +
- block/qcow2.c                    |  1 +
+ tests/qemu-iotests/003                        |  1 +
- block/vmdk.c                     |  4 +-
+ tests/qemu-iotests/005                        |  3 +-
- qemu-img.c                       | 44 +++++++++---------
+ tests/qemu-iotests/009                        |  1 +
- qemu-io-cmds.c                   | 20 +++++---
+ tests/qemu-iotests/010                        |  1 +
- qemu-io.c                        |  8 +++-
+ tests/qemu-iotests/011                        |  1 +
- tests/qemu-iotests/.gitignore    |  2 +-
+ tests/qemu-iotests/017                        |  3 +-
- tests/qemu-iotests/059.out       |  5 +-
+ tests/qemu-iotests/018                        |  3 +-
- tests/qemu-iotests/070.out       |  1 -
+ tests/qemu-iotests/019                        |  3 +-
- tests/qemu-iotests/075.out       |  7 ---
+ tests/qemu-iotests/020                        |  3 +-
- tests/qemu-iotests/076.out       |  3 --
+ tests/qemu-iotests/026                        |  4 +-
- tests/qemu-iotests/078.out       |  6 ---
+ tests/qemu-iotests/027                        |  1 +
- tests/qemu-iotests/080.out       | 18 --------
+ tests/qemu-iotests/032                        |  1 +
- tests/qemu-iotests/083.out       | 17 -------
+ tests/qemu-iotests/033                        |  1 +
- tests/qemu-iotests/088.out       |  6 ---
+ tests/qemu-iotests/034                        |  3 +-
- tests/qemu-iotests/092.out       | 12 -----
+ tests/qemu-iotests/037                        |  3 +-
- tests/qemu-iotests/116.out       |  7 ---
+ tests/qemu-iotests/039                        |  4 +-
- tests/qemu-iotests/131.out       |  1 -
+ tests/qemu-iotests/052                        |  2 +-
- tests/qemu-iotests/140.out       |  1 -
+ tests/qemu-iotests/059                        | 34 ++++++-
- tests/qemu-iotests/173           | 97 +++++++++++++++++++++++++++++++++++++++
+ tests/qemu-iotests/059.out                    | 26 +++--
- tests/qemu-iotests/173.out       | 12 +++++
+ tests/qemu-iotests/063                        |  3 +-
- tests/qemu-iotests/174           | 59 ++++++++++++++++++++++++
+ tests/qemu-iotests/071                        |  1 +
- tests/qemu-iotests/174.out       |  7 +++
+ tests/qemu-iotests/072                        |  1 +
- tests/qemu-iotests/check         | 12 +++--
+ tests/qemu-iotests/081                        |  4 +-
- tests/qemu-iotests/common.config |  6 ++-
+ tests/qemu-iotests/091                        |  4 +-
- tests/qemu-iotests/common.filter |  2 +-
+ tests/qemu-iotests/099                        |  1 +
- tests/qemu-iotests/common.qemu   | 10 ++--
+ tests/qemu-iotests/105                        |  3 +-
- tests/qemu-iotests/common.rc     | 17 +++++--
+ tests/qemu-iotests/110                        |  3 +-
- tests/qemu-iotests/group         |  2 +
+ tests/qemu-iotests/120                        |  1 +
-files changed, 366 insertions(+), 220 deletions(-)
+ tests/qemu-iotests/126                        |  2 +
- create mode 100755 tests/qemu-iotests/173
+ tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
- create mode 100644 tests/qemu-iotests/173.out
+ tests/qemu-iotests/150.out.raw                | 12 +++
- create mode 100755 tests/qemu-iotests/174
+ tests/qemu-iotests/162                        |  4 +-
- create mode 100644 tests/qemu-iotests/174.out
+ tests/qemu-iotests/175                        | 47 +++++++--
  tests/qemu-iotests/175.out                    | 16 ++-
  tests/qemu-iotests/178.out.qcow2              |  4 +-
  tests/qemu-iotests/184                        |  1 +
  tests/qemu-iotests/186                        |  1 +
  tests/qemu-iotests/197                        |  1 +
  tests/qemu-iotests/215                        |  1 +
  tests/qemu-iotests/221.out                    | 12 ++-
  tests/qemu-iotests/251                        |  1 +
  tests/qemu-iotests/253.out                    | 12 ++-
  tests/qemu-iotests/common.filter              |  4 +-
  tests/qemu-iotests/common.rc                  | 14 +++
 files changed, 391 insertions(+), 87 deletions(-)
  rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
  create mode 100644 tests/qemu-iotests/150.out.raw
+--
+.21.0

-[Qemu-devel] [PULL 19/21] qemu-io: don't allow I/O operations larger than BDRV_REQUEST_MAX_BYTES
+[Qemu-devel] [PULL 01/15] qemu-io: add pattern file for write command
-From: Alberto Garcia <berto@igalia.com>
+From: Denis Plotnikov <dplotnikov@virtuozzo.com>
-Passing a request size larger than BDRV_REQUEST_MAX_BYTES to any of the
+The patch allows to provide a pattern file for write
-I/O commands results in an error. While 'read' and 'write' handle the
+command. There was no similar ability before.
 error correctly, 'aio_read' and 'aio_write' hit an assertion:
-blk_aio_read_entry: Assertion `rwco->qiov->size == acb->bytes' failed.
+Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
+Message-id: 20190820164616.4072-1-dplotnikov@virtuozzo.com
-The reason is that the QEMU I/O code cannot handle request sizes
+Reviewed-by: Eric Blake <eblake@redhat.com>
-larger than BDRV_REQUEST_MAX_BYTES, so this patch makes qemu-io check
+[mreitz: Keep optstring in alphabetical order]
 that all values are within range.
 Signed-off-by: Alberto Garcia <berto@igalia.com>
 Message-id: 79f66648c685929a144396bda24d13a207131dcf.1485878688.git.berto@igalia.com
 [mreitz: Use BDRV_REQUEST_MAX_BYTES instead of INT_MAX]
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qemu-io-cmds.c | 20 +++++++++++++-------
+ qemu-io-cmds.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++---
-file changed, 13 insertions(+), 7 deletions(-)
+file changed, 93 insertions(+), 6 deletions(-)
 diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
 index XXXXXXX..XXXXXXX 100644
 --- a/qemu-io-cmds.c
 +++ b/qemu-io-cmds.c
-@@ -XXX,XX +XXX,XX @@ create_iovec(BlockBackend *blk, QEMUIOVector *qiov, char **argv, int nr_iov,
+@@ -XXX,XX +XXX,XX @@ static void qemu_io_free(void *p)
-             goto fail;
+     qemu_vfree(p);
-         }
+ }
--        if (len > SIZE_MAX) {
++/*
--            printf("Argument '%s' exceeds maximum size %llu\n", arg,
++ * qemu_io_alloc_from_file()
--                   (unsigned long long)SIZE_MAX);
++ *
-+        if (len > BDRV_REQUEST_MAX_BYTES) {
++ * Allocates the buffer and populates it with the content of the given file
-+            printf("Argument '%s' exceeds maximum size %" PRIu64 "\n", arg,
++ * up to @len bytes. If the file length is less than @len, then the buffer
-+                   (uint64_t)BDRV_REQUEST_MAX_BYTES);
++ * is populated with the file content cyclically.
-+            goto fail;
++ *
 + * @blk - the block backend where the buffer content is going to be written to
 + * @len - the buffer length
 + * @file_name - the file to read the content from
 + *
 + * Returns: the buffer pointer on success
 + *          NULL on error
 + */
 +static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
 +                                     const char *file_name)
 +{
 +    char *buf, *buf_origin;
 +    FILE *f = fopen(file_name, "r");
 +    int pattern_len;
 +
 +    if (!f) {
 +        perror(file_name);
 +        return NULL;
 +    }
 +
 +    if (qemuio_misalign) {
 +        len += MISALIGN_OFFSET;
 +    }
 +
 +    buf_origin = buf = blk_blockalign(blk, len);
 +
 +    if (qemuio_misalign) {
 +        buf_origin += MISALIGN_OFFSET;
 +        buf += MISALIGN_OFFSET;
 +        len -= MISALIGN_OFFSET;
 +    }
 +
 +    pattern_len = fread(buf_origin, 1, len, f);
 +
 +    if (ferror(f)) {
 +        perror(file_name);
 +        goto error;
 +    }
 +
 +    if (pattern_len == 0) {
 +        fprintf(stderr, "%s: file is empty\n", file_name);
 +        goto error;
 +    }
 +
 +    fclose(f);
 +
 +    if (len > pattern_len) {
 +        len -= pattern_len;
 +        buf += pattern_len;
 +
 +        while (len > 0) {
 +            size_t len_to_copy = MIN(pattern_len, len);
 +
 +            memcpy(buf, buf_origin, len_to_copy);
 +
 +            len -= len_to_copy;
 +            buf += len_to_copy;
 +        }
++    }
 +
-+        if (count > BDRV_REQUEST_MAX_BYTES - len) {
++    return buf_origin;
-+            printf("The total number of bytes exceed the maximum size %" PRIu64
++
-+                   "\n", (uint64_t)BDRV_REQUEST_MAX_BYTES);
++error:
-             goto fail;
++    qemu_io_free(buf_origin);
-         }
++    return NULL;
++}
-@@ -XXX,XX +XXX,XX @@ static int read_f(BlockBackend *blk, int argc, char **argv)
++
-     if (count < 0) {
+ static void dump_buffer(const void *buffer, int64_t offset, int64_t len)
-         print_cvtnum_err(count, argv[optind]);
+ {
-         return 0;
+     uint64_t i;
--    } else if (count > SIZE_MAX) {
+@@ -XXX,XX +XXX,XX @@ static void write_help(void)
-+    } else if (count > BDRV_REQUEST_MAX_BYTES) {
+ " -n, -- with -z, don't allow slow fallback\n"
-         printf("length cannot exceed %" PRIu64 ", given %s\n",
+ " -p, -- ignored for backwards compatibility\n"
--               (uint64_t) SIZE_MAX, argv[optind]);
+ " -P, -- use different pattern to fill file\n"
-+               (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]);
++" -s, -- use a pattern file to fill the write buffer\n"
-         return 0;
+ " -C, -- report statistics in a machine parsable format\n"
  " -q, -- quiet mode, do not show I/O statistics\n"
  " -u, -- with -z, allow unmapping\n"
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t write_cmd = {
      .perm       = BLK_PERM_WRITE,
      .argmin     = 2,
      .argmax     = -1,
 -    .args       = "[-bcCfnquz] [-P pattern] off len",
 +    .args       = "[-bcCfnquz] [-P pattern | -s source_file] off len",
      .oneline    = "writes a number of bytes at a specified offset",
      .help       = write_help,
  };
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
  {
      struct timespec t1, t2;
      bool Cflag = false, qflag = false, bflag = false;
 -    bool Pflag = false, zflag = false, cflag = false;
 +    bool Pflag = false, zflag = false, cflag = false, sflag = false;
      int flags = 0;
      int c, cnt, ret;
      char *buf = NULL;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
      /* Some compilers get confused and warn if this is not initialized.  */
      int64_t total = 0;
      int pattern = 0xcd;
 +    const char *file_name = NULL;
 -    while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) {
 +    while ((c = getopt(argc, argv, "bcCfnpP:qs:uz")) != -1) {
          switch (c) {
          case 'b':
              bflag = true;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
          case 'q':
              qflag = true;
              break;
 +        case 's':
 +            sflag = true;
 +            file_name = optarg;
 +            break;
          case 'u':
              flags |= BDRV_REQ_MAY_UNMAP;
              break;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
          return -EINVAL;
      }
+-    if (zflag && Pflag) {
+-        printf("-z and -P cannot be specified at the same time\n");
++    if (zflag + Pflag + sflag > 1) {
++        printf("Only one of -z, -P, and -s "
++               "can be specified at the same time\n");
+         return -EINVAL;
+     }
 @@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
-     if (count < 0) {
-         print_cvtnum_err(count, argv[optind]);
-         return 0;
--    } else if (count > SIZE_MAX) {
-+    } else if (count > BDRV_REQUEST_MAX_BYTES) {
-         printf("length cannot exceed %" PRIu64 ", given %s\n",
--               (uint64_t) SIZE_MAX, argv[optind]);
-+               (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]);
-         return 0;
      }
+     if (!zflag) {
+-        buf = qemu_io_alloc(blk, count, pattern);
++        if (sflag) {
++            buf = qemu_io_alloc_from_file(blk, count, file_name);
++            if (!buf) {
++                return -EINVAL;
++            }
++        } else {
++            buf = qemu_io_alloc(blk, count, pattern);
++        }
+     }
+     clock_gettime(CLOCK_MONOTONIC, &t1);
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 12/21] block: bdrv_invalidate_cache: invalidate children first
+[Qemu-devel] [PULL 02/15] block: fix permission update in bdrv_replace_node
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Current implementation invalidates firstly parent bds and then its
+It's wrong to OR shared permissions. It may lead to crash on further
-children. This leads to the following bug:
+permission updates.
 Also, no needs to consider previously calculated permissions, as at
 this point we already bind all new parents and bdrv_get_cumulative_perm
 result is enough. So fix the bug by just set permissions by
 bdrv_get_cumulative_perm result.
-after incoming migration, in bdrv_invalidate_cache_all:
+Bug was introduced in long ago 234ac1a9025, in 2.9.
 . invalidate parent bds - reopen it with BDRV_O_INACTIVE cleared
 . child is not yet invalidated
 . parent check that its BDRV_O_INACTIVE is cleared
 . parent writes to child
 . assert in bdrv_co_pwritev, as BDRV_O_INACTIVE is set for child
 This patch fixes it by just changing invalidate sequence: invalidate
 children first.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-id: 20170131112308.54189-1-vsementsov@virtuozzo.com
+Message-id: 20190824100740.61635-1-vsementsov@virtuozzo.com
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block.c | 11 +++++------
+ block.c | 5 ++---
-file changed, 5 insertions(+), 6 deletions(-)
+file changed, 2 insertions(+), 3 deletions(-)
 diff --git a/block.c b/block.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block.c
 +++ b/block.c
-@@ -XXX,XX +XXX,XX @@ void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
+@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
-     if (!(bs->open_flags & BDRV_O_INACTIVE)) {
+ {
-         return;
+     BdrvChild *c, *next;
      GSList *list = NULL, *p;
 -    uint64_t old_perm, old_shared;
      uint64_t perm = 0, shared = BLK_PERM_ALL;
      int ret;
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
          bdrv_unref(from);
      }
--    bs->open_flags &= ~BDRV_O_INACTIVE;
+-    bdrv_get_cumulative_perm(to, &old_perm, &old_shared);
--    if (bs->drv->bdrv_invalidate_cache) {
+-    bdrv_set_perm(to, old_perm | perm, old_shared | shared);
--        bs->drv->bdrv_invalidate_cache(bs, &local_err);
++    bdrv_get_cumulative_perm(to, &perm, &shared);
-+    QLIST_FOREACH(child, &bs->children, next) {
++    bdrv_set_perm(to, perm, shared);
-+        bdrv_invalidate_cache(child->bs, &local_err);
-         if (local_err) {
+ out:
--            bs->open_flags |= BDRV_O_INACTIVE;
+     g_slist_free(list);
              error_propagate(errp, local_err);
              return;
          }
      }
 -    QLIST_FOREACH(child, &bs->children, next) {
 -        bdrv_invalidate_cache(child->bs, &local_err);
 +    bs->open_flags &= ~BDRV_O_INACTIVE;
 +    if (bs->drv->bdrv_invalidate_cache) {
 +        bs->drv->bdrv_invalidate_cache(bs, &local_err);
          if (local_err) {
              bs->open_flags |= BDRV_O_INACTIVE;
              error_propagate(errp, local_err);
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 15/21] qemu-io: Return non-zero exit code on failure
+[Qemu-devel] [PULL 03/15] block: posix: Always allocate the first block
 From: Nir Soffer <nirsof@gmail.com>
-The result of openfile was not checked, leading to failure deep in the
+When creating an image with preallocation "off" or "falloc", the first
-actual command with confusing error message, and exiting with exit code 0.
+block of the image is typically not allocated. When using Gluster
+storage backed by XFS filesystem, reading this block using direct I/O
-Here is a simple example - trying to read with the wrong format:
+succeeds regardless of request length, fooling alignment detection.
-    $ touch file
+In this case we fallback to a safe value (4096) instead of the optimal
-    $ qemu-io -f qcow2 -c 'read -P 1 0 1024' file; echo $?
+value (512), which may lead to unneeded data copying when aligning
-    can't open device file: Image is not in qcow2 format
+requests.  Allocating the first block avoids the fallback.
-    no file open, try 'help open'
+Since we allocate the first block even with preallocation=off, we no
+longer create images with zero disk size:
-With this patch, we fail earlier with exit code 1:
+    $ ./qemu-img create -f raw test.raw 1g
-    $ ./qemu-io -f qcow2 -c 'read -P 1 0 1024' file; echo $?
+    Formatting 'test.raw', fmt=raw size=1073741824
-    can't open device file: Image is not in qcow2 format
+    $ ls -lhs test.raw
+.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw
-Failing earlier, we don't log this error now:
+And converting the image requires additional cluster:
-    no file open, try 'help open'
+    $ ./qemu-img measure -f raw -O qcow2 test.raw
-But some tests expected it; the line was removed from the test output.
+    required size: 458752
+    fully allocated size: 1074135040
-Signed-off-by: Nir Soffer <nirsof@gmail.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+When using format like vmdk with multiple files per image, we allocate
-Message-id: 20170201003120.23378-2-nirsof@gmail.com
+one block per file:
     $ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
     Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat
     $ ls -lhs test*.vmdk
 .0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
 .0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
 .0K -rw-r--r--. 1 nsoffer nsoffer  353 Aug 27 03:23 test.vmdk
 I did quick performance test for copying disks with qemu-img convert to
 new raw target image to Gluster storage with sector size of 512 bytes:
     for i in $(seq 10); do
         rm -f dst.raw
         sleep 10
         time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
     done
 Here is a table comparing the total time spent:
 Type    Before(s)   After(s)    Diff(%)
 ---------------------------------------
 real      530.028    469.123      -11.4
 user       17.204     10.768      -37.4
 sys        17.881      7.011      -60.7
 We can see very clear improvement in CPU usage.
 Signed-off-by: Nir Soffer <nsoffer@redhat.com>
 Message-id: 20190827010528.8818-2-nsoffer@redhat.com
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qemu-io.c                  |  8 ++++++--
+ block/file-posix.c                            | 51 +++++++++++++++++++
- tests/qemu-iotests/059.out |  3 ---
+ tests/qemu-iotests/059.out                    |  2 +-
- tests/qemu-iotests/070.out |  1 -
+ tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
- tests/qemu-iotests/075.out |  7 -------
+ tests/qemu-iotests/150.out.raw                | 12 +++++
- tests/qemu-iotests/076.out |  3 ---
+ tests/qemu-iotests/175                        | 19 ++++---
- tests/qemu-iotests/078.out |  6 ------
+ tests/qemu-iotests/175.out                    |  8 +--
- tests/qemu-iotests/080.out | 18 ------------------
+ tests/qemu-iotests/178.out.qcow2              |  4 +-
- tests/qemu-iotests/083.out | 17 -----------------
+ tests/qemu-iotests/221.out                    | 12 +++--
- tests/qemu-iotests/088.out |  6 ------
+ tests/qemu-iotests/253.out                    | 12 +++--
- tests/qemu-iotests/092.out | 12 ------------
+files changed, 99 insertions(+), 21 deletions(-)
- tests/qemu-iotests/116.out |  7 -------
+ rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
- tests/qemu-iotests/131.out |  1 -
+ create mode 100644 tests/qemu-iotests/150.out.raw
- tests/qemu-iotests/140.out |  1 -
-files changed, 6 insertions(+), 84 deletions(-)
+diff --git a/block/file-posix.c b/block/file-posix.c
+index XXXXXXX..XXXXXXX 100644
-diff --git a/qemu-io.c b/qemu-io.c
+--- a/block/file-posix.c
-index XXXXXXX..XXXXXXX 100644
++++ b/block/file-posix.c
---- a/qemu-io.c
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_discard(void *opaque)
-+++ b/qemu-io.c
+     return ret;
-@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
+ }
-                 exit(1);
 +/*
 + * Help alignment probing by allocating the first block.
 + *
 + * When reading with direct I/O from unallocated area on Gluster backed by XFS,
 + * reading succeeds regardless of request length. In this case we fallback to
 + * safe alignment which is not optimal. Allocating the first block avoids this
 + * fallback.
 + *
 + * fd may be opened with O_DIRECT, but we don't know the buffer alignment or
 + * request alignment, so we use safe values.
 + *
 + * Returns: 0 on success, -errno on failure. Since this is an optimization,
 + * caller may ignore failures.
 + */
 +static int allocate_first_block(int fd, size_t max_size)
 +{
 +    size_t write_size = (max_size < MAX_BLOCKSIZE)
 +        ? BDRV_SECTOR_SIZE
 +        : MAX_BLOCKSIZE;
 +    size_t max_align = MAX(MAX_BLOCKSIZE, getpagesize());
 +    void *buf;
 +    ssize_t n;
 +    int ret;
 +
 +    buf = qemu_memalign(max_align, write_size);
 +    memset(buf, 0, write_size);
 +
 +    do {
 +        n = pwrite(fd, buf, write_size, 0);
 +    } while (n == -1 && errno == EINTR);
 +
 +    ret = (n == -1) ? -errno : 0;
 +
 +    qemu_vfree(buf);
 +    return ret;
 +}
 +
  static int handle_aiocb_truncate(void *opaque)
  {
      RawPosixAIOData *aiocb = opaque;
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
                  /* posix_fallocate() doesn't set errno. */
                  error_setg_errno(errp, -result,
                                   "Could not preallocate new data");
 +            } else if (current_length == 0) {
 +                /*
 +                 * posix_fallocate() uses fallocate() if the filesystem
 +                 * supports it, or fallback to manually writing zeroes. If
 +                 * fallocate() was used, unaligned reads from the fallocated
 +                 * area in raw_probe_alignment() will succeed, hence we need to
 +                 * allocate the first block.
 +                 *
 +                 * Optimize future alignment probing; ignore failures.
 +                 */
 +                allocate_first_block(fd, offset);
              }
-             opts = qemu_opts_to_qdict(qopts, NULL);
--            openfile(NULL, flags, writethrough, opts);
-+            if (openfile(NULL, flags, writethrough, opts)) {
-+                exit(1);
-+            }
          } else {
-             if (format) {
+             result = 0;
-                 opts = qdict_new();
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
-                 qdict_put(opts, "driver", qstring_from_str(format));
+         if (ftruncate(fd, offset) != 0) {
-             }
+             result = -errno;
--            openfile(argv[optind], flags, writethrough, opts);
+             error_setg_errno(errp, -result, "Could not resize file");
-+            if (openfile(argv[optind], flags, writethrough, opts)) {
++        } else if (current_length == 0 && offset > current_length) {
-+                exit(1);
++            /* Optimize future alignment probing; ignore failures. */
-+            }
++            allocate_first_block(fd, offset);
          }
-     }
+         return result;
-     command_loop();
+     default:
 diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/059.out
 +++ b/tests/qemu-iotests/059.out
-@@ -XXX,XX +XXX,XX @@ QA output created by 059
+@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMax
- === Testing invalid granularity ===
+ image: TEST_DIR/t.vmdk
- Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+ file format: vmdk
- can't open device TEST_DIR/t.vmdk: Invalid granularity, image may be corrupt
+ virtual size: 0.977 TiB (1073741824000 bytes)
--no file open, try 'help open'
+-disk size: 16 KiB
++disk size: 1.97 MiB
- === Testing too big L2 table size ===
+ Format specific information:
- Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+     cid: XXXXXXXX
- can't open device TEST_DIR/t.vmdk: L2 table size too big
+     parent cid: XXXXXXXX
--no file open, try 'help open'
+diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out.qcow2
+similarity index 100%
- === Testing too big L1 table size ===
+rename from tests/qemu-iotests/150.out
- Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+rename to tests/qemu-iotests/150.out.qcow2
- can't open device TEST_DIR/t.vmdk: L1 size too big
+diff --git a/tests/qemu-iotests/150.out.raw b/tests/qemu-iotests/150.out.raw
--no file open, try 'help open'
+new file mode 100644
+index XXXXXXX..XXXXXXX
- === Testing monolithicFlat creation and opening ===
+--- /dev/null
- Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
++++ b/tests/qemu-iotests/150.out.raw
-diff --git a/tests/qemu-iotests/070.out b/tests/qemu-iotests/070.out
+@@ -XXX,XX +XXX,XX @@
-index XXXXXXX..XXXXXXX 100644
++QA output created by 150
---- a/tests/qemu-iotests/070.out
++
-+++ b/tests/qemu-iotests/070.out
++=== Mapping sparse conversion ===
-@@ -XXX,XX +XXX,XX @@ QA output created by 070
++
- can't open device TEST_DIR/iotest-dirtylog-10G-4M.vhdx: VHDX image file 'TEST_DIR/iotest-dirtylog-10G-4M.vhdx' opened read-only, but contains a log that needs to be replayed
++Offset          Length          File
- To replay the log, run:
++0               0x1000          TEST_DIR/t.IMGFMT
- qemu-img check -r all 'TEST_DIR/iotest-dirtylog-10G-4M.vhdx'
++
-- no file open, try 'help open'
++=== Mapping non-sparse conversion ===
- === Verify open image replays log  ===
++
- read 18874368/18874368 bytes at offset 0
++Offset          Length          File
-MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
++0               0x100000        TEST_DIR/t.IMGFMT
-diff --git a/tests/qemu-iotests/075.out b/tests/qemu-iotests/075.out
++*** done
-index XXXXXXX..XXXXXXX 100644
+diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
---- a/tests/qemu-iotests/075.out
+index XXXXXXX..XXXXXXX 100755
-+++ b/tests/qemu-iotests/075.out
+--- a/tests/qemu-iotests/175
-@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 1048064
++++ b/tests/qemu-iotests/175
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
- == block_size must be a multiple of 512 ==
+ # the file size.  This function hides the resulting difference in the
- can't open device TEST_DIR/simple-pattern.cloop: block_size 513 must be a multiple of 512
+ # stat -c '%b' output.
--no file open, try 'help open'
+ # Parameter 1: Number of blocks an empty file occupies
+-# Parameter 2: Image size in bytes
- == block_size cannot be zero ==
++# Parameter 2: Minimal number of blocks in an image
- can't open device TEST_DIR/simple-pattern.cloop: block_size cannot be zero
++# Parameter 3: Image size in bytes
--no file open, try 'help open'
+ _filter_blocks()
+ {
- == huge block_size ===
+     extra_blocks=$1
- can't open device TEST_DIR/simple-pattern.cloop: block_size 4294966784 must be 64 MB or less
+-    img_size=$2
--no file open, try 'help open'
++    min_blocks=$2
++    img_size=$3
- == offsets_size overflow ===
- can't open device TEST_DIR/simple-pattern.cloop: n_blocks 4294967295 must be 536870911 or less
+-    sed -e "s/blocks=$extra_blocks\\(\$\\|[^0-9]\\)/nothing allocated/" \
--no file open, try 'help open'
+-        -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/everything allocated/"
++    sed -e "s/blocks=$min_blocks\\(\$\\|[^0-9]\\)/min allocation/" \
- == refuse images that require too many offsets ===
++        -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/max allocation/"
- can't open device TEST_DIR/simple-pattern.cloop: image requires too many offsets, try increasing block size
+ }
--no file open, try 'help open'
+ # get standard environment, filters and checks
- == refuse images with non-monotonically increasing offsets ==
+@@ -XXX,XX +XXX,XX @@ size=$((1 * 1024 * 1024))
- can't open device TEST_DIR/simple-pattern.cloop: offsets not monotonically increasing at index 1, image file is corrupt
+ touch "$TEST_DIR/empty"
--no file open, try 'help open'
+ extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
- == refuse images with invalid compressed block size ==
++# We always write the first byte; check how many blocks this filesystem
- can't open device TEST_DIR/simple-pattern.cloop: invalid compressed block size at index 1, image file is corrupt
++# allocates to match empty image alloation.
--no file open, try 'help open'
++printf "\0" > "$TEST_DIR/empty"
 +min_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 +
  echo
  echo "== creating image with default preallocation =="
  _make_test_img $size | _filter_imgfmt
 -stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
 +stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
  for mode in off full falloc; do
      echo
      echo "== creating image with preallocation $mode =="
      IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
 -    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
 +    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
  done
  # success, all done
 diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/175.out
 +++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ QA output created by 175
  == creating image with default preallocation ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 -size=1048576, nothing allocated
 +size=1048576, min allocation
  == creating image with preallocation off ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
 -size=1048576, nothing allocated
 +size=1048576, min allocation
  == creating image with preallocation full ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
 -size=1048576, everything allocated
 +size=1048576, max allocation
  == creating image with preallocation falloc ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
 -size=1048576, everything allocated
 +size=1048576, max allocation
   *** done
 diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/178.out.qcow2
 +++ b/tests/qemu-iotests/178.out.qcow2
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
  == raw input image with data (human) ==
  Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
 -required size: 393216
 +required size: 458752
  fully allocated size: 1074135040
  wrote 512/512 bytes at offset 512
 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
  Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
  {
 -    "required": 393216,
 +    "required": 458752,
      "fully-allocated": 1074135040
  }
  wrote 512/512 bytes at offset 512
 diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/221.out
 +++ b/tests/qemu-iotests/221.out
@@ -XXX,XX +XXX,XX @@ QA output created by 221
  === Check mapping of unaligned raw image ===
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537
 -[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 -[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
  wrote 1/1 bytes at offset 65536
 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 -[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
  { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
  { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 -[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 +[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 +{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
  { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
  { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
  *** done
-diff --git a/tests/qemu-iotests/076.out b/tests/qemu-iotests/076.out
+diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out
 index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/076.out
+--- a/tests/qemu-iotests/253.out
-+++ b/tests/qemu-iotests/076.out
++++ b/tests/qemu-iotests/253.out
-@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 0
+@@ -XXX,XX +XXX,XX @@ QA output created by 253
+ === Check mapping of unaligned raw image ===
- == Negative catalog size ==
- can't open device TEST_DIR/parallels-v1: Catalog too large
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048575
--no file open, try 'help open'
+-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
- == Overflow in catalog allocation ==
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
- can't open device TEST_DIR/parallels-v1: Catalog too large
++{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
--no file open, try 'help open'
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
++{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
- == Zero sectors per track ==
+ wrote 65535/65535 bytes at offset 983040
- can't open device TEST_DIR/parallels-v1: Invalid image: Zero sectors per track
+.999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--no file open, try 'help open'
+-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
- == Read from a valid v2 image ==
++{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
- read 65536/65536 bytes at offset 0
+ { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
-diff --git a/tests/qemu-iotests/078.out b/tests/qemu-iotests/078.out
+-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
-index XXXXXXX..XXXXXXX 100644
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
---- a/tests/qemu-iotests/078.out
++{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
-+++ b/tests/qemu-iotests/078.out
+ { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 0
  == Negative catalog size ==
  can't open device TEST_DIR/empty.bochs: Catalog size is too large
 -no file open, try 'help open'
  == Overflow for catalog size * sizeof(uint32_t) ==
  can't open device TEST_DIR/empty.bochs: Catalog size is too large
 -no file open, try 'help open'
  == Too small catalog bitmap for image size ==
  can't open device TEST_DIR/empty.bochs: Catalog size is too small for this disk size
 -no file open, try 'help open'
  can't open device TEST_DIR/empty.bochs: Catalog size is too small for this disk size
 -no file open, try 'help open'
  == Negative extent size ==
  can't open device TEST_DIR/empty.bochs: Extent size 2147483648 is too large
 -no file open, try 'help open'
  == Zero extent size ==
  can't open device TEST_DIR/empty.bochs: Extent size must be at least 512
 -no file open, try 'help open'
  *** done
 diff --git a/tests/qemu-iotests/080.out b/tests/qemu-iotests/080.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/080.out
 +++ b/tests/qemu-iotests/080.out
@@ -XXX,XX +XXX,XX @@ QA output created by 080
  == Huge header size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: qcow2 header exceeds cluster size
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: qcow2 header exceeds cluster size
 -no file open, try 'help open'
  == Huge unknown header extension ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: Invalid backing file offset
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Header extension too large
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Header extension too large
 -no file open, try 'help open'
  == Huge refcount table size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: Reference count table too large
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Reference count table too large
 -no file open, try 'help open'
  == Misaligned refcount table ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: Invalid reference count table offset
 -no file open, try 'help open'
  == Huge refcount offset ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: Invalid reference count table offset
 -no file open, try 'help open'
  == Invalid snapshot table ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: Too many snapshots
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Too many snapshots
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Invalid snapshot table offset
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Invalid snapshot table offset
 -no file open, try 'help open'
  == Hitting snapshot table size limit ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 0
  == Invalid L1 table ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: Active L1 table too large
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Active L1 table too large
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Invalid L1 table offset
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow2: Invalid L1 table offset
 -no file open, try 'help open'
  == Invalid L1 table (with internal snapshot in the image) ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
@@ -XXX,XX +XXX,XX @@ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': L1 table is too small
  == Invalid backing file size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow2: Backing file name too long
 -no file open, try 'help open'
  == Invalid L2 entry (huge physical offset) ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 diff --git a/tests/qemu-iotests/083.out b/tests/qemu-iotests/083.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/083.out
 +++ b/tests/qemu-iotests/083.out
@@ -XXX,XX +XXX,XX @@ QA output created by 083
  === Check disconnect before neg1 ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect after neg1 ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect 8 neg1 ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect 16 neg1 ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect before export ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect after export ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect 4 export ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect 12 export ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect 16 export ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect before neg2 ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect after neg2 ===
@@ -XXX,XX +XXX,XX @@ read failed: Input/output error
  === Check disconnect 8 neg2 ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect 10 neg2 ===
  can't open device nbd:127.0.0.1:PORT:exportname=foo
 -no file open, try 'help open'
  === Check disconnect before request ===
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 0
  === Check disconnect before neg-classic ===
  can't open device nbd:127.0.0.1:PORT
 -no file open, try 'help open'
  === Check disconnect 8 neg-classic ===
  can't open device nbd:127.0.0.1:PORT
 -no file open, try 'help open'
  === Check disconnect 16 neg-classic ===
  can't open device nbd:127.0.0.1:PORT
 -no file open, try 'help open'
  === Check disconnect 24 neg-classic ===
  can't open device nbd:127.0.0.1:PORT
 -no file open, try 'help open'
  === Check disconnect 28 neg-classic ===
  can't open device nbd:127.0.0.1:PORT
 -no file open, try 'help open'
  === Check disconnect after neg-classic ===
 diff --git a/tests/qemu-iotests/088.out b/tests/qemu-iotests/088.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/088.out
 +++ b/tests/qemu-iotests/088.out
@@ -XXX,XX +XXX,XX @@ QA output created by 088
  == Invalid block size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.vpc: Invalid block size 0
 -no file open, try 'help open'
  can't open device TEST_DIR/t.vpc: Invalid block size 0
 -no file open, try 'help open'
  can't open device TEST_DIR/t.vpc: Invalid block size 128
 -no file open, try 'help open'
  can't open device TEST_DIR/t.vpc: Invalid block size 128
 -no file open, try 'help open'
  can't open device TEST_DIR/t.vpc: Invalid block size 305419896
 -no file open, try 'help open'
  can't open device TEST_DIR/t.vpc: Invalid block size 305419896
 -no file open, try 'help open'
  *** done
 diff --git a/tests/qemu-iotests/092.out b/tests/qemu-iotests/092.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/092.out
 +++ b/tests/qemu-iotests/092.out
@@ -XXX,XX +XXX,XX @@ QA output created by 092
  == Invalid cluster size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
 -no file open, try 'help open'
  == Invalid L2 table size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
 -no file open, try 'help open'
  == Invalid size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow: Image too large
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: Image too large
 -no file open, try 'help open'
  == Invalid backing file length ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
  can't open device TEST_DIR/t.qcow: Backing file name too long
 -no file open, try 'help open'
  can't open device TEST_DIR/t.qcow: Backing file name too long
 -no file open, try 'help open'
  *** done
 diff --git a/tests/qemu-iotests/116.out b/tests/qemu-iotests/116.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/116.out
 +++ b/tests/qemu-iotests/116.out
@@ -XXX,XX +XXX,XX @@ QA output created by 116
  == truncated header cluster ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
  can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
 -no file open, try 'help open'
  == invalid header magic ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
  can't open device TEST_DIR/t.qed: Image not in QED format
 -no file open, try 'help open'
  == invalid cluster size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
  can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
 -no file open, try 'help open'
  == invalid table size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
  can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
 -no file open, try 'help open'
  == invalid header size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
  can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
 -no file open, try 'help open'
  == invalid L1 table offset ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
  can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
 -no file open, try 'help open'
  == invalid image size ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
  can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
 -no file open, try 'help open'
  *** done
 diff --git a/tests/qemu-iotests/131.out b/tests/qemu-iotests/131.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/131.out
 +++ b/tests/qemu-iotests/131.out
@@ -XXX,XX +XXX,XX @@ read 32768/32768 bytes at offset 0
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  == Corrupt image ==
  can't open device TEST_DIR/t.parallels: parallels: Image was not closed correctly; cannot be opened read/write
 -no file open, try 'help open'
  ERROR image was not closed correctly
 errors were found on the image.
 diff --git a/tests/qemu-iotests/140.out b/tests/qemu-iotests/140.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/140.out
 +++ b/tests/qemu-iotests/140.out
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 0
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  {"return": {}}
  can't open device nbd+unix:///drv?socket=TEST_DIR/nbd: No export with name 'drv' available
 -no file open, try 'help open'
  {"return": {}}
  {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN"}
  *** done
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 18/21] qcow2: Optimize the refcount-block overlap check
+[Qemu-devel] [PULL 04/15] iotests: Test allocate_first_block() with O_DIRECT
-From: Alberto Garcia <berto@igalia.com>
+From: Nir Soffer <nirsof@gmail.com>
-The metadata overlap checks introduced in a40f1c2add help detect
+Using block_resize we can test allocate_first_block() with file
-corruption in the qcow2 image by verifying that data writes don't
+descriptor opened with O_DIRECT, ensuring that it works for any size
-overlap with existing metadata sections.
+larger than 4096 bytes.
-The 'refcount-block' check in particular iterates over the refcount
+Testing smaller sizes is tricky as the result depends on the filesystem
-table in order to get the addresses of all refcount blocks and check
+used for testing. For example on NFS any size will work since O_DIRECT
-that none of them overlap with the region where we want to write.
+does not require any alignment.
-The problem with the refcount table is that since it always occupies
+Signed-off-by: Nir Soffer <nsoffer@redhat.com>
 complete clusters its size is usually very big. With the default
 values of cluster_size=64KB and refcount_bits=16 this table holds 8192
 entries, each one of them enough to map 2GB worth of host clusters.
 So unless we're using images with several TB of allocated data this
 table is going to be mostly empty, and iterating over it is a waste of
 CPU. If the storage backend is fast enough this can have an effect on
 I/O performance.
 This patch keeps the index of the last used (i.e. non-zero) entry in
 the refcount table and updates it every time the table changes. The
 refcount-block overlap check then uses that index instead of reading
 the whole table.
 In my tests with a 4GB qcow2 file stored in RAM this doubles the
 amount of write IOPS.
 Signed-off-by: Alberto Garcia <berto@igalia.com>
 Message-id: 20170201123828.4815-1-berto@igalia.com
 Reviewed-by: Max Reitz <mreitz@redhat.com>
+Message-id: 20190827010528.8818-3-nsoffer@redhat.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/qcow2.h          |  1 +
+ tests/qemu-iotests/175     | 28 ++++++++++++++++++++++++++++
- block/qcow2-refcount.c | 24 +++++++++++++++++++++++-
+ tests/qemu-iotests/175.out |  8 ++++++++
- block/qcow2.c          |  1 +
+files changed, 36 insertions(+)
 files changed, 25 insertions(+), 1 deletion(-)
-diff --git a/block/qcow2.h b/block/qcow2.h
+diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/qcow2.h
+--- a/tests/qemu-iotests/175
-+++ b/block/qcow2.h
++++ b/tests/qemu-iotests/175
-@@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State {
+@@ -XXX,XX +XXX,XX @@ _filter_blocks()
-     uint64_t *refcount_table;
+         -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/max allocation/"
-     uint64_t refcount_table_offset;
+ }
-     uint32_t refcount_table_size;
-+    uint32_t max_refcount_table_index; /* Last used entry in refcount_table */
++# Resize image using block_resize.
-     uint64_t free_cluster_index;
++# Parameter 1: image path
-     uint64_t free_byte_offset;
++# Parameter 2: new size
++_block_resize()
 diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qcow2-refcount.c
 +++ b/block/qcow2-refcount.c
@@ -XXX,XX +XXX,XX @@ static Qcow2SetRefcountFunc *const set_refcount_funcs[] = {
  /*********************************************************/
  /* refcount handling */
 +static void update_max_refcount_table_index(BDRVQcow2State *s)
 +{
-+    unsigned i = s->refcount_table_size - 1;
++    local path=$1
-+    while (i > 0 && (s->refcount_table[i] & REFT_OFFSET_MASK) == 0) {
++    local size=$2
-+        i--;
++
-+    }
++    $QEMU -qmp stdio -nographic -nodefaults \
-+    /* Set s->max_refcount_table_index to the index of the last used entry */
++        -blockdev file,node-name=file,filename=$path,cache.direct=on \
-+    s->max_refcount_table_index = i;
++        <<EOF
 +{'execute': 'qmp_capabilities'}
 +{'execute': 'block_resize', 'arguments': {'node-name': 'file', 'size': $size}}
 +{'execute': 'quit'}
 +EOF
 +}
 +
- int qcow2_refcount_init(BlockDriverState *bs)
+ # get standard environment, filters and checks
- {
+ . ./common.rc
-     BDRVQcow2State *s = bs->opaque;
+ . ./common.filter
-@@ -XXX,XX +XXX,XX @@ int qcow2_refcount_init(BlockDriverState *bs)
+@@ -XXX,XX +XXX,XX @@ _supported_fmt raw
-         }
+ _supported_proto file
-         for(i = 0; i < s->refcount_table_size; i++)
+ _supported_os Linux
-             be64_to_cpus(&s->refcount_table[i]);
-+        update_max_refcount_table_index(s);
++_default_cache_mode none
-     }
++_supported_cache_modes none directsync
-     return 0;
++
-  fail:
+ size=$((1 * 1024 * 1024))
-@@ -XXX,XX +XXX,XX @@ static int alloc_refcount_block(BlockDriverState *bs,
-         }
+ touch "$TEST_DIR/empty"
+@@ -XXX,XX +XXX,XX @@ for mode in off full falloc; do
-         s->refcount_table[refcount_table_index] = new_block;
+     stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
-+        /* If there's a hole in s->refcount_table then it can happen
+ done
-+         * that refcount_table_index < s->max_refcount_table_index */
-+        s->max_refcount_table_index =
++for new_size in 4096 1048576; do
-+            MAX(s->max_refcount_table_index, refcount_table_index);
++    echo
++    echo "== resize empty image with block_resize =="
-         /* The new refcount block may be where the caller intended to put its
++    _make_test_img 0 | _filter_imgfmt
-          * data, so let it restart the search. */
++    _block_resize $TEST_IMG $new_size >/dev/null
-@@ -XXX,XX +XXX,XX @@ static int alloc_refcount_block(BlockDriverState *bs,
++    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $new_size
-     s->refcount_table = new_table;
++done
-     s->refcount_table_size = table_size;
++
-     s->refcount_table_offset = table_offset;
+ # success, all done
-+    update_max_refcount_table_index(s);
+ echo "*** done"
+ rm -f $seq.full
-     /* Free old table. */
+diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
      qcow2_free_clusters(bs, old_table_offset, old_table_size * sizeof(uint64_t),
@@ -XXX,XX +XXX,XX @@ write_refblocks:
      s->refcount_table = on_disk_reftable;
      s->refcount_table_offset = reftable_offset;
      s->refcount_table_size = reftable_size;
 +    update_max_refcount_table_index(s);
      return 0;
@@ -XXX,XX +XXX,XX @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
      }
      if ((chk & QCOW2_OL_REFCOUNT_BLOCK) && s->refcount_table) {
 -        for (i = 0; i < s->refcount_table_size; i++) {
 +        unsigned last_entry = s->max_refcount_table_index;
 +        assert(last_entry < s->refcount_table_size);
 +        assert(last_entry + 1 == s->refcount_table_size ||
 +               (s->refcount_table[last_entry + 1] & REFT_OFFSET_MASK) == 0);
 +        for (i = 0; i <= last_entry; i++) {
              if ((s->refcount_table[i] & REFT_OFFSET_MASK) &&
                  overlaps_with(s->refcount_table[i] & REFT_OFFSET_MASK,
                  s->cluster_size)) {
@@ -XXX,XX +XXX,XX @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
      /* Now update the rest of the in-memory information */
      old_reftable = s->refcount_table;
      s->refcount_table = new_reftable;
 +    update_max_refcount_table_index(s);
      s->refcount_bits = 1 << refcount_order;
      s->refcount_max = UINT64_C(1) << (s->refcount_bits - 1);
 diff --git a/block/qcow2.c b/block/qcow2.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2.c
+--- a/tests/qemu-iotests/175.out
-+++ b/block/qcow2.c
++++ b/tests/qemu-iotests/175.out
-@@ -XXX,XX +XXX,XX @@ static int make_completely_empty(BlockDriverState *bs)
+@@ -XXX,XX +XXX,XX @@ size=1048576, max allocation
+ == creating image with preallocation falloc ==
-     s->refcount_table_offset = s->cluster_size;
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
-     s->refcount_table_size   = s->cluster_size / sizeof(uint64_t);
+ size=1048576, max allocation
-+    s->max_refcount_table_index = 0;
++
++== resize empty image with block_resize ==
-     g_free(s->refcount_table);
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
-     s->refcount_table = new_reftable;
++size=4096, min allocation
 +
 +== resize empty image with block_resize ==
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
 +size=1048576, min allocation
   *** done
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 01/21] iotests: Fix a problem in common.filter
+[Qemu-devel] [PULL 05/15] iotests: Fix _filter_img_create()
-From: QingFeng Hao <haoqf@linux.vnet.ibm.com>
+fe646693acc changed qemu-img create's output so that it no longer prints
 single quotes around parameter values.  The subformat and adapter_type
 filters in _filter_img_create() have never been adapted to that change.
-If TEST_DIR is set to /tmp, test case 144 will fail. The reason is that
+Fixes: fe646693acc13ac48b98435d14149ab04dc597bc
-TEST_DIR resembles 144's test image name tmp.qcow2.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
-When 144 is testing $TEST_DIR/tmp.qcow2, it wants to replace
+Reviewed-by: John Snow <jsnow@redhat.com>
-$TEST_DIR/tmp.qcow2 to TEST_DIR/tmp.qcow2, but actually it will fail
+Message-id: 20190815153638.4600-2-mreitz@redhat.com
-and get TEST_DIRTEST_DIR.qcow2 in this case.
+Reviewed-by: John Snow <jsnow@redhat.com>
 The fix is just to modify the code to replace $TEST_DIR/ with TEST_DIR/.
 Signed-off-by: QingFeng Hao <haoqf@linux.vnet.ibm.com>
 Message-id: 20161216054723.96055-2-haoqf@linux.vnet.ibm.com
 Reviewed-by: Eric Blake <eblake@redhat.com>
 [mreitz: Fixed commit message and dropped superfluous escaping]
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/common.filter | 2 +-
+ tests/qemu-iotests/059.out       | 16 ++++++++--------
-file changed, 1 insertion(+), 1 deletion(-)
+ tests/qemu-iotests/common.filter |  4 ++--
 files changed, 10 insertions(+), 10 deletions(-)
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/059.out
++++ b/tests/qemu-iotests/059.out
+@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+ qemu-io: can't open device TEST_DIR/t.vmdk: L1 size too big
+ === Testing monolithicFlat creation and opening ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
+ image: TEST_DIR/t.IMGFMT
+ file format: IMGFMT
+ virtual size: 2 GiB (2147483648 bytes)
+ === Testing monolithicFlat with zeroed_grain ===
+ qemu-img: TEST_DIR/t.IMGFMT: Flat image can't enable zeroed grain
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
+ === Testing big twoGbMaxExtentFlat ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMaxExtentFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000
+ image: TEST_DIR/t.vmdk
+ file format: vmdk
+ virtual size: 0.977 TiB (1073741824000 bytes)
+@@ -XXX,XX +XXX,XX @@ Format specific information:
+ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Invalid extent line: RW 12582912 VMFS "dummy.IMGFMT" 1
+ === Testing truncated sparse ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400 subformat=monolithicSparse
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400
+ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': File truncated, expecting at least 13172736 bytes
+ === Converting to streamOptimized from image with small cluster size===
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing monolithicFlat with internally generated JSON file name ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+ qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
+ === Testing version 3 ===
+@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 64931328
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing 4TB monolithicFlat creation and IO ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104
+ image: TEST_DIR/t.IMGFMT
+ file format: IMGFMT
+ virtual size: 4 TiB (4398046511104 bytes)
+@@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 966367641600
+KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing qemu-img map on extents ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=monolithicSparse
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
+ wrote 1024/1024 bytes at offset 65024
+KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ wrote 1024/1024 bytes at offset 2147483136
+@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
+0x20000         0x3f0000        TEST_DIR/t.vmdk
+x7fff0000      0x20000         0x410000        TEST_DIR/t.vmdk
+x140000000     0x10000         0x430000        TEST_DIR/t.vmdk
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=twoGbMaxExtentSparse
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
+ wrote 1024/1024 bytes at offset 65024
+KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ wrote 1024/1024 bytes at offset 2147483136
 diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/common.filter
 +++ b/tests/qemu-iotests/common.filter
-@@ -XXX,XX +XXX,XX @@ _filter_generated_node_ids()
+@@ -XXX,XX +XXX,XX @@ _filter_img_create()
- # replace occurrences of the actual TEST_DIR value with TEST_DIR
+         -e "s# compat6=\\(on\\|off\\)##g" \
- _filter_testdir()
+         -e "s# static=\\(on\\|off\\)##g" \
- {
+         -e "s# zeroed_grain=\\(on\\|off\\)##g" \
--    sed -e "s#$TEST_DIR#TEST_DIR#g"
+-        -e "s# subformat='[^']*'##g" \
-+    sed -e "s#$TEST_DIR/#TEST_DIR/#g"
+-        -e "s# adapter_type='[^']*'##g" \
- }
++        -e "s# subformat=[^ ]*##g" \
++        -e "s# adapter_type=[^ ]*##g" \
- # replace occurrences of the actual IMGFMT value with IMGFMT
+         -e "s# hwversion=[^ ]*##g" \
          -e "s# lazy_refcounts=\\(on\\|off\\)##g" \
          -e "s# block_size=[0-9]\\+##g" \
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 11/21] block/qapi: reduce the execution time of qmp_query_blockstats
+[Qemu-devel] [PULL 06/15] vmdk: Use bdrv_dirname() for relative extent paths
-From: Dou Liyang <douly.fnst@cn.fujitsu.com>
+This makes iotest 033 pass with e.g. subformat=monolithicFlat.  It also
 turns a former error in 059 into success.
-In order to reduce the execution time, this patch optimize
+Signed-off-by: Max Reitz <mreitz@redhat.com>
-the qmp_query_blockstats():
+Message-id: 20190815153638.4600-3-mreitz@redhat.com
-Remove the next_query_bds function.
+Reviewed-by: John Snow <jsnow@redhat.com>
 Remove the bdrv_query_stats function.
 Remove some judgement sentence.
 The original qmp_query_blockstats calls next_query_bds to get
 the next objects in each loops. In the next_query_bds, it checks
 the query_nodes and blk. It also call bdrv_query_stats to get
 the stats, In the bdrv_query_stats, it checks blk and bs each
 times. This waste more times, which may stall the main loop a
 bit. And if the disk is too many and donot use the dataplane
 feature, this may affect the performance in main loop thread.
 This patch removes that two functions, and makes the structure
 clearly.
 Signed-off-by: Dou Liyang <douly.fnst@cn.fujitsu.com>
 Message-id: 1484467275-27919-3-git-send-email-douly.fnst@cn.fujitsu.com
 Reviewed-by: Markus Armbruster <armbru@redhat.com>
 [mreitz: Removed duplicate info->value assignment]
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/qapi.c | 73 ++++++++++++++++++++++++------------------------------------
+ block/vmdk.c               | 54 ++++++++++++++++++++++++--------------
-file changed, 29 insertions(+), 44 deletions(-)
+ tests/qemu-iotests/059     |  7 +++--
  tests/qemu-iotests/059.out |  4 ++-
 files changed, 42 insertions(+), 23 deletions(-)
-diff --git a/block/qapi.c b/block/qapi.c
+diff --git a/block/vmdk.c b/block/vmdk.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/qapi.c
+--- a/block/vmdk.c
-+++ b/block/qapi.c
++++ b/block/vmdk.c
-@@ -XXX,XX +XXX,XX @@ static BlockStats *bdrv_query_bds_stats(const BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ static const char *next_line(const char *s)
      return s;
  }
--static BlockStats *bdrv_query_stats(BlockBackend *blk,
+ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
--                                    const BlockDriverState *bs,
+-                              const char *desc_file_path, QDict *options,
--                                    bool query_backing)
+-                              Error **errp)
--{
++                              QDict *options, Error **errp)
 -    BlockStats *s;
 -
 -    s = bdrv_query_bds_stats(bs, query_backing);
 -
 -    if (blk) {
 -        s->has_device = true;
 -        s->device = g_strdup(blk_name(blk));
 -        bdrv_query_blk_stats(s->stats, blk);
 -    }
 -
 -    return s;
 -}
 -
  BlockInfoList *qmp_query_block(Error **errp)
  {
-     BlockInfoList *head = NULL, **p_next = &head;
+     int ret;
-@@ -XXX,XX +XXX,XX @@ BlockInfoList *qmp_query_block(Error **errp)
+     int matches;
-     return head;
+@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
      const char *p, *np;
      int64_t sectors = 0;
      int64_t flat_offset;
 +    char *desc_file_dir = NULL;
      char *extent_path;
      BdrvChild *extent_file;
      BDRVVmdkState *s = bs->opaque;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
              continue;
          }
 -        if (!path_is_absolute(fname) && !path_has_protocol(fname) &&
 -            !desc_file_path[0])
 -        {
 -            bdrv_refresh_filename(bs->file->bs);
 -            error_setg(errp, "Cannot use relative extent paths with VMDK "
 -                       "descriptor file '%s'", bs->file->bs->filename);
 -            return -EINVAL;
 -        }
 +        if (path_is_absolute(fname)) {
 +            extent_path = g_strdup(fname);
 +        } else {
 +            if (!desc_file_dir) {
 +                desc_file_dir = bdrv_dirname(bs->file->bs, errp);
 +                if (!desc_file_dir) {
 +                    bdrv_refresh_filename(bs->file->bs);
 +                    error_prepend(errp, "Cannot use relative paths with VMDK "
 +                                  "descriptor file '%s': ",
 +                                  bs->file->bs->filename);
 +                    ret = -EINVAL;
 +                    goto out;
 +                }
 +            }
 -        extent_path = path_combine(desc_file_path, fname);
 +            extent_path = g_strconcat(desc_file_dir, fname, NULL);
 +        }
          ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
          assert(ret < 32);
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
          g_free(extent_path);
          if (local_err) {
              error_propagate(errp, local_err);
 -            return -EINVAL;
 +            ret = -EINVAL;
 +            goto out;
          }
          /* save to extents array */
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
 , 0, 0, 0, 0, &extent, errp);
              if (ret < 0) {
                  bdrv_unref_child(bs, extent_file);
 -                return ret;
 +                goto out;
              }
              extent->flat_start_offset = flat_offset << 9;
          } else if (!strcmp(type, "SPARSE") || !strcmp(type, "VMFSSPARSE")) {
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
              g_free(buf);
              if (ret) {
                  bdrv_unref_child(bs, extent_file);
 -                return ret;
 +                goto out;
              }
              extent = &s->extents[s->num_extents - 1];
          } else if (!strcmp(type, "SESPARSE")) {
              ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
              if (ret) {
                  bdrv_unref_child(bs, extent_file);
 -                return ret;
 +                goto out;
              }
              extent = &s->extents[s->num_extents - 1];
          } else {
              error_setg(errp, "Unsupported extent type '%s'", type);
              bdrv_unref_child(bs, extent_file);
 -            return -ENOTSUP;
 +            ret = -ENOTSUP;
 +            goto out;
          }
          extent->type = g_strdup(type);
      }
 -    return 0;
 +
 +    ret = 0;
 +    goto out;
  invalid:
      np = next_line(p);
@@ -XXX,XX +XXX,XX @@ invalid:
          np--;
      }
      error_setg(errp, "Invalid extent line: %.*s", (int)(np - p), p);
 -    return -EINVAL;
 +    ret = -EINVAL;
 +
 +out:
 +    g_free(desc_file_dir);
 +    return ret;
  }
--static bool next_query_bds(BlockBackend **blk, BlockDriverState **bs,
+ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
--                           bool query_nodes)
+@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
 -{
 -    if (query_nodes) {
 -        *bs = bdrv_next_node(*bs);
 -        return !!*bs;
 -    }
 -
 -    *blk = blk_next(*blk);
 -    *bs = *blk ? blk_bs(*blk) : NULL;
 -
 -    return !!*blk;
 -}
 -
  BlockStatsList *qmp_query_blockstats(bool has_query_nodes,
                                       bool query_nodes,
                                       Error **errp)
  {
      BlockStatsList *head = NULL, **p_next = &head;
 -    BlockBackend *blk = NULL;
 -    BlockDriverState *bs = NULL;
 +    BlockBackend *blk;
 +    BlockDriverState *bs;
      /* Just to be safe if query_nodes is not always initialized */
 -    query_nodes = has_query_nodes && query_nodes;
 -
 -    while (next_query_bds(&blk, &bs, query_nodes)) {
 -        BlockStatsList *info = g_malloc0(sizeof(*info));
 -        AioContext *ctx = blk ? blk_get_aio_context(blk)
 -                              : bdrv_get_aio_context(bs);
 +    if (has_query_nodes && query_nodes) {
 +        for (bs = bdrv_next_node(NULL); bs; bs = bdrv_next_node(bs)) {
 +            BlockStatsList *info = g_malloc0(sizeof(*info));
 +            AioContext *ctx = bdrv_get_aio_context(bs);
 -        aio_context_acquire(ctx);
 -        info->value = bdrv_query_stats(blk, bs, !query_nodes);
 -        aio_context_release(ctx);
 +            aio_context_acquire(ctx);
 +            info->value = bdrv_query_bds_stats(bs, false);
 +            aio_context_release(ctx);
 -        *p_next = info;
 -        p_next = &info->next;
 +            *p_next = info;
 +            p_next = &info->next;
 +        }
 +    } else {
 +        for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
 +            BlockStatsList *info = g_malloc0(sizeof(*info));
 +            AioContext *ctx = blk_get_aio_context(blk);
 +            BlockStats *s;
 +
 +            aio_context_acquire(ctx);
 +            s = bdrv_query_bds_stats(blk_bs(blk), true);
 +            s->has_device = true;
 +            s->device = g_strdup(blk_name(blk));
 +            bdrv_query_blk_stats(s->stats, blk);
 +            aio_context_release(ctx);
 +
 +            info->value = s;
 +            *p_next = info;
 +            p_next = &info->next;
 +        }
      }
+     s->create_type = g_strdup(ct);
-     return head;
+     s->desc_offset = 0;
 -    ret = vmdk_parse_extents(buf, bs, bs->file->bs->exact_filename, options,
 -                             errp);
 +    ret = vmdk_parse_extents(buf, bs, options, errp);
  exit:
      return ret;
  }
 diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/059
 +++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
  echo
  echo "=== Testing monolithicFlat with internally generated JSON file name ==="
 +# Should work, because bdrv_dirname() works fine with blkdebug
  IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
 -$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" 2>&1 \
 -    | _filter_testdir | _filter_imgfmt
 +$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
 +         -c info \
 +    2>&1 \
 +    | _filter_testdir | _filter_imgfmt | _filter_img_info
  _cleanup_test_img
  echo
 diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/059.out
 +++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
  === Testing monolithicFlat with internally generated JSON file name ===
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 -qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
 +format name: IMGFMT
 +cluster size: 0 bytes
 +vm state offset: 0 bytes
  === Testing version 3 ===
  image: TEST_DIR/iotest-version3.IMGFMT
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 21/21] qemu-img: Avoid setting ret to unused value in img_convert()
+[Qemu-devel] [PULL 07/15] iotests: Keep testing broken relative extent paths
-From: Peter Maydell <peter.maydell@linaro.org>
+We had a test for a case where relative extent paths did not work, but
 unfortunately we just fixed the underlying problem, so it works now.
 This patch adds a new test case that still fails.
-Coverity points out that we assign the return value from
+Signed-off-by: Max Reitz <mreitz@redhat.com>
-bdrv_snapshot_load_tmp() to 'ret' in img_convert(), but then
+Reviewed-by: John Snow <jsnow@redhat.com>
-never use that variable. (We check for failure by looking
+Message-id: 20190815153638.4600-4-mreitz@redhat.com
-at local_err instead.) Drop the unused assignment, bringing
+Reviewed-by: John Snow <jsnow@redhat.com>
 the call into line with the following call to
 bdrv_snapshot_laod_tmp_by_id_or_name().
 (Fixes CID 1247240.)
 Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
 Message-id: 1486744104-15590-3-git-send-email-peter.maydell@linaro.org
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qemu-img.c | 8 ++++----
+ tests/qemu-iotests/059     | 27 +++++++++++++++++++++++++++
-file changed, 4 insertions(+), 4 deletions(-)
+ tests/qemu-iotests/059.out |  4 ++++
 files changed, 31 insertions(+)
-diff --git a/qemu-img.c b/qemu-img.c
+diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/059
 +++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
  echo
  echo "=== Testing monolithicFlat with internally generated JSON file name ==="
 +
 +echo '--- blkdebug ---'
  # Should work, because bdrv_dirname() works fine with blkdebug
  IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
  $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TE
      | _filter_testdir | _filter_imgfmt | _filter_img_info
  _cleanup_test_img
 +echo '--- quorum ---'
 +# Should not work, because bdrv_dirname() does not work with quorum
 +IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
 +cp "$TEST_IMG" "$TEST_IMG.orig"
 +
 +filename="json:{
 +    \"driver\": \"$IMGFMT\",
 +    \"file\": {
 +        \"driver\": \"quorum\",
 +        \"children\": [ {
 +            \"driver\": \"file\",
 +            \"filename\": \"$TEST_IMG\"
 +        }, {
 +            \"driver\": \"file\",
 +            \"filename\": \"$TEST_IMG.orig\"
 +        } ],
 +        \"vote-threshold\": 1
 +    } }"
 +
 +filename=$(echo "$filename" | tr '\n' ' ' | sed -e 's/\s\+/ /g')
 +$QEMU_IMG info "$filename" 2>&1 \
 +    | sed -e "s/'json:[^']*'/\$QUORUM_FILE/g" \
 +    | _filter_testdir | _filter_imgfmt | _filter_img_info
 +
 +
  echo
  echo "=== Testing version 3 ==="
  _use_sample_img iotest-version3.vmdk.bz2
 diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
 index XXXXXXX..XXXXXXX 100644
---- a/qemu-img.c
+--- a/tests/qemu-iotests/059.out
-+++ b/qemu-img.c
++++ b/tests/qemu-iotests/059.out
-@@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv)
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
-     }
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-     if (sn_opts) {
+ === Testing monolithicFlat with internally generated JSON file name ===
--        ret = bdrv_snapshot_load_tmp(bs[0],
++--- blkdebug ---
--                                     qemu_opt_get(sn_opts, SNAPSHOT_OPT_ID),
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
--                                     qemu_opt_get(sn_opts, SNAPSHOT_OPT_NAME),
+ format name: IMGFMT
--                                     &local_err);
+ cluster size: 0 bytes
-+        bdrv_snapshot_load_tmp(bs[0],
+ vm state offset: 0 bytes
-+                               qemu_opt_get(sn_opts, SNAPSHOT_OPT_ID),
++--- quorum ---
-+                               qemu_opt_get(sn_opts, SNAPSHOT_OPT_NAME),
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-+                               &local_err);
++qemu-img: Could not open $QUORUM_FILE: Cannot use relative paths with VMDK descriptor file $QUORUM_FILE: Cannot generate a base directory for quorum nodes
-     } else if (snapshot_name != NULL) {
-         if (bs_n > 1) {
+ === Testing version 3 ===
-             error_report("No support for concatenating multiple snapshot");
+ image: TEST_DIR/iotest-version3.IMGFMT
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 13/21] block/nfs: fix NULL pointer dereference in URI parsing
+[Qemu-devel] [PULL 08/15] vmdk: Reject invalid compressed writes
-From: Peter Lieven <pl@kamp.de>
+Compressed writes generally have to write full clusters, not just in
 theory but also in practice when it comes to vmdk's streamOptimized
 subformat.  It currently is just silently broken for writes with
 non-zero in-cluster offsets:
-parse_uint_full wants to put the parsed value into the
+$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
-variable passed via its second argument which is NULL.
+$ qemu-io -c 'write 4k 4k' -c 'read 4k 4k' foo.vmdk
 wrote 4096/4096 bytes at offset 4096
 KiB, 1 ops; 00.01 sec (443.724 KiB/sec and 110.9309 ops/sec)
 read failed: Invalid argument
-Fixes: 94d6a7a76e9df9919629428f6c598e2b97d9426c
+(The technical reason is that vmdk_write_extent() just writes the
-Cc: qemu-stable@nongnu.org
+incomplete compressed data actually to offset 4k.  When reading the
-Signed-off-by: Peter Lieven <pl@kamp.de>
+data, vmdk_read_extent() looks at offset 0 and finds the compressed data
-Reviewed-by: Eric Blake <eblake@redhat.com>
+size to be 0, because that is what it reads from there.  This yields an
-Message-id: 1485942829-10756-2-git-send-email-pl@kamp.de
+error.)
 For incomplete writes with zero in-cluster offsets, the error path when
 reading the rest of the cluster is a bit different, but the result is
 the same:
 $ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
 $ qemu-io -c 'write 0k 4k' -c 'read 4k 4k' foo.vmdk
 wrote 4096/4096 bytes at offset 0
 KiB, 1 ops; 00.01 sec (362.641 KiB/sec and 90.6603 ops/sec)
 read failed: Invalid argument
 (Here, vmdk_read_extent() finds the data and then sees that the
 uncompressed data is short.)
 It is better to reject invalid writes than to make the user believe they
 might have succeeded and then fail when trying to read it back.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 Reviewed-by: John Snow <jsnow@redhat.com>
 Message-id: 20190815153638.4600-5-mreitz@redhat.com
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/nfs.c | 3 ++-
+ block/vmdk.c | 10 ++++++++++
-file changed, 2 insertions(+), 1 deletion(-)
+file changed, 10 insertions(+)
-diff --git a/block/nfs.c b/block/nfs.c
+diff --git a/block/vmdk.c b/block/vmdk.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/nfs.c
+--- a/block/vmdk.c
-+++ b/block/nfs.c
++++ b/block/vmdk.c
-@@ -XXX,XX +XXX,XX @@ static int nfs_parse_uri(const char *filename, QDict *options, Error **errp)
+@@ -XXX,XX +XXX,XX @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset,
-     qdict_put(options, "path", qstring_from_str(uri->path));
+     if (extent->compressed) {
+         void *compressed_data;
-     for (i = 0; i < qp->n; i++) {
-+        unsigned long long val;
++        /* Only whole clusters */
-         if (!qp->p[i].value) {
++        if (offset_in_cluster ||
-             error_setg(errp, "Value for NFS parameter expected: %s",
++            n_bytes > (extent->cluster_sectors * SECTOR_SIZE) ||
-                        qp->p[i].name);
++            (n_bytes < (extent->cluster_sectors * SECTOR_SIZE) &&
-             goto out;
++             offset + n_bytes != extent->end_sector * SECTOR_SIZE))
-         }
++        {
--        if (parse_uint_full(qp->p[i].value, NULL, 0)) {
++            ret = -EINVAL;
-+        if (parse_uint_full(qp->p[i].value, &val, 0)) {
++            goto out;
-             error_setg(errp, "Illegal value for NFS parameter: %s",
++        }
-                        qp->p[i].name);
++
          if (!extent->has_marker) {
              ret = -EINVAL;
              goto out;
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 20/21] qemu-img: Use qemu_strtoul() rather than raw strtoul()
+[Qemu-devel] [PULL 09/15] iotests: Disable broken streamOptimized tests
-From: Peter Maydell <peter.maydell@linaro.org>
+streamOptimized does not support writes that do not span exactly one
 cluster.  Furthermore, it cannot rewrite already allocated clusters.
 As such, many iotests do not work with it.  Disable them.
-Some of the argument parsing in qemu-img uses strtoul() to parse
+Signed-off-by: Max Reitz <mreitz@redhat.com>
-integer arguments.  This is tricky to get correct and in fact the
+Message-id: 20190815153638.4600-6-mreitz@redhat.com
-code does not get it right, because it assigns the result of
+Reviewed-by: John Snow <jsnow@redhat.com>
 strtoul() to an 'int' variable and then tries to check for > INT_MAX.
 Coverity correctly complains that the comparison is always false.
 Rewrite to use qemu_strtoul(), which has a saner convention for
 reporting conversion failures.
 (Fixes CID 1356421, CID 1356422, CID 1356423.)
 Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
 Message-id: 1486744104-15590-2-git-send-email-peter.maydell@linaro.org
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qemu-img.c | 32 ++++++++++++++++----------------
+ tests/qemu-iotests/002 | 1 +
-file changed, 16 insertions(+), 16 deletions(-)
+ tests/qemu-iotests/003 | 1 +
  tests/qemu-iotests/005 | 3 ++-
  tests/qemu-iotests/009 | 1 +
  tests/qemu-iotests/010 | 1 +
  tests/qemu-iotests/011 | 1 +
  tests/qemu-iotests/017 | 3 ++-
  tests/qemu-iotests/018 | 3 ++-
  tests/qemu-iotests/019 | 3 ++-
  tests/qemu-iotests/020 | 3 ++-
  tests/qemu-iotests/027 | 1 +
  tests/qemu-iotests/032 | 1 +
  tests/qemu-iotests/033 | 1 +
  tests/qemu-iotests/034 | 3 ++-
  tests/qemu-iotests/037 | 3 ++-
  tests/qemu-iotests/063 | 3 ++-
  tests/qemu-iotests/072 | 1 +
  tests/qemu-iotests/105 | 3 ++-
  tests/qemu-iotests/197 | 1 +
  tests/qemu-iotests/215 | 1 +
  tests/qemu-iotests/251 | 1 +
 files changed, 30 insertions(+), 9 deletions(-)
-diff --git a/qemu-img.c b/qemu-img.c
+diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/qemu-img.c
+--- a/tests/qemu-iotests/002
-+++ b/qemu-img.c
++++ b/tests/qemu-iotests/002
-@@ -XXX,XX +XXX,XX @@ static int img_bench(int argc, char **argv)
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-             break;
-         case 'c':
+ _supported_fmt generic
-         {
+ _supported_proto generic
--            char *end;
++_unsupported_imgopts "subformat=streamOptimized"
--            errno = 0;
--            count = strtoul(optarg, &end, 0);
--            if (errno || *end || count > INT_MAX) {
+ size=128M
-+            unsigned long res;
+diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
-+
+index XXXXXXX..XXXXXXX 100755
-+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > INT_MAX) {
+--- a/tests/qemu-iotests/003
-                 error_report("Invalid request count specified");
++++ b/tests/qemu-iotests/003
-                 return 1;
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-             }
-+            count = res;
+ _supported_fmt generic
-             break;
+ _supported_proto generic
-         }
++_unsupported_imgopts "subformat=streamOptimized"
-         case 'd':
-         {
+ size=128M
--            char *end;
+ offset=67M
--            errno = 0;
+diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
--            depth = strtoul(optarg, &end, 0);
+index XXXXXXX..XXXXXXX 100755
--            if (errno || *end || depth > INT_MAX) {
+--- a/tests/qemu-iotests/005
-+            unsigned long res;
++++ b/tests/qemu-iotests/005
-+
+@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
-+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > INT_MAX) {
+ _supported_proto generic
-                 error_report("Invalid queue depth specified");
+ _supported_os Linux
-                 return 1;
+ _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
-             }
+-                     "subformat=twoGbMaxExtentSparse"
-+            depth = res;
++                     "subformat=twoGbMaxExtentSparse" \
-             break;
++                     "subformat=streamOptimized"
-         }
-         case 'f':
+ # vpc is limited to 127GB, so we can't test it here
-@@ -XXX,XX +XXX,XX @@ static int img_bench(int argc, char **argv)
+ if [ "$IMGFMT" = "vpc" ]; then
-             break;
+diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
-         case OPTION_PATTERN:
+index XXXXXXX..XXXXXXX 100755
-         {
+--- a/tests/qemu-iotests/009
--            char *end;
++++ b/tests/qemu-iotests/009
--            errno = 0;
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
--            pattern = strtoul(optarg, &end, 0);
--            if (errno || *end || pattern > 0xff) {
+ _supported_fmt generic
-+            unsigned long res;
+ _supported_proto generic
-+
++_unsupported_imgopts "subformat=streamOptimized"
-+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > 0xff) {
-                 error_report("Invalid pattern byte specified");
-                 return 1;
+ size=6G
-             }
+diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
-+            pattern = res;
+index XXXXXXX..XXXXXXX 100755
-             break;
+--- a/tests/qemu-iotests/010
-         }
++++ b/tests/qemu-iotests/010
-         case OPTION_FLUSH_INTERVAL:
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-         {
--            char *end;
+ _supported_fmt generic
--            errno = 0;
+ _supported_proto generic
--            flush_interval = strtoul(optarg, &end, 0);
++_unsupported_imgopts "subformat=streamOptimized"
--            if (errno || *end || flush_interval > INT_MAX) {
-+            unsigned long res;
-+
+ size=6G
-+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > INT_MAX) {
+diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
-                 error_report("Invalid flush interval specified");
+index XXXXXXX..XXXXXXX 100755
-                 return 1;
+--- a/tests/qemu-iotests/011
-             }
++++ b/tests/qemu-iotests/011
-+            flush_interval = res;
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-             break;
-         }
+ _supported_fmt generic
-         case OPTION_NO_DRAIN:
+ _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=6G
 diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/017
 +++ b/tests/qemu-iotests/017
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow qcow2 vmdk qed
  _supported_proto generic
  _unsupported_proto vxhs
 -_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 +_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
 +                     "subformat=streamOptimized"
  TEST_OFFSETS="0 4294967296"
 diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/018
 +++ b/tests/qemu-iotests/018
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow qcow2 vmdk qed
  _supported_proto file
  _supported_os Linux
 -_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
 +_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
 +                     "streamOptimized"
  TEST_OFFSETS="0 4294967296"
 diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/019
 +++ b/tests/qemu-iotests/019
@@ -XXX,XX +XXX,XX @@ _supported_proto file
  _supported_os Linux
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  TEST_OFFSETS="0 4294967296"
  CLUSTER_SIZE=65536
 diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/020
 +++ b/tests/qemu-iotests/020
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
  _supported_proto file
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  TEST_OFFSETS="0 4294967296"
 diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/027
 +++ b/tests/qemu-iotests/027
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt vmdk qcow qcow2 qed
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=128M
 diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/032
 +++ b/tests/qemu-iotests/032
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  # This works for any image format (though unlikely to segfault for raw)
  _supported_fmt generic
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  echo
  echo === Prepare image ===
 diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/033
 +++ b/tests/qemu-iotests/033
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto generic
 +_unsupported_imgopts "subformat=streamOptimized"
  size=128M
 diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/034
 +++ b/tests/qemu-iotests/034
@@ -XXX,XX +XXX,XX @@ _supported_proto file
  _supported_os Linux
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  CLUSTER_SIZE=4k
  size=128M
 diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/037
 +++ b/tests/qemu-iotests/037
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
  _supported_proto file
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  CLUSTER_SIZE=4k
  size=128M
 diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/063
 +++ b/tests/qemu-iotests/063
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed raw
  _supported_proto file
  _unsupported_imgopts "subformat=monolithicFlat" \
                       "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  _make_test_img 4M
 diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/072
 +++ b/tests/qemu-iotests/072
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
  _supported_proto file
 +_unsupported_imgopts "subformat=streamOptimized"
  IMG_SIZE=64M
 diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/105
 +++ b/tests/qemu-iotests/105
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow2 vmdk vhdx qed
  _supported_proto generic
  _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
 -                     "subformat=twoGbMaxExtentSparse"
 +                     "subformat=twoGbMaxExtentSparse" \
 +                     "subformat=streamOptimized"
  echo
  echo "creating large image"
 diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/197
 +++ b/tests/qemu-iotests/197
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
  _supported_proto generic
  # LUKS support may be possible, but it complicates things.
  _unsupported_fmt luks
 +_unsupported_imgopts "subformat=streamOptimized"
  echo
  echo '=== Copy-on-read ==='
 diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/215
 +++ b/tests/qemu-iotests/215
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
  _supported_proto generic
  # LUKS support may be possible, but it complicates things.
  _unsupported_fmt luks
 +_unsupported_imgopts "subformat=streamOptimized"
  echo
  echo '=== Copy-on-read ==='
 diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/251
 +++ b/tests/qemu-iotests/251
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto file
  _supported_os Linux
 +_unsupported_imgopts "subformat=streamOptimized"
  if [ "$IMGOPTSSYNTAX" = "true" ]; then
      # We use json:{} filenames here, so we cannot work with additional options.
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 03/21] qapi: Tweak error message of bdrv_query_image_info
+[Qemu-devel] [PULL 10/15] iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
-From: Fam Zheng <famz@redhat.com>
+The error message for the test case where we have a quorum node for
 which no directory name can be generated is different: For
 twoGbMaxExtentSparse, it complains that it cannot open the extent file.
 For other (sub)formats, it just notes that it cannot determine the
 backing file path.  Both are fine, but just disable twoGbMaxExtentSparse
 for simplicity's sake.
-@bs doesn't always have a device name, such as when it comes from
+Signed-off-by: Max Reitz <mreitz@redhat.com>
-"qemu-img info". Report file name instead.
+Reviewed-by: John Snow <jsnow@redhat.com>
+Message-id: 20190815153638.4600-7-mreitz@redhat.com
-Signed-off-by: Fam Zheng <famz@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
 Message-id: 20170119130759.28319-2-famz@redhat.com
 Reviewed-by: Eric Blake <eblake@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/qapi.c | 4 ++--
+ tests/qemu-iotests/110 | 3 ++-
-file changed, 2 insertions(+), 2 deletions(-)
+file changed, 2 insertions(+), 1 deletion(-)
-diff --git a/block/qapi.c b/block/qapi.c
+diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/qapi.c
+--- a/tests/qemu-iotests/110
-+++ b/block/qapi.c
++++ b/tests/qemu-iotests/110
-@@ -XXX,XX +XXX,XX @@ void bdrv_query_image_info(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ # Any format supporting backing files
-     size = bdrv_getlength(bs);
+ _supported_fmt qed qcow qcow2 vmdk
-     if (size < 0) {
+ _supported_proto file
--        error_setg_errno(errp, -size, "Can't get size of device '%s'",
+-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
--                         bdrv_get_device_name(bs));
++_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
-+        error_setg_errno(errp, -size, "Can't get image size '%s'",
++                     "subformat=twoGbMaxExtentSparse"
-+                         bs->exact_filename);
-         goto out;
+ TEST_IMG_REL=$(basename "$TEST_IMG")
      }
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 14/21] block/nfs: fix naming of runtime opts
+[Qemu-devel] [PULL 11/15] iotests: Disable 126 for flat vmdk subformats
-From: Peter Lieven <pl@kamp.de>
+iotest 126 requires backing file support, which flat vmdks cannot offer.
 Skip this test for such subformats.
-commit 94d6a7a accidentally left the naming of runtime opts and QAPI
+Signed-off-by: Max Reitz <mreitz@redhat.com>
-scheme inconsistent. As one consequence passing of parameters in the
+Message-id: 20190815153638.4600-8-mreitz@redhat.com
-URI is broken. Sync the naming of the runtime opts to the QAPI
+Reviewed-by: John Snow <jsnow@redhat.com>
 scheme.
 Please note that this is technically backwards incompatible with the 2.8
 release, but the 2.8 release is the only version that had the wrong naming.
 Furthermore release 2.8 suffered from a NULL pointer dereference during
 URI parsing.
 Fixes: 94d6a7a76e9df9919629428f6c598e2b97d9426c
 Cc: qemu-stable@nongnu.org
 Signed-off-by: Peter Lieven <pl@kamp.de>
 Message-id: 1485942829-10756-3-git-send-email-pl@kamp.de
 [mreitz: Fixed commit message]
 Reviewed-by: Eric Blake <eblake@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/nfs.c | 46 +++++++++++++++++++++++-----------------------
+ tests/qemu-iotests/126 | 2 ++
-file changed, 23 insertions(+), 23 deletions(-)
+file changed, 2 insertions(+)
-diff --git a/block/nfs.c b/block/nfs.c
+diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/nfs.c
+--- a/tests/qemu-iotests/126
-+++ b/block/nfs.c
++++ b/tests/qemu-iotests/126
-@@ -XXX,XX +XXX,XX @@ static QemuOptsList runtime_opts = {
+@@ -XXX,XX +XXX,XX @@ status=1    # failure is the default!
-             .help = "Path of the image on the host",
-         },
+ # Needs backing file support
-         {
+ _supported_fmt qcow qcow2 qed vmdk
--            .name = "uid",
++_unsupported_imgopts "subformat=monolithicFlat" \
-+            .name = "user",
++                     "subformat=twoGbMaxExtentFlat"
-             .type = QEMU_OPT_NUMBER,
+ # This is the default protocol (and we want to test the difference between
-             .help = "UID value to use when talking to the server",
+ # colons which separate a protocol prefix from the rest and colons which are
-         },
+ # just part of the filename, so we cannot test protocols which require a prefix)
          {
 -            .name = "gid",
 +            .name = "group",
              .type = QEMU_OPT_NUMBER,
              .help = "GID value to use when talking to the server",
          },
          {
 -            .name = "tcp-syncnt",
 +            .name = "tcp-syn-count",
              .type = QEMU_OPT_NUMBER,
              .help = "Number of SYNs to send during the session establish",
          },
          {
 -            .name = "readahead",
 +            .name = "readahead-size",
              .type = QEMU_OPT_NUMBER,
              .help = "Set the readahead size in bytes",
          },
          {
 -            .name = "pagecache",
 +            .name = "page-cache-size",
              .type = QEMU_OPT_NUMBER,
              .help = "Set the pagecache size in bytes",
          },
@@ -XXX,XX +XXX,XX @@ static int64_t nfs_client_open(NFSClient *client, QDict *options,
          goto fail;
      }
 -    if (qemu_opt_get(opts, "uid")) {
 -        client->uid = qemu_opt_get_number(opts, "uid", 0);
 +    if (qemu_opt_get(opts, "user")) {
 +        client->uid = qemu_opt_get_number(opts, "user", 0);
          nfs_set_uid(client->context, client->uid);
      }
 -    if (qemu_opt_get(opts, "gid")) {
 -        client->gid = qemu_opt_get_number(opts, "gid", 0);
 +    if (qemu_opt_get(opts, "group")) {
 +        client->gid = qemu_opt_get_number(opts, "group", 0);
          nfs_set_gid(client->context, client->gid);
      }
 -    if (qemu_opt_get(opts, "tcp-syncnt")) {
 -        client->tcp_syncnt = qemu_opt_get_number(opts, "tcp-syncnt", 0);
 +    if (qemu_opt_get(opts, "tcp-syn-count")) {
 +        client->tcp_syncnt = qemu_opt_get_number(opts, "tcp-syn-count", 0);
          nfs_set_tcp_syncnt(client->context, client->tcp_syncnt);
      }
  #ifdef LIBNFS_FEATURE_READAHEAD
 -    if (qemu_opt_get(opts, "readahead")) {
 +    if (qemu_opt_get(opts, "readahead-size")) {
          if (open_flags & BDRV_O_NOCACHE) {
              error_setg(errp, "Cannot enable NFS readahead "
                               "if cache.direct = on");
              goto fail;
          }
 -        client->readahead = qemu_opt_get_number(opts, "readahead", 0);
 +        client->readahead = qemu_opt_get_number(opts, "readahead-size", 0);
          if (client->readahead > QEMU_NFS_MAX_READAHEAD_SIZE) {
              error_report("NFS Warning: Truncating NFS readahead "
                           "size to %d", QEMU_NFS_MAX_READAHEAD_SIZE);
@@ -XXX,XX +XXX,XX @@ static int64_t nfs_client_open(NFSClient *client, QDict *options,
  #endif
  #ifdef LIBNFS_FEATURE_PAGECACHE
 -    if (qemu_opt_get(opts, "pagecache")) {
 +    if (qemu_opt_get(opts, "page-cache-size")) {
          if (open_flags & BDRV_O_NOCACHE) {
              error_setg(errp, "Cannot enable NFS pagecache "
                               "if cache.direct = on");
              goto fail;
          }
 -        client->pagecache = qemu_opt_get_number(opts, "pagecache", 0);
 +        client->pagecache = qemu_opt_get_number(opts, "page-cache-size", 0);
          if (client->pagecache > QEMU_NFS_MAX_PAGECACHE_SIZE) {
              error_report("NFS Warning: Truncating NFS pagecache "
                           "size to %d pages", QEMU_NFS_MAX_PAGECACHE_SIZE);
@@ -XXX,XX +XXX,XX @@ static void nfs_refresh_filename(BlockDriverState *bs, QDict *options)
      qdict_put(opts, "path", qstring_from_str(client->path));
      if (client->uid) {
 -        qdict_put(opts, "uid", qint_from_int(client->uid));
 +        qdict_put(opts, "user", qint_from_int(client->uid));
      }
      if (client->gid) {
 -        qdict_put(opts, "gid", qint_from_int(client->gid));
 +        qdict_put(opts, "group", qint_from_int(client->gid));
      }
      if (client->tcp_syncnt) {
 -        qdict_put(opts, "tcp-syncnt",
 -                      qint_from_int(client->tcp_syncnt));
 +        qdict_put(opts, "tcp-syn-cnt",
 +                  qint_from_int(client->tcp_syncnt));
      }
      if (client->readahead) {
 -        qdict_put(opts, "readahead",
 -                      qint_from_int(client->readahead));
 +        qdict_put(opts, "readahead-size",
 +                  qint_from_int(client->readahead));
      }
      if (client->pagecache) {
 -        qdict_put(opts, "pagecache",
 -                      qint_from_int(client->pagecache));
 +        qdict_put(opts, "page-cache-size",
 +                  qint_from_int(client->pagecache));
      }
      if (client->debug) {
          qdict_put(opts, "debug", qint_from_int(client->debug));
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 10/21] block/qapi: reduce the coupling between the bdrv_query_stats and bdrv_query_bds_stats
+[Qemu-devel] [PULL 12/15] file-posix: fix request_alignment typo
-From: Dou Liyang <douly.fnst@cn.fujitsu.com>
+From: Stefan Hajnoczi <stefanha@redhat.com>
-The bdrv_query_stats and bdrv_query_bds_stats functions need to call
+Fixes: a6b257a08e3d72219f03e461a52152672fec0612
-each other, that increases the coupling. it also makes the program
+       ("file-posix: Handle undetectable alignment")
-complicated and makes some unnecessary tests.
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-id: 20190827101328.4062-1-stefanha@redhat.com
-Remove the call from bdrv_query_bds_stats to bdrv_query_stats, just
+Reviewed-by: Thomas Huth <thuth@redhat.com>
 take some recursion to make it clearly.
 Avoid testing whether the blk is NULL during querying the bds stats.
 It is unnecessary.
 Signed-off-by: Dou Liyang <douly.fnst@cn.fujitsu.com>
 Message-id: 1484467275-27919-2-git-send-email-douly.fnst@cn.fujitsu.com
 Reviewed-by: Markus Armbruster <armbru@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/qapi.c | 26 ++++++++++++++------------
+ block/file-posix.c | 2 +-
-file changed, 14 insertions(+), 12 deletions(-)
+file changed, 1 insertion(+), 1 deletion(-)
-diff --git a/block/qapi.c b/block/qapi.c
+diff --git a/block/file-posix.c b/block/file-posix.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/qapi.c
+--- a/block/file-posix.c
-+++ b/block/qapi.c
++++ b/block/file-posix.c
-@@ -XXX,XX +XXX,XX @@ static void bdrv_query_info(BlockBackend *blk, BlockInfo **p_info,
+@@ -XXX,XX +XXX,XX @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
-     qapi_free_BlockInfo(info);
+         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
- }
+             align = alignments[i];
+             if (raw_is_io_aligned(fd, buf + align, max_align)) {
--static BlockStats *bdrv_query_stats(BlockBackend *blk,
+-                /* Fallback to request_aligment. */
--                                    const BlockDriverState *bs,
++                /* Fallback to request_alignment. */
--                                    bool query_backing);
+                 s->buf_align = (align != 1) ? align : bs->bl.request_alignment;
--
+                 break;
- static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
+             }
  {
      BlockAcctStats *stats = blk_get_stats(blk);
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
      }
  }
 -static void bdrv_query_bds_stats(BlockStats *s, const BlockDriverState *bs,
 +static BlockStats *bdrv_query_bds_stats(const BlockDriverState *bs,
                                   bool query_backing)
  {
 +    BlockStats *s = NULL;
 +
 +    s = g_malloc0(sizeof(*s));
 +    s->stats = g_malloc0(sizeof(*s->stats));
 +
 +    if (!bs) {
 +        return s;
 +    }
 +
      if (bdrv_get_node_name(bs)[0]) {
          s->has_node_name = true;
          s->node_name = g_strdup(bdrv_get_node_name(bs));
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_bds_stats(BlockStats *s, const BlockDriverState *bs,
      if (bs->file) {
          s->has_parent = true;
 -        s->parent = bdrv_query_stats(NULL, bs->file->bs, query_backing);
 +        s->parent = bdrv_query_bds_stats(bs->file->bs, query_backing);
      }
      if (query_backing && bs->backing) {
          s->has_backing = true;
 -        s->backing = bdrv_query_stats(NULL, bs->backing->bs, query_backing);
 +        s->backing = bdrv_query_bds_stats(bs->backing->bs, query_backing);
      }
 +    return s;
  }
  static BlockStats *bdrv_query_stats(BlockBackend *blk,
@@ -XXX,XX +XXX,XX @@ static BlockStats *bdrv_query_stats(BlockBackend *blk,
  {
      BlockStats *s;
 -    s = g_malloc0(sizeof(*s));
 -    s->stats = g_malloc0(sizeof(*s->stats));
 +    s = bdrv_query_bds_stats(bs, query_backing);
      if (blk) {
          s->has_device = true;
          s->device = g_strdup(blk_name(blk));
          bdrv_query_blk_stats(s->stats, blk);
      }
 -    if (bs) {
 -        bdrv_query_bds_stats(s, bs, query_backing);
 -    }
      return s;
  }
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 16/21] qemu-iotests: Add _unsupported_fmt helper
+[Qemu-devel] [PULL 13/15] iotests: Check for enabled drivers before testing them
-From: Nir Soffer <nirsof@gmail.com>
+From: Thomas Huth <thuth@redhat.com>
-This helper allows adding tests supporting any format expect the
+It is possible to enable only a subset of the block drivers with the
-specified formats. This may be useful to test that many formats behave
+"--block-drv-rw-whitelist" option of the "configure" script. All other
-in a common way.
+drivers are marked as unusable (or only included as read-only with the
 "--block-drv-ro-whitelist" option). If an iotest is now using such a
 disabled block driver, it is failing - which is bad, since at least the
 tests in the "auto" group should be able to deal with this situation.
 Thus let's introduce a "_require_drivers" function that can be used by
 the shell tests to check for the availability of certain drivers first,
 and marks the test as "not run" if one of the drivers is missing.
-Signed-off-by: Nir Soffer <nirsof@gmail.com>
+This patch mainly targets the test in the "auto" group which should
-Message-id: 20170201003120.23378-3-nirsof@gmail.com
+never fail in such a case, but also improves some of the other tests
-Reviewed-by: Max Reitz <mreitz@redhat.com>
+along the way. Note that we also assume that the "qcow2" and "file"
 drivers are always available - otherwise it does not make sense to
 run "make check-block" at all (which only tests with qcow2 by default).
 Signed-off-by: Thomas Huth <thuth@redhat.com>
 Message-id: 20190823133552.11680-1-thuth@redhat.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/common.rc | 11 +++++++++++
+ tests/qemu-iotests/071       |  1 +
-file changed, 11 insertions(+)
+ tests/qemu-iotests/081       |  4 +---
  tests/qemu-iotests/099       |  1 +
  tests/qemu-iotests/120       |  1 +
  tests/qemu-iotests/162       |  4 +---
  tests/qemu-iotests/184       |  1 +
  tests/qemu-iotests/186       |  1 +
  tests/qemu-iotests/common.rc | 14 ++++++++++++++
 files changed, 21 insertions(+), 6 deletions(-)
+diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/071
++++ b/tests/qemu-iotests/071
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow2
+ _supported_proto file
++_require_drivers blkdebug blkverify
+ do_run_qemu()
+ {
+diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/081
++++ b/tests/qemu-iotests/081
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt raw
+ _supported_proto file
+ _supported_os Linux
++_require_drivers quorum
+ do_run_qemu()
+ {
+@@ -XXX,XX +XXX,XX @@ run_qemu()
+                           | _filter_qemu_io | _filter_generated_node_ids
+ }
+-test_quorum=$($QEMU_IMG --help|grep quorum)
+-[ "$test_quorum" = "" ] && _supported_fmt quorum
+-
+ quorum="driver=raw,file.driver=quorum,file.vote-threshold=2"
+ quorum="$quorum,file.children.0.file.filename=$TEST_DIR/1.raw"
+ quorum="$quorum,file.children.1.file.filename=$TEST_DIR/2.raw"
+diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/099
++++ b/tests/qemu-iotests/099
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow qcow2 qed vdi vhdx vmdk vpc
+ _supported_proto file
+ _supported_os Linux
++_require_drivers blkdebug blkverify
+ _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+     "subformat=twoGbMaxExtentSparse"
+diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/120
++++ b/tests/qemu-iotests/120
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto file
+ _unsupported_fmt luks
++_require_drivers raw
+ _make_test_img 64M
+diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/162
++++ b/tests/qemu-iotests/162
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ . ./common.filter
+ _supported_fmt generic
+-
+-test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh\( \|$\)')
+-[ "$test_ssh" = "" ] && _notrun "ssh support required"
++_require_drivers ssh
+ echo
+ echo '=== NBD ==='
+diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/184
++++ b/tests/qemu-iotests/184
+@@ -XXX,XX +XXX,XX @@ trap "exit \$status" 0 1 2 3 15
+ . ./common.filter
+ _supported_os Linux
++_require_drivers throttle
+ do_run_qemu()
+ {
+diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/186
++++ b/tests/qemu-iotests/186
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow2
+ _supported_proto file
++_require_drivers null-co
+ if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
+     _notrun "Requires a PC machine"
 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/common.rc
 +++ b/tests/qemu-iotests/common.rc
-@@ -XXX,XX +XXX,XX @@ _supported_fmt()
+@@ -XXX,XX +XXX,XX @@ _require_command()
-     _notrun "not suitable for this image format: $IMGFMT"
+     [ -x "$c" ] || _notrun "$1 utility required, skipped this test"
  }
-+# tests whether $IMGFMT is one of the unsupported image format for a test
++# Check that a set of drivers has been whitelisted in the QEMU binary
 +#
-+_unsupported_fmt()
++_require_drivers()
 +{
-+    for f; do
++    available=$($QEMU -drive format=help | \
-+        if [ "$f" = "$IMGFMT" ]; then
++                sed -e '/Supported formats:/!d' -e 's/Supported formats://')
-+            _notrun "not suitable for this image format: $IMGFMT"
++    for driver
 +    do
 +        if ! echo "$available" | grep -q " $driver\( \|$\)"; then
 +            _notrun "$driver not available"
 +        fi
 +    done
 +}
 +
- # tests whether $IMGPROTO is one of the supported image protocols for a test
+ # make sure this script returns success
- #
+ true
  _supported_proto()
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 02/21] qemu-img: Improve commit invalid base message
+[Qemu-devel] [PULL 14/15] tests/check-block: Skip iotests when sanitizers are enabled
-When trying to invoke qemu-img commit with a base image file name that
+From: Thomas Huth <thuth@redhat.com>
 is not part of the top image's backing chain, the user receives a rather
 plain "Base not found" error message. This is not really helpful because
 it does not explain what "not found" means, potentially leaving the user
 wondering why qemu cannot find a file despite it clearly existing in the
 file system.
-Improve the error message by clarifying that "not found" means "not
+The sanitizers (especially the address sanitizer from Clang) are
-found in the top image's backing chain".
+sometimes printing out warnings or false positives - this spoils
 the output of the iotests, causing some of the tests to fail.
 Thus let's skip the automatic iotests during "make check" when the
 user configured QEMU with --enable-sanitizers.
-Reported-by: Ala Hino <ahino@redhat.com>
+Signed-off-by: Thomas Huth <thuth@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+Message-id: 20190823084203.29734-1-thuth@redhat.com
 Message-id: 20161201020508.24417-1-mreitz@redhat.com
 Reviewed-by: Eric Blake <eblake@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qemu-img.c | 4 +++-
+ tests/check-block.sh | 5 +++++
-file changed, 3 insertions(+), 1 deletion(-)
+file changed, 5 insertions(+)
-diff --git a/qemu-img.c b/qemu-img.c
+diff --git a/tests/check-block.sh b/tests/check-block.sh
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/qemu-img.c
+--- a/tests/check-block.sh
-+++ b/qemu-img.c
++++ b/tests/check-block.sh
-@@ -XXX,XX +XXX,XX @@ static int img_commit(int argc, char **argv)
+@@ -XXX,XX +XXX,XX @@ if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
-     if (base) {
+     exit 0
-         base_bs = bdrv_find_backing_image(bs, base);
+ fi
-         if (!base_bs) {
--            error_setg(&local_err, QERR_BASE_NOT_FOUND, base);
++if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
-+            error_setg(&local_err,
++    echo "Sanitizers are enabled ==> Not running the qemu-iotests."
-+                       "Did not find '%s' in the backing chain of '%s'",
++    exit 0
-+                       base, filename);
++fi
-             goto done;
++
-         }
+ if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
-     } else {
+     echo "No qemu-system binary available ==> Not running the qemu-iotests."
      exit 0
 --
-.11.0
+.21.0

-[Qemu-devel] [PULL 17/21] qemu-io: Add failure regression tests
+[Qemu-devel] [PULL 15/15] iotests: Unify cache mode quoting
 From: Nir Soffer <nirsof@gmail.com>
-Add regression tests checking that qemu-io fails with non-zero exit code
+Quoting cache mode is not needed, and most tests use unquoted values.
-when reading non-existing file or using the wrong image format.
+Unify all test to use the same style.
-Signed-off-by: Nir Soffer <nirsof@gmail.com>
+Message-id: 20190827173432.7656-1-nsoffer@redhat.com
-Message-id: 20170201003120.23378-4-nirsof@gmail.com
+Signed-off-by: Nir Soffer <nsoffer@redhat.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/174     | 59 ++++++++++++++++++++++++++++++++++++++++++++++
+ tests/qemu-iotests/026 | 4 ++--
- tests/qemu-iotests/174.out |  7 ++++++
+ tests/qemu-iotests/039 | 4 ++--
- tests/qemu-iotests/group   |  1 +
+ tests/qemu-iotests/052 | 2 +-
-files changed, 67 insertions(+)
+ tests/qemu-iotests/091 | 4 ++--
- create mode 100755 tests/qemu-iotests/174
+files changed, 7 insertions(+), 7 deletions(-)
  create mode 100644 tests/qemu-iotests/174.out
-diff --git a/tests/qemu-iotests/174 b/tests/qemu-iotests/174
+diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
-new file mode 100755
+index XXXXXXX..XXXXXXX 100755
-index XXXXXXX..XXXXXXX
+--- a/tests/qemu-iotests/026
---- /dev/null
++++ b/tests/qemu-iotests/026
-+++ b/tests/qemu-iotests/174
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-@@ -XXX,XX +XXX,XX @@
+ # Currently only qcow2 supports rebasing
-+#!/bin/bash
+ _supported_fmt qcow2
-+#
+ _supported_proto file
-+# Test that qemu-io fail with non-zero exit code
+-_default_cache_mode "writethrough"
-+#
+-_supported_cache_modes "writethrough" "none"
-+# Copyright (C) 2017 Nir Soffer <nirsof@gmail.com>
++_default_cache_mode writethrough
-+#
++_supported_cache_modes writethrough none
-+# This program is free software; you can redistribute it and/or modify
+ # The refcount table tests expect a certain minimum width for refcount entries
-+# it under the terms of the GNU General Public License as published by
+ # (so that the refcount table actually needs to grow); that minimum is 16 bits,
-+# the Free Software Foundation; either version 2 of the License, or
+ # being the default refcount entry width.
-+# (at your option) any later version.
+diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
-+#
+index XXXXXXX..XXXXXXX 100755
-+# This program is distributed in the hope that it will be useful,
+--- a/tests/qemu-iotests/039
-+# but WITHOUT ANY WARRANTY; without even the implied warranty of
++++ b/tests/qemu-iotests/039
-+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-+# GNU General Public License for more details.
+ _supported_fmt qcow2
-+#
+ _supported_proto file
-+# You should have received a copy of the GNU General Public License
+ _supported_os Linux
-+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+-_default_cache_mode "writethrough"
-+#
+-_supported_cache_modes "writethrough"
-+
++_default_cache_mode writethrough
-+# creator
++_supported_cache_modes writethrough
-+owner=nirsof@gmail.com
-+
+ size=128M
-+seq=`basename $0`
-+echo "QA output created by $seq"
+diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
-+
+index XXXXXXX..XXXXXXX 100755
-+here=`pwd`
+--- a/tests/qemu-iotests/052
-+status=1    # failure is the default!
++++ b/tests/qemu-iotests/052
-+
+@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
-+_cleanup()
+ _supported_proto file
-+{
-+    _cleanup_test_img
+ # Don't do O_DIRECT on tmpfs
-+}
+-_supported_cache_modes "writeback" "writethrough" "unsafe"
-+trap "_cleanup; exit \$status" 0 1 2 3 15
++_supported_cache_modes writeback writethrough unsafe
-+
-+# get standard environment, filters and checks
+ size=128M
-+. ./common.rc
+ _make_test_img $size
-+. ./common.filter
+diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
-+
+index XXXXXXX..XXXXXXX 100755
-+_unsupported_fmt raw
+--- a/tests/qemu-iotests/091
-+
++++ b/tests/qemu-iotests/091
-+
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-+size=256K
+ _supported_fmt qcow2
-+IMGFMT=raw IMGOPTS= _make_test_img $size | _filter_imgfmt
+ _supported_proto file
-+
+ _supported_os Linux
-+echo
+-_default_cache_mode "none"
-+echo "== reading wrong format should fail =="
+-_supported_cache_modes "writethrough" "none" "writeback"
-+$QEMU_IO -f $IMGFMT -c "read 0 $size" "$TEST_IMG" 2>/dev/null
++_default_cache_mode none
-+test $? -eq 1 || _fail "did not fail"
++_supported_cache_modes writethrough none writeback
-+
-+echo
+ size=1G
-+echo "== reading missing file should fail =="
 +$QEMU_IO -c "read 0 $size" "$TEST_DIR/missing" 2>/dev/null
 +test $? -eq 1 || _fail "did not fail"
 +
 +# success, all done
 +echo "*** done"
 +rm -f $seq.full
 +status=0
 diff --git a/tests/qemu-iotests/174.out b/tests/qemu-iotests/174.out
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/tests/qemu-iotests/174.out
@@ -XXX,XX +XXX,XX @@
 +QA output created by 174
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=262144
 +
 +== reading wrong format should fail ==
 +
 +== reading missing file should fail ==
 +*** done
 diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/group
 +++ b/tests/qemu-iotests/group
@@ -XXX,XX +XXX,XX @@
 rw auto quick
 auto
 rw auto
 +174 auto
 --
-.11.0
+.21.0

The following changes since commit 98b2faeaee96ab084d0b1669918688d8895c155f:

Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging (2017-02-10 18:07:02 +0000)

are available in the git repository at:

git://github.com/XanClic/qemu.git tags/pull-block-2017-02-12

for you to fetch changes up to 10d6eda1926804a09aa0710ca62933087813de0b:

qemu-img: Avoid setting ret to unused value in img_convert() (2017-02-12 00:56:32 +0100)

----------------------------------------------------------------
Block patches

----------------------------------------------------------------
Alberto Garcia (2):
      qcow2: Optimize the refcount-block overlap check
      qemu-io: don't allow I/O operations larger than BDRV_REQUEST_MAX_BYTES

Daniel P. Berrange (1):
      iotests: record separate timings per format,protocol pair

Dou Liyang (2):
      block/qapi: reduce the coupling between the bdrv_query_stats and bdrv_query_bds_stats
      block/qapi: reduce the execution time of qmp_query_blockstats

Fam Zheng (2):
      qapi: Tweak error message of bdrv_query_image_info
      iotests: Fix reference output for 059

Jeff Cody (3):
      block: check full backing filename when searching protocol filenames
      qemu-iotests: Don't create fifos / pidfiles with protocol paths
      qemu-iotest: test to lookup protocol-based image with relative backing

Max Reitz (1):
      qemu-img: Improve commit invalid base message

Nir Soffer (3):
      qemu-io: Return non-zero exit code on failure
      qemu-iotests: Add _unsupported_fmt helper
      qemu-io: Add failure regression tests

Peter Lieven (2):
      block/nfs: fix NULL pointer dereference in URI parsing
      block/nfs: fix naming of runtime opts

Peter Maydell (2):
      qemu-img: Use qemu_strtoul() rather than raw strtoul()
      qemu-img: Avoid setting ret to unused value in img_convert()

QingFeng Hao (2):
      iotests: Fix a problem in common.filter
      block/vmdk: Fix the endian problem of buf_len and lba

Vladimir Sementsov-Ogievskiy (1):
      block: bdrv_invalidate_cache: invalidate children first

From: QingFeng Hao <haoqf@linux.vnet.ibm.com>

If TEST_DIR is set to /tmp, test case 144 will fail. The reason is that
TEST_DIR resembles 144's test image name tmp.qcow2.
When 144 is testing $TEST_DIR/tmp.qcow2, it wants to replace
$TEST_DIR/tmp.qcow2 to TEST_DIR/tmp.qcow2, but actually it will fail
and get TEST_DIRTEST_DIR.qcow2 in this case.
The fix is just to modify the code to replace $TEST_DIR/ with TEST_DIR/.

Signed-off-by: QingFeng Hao <haoqf@linux.vnet.ibm.com>
Message-id: 20161216054723.96055-2-haoqf@linux.vnet.ibm.com
Reviewed-by: Eric Blake <eblake@redhat.com>
[mreitz: Fixed commit message and dropped superfluous escaping]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/common.filter | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_generated_node_ids()
 # replace occurrences of the actual TEST_DIR value with TEST_DIR
 _filter_testdir()
 {
-    sed -e "s#$TEST_DIR#TEST_DIR#g"
+    sed -e "s#$TEST_DIR/#TEST_DIR/#g"
 }
 
 # replace occurrences of the actual IMGFMT value with IMGFMT
-- 
2.11.0

When trying to invoke qemu-img commit with a base image file name that
is not part of the top image's backing chain, the user receives a rather
plain "Base not found" error message. This is not really helpful because
it does not explain what "not found" means, potentially leaving the user
wondering why qemu cannot find a file despite it clearly existing in the
file system.

Improve the error message by clarifying that "not found" means "not
found in the top image's backing chain".

Reported-by: Ala Hino <ahino@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20161201020508.24417-1-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-img.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int img_commit(int argc, char **argv)
     if (base) {
         base_bs = bdrv_find_backing_image(bs, base);
         if (!base_bs) {
-            error_setg(&local_err, QERR_BASE_NOT_FOUND, base);
+            error_setg(&local_err,
+                       "Did not find '%s' in the backing chain of '%s'",
+                       base, filename);
             goto done;
         }
     } else {
-- 
2.11.0

From: Dou Liyang <douly.fnst@cn.fujitsu.com>

The bdrv_query_stats and bdrv_query_bds_stats functions need to call
each other, that increases the coupling. it also makes the program
complicated and makes some unnecessary tests.

Remove the call from bdrv_query_bds_stats to bdrv_query_stats, just
take some recursion to make it clearly.

Avoid testing whether the blk is NULL during querying the bds stats.
It is unnecessary.

Signed-off-by: Dou Liyang <douly.fnst@cn.fujitsu.com>
Message-id: 1484467275-27919-2-git-send-email-douly.fnst@cn.fujitsu.com
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qapi.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/block/qapi.c b/block/qapi.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_info(BlockBackend *blk, BlockInfo **p_info,
     qapi_free_BlockInfo(info);
 }
 
-static BlockStats *bdrv_query_stats(BlockBackend *blk,
-                                    const BlockDriverState *bs,
-                                    bool query_backing);
-
 static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
 {
     BlockAcctStats *stats = blk_get_stats(blk);
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds, BlockBackend *blk)
     }
 }
 
-static void bdrv_query_bds_stats(BlockStats *s, const BlockDriverState *bs,
+static BlockStats *bdrv_query_bds_stats(const BlockDriverState *bs,
                                  bool query_backing)
 {
+    BlockStats *s = NULL;
+
+    s = g_malloc0(sizeof(*s));
+    s->stats = g_malloc0(sizeof(*s->stats));
+
+    if (!bs) {
+        return s;
+    }
+
     if (bdrv_get_node_name(bs)[0]) {
         s->has_node_name = true;
         s->node_name = g_strdup(bdrv_get_node_name(bs));
@@ -XXX,XX +XXX,XX @@ static void bdrv_query_bds_stats(BlockStats *s, const BlockDriverState *bs,
 
     if (bs->file) {
         s->has_parent = true;
-        s->parent = bdrv_query_stats(NULL, bs->file->bs, query_backing);
+        s->parent = bdrv_query_bds_stats(bs->file->bs, query_backing);
     }
 
     if (query_backing && bs->backing) {
         s->has_backing = true;
-        s->backing = bdrv_query_stats(NULL, bs->backing->bs, query_backing);
+        s->backing = bdrv_query_bds_stats(bs->backing->bs, query_backing);
     }
 
+    return s;
 }
 
 static BlockStats *bdrv_query_stats(BlockBackend *blk,
@@ -XXX,XX +XXX,XX @@ static BlockStats *bdrv_query_stats(BlockBackend *blk,
 {
     BlockStats *s;
 
-    s = g_malloc0(sizeof(*s));
-    s->stats = g_malloc0(sizeof(*s->stats));
+    s = bdrv_query_bds_stats(bs, query_backing);
 
     if (blk) {
         s->has_device = true;
         s->device = g_strdup(blk_name(blk));
         bdrv_query_blk_stats(s->stats, blk);
     }
-    if (bs) {
-        bdrv_query_bds_stats(s, bs, query_backing);
-    }
 
     return s;
 }
-- 
2.11.0

From: Dou Liyang <douly.fnst@cn.fujitsu.com>

In order to reduce the execution time, this patch optimize
the qmp_query_blockstats():
Remove the next_query_bds function.
Remove the bdrv_query_stats function.
Remove some judgement sentence.

The original qmp_query_blockstats calls next_query_bds to get
the next objects in each loops. In the next_query_bds, it checks
the query_nodes and blk. It also call bdrv_query_stats to get
the stats, In the bdrv_query_stats, it checks blk and bs each
times. This waste more times, which may stall the main loop a
bit. And if the disk is too many and donot use the dataplane
feature, this may affect the performance in main loop thread.

This patch removes that two functions, and makes the structure
clearly.

Signed-off-by: Dou Liyang <douly.fnst@cn.fujitsu.com>
Message-id: 1484467275-27919-3-git-send-email-douly.fnst@cn.fujitsu.com
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[mreitz: Removed duplicate info->value assignment]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qapi.c | 73 ++++++++++++++++++++++++------------------------------------
 1 file changed, 29 insertions(+), 44 deletions(-)

diff --git a/block/qapi.c b/block/qapi.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -XXX,XX +XXX,XX @@ static BlockStats *bdrv_query_bds_stats(const BlockDriverState *bs,
     return s;
 }
 
-static BlockStats *bdrv_query_stats(BlockBackend *blk,
-                                    const BlockDriverState *bs,
-                                    bool query_backing)
-{
-    BlockStats *s;
-
-    s = bdrv_query_bds_stats(bs, query_backing);
-
-    if (blk) {
-        s->has_device = true;
-        s->device = g_strdup(blk_name(blk));
-        bdrv_query_blk_stats(s->stats, blk);
-    }
-
-    return s;
-}
-
 BlockInfoList *qmp_query_block(Error **errp)
 {
     BlockInfoList *head = NULL, **p_next = &head;
@@ -XXX,XX +XXX,XX @@ BlockInfoList *qmp_query_block(Error **errp)
     return head;
 }
 
-static bool next_query_bds(BlockBackend **blk, BlockDriverState **bs,
-                           bool query_nodes)
-{
-    if (query_nodes) {
-        *bs = bdrv_next_node(*bs);
-        return !!*bs;
-    }
-
-    *blk = blk_next(*blk);
-    *bs = *blk ? blk_bs(*blk) : NULL;
-
-    return !!*blk;
-}
-
 BlockStatsList *qmp_query_blockstats(bool has_query_nodes,
                                      bool query_nodes,
                                      Error **errp)
 {
     BlockStatsList *head = NULL, **p_next = &head;
-    BlockBackend *blk = NULL;
-    BlockDriverState *bs = NULL;
+    BlockBackend *blk;
+    BlockDriverState *bs;
 
     /* Just to be safe if query_nodes is not always initialized */
-    query_nodes = has_query_nodes && query_nodes;
-
-    while (next_query_bds(&blk, &bs, query_nodes)) {
-        BlockStatsList *info = g_malloc0(sizeof(*info));
-        AioContext *ctx = blk ? blk_get_aio_context(blk)
-                              : bdrv_get_aio_context(bs);
+    if (has_query_nodes && query_nodes) {
+        for (bs = bdrv_next_node(NULL); bs; bs = bdrv_next_node(bs)) {
+            BlockStatsList *info = g_malloc0(sizeof(*info));
+            AioContext *ctx = bdrv_get_aio_context(bs);
 
-        aio_context_acquire(ctx);
-        info->value = bdrv_query_stats(blk, bs, !query_nodes);
-        aio_context_release(ctx);
+            aio_context_acquire(ctx);
+            info->value = bdrv_query_bds_stats(bs, false);
+            aio_context_release(ctx);
 
-        *p_next = info;
-        p_next = &info->next;
+            *p_next = info;
+            p_next = &info->next;
+        }
+    } else {
+        for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
+            BlockStatsList *info = g_malloc0(sizeof(*info));
+            AioContext *ctx = blk_get_aio_context(blk);
+            BlockStats *s;
+
+            aio_context_acquire(ctx);
+            s = bdrv_query_bds_stats(blk_bs(blk), true);
+            s->has_device = true;
+            s->device = g_strdup(blk_name(blk));
+            bdrv_query_blk_stats(s->stats, blk);
+            aio_context_release(ctx);
+
+            info->value = s;
+            *p_next = info;
+            p_next = &info->next;
+        }
     }
 
     return head;
-- 
2.11.0

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Current implementation invalidates firstly parent bds and then its
children. This leads to the following bug:

after incoming migration, in bdrv_invalidate_cache_all:
1. invalidate parent bds - reopen it with BDRV_O_INACTIVE cleared
2. child is not yet invalidated
3. parent check that its BDRV_O_INACTIVE is cleared
4. parent writes to child
5. assert in bdrv_co_pwritev, as BDRV_O_INACTIVE is set for child

This patch fixes it by just changing invalidate sequence: invalidate
children first.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20170131112308.54189-1-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
     if (!(bs->open_flags & BDRV_O_INACTIVE)) {
         return;
     }
-    bs->open_flags &= ~BDRV_O_INACTIVE;
 
-    if (bs->drv->bdrv_invalidate_cache) {
-        bs->drv->bdrv_invalidate_cache(bs, &local_err);
+    QLIST_FOREACH(child, &bs->children, next) {
+        bdrv_invalidate_cache(child->bs, &local_err);
         if (local_err) {
-            bs->open_flags |= BDRV_O_INACTIVE;
             error_propagate(errp, local_err);
             return;
         }
     }
 
-    QLIST_FOREACH(child, &bs->children, next) {
-        bdrv_invalidate_cache(child->bs, &local_err);
+    bs->open_flags &= ~BDRV_O_INACTIVE;
+    if (bs->drv->bdrv_invalidate_cache) {
+        bs->drv->bdrv_invalidate_cache(bs, &local_err);
         if (local_err) {
             bs->open_flags |= BDRV_O_INACTIVE;
             error_propagate(errp, local_err);
-- 
2.11.0

From: Peter Lieven <pl@kamp.de>

parse_uint_full wants to put the parsed value into the
variable passed via its second argument which is NULL.

Fixes: 94d6a7a76e9df9919629428f6c598e2b97d9426c
Cc: qemu-stable@nongnu.org
Signed-off-by: Peter Lieven <pl@kamp.de>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1485942829-10756-2-git-send-email-pl@kamp.de
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/nfs.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/block/nfs.c b/block/nfs.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -XXX,XX +XXX,XX @@ static int nfs_parse_uri(const char *filename, QDict *options, Error **errp)
     qdict_put(options, "path", qstring_from_str(uri->path));
 
     for (i = 0; i < qp->n; i++) {
+        unsigned long long val;
         if (!qp->p[i].value) {
             error_setg(errp, "Value for NFS parameter expected: %s",
                        qp->p[i].name);
             goto out;
         }
-        if (parse_uint_full(qp->p[i].value, NULL, 0)) {
+        if (parse_uint_full(qp->p[i].value, &val, 0)) {
             error_setg(errp, "Illegal value for NFS parameter: %s",
                        qp->p[i].name);
             goto out;
-- 
2.11.0

From: Peter Lieven <pl@kamp.de>

commit 94d6a7a accidentally left the naming of runtime opts and QAPI
scheme inconsistent. As one consequence passing of parameters in the
URI is broken. Sync the naming of the runtime opts to the QAPI
scheme.

Please note that this is technically backwards incompatible with the 2.8
release, but the 2.8 release is the only version that had the wrong naming.
Furthermore release 2.8 suffered from a NULL pointer dereference during
URI parsing.

Fixes: 94d6a7a76e9df9919629428f6c598e2b97d9426c
Cc: qemu-stable@nongnu.org
Signed-off-by: Peter Lieven <pl@kamp.de>
Message-id: 1485942829-10756-3-git-send-email-pl@kamp.de
[mreitz: Fixed commit message]
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/nfs.c | 46 +++++++++++++++++++++++-----------------------
 1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/block/nfs.c b/block/nfs.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -XXX,XX +XXX,XX @@ static QemuOptsList runtime_opts = {
             .help = "Path of the image on the host",
         },
         {
-            .name = "uid",
+            .name = "user",
             .type = QEMU_OPT_NUMBER,
             .help = "UID value to use when talking to the server",
         },
         {
-            .name = "gid",
+            .name = "group",
             .type = QEMU_OPT_NUMBER,
             .help = "GID value to use when talking to the server",
         },
         {
-            .name = "tcp-syncnt",
+            .name = "tcp-syn-count",
             .type = QEMU_OPT_NUMBER,
             .help = "Number of SYNs to send during the session establish",
         },
         {
-            .name = "readahead",
+            .name = "readahead-size",
             .type = QEMU_OPT_NUMBER,
             .help = "Set the readahead size in bytes",
         },
         {
-            .name = "pagecache",
+            .name = "page-cache-size",
             .type = QEMU_OPT_NUMBER,
             .help = "Set the pagecache size in bytes",
         },
@@ -XXX,XX +XXX,XX @@ static int64_t nfs_client_open(NFSClient *client, QDict *options,
         goto fail;
     }
 
-    if (qemu_opt_get(opts, "uid")) {
-        client->uid = qemu_opt_get_number(opts, "uid", 0);
+    if (qemu_opt_get(opts, "user")) {
+        client->uid = qemu_opt_get_number(opts, "user", 0);
         nfs_set_uid(client->context, client->uid);
     }
 
-    if (qemu_opt_get(opts, "gid")) {
-        client->gid = qemu_opt_get_number(opts, "gid", 0);
+    if (qemu_opt_get(opts, "group")) {
+        client->gid = qemu_opt_get_number(opts, "group", 0);
         nfs_set_gid(client->context, client->gid);
     }
 
-    if (qemu_opt_get(opts, "tcp-syncnt")) {
-        client->tcp_syncnt = qemu_opt_get_number(opts, "tcp-syncnt", 0);
+    if (qemu_opt_get(opts, "tcp-syn-count")) {
+        client->tcp_syncnt = qemu_opt_get_number(opts, "tcp-syn-count", 0);
         nfs_set_tcp_syncnt(client->context, client->tcp_syncnt);
     }
 
 #ifdef LIBNFS_FEATURE_READAHEAD
-    if (qemu_opt_get(opts, "readahead")) {
+    if (qemu_opt_get(opts, "readahead-size")) {
         if (open_flags & BDRV_O_NOCACHE) {
             error_setg(errp, "Cannot enable NFS readahead "
                              "if cache.direct = on");
             goto fail;
         }
-        client->readahead = qemu_opt_get_number(opts, "readahead", 0);
+        client->readahead = qemu_opt_get_number(opts, "readahead-size", 0);
         if (client->readahead > QEMU_NFS_MAX_READAHEAD_SIZE) {
             error_report("NFS Warning: Truncating NFS readahead "
                          "size to %d", QEMU_NFS_MAX_READAHEAD_SIZE);
@@ -XXX,XX +XXX,XX @@ static int64_t nfs_client_open(NFSClient *client, QDict *options,
 #endif
 
 #ifdef LIBNFS_FEATURE_PAGECACHE
-    if (qemu_opt_get(opts, "pagecache")) {
+    if (qemu_opt_get(opts, "page-cache-size")) {
         if (open_flags & BDRV_O_NOCACHE) {
             error_setg(errp, "Cannot enable NFS pagecache "
                              "if cache.direct = on");
             goto fail;
         }
-        client->pagecache = qemu_opt_get_number(opts, "pagecache", 0);
+        client->pagecache = qemu_opt_get_number(opts, "page-cache-size", 0);
         if (client->pagecache > QEMU_NFS_MAX_PAGECACHE_SIZE) {
             error_report("NFS Warning: Truncating NFS pagecache "
                          "size to %d pages", QEMU_NFS_MAX_PAGECACHE_SIZE);
@@ -XXX,XX +XXX,XX @@ static void nfs_refresh_filename(BlockDriverState *bs, QDict *options)
     qdict_put(opts, "path", qstring_from_str(client->path));
 
     if (client->uid) {
-        qdict_put(opts, "uid", qint_from_int(client->uid));
+        qdict_put(opts, "user", qint_from_int(client->uid));
     }
     if (client->gid) {
-        qdict_put(opts, "gid", qint_from_int(client->gid));
+        qdict_put(opts, "group", qint_from_int(client->gid));
     }
     if (client->tcp_syncnt) {
-        qdict_put(opts, "tcp-syncnt",
-                      qint_from_int(client->tcp_syncnt));
+        qdict_put(opts, "tcp-syn-cnt",
+                  qint_from_int(client->tcp_syncnt));
     }
     if (client->readahead) {
-        qdict_put(opts, "readahead",
-                      qint_from_int(client->readahead));
+        qdict_put(opts, "readahead-size",
+                  qint_from_int(client->readahead));
     }
     if (client->pagecache) {
-        qdict_put(opts, "pagecache",
-                      qint_from_int(client->pagecache));
+        qdict_put(opts, "page-cache-size",
+                  qint_from_int(client->pagecache));
     }
     if (client->debug) {
         qdict_put(opts, "debug", qint_from_int(client->debug));
-- 
2.11.0

From: Nir Soffer <nirsof@gmail.com>

The result of openfile was not checked, leading to failure deep in the
actual command with confusing error message, and exiting with exit code 0.

Here is a simple example - trying to read with the wrong format:

$ touch file
    $ qemu-io -f qcow2 -c 'read -P 1 0 1024' file; echo $?
    can't open device file: Image is not in qcow2 format
    no file open, try 'help open'
    0

With this patch, we fail earlier with exit code 1:

$ ./qemu-io -f qcow2 -c 'read -P 1 0 1024' file; echo $?
    can't open device file: Image is not in qcow2 format
    1

Failing earlier, we don't log this error now:

no file open, try 'help open'

But some tests expected it; the line was removed from the test output.

Signed-off-by: Nir Soffer <nirsof@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20170201003120.23378-2-nirsof@gmail.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-io.c                  |  8 ++++++--
 tests/qemu-iotests/059.out |  3 ---
 tests/qemu-iotests/070.out |  1 -
 tests/qemu-iotests/075.out |  7 -------
 tests/qemu-iotests/076.out |  3 ---
 tests/qemu-iotests/078.out |  6 ------
 tests/qemu-iotests/080.out | 18 ------------------
 tests/qemu-iotests/083.out | 17 -----------------
 tests/qemu-iotests/088.out |  6 ------
 tests/qemu-iotests/092.out | 12 ------------
 tests/qemu-iotests/116.out |  7 -------
 tests/qemu-iotests/131.out |  1 -
 tests/qemu-iotests/140.out |  1 -
 13 files changed, 6 insertions(+), 84 deletions(-)

diff --git a/qemu-io.c b/qemu-io.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv)
                 exit(1);
             }
             opts = qemu_opts_to_qdict(qopts, NULL);
-            openfile(NULL, flags, writethrough, opts);
+            if (openfile(NULL, flags, writethrough, opts)) {
+                exit(1);
+            }
         } else {
             if (format) {
                 opts = qdict_new();
                 qdict_put(opts, "driver", qstring_from_str(format));
             }
-            openfile(argv[optind], flags, writethrough, opts);
+            if (openfile(argv[optind], flags, writethrough, opts)) {
+                exit(1);
+            }
         }
     }
     command_loop();
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ QA output created by 059
 === Testing invalid granularity ===
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.vmdk: Invalid granularity, image may be corrupt
-no file open, try 'help open'
 
 === Testing too big L2 table size ===
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.vmdk: L2 table size too big
-no file open, try 'help open'
 
 === Testing too big L1 table size ===
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.vmdk: L1 size too big
-no file open, try 'help open'
 
 === Testing monolithicFlat creation and opening ===
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
diff --git a/tests/qemu-iotests/070.out b/tests/qemu-iotests/070.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/070.out
+++ b/tests/qemu-iotests/070.out
@@ -XXX,XX +XXX,XX @@ QA output created by 070
 can't open device TEST_DIR/iotest-dirtylog-10G-4M.vhdx: VHDX image file 'TEST_DIR/iotest-dirtylog-10G-4M.vhdx' opened read-only, but contains a log that needs to be replayed
 To replay the log, run:
 qemu-img check -r all 'TEST_DIR/iotest-dirtylog-10G-4M.vhdx'
- no file open, try 'help open'
 === Verify open image replays log  ===
 read 18874368/18874368 bytes at offset 0
 18 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
diff --git a/tests/qemu-iotests/075.out b/tests/qemu-iotests/075.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/075.out
+++ b/tests/qemu-iotests/075.out
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 1048064
 
 == block_size must be a multiple of 512 ==
 can't open device TEST_DIR/simple-pattern.cloop: block_size 513 must be a multiple of 512
-no file open, try 'help open'
 
 == block_size cannot be zero ==
 can't open device TEST_DIR/simple-pattern.cloop: block_size cannot be zero
-no file open, try 'help open'
 
 == huge block_size ===
 can't open device TEST_DIR/simple-pattern.cloop: block_size 4294966784 must be 64 MB or less
-no file open, try 'help open'
 
 == offsets_size overflow ===
 can't open device TEST_DIR/simple-pattern.cloop: n_blocks 4294967295 must be 536870911 or less
-no file open, try 'help open'
 
 == refuse images that require too many offsets ===
 can't open device TEST_DIR/simple-pattern.cloop: image requires too many offsets, try increasing block size
-no file open, try 'help open'
 
 == refuse images with non-monotonically increasing offsets ==
 can't open device TEST_DIR/simple-pattern.cloop: offsets not monotonically increasing at index 1, image file is corrupt
-no file open, try 'help open'
 
 == refuse images with invalid compressed block size ==
 can't open device TEST_DIR/simple-pattern.cloop: invalid compressed block size at index 1, image file is corrupt
-no file open, try 'help open'
 *** done
diff --git a/tests/qemu-iotests/076.out b/tests/qemu-iotests/076.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/076.out
+++ b/tests/qemu-iotests/076.out
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 0
 
 == Negative catalog size ==
 can't open device TEST_DIR/parallels-v1: Catalog too large
-no file open, try 'help open'
 
 == Overflow in catalog allocation ==
 can't open device TEST_DIR/parallels-v1: Catalog too large
-no file open, try 'help open'
 
 == Zero sectors per track ==
 can't open device TEST_DIR/parallels-v1: Invalid image: Zero sectors per track
-no file open, try 'help open'
 
 == Read from a valid v2 image ==
 read 65536/65536 bytes at offset 0
diff --git a/tests/qemu-iotests/078.out b/tests/qemu-iotests/078.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/078.out
+++ b/tests/qemu-iotests/078.out
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 0
 
 == Negative catalog size ==
 can't open device TEST_DIR/empty.bochs: Catalog size is too large
-no file open, try 'help open'
 
 == Overflow for catalog size * sizeof(uint32_t) ==
 can't open device TEST_DIR/empty.bochs: Catalog size is too large
-no file open, try 'help open'
 
 == Too small catalog bitmap for image size ==
 can't open device TEST_DIR/empty.bochs: Catalog size is too small for this disk size
-no file open, try 'help open'
 can't open device TEST_DIR/empty.bochs: Catalog size is too small for this disk size
-no file open, try 'help open'
 
 == Negative extent size ==
 can't open device TEST_DIR/empty.bochs: Extent size 2147483648 is too large
-no file open, try 'help open'
 
 == Zero extent size ==
 can't open device TEST_DIR/empty.bochs: Extent size must be at least 512
-no file open, try 'help open'
 *** done
diff --git a/tests/qemu-iotests/080.out b/tests/qemu-iotests/080.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/080.out
+++ b/tests/qemu-iotests/080.out
@@ -XXX,XX +XXX,XX @@ QA output created by 080
 == Huge header size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: qcow2 header exceeds cluster size
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: qcow2 header exceeds cluster size
-no file open, try 'help open'
 
 == Huge unknown header extension ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: Invalid backing file offset
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Header extension too large
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Header extension too large
-no file open, try 'help open'
 
 == Huge refcount table size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: Reference count table too large
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Reference count table too large
-no file open, try 'help open'
 
 == Misaligned refcount table ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: Invalid reference count table offset
-no file open, try 'help open'
 
 == Huge refcount offset ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: Invalid reference count table offset
-no file open, try 'help open'
 
 == Invalid snapshot table ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: Too many snapshots
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Too many snapshots
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Invalid snapshot table offset
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Invalid snapshot table offset
-no file open, try 'help open'
 
 == Hitting snapshot table size limit ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 0
 == Invalid L1 table ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: Active L1 table too large
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Active L1 table too large
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Invalid L1 table offset
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow2: Invalid L1 table offset
-no file open, try 'help open'
 
 == Invalid L1 table (with internal snapshot in the image) ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
@@ -XXX,XX +XXX,XX @@ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': L1 table is too small
 == Invalid backing file size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow2: Backing file name too long
-no file open, try 'help open'
 
 == Invalid L2 entry (huge physical offset) ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
diff --git a/tests/qemu-iotests/083.out b/tests/qemu-iotests/083.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/083.out
+++ b/tests/qemu-iotests/083.out
@@ -XXX,XX +XXX,XX @@ QA output created by 083
 === Check disconnect before neg1 ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect after neg1 ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect 8 neg1 ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect 16 neg1 ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect before export ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect after export ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect 4 export ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect 12 export ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect 16 export ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect before neg2 ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect after neg2 ===
 
@@ -XXX,XX +XXX,XX @@ read failed: Input/output error
 === Check disconnect 8 neg2 ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect 10 neg2 ===
 
 can't open device nbd:127.0.0.1:PORT:exportname=foo
-no file open, try 'help open'
 
 === Check disconnect before request ===
 
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 0
 === Check disconnect before neg-classic ===
 
 can't open device nbd:127.0.0.1:PORT
-no file open, try 'help open'
 
 === Check disconnect 8 neg-classic ===
 
 can't open device nbd:127.0.0.1:PORT
-no file open, try 'help open'
 
 === Check disconnect 16 neg-classic ===
 
 can't open device nbd:127.0.0.1:PORT
-no file open, try 'help open'
 
 === Check disconnect 24 neg-classic ===
 
 can't open device nbd:127.0.0.1:PORT
-no file open, try 'help open'
 
 === Check disconnect 28 neg-classic ===
 
 can't open device nbd:127.0.0.1:PORT
-no file open, try 'help open'
 
 === Check disconnect after neg-classic ===
 
diff --git a/tests/qemu-iotests/088.out b/tests/qemu-iotests/088.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/088.out
+++ b/tests/qemu-iotests/088.out
@@ -XXX,XX +XXX,XX @@ QA output created by 088
 == Invalid block size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.vpc: Invalid block size 0
-no file open, try 'help open'
 can't open device TEST_DIR/t.vpc: Invalid block size 0
-no file open, try 'help open'
 can't open device TEST_DIR/t.vpc: Invalid block size 128
-no file open, try 'help open'
 can't open device TEST_DIR/t.vpc: Invalid block size 128
-no file open, try 'help open'
 can't open device TEST_DIR/t.vpc: Invalid block size 305419896
-no file open, try 'help open'
 can't open device TEST_DIR/t.vpc: Invalid block size 305419896
-no file open, try 'help open'
 *** done
diff --git a/tests/qemu-iotests/092.out b/tests/qemu-iotests/092.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/092.out
+++ b/tests/qemu-iotests/092.out
@@ -XXX,XX +XXX,XX @@ QA output created by 092
 == Invalid cluster size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: Cluster size must be between 512 and 64k
-no file open, try 'help open'
 
 == Invalid L2 table size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: L2 table size must be between 512 and 64k
-no file open, try 'help open'
 
 == Invalid size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow: Image too large
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: Image too large
-no file open, try 'help open'
 
 == Invalid backing file length ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 can't open device TEST_DIR/t.qcow: Backing file name too long
-no file open, try 'help open'
 can't open device TEST_DIR/t.qcow: Backing file name too long
-no file open, try 'help open'
 *** done
diff --git a/tests/qemu-iotests/116.out b/tests/qemu-iotests/116.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/116.out
+++ b/tests/qemu-iotests/116.out
@@ -XXX,XX +XXX,XX @@ QA output created by 116
 == truncated header cluster ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
-no file open, try 'help open'
 
 == invalid header magic ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 can't open device TEST_DIR/t.qed: Image not in QED format
-no file open, try 'help open'
 
 == invalid cluster size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
-no file open, try 'help open'
 
 == invalid table size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
-no file open, try 'help open'
 
 == invalid header size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
-no file open, try 'help open'
 
 == invalid L1 table offset ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
-no file open, try 'help open'
 
 == invalid image size ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
 can't open device TEST_DIR/t.qed: Could not open 'TEST_DIR/t.qed': Invalid argument
-no file open, try 'help open'
 *** done
diff --git a/tests/qemu-iotests/131.out b/tests/qemu-iotests/131.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/131.out
+++ b/tests/qemu-iotests/131.out
@@ -XXX,XX +XXX,XX @@ read 32768/32768 bytes at offset 0
 32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 == Corrupt image ==
 can't open device TEST_DIR/t.parallels: parallels: Image was not closed correctly; cannot be opened read/write
-no file open, try 'help open'
 ERROR image was not closed correctly
 
 1 errors were found on the image.
diff --git a/tests/qemu-iotests/140.out b/tests/qemu-iotests/140.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/140.out
+++ b/tests/qemu-iotests/140.out
@@ -XXX,XX +XXX,XX @@ read 65536/65536 bytes at offset 0
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 {"return": {}}
 can't open device nbd+unix:///drv?socket=TEST_DIR/nbd: No export with name 'drv' available
-no file open, try 'help open'
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN"}
 *** done
-- 
2.11.0

From: Nir Soffer <nirsof@gmail.com>

This helper allows adding tests supporting any format expect the
specified formats. This may be useful to test that many formats behave
in a common way.

Signed-off-by: Nir Soffer <nirsof@gmail.com>
Message-id: 20170201003120.23378-3-nirsof@gmail.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/common.rc | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -XXX,XX +XXX,XX @@ _supported_fmt()
     _notrun "not suitable for this image format: $IMGFMT"
 }
 
+# tests whether $IMGFMT is one of the unsupported image format for a test
+#
+_unsupported_fmt()
+{
+    for f; do
+        if [ "$f" = "$IMGFMT" ]; then
+            _notrun "not suitable for this image format: $IMGFMT"
+        fi
+    done
+}
+
 # tests whether $IMGPROTO is one of the supported image protocols for a test
 #
 _supported_proto()
-- 
2.11.0

From: Nir Soffer <nirsof@gmail.com>

Add regression tests checking that qemu-io fails with non-zero exit code
when reading non-existing file or using the wrong image format.

Signed-off-by: Nir Soffer <nirsof@gmail.com>
Message-id: 20170201003120.23378-4-nirsof@gmail.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/174     | 59 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/174.out |  7 ++++++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 67 insertions(+)
 create mode 100755 tests/qemu-iotests/174
 create mode 100644 tests/qemu-iotests/174.out

diff --git a/tests/qemu-iotests/174 b/tests/qemu-iotests/174
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/174
@@ -XXX,XX +XXX,XX @@
+#!/bin/bash
+#
+# Test that qemu-io fail with non-zero exit code
+#
+# Copyright (C) 2017 Nir Soffer <nirsof@gmail.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=nirsof@gmail.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_unsupported_fmt raw
+
+
+size=256K
+IMGFMT=raw IMGOPTS= _make_test_img $size | _filter_imgfmt
+
+echo
+echo "== reading wrong format should fail =="
+$QEMU_IO -f $IMGFMT -c "read 0 $size" "$TEST_IMG" 2>/dev/null
+test $? -eq 1 || _fail "did not fail"
+
+echo
+echo "== reading missing file should fail =="
+$QEMU_IO -c "read 0 $size" "$TEST_DIR/missing" 2>/dev/null
+test $? -eq 1 || _fail "did not fail"
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/174.out b/tests/qemu-iotests/174.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/174.out
@@ -XXX,XX +XXX,XX @@
+QA output created by 174
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=262144
+
+== reading wrong format should fail ==
+
+== reading missing file should fail ==
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -XXX,XX +XXX,XX @@
 171 rw auto quick
 172 auto
 173 rw auto
+174 auto
-- 
2.11.0

From: Alberto Garcia <berto@igalia.com>

The metadata overlap checks introduced in a40f1c2add help detect
corruption in the qcow2 image by verifying that data writes don't
overlap with existing metadata sections.

The 'refcount-block' check in particular iterates over the refcount
table in order to get the addresses of all refcount blocks and check
that none of them overlap with the region where we want to write.

The problem with the refcount table is that since it always occupies
complete clusters its size is usually very big. With the default
values of cluster_size=64KB and refcount_bits=16 this table holds 8192
entries, each one of them enough to map 2GB worth of host clusters.

So unless we're using images with several TB of allocated data this
table is going to be mostly empty, and iterating over it is a waste of
CPU. If the storage backend is fast enough this can have an effect on
I/O performance.

This patch keeps the index of the last used (i.e. non-zero) entry in
the refcount table and updates it every time the table changes. The
refcount-block overlap check then uses that index instead of reading
the whole table.

In my tests with a 4GB qcow2 file stored in RAM this doubles the
amount of write IOPS.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 20170201123828.4815-1-berto@igalia.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qcow2.h          |  1 +
 block/qcow2-refcount.c | 24 +++++++++++++++++++++++-
 block/qcow2.c          |  1 +
 3 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/block/qcow2.h b/block/qcow2.h
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVQcow2State {
     uint64_t *refcount_table;
     uint64_t refcount_table_offset;
     uint32_t refcount_table_size;
+    uint32_t max_refcount_table_index; /* Last used entry in refcount_table */
     uint64_t free_cluster_index;
     uint64_t free_byte_offset;
 
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -XXX,XX +XXX,XX @@ static Qcow2SetRefcountFunc *const set_refcount_funcs[] = {
 /*********************************************************/
 /* refcount handling */
 
+static void update_max_refcount_table_index(BDRVQcow2State *s)
+{
+    unsigned i = s->refcount_table_size - 1;
+    while (i > 0 && (s->refcount_table[i] & REFT_OFFSET_MASK) == 0) {
+        i--;
+    }
+    /* Set s->max_refcount_table_index to the index of the last used entry */
+    s->max_refcount_table_index = i;
+}
+
 int qcow2_refcount_init(BlockDriverState *bs)
 {
     BDRVQcow2State *s = bs->opaque;
@@ -XXX,XX +XXX,XX @@ int qcow2_refcount_init(BlockDriverState *bs)
         }
         for(i = 0; i < s->refcount_table_size; i++)
             be64_to_cpus(&s->refcount_table[i]);
+        update_max_refcount_table_index(s);
     }
     return 0;
  fail:
@@ -XXX,XX +XXX,XX @@ static int alloc_refcount_block(BlockDriverState *bs,
         }
 
         s->refcount_table[refcount_table_index] = new_block;
+        /* If there's a hole in s->refcount_table then it can happen
+         * that refcount_table_index < s->max_refcount_table_index */
+        s->max_refcount_table_index =
+            MAX(s->max_refcount_table_index, refcount_table_index);
 
         /* The new refcount block may be where the caller intended to put its
          * data, so let it restart the search. */
@@ -XXX,XX +XXX,XX @@ static int alloc_refcount_block(BlockDriverState *bs,
     s->refcount_table = new_table;
     s->refcount_table_size = table_size;
     s->refcount_table_offset = table_offset;
+    update_max_refcount_table_index(s);
 
     /* Free old table. */
     qcow2_free_clusters(bs, old_table_offset, old_table_size * sizeof(uint64_t),
@@ -XXX,XX +XXX,XX @@ write_refblocks:
     s->refcount_table = on_disk_reftable;
     s->refcount_table_offset = reftable_offset;
     s->refcount_table_size = reftable_size;
+    update_max_refcount_table_index(s);
 
     return 0;
 
@@ -XXX,XX +XXX,XX @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
     }
 
     if ((chk & QCOW2_OL_REFCOUNT_BLOCK) && s->refcount_table) {
-        for (i = 0; i < s->refcount_table_size; i++) {
+        unsigned last_entry = s->max_refcount_table_index;
+        assert(last_entry < s->refcount_table_size);
+        assert(last_entry + 1 == s->refcount_table_size ||
+               (s->refcount_table[last_entry + 1] & REFT_OFFSET_MASK) == 0);
+        for (i = 0; i <= last_entry; i++) {
             if ((s->refcount_table[i] & REFT_OFFSET_MASK) &&
                 overlaps_with(s->refcount_table[i] & REFT_OFFSET_MASK,
                 s->cluster_size)) {
@@ -XXX,XX +XXX,XX @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
     /* Now update the rest of the in-memory information */
     old_reftable = s->refcount_table;
     s->refcount_table = new_reftable;
+    update_max_refcount_table_index(s);
 
     s->refcount_bits = 1 << refcount_order;
     s->refcount_max = UINT64_C(1) << (s->refcount_bits - 1);
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static int make_completely_empty(BlockDriverState *bs)
 
     s->refcount_table_offset = s->cluster_size;
     s->refcount_table_size   = s->cluster_size / sizeof(uint64_t);
+    s->max_refcount_table_index = 0;
 
     g_free(s->refcount_table);
     s->refcount_table = new_reftable;
-- 
2.11.0

From: Alberto Garcia <berto@igalia.com>

Passing a request size larger than BDRV_REQUEST_MAX_BYTES to any of the
I/O commands results in an error. While 'read' and 'write' handle the
error correctly, 'aio_read' and 'aio_write' hit an assertion:

blk_aio_read_entry: Assertion `rwco->qiov->size == acb->bytes' failed.

The reason is that the QEMU I/O code cannot handle request sizes
larger than BDRV_REQUEST_MAX_BYTES, so this patch makes qemu-io check
that all values are within range.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 79f66648c685929a144396bda24d13a207131dcf.1485878688.git.berto@igalia.com
[mreitz: Use BDRV_REQUEST_MAX_BYTES instead of INT_MAX]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-io-cmds.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -XXX,XX +XXX,XX @@ create_iovec(BlockBackend *blk, QEMUIOVector *qiov, char **argv, int nr_iov,
             goto fail;
         }
 
-        if (len > SIZE_MAX) {
-            printf("Argument '%s' exceeds maximum size %llu\n", arg,
-                   (unsigned long long)SIZE_MAX);
+        if (len > BDRV_REQUEST_MAX_BYTES) {
+            printf("Argument '%s' exceeds maximum size %" PRIu64 "\n", arg,
+                   (uint64_t)BDRV_REQUEST_MAX_BYTES);
+            goto fail;
+        }
+
+        if (count > BDRV_REQUEST_MAX_BYTES - len) {
+            printf("The total number of bytes exceed the maximum size %" PRIu64
+                   "\n", (uint64_t)BDRV_REQUEST_MAX_BYTES);
             goto fail;
         }
 
@@ -XXX,XX +XXX,XX @@ static int read_f(BlockBackend *blk, int argc, char **argv)
     if (count < 0) {
         print_cvtnum_err(count, argv[optind]);
         return 0;
-    } else if (count > SIZE_MAX) {
+    } else if (count > BDRV_REQUEST_MAX_BYTES) {
         printf("length cannot exceed %" PRIu64 ", given %s\n",
-               (uint64_t) SIZE_MAX, argv[optind]);
+               (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]);
         return 0;
     }
 
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
     if (count < 0) {
         print_cvtnum_err(count, argv[optind]);
         return 0;
-    } else if (count > SIZE_MAX) {
+    } else if (count > BDRV_REQUEST_MAX_BYTES) {
         printf("length cannot exceed %" PRIu64 ", given %s\n",
-               (uint64_t) SIZE_MAX, argv[optind]);
+               (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]);
         return 0;
     }
 
-- 
2.11.0

From: Peter Maydell <peter.maydell@linaro.org>

Some of the argument parsing in qemu-img uses strtoul() to parse
integer arguments.  This is tricky to get correct and in fact the
code does not get it right, because it assigns the result of
strtoul() to an 'int' variable and then tries to check for > INT_MAX.
Coverity correctly complains that the comparison is always false.

Rewrite to use qemu_strtoul(), which has a saner convention for
reporting conversion failures.

(Fixes CID 1356421, CID 1356422, CID 1356423.)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 1486744104-15590-2-git-send-email-peter.maydell@linaro.org
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-img.c | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int img_bench(int argc, char **argv)
             break;
         case 'c':
         {
-            char *end;
-            errno = 0;
-            count = strtoul(optarg, &end, 0);
-            if (errno || *end || count > INT_MAX) {
+            unsigned long res;
+
+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > INT_MAX) {
                 error_report("Invalid request count specified");
                 return 1;
             }
+            count = res;
             break;
         }
         case 'd':
         {
-            char *end;
-            errno = 0;
-            depth = strtoul(optarg, &end, 0);
-            if (errno || *end || depth > INT_MAX) {
+            unsigned long res;
+
+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > INT_MAX) {
                 error_report("Invalid queue depth specified");
                 return 1;
             }
+            depth = res;
             break;
         }
         case 'f':
@@ -XXX,XX +XXX,XX @@ static int img_bench(int argc, char **argv)
             break;
         case OPTION_PATTERN:
         {
-            char *end;
-            errno = 0;
-            pattern = strtoul(optarg, &end, 0);
-            if (errno || *end || pattern > 0xff) {
+            unsigned long res;
+
+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > 0xff) {
                 error_report("Invalid pattern byte specified");
                 return 1;
             }
+            pattern = res;
             break;
         }
         case OPTION_FLUSH_INTERVAL:
         {
-            char *end;
-            errno = 0;
-            flush_interval = strtoul(optarg, &end, 0);
-            if (errno || *end || flush_interval > INT_MAX) {
+            unsigned long res;
+
+            if (qemu_strtoul(optarg, NULL, 0, &res) < 0 || res > INT_MAX) {
                 error_report("Invalid flush interval specified");
                 return 1;
             }
+            flush_interval = res;
             break;
         }
         case OPTION_NO_DRAIN:
-- 
2.11.0

From: Peter Maydell <peter.maydell@linaro.org>

Coverity points out that we assign the return value from
bdrv_snapshot_load_tmp() to 'ret' in img_convert(), but then
never use that variable. (We check for failure by looking
at local_err instead.) Drop the unused assignment, bringing
the call into line with the following call to
bdrv_snapshot_laod_tmp_by_id_or_name().

(Fixes CID 1247240.)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 1486744104-15590-3-git-send-email-peter.maydell@linaro.org
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-img.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv)
     }
 
     if (sn_opts) {
-        ret = bdrv_snapshot_load_tmp(bs[0],
-                                     qemu_opt_get(sn_opts, SNAPSHOT_OPT_ID),
-                                     qemu_opt_get(sn_opts, SNAPSHOT_OPT_NAME),
-                                     &local_err);
+        bdrv_snapshot_load_tmp(bs[0],
+                               qemu_opt_get(sn_opts, SNAPSHOT_OPT_ID),
+                               qemu_opt_get(sn_opts, SNAPSHOT_OPT_NAME),
+                               &local_err);
     } else if (snapshot_name != NULL) {
         if (bs_n > 1) {
             error_report("No support for concatenating multiple snapshot");
-- 
2.11.0

The following changes since commit 23919ddfd56135cad3cb468a8f54d5a595f024f4:

Merge remote-tracking branch 'remotes/aperard/tags/pull-xen-20190827' into staging (2019-08-27 15:52:36 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2019-08-27

for you to fetch changes up to bb043c056cffcc2f3ce88bfdaf2e76e455c09e2c:

iotests: Unify cache mode quoting (2019-08-27 19:48:44 +0200)

----------------------------------------------------------------
Block patches:
- qemu-io now accepts a file to read a write pattern from
- Ensure that raw files have their first block allocated so we can probe
  the O_DIRECT alignment if necessary
- Various fixes

----------------------------------------------------------------
Denis Plotnikov (1):
  qemu-io: add pattern file for write command

Max Reitz (7):
  iotests: Fix _filter_img_create()
  vmdk: Use bdrv_dirname() for relative extent paths
  iotests: Keep testing broken relative extent paths
  vmdk: Reject invalid compressed writes
  iotests: Disable broken streamOptimized tests
  iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
  iotests: Disable 126 for flat vmdk subformats

Nir Soffer (3):
  block: posix: Always allocate the first block
  iotests: Test allocate_first_block() with O_DIRECT
  iotests: Unify cache mode quoting

Stefan Hajnoczi (1):
  file-posix: fix request_alignment typo

Thomas Huth (2):
  iotests: Check for enabled drivers before testing them
  tests/check-block: Skip iotests when sanitizers are enabled

Vladimir Sementsov-Ogievskiy (1):
  block: fix permission update in bdrv_replace_node

block.c                                       |  5 +-
 block/file-posix.c                            | 53 +++++++++-
 block/vmdk.c                                  | 64 ++++++++----
 qemu-io-cmds.c                                | 99 +++++++++++++++++--
 tests/check-block.sh                          |  5 +
 tests/qemu-iotests/002                        |  1 +
 tests/qemu-iotests/003                        |  1 +
 tests/qemu-iotests/005                        |  3 +-
 tests/qemu-iotests/009                        |  1 +
 tests/qemu-iotests/010                        |  1 +
 tests/qemu-iotests/011                        |  1 +
 tests/qemu-iotests/017                        |  3 +-
 tests/qemu-iotests/018                        |  3 +-
 tests/qemu-iotests/019                        |  3 +-
 tests/qemu-iotests/020                        |  3 +-
 tests/qemu-iotests/026                        |  4 +-
 tests/qemu-iotests/027                        |  1 +
 tests/qemu-iotests/032                        |  1 +
 tests/qemu-iotests/033                        |  1 +
 tests/qemu-iotests/034                        |  3 +-
 tests/qemu-iotests/037                        |  3 +-
 tests/qemu-iotests/039                        |  4 +-
 tests/qemu-iotests/052                        |  2 +-
 tests/qemu-iotests/059                        | 34 ++++++-
 tests/qemu-iotests/059.out                    | 26 +++--
 tests/qemu-iotests/063                        |  3 +-
 tests/qemu-iotests/071                        |  1 +
 tests/qemu-iotests/072                        |  1 +
 tests/qemu-iotests/081                        |  4 +-
 tests/qemu-iotests/091                        |  4 +-
 tests/qemu-iotests/099                        |  1 +
 tests/qemu-iotests/105                        |  3 +-
 tests/qemu-iotests/110                        |  3 +-
 tests/qemu-iotests/120                        |  1 +
 tests/qemu-iotests/126                        |  2 +
 tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
 tests/qemu-iotests/150.out.raw                | 12 +++
 tests/qemu-iotests/162                        |  4 +-
 tests/qemu-iotests/175                        | 47 +++++++--
 tests/qemu-iotests/175.out                    | 16 ++-
 tests/qemu-iotests/178.out.qcow2              |  4 +-
 tests/qemu-iotests/184                        |  1 +
 tests/qemu-iotests/186                        |  1 +
 tests/qemu-iotests/197                        |  1 +
 tests/qemu-iotests/215                        |  1 +
 tests/qemu-iotests/221.out                    | 12 ++-
 tests/qemu-iotests/251                        |  1 +
 tests/qemu-iotests/253.out                    | 12 ++-
 tests/qemu-iotests/common.filter              |  4 +-
 tests/qemu-iotests/common.rc                  | 14 +++
 50 files changed, 391 insertions(+), 87 deletions(-)
 rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
 create mode 100644 tests/qemu-iotests/150.out.raw

-- 
2.21.0

From: Denis Plotnikov <dplotnikov@virtuozzo.com>

The patch allows to provide a pattern file for write
command. There was no similar ability before.

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Message-id: 20190820164616.4072-1-dplotnikov@virtuozzo.com
Reviewed-by: Eric Blake <eblake@redhat.com>
[mreitz: Keep optstring in alphabetical order]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-io-cmds.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 93 insertions(+), 6 deletions(-)

diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -XXX,XX +XXX,XX @@ static void qemu_io_free(void *p)
     qemu_vfree(p);
 }
 
+/*
+ * qemu_io_alloc_from_file()
+ *
+ * Allocates the buffer and populates it with the content of the given file
+ * up to @len bytes. If the file length is less than @len, then the buffer
+ * is populated with the file content cyclically.
+ *
+ * @blk - the block backend where the buffer content is going to be written to
+ * @len - the buffer length
+ * @file_name - the file to read the content from
+ *
+ * Returns: the buffer pointer on success
+ *          NULL on error
+ */
+static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
+                                     const char *file_name)
+{
+    char *buf, *buf_origin;
+    FILE *f = fopen(file_name, "r");
+    int pattern_len;
+
+    if (!f) {
+        perror(file_name);
+        return NULL;
+    }
+
+    if (qemuio_misalign) {
+        len += MISALIGN_OFFSET;
+    }
+
+    buf_origin = buf = blk_blockalign(blk, len);
+
+    if (qemuio_misalign) {
+        buf_origin += MISALIGN_OFFSET;
+        buf += MISALIGN_OFFSET;
+        len -= MISALIGN_OFFSET;
+    }
+
+    pattern_len = fread(buf_origin, 1, len, f);
+
+    if (ferror(f)) {
+        perror(file_name);
+        goto error;
+    }
+
+    if (pattern_len == 0) {
+        fprintf(stderr, "%s: file is empty\n", file_name);
+        goto error;
+    }
+
+    fclose(f);
+
+    if (len > pattern_len) {
+        len -= pattern_len;
+        buf += pattern_len;
+
+        while (len > 0) {
+            size_t len_to_copy = MIN(pattern_len, len);
+
+            memcpy(buf, buf_origin, len_to_copy);
+
+            len -= len_to_copy;
+            buf += len_to_copy;
+        }
+    }
+
+    return buf_origin;
+
+error:
+    qemu_io_free(buf_origin);
+    return NULL;
+}
+
 static void dump_buffer(const void *buffer, int64_t offset, int64_t len)
 {
     uint64_t i;
@@ -XXX,XX +XXX,XX @@ static void write_help(void)
 " -n, -- with -z, don't allow slow fallback\n"
 " -p, -- ignored for backwards compatibility\n"
 " -P, -- use different pattern to fill file\n"
+" -s, -- use a pattern file to fill the write buffer\n"
 " -C, -- report statistics in a machine parsable format\n"
 " -q, -- quiet mode, do not show I/O statistics\n"
 " -u, -- with -z, allow unmapping\n"
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t write_cmd = {
     .perm       = BLK_PERM_WRITE,
     .argmin     = 2,
     .argmax     = -1,
-    .args       = "[-bcCfnquz] [-P pattern] off len",
+    .args       = "[-bcCfnquz] [-P pattern | -s source_file] off len",
     .oneline    = "writes a number of bytes at a specified offset",
     .help       = write_help,
 };
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
 {
     struct timespec t1, t2;
     bool Cflag = false, qflag = false, bflag = false;
-    bool Pflag = false, zflag = false, cflag = false;
+    bool Pflag = false, zflag = false, cflag = false, sflag = false;
     int flags = 0;
     int c, cnt, ret;
     char *buf = NULL;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
     /* Some compilers get confused and warn if this is not initialized.  */
     int64_t total = 0;
     int pattern = 0xcd;
+    const char *file_name = NULL;
 
-    while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) {
+    while ((c = getopt(argc, argv, "bcCfnpP:qs:uz")) != -1) {
         switch (c) {
         case 'b':
             bflag = true;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
         case 'q':
             qflag = true;
             break;
+        case 's':
+            sflag = true;
+            file_name = optarg;
+            break;
         case 'u':
             flags |= BDRV_REQ_MAY_UNMAP;
             break;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
         return -EINVAL;
     }
 
-    if (zflag && Pflag) {
-        printf("-z and -P cannot be specified at the same time\n");
+    if (zflag + Pflag + sflag > 1) {
+        printf("Only one of -z, -P, and -s "
+               "can be specified at the same time\n");
         return -EINVAL;
     }
 
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
     }
 
     if (!zflag) {
-        buf = qemu_io_alloc(blk, count, pattern);
+        if (sflag) {
+            buf = qemu_io_alloc_from_file(blk, count, file_name);
+            if (!buf) {
+                return -EINVAL;
+            }
+        } else {
+            buf = qemu_io_alloc(blk, count, pattern);
+        }
     }
 
     clock_gettime(CLOCK_MONOTONIC, &t1);
-- 
2.21.0

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

It's wrong to OR shared permissions. It may lead to crash on further
permission updates.
Also, no needs to consider previously calculated permissions, as at
this point we already bind all new parents and bdrv_get_cumulative_perm
result is enough. So fix the bug by just set permissions by
bdrv_get_cumulative_perm result.

Bug was introduced in long ago 234ac1a9025, in 2.9.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190824100740.61635-1-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
 {
     BdrvChild *c, *next;
     GSList *list = NULL, *p;
-    uint64_t old_perm, old_shared;
     uint64_t perm = 0, shared = BLK_PERM_ALL;
     int ret;
 
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
         bdrv_unref(from);
     }
 
-    bdrv_get_cumulative_perm(to, &old_perm, &old_shared);
-    bdrv_set_perm(to, old_perm | perm, old_shared | shared);
+    bdrv_get_cumulative_perm(to, &perm, &shared);
+    bdrv_set_perm(to, perm, shared);
 
 out:
     g_slist_free(list);
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster
storage backed by XFS filesystem, reading this block using direct I/O
succeeds regardless of request length, fooling alignment detection.

In this case we fallback to a safe value (4096) instead of the optimal
value (512), which may lead to unneeded data copying when aligning
requests.  Allocating the first block avoids the fallback.

Since we allocate the first block even with preallocation=off, we no
longer create images with zero disk size:

$ ./qemu-img create -f raw test.raw 1g
    Formatting 'test.raw', fmt=raw size=1073741824

$ ls -lhs test.raw
    4.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw

And converting the image requires additional cluster:

$ ./qemu-img measure -f raw -O qcow2 test.raw
    required size: 458752
    fully allocated size: 1074135040

When using format like vmdk with multiple files per image, we allocate
one block per file:

$ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
    Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

$ ls -lhs test*.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer  353 Aug 27 03:23 test.vmdk

I did quick performance test for copying disks with qemu-img convert to
new raw target image to Gluster storage with sector size of 512 bytes:

for i in $(seq 10); do
        rm -f dst.raw
        sleep 10
        time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
    done

Here is a table comparing the total time spent:

Type    Before(s)   After(s)    Diff(%)
---------------------------------------
real      530.028    469.123      -11.4
user       17.204     10.768      -37.4
sys        17.881      7.011      -60.7

We can see very clear improvement in CPU usage.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827010528.8818-2-nsoffer@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/file-posix.c                            | 51 +++++++++++++++++++
 tests/qemu-iotests/059.out                    |  2 +-
 tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
 tests/qemu-iotests/150.out.raw                | 12 +++++
 tests/qemu-iotests/175                        | 19 ++++---
 tests/qemu-iotests/175.out                    |  8 +--
 tests/qemu-iotests/178.out.qcow2              |  4 +-
 tests/qemu-iotests/221.out                    | 12 +++--
 tests/qemu-iotests/253.out                    | 12 +++--
 9 files changed, 99 insertions(+), 21 deletions(-)
 rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
 create mode 100644 tests/qemu-iotests/150.out.raw

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_discard(void *opaque)
     return ret;
 }
 
+/*
+ * Help alignment probing by allocating the first block.
+ *
+ * When reading with direct I/O from unallocated area on Gluster backed by XFS,
+ * reading succeeds regardless of request length. In this case we fallback to
+ * safe alignment which is not optimal. Allocating the first block avoids this
+ * fallback.
+ *
+ * fd may be opened with O_DIRECT, but we don't know the buffer alignment or
+ * request alignment, so we use safe values.
+ *
+ * Returns: 0 on success, -errno on failure. Since this is an optimization,
+ * caller may ignore failures.
+ */
+static int allocate_first_block(int fd, size_t max_size)
+{
+    size_t write_size = (max_size < MAX_BLOCKSIZE)
+        ? BDRV_SECTOR_SIZE
+        : MAX_BLOCKSIZE;
+    size_t max_align = MAX(MAX_BLOCKSIZE, getpagesize());
+    void *buf;
+    ssize_t n;
+    int ret;
+
+    buf = qemu_memalign(max_align, write_size);
+    memset(buf, 0, write_size);
+
+    do {
+        n = pwrite(fd, buf, write_size, 0);
+    } while (n == -1 && errno == EINTR);
+
+    ret = (n == -1) ? -errno : 0;
+
+    qemu_vfree(buf);
+    return ret;
+}
+
 static int handle_aiocb_truncate(void *opaque)
 {
     RawPosixAIOData *aiocb = opaque;
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
                 /* posix_fallocate() doesn't set errno. */
                 error_setg_errno(errp, -result,
                                  "Could not preallocate new data");
+            } else if (current_length == 0) {
+                /*
+                 * posix_fallocate() uses fallocate() if the filesystem
+                 * supports it, or fallback to manually writing zeroes. If
+                 * fallocate() was used, unaligned reads from the fallocated
+                 * area in raw_probe_alignment() will succeed, hence we need to
+                 * allocate the first block.
+                 *
+                 * Optimize future alignment probing; ignore failures.
+                 */
+                allocate_first_block(fd, offset);
             }
         } else {
             result = 0;
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
         if (ftruncate(fd, offset) != 0) {
             result = -errno;
             error_setg_errno(errp, -result, "Could not resize file");
+        } else if (current_length == 0 && offset > current_length) {
+            /* Optimize future alignment probing; ignore failures. */
+            allocate_first_block(fd, offset);
         }
         return result;
     default:
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMax
 image: TEST_DIR/t.vmdk
 file format: vmdk
 virtual size: 0.977 TiB (1073741824000 bytes)
-disk size: 16 KiB
+disk size: 1.97 MiB
 Format specific information:
     cid: XXXXXXXX
     parent cid: XXXXXXXX
diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out.qcow2
similarity index 100%
rename from tests/qemu-iotests/150.out
rename to tests/qemu-iotests/150.out.qcow2
diff --git a/tests/qemu-iotests/150.out.raw b/tests/qemu-iotests/150.out.raw
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/150.out.raw
@@ -XXX,XX +XXX,XX @@
+QA output created by 150
+
+=== Mapping sparse conversion ===
+
+Offset          Length          File
+0               0x1000          TEST_DIR/t.IMGFMT
+
+=== Mapping non-sparse conversion ===
+
+Offset          Length          File
+0               0x100000        TEST_DIR/t.IMGFMT
+*** done
diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # the file size.  This function hides the resulting difference in the
 # stat -c '%b' output.
 # Parameter 1: Number of blocks an empty file occupies
-# Parameter 2: Image size in bytes
+# Parameter 2: Minimal number of blocks in an image
+# Parameter 3: Image size in bytes
 _filter_blocks()
 {
     extra_blocks=$1
-    img_size=$2
+    min_blocks=$2
+    img_size=$3
 
-    sed -e "s/blocks=$extra_blocks\$\$\\|[^0-9]\$/nothing allocated/" \
-        -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/everything allocated/"
+    sed -e "s/blocks=$min_blocks\$\$\\|[^0-9]\$/min allocation/" \
+        -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/max allocation/"
 }
 
 # get standard environment, filters and checks
@@ -XXX,XX +XXX,XX @@ size=$((1 * 1024 * 1024))
 touch "$TEST_DIR/empty"
 extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 
+# We always write the first byte; check how many blocks this filesystem
+# allocates to match empty image alloation.
+printf "\0" > "$TEST_DIR/empty"
+min_blocks=$(stat -c '%b' "$TEST_DIR/empty")
+
 echo
 echo "== creating image with default preallocation =="
 _make_test_img $size | _filter_imgfmt
-stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
+stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 
 for mode in off full falloc; do
     echo
     echo "== creating image with preallocation $mode =="
     IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
-    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
+    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 done
 
 # success, all done
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ QA output created by 175
 
 == creating image with default preallocation ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
-size=1048576, nothing allocated
+size=1048576, min allocation
 
 == creating image with preallocation off ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
-size=1048576, nothing allocated
+size=1048576, min allocation
 
 == creating image with preallocation full ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
-size=1048576, everything allocated
+size=1048576, max allocation
 
 == creating image with preallocation falloc ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
-size=1048576, everything allocated
+size=1048576, max allocation
  *** done
diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/178.out.qcow2
+++ b/tests/qemu-iotests/178.out.qcow2
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
 == raw input image with data (human) ==
 
 Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
-required size: 393216
+required size: 458752
 fully allocated size: 1074135040
 wrote 512/512 bytes at offset 512
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
 
 Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
 {
-    "required": 393216,
+    "required": 458752,
     "fully-allocated": 1074135040
 }
 wrote 512/512 bytes at offset 512
diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/221.out
+++ b/tests/qemu-iotests/221.out
@@ -XXX,XX +XXX,XX @@ QA output created by 221
 === Check mapping of unaligned raw image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537
-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 wrote 1/1 bytes at offset 65536
 1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 *** done
diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/253.out
+++ b/tests/qemu-iotests/253.out
@@ -XXX,XX +XXX,XX @@ QA output created by 253
 === Check mapping of unaligned raw image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048575
-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 wrote 65535/65535 bytes at offset 983040
 63.999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
 *** done
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Using block_resize we can test allocate_first_block() with file
descriptor opened with O_DIRECT, ensuring that it works for any size
larger than 4096 bytes.

Testing smaller sizes is tricky as the result depends on the filesystem
used for testing. For example on NFS any size will work since O_DIRECT
does not require any alignment.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190827010528.8818-3-nsoffer@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/175     | 28 ++++++++++++++++++++++++++++
 tests/qemu-iotests/175.out |  8 ++++++++
 2 files changed, 36 insertions(+)

diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -XXX,XX +XXX,XX @@ _filter_blocks()
         -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/max allocation/"
 }
 
+# Resize image using block_resize.
+# Parameter 1: image path
+# Parameter 2: new size
+_block_resize()
+{
+    local path=$1
+    local size=$2
+
+    $QEMU -qmp stdio -nographic -nodefaults \
+        -blockdev file,node-name=file,filename=$path,cache.direct=on \
+        <<EOF
+{'execute': 'qmp_capabilities'}
+{'execute': 'block_resize', 'arguments': {'node-name': 'file', 'size': $size}}
+{'execute': 'quit'}
+EOF
+}
+
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
@@ -XXX,XX +XXX,XX @@ _supported_fmt raw
 _supported_proto file
 _supported_os Linux
 
+_default_cache_mode none
+_supported_cache_modes none directsync
+
 size=$((1 * 1024 * 1024))
 
 touch "$TEST_DIR/empty"
@@ -XXX,XX +XXX,XX @@ for mode in off full falloc; do
     stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 done
 
+for new_size in 4096 1048576; do
+    echo
+    echo "== resize empty image with block_resize =="
+    _make_test_img 0 | _filter_imgfmt
+    _block_resize $TEST_IMG $new_size >/dev/null
+    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $new_size
+done
+
 # success, all done
 echo "*** done"
 rm -f $seq.full
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ size=1048576, max allocation
 == creating image with preallocation falloc ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
 size=1048576, max allocation
+
+== resize empty image with block_resize ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
+size=4096, min allocation
+
+== resize empty image with block_resize ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
+size=1048576, min allocation
  *** done
-- 
2.21.0

fe646693acc changed qemu-img create's output so that it no longer prints
single quotes around parameter values.  The subformat and adapter_type
filters in _filter_img_create() have never been adapted to that change.

Fixes: fe646693acc13ac48b98435d14149ab04dc597bc
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-2-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/059.out       | 16 ++++++++--------
 tests/qemu-iotests/common.filter |  4 ++--
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-io: can't open device TEST_DIR/t.vmdk: L1 size too big
 
 === Testing monolithicFlat creation and opening ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 2 GiB (2147483648 bytes)
 
 === Testing monolithicFlat with zeroed_grain ===
 qemu-img: TEST_DIR/t.IMGFMT: Flat image can't enable zeroed grain
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
 
 === Testing big twoGbMaxExtentFlat ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMaxExtentFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000
 image: TEST_DIR/t.vmdk
 file format: vmdk
 virtual size: 0.977 TiB (1073741824000 bytes)
@@ -XXX,XX +XXX,XX @@ Format specific information:
 qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Invalid extent line: RW 12582912 VMFS "dummy.IMGFMT" 1
 
 === Testing truncated sparse ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400
 qemu-img: Could not open 'TEST_DIR/t.IMGFMT': File truncated, expecting at least 13172736 bytes
 
 === Converting to streamOptimized from image with small cluster size===
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing monolithicFlat with internally generated JSON file name ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
 
 === Testing version 3 ===
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 64931328
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing 4TB monolithicFlat creation and IO ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 4 TiB (4398046511104 bytes)
@@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 966367641600
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing qemu-img map on extents ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
 wrote 1024/1024 bytes at offset 65024
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 1024/1024 bytes at offset 2147483136
@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
 0               0x20000         0x3f0000        TEST_DIR/t.vmdk
 0x7fff0000      0x20000         0x410000        TEST_DIR/t.vmdk
 0x140000000     0x10000         0x430000        TEST_DIR/t.vmdk
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=twoGbMaxExtentSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
 wrote 1024/1024 bytes at offset 65024
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 1024/1024 bytes at offset 2147483136
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_img_create()
         -e "s# compat6=\$on\\|off\$##g" \
         -e "s# static=\$on\\|off\$##g" \
         -e "s# zeroed_grain=\$on\\|off\$##g" \
-        -e "s# subformat='[^']*'##g" \
-        -e "s# adapter_type='[^']*'##g" \
+        -e "s# subformat=[^ ]*##g" \
+        -e "s# adapter_type=[^ ]*##g" \
         -e "s# hwversion=[^ ]*##g" \
         -e "s# lazy_refcounts=\$on\\|off\$##g" \
         -e "s# block_size=[0-9]\\+##g" \
-- 
2.21.0

This makes iotest 033 pass with e.g. subformat=monolithicFlat.  It also
turns a former error in 059 into success.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190815153638.4600-3-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c               | 54 ++++++++++++++++++++++++--------------
 tests/qemu-iotests/059     |  7 +++--
 tests/qemu-iotests/059.out |  4 ++-
 3 files changed, 42 insertions(+), 23 deletions(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static const char *next_line(const char *s)
 }
 
 static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
-                              const char *desc_file_path, QDict *options,
-                              Error **errp)
+                              QDict *options, Error **errp)
 {
     int ret;
     int matches;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
     const char *p, *np;
     int64_t sectors = 0;
     int64_t flat_offset;
+    char *desc_file_dir = NULL;
     char *extent_path;
     BdrvChild *extent_file;
     BDRVVmdkState *s = bs->opaque;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
             continue;
         }
 
-        if (!path_is_absolute(fname) && !path_has_protocol(fname) &&
-            !desc_file_path[0])
-        {
-            bdrv_refresh_filename(bs->file->bs);
-            error_setg(errp, "Cannot use relative extent paths with VMDK "
-                       "descriptor file '%s'", bs->file->bs->filename);
-            return -EINVAL;
-        }
+        if (path_is_absolute(fname)) {
+            extent_path = g_strdup(fname);
+        } else {
+            if (!desc_file_dir) {
+                desc_file_dir = bdrv_dirname(bs->file->bs, errp);
+                if (!desc_file_dir) {
+                    bdrv_refresh_filename(bs->file->bs);
+                    error_prepend(errp, "Cannot use relative paths with VMDK "
+                                  "descriptor file '%s': ",
+                                  bs->file->bs->filename);
+                    ret = -EINVAL;
+                    goto out;
+                }
+            }
 
-        extent_path = path_combine(desc_file_path, fname);
+            extent_path = g_strconcat(desc_file_dir, fname, NULL);
+        }
 
         ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
         assert(ret < 32);
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
         g_free(extent_path);
         if (local_err) {
             error_propagate(errp, local_err);
-            return -EINVAL;
+            ret = -EINVAL;
+            goto out;
         }
 
         /* save to extents array */
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
                             0, 0, 0, 0, 0, &extent, errp);
             if (ret < 0) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent->flat_start_offset = flat_offset << 9;
         } else if (!strcmp(type, "SPARSE") || !strcmp(type, "VMFSSPARSE")) {
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
             g_free(buf);
             if (ret) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent = &s->extents[s->num_extents - 1];
         } else if (!strcmp(type, "SESPARSE")) {
             ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
             if (ret) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent = &s->extents[s->num_extents - 1];
         } else {
             error_setg(errp, "Unsupported extent type '%s'", type);
             bdrv_unref_child(bs, extent_file);
-            return -ENOTSUP;
+            ret = -ENOTSUP;
+            goto out;
         }
         extent->type = g_strdup(type);
     }
-    return 0;
+
+    ret = 0;
+    goto out;
 
 invalid:
     np = next_line(p);
@@ -XXX,XX +XXX,XX @@ invalid:
         np--;
     }
     error_setg(errp, "Invalid extent line: %.*s", (int)(np - p), p);
-    return -EINVAL;
+    ret = -EINVAL;
+
+out:
+    g_free(desc_file_dir);
+    return ret;
 }
 
 static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
     }
     s->create_type = g_strdup(ct);
     s->desc_offset = 0;
-    ret = vmdk_parse_extents(buf, bs, bs->file->bs->exact_filename, options,
-                             errp);
+    ret = vmdk_parse_extents(buf, bs, options, errp);
 exit:
     return ret;
 }
diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
 
 echo
 echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+# Should work, because bdrv_dirname() works fine with blkdebug
 IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
-$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" 2>&1 \
-    | _filter_testdir | _filter_imgfmt
+$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
+         -c info \
+    2>&1 \
+    | _filter_testdir | _filter_imgfmt | _filter_img_info
 _cleanup_test_img
 
 echo
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 
 === Testing monolithicFlat with internally generated JSON file name ===
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
+format name: IMGFMT
+cluster size: 0 bytes
+vm state offset: 0 bytes
 
 === Testing version 3 ===
 image: TEST_DIR/iotest-version3.IMGFMT
-- 
2.21.0

We had a test for a case where relative extent paths did not work, but
unfortunately we just fixed the underlying problem, so it works now.
This patch adds a new test case that still fails.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-4-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/059     | 27 +++++++++++++++++++++++++++
 tests/qemu-iotests/059.out |  4 ++++
 2 files changed, 31 insertions(+)

diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
 
 echo
 echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+
+echo '--- blkdebug ---'
 # Should work, because bdrv_dirname() works fine with blkdebug
 IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
 $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TE
     | _filter_testdir | _filter_imgfmt | _filter_img_info
 _cleanup_test_img
 
+echo '--- quorum ---'
+# Should not work, because bdrv_dirname() does not work with quorum
+IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
+cp "$TEST_IMG" "$TEST_IMG.orig"
+
+filename="json:{
+    \"driver\": \"$IMGFMT\",
+    \"file\": {
+        \"driver\": \"quorum\",
+        \"children\": [ {
+            \"driver\": \"file\",
+            \"filename\": \"$TEST_IMG\"
+        }, {
+            \"driver\": \"file\",
+            \"filename\": \"$TEST_IMG.orig\"
+        } ],
+        \"vote-threshold\": 1
+    } }"
+
+filename=$(echo "$filename" | tr '\n' ' ' | sed -e 's/\s\+/ /g')
+$QEMU_IMG info "$filename" 2>&1 \
+    | sed -e "s/'json:[^']*'/\$QUORUM_FILE/g" \
+    | _filter_testdir | _filter_imgfmt | _filter_img_info
+
+
 echo
 echo "=== Testing version 3 ==="
 _use_sample_img iotest-version3.vmdk.bz2
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing monolithicFlat with internally generated JSON file name ===
+--- blkdebug ---
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 format name: IMGFMT
 cluster size: 0 bytes
 vm state offset: 0 bytes
+--- quorum ---
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+qemu-img: Could not open $QUORUM_FILE: Cannot use relative paths with VMDK descriptor file $QUORUM_FILE: Cannot generate a base directory for quorum nodes
 
 === Testing version 3 ===
 image: TEST_DIR/iotest-version3.IMGFMT
-- 
2.21.0

Compressed writes generally have to write full clusters, not just in
theory but also in practice when it comes to vmdk's streamOptimized
subformat.  It currently is just silently broken for writes with
non-zero in-cluster offsets:

$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 4k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 4096
4 KiB, 1 ops; 00.01 sec (443.724 KiB/sec and 110.9309 ops/sec)
read failed: Invalid argument

(The technical reason is that vmdk_write_extent() just writes the
incomplete compressed data actually to offset 4k.  When reading the
data, vmdk_read_extent() looks at offset 0 and finds the compressed data
size to be 0, because that is what it reads from there.  This yields an
error.)

For incomplete writes with zero in-cluster offsets, the error path when
reading the rest of the cluster is a bit different, but the result is
the same:

$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 0k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 0
4 KiB, 1 ops; 00.01 sec (362.641 KiB/sec and 90.6603 ops/sec)
read failed: Invalid argument

(Here, vmdk_read_extent() finds the data and then sees that the
uncompressed data is short.)

It is better to reject invalid writes than to make the user believe they
might have succeeded and then fail when trying to read it back.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-5-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset,
     if (extent->compressed) {
         void *compressed_data;
 
+        /* Only whole clusters */
+        if (offset_in_cluster ||
+            n_bytes > (extent->cluster_sectors * SECTOR_SIZE) ||
+            (n_bytes < (extent->cluster_sectors * SECTOR_SIZE) &&
+             offset + n_bytes != extent->end_sector * SECTOR_SIZE))
+        {
+            ret = -EINVAL;
+            goto out;
+        }
+
         if (!extent->has_marker) {
             ret = -EINVAL;
             goto out;
-- 
2.21.0

streamOptimized does not support writes that do not span exactly one
cluster.  Furthermore, it cannot rewrite already allocated clusters.
As such, many iotests do not work with it.  Disable them.

diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/002
+++ b/tests/qemu-iotests/002
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/003
+++ b/tests/qemu-iotests/003
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 size=128M
 offset=67M
diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 _supported_os Linux
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 # vpc is limited to 127GB, so we can't test it here
 if [ "$IMGFMT" = "vpc" ]; then
diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/009
+++ b/tests/qemu-iotests/009
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/010
+++ b/tests/qemu-iotests/010
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/011
+++ b/tests/qemu-iotests/011
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/017
+++ b/tests/qemu-iotests/017
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto generic
 _unsupported_proto vxhs
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/018
+++ b/tests/qemu-iotests/018
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _supported_os Linux
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/019
+++ b/tests/qemu-iotests/019
@@ -XXX,XX +XXX,XX @@ _supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 CLUSTER_SIZE=65536
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/027
+++ b/tests/qemu-iotests/027
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vmdk qcow qcow2 qed
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/032
+++ b/tests/qemu-iotests/032
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This works for any image format (though unlikely to segfault for raw)
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo === Prepare image ===
diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/034
+++ b/tests/qemu-iotests/034
@@ -XXX,XX +XXX,XX @@ _supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 CLUSTER_SIZE=4k
 size=128M
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 CLUSTER_SIZE=4k
 size=128M
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed raw
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 _make_test_img 4M
 
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
 _supported_proto file
+_unsupported_imgopts "subformat=streamOptimized"
 
 IMG_SIZE=64M
 
diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/105
+++ b/tests/qemu-iotests/105
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2 vmdk vhdx qed
 _supported_proto generic
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 echo
 echo "creating large image"
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/251
+++ b/tests/qemu-iotests/251
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt generic
 _supported_proto file
 _supported_os Linux
+_unsupported_imgopts "subformat=streamOptimized"
 
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
     # We use json:{} filenames here, so we cannot work with additional options.
-- 
2.21.0

The error message for the test case where we have a quorum node for
which no directory name can be generated is different: For
twoGbMaxExtentSparse, it complains that it cannot open the extent file.
For other (sub)formats, it just notes that it cannot determine the
backing file path.  Both are fine, but just disable twoGbMaxExtentSparse
for simplicity's sake.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-7-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/110 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "subformat=twoGbMaxExtentSparse"
 
 TEST_IMG_REL=$(basename "$TEST_IMG")
 
-- 
2.21.0

From: Stefan Hajnoczi <stefanha@redhat.com>

Fixes: a6b257a08e3d72219f03e461a52152672fec0612
       ("file-posix: Handle undetectable alignment")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20190827101328.4062-1-stefanha@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/file-posix.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
             align = alignments[i];
             if (raw_is_io_aligned(fd, buf + align, max_align)) {
-                /* Fallback to request_aligment. */
+                /* Fallback to request_alignment. */
                 s->buf_align = (align != 1) ? align : bs->bl.request_alignment;
                 break;
             }
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

It is possible to enable only a subset of the block drivers with the
"--block-drv-rw-whitelist" option of the "configure" script. All other
drivers are marked as unusable (or only included as read-only with the
"--block-drv-ro-whitelist" option). If an iotest is now using such a
disabled block driver, it is failing - which is bad, since at least the
tests in the "auto" group should be able to deal with this situation.
Thus let's introduce a "_require_drivers" function that can be used by
the shell tests to check for the availability of certain drivers first,
and marks the test as "not run" if one of the drivers is missing.

This patch mainly targets the test in the "auto" group which should
never fail in such a case, but also improves some of the other tests
along the way. Note that we also assume that the "qcow2" and "file"
drivers are always available - otherwise it does not make sense to
run "make check-block" at all (which only tests with qcow2 by default).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190823133552.11680-1-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/071       |  1 +
 tests/qemu-iotests/081       |  4 +---
 tests/qemu-iotests/099       |  1 +
 tests/qemu-iotests/120       |  1 +
 tests/qemu-iotests/162       |  4 +---
 tests/qemu-iotests/184       |  1 +
 tests/qemu-iotests/186       |  1 +
 tests/qemu-iotests/common.rc | 14 ++++++++++++++
 8 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
+_require_drivers blkdebug blkverify
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/081
+++ b/tests/qemu-iotests/081
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt raw
 _supported_proto file
 _supported_os Linux
+_require_drivers quorum
 
 do_run_qemu()
 {
@@ -XXX,XX +XXX,XX @@ run_qemu()
                           | _filter_qemu_io | _filter_generated_node_ids
 }
 
-test_quorum=$($QEMU_IMG --help|grep quorum)
-[ "$test_quorum" = "" ] && _supported_fmt quorum
-
 quorum="driver=raw,file.driver=quorum,file.vote-threshold=2"
 quorum="$quorum,file.children.0.file.filename=$TEST_DIR/1.raw"
 quorum="$quorum,file.children.1.file.filename=$TEST_DIR/2.raw"
diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/099
+++ b/tests/qemu-iotests/099
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 qed vdi vhdx vmdk vpc
 _supported_proto file
 _supported_os Linux
+_require_drivers blkdebug blkverify
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
     "subformat=twoGbMaxExtentSparse"
 
diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/120
+++ b/tests/qemu-iotests/120
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt generic
 _supported_proto file
 _unsupported_fmt luks
+_require_drivers raw
 
 _make_test_img 64M
 
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt generic
-
-test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh$ \|$$')
-[ "$test_ssh" = "" ] && _notrun "ssh support required"
+_require_drivers ssh
 
 echo
 echo '=== NBD ==='
diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/184
+++ b/tests/qemu-iotests/184
@@ -XXX,XX +XXX,XX @@ trap "exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_os Linux
+_require_drivers throttle
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/186
+++ b/tests/qemu-iotests/186
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
+_require_drivers null-co
 
 if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
     _notrun "Requires a PC machine"
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -XXX,XX +XXX,XX @@ _require_command()
     [ -x "$c" ] || _notrun "$1 utility required, skipped this test"
 }
 
+# Check that a set of drivers has been whitelisted in the QEMU binary
+#
+_require_drivers()
+{
+    available=$($QEMU -drive format=help | \
+                sed -e '/Supported formats:/!d' -e 's/Supported formats://')
+    for driver
+    do
+        if ! echo "$available" | grep -q " $driver$ \|$$"; then
+            _notrun "$driver not available"
+        fi
+    done
+}
+
 # make sure this script returns success
 true
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

The sanitizers (especially the address sanitizer from Clang) are
sometimes printing out warnings or false positives - this spoils
the output of the iotests, causing some of the tests to fail.
Thus let's skip the automatic iotests during "make check" when the
user configured QEMU with --enable-sanitizers.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190823084203.29734-1-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/check-block.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tests/check-block.sh b/tests/check-block.sh
index XXXXXXX..XXXXXXX 100755
--- a/tests/check-block.sh
+++ b/tests/check-block.sh
@@ -XXX,XX +XXX,XX @@ if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
     exit 0
 fi
 
+if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
+    echo "Sanitizers are enabled ==> Not running the qemu-iotests."
+    exit 0
+fi
+
 if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
     echo "No qemu-system binary available ==> Not running the qemu-iotests."
     exit 0
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Quoting cache mode is not needed, and most tests use unquoted values.
Unify all test to use the same style.

Message-id: 20190827173432.7656-1-nsoffer@redhat.com
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/026 | 4 ++--
 tests/qemu-iotests/039 | 4 ++--
 tests/qemu-iotests/052 | 2 +-
 tests/qemu-iotests/091 | 4 ++--
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/026
+++ b/tests/qemu-iotests/026
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Currently only qcow2 supports rebasing
 _supported_fmt qcow2
 _supported_proto file
-_default_cache_mode "writethrough"
-_supported_cache_modes "writethrough" "none"
+_default_cache_mode writethrough
+_supported_cache_modes writethrough none
 # The refcount table tests expect a certain minimum width for refcount entries
 # (so that the refcount table actually needs to grow); that minimum is 16 bits,
 # being the default refcount entry width.
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
-_default_cache_mode "writethrough"
-_supported_cache_modes "writethrough"
+_default_cache_mode writethrough
+_supported_cache_modes writethrough
 
 size=128M
 
diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/052
+++ b/tests/qemu-iotests/052
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto file
 
 # Don't do O_DIRECT on tmpfs
-_supported_cache_modes "writeback" "writethrough" "unsafe"
+_supported_cache_modes writeback writethrough unsafe
 
 size=128M
 _make_test_img $size
diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/091
+++ b/tests/qemu-iotests/091
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
-_default_cache_mode "none"
-_supported_cache_modes "writethrough" "none" "writeback"
+_default_cache_mode none
+_supported_cache_modes writethrough none writeback
 
 size=1G
 
-- 
2.21.0