Series comparison

-[PULL 0/3] Block patches
+[Qemu-devel] [PULL v2 00/16] Block patches
-The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:
+The following changes since commit 54b89db5309d5fa8b5d3fe5fe56f81704e2f9706:
-  Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)
+  Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2019-09-03 09:43:26 +0100)
 are available in the Git repository at:
-  https://gitlab.com/stefanha/qemu.git tags/block-pull-request
+  https://github.com/XanClic/qemu.git tags/pull-block-2019-09-03
-for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:
+for you to fetch changes up to 755c5fe79d88717600356d3edf04835bba43dcb6:
-  util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)
+  iotests: Unify cache mode quoting (2019-09-03 14:56:06 +0200)
 ----------------------------------------------------------------
-Pull request
+Block patches:
+- qemu-io now accepts a file to read a write pattern from
-Add new thread-pool-min/thread-pool-max parameters to control the thread pool
+- Ensure that raw files have their first block allocated so we can probe
-used for async I/O.
+  the O_DIRECT alignment if necessary
 - Various fixes
 ----------------------------------------------------------------
+v2:
+- Added a patch we already had on the list to keep the iotests passing
+  when $DISPLAY is not set
-Nicolas Saenz Julienne (3):
+----------------------------------------------------------------
-  Introduce event-loop-base abstract class
+Denis Plotnikov (1):
-  util/main-loop: Introduce the main loop into QOM
+  qemu-io: add pattern file for write command
   util/event-loop-base: Introduce options to set the thread pool size
- qapi/qom.json                    |  43 ++++++++--
+Max Reitz (8):
- meson.build                      |  26 +++---
+  iotests: Fix _filter_img_create()
- include/block/aio.h              |  10 +++
+  vmdk: Use bdrv_dirname() for relative extent paths
- include/block/thread-pool.h      |   3 +
+  iotests: Keep testing broken relative extent paths
- include/qemu/main-loop.h         |  10 +++
+  vmdk: Reject invalid compressed writes
- include/sysemu/event-loop-base.h |  41 +++++++++
+  iotests: Disable broken streamOptimized tests
- include/sysemu/iothread.h        |   6 +-
+  iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
- event-loop-base.c                | 140 +++++++++++++++++++++++++++++++
+  iotests: Disable 126 for flat vmdk subformats
- iothread.c                       |  68 +++++----------
+  iotests: Add -display none to the qemu options
- util/aio-posix.c                 |   1 +
- util/async.c                     |  20 +++++
+Nir Soffer (3):
- util/main-loop.c                 |  65 ++++++++++++++
+  block: posix: Always allocate the first block
- util/thread-pool.c               |  55 +++++++++++-
+  iotests: Test allocate_first_block() with O_DIRECT
-files changed, 419 insertions(+), 69 deletions(-)
+  iotests: Unify cache mode quoting
- create mode 100644 include/sysemu/event-loop-base.h
- create mode 100644 event-loop-base.c
+Stefan Hajnoczi (1):
   file-posix: fix request_alignment typo
 Thomas Huth (2):
   iotests: Check for enabled drivers before testing them
   tests/check-block: Skip iotests when sanitizers are enabled
 Vladimir Sementsov-Ogievskiy (1):
   block: fix permission update in bdrv_replace_node
  block.c                                       |  5 +-
  block/file-posix.c                            | 53 +++++++++-
  block/vmdk.c                                  | 64 ++++++++----
  qemu-io-cmds.c                                | 99 +++++++++++++++++--
  tests/check-block.sh                          |  5 +
  tests/qemu-iotests/002                        |  1 +
  tests/qemu-iotests/003                        |  1 +
  tests/qemu-iotests/005                        |  3 +-
  tests/qemu-iotests/009                        |  1 +
  tests/qemu-iotests/010                        |  1 +
  tests/qemu-iotests/011                        |  1 +
  tests/qemu-iotests/017                        |  3 +-
  tests/qemu-iotests/018                        |  3 +-
  tests/qemu-iotests/019                        |  3 +-
  tests/qemu-iotests/020                        |  3 +-
  tests/qemu-iotests/026                        |  4 +-
  tests/qemu-iotests/027                        |  1 +
  tests/qemu-iotests/032                        |  1 +
  tests/qemu-iotests/033                        |  1 +
  tests/qemu-iotests/034                        |  3 +-
  tests/qemu-iotests/037                        |  3 +-
  tests/qemu-iotests/039                        |  4 +-
  tests/qemu-iotests/052                        |  2 +-
  tests/qemu-iotests/059                        | 34 ++++++-
  tests/qemu-iotests/059.out                    | 26 +++--
  tests/qemu-iotests/063                        |  3 +-
  tests/qemu-iotests/071                        |  1 +
  tests/qemu-iotests/072                        |  1 +
  tests/qemu-iotests/081                        |  4 +-
  tests/qemu-iotests/091                        |  4 +-
  tests/qemu-iotests/099                        |  1 +
  tests/qemu-iotests/105                        |  3 +-
  tests/qemu-iotests/110                        |  3 +-
  tests/qemu-iotests/120                        |  1 +
  tests/qemu-iotests/126                        |  2 +
  tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
  tests/qemu-iotests/150.out.raw                | 12 +++
  tests/qemu-iotests/162                        |  4 +-
  tests/qemu-iotests/175                        | 47 +++++++--
  tests/qemu-iotests/175.out                    | 16 ++-
  tests/qemu-iotests/178.out.qcow2              |  4 +-
  tests/qemu-iotests/184                        |  1 +
  tests/qemu-iotests/186                        |  1 +
  tests/qemu-iotests/197                        |  1 +
  tests/qemu-iotests/215                        |  1 +
  tests/qemu-iotests/221.out                    | 12 ++-
  tests/qemu-iotests/251                        |  1 +
  tests/qemu-iotests/253.out                    | 12 ++-
  tests/qemu-iotests/check                      |  6 +-
  tests/qemu-iotests/common.filter              |  4 +-
  tests/qemu-iotests/common.rc                  | 14 +++
 files changed, 394 insertions(+), 90 deletions(-)
  rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
  create mode 100644 tests/qemu-iotests/150.out.raw
 --
-.35.1
+.21.0

-[PULL 3/3] util/event-loop-base: Introduce options to set the thread pool size
+[Qemu-devel] [PULL v2 01/16] qemu-io: add pattern file for write command
-From: Nicolas Saenz Julienne <nsaenzju@redhat.com>
+From: Denis Plotnikov <dplotnikov@virtuozzo.com>
-The thread pool regulates itself: when idle, it kills threads until
+The patch allows to provide a pattern file for write
-empty, when in demand, it creates new threads until full. This behaviour
+command. There was no similar ability before.
 doesn't play well with latency sensitive workloads where the price of
 creating a new thread is too high. For example, when paired with qemu's
 '-mlock', or using safety features like SafeStack, creating a new thread
 has been measured take multiple milliseconds.
-In order to mitigate this let's introduce a new 'EventLoopBase'
+Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
-property to set the thread pool size. The threads will be created during
+Message-id: 20190820164616.4072-1-dplotnikov@virtuozzo.com
-the pool's initialization or upon updating the property's value, remain
+Reviewed-by: Eric Blake <eblake@redhat.com>
-available during its lifetime regardless of demand, and destroyed upon
+[mreitz: Keep optstring in alphabetical order]
-freeing it. A properly characterized workload will then be able to
+Signed-off-by: Max Reitz <mreitz@redhat.com>
-configure the pool to avoid any latency spikes.
+---
  qemu-io-cmds.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++---
 file changed, 93 insertions(+), 6 deletions(-)
-Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
+diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Acked-by: Markus Armbruster <armbru@redhat.com>
 Message-id: 20220425075723.20019-4-nsaenzju@redhat.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
  qapi/qom.json                    | 10 +++++-
  include/block/aio.h              | 10 ++++++
  include/block/thread-pool.h      |  3 ++
  include/sysemu/event-loop-base.h |  4 +++
  event-loop-base.c                | 23 +++++++++++++
  iothread.c                       |  3 ++
  util/aio-posix.c                 |  1 +
  util/async.c                     | 20 ++++++++++++
  util/main-loop.c                 |  9 ++++++
  util/thread-pool.c               | 55 +++++++++++++++++++++++++++++---
 files changed, 133 insertions(+), 5 deletions(-)
 diff --git a/qapi/qom.json b/qapi/qom.json
 index XXXXXXX..XXXXXXX 100644
---- a/qapi/qom.json
+--- a/qemu-io-cmds.c
-+++ b/qapi/qom.json
++++ b/qemu-io-cmds.c
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static void qemu_io_free(void *p)
- #                 0 means that the engine will use its default.
+     qemu_vfree(p);
- #                 (default: 0)
+ }
- #
-+# @thread-pool-min: minimum number of threads reserved in the thread pool
++/*
-+#                   (default:0)
++ * qemu_io_alloc_from_file()
-+#
++ *
-+# @thread-pool-max: maximum number of threads the thread pool can contain
++ * Allocates the buffer and populates it with the content of the given file
-+#                   (default:64)
++ * up to @len bytes. If the file length is less than @len, then the buffer
-+#
++ * is populated with the file content cyclically.
- # Since: 7.1
++ *
- ##
++ * @blk - the block backend where the buffer content is going to be written to
- { 'struct': 'EventLoopBaseProperties',
++ * @len - the buffer length
--  'data': { '*aio-max-batch': 'int' } }
++ * @file_name - the file to read the content from
-+  'data': { '*aio-max-batch': 'int',
++ *
-+            '*thread-pool-min': 'int',
++ * Returns: the buffer pointer on success
-+            '*thread-pool-max': 'int' } }
++ *          NULL on error
  ##
  # @IothreadProperties:
 diff --git a/include/block/aio.h b/include/block/aio.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/aio.h
 +++ b/include/block/aio.h
@@ -XXX,XX +XXX,XX @@ struct AioContext {
      QSLIST_HEAD(, Coroutine) scheduled_coroutines;
      QEMUBH *co_schedule_bh;
 +    int thread_pool_min;
 +    int thread_pool_max;
      /* Thread pool for performing work and receiving completion callbacks.
       * Has its own locking.
       */
@@ -XXX,XX +XXX,XX @@ void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns,
  void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch,
                                  Error **errp);
 +/**
 + * aio_context_set_thread_pool_params:
 + * @ctx: the aio context
 + * @min: min number of threads to have readily available in the thread pool
 + * @min: max number of threads the thread pool can contain
 + */
-+void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
++static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
-+                                        int64_t max, Error **errp);
++                                     const char *file_name)
- #endif
++{
-diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
++    char *buf, *buf_origin;
-index XXXXXXX..XXXXXXX 100644
++    FILE *f = fopen(file_name, "r");
---- a/include/block/thread-pool.h
++    int pattern_len;
 +++ b/include/block/thread-pool.h
@@ -XXX,XX +XXX,XX @@
  #include "block/block.h"
 +#define THREAD_POOL_MAX_THREADS_DEFAULT         64
 +
- typedef int ThreadPoolFunc(void *opaque);
++    if (!f) {
++        perror(file_name);
- typedef struct ThreadPool ThreadPool;
++        return NULL;
-@@ -XXX,XX +XXX,XX @@ BlockAIOCB *thread_pool_submit_aio(ThreadPool *pool,
++    }
  int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
          ThreadPoolFunc *func, void *arg);
  void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
 +void thread_pool_update_params(ThreadPool *pool, struct AioContext *ctx);
  #endif
 diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/sysemu/event-loop-base.h
 +++ b/include/sysemu/event-loop-base.h
@@ -XXX,XX +XXX,XX @@ struct EventLoopBase {
      /* AioContext AIO engine parameters */
      int64_t aio_max_batch;
 +
-+    /* AioContext thread pool parameters */
++    if (qemuio_misalign) {
-+    int64_t thread_pool_min;
++        len += MISALIGN_OFFSET;
-+    int64_t thread_pool_max;
++    }
  };
  #endif
 diff --git a/event-loop-base.c b/event-loop-base.c
 index XXXXXXX..XXXXXXX 100644
 --- a/event-loop-base.c
 +++ b/event-loop-base.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/osdep.h"
  #include "qom/object_interfaces.h"
  #include "qapi/error.h"
 +#include "block/thread-pool.h"
  #include "sysemu/event-loop-base.h"
  typedef struct {
@@ -XXX,XX +XXX,XX @@ typedef struct {
      ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
  } EventLoopBaseParamInfo;
 +static void event_loop_base_instance_init(Object *obj)
 +{
 +    EventLoopBase *base = EVENT_LOOP_BASE(obj);
 +
-+    base->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
++    buf_origin = buf = blk_blockalign(blk, len);
 +
 +    if (qemuio_misalign) {
 +        buf_origin += MISALIGN_OFFSET;
 +        buf += MISALIGN_OFFSET;
 +        len -= MISALIGN_OFFSET;
 +    }
 +
 +    pattern_len = fread(buf_origin, 1, len, f);
 +
 +    if (ferror(f)) {
 +        perror(file_name);
 +        goto error;
 +    }
 +
 +    if (pattern_len == 0) {
 +        fprintf(stderr, "%s: file is empty\n", file_name);
 +        goto error;
 +    }
 +
 +    fclose(f);
 +
 +    if (len > pattern_len) {
 +        len -= pattern_len;
 +        buf += pattern_len;
 +
 +        while (len > 0) {
 +            size_t len_to_copy = MIN(pattern_len, len);
 +
 +            memcpy(buf, buf_origin, len_to_copy);
 +
 +            len -= len_to_copy;
 +            buf += len_to_copy;
 +        }
 +    }
 +
 +    return buf_origin;
 +
 +error:
 +    qemu_io_free(buf_origin);
 +    return NULL;
 +}
 +
- static EventLoopBaseParamInfo aio_max_batch_info = {
+ static void dump_buffer(const void *buffer, int64_t offset, int64_t len)
-     "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
+ {
      uint64_t i;
@@ -XXX,XX +XXX,XX @@ static void write_help(void)
  " -n, -- with -z, don't allow slow fallback\n"
  " -p, -- ignored for backwards compatibility\n"
  " -P, -- use different pattern to fill file\n"
 +" -s, -- use a pattern file to fill the write buffer\n"
  " -C, -- report statistics in a machine parsable format\n"
  " -q, -- quiet mode, do not show I/O statistics\n"
  " -u, -- with -z, allow unmapping\n"
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t write_cmd = {
      .perm       = BLK_PERM_WRITE,
      .argmin     = 2,
      .argmax     = -1,
 -    .args       = "[-bcCfnquz] [-P pattern] off len",
 +    .args       = "[-bcCfnquz] [-P pattern | -s source_file] off len",
      .oneline    = "writes a number of bytes at a specified offset",
      .help       = write_help,
  };
-+static EventLoopBaseParamInfo thread_pool_min_info = {
+@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
 +    "thread-pool-min", offsetof(EventLoopBase, thread_pool_min),
 +};
 +static EventLoopBaseParamInfo thread_pool_max_info = {
 +    "thread-pool-max", offsetof(EventLoopBase, thread_pool_max),
 +};
  static void event_loop_base_get_param(Object *obj, Visitor *v,
          const char *name, void *opaque, Error **errp)
@@ -XXX,XX +XXX,XX @@ static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
                                event_loop_base_get_param,
                                event_loop_base_set_param,
                                NULL, &aio_max_batch_info);
 +    object_class_property_add(klass, "thread-pool-min", "int",
 +                              event_loop_base_get_param,
 +                              event_loop_base_set_param,
 +                              NULL, &thread_pool_min_info);
 +    object_class_property_add(klass, "thread-pool-max", "int",
 +                              event_loop_base_get_param,
 +                              event_loop_base_set_param,
 +                              NULL, &thread_pool_max_info);
  }
  static const TypeInfo event_loop_base_info = {
      .name = TYPE_EVENT_LOOP_BASE,
      .parent = TYPE_OBJECT,
      .instance_size = sizeof(EventLoopBase),
 +    .instance_init = event_loop_base_instance_init,
      .class_size = sizeof(EventLoopBaseClass),
      .class_init = event_loop_base_class_init,
      .abstract = true,
 diff --git a/iothread.c b/iothread.c
 index XXXXXXX..XXXXXXX 100644
 --- a/iothread.c
 +++ b/iothread.c
@@ -XXX,XX +XXX,XX @@ static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
      aio_context_set_aio_params(iothread->ctx,
                                 iothread->parent_obj.aio_max_batch,
                                 errp);
 +
 +    aio_context_set_thread_pool_params(iothread->ctx, base->thread_pool_min,
 +                                       base->thread_pool_max, errp);
  }
 diff --git a/util/aio-posix.c b/util/aio-posix.c
 index XXXXXXX..XXXXXXX 100644
 --- a/util/aio-posix.c
 +++ b/util/aio-posix.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/osdep.h"
  #include "block/block.h"
 +#include "block/thread-pool.h"
  #include "qemu/main-loop.h"
  #include "qemu/rcu.h"
  #include "qemu/rcu_queue.h"
 diff --git a/util/async.c b/util/async.c
 index XXXXXXX..XXXXXXX 100644
 --- a/util/async.c
 +++ b/util/async.c
@@ -XXX,XX +XXX,XX @@ AioContext *aio_context_new(Error **errp)
      ctx->aio_max_batch = 0;
 +    ctx->thread_pool_min = 0;
 +    ctx->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
 +
      return ctx;
  fail:
      g_source_destroy(&ctx->source);
@@ -XXX,XX +XXX,XX @@ void qemu_set_current_aio_context(AioContext *ctx)
      assert(!get_my_aiocontext());
      set_my_aiocontext(ctx);
  }
 +
 +void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
 +                                        int64_t max, Error **errp)
 +{
 +
 +    if (min > max || !max || min > INT_MAX || max > INT_MAX) {
 +        error_setg(errp, "bad thread-pool-min/thread-pool-max values");
 +        return;
 +    }
 +
 +    ctx->thread_pool_min = min;
 +    ctx->thread_pool_max = max;
 +
 +    if (ctx->thread_pool) {
 +        thread_pool_update_params(ctx->thread_pool, ctx);
 +    }
 +}
 diff --git a/util/main-loop.c b/util/main-loop.c
 index XXXXXXX..XXXXXXX 100644
 --- a/util/main-loop.c
 +++ b/util/main-loop.c
@@ -XXX,XX +XXX,XX @@
  #include "sysemu/replay.h"
  #include "qemu/main-loop.h"
  #include "block/aio.h"
 +#include "block/thread-pool.h"
  #include "qemu/error-report.h"
  #include "qemu/queue.h"
  #include "qemu/compiler.h"
@@ -XXX,XX +XXX,XX @@ int qemu_init_main_loop(Error **errp)
  static void main_loop_update_params(EventLoopBase *base, Error **errp)
  {
-+    ERRP_GUARD();
+     struct timespec t1, t2;
-+
+     bool Cflag = false, qflag = false, bflag = false;
-     if (!qemu_aio_context) {
+-    bool Pflag = false, zflag = false, cflag = false;
-         error_setg(errp, "qemu aio context not ready");
++    bool Pflag = false, zflag = false, cflag = false, sflag = false;
-         return;
+     int flags = 0;
      int c, cnt, ret;
      char *buf = NULL;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
      /* Some compilers get confused and warn if this is not initialized.  */
      int64_t total = 0;
      int pattern = 0xcd;
 +    const char *file_name = NULL;
 -    while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) {
 +    while ((c = getopt(argc, argv, "bcCfnpP:qs:uz")) != -1) {
          switch (c) {
          case 'b':
              bflag = true;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
          case 'q':
              qflag = true;
              break;
 +        case 's':
 +            sflag = true;
 +            file_name = optarg;
 +            break;
          case 'u':
              flags |= BDRV_REQ_MAY_UNMAP;
              break;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
          return -EINVAL;
      }
-     aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
+-    if (zflag && Pflag) {
-+    if (*errp) {
+-        printf("-z and -P cannot be specified at the same time\n");
-+        return;
++    if (zflag + Pflag + sflag > 1) {
-+    }
++        printf("Only one of -z, -P, and -s "
-+
++               "can be specified at the same time\n");
-+    aio_context_set_thread_pool_params(qemu_aio_context, base->thread_pool_min,
+         return -EINVAL;
-+                                       base->thread_pool_max, errp);
+     }
- }
+@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
- MainLoop *mloop;
+     }
-diff --git a/util/thread-pool.c b/util/thread-pool.c
-index XXXXXXX..XXXXXXX 100644
+     if (!zflag) {
---- a/util/thread-pool.c
+-        buf = qemu_io_alloc(blk, count, pattern);
-+++ b/util/thread-pool.c
++        if (sflag) {
-@@ -XXX,XX +XXX,XX @@ struct ThreadPool {
++            buf = qemu_io_alloc_from_file(blk, count, file_name);
-     QemuMutex lock;
++            if (!buf) {
-     QemuCond worker_stopped;
++                return -EINVAL;
-     QemuSemaphore sem;
++            }
--    int max_threads;
++        } else {
-     QEMUBH *new_thread_bh;
++            buf = qemu_io_alloc(blk, count, pattern);
++        }
-     /* The following variables are only accessed from one AioContext. */
+     }
-@@ -XXX,XX +XXX,XX @@ struct ThreadPool {
-     int new_threads;     /* backlog of threads we need to create */
+     clock_gettime(CLOCK_MONOTONIC, &t1);
      int pending_threads; /* threads created but not running yet */
      bool stopping;
 +    int min_threads;
 +    int max_threads;
  };
 +static inline bool back_to_sleep(ThreadPool *pool, int ret)
 +{
 +    /*
 +     * The semaphore timed out, we should exit the loop except when:
 +     *  - There is work to do, we raced with the signal.
 +     *  - The max threads threshold just changed, we raced with the signal.
 +     *  - The thread pool forces a minimum number of readily available threads.
 +     */
 +    if (ret == -1 && (!QTAILQ_EMPTY(&pool->request_list) ||
 +            pool->cur_threads > pool->max_threads ||
 +            pool->cur_threads <= pool->min_threads)) {
 +            return true;
 +    }
 +
 +    return false;
 +}
 +
  static void *worker_thread(void *opaque)
  {
      ThreadPool *pool = opaque;
@@ -XXX,XX +XXX,XX @@ static void *worker_thread(void *opaque)
              ret = qemu_sem_timedwait(&pool->sem, 10000);
              qemu_mutex_lock(&pool->lock);
              pool->idle_threads--;
 -        } while (ret == -1 && !QTAILQ_EMPTY(&pool->request_list));
 -        if (ret == -1 || pool->stopping) {
 +        } while (back_to_sleep(pool, ret));
 +        if (ret == -1 || pool->stopping ||
 +            pool->cur_threads > pool->max_threads) {
              break;
          }
@@ -XXX,XX +XXX,XX @@ void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg)
      thread_pool_submit_aio(pool, func, arg, NULL, NULL);
  }
 +void thread_pool_update_params(ThreadPool *pool, AioContext *ctx)
 +{
 +    qemu_mutex_lock(&pool->lock);
 +
 +    pool->min_threads = ctx->thread_pool_min;
 +    pool->max_threads = ctx->thread_pool_max;
 +
 +    /*
 +     * We either have to:
 +     *  - Increase the number available of threads until over the min_threads
 +     *    threshold.
 +     *  - Decrease the number of available threads until under the max_threads
 +     *    threshold.
 +     *  - Do nothing. The current number of threads fall in between the min and
 +     *    max thresholds. We'll let the pool manage itself.
 +     */
 +    for (int i = pool->cur_threads; i < pool->min_threads; i++) {
 +        spawn_thread(pool);
 +    }
 +
 +    for (int i = pool->cur_threads; i > pool->max_threads; i--) {
 +        qemu_sem_post(&pool->sem);
 +    }
 +
 +    qemu_mutex_unlock(&pool->lock);
 +}
 +
  static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
  {
      if (!ctx) {
@@ -XXX,XX +XXX,XX @@ static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
      qemu_mutex_init(&pool->lock);
      qemu_cond_init(&pool->worker_stopped);
      qemu_sem_init(&pool->sem, 0);
 -    pool->max_threads = 64;
      pool->new_thread_bh = aio_bh_new(ctx, spawn_thread_bh_fn, pool);
      QLIST_INIT(&pool->head);
      QTAILQ_INIT(&pool->request_list);
 +
 +    thread_pool_update_params(pool, ctx);
  }
  ThreadPool *thread_pool_new(AioContext *ctx)
 --
-.35.1
+.21.0

-New patch
+[Qemu-devel] [PULL v2 02/16] block: fix permission update in bdrv_replace_node
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+It's wrong to OR shared permissions. It may lead to crash on further
+permission updates.
+Also, no needs to consider previously calculated permissions, as at
+this point we already bind all new parents and bdrv_get_cumulative_perm
+result is enough. So fix the bug by just set permissions by
+bdrv_get_cumulative_perm result.
+Bug was introduced in long ago 234ac1a9025, in 2.9.
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Message-id: 20190824100740.61635-1-vsementsov@virtuozzo.com
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ block.c | 5 ++---
+file changed, 2 insertions(+), 3 deletions(-)
+diff --git a/block.c b/block.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block.c
++++ b/block.c
+@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
+ {
+     BdrvChild *c, *next;
+     GSList *list = NULL, *p;
+-    uint64_t old_perm, old_shared;
+     uint64_t perm = 0, shared = BLK_PERM_ALL;
+     int ret;
+@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
+         bdrv_unref(from);
+     }
+-    bdrv_get_cumulative_perm(to, &old_perm, &old_shared);
+-    bdrv_set_perm(to, old_perm | perm, old_shared | shared);
++    bdrv_get_cumulative_perm(to, &perm, &shared);
++    bdrv_set_perm(to, perm, shared);
+ out:
+     g_slist_free(list);
+--
+.21.0

-[PULL 1/3] Introduce event-loop-base abstract class
+[Qemu-devel] [PULL v2 03/16] block: posix: Always allocate the first block
-From: Nicolas Saenz Julienne <nsaenzju@redhat.com>
+From: Nir Soffer <nirsof@gmail.com>
-Introduce the 'event-loop-base' abstract class, it'll hold the
+When creating an image with preallocation "off" or "falloc", the first
-properties common to all event loops and provide the necessary hooks for
+block of the image is typically not allocated. When using Gluster
-their creation and maintenance. Then have iothread inherit from it.
+storage backed by XFS filesystem, reading this block using direct I/O
+succeeds regardless of request length, fooling alignment detection.
-EventLoopBaseClass is defined as user creatable and provides a hook for
-its children to attach themselves to the user creatable class 'complete'
+In this case we fallback to a safe value (4096) instead of the optimal
-function. It also provides an update_params() callback to propagate
+value (512), which may lead to unneeded data copying when aligning
-property changes onto its children.
+requests.  Allocating the first block avoids the fallback.
-The new 'event-loop-base' class will live in the root directory. It is
+Since we allocate the first block even with preallocation=off, we no
-built on its own using the 'link_whole' option (there are no direct
+longer create images with zero disk size:
-function dependencies between the class and its children, it all happens
-trough 'constructor' magic). And also imposes new compilation
+    $ ./qemu-img create -f raw test.raw 1g
-dependencies:
+    Formatting 'test.raw', fmt=raw size=1073741824
-    qom <- event-loop-base <- blockdev (iothread.c)
+    $ ls -lhs test.raw
+.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw
-And in subsequent patches:
+And converting the image requires additional cluster:
-    qom <- event-loop-base <- qemuutil (util/main-loop.c)
+    $ ./qemu-img measure -f raw -O qcow2 test.raw
-All this forced some amount of reordering in meson.build:
+    required size: 458752
+    fully allocated size: 1074135040
- - Moved qom build definition before qemuutil. Doing it the other way
-   around (i.e. moving qemuutil after qom) isn't possible as a lot of
+When using format like vmdk with multiple files per image, we allocate
-   core libraries that live in between the two depend on it.
+one block per file:
- - Process the 'hw' subdir earlier, as it introduces files into the
+    $ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
-   'qom' source set.
+    Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat
-No functional changes intended.
+    $ ls -lhs test*.vmdk
+.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
-Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
+.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+.0K -rw-r--r--. 1 nsoffer nsoffer  353 Aug 27 03:23 test.vmdk
-Acked-by: Markus Armbruster <armbru@redhat.com>
-Message-id: 20220425075723.20019-2-nsaenzju@redhat.com
+I did quick performance test for copying disks with qemu-img convert to
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+new raw target image to Gluster storage with sector size of 512 bytes:
     for i in $(seq 10); do
         rm -f dst.raw
         sleep 10
         time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
     done
 Here is a table comparing the total time spent:
 Type    Before(s)   After(s)    Diff(%)
 ---------------------------------------
 real      530.028    469.123      -11.4
 user       17.204     10.768      -37.4
 sys        17.881      7.011      -60.7
 We can see very clear improvement in CPU usage.
 Signed-off-by: Nir Soffer <nsoffer@redhat.com>
 Message-id: 20190827010528.8818-2-nsoffer@redhat.com
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- qapi/qom.json                    |  22 +++++--
+ block/file-posix.c                            | 51 +++++++++++++++++++
- meson.build                      |  23 ++++---
+ tests/qemu-iotests/059.out                    |  2 +-
- include/sysemu/event-loop-base.h |  36 +++++++++++
+ tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
- include/sysemu/iothread.h        |   6 +-
+ tests/qemu-iotests/150.out.raw                | 12 +++++
- event-loop-base.c                | 104 +++++++++++++++++++++++++++++++
+ tests/qemu-iotests/175                        | 19 ++++---
- iothread.c                       |  65 ++++++-------------
+ tests/qemu-iotests/175.out                    |  8 +--
-files changed, 192 insertions(+), 64 deletions(-)
+ tests/qemu-iotests/178.out.qcow2              |  4 +-
- create mode 100644 include/sysemu/event-loop-base.h
+ tests/qemu-iotests/221.out                    | 12 +++--
- create mode 100644 event-loop-base.c
+ tests/qemu-iotests/253.out                    | 12 +++--
+files changed, 99 insertions(+), 21 deletions(-)
-diff --git a/qapi/qom.json b/qapi/qom.json
+ rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
-index XXXXXXX..XXXXXXX 100644
+ create mode 100644 tests/qemu-iotests/150.out.raw
---- a/qapi/qom.json
-+++ b/qapi/qom.json
+diff --git a/block/file-posix.c b/block/file-posix.c
-@@ -XXX,XX +XXX,XX @@
+index XXXXXXX..XXXXXXX 100644
-             '*repeat': 'bool',
+--- a/block/file-posix.c
-             '*grab-toggle': 'GrabToggleKeys' } }
++++ b/block/file-posix.c
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_discard(void *opaque)
-+##
+     return ret;
-+# @EventLoopBaseProperties:
+ }
-+#
-+# Common properties for event loops
++/*
-+#
++ * Help alignment probing by allocating the first block.
-+# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
++ *
-+#                 0 means that the engine will use its default.
++ * When reading with direct I/O from unallocated area on Gluster backed by XFS,
-+#                 (default: 0)
++ * reading succeeds regardless of request length. In this case we fallback to
-+#
++ * safe alignment which is not optimal. Allocating the first block avoids this
-+# Since: 7.1
++ * fallback.
-+##
++ *
-+{ 'struct': 'EventLoopBaseProperties',
++ * fd may be opened with O_DIRECT, but we don't know the buffer alignment or
-+  'data': { '*aio-max-batch': 'int' } }
++ * request alignment, so we use safe values.
-+
++ *
- ##
++ * Returns: 0 on success, -errno on failure. Since this is an optimization,
- # @IothreadProperties:
++ * caller may ignore failures.
- #
++ */
-@@ -XXX,XX +XXX,XX @@
++static int allocate_first_block(int fd, size_t max_size)
- #               algorithm detects it is spending too long polling without
++{
- #               encountering events. 0 selects a default behaviour (default: 0)
++    size_t write_size = (max_size < MAX_BLOCKSIZE)
- #
++        ? BDRV_SECTOR_SIZE
--# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
++        : MAX_BLOCKSIZE;
--#                 0 means that the engine will use its default
++    size_t max_align = MAX(MAX_BLOCKSIZE, getpagesize());
--#                 (default:0, since 6.1)
++    void *buf;
-+# The @aio-max-batch option is available since 6.1.
++    ssize_t n;
- #
++    int ret;
- # Since: 2.0
++
- ##
++    buf = qemu_memalign(max_align, write_size);
- { 'struct': 'IothreadProperties',
++    memset(buf, 0, write_size);
-+  'base': 'EventLoopBaseProperties',
++
-   'data': { '*poll-max-ns': 'int',
++    do {
-             '*poll-grow': 'int',
++        n = pwrite(fd, buf, write_size, 0);
--            '*poll-shrink': 'int',
++    } while (n == -1 && errno == EINTR);
--            '*aio-max-batch': 'int' } }
++
-+            '*poll-shrink': 'int' } }
++    ret = (n == -1) ? -errno : 0;
++
- ##
++    qemu_vfree(buf);
- # @MemoryBackendProperties:
++    return ret;
-diff --git a/meson.build b/meson.build
++}
-index XXXXXXX..XXXXXXX 100644
++
---- a/meson.build
+ static int handle_aiocb_truncate(void *opaque)
-+++ b/meson.build
+ {
-@@ -XXX,XX +XXX,XX @@ subdir('qom')
+     RawPosixAIOData *aiocb = opaque;
- subdir('authz')
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
- subdir('crypto')
+                 /* posix_fallocate() doesn't set errno. */
- subdir('ui')
+                 error_setg_errno(errp, -result,
-+subdir('hw')
+                                  "Could not preallocate new data");
++            } else if (current_length == 0) {
++                /*
- if enable_modules
++                 * posix_fallocate() uses fallocate() if the filesystem
-@@ -XXX,XX +XXX,XX @@ if enable_modules
++                 * supports it, or fallback to manually writing zeroes. If
-   modulecommon = declare_dependency(link_whole: libmodulecommon, compile_args: '-DBUILD_DSO')
++                 * fallocate() was used, unaligned reads from the fallocated
- endif
++                 * area in raw_probe_alignment() will succeed, hence we need to
++                 * allocate the first block.
-+qom_ss = qom_ss.apply(config_host, strict: false)
++                 *
-+libqom = static_library('qom', qom_ss.sources() + genh,
++                 * Optimize future alignment probing; ignore failures.
-+                        dependencies: [qom_ss.dependencies()],
++                 */
-+                        name_suffix: 'fa')
++                allocate_first_block(fd, offset);
-+qom = declare_dependency(link_whole: libqom)
+             }
-+
+         } else {
-+event_loop_base = files('event-loop-base.c')
+             result = 0;
-+event_loop_base = static_library('event-loop-base', sources: event_loop_base + genh,
+@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
-+                                 build_by_default: true)
+         if (ftruncate(fd, offset) != 0) {
-+event_loop_base = declare_dependency(link_whole: event_loop_base,
+             result = -errno;
-+                                     dependencies: [qom])
+             error_setg_errno(errp, -result, "Could not resize file");
-+
++        } else if (current_length == 0 && offset > current_length) {
- stub_ss = stub_ss.apply(config_all, strict: false)
++            /* Optimize future alignment probing; ignore failures. */
++            allocate_first_block(fd, offset);
- util_ss.add_all(trace_ss)
+         }
-@@ -XXX,XX +XXX,XX @@ subdir('monitor')
+         return result;
- subdir('net')
+     default:
- subdir('replay')
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
- subdir('semihosting')
+index XXXXXXX..XXXXXXX 100644
--subdir('hw')
+--- a/tests/qemu-iotests/059.out
- subdir('tcg')
++++ b/tests/qemu-iotests/059.out
- subdir('fpu')
+@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMax
- subdir('accel')
+ image: TEST_DIR/t.vmdk
-@@ -XXX,XX +XXX,XX @@ qemu_syms = custom_target('qemu.syms', output: 'qemu.syms',
+ file format: vmdk
-                              capture: true,
+ virtual size: 0.977 TiB (1073741824000 bytes)
-                              command: [undefsym, nm, '@INPUT@'])
+-disk size: 16 KiB
++disk size: 1.97 MiB
--qom_ss = qom_ss.apply(config_host, strict: false)
+ Format specific information:
--libqom = static_library('qom', qom_ss.sources() + genh,
+     cid: XXXXXXXX
--                        dependencies: [qom_ss.dependencies()],
+     parent cid: XXXXXXXX
--                        name_suffix: 'fa')
+diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out.qcow2
--
+similarity index 100%
--qom = declare_dependency(link_whole: libqom)
+rename from tests/qemu-iotests/150.out
--
+rename to tests/qemu-iotests/150.out.qcow2
- authz_ss = authz_ss.apply(config_host, strict: false)
+diff --git a/tests/qemu-iotests/150.out.raw b/tests/qemu-iotests/150.out.raw
  libauthz = static_library('authz', authz_ss.sources() + genh,
                            dependencies: [authz_ss.dependencies()],
@@ -XXX,XX +XXX,XX @@ libblockdev = static_library('blockdev', blockdev_ss.sources() + genh,
                               build_by_default: false)
  blockdev = declare_dependency(link_whole: [libblockdev],
 -                              dependencies: [block])
 +                              dependencies: [block, event_loop_base])
  qmp_ss = qmp_ss.apply(config_host, strict: false)
  libqmp = static_library('qmp', qmp_ss.sources() + genh,
 diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
-+++ b/include/sysemu/event-loop-base.h
++++ b/tests/qemu-iotests/150.out.raw
 @@ -XXX,XX +XXX,XX @@
-+/*
++QA output created by 150
-+ * QEMU event-loop backend
++
-+ *
++=== Mapping sparse conversion ===
-+ * Copyright (C) 2022 Red Hat Inc
++
-+ *
++Offset          Length          File
-+ * Authors:
++0               0x1000          TEST_DIR/t.IMGFMT
-+ *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
++
-+ *
++=== Mapping non-sparse conversion ===
-+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
++
-+ * See the COPYING file in the top-level directory.
++Offset          Length          File
-+ */
++0               0x100000        TEST_DIR/t.IMGFMT
-+#ifndef QEMU_EVENT_LOOP_BASE_H
++*** done
-+#define QEMU_EVENT_LOOP_BASE_H
+diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
-+
+index XXXXXXX..XXXXXXX 100755
-+#include "qom/object.h"
+--- a/tests/qemu-iotests/175
-+#include "block/aio.h"
++++ b/tests/qemu-iotests/175
-+#include "qemu/typedefs.h"
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
-+
+ # the file size.  This function hides the resulting difference in the
-+#define TYPE_EVENT_LOOP_BASE         "event-loop-base"
+ # stat -c '%b' output.
-+OBJECT_DECLARE_TYPE(EventLoopBase, EventLoopBaseClass,
+ # Parameter 1: Number of blocks an empty file occupies
-+                    EVENT_LOOP_BASE)
+-# Parameter 2: Image size in bytes
-+
++# Parameter 2: Minimal number of blocks in an image
-+struct EventLoopBaseClass {
++# Parameter 3: Image size in bytes
-+    ObjectClass parent_class;
+ _filter_blocks()
-+
+ {
-+    void (*init)(EventLoopBase *base, Error **errp);
+     extra_blocks=$1
-+    void (*update_params)(EventLoopBase *base, Error **errp);
+-    img_size=$2
-+};
++    min_blocks=$2
-+
++    img_size=$3
-+struct EventLoopBase {
-+    Object parent;
+-    sed -e "s/blocks=$extra_blocks\\(\$\\|[^0-9]\\)/nothing allocated/" \
-+
+-        -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/everything allocated/"
-+    /* AioContext AIO engine parameters */
++    sed -e "s/blocks=$min_blocks\\(\$\\|[^0-9]\\)/min allocation/" \
-+    int64_t aio_max_batch;
++        -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/max allocation/"
 +};
 +#endif
 diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/sysemu/iothread.h
 +++ b/include/sysemu/iothread.h
@@ -XXX,XX +XXX,XX @@
  #include "block/aio.h"
  #include "qemu/thread.h"
  #include "qom/object.h"
 +#include "sysemu/event-loop-base.h"
  #define TYPE_IOTHREAD "iothread"
  struct IOThread {
 -    Object parent_obj;
 +    EventLoopBase parent_obj;
      QemuThread thread;
      AioContext *ctx;
@@ -XXX,XX +XXX,XX @@ struct IOThread {
      int64_t poll_max_ns;
      int64_t poll_grow;
      int64_t poll_shrink;
 -
 -    /* AioContext AIO engine parameters */
 -    int64_t aio_max_batch;
  };
  typedef struct IOThread IOThread;
 diff --git a/event-loop-base.c b/event-loop-base.c
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/event-loop-base.c
@@ -XXX,XX +XXX,XX @@
 +/*
 + * QEMU event-loop base
 + *
 + * Copyright (C) 2022 Red Hat Inc
 + *
 + * Authors:
 + *  Stefan Hajnoczi <stefanha@redhat.com>
 + *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2 or later.
 + * See the COPYING file in the top-level directory.
 + */
 +
 +#include "qemu/osdep.h"
 +#include "qom/object_interfaces.h"
 +#include "qapi/error.h"
 +#include "sysemu/event-loop-base.h"
 +
 +typedef struct {
 +    const char *name;
 +    ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
 +} EventLoopBaseParamInfo;
 +
 +static EventLoopBaseParamInfo aio_max_batch_info = {
 +    "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
 +};
 +
 +static void event_loop_base_get_param(Object *obj, Visitor *v,
 +        const char *name, void *opaque, Error **errp)
 +{
 +    EventLoopBase *event_loop_base = EVENT_LOOP_BASE(obj);
 +    EventLoopBaseParamInfo *info = opaque;
 +    int64_t *field = (void *)event_loop_base + info->offset;
 +
 +    visit_type_int64(v, name, field, errp);
 +}
 +
 +static void event_loop_base_set_param(Object *obj, Visitor *v,
 +        const char *name, void *opaque, Error **errp)
 +{
 +    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(obj);
 +    EventLoopBase *base = EVENT_LOOP_BASE(obj);
 +    EventLoopBaseParamInfo *info = opaque;
 +    int64_t *field = (void *)base + info->offset;
 +    int64_t value;
 +
 +    if (!visit_type_int64(v, name, &value, errp)) {
 +        return;
 +    }
 +
 +    if (value < 0) {
 +        error_setg(errp, "%s value must be in range [0, %" PRId64 "]",
 +                   info->name, INT64_MAX);
 +        return;
 +    }
 +
 +    *field = value;
 +
 +    if (bc->update_params) {
 +        bc->update_params(base, errp);
 +    }
 +
 +    return;
 +}
 +
 +static void event_loop_base_complete(UserCreatable *uc, Error **errp)
 +{
 +    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
 +    EventLoopBase *base = EVENT_LOOP_BASE(uc);
 +
 +    if (bc->init) {
 +        bc->init(base, errp);
 +    }
 +}
 +
 +static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
 +{
 +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
 +    ucc->complete = event_loop_base_complete;
 +
 +    object_class_property_add(klass, "aio-max-batch", "int",
 +                              event_loop_base_get_param,
 +                              event_loop_base_set_param,
 +                              NULL, &aio_max_batch_info);
 +}
 +
 +static const TypeInfo event_loop_base_info = {
 +    .name = TYPE_EVENT_LOOP_BASE,
 +    .parent = TYPE_OBJECT,
 +    .instance_size = sizeof(EventLoopBase),
 +    .class_size = sizeof(EventLoopBaseClass),
 +    .class_init = event_loop_base_class_init,
 +    .abstract = true,
 +    .interfaces = (InterfaceInfo[]) {
 +        { TYPE_USER_CREATABLE },
 +        { }
 +    }
 +};
 +
 +static void register_types(void)
 +{
 +    type_register_static(&event_loop_base_info);
 +}
 +type_init(register_types);
 diff --git a/iothread.c b/iothread.c
 index XXXXXXX..XXXXXXX 100644
 --- a/iothread.c
 +++ b/iothread.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/module.h"
  #include "block/aio.h"
  #include "block/block.h"
 +#include "sysemu/event-loop-base.h"
  #include "sysemu/iothread.h"
  #include "qapi/error.h"
  #include "qapi/qapi-commands-misc.h"
@@ -XXX,XX +XXX,XX @@ static void iothread_init_gcontext(IOThread *iothread)
      iothread->main_loop = g_main_loop_new(iothread->worker_context, TRUE);
  }
--static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
+ # get standard environment, filters and checks
-+static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
+@@ -XXX,XX +XXX,XX @@ size=$((1 * 1024 * 1024))
  touch "$TEST_DIR/empty"
  extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 +# We always write the first byte; check how many blocks this filesystem
 +# allocates to match empty image alloation.
 +printf "\0" > "$TEST_DIR/empty"
 +min_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 +
  echo
  echo "== creating image with default preallocation =="
  _make_test_img $size | _filter_imgfmt
 -stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
 +stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
  for mode in off full falloc; do
      echo
      echo "== creating image with preallocation $mode =="
      IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
 -    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
 +    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
  done
  # success, all done
 diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/175.out
 +++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ QA output created by 175
  == creating image with default preallocation ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
 -size=1048576, nothing allocated
 +size=1048576, min allocation
  == creating image with preallocation off ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
 -size=1048576, nothing allocated
 +size=1048576, min allocation
  == creating image with preallocation full ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
 -size=1048576, everything allocated
 +size=1048576, max allocation
  == creating image with preallocation falloc ==
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
 -size=1048576, everything allocated
 +size=1048576, max allocation
   *** done
 diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/178.out.qcow2
 +++ b/tests/qemu-iotests/178.out.qcow2
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
  == raw input image with data (human) ==
  Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
 -required size: 393216
 +required size: 458752
  fully allocated size: 1074135040
  wrote 512/512 bytes at offset 512
 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
  Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
  {
-+    IOThread *iothread = IOTHREAD(base);
+-    "required": 393216,
-     ERRP_GUARD();
++    "required": 458752,
+     "fully-allocated": 1074135040
 +    if (!iothread->ctx) {
 +        return;
 +    }
 +
      aio_context_set_poll_params(iothread->ctx,
                                  iothread->poll_max_ns,
                                  iothread->poll_grow,
@@ -XXX,XX +XXX,XX @@ static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
      }
      aio_context_set_aio_params(iothread->ctx,
 -                               iothread->aio_max_batch,
 +                               iothread->parent_obj.aio_max_batch,
                                 errp);
  }
+ wrote 512/512 bytes at offset 512
--static void iothread_complete(UserCreatable *obj, Error **errp)
+diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out
-+
+index XXXXXXX..XXXXXXX 100644
-+static void iothread_init(EventLoopBase *base, Error **errp)
+--- a/tests/qemu-iotests/221.out
- {
++++ b/tests/qemu-iotests/221.out
-     Error *local_error = NULL;
+@@ -XXX,XX +XXX,XX @@ QA output created by 221
--    IOThread *iothread = IOTHREAD(obj);
+ === Check mapping of unaligned raw image ===
-+    IOThread *iothread = IOTHREAD(base);
-     char *thread_name;
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537
+-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-     iothread->stopping = false;
+-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-@@ -XXX,XX +XXX,XX @@ static void iothread_complete(UserCreatable *obj, Error **errp)
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
-      */
++{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-     iothread_init_gcontext(iothread);
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
++{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
--    iothread_set_aio_context_params(iothread, &local_error);
+ wrote 1/1 bytes at offset 65536
-+    iothread_set_aio_context_params(base, &local_error);
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-     if (local_error) {
+-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
-         error_propagate(errp, local_error);
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
-         aio_context_unref(iothread->ctx);
++{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
-@@ -XXX,XX +XXX,XX @@ static void iothread_complete(UserCreatable *obj, Error **errp)
+ { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
-      * to inherit.
+ { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-      */
+-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
-     thread_name = g_strdup_printf("IO %s",
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
--                        object_get_canonical_path_component(OBJECT(obj)));
++{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
-+                        object_get_canonical_path_component(OBJECT(base)));
+ { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
-     qemu_thread_create(&iothread->thread, thread_name, iothread_run,
+ { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-                        iothread, QEMU_THREAD_JOINABLE);
+ *** done
-     g_free(thread_name);
+diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out
-@@ -XXX,XX +XXX,XX @@ static IOThreadParamInfo poll_grow_info = {
+index XXXXXXX..XXXXXXX 100644
- static IOThreadParamInfo poll_shrink_info = {
+--- a/tests/qemu-iotests/253.out
-     "poll-shrink", offsetof(IOThread, poll_shrink),
++++ b/tests/qemu-iotests/253.out
- };
+@@ -XXX,XX +XXX,XX @@ QA output created by 253
--static IOThreadParamInfo aio_max_batch_info = {
+ === Check mapping of unaligned raw image ===
--    "aio-max-batch", offsetof(IOThread, aio_max_batch),
--};
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048575
+-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
- static void iothread_get_param(Object *obj, Visitor *v,
+-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-         const char *name, IOThreadParamInfo *info, Error **errp)
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
-@@ -XXX,XX +XXX,XX @@ static void iothread_set_poll_param(Object *obj, Visitor *v,
++{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-     }
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
- }
++{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+ wrote 65535/65535 bytes at offset 983040
--static void iothread_get_aio_param(Object *obj, Visitor *v,
+.999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
--        const char *name, void *opaque, Error **errp)
+-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
--{
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
--    IOThreadParamInfo *info = opaque;
++{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
--
+ { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
--    iothread_get_param(obj, v, name, info, errp);
+-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
--}
++[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
--
++{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
--static void iothread_set_aio_param(Object *obj, Visitor *v,
+ { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
--        const char *name, void *opaque, Error **errp)
+ *** done
 -{
 -    IOThread *iothread = IOTHREAD(obj);
 -    IOThreadParamInfo *info = opaque;
 -
 -    if (!iothread_set_param(obj, v, name, info, errp)) {
 -        return;
 -    }
 -
 -    if (iothread->ctx) {
 -        aio_context_set_aio_params(iothread->ctx,
 -                                   iothread->aio_max_batch,
 -                                   errp);
 -    }
 -}
 -
  static void iothread_class_init(ObjectClass *klass, void *class_data)
  {
 -    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
 -    ucc->complete = iothread_complete;
 +    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(klass);
 +
 +    bc->init = iothread_init;
 +    bc->update_params = iothread_set_aio_context_params;
      object_class_property_add(klass, "poll-max-ns", "int",
                                iothread_get_poll_param,
@@ -XXX,XX +XXX,XX @@ static void iothread_class_init(ObjectClass *klass, void *class_data)
                                iothread_get_poll_param,
                                iothread_set_poll_param,
                                NULL, &poll_shrink_info);
 -    object_class_property_add(klass, "aio-max-batch", "int",
 -                              iothread_get_aio_param,
 -                              iothread_set_aio_param,
 -                              NULL, &aio_max_batch_info);
  }
  static const TypeInfo iothread_info = {
      .name = TYPE_IOTHREAD,
 -    .parent = TYPE_OBJECT,
 +    .parent = TYPE_EVENT_LOOP_BASE,
      .class_init = iothread_class_init,
      .instance_size = sizeof(IOThread),
      .instance_init = iothread_instance_init,
      .instance_finalize = iothread_instance_finalize,
 -    .interfaces = (InterfaceInfo[]) {
 -        {TYPE_USER_CREATABLE},
 -        {}
 -    },
  };
  static void iothread_register_types(void)
@@ -XXX,XX +XXX,XX @@ static int query_one_iothread(Object *object, void *opaque)
      info->poll_max_ns = iothread->poll_max_ns;
      info->poll_grow = iothread->poll_grow;
      info->poll_shrink = iothread->poll_shrink;
 -    info->aio_max_batch = iothread->aio_max_batch;
 +    info->aio_max_batch = iothread->parent_obj.aio_max_batch;
      QAPI_LIST_APPEND(*tail, info);
      return 0;
 --
-.35.1
+.21.0

-New patch
+[Qemu-devel] [PULL v2 04/16] iotests: Test allocate_first_block() with O_DIRECT
+From: Nir Soffer <nirsof@gmail.com>
+Using block_resize we can test allocate_first_block() with file
+descriptor opened with O_DIRECT, ensuring that it works for any size
+larger than 4096 bytes.
+Testing smaller sizes is tricky as the result depends on the filesystem
+used for testing. For example on NFS any size will work since O_DIRECT
+does not require any alignment.
+Signed-off-by: Nir Soffer <nsoffer@redhat.com>
+Reviewed-by: Max Reitz <mreitz@redhat.com>
+Message-id: 20190827010528.8818-3-nsoffer@redhat.com
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/175     | 28 ++++++++++++++++++++++++++++
+ tests/qemu-iotests/175.out |  8 ++++++++
+files changed, 36 insertions(+)
+diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/175
++++ b/tests/qemu-iotests/175
+@@ -XXX,XX +XXX,XX @@ _filter_blocks()
+         -e "s/blocks=$((extra_blocks + img_size / 512))\\(\$\\|[^0-9]\\)/max allocation/"
+ }
++# Resize image using block_resize.
++# Parameter 1: image path
++# Parameter 2: new size
++_block_resize()
++{
++    local path=$1
++    local size=$2
++
++    $QEMU -qmp stdio -nographic -nodefaults \
++        -blockdev file,node-name=file,filename=$path,cache.direct=on \
++        <<EOF
++{'execute': 'qmp_capabilities'}
++{'execute': 'block_resize', 'arguments': {'node-name': 'file', 'size': $size}}
++{'execute': 'quit'}
++EOF
++}
++
+ # get standard environment, filters and checks
+ . ./common.rc
+ . ./common.filter
+@@ -XXX,XX +XXX,XX @@ _supported_fmt raw
+ _supported_proto file
+ _supported_os Linux
++_default_cache_mode none
++_supported_cache_modes none directsync
++
+ size=$((1 * 1024 * 1024))
+ touch "$TEST_DIR/empty"
+@@ -XXX,XX +XXX,XX @@ for mode in off full falloc; do
+     stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
+ done
++for new_size in 4096 1048576; do
++    echo
++    echo "== resize empty image with block_resize =="
++    _make_test_img 0 | _filter_imgfmt
++    _block_resize $TEST_IMG $new_size >/dev/null
++    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $new_size
++done
++
+ # success, all done
+ echo "*** done"
+ rm -f $seq.full
+diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/175.out
++++ b/tests/qemu-iotests/175.out
+@@ -XXX,XX +XXX,XX @@ size=1048576, max allocation
+ == creating image with preallocation falloc ==
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
+ size=1048576, max allocation
++
++== resize empty image with block_resize ==
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
++size=4096, min allocation
++
++== resize empty image with block_resize ==
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
++size=1048576, min allocation
+  *** done
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 05/16] iotests: Fix _filter_img_create()
+fe646693acc changed qemu-img create's output so that it no longer prints
+single quotes around parameter values.  The subformat and adapter_type
+filters in _filter_img_create() have never been adapted to that change.
+Fixes: fe646693acc13ac48b98435d14149ab04dc597bc
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
+Message-id: 20190815153638.4600-2-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/059.out       | 16 ++++++++--------
+ tests/qemu-iotests/common.filter |  4 ++--
+files changed, 10 insertions(+), 10 deletions(-)
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/059.out
++++ b/tests/qemu-iotests/059.out
+@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+ qemu-io: can't open device TEST_DIR/t.vmdk: L1 size too big
+ === Testing monolithicFlat creation and opening ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
+ image: TEST_DIR/t.IMGFMT
+ file format: IMGFMT
+ virtual size: 2 GiB (2147483648 bytes)
+ === Testing monolithicFlat with zeroed_grain ===
+ qemu-img: TEST_DIR/t.IMGFMT: Flat image can't enable zeroed grain
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
+ === Testing big twoGbMaxExtentFlat ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMaxExtentFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000
+ image: TEST_DIR/t.vmdk
+ file format: vmdk
+ virtual size: 0.977 TiB (1073741824000 bytes)
+@@ -XXX,XX +XXX,XX @@ Format specific information:
+ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Invalid extent line: RW 12582912 VMFS "dummy.IMGFMT" 1
+ === Testing truncated sparse ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400 subformat=monolithicSparse
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400
+ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': File truncated, expecting at least 13172736 bytes
+ === Converting to streamOptimized from image with small cluster size===
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing monolithicFlat with internally generated JSON file name ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+ qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
+ === Testing version 3 ===
+@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 64931328
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing 4TB monolithicFlat creation and IO ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104 subformat=monolithicFlat
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104
+ image: TEST_DIR/t.IMGFMT
+ file format: IMGFMT
+ virtual size: 4 TiB (4398046511104 bytes)
+@@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 966367641600
+KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing qemu-img map on extents ===
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=monolithicSparse
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
+ wrote 1024/1024 bytes at offset 65024
+KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ wrote 1024/1024 bytes at offset 2147483136
+@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
+0x20000         0x3f0000        TEST_DIR/t.vmdk
+x7fff0000      0x20000         0x410000        TEST_DIR/t.vmdk
+x140000000     0x10000         0x430000        TEST_DIR/t.vmdk
+-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=twoGbMaxExtentSparse
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
+ wrote 1024/1024 bytes at offset 65024
+KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ wrote 1024/1024 bytes at offset 2147483136
+diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/common.filter
++++ b/tests/qemu-iotests/common.filter
+@@ -XXX,XX +XXX,XX @@ _filter_img_create()
+         -e "s# compat6=\\(on\\|off\\)##g" \
+         -e "s# static=\\(on\\|off\\)##g" \
+         -e "s# zeroed_grain=\\(on\\|off\\)##g" \
+-        -e "s# subformat='[^']*'##g" \
+-        -e "s# adapter_type='[^']*'##g" \
++        -e "s# subformat=[^ ]*##g" \
++        -e "s# adapter_type=[^ ]*##g" \
+         -e "s# hwversion=[^ ]*##g" \
+         -e "s# lazy_refcounts=\\(on\\|off\\)##g" \
+         -e "s# block_size=[0-9]\\+##g" \
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 06/16] vmdk: Use bdrv_dirname() for relative extent paths
+This makes iotest 033 pass with e.g. subformat=monolithicFlat.  It also
+turns a former error in 059 into success.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Message-id: 20190815153638.4600-3-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ block/vmdk.c               | 54 ++++++++++++++++++++++++--------------
+ tests/qemu-iotests/059     |  7 +++--
+ tests/qemu-iotests/059.out |  4 ++-
+files changed, 42 insertions(+), 23 deletions(-)
+diff --git a/block/vmdk.c b/block/vmdk.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/vmdk.c
++++ b/block/vmdk.c
+@@ -XXX,XX +XXX,XX @@ static const char *next_line(const char *s)
+ }
+ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
+-                              const char *desc_file_path, QDict *options,
+-                              Error **errp)
++                              QDict *options, Error **errp)
+ {
+     int ret;
+     int matches;
+@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
+     const char *p, *np;
+     int64_t sectors = 0;
+     int64_t flat_offset;
++    char *desc_file_dir = NULL;
+     char *extent_path;
+     BdrvChild *extent_file;
+     BDRVVmdkState *s = bs->opaque;
+@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
+             continue;
+         }
+-        if (!path_is_absolute(fname) && !path_has_protocol(fname) &&
+-            !desc_file_path[0])
+-        {
+-            bdrv_refresh_filename(bs->file->bs);
+-            error_setg(errp, "Cannot use relative extent paths with VMDK "
+-                       "descriptor file '%s'", bs->file->bs->filename);
+-            return -EINVAL;
+-        }
++        if (path_is_absolute(fname)) {
++            extent_path = g_strdup(fname);
++        } else {
++            if (!desc_file_dir) {
++                desc_file_dir = bdrv_dirname(bs->file->bs, errp);
++                if (!desc_file_dir) {
++                    bdrv_refresh_filename(bs->file->bs);
++                    error_prepend(errp, "Cannot use relative paths with VMDK "
++                                  "descriptor file '%s': ",
++                                  bs->file->bs->filename);
++                    ret = -EINVAL;
++                    goto out;
++                }
++            }
+-        extent_path = path_combine(desc_file_path, fname);
++            extent_path = g_strconcat(desc_file_dir, fname, NULL);
++        }
+         ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
+         assert(ret < 32);
+@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
+         g_free(extent_path);
+         if (local_err) {
+             error_propagate(errp, local_err);
+-            return -EINVAL;
++            ret = -EINVAL;
++            goto out;
+         }
+         /* save to extents array */
+@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
+, 0, 0, 0, 0, &extent, errp);
+             if (ret < 0) {
+                 bdrv_unref_child(bs, extent_file);
+-                return ret;
++                goto out;
+             }
+             extent->flat_start_offset = flat_offset << 9;
+         } else if (!strcmp(type, "SPARSE") || !strcmp(type, "VMFSSPARSE")) {
+@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
+             g_free(buf);
+             if (ret) {
+                 bdrv_unref_child(bs, extent_file);
+-                return ret;
++                goto out;
+             }
+             extent = &s->extents[s->num_extents - 1];
+         } else if (!strcmp(type, "SESPARSE")) {
+             ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
+             if (ret) {
+                 bdrv_unref_child(bs, extent_file);
+-                return ret;
++                goto out;
+             }
+             extent = &s->extents[s->num_extents - 1];
+         } else {
+             error_setg(errp, "Unsupported extent type '%s'", type);
+             bdrv_unref_child(bs, extent_file);
+-            return -ENOTSUP;
++            ret = -ENOTSUP;
++            goto out;
+         }
+         extent->type = g_strdup(type);
+     }
+-    return 0;
++
++    ret = 0;
++    goto out;
+ invalid:
+     np = next_line(p);
+@@ -XXX,XX +XXX,XX @@ invalid:
+         np--;
+     }
+     error_setg(errp, "Invalid extent line: %.*s", (int)(np - p), p);
+-    return -EINVAL;
++    ret = -EINVAL;
++
++out:
++    g_free(desc_file_dir);
++    return ret;
+ }
+ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
+@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
+     }
+     s->create_type = g_strdup(ct);
+     s->desc_offset = 0;
+-    ret = vmdk_parse_extents(buf, bs, bs->file->bs->exact_filename, options,
+-                             errp);
++    ret = vmdk_parse_extents(buf, bs, options, errp);
+ exit:
+     return ret;
+ }
+diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/059
++++ b/tests/qemu-iotests/059
+@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
+ echo
+ echo "=== Testing monolithicFlat with internally generated JSON file name ==="
++# Should work, because bdrv_dirname() works fine with blkdebug
+ IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
+-$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" 2>&1 \
+-    | _filter_testdir | _filter_imgfmt
++$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
++         -c info \
++    2>&1 \
++    | _filter_testdir | _filter_imgfmt | _filter_img_info
+ _cleanup_test_img
+ echo
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/059.out
++++ b/tests/qemu-iotests/059.out
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
+ === Testing monolithicFlat with internally generated JSON file name ===
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+-qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
++format name: IMGFMT
++cluster size: 0 bytes
++vm state offset: 0 bytes
+ === Testing version 3 ===
+ image: TEST_DIR/iotest-version3.IMGFMT
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 07/16] iotests: Keep testing broken relative extent paths
+We had a test for a case where relative extent paths did not work, but
+unfortunately we just fixed the underlying problem, so it works now.
+This patch adds a new test case that still fails.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
+Message-id: 20190815153638.4600-4-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/059     | 27 +++++++++++++++++++++++++++
+ tests/qemu-iotests/059.out |  4 ++++
+files changed, 31 insertions(+)
+diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/059
++++ b/tests/qemu-iotests/059
+@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
+ echo
+ echo "=== Testing monolithicFlat with internally generated JSON file name ==="
++
++echo '--- blkdebug ---'
+ # Should work, because bdrv_dirname() works fine with blkdebug
+ IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
+ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
+@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TE
+     | _filter_testdir | _filter_imgfmt | _filter_img_info
+ _cleanup_test_img
++echo '--- quorum ---'
++# Should not work, because bdrv_dirname() does not work with quorum
++IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
++cp "$TEST_IMG" "$TEST_IMG.orig"
++
++filename="json:{
++    \"driver\": \"$IMGFMT\",
++    \"file\": {
++        \"driver\": \"quorum\",
++        \"children\": [ {
++            \"driver\": \"file\",
++            \"filename\": \"$TEST_IMG\"
++        }, {
++            \"driver\": \"file\",
++            \"filename\": \"$TEST_IMG.orig\"
++        } ],
++        \"vote-threshold\": 1
++    } }"
++
++filename=$(echo "$filename" | tr '\n' ' ' | sed -e 's/\s\+/ /g')
++$QEMU_IMG info "$filename" 2>&1 \
++    | sed -e "s/'json:[^']*'/\$QUORUM_FILE/g" \
++    | _filter_testdir | _filter_imgfmt | _filter_img_info
++
++
+ echo
+ echo "=== Testing version 3 ==="
+ _use_sample_img iotest-version3.vmdk.bz2
+diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/059.out
++++ b/tests/qemu-iotests/059.out
+@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
+bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ === Testing monolithicFlat with internally generated JSON file name ===
++--- blkdebug ---
+ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+ format name: IMGFMT
+ cluster size: 0 bytes
+ vm state offset: 0 bytes
++--- quorum ---
++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
++qemu-img: Could not open $QUORUM_FILE: Cannot use relative paths with VMDK descriptor file $QUORUM_FILE: Cannot generate a base directory for quorum nodes
+ === Testing version 3 ===
+ image: TEST_DIR/iotest-version3.IMGFMT
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 08/16] vmdk: Reject invalid compressed writes
+Compressed writes generally have to write full clusters, not just in
+theory but also in practice when it comes to vmdk's streamOptimized
+subformat.  It currently is just silently broken for writes with
+non-zero in-cluster offsets:
+$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
+$ qemu-io -c 'write 4k 4k' -c 'read 4k 4k' foo.vmdk
+wrote 4096/4096 bytes at offset 4096
+KiB, 1 ops; 00.01 sec (443.724 KiB/sec and 110.9309 ops/sec)
+read failed: Invalid argument
+(The technical reason is that vmdk_write_extent() just writes the
+incomplete compressed data actually to offset 4k.  When reading the
+data, vmdk_read_extent() looks at offset 0 and finds the compressed data
+size to be 0, because that is what it reads from there.  This yields an
+error.)
+For incomplete writes with zero in-cluster offsets, the error path when
+reading the rest of the cluster is a bit different, but the result is
+the same:
+$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
+$ qemu-io -c 'write 0k 4k' -c 'read 4k 4k' foo.vmdk
+wrote 4096/4096 bytes at offset 0
+KiB, 1 ops; 00.01 sec (362.641 KiB/sec and 90.6603 ops/sec)
+read failed: Invalid argument
+(Here, vmdk_read_extent() finds the data and then sees that the
+uncompressed data is short.)
+It is better to reject invalid writes than to make the user believe they
+might have succeeded and then fail when trying to read it back.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
+Message-id: 20190815153638.4600-5-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ block/vmdk.c | 10 ++++++++++
+file changed, 10 insertions(+)
+diff --git a/block/vmdk.c b/block/vmdk.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/vmdk.c
++++ b/block/vmdk.c
+@@ -XXX,XX +XXX,XX @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset,
+     if (extent->compressed) {
+         void *compressed_data;
++        /* Only whole clusters */
++        if (offset_in_cluster ||
++            n_bytes > (extent->cluster_sectors * SECTOR_SIZE) ||
++            (n_bytes < (extent->cluster_sectors * SECTOR_SIZE) &&
++             offset + n_bytes != extent->end_sector * SECTOR_SIZE))
++        {
++            ret = -EINVAL;
++            goto out;
++        }
++
+         if (!extent->has_marker) {
+             ret = -EINVAL;
+             goto out;
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 09/16] iotests: Disable broken streamOptimized tests
+streamOptimized does not support writes that do not span exactly one
+cluster.  Furthermore, it cannot rewrite already allocated clusters.
+As such, many iotests do not work with it.  Disable them.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Message-id: 20190815153638.4600-6-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/002 | 1 +
+ tests/qemu-iotests/003 | 1 +
+ tests/qemu-iotests/005 | 3 ++-
+ tests/qemu-iotests/009 | 1 +
+ tests/qemu-iotests/010 | 1 +
+ tests/qemu-iotests/011 | 1 +
+ tests/qemu-iotests/017 | 3 ++-
+ tests/qemu-iotests/018 | 3 ++-
+ tests/qemu-iotests/019 | 3 ++-
+ tests/qemu-iotests/020 | 3 ++-
+ tests/qemu-iotests/027 | 1 +
+ tests/qemu-iotests/032 | 1 +
+ tests/qemu-iotests/033 | 1 +
+ tests/qemu-iotests/034 | 3 ++-
+ tests/qemu-iotests/037 | 3 ++-
+ tests/qemu-iotests/063 | 3 ++-
+ tests/qemu-iotests/072 | 1 +
+ tests/qemu-iotests/105 | 3 ++-
+ tests/qemu-iotests/197 | 1 +
+ tests/qemu-iotests/215 | 1 +
+ tests/qemu-iotests/251 | 1 +
+files changed, 30 insertions(+), 9 deletions(-)
+diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/002
++++ b/tests/qemu-iotests/002
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ size=128M
+diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/003
++++ b/tests/qemu-iotests/003
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ size=128M
+ offset=67M
+diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/005
++++ b/tests/qemu-iotests/005
+@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
+ _supported_proto generic
+ _supported_os Linux
+ _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
+-                     "subformat=twoGbMaxExtentSparse"
++                     "subformat=twoGbMaxExtentSparse" \
++                     "subformat=streamOptimized"
+ # vpc is limited to 127GB, so we can't test it here
+ if [ "$IMGFMT" = "vpc" ]; then
+diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/009
++++ b/tests/qemu-iotests/009
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ size=6G
+diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/010
++++ b/tests/qemu-iotests/010
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ size=6G
+diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/011
++++ b/tests/qemu-iotests/011
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ size=6G
+diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/017
++++ b/tests/qemu-iotests/017
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow qcow2 vmdk qed
+ _supported_proto generic
+ _unsupported_proto vxhs
+-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
++_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
++                     "subformat=streamOptimized"
+ TEST_OFFSETS="0 4294967296"
+diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/018
++++ b/tests/qemu-iotests/018
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow qcow2 vmdk qed
+ _supported_proto file
+ _supported_os Linux
+-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
++_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
++                     "streamOptimized"
+ TEST_OFFSETS="0 4294967296"
+diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/019
++++ b/tests/qemu-iotests/019
+@@ -XXX,XX +XXX,XX @@ _supported_proto file
+ _supported_os Linux
+ _unsupported_imgopts "subformat=monolithicFlat" \
+                      "subformat=twoGbMaxExtentFlat" \
+-                     "subformat=twoGbMaxExtentSparse"
++                     "subformat=twoGbMaxExtentSparse" \
++                     "subformat=streamOptimized"
+ TEST_OFFSETS="0 4294967296"
+ CLUSTER_SIZE=65536
+diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/020
++++ b/tests/qemu-iotests/020
+@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
+ _supported_proto file
+ _unsupported_imgopts "subformat=monolithicFlat" \
+                      "subformat=twoGbMaxExtentFlat" \
+-                     "subformat=twoGbMaxExtentSparse"
++                     "subformat=twoGbMaxExtentSparse" \
++                     "subformat=streamOptimized"
+ TEST_OFFSETS="0 4294967296"
+diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/027
++++ b/tests/qemu-iotests/027
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt vmdk qcow qcow2 qed
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ size=128M
+diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/032
++++ b/tests/qemu-iotests/032
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ # This works for any image format (though unlikely to segfault for raw)
+ _supported_fmt generic
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ echo
+ echo === Prepare image ===
+diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/033
++++ b/tests/qemu-iotests/033
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto generic
++_unsupported_imgopts "subformat=streamOptimized"
+ size=128M
+diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/034
++++ b/tests/qemu-iotests/034
+@@ -XXX,XX +XXX,XX @@ _supported_proto file
+ _supported_os Linux
+ _unsupported_imgopts "subformat=monolithicFlat" \
+                      "subformat=twoGbMaxExtentFlat" \
+-                     "subformat=twoGbMaxExtentSparse"
++                     "subformat=twoGbMaxExtentSparse" \
++                     "subformat=streamOptimized"
+ CLUSTER_SIZE=4k
+ size=128M
+diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/037
++++ b/tests/qemu-iotests/037
+@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
+ _supported_proto file
+ _unsupported_imgopts "subformat=monolithicFlat" \
+                      "subformat=twoGbMaxExtentFlat" \
+-                     "subformat=twoGbMaxExtentSparse"
++                     "subformat=twoGbMaxExtentSparse" \
++                     "subformat=streamOptimized"
+ CLUSTER_SIZE=4k
+ size=128M
+diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/063
++++ b/tests/qemu-iotests/063
+@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed raw
+ _supported_proto file
+ _unsupported_imgopts "subformat=monolithicFlat" \
+                      "subformat=twoGbMaxExtentFlat" \
+-                     "subformat=twoGbMaxExtentSparse"
++                     "subformat=twoGbMaxExtentSparse" \
++                     "subformat=streamOptimized"
+ _make_test_img 4M
+diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/072
++++ b/tests/qemu-iotests/072
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
+ _supported_proto file
++_unsupported_imgopts "subformat=streamOptimized"
+ IMG_SIZE=64M
+diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/105
++++ b/tests/qemu-iotests/105
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow2 vmdk vhdx qed
+ _supported_proto generic
+ _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
+-                     "subformat=twoGbMaxExtentSparse"
++                     "subformat=twoGbMaxExtentSparse" \
++                     "subformat=streamOptimized"
+ echo
+ echo "creating large image"
+diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/197
++++ b/tests/qemu-iotests/197
+@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
+ _supported_proto generic
+ # LUKS support may be possible, but it complicates things.
+ _unsupported_fmt luks
++_unsupported_imgopts "subformat=streamOptimized"
+ echo
+ echo '=== Copy-on-read ==='
+diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/215
++++ b/tests/qemu-iotests/215
+@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
+ _supported_proto generic
+ # LUKS support may be possible, but it complicates things.
+ _unsupported_fmt luks
++_unsupported_imgopts "subformat=streamOptimized"
+ echo
+ echo '=== Copy-on-read ==='
+diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/251
++++ b/tests/qemu-iotests/251
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt generic
+ _supported_proto file
+ _supported_os Linux
++_unsupported_imgopts "subformat=streamOptimized"
+ if [ "$IMGOPTSSYNTAX" = "true" ]; then
+     # We use json:{} filenames here, so we cannot work with additional options.
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 10/16] iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
+The error message for the test case where we have a quorum node for
+which no directory name can be generated is different: For
+twoGbMaxExtentSparse, it complains that it cannot open the extent file.
+For other (sub)formats, it just notes that it cannot determine the
+backing file path.  Both are fine, but just disable twoGbMaxExtentSparse
+for simplicity's sake.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
+Message-id: 20190815153638.4600-7-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/110 | 3 ++-
+file changed, 2 insertions(+), 1 deletion(-)
+diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/110
++++ b/tests/qemu-iotests/110
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ # Any format supporting backing files
+ _supported_fmt qed qcow qcow2 vmdk
+ _supported_proto file
+-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
++_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
++                     "subformat=twoGbMaxExtentSparse"
+ TEST_IMG_REL=$(basename "$TEST_IMG")
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 11/16] iotests: Disable 126 for flat vmdk subformats
+iotest 126 requires backing file support, which flat vmdks cannot offer.
+Skip this test for such subformats.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Message-id: 20190815153638.4600-8-mreitz@redhat.com
+Reviewed-by: John Snow <jsnow@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/126 | 2 ++
+file changed, 2 insertions(+)
+diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/126
++++ b/tests/qemu-iotests/126
+@@ -XXX,XX +XXX,XX @@ status=1    # failure is the default!
+ # Needs backing file support
+ _supported_fmt qcow qcow2 qed vmdk
++_unsupported_imgopts "subformat=monolithicFlat" \
++                     "subformat=twoGbMaxExtentFlat"
+ # This is the default protocol (and we want to test the difference between
+ # colons which separate a protocol prefix from the rest and colons which are
+ # just part of the filename, so we cannot test protocols which require a prefix)
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 12/16] file-posix: fix request_alignment typo
+From: Stefan Hajnoczi <stefanha@redhat.com>
+Fixes: a6b257a08e3d72219f03e461a52152672fec0612
+       ("file-posix: Handle undetectable alignment")
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-id: 20190827101328.4062-1-stefanha@redhat.com
+Reviewed-by: Thomas Huth <thuth@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ block/file-posix.c | 2 +-
+file changed, 1 insertion(+), 1 deletion(-)
+diff --git a/block/file-posix.c b/block/file-posix.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/file-posix.c
++++ b/block/file-posix.c
+@@ -XXX,XX +XXX,XX @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
+         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
+             align = alignments[i];
+             if (raw_is_io_aligned(fd, buf + align, max_align)) {
+-                /* Fallback to request_aligment. */
++                /* Fallback to request_alignment. */
+                 s->buf_align = (align != 1) ? align : bs->bl.request_alignment;
+                 break;
+             }
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 13/16] iotests: Add -display none to the qemu options
+Without this argument, qemu will print an angry message about not being
+able to connect to a display server if $DISPLAY is not set.  For me,
+that breaks iotests.supported_formats() because it thus only sees
+["Could", "not", "connect"] as the supported formats.
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+Message-id: 20190819201851.24418-2-mreitz@redhat.com
+Reviewed-by: Thomas Huth <thuth@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/check | 6 +++---
+file changed, 3 insertions(+), 3 deletions(-)
+diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/check
++++ b/tests/qemu-iotests/check
+@@ -XXX,XX +XXX,XX @@ export QEMU_PROG="$(type -p "$QEMU_PROG")"
+ case "$QEMU_PROG" in
+     *qemu-system-arm|*qemu-system-aarch64)
+-        export QEMU_OPTIONS="-nodefaults -machine virt,accel=qtest"
++        export QEMU_OPTIONS="-nodefaults -display none -machine virt,accel=qtest"
+         ;;
+     *qemu-system-tricore)
+-        export QEMU_OPTIONS="-nodefaults -machine tricore_testboard,accel=qtest"
++        export QEMU_OPTIONS="-nodefaults -display none -machine tricore_testboard,accel=qtest"
+         ;;
+     *)
+-        export QEMU_OPTIONS="-nodefaults -machine accel=qtest"
++        export QEMU_OPTIONS="-nodefaults -display none -machine accel=qtest"
+         ;;
+ esac
+--
+.21.0

-[PULL 2/3] util/main-loop: Introduce the main loop into QOM
+[Qemu-devel] [PULL v2 14/16] iotests: Check for enabled drivers before testing them
-From: Nicolas Saenz Julienne <nsaenzju@redhat.com>
+From: Thomas Huth <thuth@redhat.com>
-'event-loop-base' provides basic property handling for all 'AioContext'
+It is possible to enable only a subset of the block drivers with the
-based event loops. So let's define a new 'MainLoopClass' that inherits
+"--block-drv-rw-whitelist" option of the "configure" script. All other
-from it. This will permit tweaking the main loop's properties through
+drivers are marked as unusable (or only included as read-only with the
-qapi as well as through the command line using the '-object' keyword[1].
+"--block-drv-ro-whitelist" option). If an iotest is now using such a
-Only one instance of 'MainLoopClass' might be created at any time.
+disabled block driver, it is failing - which is bad, since at least the
 tests in the "auto" group should be able to deal with this situation.
 Thus let's introduce a "_require_drivers" function that can be used by
 the shell tests to check for the availability of certain drivers first,
 and marks the test as "not run" if one of the drivers is missing.
-'EventLoopBaseClass' learns a new callback, 'can_be_deleted()' so as to
+This patch mainly targets the test in the "auto" group which should
-mark 'MainLoop' as non-deletable.
+never fail in such a case, but also improves some of the other tests
 along the way. Note that we also assume that the "qcow2" and "file"
 drivers are always available - otherwise it does not make sense to
 run "make check-block" at all (which only tests with qcow2 by default).
-[1] For example:
+Signed-off-by: Thomas Huth <thuth@redhat.com>
-      -object main-loop,id=main-loop,aio-max-batch=<value>
+Message-id: 20190823133552.11680-1-thuth@redhat.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
  tests/qemu-iotests/071       |  1 +
  tests/qemu-iotests/081       |  4 +---
  tests/qemu-iotests/099       |  1 +
  tests/qemu-iotests/120       |  1 +
  tests/qemu-iotests/162       |  4 +---
  tests/qemu-iotests/184       |  1 +
  tests/qemu-iotests/186       |  1 +
  tests/qemu-iotests/common.rc | 14 ++++++++++++++
 files changed, 21 insertions(+), 6 deletions(-)
-Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
+diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+index XXXXXXX..XXXXXXX 100755
-Acked-by: Markus Armbruster <armbru@redhat.com>
+--- a/tests/qemu-iotests/071
-Message-id: 20220425075723.20019-3-nsaenzju@redhat.com
++++ b/tests/qemu-iotests/071
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
----
- qapi/qom.json                    | 13 ++++++++
+ _supported_fmt qcow2
- meson.build                      |  3 +-
+ _supported_proto file
- include/qemu/main-loop.h         | 10 ++++++
++_require_drivers blkdebug blkverify
- include/sysemu/event-loop-base.h |  1 +
- event-loop-base.c                | 13 ++++++++
+ do_run_qemu()
- util/main-loop.c                 | 56 ++++++++++++++++++++++++++++++++
+ {
-files changed, 95 insertions(+), 1 deletion(-)
+diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
+index XXXXXXX..XXXXXXX 100755
-diff --git a/qapi/qom.json b/qapi/qom.json
+--- a/tests/qemu-iotests/081
 +++ b/tests/qemu-iotests/081
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt raw
  _supported_proto file
  _supported_os Linux
 +_require_drivers quorum
  do_run_qemu()
  {
@@ -XXX,XX +XXX,XX @@ run_qemu()
                            | _filter_qemu_io | _filter_generated_node_ids
  }
 -test_quorum=$($QEMU_IMG --help|grep quorum)
 -[ "$test_quorum" = "" ] && _supported_fmt quorum
 -
  quorum="driver=raw,file.driver=quorum,file.vote-threshold=2"
  quorum="$quorum,file.children.0.file.filename=$TEST_DIR/1.raw"
  quorum="$quorum,file.children.1.file.filename=$TEST_DIR/2.raw"
 diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/099
 +++ b/tests/qemu-iotests/099
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow qcow2 qed vdi vhdx vmdk vpc
  _supported_proto file
  _supported_os Linux
 +_require_drivers blkdebug blkverify
  _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
      "subformat=twoGbMaxExtentSparse"
 diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/120
 +++ b/tests/qemu-iotests/120
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt generic
  _supported_proto file
  _unsupported_fmt luks
 +_require_drivers raw
  _make_test_img 64M
 diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/162
 +++ b/tests/qemu-iotests/162
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  . ./common.filter
  _supported_fmt generic
 -
 -test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh\( \|$\)')
 -[ "$test_ssh" = "" ] && _notrun "ssh support required"
 +_require_drivers ssh
  echo
  echo '=== NBD ==='
 diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/184
 +++ b/tests/qemu-iotests/184
@@ -XXX,XX +XXX,XX @@ trap "exit \$status" 0 1 2 3 15
  . ./common.filter
  _supported_os Linux
 +_require_drivers throttle
  do_run_qemu()
  {
 diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/186
 +++ b/tests/qemu-iotests/186
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
  _supported_fmt qcow2
  _supported_proto file
 +_require_drivers null-co
  if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
      _notrun "Requires a PC machine"
 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
 index XXXXXXX..XXXXXXX 100644
---- a/qapi/qom.json
+--- a/tests/qemu-iotests/common.rc
-+++ b/qapi/qom.json
++++ b/tests/qemu-iotests/common.rc
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ _require_command()
-             '*poll-grow': 'int',
+     [ -x "$c" ] || _notrun "$1 utility required, skipped this test"
-             '*poll-shrink': 'int' } }
+ }
-+##
++# Check that a set of drivers has been whitelisted in the QEMU binary
 +# @MainLoopProperties:
 +#
-+# Properties for the main-loop object.
++_require_drivers()
 +#
 +# Since: 7.1
 +##
 +{ 'struct': 'MainLoopProperties',
 +  'base': 'EventLoopBaseProperties',
 +  'data': {} }
 +
  ##
  # @MemoryBackendProperties:
  #
@@ -XXX,XX +XXX,XX @@
      { 'name': 'input-linux',
        'if': 'CONFIG_LINUX' },
      'iothread',
 +    'main-loop',
      { 'name': 'memory-backend-epc',
        'if': 'CONFIG_LINUX' },
      'memory-backend-file',
@@ -XXX,XX +XXX,XX @@
        'input-linux':                { 'type': 'InputLinuxProperties',
                                        'if': 'CONFIG_LINUX' },
        'iothread':                   'IothreadProperties',
 +      'main-loop':                  'MainLoopProperties',
        'memory-backend-epc':         { 'type': 'MemoryBackendEpcProperties',
                                        'if': 'CONFIG_LINUX' },
        'memory-backend-file':        'MemoryBackendFileProperties',
 diff --git a/meson.build b/meson.build
 index XXXXXXX..XXXXXXX 100644
 --- a/meson.build
 +++ b/meson.build
@@ -XXX,XX +XXX,XX @@ libqemuutil = static_library('qemuutil',
                               sources: util_ss.sources() + stub_ss.sources() + genh,
                               dependencies: [util_ss.dependencies(), libm, threads, glib, socket, malloc, pixman])
  qemuutil = declare_dependency(link_with: libqemuutil,
 -                              sources: genh + version_res)
 +                              sources: genh + version_res,
 +                              dependencies: [event_loop_base])
  if have_system or have_user
    decodetree = generator(find_program('scripts/decodetree.py'),
 diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/qemu/main-loop.h
 +++ b/include/qemu/main-loop.h
@@ -XXX,XX +XXX,XX @@
  #define QEMU_MAIN_LOOP_H
  #include "block/aio.h"
 +#include "qom/object.h"
 +#include "sysemu/event-loop-base.h"
  #define SIG_IPI SIGUSR1
 +#define TYPE_MAIN_LOOP  "main-loop"
 +OBJECT_DECLARE_TYPE(MainLoop, MainLoopClass, MAIN_LOOP)
 +
 +struct MainLoop {
 +    EventLoopBase parent_obj;
 +};
 +typedef struct MainLoop MainLoop;
 +
  /**
   * qemu_init_main_loop: Set up the process so that it can run the main loop.
   *
 diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/sysemu/event-loop-base.h
 +++ b/include/sysemu/event-loop-base.h
@@ -XXX,XX +XXX,XX @@ struct EventLoopBaseClass {
      void (*init)(EventLoopBase *base, Error **errp);
      void (*update_params)(EventLoopBase *base, Error **errp);
 +    bool (*can_be_deleted)(EventLoopBase *base);
  };
  struct EventLoopBase {
 diff --git a/event-loop-base.c b/event-loop-base.c
 index XXXXXXX..XXXXXXX 100644
 --- a/event-loop-base.c
 +++ b/event-loop-base.c
@@ -XXX,XX +XXX,XX @@ static void event_loop_base_complete(UserCreatable *uc, Error **errp)
      }
  }
 +static bool event_loop_base_can_be_deleted(UserCreatable *uc)
 +{
-+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
++    available=$($QEMU -drive format=help | \
-+    EventLoopBase *backend = EVENT_LOOP_BASE(uc);
++                sed -e '/Supported formats:/!d' -e 's/Supported formats://')
-+
++    for driver
-+    if (bc->can_be_deleted) {
++    do
-+        return bc->can_be_deleted(backend);
++        if ! echo "$available" | grep -q " $driver\( \|$\)"; then
-+    }
++            _notrun "$driver not available"
-+
++        fi
-+    return true;
++    done
 +}
 +
- static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
+ # make sure this script returns success
- {
+ true
      UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
      ucc->complete = event_loop_base_complete;
 +    ucc->can_be_deleted = event_loop_base_can_be_deleted;
      object_class_property_add(klass, "aio-max-batch", "int",
                                event_loop_base_get_param,
 diff --git a/util/main-loop.c b/util/main-loop.c
 index XXXXXXX..XXXXXXX 100644
 --- a/util/main-loop.c
 +++ b/util/main-loop.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/error-report.h"
  #include "qemu/queue.h"
  #include "qemu/compiler.h"
 +#include "qom/object.h"
  #ifndef _WIN32
  #include <sys/wait.h>
@@ -XXX,XX +XXX,XX @@ int qemu_init_main_loop(Error **errp)
      return 0;
  }
 +static void main_loop_update_params(EventLoopBase *base, Error **errp)
 +{
 +    if (!qemu_aio_context) {
 +        error_setg(errp, "qemu aio context not ready");
 +        return;
 +    }
 +
 +    aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
 +}
 +
 +MainLoop *mloop;
 +
 +static void main_loop_init(EventLoopBase *base, Error **errp)
 +{
 +    MainLoop *m = MAIN_LOOP(base);
 +
 +    if (mloop) {
 +        error_setg(errp, "only one main-loop instance allowed");
 +        return;
 +    }
 +
 +    main_loop_update_params(base, errp);
 +
 +    mloop = m;
 +    return;
 +}
 +
 +static bool main_loop_can_be_deleted(EventLoopBase *base)
 +{
 +    return false;
 +}
 +
 +static void main_loop_class_init(ObjectClass *oc, void *class_data)
 +{
 +    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(oc);
 +
 +    bc->init = main_loop_init;
 +    bc->update_params = main_loop_update_params;
 +    bc->can_be_deleted = main_loop_can_be_deleted;
 +}
 +
 +static const TypeInfo main_loop_info = {
 +    .name = TYPE_MAIN_LOOP,
 +    .parent = TYPE_EVENT_LOOP_BASE,
 +    .class_init = main_loop_class_init,
 +    .instance_size = sizeof(MainLoop),
 +};
 +
 +static void main_loop_register_types(void)
 +{
 +    type_register_static(&main_loop_info);
 +}
 +
 +type_init(main_loop_register_types)
 +
  static int max_priority;
  #ifndef _WIN32
 --
-.35.1
+.21.0

-New patch
+[Qemu-devel] [PULL v2 15/16] tests/check-block: Skip iotests when sanitizers are enabled
+From: Thomas Huth <thuth@redhat.com>
+The sanitizers (especially the address sanitizer from Clang) are
+sometimes printing out warnings or false positives - this spoils
+the output of the iotests, causing some of the tests to fail.
+Thus let's skip the automatic iotests during "make check" when the
+user configured QEMU with --enable-sanitizers.
+Signed-off-by: Thomas Huth <thuth@redhat.com>
+Message-id: 20190823084203.29734-1-thuth@redhat.com
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/check-block.sh | 5 +++++
+file changed, 5 insertions(+)
+diff --git a/tests/check-block.sh b/tests/check-block.sh
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/check-block.sh
++++ b/tests/check-block.sh
+@@ -XXX,XX +XXX,XX @@ if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
+     exit 0
+ fi
++if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
++    echo "Sanitizers are enabled ==> Not running the qemu-iotests."
++    exit 0
++fi
++
+ if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
+     echo "No qemu-system binary available ==> Not running the qemu-iotests."
+     exit 0
+--
+.21.0

-New patch
+[Qemu-devel] [PULL v2 16/16] iotests: Unify cache mode quoting
+From: Nir Soffer <nirsof@gmail.com>
+Quoting cache mode is not needed, and most tests use unquoted values.
+Unify all test to use the same style.
+Message-id: 20190827173432.7656-1-nsoffer@redhat.com
+Signed-off-by: Nir Soffer <nsoffer@redhat.com>
+Signed-off-by: Max Reitz <mreitz@redhat.com>
+---
+ tests/qemu-iotests/026 | 4 ++--
+ tests/qemu-iotests/039 | 4 ++--
+ tests/qemu-iotests/052 | 2 +-
+ tests/qemu-iotests/091 | 4 ++--
+files changed, 7 insertions(+), 7 deletions(-)
+diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/026
++++ b/tests/qemu-iotests/026
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ # Currently only qcow2 supports rebasing
+ _supported_fmt qcow2
+ _supported_proto file
+-_default_cache_mode "writethrough"
+-_supported_cache_modes "writethrough" "none"
++_default_cache_mode writethrough
++_supported_cache_modes writethrough none
+ # The refcount table tests expect a certain minimum width for refcount entries
+ # (so that the refcount table actually needs to grow); that minimum is 16 bits,
+ # being the default refcount entry width.
+diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/039
++++ b/tests/qemu-iotests/039
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow2
+ _supported_proto file
+ _supported_os Linux
+-_default_cache_mode "writethrough"
+-_supported_cache_modes "writethrough"
++_default_cache_mode writethrough
++_supported_cache_modes writethrough
+ size=128M
+diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/052
++++ b/tests/qemu-iotests/052
+@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
+ _supported_proto file
+ # Don't do O_DIRECT on tmpfs
+-_supported_cache_modes "writeback" "writethrough" "unsafe"
++_supported_cache_modes writeback writethrough unsafe
+ size=128M
+ _make_test_img $size
+diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/091
++++ b/tests/qemu-iotests/091
+@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
+ _supported_fmt qcow2
+ _supported_proto file
+ _supported_os Linux
+-_default_cache_mode "none"
+-_supported_cache_modes "writethrough" "none" "writeback"
++_default_cache_mode none
++_supported_cache_modes writethrough none writeback
+ size=1G
+--
+.21.0

The following changes since commit 9cf289af47bcfae5c75de37d8e5d6fd23705322c:

Merge tag 'qga-pull-request' of gitlab.com:marcandre.lureau/qemu into staging (2022-05-04 03:42:49 -0700)

are available in the Git repository at:

https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to bef2e050d6a7feb865854c65570c496ac5a8cf53:

util/event-loop-base: Introduce options to set the thread pool size (2022-05-04 17:02:19 +0100)

----------------------------------------------------------------
Pull request

Add new thread-pool-min/thread-pool-max parameters to control the thread pool
used for async I/O.

----------------------------------------------------------------

Nicolas Saenz Julienne (3):
  Introduce event-loop-base abstract class
  util/main-loop: Introduce the main loop into QOM
  util/event-loop-base: Introduce options to set the thread pool size

-- 
2.35.1

From: Nicolas Saenz Julienne <nsaenzju@redhat.com>

Introduce the 'event-loop-base' abstract class, it'll hold the
properties common to all event loops and provide the necessary hooks for
their creation and maintenance. Then have iothread inherit from it.

EventLoopBaseClass is defined as user creatable and provides a hook for
its children to attach themselves to the user creatable class 'complete'
function. It also provides an update_params() callback to propagate
property changes onto its children.

The new 'event-loop-base' class will live in the root directory. It is
built on its own using the 'link_whole' option (there are no direct
function dependencies between the class and its children, it all happens
trough 'constructor' magic). And also imposes new compilation
dependencies:

qom <- event-loop-base <- blockdev (iothread.c)

And in subsequent patches:

qom <- event-loop-base <- qemuutil (util/main-loop.c)

All this forced some amount of reordering in meson.build:

- Moved qom build definition before qemuutil. Doing it the other way
   around (i.e. moving qemuutil after qom) isn't possible as a lot of
   core libraries that live in between the two depend on it.

- Process the 'hw' subdir earlier, as it introduces files into the
   'qom' source set.

No functional changes intended.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-2-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qapi/qom.json                    |  22 +++++--
 meson.build                      |  23 ++++---
 include/sysemu/event-loop-base.h |  36 +++++++++++
 include/sysemu/iothread.h        |   6 +-
 event-loop-base.c                | 104 +++++++++++++++++++++++++++++++
 iothread.c                       |  65 ++++++-------------
 6 files changed, 192 insertions(+), 64 deletions(-)
 create mode 100644 include/sysemu/event-loop-base.h
 create mode 100644 event-loop-base.c

diff --git a/qapi/qom.json b/qapi/qom.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -XXX,XX +XXX,XX @@
             '*repeat': 'bool',
             '*grab-toggle': 'GrabToggleKeys' } }
 
+##
+# @EventLoopBaseProperties:
+#
+# Common properties for event loops
+#
+# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
+#                 0 means that the engine will use its default.
+#                 (default: 0)
+#
+# Since: 7.1
+##
+{ 'struct': 'EventLoopBaseProperties',
+  'data': { '*aio-max-batch': 'int' } }
+
 ##
 # @IothreadProperties:
 #
@@ -XXX,XX +XXX,XX @@
 #               algorithm detects it is spending too long polling without
 #               encountering events. 0 selects a default behaviour (default: 0)
 #
-# @aio-max-batch: maximum number of requests in a batch for the AIO engine,
-#                 0 means that the engine will use its default
-#                 (default:0, since 6.1)
+# The @aio-max-batch option is available since 6.1.
 #
 # Since: 2.0
 ##
 { 'struct': 'IothreadProperties',
+  'base': 'EventLoopBaseProperties',
   'data': { '*poll-max-ns': 'int',
             '*poll-grow': 'int',
-            '*poll-shrink': 'int',
-            '*aio-max-batch': 'int' } }
+            '*poll-shrink': 'int' } }
 
 ##
 # @MemoryBackendProperties:
diff --git a/meson.build b/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/meson.build
+++ b/meson.build
@@ -XXX,XX +XXX,XX @@ subdir('qom')
 subdir('authz')
 subdir('crypto')
 subdir('ui')
+subdir('hw')
 
 
 if enable_modules
@@ -XXX,XX +XXX,XX @@ if enable_modules
   modulecommon = declare_dependency(link_whole: libmodulecommon, compile_args: '-DBUILD_DSO')
 endif
 
+qom_ss = qom_ss.apply(config_host, strict: false)
+libqom = static_library('qom', qom_ss.sources() + genh,
+                        dependencies: [qom_ss.dependencies()],
+                        name_suffix: 'fa')
+qom = declare_dependency(link_whole: libqom)
+
+event_loop_base = files('event-loop-base.c')
+event_loop_base = static_library('event-loop-base', sources: event_loop_base + genh,
+                                 build_by_default: true)
+event_loop_base = declare_dependency(link_whole: event_loop_base,
+                                     dependencies: [qom])
+
 stub_ss = stub_ss.apply(config_all, strict: false)
 
 util_ss.add_all(trace_ss)
@@ -XXX,XX +XXX,XX @@ subdir('monitor')
 subdir('net')
 subdir('replay')
 subdir('semihosting')
-subdir('hw')
 subdir('tcg')
 subdir('fpu')
 subdir('accel')
@@ -XXX,XX +XXX,XX @@ qemu_syms = custom_target('qemu.syms', output: 'qemu.syms',
                              capture: true,
                              command: [undefsym, nm, '@INPUT@'])
 
-qom_ss = qom_ss.apply(config_host, strict: false)
-libqom = static_library('qom', qom_ss.sources() + genh,
-                        dependencies: [qom_ss.dependencies()],
-                        name_suffix: 'fa')
-
-qom = declare_dependency(link_whole: libqom)
-
 authz_ss = authz_ss.apply(config_host, strict: false)
 libauthz = static_library('authz', authz_ss.sources() + genh,
                           dependencies: [authz_ss.dependencies()],
@@ -XXX,XX +XXX,XX @@ libblockdev = static_library('blockdev', blockdev_ss.sources() + genh,
                              build_by_default: false)
 
 blockdev = declare_dependency(link_whole: [libblockdev],
-                              dependencies: [block])
+                              dependencies: [block, event_loop_base])
 
 qmp_ss = qmp_ss.apply(config_host, strict: false)
 libqmp = static_library('qmp', qmp_ss.sources() + genh,
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/include/sysemu/event-loop-base.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU event-loop backend
+ *
+ * Copyright (C) 2022 Red Hat Inc
+ *
+ * Authors:
+ *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef QEMU_EVENT_LOOP_BASE_H
+#define QEMU_EVENT_LOOP_BASE_H
+
+#include "qom/object.h"
+#include "block/aio.h"
+#include "qemu/typedefs.h"
+
+#define TYPE_EVENT_LOOP_BASE         "event-loop-base"
+OBJECT_DECLARE_TYPE(EventLoopBase, EventLoopBaseClass,
+                    EVENT_LOOP_BASE)
+
+struct EventLoopBaseClass {
+    ObjectClass parent_class;
+
+    void (*init)(EventLoopBase *base, Error **errp);
+    void (*update_params)(EventLoopBase *base, Error **errp);
+};
+
+struct EventLoopBase {
+    Object parent;
+
+    /* AioContext AIO engine parameters */
+    int64_t aio_max_batch;
+};
+#endif
diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/iothread.h
+++ b/include/sysemu/iothread.h
@@ -XXX,XX +XXX,XX @@
 #include "block/aio.h"
 #include "qemu/thread.h"
 #include "qom/object.h"
+#include "sysemu/event-loop-base.h"
 
 #define TYPE_IOTHREAD "iothread"
 
 struct IOThread {
-    Object parent_obj;
+    EventLoopBase parent_obj;
 
     QemuThread thread;
     AioContext *ctx;
@@ -XXX,XX +XXX,XX @@ struct IOThread {
     int64_t poll_max_ns;
     int64_t poll_grow;
     int64_t poll_shrink;
-
-    /* AioContext AIO engine parameters */
-    int64_t aio_max_batch;
 };
 typedef struct IOThread IOThread;
 
diff --git a/event-loop-base.c b/event-loop-base.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/event-loop-base.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * QEMU event-loop base
+ *
+ * Copyright (C) 2022 Red Hat Inc
+ *
+ * Authors:
+ *  Stefan Hajnoczi <stefanha@redhat.com>
+ *  Nicolas Saenz Julienne <nsaenzju@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qom/object_interfaces.h"
+#include "qapi/error.h"
+#include "sysemu/event-loop-base.h"
+
+typedef struct {
+    const char *name;
+    ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
+} EventLoopBaseParamInfo;
+
+static EventLoopBaseParamInfo aio_max_batch_info = {
+    "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
+};
+
+static void event_loop_base_get_param(Object *obj, Visitor *v,
+        const char *name, void *opaque, Error **errp)
+{
+    EventLoopBase *event_loop_base = EVENT_LOOP_BASE(obj);
+    EventLoopBaseParamInfo *info = opaque;
+    int64_t *field = (void *)event_loop_base + info->offset;
+
+    visit_type_int64(v, name, field, errp);
+}
+
+static void event_loop_base_set_param(Object *obj, Visitor *v,
+        const char *name, void *opaque, Error **errp)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(obj);
+    EventLoopBase *base = EVENT_LOOP_BASE(obj);
+    EventLoopBaseParamInfo *info = opaque;
+    int64_t *field = (void *)base + info->offset;
+    int64_t value;
+
+    if (!visit_type_int64(v, name, &value, errp)) {
+        return;
+    }
+
+    if (value < 0) {
+        error_setg(errp, "%s value must be in range [0, %" PRId64 "]",
+                   info->name, INT64_MAX);
+        return;
+    }
+
+    *field = value;
+
+    if (bc->update_params) {
+        bc->update_params(base, errp);
+    }
+
+    return;
+}
+
+static void event_loop_base_complete(UserCreatable *uc, Error **errp)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
+    EventLoopBase *base = EVENT_LOOP_BASE(uc);
+
+    if (bc->init) {
+        bc->init(base, errp);
+    }
+}
+
+static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
+    ucc->complete = event_loop_base_complete;
+
+    object_class_property_add(klass, "aio-max-batch", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &aio_max_batch_info);
+}
+
+static const TypeInfo event_loop_base_info = {
+    .name = TYPE_EVENT_LOOP_BASE,
+    .parent = TYPE_OBJECT,
+    .instance_size = sizeof(EventLoopBase),
+    .class_size = sizeof(EventLoopBaseClass),
+    .class_init = event_loop_base_class_init,
+    .abstract = true,
+    .interfaces = (InterfaceInfo[]) {
+        { TYPE_USER_CREATABLE },
+        { }
+    }
+};
+
+static void register_types(void)
+{
+    type_register_static(&event_loop_base_info);
+}
+type_init(register_types);
diff --git a/iothread.c b/iothread.c
index XXXXXXX..XXXXXXX 100644
--- a/iothread.c
+++ b/iothread.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/module.h"
 #include "block/aio.h"
 #include "block/block.h"
+#include "sysemu/event-loop-base.h"
 #include "sysemu/iothread.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-misc.h"
@@ -XXX,XX +XXX,XX @@ static void iothread_init_gcontext(IOThread *iothread)
     iothread->main_loop = g_main_loop_new(iothread->worker_context, TRUE);
 }
 
-static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
+static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
 {
+    IOThread *iothread = IOTHREAD(base);
     ERRP_GUARD();
 
+    if (!iothread->ctx) {
+        return;
+    }
+
     aio_context_set_poll_params(iothread->ctx,
                                 iothread->poll_max_ns,
                                 iothread->poll_grow,
@@ -XXX,XX +XXX,XX @@ static void iothread_set_aio_context_params(IOThread *iothread, Error **errp)
     }
 
     aio_context_set_aio_params(iothread->ctx,
-                               iothread->aio_max_batch,
+                               iothread->parent_obj.aio_max_batch,
                                errp);
 }
 
-static void iothread_complete(UserCreatable *obj, Error **errp)
+
+static void iothread_init(EventLoopBase *base, Error **errp)
 {
     Error *local_error = NULL;
-    IOThread *iothread = IOTHREAD(obj);
+    IOThread *iothread = IOTHREAD(base);
     char *thread_name;
 
     iothread->stopping = false;
@@ -XXX,XX +XXX,XX @@ static void iothread_complete(UserCreatable *obj, Error **errp)
      */
     iothread_init_gcontext(iothread);
 
-    iothread_set_aio_context_params(iothread, &local_error);
+    iothread_set_aio_context_params(base, &local_error);
     if (local_error) {
         error_propagate(errp, local_error);
         aio_context_unref(iothread->ctx);
@@ -XXX,XX +XXX,XX @@ static void iothread_complete(UserCreatable *obj, Error **errp)
      * to inherit.
      */
     thread_name = g_strdup_printf("IO %s",
-                        object_get_canonical_path_component(OBJECT(obj)));
+                        object_get_canonical_path_component(OBJECT(base)));
     qemu_thread_create(&iothread->thread, thread_name, iothread_run,
                        iothread, QEMU_THREAD_JOINABLE);
     g_free(thread_name);
@@ -XXX,XX +XXX,XX @@ static IOThreadParamInfo poll_grow_info = {
 static IOThreadParamInfo poll_shrink_info = {
     "poll-shrink", offsetof(IOThread, poll_shrink),
 };
-static IOThreadParamInfo aio_max_batch_info = {
-    "aio-max-batch", offsetof(IOThread, aio_max_batch),
-};
 
 static void iothread_get_param(Object *obj, Visitor *v,
         const char *name, IOThreadParamInfo *info, Error **errp)
@@ -XXX,XX +XXX,XX @@ static void iothread_set_poll_param(Object *obj, Visitor *v,
     }
 }
 
-static void iothread_get_aio_param(Object *obj, Visitor *v,
-        const char *name, void *opaque, Error **errp)
-{
-    IOThreadParamInfo *info = opaque;
-
-    iothread_get_param(obj, v, name, info, errp);
-}
-
-static void iothread_set_aio_param(Object *obj, Visitor *v,
-        const char *name, void *opaque, Error **errp)
-{
-    IOThread *iothread = IOTHREAD(obj);
-    IOThreadParamInfo *info = opaque;
-
-    if (!iothread_set_param(obj, v, name, info, errp)) {
-        return;
-    }
-
-    if (iothread->ctx) {
-        aio_context_set_aio_params(iothread->ctx,
-                                   iothread->aio_max_batch,
-                                   errp);
-    }
-}
-
 static void iothread_class_init(ObjectClass *klass, void *class_data)
 {
-    UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
-    ucc->complete = iothread_complete;
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(klass);
+
+    bc->init = iothread_init;
+    bc->update_params = iothread_set_aio_context_params;
 
     object_class_property_add(klass, "poll-max-ns", "int",
                               iothread_get_poll_param,
@@ -XXX,XX +XXX,XX @@ static void iothread_class_init(ObjectClass *klass, void *class_data)
                               iothread_get_poll_param,
                               iothread_set_poll_param,
                               NULL, &poll_shrink_info);
-    object_class_property_add(klass, "aio-max-batch", "int",
-                              iothread_get_aio_param,
-                              iothread_set_aio_param,
-                              NULL, &aio_max_batch_info);
 }
 
 static const TypeInfo iothread_info = {
     .name = TYPE_IOTHREAD,
-    .parent = TYPE_OBJECT,
+    .parent = TYPE_EVENT_LOOP_BASE,
     .class_init = iothread_class_init,
     .instance_size = sizeof(IOThread),
     .instance_init = iothread_instance_init,
     .instance_finalize = iothread_instance_finalize,
-    .interfaces = (InterfaceInfo[]) {
-        {TYPE_USER_CREATABLE},
-        {}
-    },
 };
 
 static void iothread_register_types(void)
@@ -XXX,XX +XXX,XX @@ static int query_one_iothread(Object *object, void *opaque)
     info->poll_max_ns = iothread->poll_max_ns;
     info->poll_grow = iothread->poll_grow;
     info->poll_shrink = iothread->poll_shrink;
-    info->aio_max_batch = iothread->aio_max_batch;
+    info->aio_max_batch = iothread->parent_obj.aio_max_batch;
 
     QAPI_LIST_APPEND(*tail, info);
     return 0;
-- 
2.35.1

From: Nicolas Saenz Julienne <nsaenzju@redhat.com>

'event-loop-base' provides basic property handling for all 'AioContext'
based event loops. So let's define a new 'MainLoopClass' that inherits
from it. This will permit tweaking the main loop's properties through
qapi as well as through the command line using the '-object' keyword[1].
Only one instance of 'MainLoopClass' might be created at any time.

'EventLoopBaseClass' learns a new callback, 'can_be_deleted()' so as to
mark 'MainLoop' as non-deletable.

[1] For example:
      -object main-loop,id=main-loop,aio-max-batch=<value>

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-3-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qapi/qom.json                    | 13 ++++++++
 meson.build                      |  3 +-
 include/qemu/main-loop.h         | 10 ++++++
 include/sysemu/event-loop-base.h |  1 +
 event-loop-base.c                | 13 ++++++++
 util/main-loop.c                 | 56 ++++++++++++++++++++++++++++++++
 6 files changed, 95 insertions(+), 1 deletion(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -XXX,XX +XXX,XX @@
             '*poll-grow': 'int',
             '*poll-shrink': 'int' } }
 
+##
+# @MainLoopProperties:
+#
+# Properties for the main-loop object.
+#
+# Since: 7.1
+##
+{ 'struct': 'MainLoopProperties',
+  'base': 'EventLoopBaseProperties',
+  'data': {} }
+
 ##
 # @MemoryBackendProperties:
 #
@@ -XXX,XX +XXX,XX @@
     { 'name': 'input-linux',
       'if': 'CONFIG_LINUX' },
     'iothread',
+    'main-loop',
     { 'name': 'memory-backend-epc',
       'if': 'CONFIG_LINUX' },
     'memory-backend-file',
@@ -XXX,XX +XXX,XX @@
       'input-linux':                { 'type': 'InputLinuxProperties',
                                       'if': 'CONFIG_LINUX' },
       'iothread':                   'IothreadProperties',
+      'main-loop':                  'MainLoopProperties',
       'memory-backend-epc':         { 'type': 'MemoryBackendEpcProperties',
                                       'if': 'CONFIG_LINUX' },
       'memory-backend-file':        'MemoryBackendFileProperties',
diff --git a/meson.build b/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/meson.build
+++ b/meson.build
@@ -XXX,XX +XXX,XX @@ libqemuutil = static_library('qemuutil',
                              sources: util_ss.sources() + stub_ss.sources() + genh,
                              dependencies: [util_ss.dependencies(), libm, threads, glib, socket, malloc, pixman])
 qemuutil = declare_dependency(link_with: libqemuutil,
-                              sources: genh + version_res)
+                              sources: genh + version_res,
+                              dependencies: [event_loop_base])
 
 if have_system or have_user
   decodetree = generator(find_program('scripts/decodetree.py'),
diff --git a/include/qemu/main-loop.h b/include/qemu/main-loop.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/main-loop.h
+++ b/include/qemu/main-loop.h
@@ -XXX,XX +XXX,XX @@
 #define QEMU_MAIN_LOOP_H
 
 #include "block/aio.h"
+#include "qom/object.h"
+#include "sysemu/event-loop-base.h"
 
 #define SIG_IPI SIGUSR1
 
+#define TYPE_MAIN_LOOP  "main-loop"
+OBJECT_DECLARE_TYPE(MainLoop, MainLoopClass, MAIN_LOOP)
+
+struct MainLoop {
+    EventLoopBase parent_obj;
+};
+typedef struct MainLoop MainLoop;
+
 /**
  * qemu_init_main_loop: Set up the process so that it can run the main loop.
  *
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/event-loop-base.h
+++ b/include/sysemu/event-loop-base.h
@@ -XXX,XX +XXX,XX @@ struct EventLoopBaseClass {
 
     void (*init)(EventLoopBase *base, Error **errp);
     void (*update_params)(EventLoopBase *base, Error **errp);
+    bool (*can_be_deleted)(EventLoopBase *base);
 };
 
 struct EventLoopBase {
diff --git a/event-loop-base.c b/event-loop-base.c
index XXXXXXX..XXXXXXX 100644
--- a/event-loop-base.c
+++ b/event-loop-base.c
@@ -XXX,XX +XXX,XX @@ static void event_loop_base_complete(UserCreatable *uc, Error **errp)
     }
 }
 
+static bool event_loop_base_can_be_deleted(UserCreatable *uc)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_GET_CLASS(uc);
+    EventLoopBase *backend = EVENT_LOOP_BASE(uc);
+
+    if (bc->can_be_deleted) {
+        return bc->can_be_deleted(backend);
+    }
+
+    return true;
+}
+
 static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
 {
     UserCreatableClass *ucc = USER_CREATABLE_CLASS(klass);
     ucc->complete = event_loop_base_complete;
+    ucc->can_be_deleted = event_loop_base_can_be_deleted;
 
     object_class_property_add(klass, "aio-max-batch", "int",
                               event_loop_base_get_param,
diff --git a/util/main-loop.c b/util/main-loop.c
index XXXXXXX..XXXXXXX 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/error-report.h"
 #include "qemu/queue.h"
 #include "qemu/compiler.h"
+#include "qom/object.h"
 
 #ifndef _WIN32
 #include <sys/wait.h>
@@ -XXX,XX +XXX,XX @@ int qemu_init_main_loop(Error **errp)
     return 0;
 }
 
+static void main_loop_update_params(EventLoopBase *base, Error **errp)
+{
+    if (!qemu_aio_context) {
+        error_setg(errp, "qemu aio context not ready");
+        return;
+    }
+
+    aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
+}
+
+MainLoop *mloop;
+
+static void main_loop_init(EventLoopBase *base, Error **errp)
+{
+    MainLoop *m = MAIN_LOOP(base);
+
+    if (mloop) {
+        error_setg(errp, "only one main-loop instance allowed");
+        return;
+    }
+
+    main_loop_update_params(base, errp);
+
+    mloop = m;
+    return;
+}
+
+static bool main_loop_can_be_deleted(EventLoopBase *base)
+{
+    return false;
+}
+
+static void main_loop_class_init(ObjectClass *oc, void *class_data)
+{
+    EventLoopBaseClass *bc = EVENT_LOOP_BASE_CLASS(oc);
+
+    bc->init = main_loop_init;
+    bc->update_params = main_loop_update_params;
+    bc->can_be_deleted = main_loop_can_be_deleted;
+}
+
+static const TypeInfo main_loop_info = {
+    .name = TYPE_MAIN_LOOP,
+    .parent = TYPE_EVENT_LOOP_BASE,
+    .class_init = main_loop_class_init,
+    .instance_size = sizeof(MainLoop),
+};
+
+static void main_loop_register_types(void)
+{
+    type_register_static(&main_loop_info);
+}
+
+type_init(main_loop_register_types)
+
 static int max_priority;
 
 #ifndef _WIN32
-- 
2.35.1

From: Nicolas Saenz Julienne <nsaenzju@redhat.com>

The thread pool regulates itself: when idle, it kills threads until
empty, when in demand, it creates new threads until full. This behaviour
doesn't play well with latency sensitive workloads where the price of
creating a new thread is too high. For example, when paired with qemu's
'-mlock', or using safety features like SafeStack, creating a new thread
has been measured take multiple milliseconds.

In order to mitigate this let's introduce a new 'EventLoopBase'
property to set the thread pool size. The threads will be created during
the pool's initialization or upon updating the property's value, remain
available during its lifetime regardless of demand, and destroyed upon
freeing it. A properly characterized workload will then be able to
configure the pool to avoid any latency spikes.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-4-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 qapi/qom.json                    | 10 +++++-
 include/block/aio.h              | 10 ++++++
 include/block/thread-pool.h      |  3 ++
 include/sysemu/event-loop-base.h |  4 +++
 event-loop-base.c                | 23 +++++++++++++
 iothread.c                       |  3 ++
 util/aio-posix.c                 |  1 +
 util/async.c                     | 20 ++++++++++++
 util/main-loop.c                 |  9 ++++++
 util/thread-pool.c               | 55 +++++++++++++++++++++++++++++---
 10 files changed, 133 insertions(+), 5 deletions(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -XXX,XX +XXX,XX @@
 #                 0 means that the engine will use its default.
 #                 (default: 0)
 #
+# @thread-pool-min: minimum number of threads reserved in the thread pool
+#                   (default:0)
+#
+# @thread-pool-max: maximum number of threads the thread pool can contain
+#                   (default:64)
+#
 # Since: 7.1
 ##
 { 'struct': 'EventLoopBaseProperties',
-  'data': { '*aio-max-batch': 'int' } }
+  'data': { '*aio-max-batch': 'int',
+            '*thread-pool-min': 'int',
+            '*thread-pool-max': 'int' } }
 
 ##
 # @IothreadProperties:
diff --git a/include/block/aio.h b/include/block/aio.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -XXX,XX +XXX,XX @@ struct AioContext {
     QSLIST_HEAD(, Coroutine) scheduled_coroutines;
     QEMUBH *co_schedule_bh;
 
+    int thread_pool_min;
+    int thread_pool_max;
     /* Thread pool for performing work and receiving completion callbacks.
      * Has its own locking.
      */
@@ -XXX,XX +XXX,XX @@ void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns,
 void aio_context_set_aio_params(AioContext *ctx, int64_t max_batch,
                                 Error **errp);
 
+/**
+ * aio_context_set_thread_pool_params:
+ * @ctx: the aio context
+ * @min: min number of threads to have readily available in the thread pool
+ * @min: max number of threads the thread pool can contain
+ */
+void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
+                                        int64_t max, Error **errp);
 #endif
diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/thread-pool.h
+++ b/include/block/thread-pool.h
@@ -XXX,XX +XXX,XX @@
 
 #include "block/block.h"
 
+#define THREAD_POOL_MAX_THREADS_DEFAULT         64
+
 typedef int ThreadPoolFunc(void *opaque);
 
 typedef struct ThreadPool ThreadPool;
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *thread_pool_submit_aio(ThreadPool *pool,
 int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
         ThreadPoolFunc *func, void *arg);
 void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
+void thread_pool_update_params(ThreadPool *pool, struct AioContext *ctx);
 
 #endif
diff --git a/include/sysemu/event-loop-base.h b/include/sysemu/event-loop-base.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/event-loop-base.h
+++ b/include/sysemu/event-loop-base.h
@@ -XXX,XX +XXX,XX @@ struct EventLoopBase {
 
     /* AioContext AIO engine parameters */
     int64_t aio_max_batch;
+
+    /* AioContext thread pool parameters */
+    int64_t thread_pool_min;
+    int64_t thread_pool_max;
 };
 #endif
diff --git a/event-loop-base.c b/event-loop-base.c
index XXXXXXX..XXXXXXX 100644
--- a/event-loop-base.c
+++ b/event-loop-base.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 #include "qapi/error.h"
+#include "block/thread-pool.h"
 #include "sysemu/event-loop-base.h"
 
 typedef struct {
@@ -XXX,XX +XXX,XX @@ typedef struct {
     ptrdiff_t offset; /* field's byte offset in EventLoopBase struct */
 } EventLoopBaseParamInfo;
 
+static void event_loop_base_instance_init(Object *obj)
+{
+    EventLoopBase *base = EVENT_LOOP_BASE(obj);
+
+    base->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
+}
+
 static EventLoopBaseParamInfo aio_max_batch_info = {
     "aio-max-batch", offsetof(EventLoopBase, aio_max_batch),
 };
+static EventLoopBaseParamInfo thread_pool_min_info = {
+    "thread-pool-min", offsetof(EventLoopBase, thread_pool_min),
+};
+static EventLoopBaseParamInfo thread_pool_max_info = {
+    "thread-pool-max", offsetof(EventLoopBase, thread_pool_max),
+};
 
 static void event_loop_base_get_param(Object *obj, Visitor *v,
         const char *name, void *opaque, Error **errp)
@@ -XXX,XX +XXX,XX @@ static void event_loop_base_class_init(ObjectClass *klass, void *class_data)
                               event_loop_base_get_param,
                               event_loop_base_set_param,
                               NULL, &aio_max_batch_info);
+    object_class_property_add(klass, "thread-pool-min", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &thread_pool_min_info);
+    object_class_property_add(klass, "thread-pool-max", "int",
+                              event_loop_base_get_param,
+                              event_loop_base_set_param,
+                              NULL, &thread_pool_max_info);
 }
 
 static const TypeInfo event_loop_base_info = {
     .name = TYPE_EVENT_LOOP_BASE,
     .parent = TYPE_OBJECT,
     .instance_size = sizeof(EventLoopBase),
+    .instance_init = event_loop_base_instance_init,
     .class_size = sizeof(EventLoopBaseClass),
     .class_init = event_loop_base_class_init,
     .abstract = true,
diff --git a/iothread.c b/iothread.c
index XXXXXXX..XXXXXXX 100644
--- a/iothread.c
+++ b/iothread.c
@@ -XXX,XX +XXX,XX @@ static void iothread_set_aio_context_params(EventLoopBase *base, Error **errp)
     aio_context_set_aio_params(iothread->ctx,
                                iothread->parent_obj.aio_max_batch,
                                errp);
+
+    aio_context_set_thread_pool_params(iothread->ctx, base->thread_pool_min,
+                                       base->thread_pool_max, errp);
 }
 
 
diff --git a/util/aio-posix.c b/util/aio-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -XXX,XX +XXX,XX @@
 
 #include "qemu/osdep.h"
 #include "block/block.h"
+#include "block/thread-pool.h"
 #include "qemu/main-loop.h"
 #include "qemu/rcu.h"
 #include "qemu/rcu_queue.h"
diff --git a/util/async.c b/util/async.c
index XXXXXXX..XXXXXXX 100644
--- a/util/async.c
+++ b/util/async.c
@@ -XXX,XX +XXX,XX @@ AioContext *aio_context_new(Error **errp)
 
     ctx->aio_max_batch = 0;
 
+    ctx->thread_pool_min = 0;
+    ctx->thread_pool_max = THREAD_POOL_MAX_THREADS_DEFAULT;
+
     return ctx;
 fail:
     g_source_destroy(&ctx->source);
@@ -XXX,XX +XXX,XX @@ void qemu_set_current_aio_context(AioContext *ctx)
     assert(!get_my_aiocontext());
     set_my_aiocontext(ctx);
 }
+
+void aio_context_set_thread_pool_params(AioContext *ctx, int64_t min,
+                                        int64_t max, Error **errp)
+{
+
+    if (min > max || !max || min > INT_MAX || max > INT_MAX) {
+        error_setg(errp, "bad thread-pool-min/thread-pool-max values");
+        return;
+    }
+
+    ctx->thread_pool_min = min;
+    ctx->thread_pool_max = max;
+
+    if (ctx->thread_pool) {
+        thread_pool_update_params(ctx->thread_pool, ctx);
+    }
+}
diff --git a/util/main-loop.c b/util/main-loop.c
index XXXXXXX..XXXXXXX 100644
--- a/util/main-loop.c
+++ b/util/main-loop.c
@@ -XXX,XX +XXX,XX @@
 #include "sysemu/replay.h"
 #include "qemu/main-loop.h"
 #include "block/aio.h"
+#include "block/thread-pool.h"
 #include "qemu/error-report.h"
 #include "qemu/queue.h"
 #include "qemu/compiler.h"
@@ -XXX,XX +XXX,XX @@ int qemu_init_main_loop(Error **errp)
 
 static void main_loop_update_params(EventLoopBase *base, Error **errp)
 {
+    ERRP_GUARD();
+
     if (!qemu_aio_context) {
         error_setg(errp, "qemu aio context not ready");
         return;
     }
 
     aio_context_set_aio_params(qemu_aio_context, base->aio_max_batch, errp);
+    if (*errp) {
+        return;
+    }
+
+    aio_context_set_thread_pool_params(qemu_aio_context, base->thread_pool_min,
+                                       base->thread_pool_max, errp);
 }
 
 MainLoop *mloop;
diff --git a/util/thread-pool.c b/util/thread-pool.c
index XXXXXXX..XXXXXXX 100644
--- a/util/thread-pool.c
+++ b/util/thread-pool.c
@@ -XXX,XX +XXX,XX @@ struct ThreadPool {
     QemuMutex lock;
     QemuCond worker_stopped;
     QemuSemaphore sem;
-    int max_threads;
     QEMUBH *new_thread_bh;
 
     /* The following variables are only accessed from one AioContext. */
@@ -XXX,XX +XXX,XX @@ struct ThreadPool {
     int new_threads;     /* backlog of threads we need to create */
     int pending_threads; /* threads created but not running yet */
     bool stopping;
+    int min_threads;
+    int max_threads;
 };
 
+static inline bool back_to_sleep(ThreadPool *pool, int ret)
+{
+    /*
+     * The semaphore timed out, we should exit the loop except when:
+     *  - There is work to do, we raced with the signal.
+     *  - The max threads threshold just changed, we raced with the signal.
+     *  - The thread pool forces a minimum number of readily available threads.
+     */
+    if (ret == -1 && (!QTAILQ_EMPTY(&pool->request_list) ||
+            pool->cur_threads > pool->max_threads ||
+            pool->cur_threads <= pool->min_threads)) {
+            return true;
+    }
+
+    return false;
+}
+
 static void *worker_thread(void *opaque)
 {
     ThreadPool *pool = opaque;
@@ -XXX,XX +XXX,XX @@ static void *worker_thread(void *opaque)
             ret = qemu_sem_timedwait(&pool->sem, 10000);
             qemu_mutex_lock(&pool->lock);
             pool->idle_threads--;
-        } while (ret == -1 && !QTAILQ_EMPTY(&pool->request_list));
-        if (ret == -1 || pool->stopping) {
+        } while (back_to_sleep(pool, ret));
+        if (ret == -1 || pool->stopping ||
+            pool->cur_threads > pool->max_threads) {
             break;
         }
 
@@ -XXX,XX +XXX,XX @@ void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg)
     thread_pool_submit_aio(pool, func, arg, NULL, NULL);
 }
 
+void thread_pool_update_params(ThreadPool *pool, AioContext *ctx)
+{
+    qemu_mutex_lock(&pool->lock);
+
+    pool->min_threads = ctx->thread_pool_min;
+    pool->max_threads = ctx->thread_pool_max;
+
+    /*
+     * We either have to:
+     *  - Increase the number available of threads until over the min_threads
+     *    threshold.
+     *  - Decrease the number of available threads until under the max_threads
+     *    threshold.
+     *  - Do nothing. The current number of threads fall in between the min and
+     *    max thresholds. We'll let the pool manage itself.
+     */
+    for (int i = pool->cur_threads; i < pool->min_threads; i++) {
+        spawn_thread(pool);
+    }
+
+    for (int i = pool->cur_threads; i > pool->max_threads; i--) {
+        qemu_sem_post(&pool->sem);
+    }
+
+    qemu_mutex_unlock(&pool->lock);
+}
+
 static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
 {
     if (!ctx) {
@@ -XXX,XX +XXX,XX @@ static void thread_pool_init_one(ThreadPool *pool, AioContext *ctx)
     qemu_mutex_init(&pool->lock);
     qemu_cond_init(&pool->worker_stopped);
     qemu_sem_init(&pool->sem, 0);
-    pool->max_threads = 64;
     pool->new_thread_bh = aio_bh_new(ctx, spawn_thread_bh_fn, pool);
 
     QLIST_INIT(&pool->head);
     QTAILQ_INIT(&pool->request_list);
+
+    thread_pool_update_params(pool, ctx);
 }
 
 ThreadPool *thread_pool_new(AioContext *ctx)
-- 
2.35.1

The following changes since commit 54b89db5309d5fa8b5d3fe5fe56f81704e2f9706:

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2019-09-03 09:43:26 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2019-09-03

for you to fetch changes up to 755c5fe79d88717600356d3edf04835bba43dcb6:

iotests: Unify cache mode quoting (2019-09-03 14:56:06 +0200)

----------------------------------------------------------------
Block patches:
- qemu-io now accepts a file to read a write pattern from
- Ensure that raw files have their first block allocated so we can probe
  the O_DIRECT alignment if necessary
- Various fixes

----------------------------------------------------------------
v2:
- Added a patch we already had on the list to keep the iotests passing
  when $DISPLAY is not set

----------------------------------------------------------------
Denis Plotnikov (1):
  qemu-io: add pattern file for write command

Max Reitz (8):
  iotests: Fix _filter_img_create()
  vmdk: Use bdrv_dirname() for relative extent paths
  iotests: Keep testing broken relative extent paths
  vmdk: Reject invalid compressed writes
  iotests: Disable broken streamOptimized tests
  iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
  iotests: Disable 126 for flat vmdk subformats
  iotests: Add -display none to the qemu options

Nir Soffer (3):
  block: posix: Always allocate the first block
  iotests: Test allocate_first_block() with O_DIRECT
  iotests: Unify cache mode quoting

Stefan Hajnoczi (1):
  file-posix: fix request_alignment typo

Thomas Huth (2):
  iotests: Check for enabled drivers before testing them
  tests/check-block: Skip iotests when sanitizers are enabled

Vladimir Sementsov-Ogievskiy (1):
  block: fix permission update in bdrv_replace_node

block.c                                       |  5 +-
 block/file-posix.c                            | 53 +++++++++-
 block/vmdk.c                                  | 64 ++++++++----
 qemu-io-cmds.c                                | 99 +++++++++++++++++--
 tests/check-block.sh                          |  5 +
 tests/qemu-iotests/002                        |  1 +
 tests/qemu-iotests/003                        |  1 +
 tests/qemu-iotests/005                        |  3 +-
 tests/qemu-iotests/009                        |  1 +
 tests/qemu-iotests/010                        |  1 +
 tests/qemu-iotests/011                        |  1 +
 tests/qemu-iotests/017                        |  3 +-
 tests/qemu-iotests/018                        |  3 +-
 tests/qemu-iotests/019                        |  3 +-
 tests/qemu-iotests/020                        |  3 +-
 tests/qemu-iotests/026                        |  4 +-
 tests/qemu-iotests/027                        |  1 +
 tests/qemu-iotests/032                        |  1 +
 tests/qemu-iotests/033                        |  1 +
 tests/qemu-iotests/034                        |  3 +-
 tests/qemu-iotests/037                        |  3 +-
 tests/qemu-iotests/039                        |  4 +-
 tests/qemu-iotests/052                        |  2 +-
 tests/qemu-iotests/059                        | 34 ++++++-
 tests/qemu-iotests/059.out                    | 26 +++--
 tests/qemu-iotests/063                        |  3 +-
 tests/qemu-iotests/071                        |  1 +
 tests/qemu-iotests/072                        |  1 +
 tests/qemu-iotests/081                        |  4 +-
 tests/qemu-iotests/091                        |  4 +-
 tests/qemu-iotests/099                        |  1 +
 tests/qemu-iotests/105                        |  3 +-
 tests/qemu-iotests/110                        |  3 +-
 tests/qemu-iotests/120                        |  1 +
 tests/qemu-iotests/126                        |  2 +
 tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
 tests/qemu-iotests/150.out.raw                | 12 +++
 tests/qemu-iotests/162                        |  4 +-
 tests/qemu-iotests/175                        | 47 +++++++--
 tests/qemu-iotests/175.out                    | 16 ++-
 tests/qemu-iotests/178.out.qcow2              |  4 +-
 tests/qemu-iotests/184                        |  1 +
 tests/qemu-iotests/186                        |  1 +
 tests/qemu-iotests/197                        |  1 +
 tests/qemu-iotests/215                        |  1 +
 tests/qemu-iotests/221.out                    | 12 ++-
 tests/qemu-iotests/251                        |  1 +
 tests/qemu-iotests/253.out                    | 12 ++-
 tests/qemu-iotests/check                      |  6 +-
 tests/qemu-iotests/common.filter              |  4 +-
 tests/qemu-iotests/common.rc                  | 14 +++
 51 files changed, 394 insertions(+), 90 deletions(-)
 rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
 create mode 100644 tests/qemu-iotests/150.out.raw

-- 
2.21.0

From: Denis Plotnikov <dplotnikov@virtuozzo.com>

The patch allows to provide a pattern file for write
command. There was no similar ability before.

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Message-id: 20190820164616.4072-1-dplotnikov@virtuozzo.com
Reviewed-by: Eric Blake <eblake@redhat.com>
[mreitz: Keep optstring in alphabetical order]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 qemu-io-cmds.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 93 insertions(+), 6 deletions(-)

diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -XXX,XX +XXX,XX @@ static void qemu_io_free(void *p)
     qemu_vfree(p);
 }
 
+/*
+ * qemu_io_alloc_from_file()
+ *
+ * Allocates the buffer and populates it with the content of the given file
+ * up to @len bytes. If the file length is less than @len, then the buffer
+ * is populated with the file content cyclically.
+ *
+ * @blk - the block backend where the buffer content is going to be written to
+ * @len - the buffer length
+ * @file_name - the file to read the content from
+ *
+ * Returns: the buffer pointer on success
+ *          NULL on error
+ */
+static void *qemu_io_alloc_from_file(BlockBackend *blk, size_t len,
+                                     const char *file_name)
+{
+    char *buf, *buf_origin;
+    FILE *f = fopen(file_name, "r");
+    int pattern_len;
+
+    if (!f) {
+        perror(file_name);
+        return NULL;
+    }
+
+    if (qemuio_misalign) {
+        len += MISALIGN_OFFSET;
+    }
+
+    buf_origin = buf = blk_blockalign(blk, len);
+
+    if (qemuio_misalign) {
+        buf_origin += MISALIGN_OFFSET;
+        buf += MISALIGN_OFFSET;
+        len -= MISALIGN_OFFSET;
+    }
+
+    pattern_len = fread(buf_origin, 1, len, f);
+
+    if (ferror(f)) {
+        perror(file_name);
+        goto error;
+    }
+
+    if (pattern_len == 0) {
+        fprintf(stderr, "%s: file is empty\n", file_name);
+        goto error;
+    }
+
+    fclose(f);
+
+    if (len > pattern_len) {
+        len -= pattern_len;
+        buf += pattern_len;
+
+        while (len > 0) {
+            size_t len_to_copy = MIN(pattern_len, len);
+
+            memcpy(buf, buf_origin, len_to_copy);
+
+            len -= len_to_copy;
+            buf += len_to_copy;
+        }
+    }
+
+    return buf_origin;
+
+error:
+    qemu_io_free(buf_origin);
+    return NULL;
+}
+
 static void dump_buffer(const void *buffer, int64_t offset, int64_t len)
 {
     uint64_t i;
@@ -XXX,XX +XXX,XX @@ static void write_help(void)
 " -n, -- with -z, don't allow slow fallback\n"
 " -p, -- ignored for backwards compatibility\n"
 " -P, -- use different pattern to fill file\n"
+" -s, -- use a pattern file to fill the write buffer\n"
 " -C, -- report statistics in a machine parsable format\n"
 " -q, -- quiet mode, do not show I/O statistics\n"
 " -u, -- with -z, allow unmapping\n"
@@ -XXX,XX +XXX,XX @@ static const cmdinfo_t write_cmd = {
     .perm       = BLK_PERM_WRITE,
     .argmin     = 2,
     .argmax     = -1,
-    .args       = "[-bcCfnquz] [-P pattern] off len",
+    .args       = "[-bcCfnquz] [-P pattern | -s source_file] off len",
     .oneline    = "writes a number of bytes at a specified offset",
     .help       = write_help,
 };
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
 {
     struct timespec t1, t2;
     bool Cflag = false, qflag = false, bflag = false;
-    bool Pflag = false, zflag = false, cflag = false;
+    bool Pflag = false, zflag = false, cflag = false, sflag = false;
     int flags = 0;
     int c, cnt, ret;
     char *buf = NULL;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
     /* Some compilers get confused and warn if this is not initialized.  */
     int64_t total = 0;
     int pattern = 0xcd;
+    const char *file_name = NULL;
 
-    while ((c = getopt(argc, argv, "bcCfnpP:quz")) != -1) {
+    while ((c = getopt(argc, argv, "bcCfnpP:qs:uz")) != -1) {
         switch (c) {
         case 'b':
             bflag = true;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
         case 'q':
             qflag = true;
             break;
+        case 's':
+            sflag = true;
+            file_name = optarg;
+            break;
         case 'u':
             flags |= BDRV_REQ_MAY_UNMAP;
             break;
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
         return -EINVAL;
     }
 
-    if (zflag && Pflag) {
-        printf("-z and -P cannot be specified at the same time\n");
+    if (zflag + Pflag + sflag > 1) {
+        printf("Only one of -z, -P, and -s "
+               "can be specified at the same time\n");
         return -EINVAL;
     }
 
@@ -XXX,XX +XXX,XX @@ static int write_f(BlockBackend *blk, int argc, char **argv)
     }
 
     if (!zflag) {
-        buf = qemu_io_alloc(blk, count, pattern);
+        if (sflag) {
+            buf = qemu_io_alloc_from_file(blk, count, file_name);
+            if (!buf) {
+                return -EINVAL;
+            }
+        } else {
+            buf = qemu_io_alloc(blk, count, pattern);
+        }
     }
 
     clock_gettime(CLOCK_MONOTONIC, &t1);
-- 
2.21.0

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

It's wrong to OR shared permissions. It may lead to crash on further
permission updates.
Also, no needs to consider previously calculated permissions, as at
this point we already bind all new parents and bdrv_get_cumulative_perm
result is enough. So fix the bug by just set permissions by
bdrv_get_cumulative_perm result.

Bug was introduced in long ago 234ac1a9025, in 2.9.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190824100740.61635-1-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
 {
     BdrvChild *c, *next;
     GSList *list = NULL, *p;
-    uint64_t old_perm, old_shared;
     uint64_t perm = 0, shared = BLK_PERM_ALL;
     int ret;
 
@@ -XXX,XX +XXX,XX @@ void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
         bdrv_unref(from);
     }
 
-    bdrv_get_cumulative_perm(to, &old_perm, &old_shared);
-    bdrv_set_perm(to, old_perm | perm, old_shared | shared);
+    bdrv_get_cumulative_perm(to, &perm, &shared);
+    bdrv_set_perm(to, perm, shared);
 
 out:
     g_slist_free(list);
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

When creating an image with preallocation "off" or "falloc", the first
block of the image is typically not allocated. When using Gluster
storage backed by XFS filesystem, reading this block using direct I/O
succeeds regardless of request length, fooling alignment detection.

In this case we fallback to a safe value (4096) instead of the optimal
value (512), which may lead to unneeded data copying when aligning
requests.  Allocating the first block avoids the fallback.

Since we allocate the first block even with preallocation=off, we no
longer create images with zero disk size:

$ ./qemu-img create -f raw test.raw 1g
    Formatting 'test.raw', fmt=raw size=1073741824

$ ls -lhs test.raw
    4.0K -rw-r--r--. 1 nsoffer nsoffer 1.0G Aug 16 23:48 test.raw

And converting the image requires additional cluster:

$ ./qemu-img measure -f raw -O qcow2 test.raw
    required size: 458752
    fully allocated size: 1074135040

When using format like vmdk with multiple files per image, we allocate
one block per file:

$ ./qemu-img create -f vmdk -o subformat=twoGbMaxExtentFlat test.vmdk 4g
    Formatting 'test.vmdk', fmt=vmdk size=4294967296 compat6=off hwversion=undefined subformat=twoGbMaxExtentFlat

$ ls -lhs test*.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f001.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer 2.0G Aug 27 03:23 test-f002.vmdk
    4.0K -rw-r--r--. 1 nsoffer nsoffer  353 Aug 27 03:23 test.vmdk

I did quick performance test for copying disks with qemu-img convert to
new raw target image to Gluster storage with sector size of 512 bytes:

for i in $(seq 10); do
        rm -f dst.raw
        sleep 10
        time ./qemu-img convert -f raw -O raw -t none -T none src.raw dst.raw
    done

Here is a table comparing the total time spent:

Type    Before(s)   After(s)    Diff(%)
---------------------------------------
real      530.028    469.123      -11.4
user       17.204     10.768      -37.4
sys        17.881      7.011      -60.7

We can see very clear improvement in CPU usage.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827010528.8818-2-nsoffer@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/file-posix.c                            | 51 +++++++++++++++++++
 tests/qemu-iotests/059.out                    |  2 +-
 tests/qemu-iotests/{150.out => 150.out.qcow2} |  0
 tests/qemu-iotests/150.out.raw                | 12 +++++
 tests/qemu-iotests/175                        | 19 ++++---
 tests/qemu-iotests/175.out                    |  8 +--
 tests/qemu-iotests/178.out.qcow2              |  4 +-
 tests/qemu-iotests/221.out                    | 12 +++--
 tests/qemu-iotests/253.out                    | 12 +++--
 9 files changed, 99 insertions(+), 21 deletions(-)
 rename tests/qemu-iotests/{150.out => 150.out.qcow2} (100%)
 create mode 100644 tests/qemu-iotests/150.out.raw

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_discard(void *opaque)
     return ret;
 }
 
+/*
+ * Help alignment probing by allocating the first block.
+ *
+ * When reading with direct I/O from unallocated area on Gluster backed by XFS,
+ * reading succeeds regardless of request length. In this case we fallback to
+ * safe alignment which is not optimal. Allocating the first block avoids this
+ * fallback.
+ *
+ * fd may be opened with O_DIRECT, but we don't know the buffer alignment or
+ * request alignment, so we use safe values.
+ *
+ * Returns: 0 on success, -errno on failure. Since this is an optimization,
+ * caller may ignore failures.
+ */
+static int allocate_first_block(int fd, size_t max_size)
+{
+    size_t write_size = (max_size < MAX_BLOCKSIZE)
+        ? BDRV_SECTOR_SIZE
+        : MAX_BLOCKSIZE;
+    size_t max_align = MAX(MAX_BLOCKSIZE, getpagesize());
+    void *buf;
+    ssize_t n;
+    int ret;
+
+    buf = qemu_memalign(max_align, write_size);
+    memset(buf, 0, write_size);
+
+    do {
+        n = pwrite(fd, buf, write_size, 0);
+    } while (n == -1 && errno == EINTR);
+
+    ret = (n == -1) ? -errno : 0;
+
+    qemu_vfree(buf);
+    return ret;
+}
+
 static int handle_aiocb_truncate(void *opaque)
 {
     RawPosixAIOData *aiocb = opaque;
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
                 /* posix_fallocate() doesn't set errno. */
                 error_setg_errno(errp, -result,
                                  "Could not preallocate new data");
+            } else if (current_length == 0) {
+                /*
+                 * posix_fallocate() uses fallocate() if the filesystem
+                 * supports it, or fallback to manually writing zeroes. If
+                 * fallocate() was used, unaligned reads from the fallocated
+                 * area in raw_probe_alignment() will succeed, hence we need to
+                 * allocate the first block.
+                 *
+                 * Optimize future alignment probing; ignore failures.
+                 */
+                allocate_first_block(fd, offset);
             }
         } else {
             result = 0;
@@ -XXX,XX +XXX,XX @@ static int handle_aiocb_truncate(void *opaque)
         if (ftruncate(fd, offset) != 0) {
             result = -errno;
             error_setg_errno(errp, -result, "Could not resize file");
+        } else if (current_length == 0 && offset > current_length) {
+            /* Optimize future alignment probing; ignore failures. */
+            allocate_first_block(fd, offset);
         }
         return result;
     default:
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMax
 image: TEST_DIR/t.vmdk
 file format: vmdk
 virtual size: 0.977 TiB (1073741824000 bytes)
-disk size: 16 KiB
+disk size: 1.97 MiB
 Format specific information:
     cid: XXXXXXXX
     parent cid: XXXXXXXX
diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out.qcow2
similarity index 100%
rename from tests/qemu-iotests/150.out
rename to tests/qemu-iotests/150.out.qcow2
diff --git a/tests/qemu-iotests/150.out.raw b/tests/qemu-iotests/150.out.raw
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/150.out.raw
@@ -XXX,XX +XXX,XX @@
+QA output created by 150
+
+=== Mapping sparse conversion ===
+
+Offset          Length          File
+0               0x1000          TEST_DIR/t.IMGFMT
+
+=== Mapping non-sparse conversion ===
+
+Offset          Length          File
+0               0x100000        TEST_DIR/t.IMGFMT
+*** done
diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # the file size.  This function hides the resulting difference in the
 # stat -c '%b' output.
 # Parameter 1: Number of blocks an empty file occupies
-# Parameter 2: Image size in bytes
+# Parameter 2: Minimal number of blocks in an image
+# Parameter 3: Image size in bytes
 _filter_blocks()
 {
     extra_blocks=$1
-    img_size=$2
+    min_blocks=$2
+    img_size=$3
 
-    sed -e "s/blocks=$extra_blocks\$\$\\|[^0-9]\$/nothing allocated/" \
-        -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/everything allocated/"
+    sed -e "s/blocks=$min_blocks\$\$\\|[^0-9]\$/min allocation/" \
+        -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/max allocation/"
 }
 
 # get standard environment, filters and checks
@@ -XXX,XX +XXX,XX @@ size=$((1 * 1024 * 1024))
 touch "$TEST_DIR/empty"
 extra_blocks=$(stat -c '%b' "$TEST_DIR/empty")
 
+# We always write the first byte; check how many blocks this filesystem
+# allocates to match empty image alloation.
+printf "\0" > "$TEST_DIR/empty"
+min_blocks=$(stat -c '%b' "$TEST_DIR/empty")
+
 echo
 echo "== creating image with default preallocation =="
 _make_test_img $size | _filter_imgfmt
-stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
+stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 
 for mode in off full falloc; do
     echo
     echo "== creating image with preallocation $mode =="
     IMGOPTS=preallocation=$mode _make_test_img $size | _filter_imgfmt
-    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $size
+    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 done
 
 # success, all done
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ QA output created by 175
 
 == creating image with default preallocation ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
-size=1048576, nothing allocated
+size=1048576, min allocation
 
 == creating image with preallocation off ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=off
-size=1048576, nothing allocated
+size=1048576, min allocation
 
 == creating image with preallocation full ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=full
-size=1048576, everything allocated
+size=1048576, max allocation
 
 == creating image with preallocation falloc ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
-size=1048576, everything allocated
+size=1048576, max allocation
  *** done
diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/178.out.qcow2
+++ b/tests/qemu-iotests/178.out.qcow2
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
 == raw input image with data (human) ==
 
 Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
-required size: 393216
+required size: 458752
 fully allocated size: 1074135040
 wrote 512/512 bytes at offset 512
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ converted image file size in bytes: 196608
 
 Formatting 'TEST_DIR/t.qcow2', fmt=IMGFMT size=1073741824
 {
-    "required": 393216,
+    "required": 458752,
     "fully-allocated": 1074135040
 }
 wrote 512/512 bytes at offset 512
diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/221.out
+++ b/tests/qemu-iotests/221.out
@@ -XXX,XX +XXX,XX @@ QA output created by 221
 === Check mapping of unaligned raw image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537
-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 wrote 1/1 bytes at offset 65536
 1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
 { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 *** done
diff --git a/tests/qemu-iotests/253.out b/tests/qemu-iotests/253.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/253.out
+++ b/tests/qemu-iotests/253.out
@@ -XXX,XX +XXX,XX @@ QA output created by 253
 === Check mapping of unaligned raw image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048575
-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
-[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 1044480, "depth": 0, "zero": true, "data": false, "offset": OFFSET}]
 wrote 65535/65535 bytes at offset 983040
 63.999 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
-[{ "start": 0, "length": 983040, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
+[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET},
+{ "start": 4096, "length": 978944, "depth": 0, "zero": true, "data": false, "offset": OFFSET},
 { "start": 983040, "length": 65536, "depth": 0, "zero": false, "data": true, "offset": OFFSET}]
 *** done
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Using block_resize we can test allocate_first_block() with file
descriptor opened with O_DIRECT, ensuring that it works for any size
larger than 4096 bytes.

Testing smaller sizes is tricky as the result depends on the filesystem
used for testing. For example on NFS any size will work since O_DIRECT
does not require any alignment.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190827010528.8818-3-nsoffer@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/175     | 28 ++++++++++++++++++++++++++++
 tests/qemu-iotests/175.out |  8 ++++++++
 2 files changed, 36 insertions(+)

diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/175
+++ b/tests/qemu-iotests/175
@@ -XXX,XX +XXX,XX @@ _filter_blocks()
         -e "s/blocks=$((extra_blocks + img_size / 512))\$\$\\|[^0-9]\$/max allocation/"
 }
 
+# Resize image using block_resize.
+# Parameter 1: image path
+# Parameter 2: new size
+_block_resize()
+{
+    local path=$1
+    local size=$2
+
+    $QEMU -qmp stdio -nographic -nodefaults \
+        -blockdev file,node-name=file,filename=$path,cache.direct=on \
+        <<EOF
+{'execute': 'qmp_capabilities'}
+{'execute': 'block_resize', 'arguments': {'node-name': 'file', 'size': $size}}
+{'execute': 'quit'}
+EOF
+}
+
 # get standard environment, filters and checks
 . ./common.rc
 . ./common.filter
@@ -XXX,XX +XXX,XX @@ _supported_fmt raw
 _supported_proto file
 _supported_os Linux
 
+_default_cache_mode none
+_supported_cache_modes none directsync
+
 size=$((1 * 1024 * 1024))
 
 touch "$TEST_DIR/empty"
@@ -XXX,XX +XXX,XX @@ for mode in off full falloc; do
     stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size
 done
 
+for new_size in 4096 1048576; do
+    echo
+    echo "== resize empty image with block_resize =="
+    _make_test_img 0 | _filter_imgfmt
+    _block_resize $TEST_IMG $new_size >/dev/null
+    stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $new_size
+done
+
 # success, all done
 echo "*** done"
 rm -f $seq.full
diff --git a/tests/qemu-iotests/175.out b/tests/qemu-iotests/175.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/175.out
+++ b/tests/qemu-iotests/175.out
@@ -XXX,XX +XXX,XX @@ size=1048576, max allocation
 == creating image with preallocation falloc ==
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 preallocation=falloc
 size=1048576, max allocation
+
+== resize empty image with block_resize ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
+size=4096, min allocation
+
+== resize empty image with block_resize ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=0
+size=1048576, min allocation
  *** done
-- 
2.21.0

fe646693acc changed qemu-img create's output so that it no longer prints
single quotes around parameter values.  The subformat and adapter_type
filters in _filter_img_create() have never been adapted to that change.

Fixes: fe646693acc13ac48b98435d14149ab04dc597bc
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-2-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/059.out       | 16 ++++++++--------
 tests/qemu-iotests/common.filter |  4 ++--
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-io: can't open device TEST_DIR/t.vmdk: L1 size too big
 
 === Testing monolithicFlat creation and opening ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 2 GiB (2147483648 bytes)
 
 === Testing monolithicFlat with zeroed_grain ===
 qemu-img: TEST_DIR/t.IMGFMT: Flat image can't enable zeroed grain
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
 
 === Testing big twoGbMaxExtentFlat ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMaxExtentFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000
 image: TEST_DIR/t.vmdk
 file format: vmdk
 virtual size: 0.977 TiB (1073741824000 bytes)
@@ -XXX,XX +XXX,XX @@ Format specific information:
 qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Invalid extent line: RW 12582912 VMFS "dummy.IMGFMT" 1
 
 === Testing truncated sparse ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400
 qemu-img: Could not open 'TEST_DIR/t.IMGFMT': File truncated, expecting at least 13172736 bytes
 
 === Converting to streamOptimized from image with small cluster size===
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing monolithicFlat with internally generated JSON file name ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
 
 === Testing version 3 ===
@@ -XXX,XX +XXX,XX @@ read 512/512 bytes at offset 64931328
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing 4TB monolithicFlat creation and IO ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104
 image: TEST_DIR/t.IMGFMT
 file format: IMGFMT
 virtual size: 4 TiB (4398046511104 bytes)
@@ -XXX,XX +XXX,XX @@ read 1024/1024 bytes at offset 966367641600
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing qemu-img map on extents ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
 wrote 1024/1024 bytes at offset 65024
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 1024/1024 bytes at offset 2147483136
@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
 0               0x20000         0x3f0000        TEST_DIR/t.vmdk
 0x7fff0000      0x20000         0x410000        TEST_DIR/t.vmdk
 0x140000000     0x10000         0x430000        TEST_DIR/t.vmdk
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=twoGbMaxExtentSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
 wrote 1024/1024 bytes at offset 65024
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 1024/1024 bytes at offset 2147483136
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -XXX,XX +XXX,XX @@ _filter_img_create()
         -e "s# compat6=\$on\\|off\$##g" \
         -e "s# static=\$on\\|off\$##g" \
         -e "s# zeroed_grain=\$on\\|off\$##g" \
-        -e "s# subformat='[^']*'##g" \
-        -e "s# adapter_type='[^']*'##g" \
+        -e "s# subformat=[^ ]*##g" \
+        -e "s# adapter_type=[^ ]*##g" \
         -e "s# hwversion=[^ ]*##g" \
         -e "s# lazy_refcounts=\$on\\|off\$##g" \
         -e "s# block_size=[0-9]\\+##g" \
-- 
2.21.0

This makes iotest 033 pass with e.g. subformat=monolithicFlat.  It also
turns a former error in 059 into success.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190815153638.4600-3-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c               | 54 ++++++++++++++++++++++++--------------
 tests/qemu-iotests/059     |  7 +++--
 tests/qemu-iotests/059.out |  4 ++-
 3 files changed, 42 insertions(+), 23 deletions(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static const char *next_line(const char *s)
 }
 
 static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
-                              const char *desc_file_path, QDict *options,
-                              Error **errp)
+                              QDict *options, Error **errp)
 {
     int ret;
     int matches;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
     const char *p, *np;
     int64_t sectors = 0;
     int64_t flat_offset;
+    char *desc_file_dir = NULL;
     char *extent_path;
     BdrvChild *extent_file;
     BDRVVmdkState *s = bs->opaque;
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
             continue;
         }
 
-        if (!path_is_absolute(fname) && !path_has_protocol(fname) &&
-            !desc_file_path[0])
-        {
-            bdrv_refresh_filename(bs->file->bs);
-            error_setg(errp, "Cannot use relative extent paths with VMDK "
-                       "descriptor file '%s'", bs->file->bs->filename);
-            return -EINVAL;
-        }
+        if (path_is_absolute(fname)) {
+            extent_path = g_strdup(fname);
+        } else {
+            if (!desc_file_dir) {
+                desc_file_dir = bdrv_dirname(bs->file->bs, errp);
+                if (!desc_file_dir) {
+                    bdrv_refresh_filename(bs->file->bs);
+                    error_prepend(errp, "Cannot use relative paths with VMDK "
+                                  "descriptor file '%s': ",
+                                  bs->file->bs->filename);
+                    ret = -EINVAL;
+                    goto out;
+                }
+            }
 
-        extent_path = path_combine(desc_file_path, fname);
+            extent_path = g_strconcat(desc_file_dir, fname, NULL);
+        }
 
         ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
         assert(ret < 32);
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
         g_free(extent_path);
         if (local_err) {
             error_propagate(errp, local_err);
-            return -EINVAL;
+            ret = -EINVAL;
+            goto out;
         }
 
         /* save to extents array */
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
                             0, 0, 0, 0, 0, &extent, errp);
             if (ret < 0) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent->flat_start_offset = flat_offset << 9;
         } else if (!strcmp(type, "SPARSE") || !strcmp(type, "VMFSSPARSE")) {
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
             g_free(buf);
             if (ret) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent = &s->extents[s->num_extents - 1];
         } else if (!strcmp(type, "SESPARSE")) {
             ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
             if (ret) {
                 bdrv_unref_child(bs, extent_file);
-                return ret;
+                goto out;
             }
             extent = &s->extents[s->num_extents - 1];
         } else {
             error_setg(errp, "Unsupported extent type '%s'", type);
             bdrv_unref_child(bs, extent_file);
-            return -ENOTSUP;
+            ret = -ENOTSUP;
+            goto out;
         }
         extent->type = g_strdup(type);
     }
-    return 0;
+
+    ret = 0;
+    goto out;
 
 invalid:
     np = next_line(p);
@@ -XXX,XX +XXX,XX @@ invalid:
         np--;
     }
     error_setg(errp, "Invalid extent line: %.*s", (int)(np - p), p);
-    return -EINVAL;
+    ret = -EINVAL;
+
+out:
+    g_free(desc_file_dir);
+    return ret;
 }
 
 static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
     }
     s->create_type = g_strdup(ct);
     s->desc_offset = 0;
-    ret = vmdk_parse_extents(buf, bs, bs->file->bs->exact_filename, options,
-                             errp);
+    ret = vmdk_parse_extents(buf, bs, options, errp);
 exit:
     return ret;
 }
diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
 
 echo
 echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+# Should work, because bdrv_dirname() works fine with blkdebug
 IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
-$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" 2>&1 \
-    | _filter_testdir | _filter_imgfmt
+$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
+         -c info \
+    2>&1 \
+    | _filter_testdir | _filter_imgfmt | _filter_img_info
 _cleanup_test_img
 
 echo
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 
 === Testing monolithicFlat with internally generated JSON file name ===
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
+format name: IMGFMT
+cluster size: 0 bytes
+vm state offset: 0 bytes
 
 === Testing version 3 ===
 image: TEST_DIR/iotest-version3.IMGFMT
-- 
2.21.0

We had a test for a case where relative extent paths did not work, but
unfortunately we just fixed the underlying problem, so it works now.
This patch adds a new test case that still fails.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-4-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/059     | 27 +++++++++++++++++++++++++++
 tests/qemu-iotests/059.out |  4 ++++
 2 files changed, 31 insertions(+)

diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -XXX,XX +XXX,XX @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
 
 echo
 echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+
+echo '--- blkdebug ---'
 # Should work, because bdrv_dirname() works fine with blkdebug
 IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
 $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
@@ -XXX,XX +XXX,XX @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TE
     | _filter_testdir | _filter_imgfmt | _filter_img_info
 _cleanup_test_img
 
+echo '--- quorum ---'
+# Should not work, because bdrv_dirname() does not work with quorum
+IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
+cp "$TEST_IMG" "$TEST_IMG.orig"
+
+filename="json:{
+    \"driver\": \"$IMGFMT\",
+    \"file\": {
+        \"driver\": \"quorum\",
+        \"children\": [ {
+            \"driver\": \"file\",
+            \"filename\": \"$TEST_IMG\"
+        }, {
+            \"driver\": \"file\",
+            \"filename\": \"$TEST_IMG.orig\"
+        } ],
+        \"vote-threshold\": 1
+    } }"
+
+filename=$(echo "$filename" | tr '\n' ' ' | sed -e 's/\s\+/ /g')
+$QEMU_IMG info "$filename" 2>&1 \
+    | sed -e "s/'json:[^']*'/\$QUORUM_FILE/g" \
+    | _filter_testdir | _filter_imgfmt | _filter_img_info
+
+
 echo
 echo "=== Testing version 3 ==="
 _use_sample_img iotest-version3.vmdk.bz2
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ wrote 512/512 bytes at offset 10240
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 === Testing monolithicFlat with internally generated JSON file name ===
+--- blkdebug ---
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 format name: IMGFMT
 cluster size: 0 bytes
 vm state offset: 0 bytes
+--- quorum ---
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+qemu-img: Could not open $QUORUM_FILE: Cannot use relative paths with VMDK descriptor file $QUORUM_FILE: Cannot generate a base directory for quorum nodes
 
 === Testing version 3 ===
 image: TEST_DIR/iotest-version3.IMGFMT
-- 
2.21.0

Compressed writes generally have to write full clusters, not just in
theory but also in practice when it comes to vmdk's streamOptimized
subformat.  It currently is just silently broken for writes with
non-zero in-cluster offsets:

$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 4k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 4096
4 KiB, 1 ops; 00.01 sec (443.724 KiB/sec and 110.9309 ops/sec)
read failed: Invalid argument

(The technical reason is that vmdk_write_extent() just writes the
incomplete compressed data actually to offset 4k.  When reading the
data, vmdk_read_extent() looks at offset 0 and finds the compressed data
size to be 0, because that is what it reads from there.  This yields an
error.)

For incomplete writes with zero in-cluster offsets, the error path when
reading the rest of the cluster is a bit different, but the result is
the same:

$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 0k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 0
4 KiB, 1 ops; 00.01 sec (362.641 KiB/sec and 90.6603 ops/sec)
read failed: Invalid argument

(Here, vmdk_read_extent() finds the data and then sees that the
uncompressed data is short.)

It is better to reject invalid writes than to make the user believe they
might have succeeded and then fail when trying to read it back.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-5-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset,
     if (extent->compressed) {
         void *compressed_data;
 
+        /* Only whole clusters */
+        if (offset_in_cluster ||
+            n_bytes > (extent->cluster_sectors * SECTOR_SIZE) ||
+            (n_bytes < (extent->cluster_sectors * SECTOR_SIZE) &&
+             offset + n_bytes != extent->end_sector * SECTOR_SIZE))
+        {
+            ret = -EINVAL;
+            goto out;
+        }
+
         if (!extent->has_marker) {
             ret = -EINVAL;
             goto out;
-- 
2.21.0

streamOptimized does not support writes that do not span exactly one
cluster.  Furthermore, it cannot rewrite already allocated clusters.
As such, many iotests do not work with it.  Disable them.

diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/002
+++ b/tests/qemu-iotests/002
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/003
+++ b/tests/qemu-iotests/003
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 size=128M
 offset=67M
diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 _supported_os Linux
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 # vpc is limited to 127GB, so we can't test it here
 if [ "$IMGFMT" = "vpc" ]; then
diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/009
+++ b/tests/qemu-iotests/009
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/010
+++ b/tests/qemu-iotests/010
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/011
+++ b/tests/qemu-iotests/011
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=6G
diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/017
+++ b/tests/qemu-iotests/017
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto generic
 _unsupported_proto vxhs
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/018
+++ b/tests/qemu-iotests/018
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _supported_os Linux
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/019
+++ b/tests/qemu-iotests/019
@@ -XXX,XX +XXX,XX @@ _supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 CLUSTER_SIZE=65536
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 TEST_OFFSETS="0 4294967296"
 
diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/027
+++ b/tests/qemu-iotests/027
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vmdk qcow qcow2 qed
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/032
+++ b/tests/qemu-iotests/032
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # This works for any image format (though unlikely to segfault for raw)
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo === Prepare image ===
diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt generic
 _supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
 
 
 size=128M
diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/034
+++ b/tests/qemu-iotests/034
@@ -XXX,XX +XXX,XX @@ _supported_proto file
 _supported_os Linux
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 CLUSTER_SIZE=4k
 size=128M
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 CLUSTER_SIZE=4k
 size=128M
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -XXX,XX +XXX,XX @@ _supported_fmt qcow qcow2 vmdk qed raw
 _supported_proto file
 _unsupported_imgopts "subformat=monolithicFlat" \
                      "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 _make_test_img 4M
 
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
 _supported_proto file
+_unsupported_imgopts "subformat=streamOptimized"
 
 IMG_SIZE=64M
 
diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/105
+++ b/tests/qemu-iotests/105
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2 vmdk vhdx qed
 _supported_proto generic
 _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
-                     "subformat=twoGbMaxExtentSparse"
+                     "subformat=twoGbMaxExtentSparse" \
+                     "subformat=streamOptimized"
 
 echo
 echo "creating large image"
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto generic
 # LUKS support may be possible, but it complicates things.
 _unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
 
 echo
 echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/251
+++ b/tests/qemu-iotests/251
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt generic
 _supported_proto file
 _supported_os Linux
+_unsupported_imgopts "subformat=streamOptimized"
 
 if [ "$IMGOPTSSYNTAX" = "true" ]; then
     # We use json:{} filenames here, so we cannot work with additional options.
-- 
2.21.0

The error message for the test case where we have a quorum node for
which no directory name can be generated is different: For
twoGbMaxExtentSparse, it complains that it cannot open the extent file.
For other (sub)formats, it just notes that it cannot determine the
backing file path.  Both are fine, but just disable twoGbMaxExtentSparse
for simplicity's sake.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190815153638.4600-7-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/110 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Any format supporting backing files
 _supported_fmt qed qcow qcow2 vmdk
 _supported_proto file
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+                     "subformat=twoGbMaxExtentSparse"
 
 TEST_IMG_REL=$(basename "$TEST_IMG")
 
-- 
2.21.0

From: Stefan Hajnoczi <stefanha@redhat.com>

Fixes: a6b257a08e3d72219f03e461a52152672fec0612
       ("file-posix: Handle undetectable alignment")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20190827101328.4062-1-stefanha@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/file-posix.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index XXXXXXX..XXXXXXX 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -XXX,XX +XXX,XX @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
             align = alignments[i];
             if (raw_is_io_aligned(fd, buf + align, max_align)) {
-                /* Fallback to request_aligment. */
+                /* Fallback to request_alignment. */
                 s->buf_align = (align != 1) ? align : bs->bl.request_alignment;
                 break;
             }
-- 
2.21.0

Without this argument, qemu will print an angry message about not being
able to connect to a display server if $DISPLAY is not set.  For me,
that breaks iotests.supported_formats() because it thus only sees
["Could", "not", "connect"] as the supported formats.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20190819201851.24418-2-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/check | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -XXX,XX +XXX,XX @@ export QEMU_PROG="$(type -p "$QEMU_PROG")"
 
 case "$QEMU_PROG" in
     *qemu-system-arm|*qemu-system-aarch64)
-        export QEMU_OPTIONS="-nodefaults -machine virt,accel=qtest"
+        export QEMU_OPTIONS="-nodefaults -display none -machine virt,accel=qtest"
         ;;
     *qemu-system-tricore)
-        export QEMU_OPTIONS="-nodefaults -machine tricore_testboard,accel=qtest"
+        export QEMU_OPTIONS="-nodefaults -display none -machine tricore_testboard,accel=qtest"
         ;;
     *)
-        export QEMU_OPTIONS="-nodefaults -machine accel=qtest"
+        export QEMU_OPTIONS="-nodefaults -display none -machine accel=qtest"
         ;;
 esac
 
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

It is possible to enable only a subset of the block drivers with the
"--block-drv-rw-whitelist" option of the "configure" script. All other
drivers are marked as unusable (or only included as read-only with the
"--block-drv-ro-whitelist" option). If an iotest is now using such a
disabled block driver, it is failing - which is bad, since at least the
tests in the "auto" group should be able to deal with this situation.
Thus let's introduce a "_require_drivers" function that can be used by
the shell tests to check for the availability of certain drivers first,
and marks the test as "not run" if one of the drivers is missing.

This patch mainly targets the test in the "auto" group which should
never fail in such a case, but also improves some of the other tests
along the way. Note that we also assume that the "qcow2" and "file"
drivers are always available - otherwise it does not make sense to
run "make check-block" at all (which only tests with qcow2 by default).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190823133552.11680-1-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/071       |  1 +
 tests/qemu-iotests/081       |  4 +---
 tests/qemu-iotests/099       |  1 +
 tests/qemu-iotests/120       |  1 +
 tests/qemu-iotests/162       |  4 +---
 tests/qemu-iotests/184       |  1 +
 tests/qemu-iotests/186       |  1 +
 tests/qemu-iotests/common.rc | 14 ++++++++++++++
 8 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
+_require_drivers blkdebug blkverify
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/081
+++ b/tests/qemu-iotests/081
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt raw
 _supported_proto file
 _supported_os Linux
+_require_drivers quorum
 
 do_run_qemu()
 {
@@ -XXX,XX +XXX,XX @@ run_qemu()
                           | _filter_qemu_io | _filter_generated_node_ids
 }
 
-test_quorum=$($QEMU_IMG --help|grep quorum)
-[ "$test_quorum" = "" ] && _supported_fmt quorum
-
 quorum="driver=raw,file.driver=quorum,file.vote-threshold=2"
 quorum="$quorum,file.children.0.file.filename=$TEST_DIR/1.raw"
 quorum="$quorum,file.children.1.file.filename=$TEST_DIR/2.raw"
diff --git a/tests/qemu-iotests/099 b/tests/qemu-iotests/099
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/099
+++ b/tests/qemu-iotests/099
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow qcow2 qed vdi vhdx vmdk vpc
 _supported_proto file
 _supported_os Linux
+_require_drivers blkdebug blkverify
 _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
     "subformat=twoGbMaxExtentSparse"
 
diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/120
+++ b/tests/qemu-iotests/120
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt generic
 _supported_proto file
 _unsupported_fmt luks
+_require_drivers raw
 
 _make_test_img 64M
 
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_fmt generic
-
-test_ssh=$($QEMU_IMG --help | grep '^Supported formats:.* ssh$ \|$$')
-[ "$test_ssh" = "" ] && _notrun "ssh support required"
+_require_drivers ssh
 
 echo
 echo '=== NBD ==='
diff --git a/tests/qemu-iotests/184 b/tests/qemu-iotests/184
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/184
+++ b/tests/qemu-iotests/184
@@ -XXX,XX +XXX,XX @@ trap "exit \$status" 0 1 2 3 15
 . ./common.filter
 
 _supported_os Linux
+_require_drivers throttle
 
 do_run_qemu()
 {
diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/186
+++ b/tests/qemu-iotests/186
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 
 _supported_fmt qcow2
 _supported_proto file
+_require_drivers null-co
 
 if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then
     _notrun "Requires a PC machine"
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -XXX,XX +XXX,XX @@ _require_command()
     [ -x "$c" ] || _notrun "$1 utility required, skipped this test"
 }
 
+# Check that a set of drivers has been whitelisted in the QEMU binary
+#
+_require_drivers()
+{
+    available=$($QEMU -drive format=help | \
+                sed -e '/Supported formats:/!d' -e 's/Supported formats://')
+    for driver
+    do
+        if ! echo "$available" | grep -q " $driver$ \|$$"; then
+            _notrun "$driver not available"
+        fi
+    done
+}
+
 # make sure this script returns success
 true
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

The sanitizers (especially the address sanitizer from Clang) are
sometimes printing out warnings or false positives - this spoils
the output of the iotests, causing some of the tests to fail.
Thus let's skip the automatic iotests during "make check" when the
user configured QEMU with --enable-sanitizers.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190823084203.29734-1-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/check-block.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tests/check-block.sh b/tests/check-block.sh
index XXXXXXX..XXXXXXX 100755
--- a/tests/check-block.sh
+++ b/tests/check-block.sh
@@ -XXX,XX +XXX,XX @@ if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then
     exit 0
 fi
 
+if grep -q "CFLAGS.*-fsanitize" config-host.mak 2>/dev/null ; then
+    echo "Sanitizers are enabled ==> Not running the qemu-iotests."
+    exit 0
+fi
+
 if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
     echo "No qemu-system binary available ==> Not running the qemu-iotests."
     exit 0
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Quoting cache mode is not needed, and most tests use unquoted values.
Unify all test to use the same style.

Message-id: 20190827173432.7656-1-nsoffer@redhat.com
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/026 | 4 ++--
 tests/qemu-iotests/039 | 4 ++--
 tests/qemu-iotests/052 | 2 +-
 tests/qemu-iotests/091 | 4 ++--
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/026
+++ b/tests/qemu-iotests/026
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 # Currently only qcow2 supports rebasing
 _supported_fmt qcow2
 _supported_proto file
-_default_cache_mode "writethrough"
-_supported_cache_modes "writethrough" "none"
+_default_cache_mode writethrough
+_supported_cache_modes writethrough none
 # The refcount table tests expect a certain minimum width for refcount entries
 # (so that the refcount table actually needs to grow); that minimum is 16 bits,
 # being the default refcount entry width.
diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
-_default_cache_mode "writethrough"
-_supported_cache_modes "writethrough"
+_default_cache_mode writethrough
+_supported_cache_modes writethrough
 
 size=128M
 
diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/052
+++ b/tests/qemu-iotests/052
@@ -XXX,XX +XXX,XX @@ _supported_fmt generic
 _supported_proto file
 
 # Don't do O_DIRECT on tmpfs
-_supported_cache_modes "writeback" "writethrough" "unsafe"
+_supported_cache_modes writeback writethrough unsafe
 
 size=128M
 _make_test_img $size
diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/091
+++ b/tests/qemu-iotests/091
@@ -XXX,XX +XXX,XX @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
-_default_cache_mode "none"
-_supported_cache_modes "writethrough" "none" "writeback"
+_default_cache_mode none
+_supported_cache_modes writethrough none writeback
 
 size=1G
 
-- 
2.21.0