Series comparison

-[PULL 00/17] Block patches
+[PULL 00/23] Block patches
-The following changes since commit b150cb8f67bf491a49a1cb1c7da151eeacbdbcc9:
+The following changes since commit 9d662a6b22a0838a85c5432385f35db2488a33a5:
-  Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2020-09-29 13:18:54 +0100)
+  Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220305' into staging (2022-03-05 18:03:15 +0000)
 are available in the Git repository at:
-  https://gitlab.com/stefanha/qemu.git tags/block-pull-request
+  https://gitlab.com/hreitz/qemu.git tags/pull-block-2022-03-07
-for you to fetch changes up to bc47831ff28d6f5830c9c8d74220131dc54c5253:
+for you to fetch changes up to 743da0b401cdc3ee94bc519975e339a3cdbe0ad1:
-  util/vfio-helpers: Rework the IOVA allocator to avoid IOVA reserved regions (2020-09-30 10:23:05 +0100)
+  iotests/image-fleecing: test push backup with fleecing (2022-03-07 09:33:31 +0100)
 ----------------------------------------------------------------
-Pull request
+Block patches for 7.0-rc0:
+- New fleecing backup scheme
-Note I have switched from GitHub to GitLab.
+- iotest fixes
 - Fixes for the curl block driver
 - Fix for the preallocate block driver
 - IDE fix for zero-length TRIM requests
 ----------------------------------------------------------------
+Hanna Reitz (2):
+  ide: Increment BB in-flight counter for TRIM BH
+  iotests: Write test output to TEST_DIR
-Eric Auger (2):
+Peter Maydell (2):
-  util/vfio-helpers: Collect IOVA reserved regions
+  block/curl.c: Set error message string if curl_init_state() fails
-  util/vfio-helpers: Rework the IOVA allocator to avoid IOVA reserved
+  block/curl.c: Check error return from curl_easy_setopt()
     regions
-Philippe Mathieu-Daudé (6):
+Thomas Huth (2):
-  util/vfio-helpers: Pass page protections to qemu_vfio_pci_map_bar()
+  tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'
-  block/nvme: Map doorbells pages write-only
+  tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP mode
   block/nvme: Reduce I/O registers scope
   block/nvme: Drop NVMeRegs structure, directly use NvmeBar
   block/nvme: Use register definitions from 'block/nvme.h'
   block/nvme: Replace magic value by SCALE_MS definition
-Stefano Garzarella (1):
+Vladimir Sementsov-Ogievskiy (17):
-  docs: add 'io_uring' option to 'aio' param in qemu-options.hx
+  block: fix preallocate filter: don't do unaligned preallocate requests
   block/block-copy: move copy_bitmap initialization to
     block_copy_state_new()
   block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
   block/block-copy: block_copy_state_new(): add bitmap parameter
   block/copy-before-write: add bitmap open parameter
   block/block-copy: add block_copy_reset()
   block: intoduce reqlist
   block/reqlist: reqlist_find_conflict(): use ranges_overlap()
   block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
   block/reqlist: add reqlist_wait_all()
   block/io: introduce block driver snapshot-access API
   block: introduce snapshot-access block driver
   block: copy-before-write: realize snapshot-access API
   iotests/image-fleecing: add test-case for fleecing format node
   iotests.py: add qemu_io_pipe_and_status()
   iotests/image-fleecing: add test case with bitmap
   iotests/image-fleecing: test push backup with fleecing
-Vladimir Sementsov-Ogievskiy (8):
+ qapi/block-core.json                        |  14 +-
-  block: return error-code from bdrv_invalidate_cache
+ include/block/block-common.h                |   3 +-
-  block/io: refactor coroutine wrappers
+ include/block/block-copy.h                  |   2 +
-  block: declare some coroutine functions in block/coroutines.h
+ include/block/block_int-common.h            |  24 ++
-  scripts: add block-coroutine-wrapper.py
+ include/block/block_int-io.h                |   9 +
-  block: generate coroutine-wrapper code
+ include/block/dirty-bitmap.h                |   4 +-
-  block: drop bdrv_prwv
+ include/block/reqlist.h                     |  75 ++++++
-  block/io: refactor save/load vmstate
+ include/qemu/hbitmap.h                      |  12 +
-  include/block/block.h: drop non-ascii quotation mark
+ block/block-copy.c                          | 150 +++++------
+ block/copy-before-write.c                   | 265 +++++++++++++++++++-
- block/block-gen.h                      |  49 ++++
+ block/curl.c                                |  92 ++++---
- block/coroutines.h                     |  65 +++++
+ block/dirty-bitmap.c                        |  15 +-
- include/block/block.h                  |  36 ++-
+ block/io.c                                  |  76 ++++++
- include/qemu/vfio-helpers.h            |   2 +-
+ block/monitor/bitmap-qmp-cmds.c             |   5 +-
- block.c                                |  97 +------
+ block/preallocate.c                         |  15 +-
- block/io.c                             | 339 ++++---------------------
+ block/reqlist.c                             |  85 +++++++
- block/nvme.c                           |  73 +++---
+ block/snapshot-access.c                     | 132 ++++++++++
- tests/test-bdrv-drain.c                |   2 +-
+ hw/ide/core.c                               |   7 +
- util/vfio-helpers.c                    | 133 +++++++++-
+ util/hbitmap.c                              |  33 +++
- block/meson.build                      |   8 +
+ MAINTAINERS                                 |   5 +-
- docs/devel/block-coroutine-wrapper.rst |  54 ++++
+ block/meson.build                           |   2 +
- docs/devel/index.rst                   |   1 +
+ tests/qemu-iotests/040                      |   1 +
- qemu-options.hx                        |  10 +-
+ tests/qemu-iotests/257.out                  | 224 +++++++++++++++++
- scripts/block-coroutine-wrapper.py     | 188 ++++++++++++++
+ tests/qemu-iotests/common.rc                |   6 +-
-files changed, 629 insertions(+), 428 deletions(-)
+ tests/qemu-iotests/iotests.py               |   8 +-
- create mode 100644 block/block-gen.h
+ tests/qemu-iotests/testenv.py               |   5 +-
- create mode 100644 block/coroutines.h
+ tests/qemu-iotests/testrunner.py            |  19 +-
- create mode 100644 docs/devel/block-coroutine-wrapper.rst
+ tests/qemu-iotests/tests/image-fleecing     | 185 +++++++++++---
- create mode 100644 scripts/block-coroutine-wrapper.py
+ tests/qemu-iotests/tests/image-fleecing.out | 221 +++++++++++++++-
 files changed, 1499 insertions(+), 195 deletions(-)
  create mode 100644 include/block/reqlist.h
  create mode 100644 block/reqlist.c
  create mode 100644 block/snapshot-access.c
 --
-.26.2
+.34.1

-New patch
+[PULL 01/23] ide: Increment BB in-flight counter for TRIM BH
+When we still have an AIOCB registered for DMA operations, we try to
+settle the respective operation by draining the BlockBackend associated
+with the IDE device.
+However, this assumes that every DMA operation is associated with an
+increment of the BlockBackend’s in-flight counter (e.g. through some
+ongoing I/O operation), so that draining the BB until its in-flight
+counter reaches 0 will settle all DMA operations.  That is not the case:
+For TRIM, the guest can issue a zero-length operation that will not
+result in any I/O operation forwarded to the BlockBackend, and also not
+increment the in-flight counter in any other way.  In such a case,
+blk_drain() will be a no-op if no other operations are in flight.
+It is clear that if blk_drain() is a no-op, the value of
+s->bus->dma->aiocb will not change between checking it in the `if`
+condition and asserting that it is NULL after blk_drain().
+The particular problem is that ide_issue_trim() creates a BH
+(ide_trim_bh_cb()) to settle the TRIM request: iocb->common.cb() is
+ide_dma_cb(), which will either create a new request, or find the
+transfer to be done and call ide_set_inactive(), which clears
+s->bus->dma->aiocb.  Therefore, the blk_drain() must wait for
+ide_trim_bh_cb() to run, which currently it will not always do.
+To fix this issue, we increment the BlockBackend's in-flight counter
+when the TRIM operation begins (in ide_issue_trim(), when the
+ide_trim_bh_cb() BH is created) and decrement it when ide_trim_bh_cb()
+is done.
+Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2029980
+Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+Message-Id: <20220120142259.120189-1-hreitz@redhat.com>
+Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
+Reviewed-by: John Snow <jsnow@redhat.com>
+Tested-by: John Snow <jsnow@redhat.com>
+---
+ hw/ide/core.c | 7 +++++++
+file changed, 7 insertions(+)
+diff --git a/hw/ide/core.c b/hw/ide/core.c
+index XXXXXXX..XXXXXXX 100644
+--- a/hw/ide/core.c
++++ b/hw/ide/core.c
+@@ -XXX,XX +XXX,XX @@ static const AIOCBInfo trim_aiocb_info = {
+ static void ide_trim_bh_cb(void *opaque)
+ {
+     TrimAIOCB *iocb = opaque;
++    BlockBackend *blk = iocb->s->blk;
+     iocb->common.cb(iocb->common.opaque, iocb->ret);
+     qemu_bh_delete(iocb->bh);
+     iocb->bh = NULL;
+     qemu_aio_unref(iocb);
++
++    /* Paired with an increment in ide_issue_trim() */
++    blk_dec_in_flight(blk);
+ }
+ static void ide_issue_trim_cb(void *opaque, int ret)
+@@ -XXX,XX +XXX,XX @@ BlockAIOCB *ide_issue_trim(
+     IDEState *s = opaque;
+     TrimAIOCB *iocb;
++    /* Paired with a decrement in ide_trim_bh_cb() */
++    blk_inc_in_flight(s->blk);
++
+     iocb = blk_aio_get(&trim_aiocb_info, s->blk, cb, cb_opaque);
+     iocb->s = s;
+     iocb->bh = qemu_bh_new(ide_trim_bh_cb, iocb);
+--
+.34.1

-[PULL 05/17] block/nvme: Use register definitions from 'block/nvme.h'
+[PULL 02/23] block/curl.c: Set error message string if curl_init_state() fails
-From: Philippe Mathieu-Daudé <philmd@redhat.com>
+From: Peter Maydell <peter.maydell@linaro.org>
-Use the NVMe register definitions from "block/nvme.h" which
+In curl_open(), the 'out' label assumes that the state->errmsg string
-ease a bit reviewing the code while matching the datasheet.
+has been set (either by curl_easy_perform() or by manually copying a
 string into it); however if curl_init_state() fails we will jump to
 that label without setting the string.  Add the missing error string
 setup.
-Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+(We can't be specific about the cause of failure: the documentation
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+of curl_easy_init() just says "If this function returns NULL,
-Message-Id: <20200922083821.578519-6-philmd@redhat.com>
+something went wrong".)
 Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
 Message-Id: <20220222152341.850419-2-peter.maydell@linaro.org>
 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
 Reviewed-by: Hanna Reitz <hreitz@redhat.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- block/nvme.c | 21 +++++++++++----------
+ block/curl.c | 2 ++
-file changed, 11 insertions(+), 10 deletions(-)
+file changed, 2 insertions(+)
-diff --git a/block/nvme.c b/block/nvme.c
+diff --git a/block/curl.c b/block/curl.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/nvme.c
+--- a/block/curl.c
-+++ b/block/nvme.c
++++ b/block/curl.c
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
+@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
-      * Initialization". */
+     // Get file size
-     cap = le64_to_cpu(regs->cap);
+     if (curl_init_state(s, state) < 0) {
--    if (!(cap & (1ULL << 37))) {
++        pstrcpy(state->errmsg, CURL_ERROR_SIZE,
-+    if (!NVME_CAP_CSS(cap)) {
++                "curl library initialization failed.");
          error_setg(errp, "Device doesn't support NVMe command set");
          ret = -EINVAL;
          goto out;
      }
--    s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
--    s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
-+    s->page_size = MAX(4096, 1 << NVME_CAP_MPSMIN(cap));
-+    s->doorbell_scale = (4 << NVME_CAP_DSTRD(cap)) / sizeof(uint32_t);
-     bs->bl.opt_mem_alignment = s->page_size;
--    timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
-+    timeout_ms = MIN(500 * NVME_CAP_TO(cap), 30000);
-     /* Reset device to get a clean state. */
-     regs->cc = cpu_to_le32(le32_to_cpu(regs->cc) & 0xFE);
-     /* Wait for CSTS.RDY = 0. */
-     deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCALE_MS;
--    while (le32_to_cpu(regs->csts) & 0x1) {
-+    while (NVME_CSTS_RDY(le32_to_cpu(regs->csts))) {
-         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
-             error_setg(errp, "Timeout while waiting for device to reset (%"
-                              PRId64 " ms)",
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
-     }
-     s->nr_queues = 1;
-     QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000);
--    regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
-+    regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << AQA_ACQS_SHIFT) |
-+                            (NVME_QUEUE_SIZE << AQA_ASQS_SHIFT));
-     regs->asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
-     regs->acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
-     /* After setting up all control registers we can enable device now. */
--    regs->cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
--                              (ctz32(NVME_SQ_ENTRY_BYTES) << 16) |
--                              0x1);
-+    regs->cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << CC_IOCQES_SHIFT) |
-+                           (ctz32(NVME_SQ_ENTRY_BYTES) << CC_IOSQES_SHIFT) |
-+                           CC_EN_MASK);
-     /* Wait for CSTS.RDY = 1. */
-     now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
-     deadline = now + timeout_ms * 1000000;
--    while (!(le32_to_cpu(regs->csts) & 0x1)) {
-+    while (!NVME_CSTS_RDY(le32_to_cpu(regs->csts))) {
-         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
-             error_setg(errp, "Timeout while waiting for device to start (%"
-                              PRId64 " ms)",
 --
-.26.2
+.34.1

-New patch
+[PULL 03/23] block/curl.c: Check error return from curl_easy_setopt()
+From: Peter Maydell <peter.maydell@linaro.org>
+Coverity points out that we aren't checking the return value
+from curl_easy_setopt() for any of the calls to it we make
+in block/curl.c.
+Some of these options are documented as always succeeding (e.g.
+CURLOPT_VERBOSE) but others have documented failure cases (e.g.
+CURLOPT_URL).  For consistency we check every call, even the ones
+that theoretically cannot fail.
+Fixes: Coverity CID 1459336, 1459482, 1460331
+Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
+Message-Id: <20220222152341.850419-3-peter.maydell@linaro.org>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+---
+ block/curl.c | 90 +++++++++++++++++++++++++++++++++-------------------
+file changed, 57 insertions(+), 33 deletions(-)
+diff --git a/block/curl.c b/block/curl.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/curl.c
++++ b/block/curl.c
+@@ -XXX,XX +XXX,XX @@ static int curl_init_state(BDRVCURLState *s, CURLState *state)
+         if (!state->curl) {
+             return -EIO;
+         }
+-        curl_easy_setopt(state->curl, CURLOPT_URL, s->url);
+-        curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
+-                         (long) s->sslverify);
+-        curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYHOST,
+-                         s->sslverify ? 2L : 0L);
++        if (curl_easy_setopt(state->curl, CURLOPT_URL, s->url) ||
++            curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
++                             (long) s->sslverify) ||
++            curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYHOST,
++                             s->sslverify ? 2L : 0L)) {
++            goto err;
++        }
+         if (s->cookie) {
+-            curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie);
++            if (curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie)) {
++                goto err;
++            }
++        }
++        if (curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, (long)s->timeout) ||
++            curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
++                             (void *)curl_read_cb) ||
++            curl_easy_setopt(state->curl, CURLOPT_WRITEDATA, (void *)state) ||
++            curl_easy_setopt(state->curl, CURLOPT_PRIVATE, (void *)state) ||
++            curl_easy_setopt(state->curl, CURLOPT_AUTOREFERER, 1) ||
++            curl_easy_setopt(state->curl, CURLOPT_FOLLOWLOCATION, 1) ||
++            curl_easy_setopt(state->curl, CURLOPT_NOSIGNAL, 1) ||
++            curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg) ||
++            curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1)) {
++            goto err;
+         }
+-        curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, (long)s->timeout);
+-        curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
+-                         (void *)curl_read_cb);
+-        curl_easy_setopt(state->curl, CURLOPT_WRITEDATA, (void *)state);
+-        curl_easy_setopt(state->curl, CURLOPT_PRIVATE, (void *)state);
+-        curl_easy_setopt(state->curl, CURLOPT_AUTOREFERER, 1);
+-        curl_easy_setopt(state->curl, CURLOPT_FOLLOWLOCATION, 1);
+-        curl_easy_setopt(state->curl, CURLOPT_NOSIGNAL, 1);
+-        curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg);
+-        curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1);
+-
+         if (s->username) {
+-            curl_easy_setopt(state->curl, CURLOPT_USERNAME, s->username);
++            if (curl_easy_setopt(state->curl, CURLOPT_USERNAME, s->username)) {
++                goto err;
++            }
+         }
+         if (s->password) {
+-            curl_easy_setopt(state->curl, CURLOPT_PASSWORD, s->password);
++            if (curl_easy_setopt(state->curl, CURLOPT_PASSWORD, s->password)) {
++                goto err;
++            }
+         }
+         if (s->proxyusername) {
+-            curl_easy_setopt(state->curl,
+-                             CURLOPT_PROXYUSERNAME, s->proxyusername);
++            if (curl_easy_setopt(state->curl,
++                                 CURLOPT_PROXYUSERNAME, s->proxyusername)) {
++                goto err;
++            }
+         }
+         if (s->proxypassword) {
+-            curl_easy_setopt(state->curl,
+-                             CURLOPT_PROXYPASSWORD, s->proxypassword);
++            if (curl_easy_setopt(state->curl,
++                                 CURLOPT_PROXYPASSWORD, s->proxypassword)) {
++                goto err;
++            }
+         }
+         /* Restrict supported protocols to avoid security issues in the more
+@@ -XXX,XX +XXX,XX @@ static int curl_init_state(BDRVCURLState *s, CURLState *state)
+          * Restricting protocols is only supported from 7.19.4 upwards.
+          */
+ #if LIBCURL_VERSION_NUM >= 0x071304
+-        curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, PROTOCOLS);
+-        curl_easy_setopt(state->curl, CURLOPT_REDIR_PROTOCOLS, PROTOCOLS);
++        if (curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, PROTOCOLS) ||
++            curl_easy_setopt(state->curl, CURLOPT_REDIR_PROTOCOLS, PROTOCOLS)) {
++            goto err;
++        }
+ #endif
+ #ifdef DEBUG_VERBOSE
+-        curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1);
++        if (curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1)) {
++            goto err;
++        }
+ #endif
+     }
+     state->s = s;
+     return 0;
++
++err:
++    curl_easy_cleanup(state->curl);
++    state->curl = NULL;
++    return -EIO;
+ }
+ /* Called with s->mutex held.  */
+@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
+     }
+     s->accept_range = false;
+-    curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1);
+-    curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION,
+-                     curl_header_cb);
+-    curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s);
++    if (curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1) ||
++        curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION, curl_header_cb) ||
++        curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s)) {
++        pstrcpy(state->errmsg, CURL_ERROR_SIZE,
++                "curl library initialization failed.");
++        goto out;
++    }
+     if (curl_easy_perform(state->curl))
+         goto out;
+     if (curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &d)) {
+@@ -XXX,XX +XXX,XX @@ static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb)
+     snprintf(state->range, 127, "%" PRIu64 "-%" PRIu64, start, end);
+     trace_curl_setup_preadv(acb->bytes, start, state->range);
+-    curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
+-
+-    if (curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
++    if (curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range) ||
++        curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
+         state->acb[0] = NULL;
+         acb->ret = -EIO;
+--
+.34.1

-[PULL 09/17] block: declare some coroutine functions in block/coroutines.h
+[PULL 04/23] block: fix preallocate filter: don't do unaligned preallocate requests
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-We are going to keep coroutine-wrappers code (structure-packing
+There is a bug in handling BDRV_REQ_NO_WAIT flag: we still may wait in
-parameters, BDRV_POLL wrapper functions) in separate auto-generated
+wait_serialising_requests() if request is unaligned. And this is
-files. So, we'll need a header with declaration of original _co_
+possible for the only user of this flag (preallocate filter) if
-functions, for those which are static now. As well, we'll need
+underlying file is unaligned to its request_alignment on start.
-declarations for wrapper functions. Do these declarations now, as a
-preparation step.
+So, we have to fix preallocate filter to do only aligned preallocate
 requests.
 Next, we should fix generic block/io.c somehow. Keeping in mind that
 preallocate is the only user of BDRV_REQ_NO_WAIT and that we have to
 fix its behavior now, it seems more safe to just assert that we never
 use BDRV_REQ_NO_WAIT with unaligned requests and add corresponding
 comment. Let's do so.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Denis V. Lunev <den@openvz.org>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Message-Id: <20220215121609.38570-1-vsementsov@virtuozzo.com>
-Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+[hreitz: Rebased on block GS/IO split]
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20200924185414.28642-4-vsementsov@virtuozzo.com>
 ---
- block/coroutines.h | 67 ++++++++++++++++++++++++++++++++++++++++++++++
+ include/block/block-common.h |  3 ++-
- block.c            |  8 +++---
+ block/io.c                   |  4 ++++
- block/io.c         | 34 +++++++++++------------
+ block/preallocate.c          | 15 ++++++++++++---
-files changed, 88 insertions(+), 21 deletions(-)
+files changed, 18 insertions(+), 4 deletions(-)
  create mode 100644 block/coroutines.h
-diff --git a/block/coroutines.h b/block/coroutines.h
+diff --git a/include/block/block-common.h b/include/block/block-common.h
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/block/coroutines.h
@@ -XXX,XX +XXX,XX @@
 +/*
 + * Block layer I/O functions
 + *
 + * Copyright (c) 2003 Fabrice Bellard
 + *
 + * Permission is hereby granted, free of charge, to any person obtaining a copy
 + * of this software and associated documentation files (the "Software"), to deal
 + * in the Software without restriction, including without limitation the rights
 + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 + * copies of the Software, and to permit persons to whom the Software is
 + * furnished to do so, subject to the following conditions:
 + *
 + * The above copyright notice and this permission notice shall be included in
 + * all copies or substantial portions of the Software.
 + *
 + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
 + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 + * THE SOFTWARE.
 + */
 +
 +#ifndef BLOCK_COROUTINES_INT_H
 +#define BLOCK_COROUTINES_INT_H
 +
 +#include "block/block_int.h"
 +
 +int coroutine_fn bdrv_co_check(BlockDriverState *bs,
 +                               BdrvCheckResult *res, BdrvCheckMode fix);
 +int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp);
 +
 +int coroutine_fn
 +bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
 +             bool is_write, BdrvRequestFlags flags);
 +int
 +bdrv_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
 +          bool is_write, BdrvRequestFlags flags);
 +
 +int coroutine_fn
 +bdrv_co_common_block_status_above(BlockDriverState *bs,
 +                                  BlockDriverState *base,
 +                                  bool want_zero,
 +                                  int64_t offset,
 +                                  int64_t bytes,
 +                                  int64_t *pnum,
 +                                  int64_t *map,
 +                                  BlockDriverState **file);
 +int
 +bdrv_common_block_status_above(BlockDriverState *bs,
 +                               BlockDriverState *base,
 +                               bool want_zero,
 +                               int64_t offset,
 +                               int64_t bytes,
 +                               int64_t *pnum,
 +                               int64_t *map,
 +                               BlockDriverState **file);
 +
 +int coroutine_fn
 +bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
 +                   bool is_read);
 +int
 +bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
 +                bool is_read);
 +
 +#endif /* BLOCK_COROUTINES_INT_H */
 diff --git a/block.c b/block.c
 index XXXXXXX..XXXXXXX 100644
---- a/block.c
+--- a/include/block/block-common.h
-+++ b/block.c
++++ b/include/block/block-common.h
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ typedef enum {
- #include "qemu/timer.h"
- #include "qemu/cutils.h"
+     /*
- #include "qemu/id.h"
+      * If we need to wait for other requests, just fail immediately. Used
-+#include "block/coroutines.h"
+-     * only together with BDRV_REQ_SERIALISING.
++     * only together with BDRV_REQ_SERIALISING. Used only with requests aligned
- #ifdef CONFIG_BSD
++     * to request_alignment (corresponding assertions are in block/io.c).
- #include <sys/ioctl.h>
+      */
-@@ -XXX,XX +XXX,XX @@ static void bdrv_delete(BlockDriverState *bs)
+     BDRV_REQ_NO_WAIT = 0x400,
-  * free of errors) or -errno when an internal error occurred. The results of the
   * check are stored in res.
   */
 -static int coroutine_fn bdrv_co_check(BlockDriverState *bs,
 -                                      BdrvCheckResult *res, BdrvCheckMode fix)
 +int coroutine_fn bdrv_co_check(BlockDriverState *bs,
 +                               BdrvCheckResult *res, BdrvCheckMode fix)
  {
      if (bs->drv == NULL) {
          return -ENOMEDIUM;
@@ -XXX,XX +XXX,XX @@ void bdrv_init_with_whitelist(void)
      bdrv_init();
  }
 -static int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
 -                                                 Error **errp)
 +int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
  {
      BdrvChild *child, *parent;
      uint64_t perm, shared_perm;
 diff --git a/block/io.c b/block/io.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/io.c
 +++ b/block/io.c
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_do_zero_pwritev(BdrvChild *child,
- #include "block/blockjob.h"
- #include "block/blockjob_int.h"
+     padding = bdrv_init_padding(bs, offset, bytes, &pad);
- #include "block/block_int.h"
+     if (padding) {
-+#include "block/coroutines.h"
++        assert(!(flags & BDRV_REQ_NO_WAIT));
- #include "qemu/cutils.h"
+         bdrv_make_request_serialising(req, align);
- #include "qapi/error.h"
- #include "qemu/error-report.h"
+         bdrv_padding_rmw_read(child, req, &pad, true);
-@@ -XXX,XX +XXX,XX @@ typedef struct RwCo {
+@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child,
-     BdrvRequestFlags flags;
+          * serialize the request to prevent interactions of the
- } RwCo;
+          * widened region with other transactions.
+          */
--static int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
++        assert(!(flags & BDRV_REQ_NO_WAIT));
--                                     QEMUIOVector *qiov, bool is_write,
+         bdrv_make_request_serialising(&req, align);
--                                     BdrvRequestFlags flags)
+         bdrv_padding_rmw_read(child, &req, &pad, false);
-+int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
+     }
-+                              QEMUIOVector *qiov, bool is_write,
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_copy_range_internal(
-+                              BdrvRequestFlags flags)
+     /* TODO We can support BDRV_REQ_NO_FALLBACK here */
- {
+     assert(!(read_flags & BDRV_REQ_NO_FALLBACK));
-     if (is_write) {
+     assert(!(write_flags & BDRV_REQ_NO_FALLBACK));
-         return bdrv_co_pwritev(child, offset, qiov->size, qiov, flags);
++    assert(!(read_flags & BDRV_REQ_NO_WAIT));
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_rw_co_entry(void *opaque)
++    assert(!(write_flags & BDRV_REQ_NO_WAIT));
- /*
-  * Process a vectored synchronous request using coroutines
+     if (!dst || !dst->bs || !bdrv_is_inserted(dst->bs)) {
-  */
+         return -ENOMEDIUM;
--static int bdrv_prwv(BdrvChild *child, int64_t offset,
+diff --git a/block/preallocate.c b/block/preallocate.c
--                     QEMUIOVector *qiov, bool is_write,
+index XXXXXXX..XXXXXXX 100644
--                     BdrvRequestFlags flags)
+--- a/block/preallocate.c
-+int bdrv_prwv(BdrvChild *child, int64_t offset,
++++ b/block/preallocate.c
-+              QEMUIOVector *qiov, bool is_write,
+@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn handle_write(BlockDriverState *bs, int64_t offset,
-+              BdrvRequestFlags flags)
+     int64_t end = offset + bytes;
- {
+     int64_t prealloc_start, prealloc_end;
-     RwCo rwco = {
+     int ret;
-         .child = child,
++    uint32_t file_align = bs->file->bs->bl.request_alignment;
-@@ -XXX,XX +XXX,XX @@ early_out:
++    uint32_t prealloc_align = MAX(s->opts.prealloc_align, file_align);
-     return ret;
++
- }
++    assert(QEMU_IS_ALIGNED(prealloc_align, file_align));
--static int coroutine_fn
+     if (!has_prealloc_perms(bs)) {
-+int coroutine_fn
+         /* We don't have state neither should try to recover it */
- bdrv_co_common_block_status_above(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn handle_write(BlockDriverState *bs, int64_t offset,
-                                   BlockDriverState *base,
-                                   bool want_zero,
+     /* Now we want new preallocation, as request writes beyond s->file_end. */
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_block_status_above_co_entry(void *opaque)
-  *
+-    prealloc_start = want_merge_zero ? MIN(offset, s->file_end) : s->file_end;
-  * See bdrv_co_block_status_above() for details.
+-    prealloc_end = QEMU_ALIGN_UP(end + s->opts.prealloc_size,
-  */
+-                                 s->opts.prealloc_align);
--static int bdrv_common_block_status_above(BlockDriverState *bs,
++    prealloc_start = QEMU_ALIGN_UP(
--                                          BlockDriverState *base,
++            want_merge_zero ? MIN(offset, s->file_end) : s->file_end,
--                                          bool want_zero, int64_t offset,
++            file_align);
--                                          int64_t bytes, int64_t *pnum,
++    prealloc_end = QEMU_ALIGN_UP(
--                                          int64_t *map,
++            MAX(prealloc_start, end) + s->opts.prealloc_size,
--                                          BlockDriverState **file)
++            prealloc_align);
-+int bdrv_common_block_status_above(BlockDriverState *bs,
++
-+                                   BlockDriverState *base,
++    want_merge_zero = want_merge_zero && (prealloc_start <= offset);
-+                                   bool want_zero, int64_t offset,
-+                                   int64_t bytes, int64_t *pnum,
+     ret = bdrv_co_pwrite_zeroes(
-+                                   int64_t *map,
+             bs->file, prealloc_start, prealloc_end - prealloc_start,
 +                                   BlockDriverState **file)
  {
      BdrvCoBlockStatusData data = {
          .bs = bs,
@@ -XXX,XX +XXX,XX @@ typedef struct BdrvVmstateCo {
      bool                is_read;
  } BdrvVmstateCo;
 -static int coroutine_fn
 +int coroutine_fn
  bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
                     bool is_read)
  {
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_rw_vmstate_entry(void *opaque)
      return bdrv_co_rw_vmstate(co->bs, co->qiov, co->pos, co->is_read);
  }
 -static inline int
 -bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
 -                bool is_read)
 +int bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
 +                    bool is_read)
  {
      BdrvVmstateCo data = {
          .bs         = bs,
 --
-.26.2
+.34.1

-New patch
+[PULL 05/23] tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'
+From: Thomas Huth <thuth@redhat.com>
+iotest 040 already has some checks for the availability of the 'throttle'
+driver, but some new code has been added in the course of time that
+depends on 'throttle' but does not check for its availability. Add
+a check to the TestCommitWithFilters class so that this iotest now
+also passes again if 'throttle' has not been enabled in the QEMU
+binaries.
+Signed-off-by: Thomas Huth <thuth@redhat.com>
+Message-Id: <20220223123127.3206042-1-thuth@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+---
+ tests/qemu-iotests/040 | 1 +
+file changed, 1 insertion(+)
+diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
+index XXXXXXX..XXXXXXX 100755
+--- a/tests/qemu-iotests/040
++++ b/tests/qemu-iotests/040
+@@ -XXX,XX +XXX,XX @@ class TestCommitWithFilters(iotests.QMPTestCase):
+                              pattern_file)
+             self.assertFalse('Pattern verification failed' in result)
++    @iotests.skip_if_unsupported(['throttle'])
+     def setUp(self):
+         qemu_img('create', '-f', iotests.imgfmt, self.img0, '64M')
+         qemu_img('create', '-f', iotests.imgfmt, self.img1, '64M')
+--
+.34.1

-New patch
+[PULL 06/23] tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP mode
+From: Thomas Huth <thuth@redhat.com>
+In TAP mode, the stdout is reserved for the TAP protocol, so we
+have to make sure to mark other lines with a comment '#' character
+at the beginning to avoid that the TAP parser at the other end
+gets confused.
+To test this condition, run "configure" for example with:
+ --block-drv-rw-whitelist=copy-before-write,qcow2,raw,file,host_device,blkdebug,null-co,copy-on-read
+so that iotest 041 will report that some tests are not run due to
+the missing "quorum" driver. Without this change, "make check-block"
+fails since the meson tap parser gets confused by these messages.
+Signed-off-by: Thomas Huth <thuth@redhat.com>
+Message-Id: <20220223124353.3273898-1-thuth@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+---
+ tests/qemu-iotests/testrunner.py | 5 ++++-
+file changed, 4 insertions(+), 1 deletion(-)
+diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/testrunner.py
++++ b/tests/qemu-iotests/testrunner.py
+@@ -XXX,XX +XXX,XX @@ def run_test(self, test: str,
+                                  description=res.description)
+         if res.casenotrun:
+-            print(res.casenotrun)
++            if self.tap:
++                print('#' + res.casenotrun.replace('\n', '\n#'))
++            else:
++                print(res.casenotrun)
+         return res
+--
+.34.1

-New patch
+[PULL 07/23] iotests: Write test output to TEST_DIR
+Drop the use of OUTPUT_DIR (test/qemu-iotests under the build
+directory), and instead write test output files (.out.bad, .notrun, and
+.casenotrun) to TEST_DIR.
+With this, the same test can be run concurrently without the separate
+instances interfering, because they will need separate TEST_DIRs anyway.
+Running the same test separately is useful when running the iotests with
+various format/protocol combinations in parallel, or when you just want
+to aggressively exercise a single test (e.g. when it fails only
+sporadically).
+Putting this output into TEST_DIR means that it will stick around for
+inspection after the test run is done (though running the same test in
+the same TEST_DIR will overwrite it, just as it used to be); but given
+that TEST_DIR is a scratch directory, it should be clear that users can
+delete all of its content at any point.  (And if TEST_DIR is on tmpfs,
+it will just disappear on shutdown.)  Contrarily, alternative approaches
+that would put these output files into OUTPUT_DIR with some prefix to
+differentiate between separate test runs might easily lead to cluttering
+OUTPUT_DIR.
+(This change means OUTPUT_DIR is no longer written to by the iotests, so
+we can drop its usage altogether.)
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+Message-Id: <20220221172909.762858-1-hreitz@redhat.com>
+[hreitz: Simplified `Path(os.path.join(x, y))` to `Path(x, y)`, as
+         suggested by Vladimir; and rebased on 9086c7639822b6
+         ("tests/qemu-iotests: Rework the checks and spots using GNU
+         sed")]
+Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+---
+ tests/qemu-iotests/common.rc     |  6 +++---
+ tests/qemu-iotests/iotests.py    |  5 ++---
+ tests/qemu-iotests/testenv.py    |  5 +----
+ tests/qemu-iotests/testrunner.py | 14 ++++++++------
+files changed, 14 insertions(+), 16 deletions(-)
+diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/common.rc
++++ b/tests/qemu-iotests/common.rc
+@@ -XXX,XX +XXX,XX @@
+ # bail out, setting up .notrun file
+ _notrun()
+ {
+-    echo "$*" >"$OUTPUT_DIR/$seq.notrun"
++    echo "$*" >"$TEST_DIR/$seq.notrun"
+     echo "$seq not run: $*"
+     status=0
+     exit
+@@ -XXX,XX +XXX,XX @@ _img_info()
+ #
+ _casenotrun()
+ {
+-    echo "    [case not run] $*" >>"$OUTPUT_DIR/$seq.casenotrun"
++    echo "    [case not run] $*" >>"$TEST_DIR/$seq.casenotrun"
+ }
+ # just plain bail out
+ #
+ _fail()
+ {
+-    echo "$*" | tee -a "$OUTPUT_DIR/$seq.full"
++    echo "$*" | tee -a "$TEST_DIR/$seq.full"
+     echo "(see $seq.full for details)"
+     status=1
+     exit 1
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/iotests.py
++++ b/tests/qemu-iotests/iotests.py
+@@ -XXX,XX +XXX,XX @@
+ imgfmt = os.environ.get('IMGFMT', 'raw')
+ imgproto = os.environ.get('IMGPROTO', 'file')
+-output_dir = os.environ.get('OUTPUT_DIR', '.')
+ try:
+     test_dir = os.environ['TEST_DIR']
+@@ -XXX,XX +XXX,XX @@ def notrun(reason):
+     # Each test in qemu-iotests has a number ("seq")
+     seq = os.path.basename(sys.argv[0])
+-    with open('%s/%s.notrun' % (output_dir, seq), 'w', encoding='utf-8') \
++    with open('%s/%s.notrun' % (test_dir, seq), 'w', encoding='utf-8') \
+             as outfile:
+         outfile.write(reason + '\n')
+     logger.warning("%s not run: %s", seq, reason)
+@@ -XXX,XX +XXX,XX @@ def case_notrun(reason):
+     # Each test in qemu-iotests has a number ("seq")
+     seq = os.path.basename(sys.argv[0])
+-    with open('%s/%s.casenotrun' % (output_dir, seq), 'a', encoding='utf-8') \
++    with open('%s/%s.casenotrun' % (test_dir, seq), 'a', encoding='utf-8') \
+             as outfile:
+         outfile.write('    [case not run] ' + reason + '\n')
+diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/testenv.py
++++ b/tests/qemu-iotests/testenv.py
+@@ -XXX,XX +XXX,XX @@ class TestEnv(ContextManager['TestEnv']):
+     # pylint: disable=too-many-instance-attributes
+     env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
+-                     'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
++                     'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
+                      'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG',
+                      'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
+                      'QEMU_IO_OPTIONS', 'QEMU_IO_OPTIONS_NO_FMT',
+@@ -XXX,XX +XXX,XX @@ def init_directories(self) -> None:
+              TEST_DIR
+              SOCK_DIR
+              SAMPLE_IMG_DIR
+-             OUTPUT_DIR
+         """
+         # Path where qemu goodies live in this source tree.
+@@ -XXX,XX +XXX,XX @@ def init_directories(self) -> None:
+                                         os.path.join(self.source_iotests,
+                                                      'sample_images'))
+-        self.output_dir = os.getcwd()  # OUTPUT_DIR
+-
+     def init_binaries(self) -> None:
+         """Init binary path variables:
+              PYTHON (for bash tests)
+diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
+index XXXXXXX..XXXXXXX 100644
+--- a/tests/qemu-iotests/testrunner.py
++++ b/tests/qemu-iotests/testrunner.py
+@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
+         """
+         f_test = Path(test)
+-        f_bad = Path(f_test.name + '.out.bad')
+-        f_notrun = Path(f_test.name + '.notrun')
+-        f_casenotrun = Path(f_test.name + '.casenotrun')
+         f_reference = Path(self.find_reference(test))
+         if not f_test.exists():
+@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
+                               description='No qualified output '
+                                           f'(expected {f_reference})')
+-        for p in (f_bad, f_notrun, f_casenotrun):
+-            silent_unlink(p)
+-
+         args = [str(f_test.resolve())]
+         env = self.env.prepare_subprocess(args)
+         if mp:
+@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
+                 env[d] = os.path.join(env[d], f_test.name)
+                 Path(env[d]).mkdir(parents=True, exist_ok=True)
++        test_dir = env['TEST_DIR']
++        f_bad = Path(test_dir, f_test.name + '.out.bad')
++        f_notrun = Path(test_dir, f_test.name + '.notrun')
++        f_casenotrun = Path(test_dir, f_test.name + '.casenotrun')
++
++        for p in (f_notrun, f_casenotrun):
++            silent_unlink(p)
++
+         t0 = time.time()
+         with f_bad.open('w', encoding="utf-8") as f:
+             with subprocess.Popen(args, cwd=str(f_test.parent), env=env,
+--
+.34.1

-[PULL 01/17] util/vfio-helpers: Pass page protections to qemu_vfio_pci_map_bar()
+[PULL 08/23] block/block-copy: move copy_bitmap initialization to block_copy_state_new()
-From: Philippe Mathieu-Daudé <philmd@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Pages are currently mapped READ/WRITE. To be able to use different
+We are going to complicate bitmap initialization in the further
-protections, add a new argument to qemu_vfio_pci_map_bar().
+commit. And in future, backup job will be able to work without filter
 (when source is immutable), so we'll need same bitmap initialization in
 copy-before-write filter and in backup job. So, it's reasonable to do
 it in block-copy.
-Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+Note that for now cbw_open() is the only caller of
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+block_copy_state_new().
-Message-Id: <20200922083821.578519-2-philmd@redhat.com>
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Reviewed-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20220303194349.2304213-2-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- include/qemu/vfio-helpers.h | 2 +-
+ block/block-copy.c        | 1 +
- block/nvme.c                | 3 ++-
+ block/copy-before-write.c | 4 ----
- util/vfio-helpers.c         | 4 ++--
+files changed, 1 insertion(+), 4 deletions(-)
 files changed, 5 insertions(+), 4 deletions(-)
-diff --git a/include/qemu/vfio-helpers.h b/include/qemu/vfio-helpers.h
+diff --git a/block/block-copy.c b/block/block-copy.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/qemu/vfio-helpers.h
+--- a/block/block-copy.c
-+++ b/include/qemu/vfio-helpers.h
++++ b/block/block-copy.c
-@@ -XXX,XX +XXX,XX @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size,
+@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
- int qemu_vfio_dma_reset_temporary(QEMUVFIOState *s);
+         return NULL;
- void qemu_vfio_dma_unmap(QEMUVFIOState *s, void *host);
+     }
- void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index,
+     bdrv_disable_dirty_bitmap(copy_bitmap);
--                            uint64_t offset, uint64_t size,
++    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
-+                            uint64_t offset, uint64_t size, int prot,
-                             Error **errp);
+     /*
- void qemu_vfio_pci_unmap_bar(QEMUVFIOState *s, int index, void *bar,
+      * If source is in backing chain of target assume that target is going to be
-                              uint64_t offset, uint64_t size);
+diff --git a/block/copy-before-write.c b/block/copy-before-write.c
 diff --git a/block/nvme.c b/block/nvme.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/nvme.c
+--- a/block/copy-before-write.c
-+++ b/block/nvme.c
++++ b/block/copy-before-write.c
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
+@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
-         goto out;
+                     Error **errp)
  {
      BDRVCopyBeforeWriteState *s = bs->opaque;
 -    BdrvDirtyBitmap *copy_bitmap;
      bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
                                 BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
          return -EINVAL;
      }
--    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, NVME_BAR_SIZE, errp);
+-    copy_bitmap = block_copy_dirty_bitmap(s->bcs);
-+    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, NVME_BAR_SIZE,
+-    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
-+                                    PROT_READ | PROT_WRITE, errp);
+-
-     if (!s->regs) {
+     return 0;
-         ret = -EINVAL;
+ }
-         goto out;
 diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c
 index XXXXXXX..XXXXXXX 100644
 --- a/util/vfio-helpers.c
 +++ b/util/vfio-helpers.c
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_pci_init_bar(QEMUVFIOState *s, int index, Error **errp)
   * Map a PCI bar area.
   */
  void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index,
 -                            uint64_t offset, uint64_t size,
 +                            uint64_t offset, uint64_t size, int prot,
                              Error **errp)
  {
      void *p;
      assert_bar_index_valid(s, index);
      p = mmap(NULL, MIN(size, s->bar_region_info[index].size - offset),
 -             PROT_READ | PROT_WRITE, MAP_SHARED,
 +             prot, MAP_SHARED,
               s->device, s->bar_region_info[index].offset + offset);
      if (p == MAP_FAILED) {
          error_setg_errno(errp, errno, "Failed to map BAR region");
 --
-.26.2
+.34.1

-New patch
+[PULL 09/23] block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+That simplifies handling failure in existing code and in further new
+usage of bdrv_merge_dirty_bitmap().
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
+Message-Id: <20220303194349.2304213-3-vsementsov@virtuozzo.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
+---
+ include/block/dirty-bitmap.h    | 2 +-
+ block/dirty-bitmap.c            | 9 +++++++--
+ block/monitor/bitmap-qmp-cmds.c | 5 +----
+files changed, 9 insertions(+), 7 deletions(-)
+diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
+index XXXXXXX..XXXXXXX 100644
+--- a/include/block/dirty-bitmap.h
++++ b/include/block/dirty-bitmap.h
+@@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_set_persistence(BdrvDirtyBitmap *bitmap,
+                                        bool persistent);
+ void bdrv_dirty_bitmap_set_inconsistent(BdrvDirtyBitmap *bitmap);
+ void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, bool busy);
+-void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
++bool bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
+                              HBitmap **backup, Error **errp);
+ void bdrv_dirty_bitmap_skip_store(BdrvDirtyBitmap *bitmap, bool skip);
+ bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset);
+diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/dirty-bitmap.c
++++ b/block/dirty-bitmap.c
+@@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
+  * Ensures permissions on bitmaps are reasonable; use for public API.
+  *
+  * @backup: If provided, make a copy of dest here prior to merge.
++ *
++ * Returns true on success, false on failure. In case of failure bitmaps are
++ * untouched.
+  */
+-void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
++bool bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
+                              HBitmap **backup, Error **errp)
+ {
+-    bool ret;
++    bool ret = false;
+     bdrv_dirty_bitmaps_lock(dest->bs);
+     if (src->bs != dest->bs) {
+@@ -XXX,XX +XXX,XX @@ out:
+     if (src->bs != dest->bs) {
+         bdrv_dirty_bitmaps_unlock(src->bs);
+     }
++
++    return ret;
+ }
+ /**
+diff --git a/block/monitor/bitmap-qmp-cmds.c b/block/monitor/bitmap-qmp-cmds.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/monitor/bitmap-qmp-cmds.c
++++ b/block/monitor/bitmap-qmp-cmds.c
+@@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target,
+     BlockDriverState *bs;
+     BdrvDirtyBitmap *dst, *src, *anon;
+     BlockDirtyBitmapMergeSourceList *lst;
+-    Error *local_err = NULL;
+     GLOBAL_STATE_CODE();
+@@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target,
+             abort();
+         }
+-        bdrv_merge_dirty_bitmap(anon, src, NULL, &local_err);
+-        if (local_err) {
+-            error_propagate(errp, local_err);
++        if (!bdrv_merge_dirty_bitmap(anon, src, NULL, errp)) {
+             dst = NULL;
+             goto out;
+         }
+--
+.34.1

-[PULL 14/17] include/block/block.h: drop non-ascii quotation mark
+[PULL 10/23] block/block-copy: block_copy_state_new(): add bitmap parameter
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-This is the only non-ascii character in the file and it doesn't really
+This will be used in the following commit to bring "incremental" mode
-needed here. Let's use normal "'" symbol for consistency with the rest
+to copy-before-write filter.
 occurrences of "'" in the file.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-Id: <20220303194349.2304213-4-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- include/block/block.h | 2 +-
+ include/block/block-copy.h |  1 +
-file changed, 1 insertion(+), 1 deletion(-)
+ block/block-copy.c         | 14 +++++++++++++-
  block/copy-before-write.c  |  2 +-
 files changed, 15 insertions(+), 2 deletions(-)
-diff --git a/include/block/block.h b/include/block/block.h
+diff --git a/include/block/block-copy.h b/include/block/block-copy.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/block/block.h
+--- a/include/block/block-copy.h
-+++ b/include/block/block.h
++++ b/include/block/block-copy.h
-@@ -XXX,XX +XXX,XX @@ enum BdrvChildRoleBits {
+@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState BlockCopyState;
-     BDRV_CHILD_FILTERED     = (1 << 2),
+ typedef struct BlockCopyCallState BlockCopyCallState;
  BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
 +                                     const BdrvDirtyBitmap *bitmap,
                                       Error **errp);
  /* Function should be called prior any actual copy request */
 diff --git a/block/block-copy.c b/block/block-copy.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/block-copy.c
 +++ b/block/block-copy.c
@@ -XXX,XX +XXX,XX @@ static int64_t block_copy_calculate_cluster_size(BlockDriverState *target,
  }
  BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
 +                                     const BdrvDirtyBitmap *bitmap,
                                       Error **errp)
  {
 +    ERRP_GUARD();
      BlockCopyState *s;
      int64_t cluster_size;
      BdrvDirtyBitmap *copy_bitmap;
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
          return NULL;
      }
      bdrv_disable_dirty_bitmap(copy_bitmap);
 -    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
 +    if (bitmap) {
 +        if (!bdrv_merge_dirty_bitmap(copy_bitmap, bitmap, NULL, errp)) {
 +            error_prepend(errp, "Failed to merge bitmap '%s' to internal "
 +                          "copy-bitmap: ", bdrv_dirty_bitmap_name(bitmap));
 +            bdrv_release_dirty_bitmap(copy_bitmap);
 +            return NULL;
 +        }
 +    } else {
 +        bdrv_set_dirty_bitmap(copy_bitmap, 0,
 +                              bdrv_dirty_bitmap_size(copy_bitmap));
 +    }
      /*
--     * Child from which to read all data that isn’t allocated in the
+      * If source is in backing chain of target assume that target is going to be
-+     * Child from which to read all data that isn't allocated in the
+diff --git a/block/copy-before-write.c b/block/copy-before-write.c
-      * parent (i.e., the backing child); such data is copied to the
+index XXXXXXX..XXXXXXX 100644
-      * parent through COW (and optionally COR).
+--- a/block/copy-before-write.c
-      * This field is mutually exclusive with DATA, METADATA, and
++++ b/block/copy-before-write.c
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
              ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
               bs->file->bs->supported_zero_flags);
 -    s->bcs = block_copy_state_new(bs->file, s->target, errp);
 +    s->bcs = block_copy_state_new(bs->file, s->target, NULL, errp);
      if (!s->bcs) {
          error_prepend(errp, "Cannot create block-copy-state: ");
          return -EINVAL;
 --
-.26.2
+.34.1

-[PULL 16/17] util/vfio-helpers: Collect IOVA reserved regions
+[PULL 11/23] block/copy-before-write: add bitmap open parameter
-From: Eric Auger <eric.auger@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-The IOVA allocator currently ignores host reserved regions.
+This brings "incremental" mode to copy-before-write filter: user can
-As a result some chosen IOVAs may collide with some of them,
+specify bitmap so that filter will copy only "dirty" areas.
 resulting in VFIO MAP_DMA errors later on. This happens on ARM
 where the MSI reserved window quickly is encountered:
 [0x8000000, 0x8100000]. since 5.4 kernel, VFIO returns the usable
 IOVA regions. So let's enumerate them in the prospect to avoid
 them, later on.
-Signed-off-by: Eric Auger <eric.auger@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-id: 20200929085550.30926-2-eric.auger@redhat.com
+Message-Id: <20220303194349.2304213-5-vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- util/vfio-helpers.c | 72 +++++++++++++++++++++++++++++++++++++++++++--
+ qapi/block-core.json      | 10 +++++++-
-file changed, 70 insertions(+), 2 deletions(-)
+ block/copy-before-write.c | 51 ++++++++++++++++++++++++++++++++++++++-
 files changed, 59 insertions(+), 2 deletions(-)
-diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c
+diff --git a/qapi/block-core.json b/qapi/block-core.json
 index XXXXXXX..XXXXXXX 100644
---- a/util/vfio-helpers.c
+--- a/qapi/block-core.json
-+++ b/util/vfio-helpers.c
++++ b/qapi/block-core.json
-@@ -XXX,XX +XXX,XX @@ typedef struct {
+@@ -XXX,XX +XXX,XX @@
-     uint64_t iova;
+ #
- } IOVAMapping;
+ # @target: The target for copy-before-write operations.
+ #
-+struct IOVARange {
++# @bitmap: If specified, copy-before-write filter will do
-+    uint64_t start;
++#          copy-before-write operations only for dirty regions of the
-+    uint64_t end;
++#          bitmap. Bitmap size must be equal to length of file and
-+};
++#          target child of the filter. Note also, that bitmap is used
 +#          only to initialize internal bitmap of the process, so further
 +#          modifications (or removing) of specified bitmap doesn't
 +#          influence the filter. (Since 7.0)
 +#
  # Since: 6.2
  ##
  { 'struct': 'BlockdevOptionsCbw',
    'base': 'BlockdevOptionsGenericFormat',
 -  'data': { 'target': 'BlockdevRef' } }
 +  'data': { 'target': 'BlockdevRef', '*bitmap': 'BlockDirtyBitmap' } }
  ##
  # @BlockdevOptions:
 diff --git a/block/copy-before-write.c b/block/copy-before-write.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/copy-before-write.c
 +++ b/block/copy-before-write.c
@@ -XXX,XX +XXX,XX @@
  #include "block/copy-before-write.h"
 +#include "qapi/qapi-visit-block-core.h"
 +
- struct QEMUVFIOState {
+ typedef struct BDRVCopyBeforeWriteState {
-     QemuMutex lock;
+     BlockCopyState *bcs;
+     BdrvChild *target;
-@@ -XXX,XX +XXX,XX @@ struct QEMUVFIOState {
+@@ -XXX,XX +XXX,XX @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
-     int device;
+     }
      RAMBlockNotifier ram_notifier;
      struct vfio_region_info config_region_info, bar_region_info[6];
 +    struct IOVARange *usable_iova_ranges;
 +    uint8_t nb_iova_ranges;
      /* These fields are protected by @lock */
      /* VFIO's IO virtual address space is managed by splitting into a few
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_pci_write_config(QEMUVFIOState *s, void *buf, int size, int
      return ret == size ? 0 : -errno;
  }
-+static void collect_usable_iova_ranges(QEMUVFIOState *s, void *buf)
++static bool cbw_parse_bitmap_option(QDict *options, BdrvDirtyBitmap **bitmap,
 +                                    Error **errp)
 +{
-+    struct vfio_iommu_type1_info *info = (struct vfio_iommu_type1_info *)buf;
++    QDict *bitmap_qdict = NULL;
-+    struct vfio_info_cap_header *cap = (void *)buf + info->cap_offset;
++    BlockDirtyBitmap *bmp_param = NULL;
-+    struct vfio_iommu_type1_info_cap_iova_range *cap_iova_range;
++    Visitor *v = NULL;
-+    int i;
++    bool ret = false;
 +
-+    while (cap->id != VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE) {
++    *bitmap = NULL;
-+        if (!cap->next) {
++
-+            return;
++    qdict_extract_subqdict(options, &bitmap_qdict, "bitmap.");
-+        }
++    if (!qdict_size(bitmap_qdict)) {
-+        cap = (struct vfio_info_cap_header *)(buf + cap->next);
++        ret = true;
 +        goto out;
 +    }
 +
-+    cap_iova_range = (struct vfio_iommu_type1_info_cap_iova_range *)cap;
++    v = qobject_input_visitor_new_flat_confused(bitmap_qdict, errp);
-+
++    if (!v) {
-+    s->nb_iova_ranges = cap_iova_range->nr_iovas;
++        goto out;
 +    if (s->nb_iova_ranges > 1) {
 +        s->usable_iova_ranges =
 +            g_realloc(s->usable_iova_ranges,
 +                      s->nb_iova_ranges * sizeof(struct IOVARange));
 +    }
 +
-+    for (i = 0; i < s->nb_iova_ranges; i++) {
++    visit_type_BlockDirtyBitmap(v, NULL, &bmp_param, errp);
-+        s->usable_iova_ranges[i].start = cap_iova_range->iova_ranges[i].start;
++    if (!bmp_param) {
-+        s->usable_iova_ranges[i].end = cap_iova_range->iova_ranges[i].end;
++        goto out;
 +    }
++
++    *bitmap = block_dirty_bitmap_lookup(bmp_param->node, bmp_param->name, NULL,
++                                        errp);
++    if (!*bitmap) {
++        goto out;
++    }
++
++    ret = true;
++
++out:
++    qapi_free_BlockDirtyBitmap(bmp_param);
++    visit_free(v);
++    qobject_unref(bitmap_qdict);
++
++    return ret;
 +}
 +
- static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
+ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
-                               Error **errp)
+                     Error **errp)
  {
-@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
+     BDRVCopyBeforeWriteState *s = bs->opaque;
-     int i;
++    BdrvDirtyBitmap *bitmap = NULL;
-     uint16_t pci_cmd;
-     struct vfio_group_status group_status = { .argsz = sizeof(group_status) };
+     bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
--    struct vfio_iommu_type1_info iommu_info = { .argsz = sizeof(iommu_info) };
+                                BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
-+    struct vfio_iommu_type1_info *iommu_info = NULL;
+@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
-+    size_t iommu_info_size = sizeof(*iommu_info);
+         return -EINVAL;
      struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
      char *group_file = NULL;
 +    s->usable_iova_ranges = NULL;
 +
      /* Create a new container */
      s->container = open("/dev/vfio/vfio", O_RDWR);
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
          goto fail;
      }
-+    iommu_info = g_malloc0(iommu_info_size);
++    if (!cbw_parse_bitmap_option(options, &bitmap, errp)) {
-+    iommu_info->argsz = iommu_info_size;
++        return -EINVAL;
 +
      /* Get additional IOMMU info */
 -    if (ioctl(s->container, VFIO_IOMMU_GET_INFO, &iommu_info)) {
 +    if (ioctl(s->container, VFIO_IOMMU_GET_INFO, iommu_info)) {
          error_setg_errno(errp, errno, "Failed to get IOMMU info");
          ret = -errno;
          goto fail;
      }
 +    /*
 +     * if the kernel does not report usable IOVA regions, choose
 +     * the legacy [QEMU_VFIO_IOVA_MIN, QEMU_VFIO_IOVA_MAX -1] region
 +     */
 +    s->nb_iova_ranges = 1;
 +    s->usable_iova_ranges = g_new0(struct IOVARange, 1);
 +    s->usable_iova_ranges[0].start = QEMU_VFIO_IOVA_MIN;
 +    s->usable_iova_ranges[0].end = QEMU_VFIO_IOVA_MAX - 1;
 +
 +    if (iommu_info->argsz > iommu_info_size) {
 +        iommu_info_size = iommu_info->argsz;
 +        iommu_info = g_realloc(iommu_info, iommu_info_size);
 +        if (ioctl(s->container, VFIO_IOMMU_GET_INFO, iommu_info)) {
 +            ret = -errno;
 +            goto fail;
 +        }
 +        collect_usable_iova_ranges(s, iommu_info);
 +    }
 +
-     s->device = ioctl(s->group, VFIO_GROUP_GET_DEVICE_FD, device);
+     bs->total_sectors = bs->file->bs->total_sectors;
+     bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
-     if (s->device < 0) {
+             (BDRV_REQ_FUA & bs->file->bs->supported_write_flags);
-@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
+@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
-     if (ret) {
+             ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
-         goto fail;
+              bs->file->bs->supported_zero_flags);
-     }
-+    g_free(iommu_info);
+-    s->bcs = block_copy_state_new(bs->file, s->target, NULL, errp);
-     return 0;
++    s->bcs = block_copy_state_new(bs->file, s->target, bitmap, errp);
- fail:
+     if (!s->bcs) {
-+    g_free(s->usable_iova_ranges);
+         error_prepend(errp, "Cannot create block-copy-state: ");
-+    s->usable_iova_ranges = NULL;
+         return -EINVAL;
 +    s->nb_iova_ranges = 0;
 +    g_free(iommu_info);
      close(s->group);
  fail_container:
      close(s->container);
@@ -XXX,XX +XXX,XX @@ void qemu_vfio_close(QEMUVFIOState *s)
          qemu_vfio_undo_mapping(s, &s->mappings[i], NULL);
      }
      ram_block_notifier_remove(&s->ram_notifier);
 +    g_free(s->usable_iova_ranges);
 +    s->nb_iova_ranges = 0;
      qemu_vfio_reset(s);
      close(s->device);
      close(s->group);
 --
-.26.2
+.34.1

-[PULL 12/17] block: drop bdrv_prwv
+[PULL 12/23] block/block-copy: add block_copy_reset()
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Now that we are not maintaining boilerplate code for coroutine
+Split block_copy_reset() out of block_copy_reset_unallocated() to be
-wrappers, there is no more sense in keeping the extra indirection layer
+used separately later.
 of bdrv_prwv().  Let's drop it and instead generate pure bdrv_preadv()
 and bdrv_pwritev().
 Currently, bdrv_pwritev() and bdrv_preadv() are returning bytes on
 success, auto generated functions will instead return zero, as their
 _co_ prototype. Still, it's simple to make the conversion safe: the
 only external user of bdrv_pwritev() is test-bdrv-drain, and it is
 comfortable enough with bdrv_co_pwritev() instead. So prototypes are
 moved to local block/coroutines.h. Next, the only internal use is
 bdrv_pread() and bdrv_pwrite(), which are modified to return bytes on
 success.
 Of course, it would be great to convert bdrv_pread() and bdrv_pwrite()
 to return 0 on success. But this requires audit (and probably
 conversion) of all their users, let's leave it for another day
 refactoring.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Message-Id: <20220303194349.2304213-6-vsementsov@virtuozzo.com>
-Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Message-Id: <20200924185414.28642-7-vsementsov@virtuozzo.com>
 ---
- block/coroutines.h      | 10 ++++-----
+ include/block/block-copy.h |  1 +
- include/block/block.h   |  2 --
+ block/block-copy.c         | 21 +++++++++++++--------
- block/io.c              | 49 ++++++++---------------------------------
+files changed, 14 insertions(+), 8 deletions(-)
  tests/test-bdrv-drain.c |  2 +-
 files changed, 15 insertions(+), 48 deletions(-)
-diff --git a/block/coroutines.h b/block/coroutines.h
+diff --git a/include/block/block-copy.h b/include/block/block-copy.h
 index XXXXXXX..XXXXXXX 100644
---- a/block/coroutines.h
+--- a/include/block/block-copy.h
-+++ b/block/coroutines.h
++++ b/include/block/block-copy.h
-@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_check(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
-                                BdrvCheckResult *res, BdrvCheckMode fix);
- int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp);
+ void block_copy_state_free(BlockCopyState *s);
--int coroutine_fn
++void block_copy_reset(BlockCopyState *s, int64_t offset, int64_t bytes);
--bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
+ int64_t block_copy_reset_unallocated(BlockCopyState *s,
--             bool is_write, BdrvRequestFlags flags);
+                                      int64_t offset, int64_t *count);
- int generated_co_wrapper
--bdrv_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
+diff --git a/block/block-copy.c b/block/block-copy.c
 -          bool is_write, BdrvRequestFlags flags);
 +bdrv_preadv(BdrvChild *child, int64_t offset, unsigned int bytes,
 +            QEMUIOVector *qiov, BdrvRequestFlags flags);
 +int generated_co_wrapper
 +bdrv_pwritev(BdrvChild *child, int64_t offset, unsigned int bytes,
 +             QEMUIOVector *qiov, BdrvRequestFlags flags);
  int coroutine_fn
  bdrv_co_common_block_status_above(BlockDriverState *bs,
 diff --git a/include/block/block.h b/include/block/block.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/block/block.h
+--- a/block/block-copy.c
-+++ b/include/block/block.h
++++ b/block/block-copy.c
-@@ -XXX,XX +XXX,XX @@ int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
+@@ -XXX,XX +XXX,XX @@ static int block_copy_is_cluster_allocated(BlockCopyState *s, int64_t offset,
                         int bytes, BdrvRequestFlags flags);
  int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags);
  int bdrv_pread(BdrvChild *child, int64_t offset, void *buf, int bytes);
 -int bdrv_preadv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov);
  int bdrv_pwrite(BdrvChild *child, int64_t offset, const void *buf, int bytes);
 -int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov);
  int bdrv_pwrite_sync(BdrvChild *child, int64_t offset,
                       const void *buf, int count);
  /*
 diff --git a/block/io.c b/block/io.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/io.c
 +++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ static int bdrv_check_byte_request(BlockDriverState *bs, int64_t offset,
      return 0;
  }
 -int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
 -                              QEMUIOVector *qiov, bool is_write,
 -                              BdrvRequestFlags flags)
 -{
 -    if (is_write) {
 -        return bdrv_co_pwritev(child, offset, qiov->size, qiov, flags);
 -    } else {
 -        return bdrv_co_preadv(child, offset, qiov->size, qiov, flags);
 -    }
 -}
 -
  int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
                         int bytes, BdrvRequestFlags flags)
  {
 -    QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, NULL, bytes);
 -
 -    return bdrv_prwv(child, offset, &qiov, true, BDRV_REQ_ZERO_WRITE | flags);
 +    return bdrv_pwritev(child, offset, bytes, NULL,
 +                        BDRV_REQ_ZERO_WRITE | flags);
  }
  /*
@@ -XXX,XX +XXX,XX @@ int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags)
      }
  }
--/* return < 0 if error. See bdrv_pwrite() for the return codes */
++void block_copy_reset(BlockCopyState *s, int64_t offset, int64_t bytes)
--int bdrv_preadv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
++{
--{
++    QEMU_LOCK_GUARD(&s->lock);
--    int ret;
++
--
++    bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
--    ret = bdrv_prwv(child, offset, qiov, false, 0);
++    if (s->progress) {
--    if (ret < 0) {
++        progress_set_remaining(s->progress,
--        return ret;
++                               bdrv_get_dirty_count(s->copy_bitmap) +
--    }
++                               s->in_flight_bytes);
--
++    }
--    return qiov->size;
++}
--}
++
--
+ /*
- /* See bdrv_pwrite() for the return codes */
+  * Reset bits in copy_bitmap starting at offset if they represent unallocated
- int bdrv_pread(BdrvChild *child, int64_t offset, void *buf, int bytes)
+  * data in the image. May reset subsequent contiguous bits.
- {
+@@ -XXX,XX +XXX,XX @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
-+    int ret;
+     bytes = clusters * s->cluster_size;
-     QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
+     if (!ret) {
-     if (bytes < 0) {
+-        qemu_co_mutex_lock(&s->lock);
-         return -EINVAL;
+-        bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
 -        if (s->progress) {
 -            progress_set_remaining(s->progress,
 -                                   bdrv_get_dirty_count(s->copy_bitmap) +
 -                                   s->in_flight_bytes);
 -        }
 -        qemu_co_mutex_unlock(&s->lock);
 +        block_copy_reset(s, offset, bytes);
      }
--    return bdrv_preadv(child, offset, &qiov);
+     *count = bytes;
 -}
 +    ret = bdrv_preadv(child, offset, bytes, &qiov,  0);
 -int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
 -{
 -    int ret;
 -
 -    ret = bdrv_prwv(child, offset, qiov, true, 0);
 -    if (ret < 0) {
 -        return ret;
 -    }
 -
 -    return qiov->size;
 +    return ret < 0 ? ret : bytes;
  }
  /* Return no. of bytes on success or < 0 on error. Important errors are:
@@ -XXX,XX +XXX,XX @@ int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
  */
  int bdrv_pwrite(BdrvChild *child, int64_t offset, const void *buf, int bytes)
  {
 +    int ret;
      QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
      if (bytes < 0) {
          return -EINVAL;
      }
 -    return bdrv_pwritev(child, offset, &qiov);
 +    ret = bdrv_pwritev(child, offset, bytes, &qiov, 0);
 +
 +    return ret < 0 ? ret : bytes;
  }
  /*
 diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/test-bdrv-drain.c
 +++ b/tests/test-bdrv-drain.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_replace_test_co_preadv(BlockDriverState *bs,
          }
          s->io_co = NULL;
 -        ret = bdrv_preadv(bs->backing, offset, qiov);
 +        ret = bdrv_co_preadv(bs->backing, offset, bytes, qiov, 0);
          s->has_read = true;
          /* Wake up drain_co if it runs */
 --
-.26.2
+.34.1

-[PULL 11/17] block: generate coroutine-wrapper code
+[PULL 13/23] block: intoduce reqlist
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Use code generation implemented in previous commit to generated
+Split intersecting-requests functionality out of block-copy to be
-coroutine wrappers in block.c and block/io.c
+reused in copy-before-write filter.
 Note: while being here, fix tiny typo in MAINTAINERS.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Message-Id: <20220303194349.2304213-7-vsementsov@virtuozzo.com>
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20200924185414.28642-6-vsementsov@virtuozzo.com>
 ---
- block/coroutines.h    |   6 +-
+ include/block/reqlist.h |  67 +++++++++++++++++++++++
- include/block/block.h |  16 ++--
+ block/block-copy.c      | 116 +++++++++++++---------------------------
- block.c               |  73 ---------------
+ block/reqlist.c         |  76 ++++++++++++++++++++++++++
- block/io.c            | 212 ------------------------------------------
+ MAINTAINERS             |   4 +-
-files changed, 13 insertions(+), 294 deletions(-)
+ block/meson.build       |   1 +
 files changed, 184 insertions(+), 80 deletions(-)
  create mode 100644 include/block/reqlist.h
  create mode 100644 block/reqlist.c
-diff --git a/block/coroutines.h b/block/coroutines.h
+diff --git a/include/block/reqlist.h b/include/block/reqlist.h
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/include/block/reqlist.h
@@ -XXX,XX +XXX,XX @@
 +/*
 + * reqlist API
 + *
 + * Copyright (C) 2013 Proxmox Server Solutions
 + * Copyright (c) 2021 Virtuozzo International GmbH.
 + *
 + * Authors:
 + *  Dietmar Maurer (dietmar@proxmox.com)
 + *  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2 or later.
 + * See the COPYING file in the top-level directory.
 + */
 +
 +#ifndef REQLIST_H
 +#define REQLIST_H
 +
 +#include "qemu/coroutine.h"
 +
 +/*
 + * The API is not thread-safe and shouldn't be. The struct is public to be part
 + * of other structures and protected by third-party locks, see
 + * block/block-copy.c for example.
 + */
 +
 +typedef struct BlockReq {
 +    int64_t offset;
 +    int64_t bytes;
 +
 +    CoQueue wait_queue; /* coroutines blocked on this req */
 +    QLIST_ENTRY(BlockReq) list;
 +} BlockReq;
 +
 +typedef QLIST_HEAD(, BlockReq) BlockReqList;
 +
 +/*
 + * Initialize new request and add it to the list. Caller must be sure that
 + * there are no conflicting requests in the list.
 + */
 +void reqlist_init_req(BlockReqList *reqs, BlockReq *req, int64_t offset,
 +                      int64_t bytes);
 +/* Search for request in the list intersecting with @offset/@bytes area. */
 +BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
 +                                int64_t bytes);
 +
 +/*
 + * If there are no intersecting requests return false. Otherwise, wait for the
 + * first found intersecting request to finish and return true.
 + *
 + * @lock is passed to qemu_co_queue_wait()
 + * False return value proves that lock was released at no point.
 + */
 +bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
 +                                   int64_t bytes, CoMutex *lock);
 +
 +/*
 + * Shrink request and wake all waiting coroutines (maybe some of them are not
 + * intersecting with shrunk request).
 + */
 +void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes);
 +
 +/*
 + * Remove request and wake all waiting coroutines. Do not release any memory.
 + */
 +void coroutine_fn reqlist_remove_req(BlockReq *req);
 +
 +#endif /* REQLIST_H */
 diff --git a/block/block-copy.c b/block/block-copy.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/coroutines.h
+--- a/block/block-copy.c
-+++ b/block/coroutines.h
++++ b/block/block-copy.c
-@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp);
+@@ -XXX,XX +XXX,XX @@
- int coroutine_fn
+ #include "trace.h"
- bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
+ #include "qapi/error.h"
-              bool is_write, BdrvRequestFlags flags);
+ #include "block/block-copy.h"
--int
++#include "block/reqlist.h"
-+int generated_co_wrapper
+ #include "sysemu/block-backend.h"
- bdrv_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
+ #include "qemu/units.h"
-           bool is_write, BdrvRequestFlags flags);
+ #include "qemu/coroutine.h"
+@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyTask {
-@@ -XXX,XX +XXX,XX @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
+      */
-                                   int64_t *pnum,
+     BlockCopyState *s;
-                                   int64_t *map,
+     BlockCopyCallState *call_state;
-                                   BlockDriverState **file);
+-    int64_t offset;
--int
+     /*
-+int generated_co_wrapper
+      * @method can also be set again in the while loop of
- bdrv_common_block_status_above(BlockDriverState *bs,
+      * block_copy_dirty_clusters(), but it is never accessed concurrently
-                                BlockDriverState *base,
+@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyTask {
-                                bool want_zero,
+     BlockCopyMethod method;
-@@ -XXX,XX +XXX,XX @@ bdrv_common_block_status_above(BlockDriverState *bs,
- int coroutine_fn
+     /*
- bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
+-     * Fields whose state changes throughout the execution
-                    bool is_read);
+-     * Protected by lock in BlockCopyState.
--int
++     * Generally, req is protected by lock in BlockCopyState, Still req.offset
-+int generated_co_wrapper
++     * is only set on task creation, so may be read concurrently after creation.
- bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
++     * req.bytes is changed at most once, and need only protecting the case of
-                 bool is_read);
++     * parallel read while updating @bytes value in block_copy_task_shrink().
+      */
-diff --git a/include/block/block.h b/include/block/block.h
+-    CoQueue wait_queue; /* coroutines blocked on this task */
-index XXXXXXX..XXXXXXX 100644
+-    /*
---- a/include/block/block.h
+-     * Only protect the case of parallel read while updating @bytes
-+++ b/include/block/block.h
+-     * value in block_copy_task_shrink().
-@@ -XXX,XX +XXX,XX @@ void bdrv_refresh_filename(BlockDriverState *bs);
+-     */
- int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact,
+-    int64_t bytes;
-                                   PreallocMode prealloc, BdrvRequestFlags flags,
+-    QLIST_ENTRY(BlockCopyTask) list;
-                                   Error **errp);
++    BlockReq req;
--int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact,
+ } BlockCopyTask;
--                  PreallocMode prealloc, BdrvRequestFlags flags, Error **errp);
-+int generated_co_wrapper
+ static int64_t task_end(BlockCopyTask *task)
-+bdrv_truncate(BdrvChild *child, int64_t offset, bool exact,
+ {
-+              PreallocMode prealloc, BdrvRequestFlags flags, Error **errp);
+-    return task->offset + task->bytes;
++    return task->req.offset + task->req.bytes;
  int64_t bdrv_nb_sectors(BlockDriverState *bs);
  int64_t bdrv_getlength(BlockDriverState *bs);
@@ -XXX,XX +XXX,XX @@ typedef enum {
      BDRV_FIX_ERRORS   = 2,
  } BdrvCheckMode;
 -int bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix);
 +int generated_co_wrapper bdrv_check(BlockDriverState *bs, BdrvCheckResult *res,
 +                                    BdrvCheckMode fix);
  /* The units of offset and total_work_size may be chosen arbitrarily by the
   * block driver; total_work_size may change during the course of the amendment
@@ -XXX,XX +XXX,XX @@ void bdrv_aio_cancel_async(BlockAIOCB *acb);
  int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf);
  /* Invalidate any cached metadata used by image formats */
 -int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp);
 +int generated_co_wrapper bdrv_invalidate_cache(BlockDriverState *bs,
 +                                               Error **errp);
  void bdrv_invalidate_cache_all(Error **errp);
  int bdrv_inactivate_all(void);
  /* Ensure contents are flushed to disk.  */
 -int bdrv_flush(BlockDriverState *bs);
 +int generated_co_wrapper bdrv_flush(BlockDriverState *bs);
  int coroutine_fn bdrv_co_flush(BlockDriverState *bs);
  int bdrv_flush_all(void);
  void bdrv_close_all(void);
@@ -XXX,XX +XXX,XX @@ void bdrv_drain_all(void);
      AIO_WAIT_WHILE(bdrv_get_aio_context(bs_),              \
                     cond); })
 -int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
 +int generated_co_wrapper bdrv_pdiscard(BdrvChild *child, int64_t offset,
 +                                       int64_t bytes);
  int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
  int bdrv_has_zero_init_1(BlockDriverState *bs);
  int bdrv_has_zero_init(BlockDriverState *bs);
 diff --git a/block.c b/block.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block.c
 +++ b/block.c
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_check(BlockDriverState *bs,
      return bs->drv->bdrv_co_check(bs, res, fix);
  }
--typedef struct CheckCo {
+ typedef struct BlockCopyState {
--    BlockDriverState *bs;
+@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState {
--    BdrvCheckResult *res;
+     CoMutex lock;
--    BdrvCheckMode fix;
+     int64_t in_flight_bytes;
--    int ret;
+     BlockCopyMethod method;
--} CheckCo;
+-    QLIST_HEAD(, BlockCopyTask) tasks; /* All tasks from all block-copy calls */
--
++    BlockReqList reqs;
--static void coroutine_fn bdrv_check_co_entry(void *opaque)
+     QLIST_HEAD(, BlockCopyCallState) calls;
      /*
       * skip_unallocated:
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState {
      RateLimit rate_limit;
  } BlockCopyState;
 -/* Called with lock held */
 -static BlockCopyTask *find_conflicting_task(BlockCopyState *s,
 -                                            int64_t offset, int64_t bytes)
 -{
--    CheckCo *cco = opaque;
+-    BlockCopyTask *t;
--    cco->ret = bdrv_co_check(cco->bs, cco->res, cco->fix);
+-
--    aio_wait_kick();
+-    QLIST_FOREACH(t, &s->tasks, list) {
 -        if (offset + bytes > t->offset && offset < t->offset + t->bytes) {
 -            return t;
 -        }
 -    }
 -
 -    return NULL;
 -}
 -
--int bdrv_check(BlockDriverState *bs,
+-/*
--               BdrvCheckResult *res, BdrvCheckMode fix)
+- * If there are no intersecting tasks return false. Otherwise, wait for the
 - * first found intersecting tasks to finish and return true.
 - *
 - * Called with lock held. May temporary release the lock.
 - * Return value of 0 proves that lock was NOT released.
 - */
 -static bool coroutine_fn block_copy_wait_one(BlockCopyState *s, int64_t offset,
 -                                             int64_t bytes)
 -{
--    Coroutine *co;
+-    BlockCopyTask *task = find_conflicting_task(s, offset, bytes);
--    CheckCo cco = {
+-
--        .bs = bs,
+-    if (!task) {
--        .res = res,
+-        return false;
 -        .ret = -EINPROGRESS,
 -        .fix = fix,
 -    };
 -
 -    if (qemu_in_coroutine()) {
 -        /* Fast-path if already in coroutine context */
 -        bdrv_check_co_entry(&cco);
 -    } else {
 -        co = qemu_coroutine_create(bdrv_check_co_entry, &cco);
 -        bdrv_coroutine_enter(bs, co);
 -        BDRV_POLL_WHILE(bs, cco.ret == -EINPROGRESS);
 -    }
 -
--    return cco.ret;
+-    qemu_co_queue_wait(&task->wait_queue, &s->lock);
 -
 -    return true;
 -}
 -
- /*
+ /* Called with lock held */
-  * Return values:
+ static int64_t block_copy_chunk_size(BlockCopyState *s)
   * 0        - success
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
      return 0;
  }
 -typedef struct InvalidateCacheCo {
 -    BlockDriverState *bs;
 -    Error **errp;
 -    bool done;
 -    int ret;
 -} InvalidateCacheCo;
 -
 -static void coroutine_fn bdrv_invalidate_cache_co_entry(void *opaque)
 -{
 -    InvalidateCacheCo *ico = opaque;
 -    ico->ret = bdrv_co_invalidate_cache(ico->bs, ico->errp);
 -    ico->done = true;
 -    aio_wait_kick();
 -}
 -
 -int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
 -{
 -    Coroutine *co;
 -    InvalidateCacheCo ico = {
 -        .bs = bs,
 -        .done = false,
 -        .errp = errp
 -    };
 -
 -    if (qemu_in_coroutine()) {
 -        /* Fast-path if already in coroutine context */
 -        bdrv_invalidate_cache_co_entry(&ico);
 -    } else {
 -        co = qemu_coroutine_create(bdrv_invalidate_cache_co_entry, &ico);
 -        bdrv_coroutine_enter(bs, co);
 -        BDRV_POLL_WHILE(bs, !ico.done);
 -    }
 -
 -    return ico.ret;
 -}
 -
  void bdrv_invalidate_cache_all(Error **errp)
  {
-     BlockDriverState *bs;
+@@ -XXX,XX +XXX,XX @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state,
-diff --git a/block/io.c b/block/io.c
+     bytes = QEMU_ALIGN_UP(bytes, s->cluster_size);
-index XXXXXXX..XXXXXXX 100644
---- a/block/io.c
+     /* region is dirty, so no existent tasks possible in it */
-+++ b/block/io.c
+-    assert(!find_conflicting_task(s, offset, bytes));
-@@ -XXX,XX +XXX,XX @@ static int bdrv_check_byte_request(BlockDriverState *bs, int64_t offset,
++    assert(!reqlist_find_conflict(&s->reqs, offset, bytes));
-     return 0;
- }
+     bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
+     s->in_flight_bytes += bytes;
--typedef int coroutine_fn BdrvRequestEntry(void *opaque);
+@@ -XXX,XX +XXX,XX @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state,
--typedef struct BdrvRunCo {
+         .task.func = block_copy_task_entry,
--    BdrvRequestEntry *entry;
+         .s = s,
--    void *opaque;
+         .call_state = call_state,
 -    int ret;
 -    bool done;
 -    Coroutine *co; /* Coroutine, running bdrv_run_co_entry, for debugging */
 -} BdrvRunCo;
 -
 -static void coroutine_fn bdrv_run_co_entry(void *opaque)
 -{
 -    BdrvRunCo *arg = opaque;
 -
 -    arg->ret = arg->entry(arg->opaque);
 -    arg->done = true;
 -    aio_wait_kick();
 -}
 -
 -static int bdrv_run_co(BlockDriverState *bs, BdrvRequestEntry *entry,
 -                       void *opaque)
 -{
 -    if (qemu_in_coroutine()) {
 -        /* Fast-path if already in coroutine context */
 -        return entry(opaque);
 -    } else {
 -        BdrvRunCo s = { .entry = entry, .opaque = opaque };
 -
 -        s.co = qemu_coroutine_create(bdrv_run_co_entry, &s);
 -        bdrv_coroutine_enter(bs, s.co);
 -
 -        BDRV_POLL_WHILE(bs, !s.done);
 -
 -        return s.ret;
 -    }
 -}
 -
 -typedef struct RwCo {
 -    BdrvChild *child;
 -    int64_t offset;
 -    QEMUIOVector *qiov;
 -    bool is_write;
 -    BdrvRequestFlags flags;
 -} RwCo;
 -
  int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
                                QEMUIOVector *qiov, bool is_write,
                                BdrvRequestFlags flags)
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
      }
  }
 -static int coroutine_fn bdrv_rw_co_entry(void *opaque)
 -{
 -    RwCo *rwco = opaque;
 -
 -    return bdrv_co_prwv(rwco->child, rwco->offset, rwco->qiov,
 -                        rwco->is_write, rwco->flags);
 -}
 -
 -/*
 - * Process a vectored synchronous request using coroutines
 - */
 -int bdrv_prwv(BdrvChild *child, int64_t offset,
 -              QEMUIOVector *qiov, bool is_write,
 -              BdrvRequestFlags flags)
 -{
 -    RwCo rwco = {
 -        .child = child,
 -        .offset = offset,
 -        .qiov = qiov,
 -        .is_write = is_write,
 -        .flags = flags,
 -    };
 -
 -    return bdrv_run_co(child->bs, bdrv_rw_co_entry, &rwco);
 -}
 -
  int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
                         int bytes, BdrvRequestFlags flags)
  {
@@ -XXX,XX +XXX,XX @@ int bdrv_flush_all(void)
      return result;
  }
 -
 -typedef struct BdrvCoBlockStatusData {
 -    BlockDriverState *bs;
 -    BlockDriverState *base;
 -    bool want_zero;
 -    int64_t offset;
 -    int64_t bytes;
 -    int64_t *pnum;
 -    int64_t *map;
 -    BlockDriverState **file;
 -} BdrvCoBlockStatusData;
 -
  /*
   * Returns the allocation status of the specified sectors.
   * Drivers not implementing the functionality are assumed to not support
@@ -XXX,XX +XXX,XX @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
      return ret;
  }
 -/* Coroutine wrapper for bdrv_block_status_above() */
 -static int coroutine_fn bdrv_block_status_above_co_entry(void *opaque)
 -{
 -    BdrvCoBlockStatusData *data = opaque;
 -
 -    return bdrv_co_common_block_status_above(data->bs, data->base,
 -                                             data->want_zero,
 -                                             data->offset, data->bytes,
 -                                             data->pnum, data->map, data->file);
 -}
 -
 -/*
 - * Synchronous wrapper around bdrv_co_block_status_above().
 - *
 - * See bdrv_co_block_status_above() for details.
 - */
 -int bdrv_common_block_status_above(BlockDriverState *bs,
 -                                   BlockDriverState *base,
 -                                   bool want_zero, int64_t offset,
 -                                   int64_t bytes, int64_t *pnum,
 -                                   int64_t *map,
 -                                   BlockDriverState **file)
 -{
 -    BdrvCoBlockStatusData data = {
 -        .bs = bs,
 -        .base = base,
 -        .want_zero = want_zero,
 -        .offset = offset,
 -        .bytes = bytes,
--        .pnum = pnum,
+         .method = s->method,
--        .map = map,
+     };
--        .file = file,
+-    qemu_co_queue_init(&task->wait_queue);
--    };
+-    QLIST_INSERT_HEAD(&s->tasks, task, list);
--
++    reqlist_init_req(&s->reqs, &task->req, offset, bytes);
--    return bdrv_run_co(bs, bdrv_block_status_above_co_entry, &data);
--}
+     return task;
 -
  int bdrv_block_status_above(BlockDriverState *bs, BlockDriverState *base,
                              int64_t offset, int64_t bytes, int64_t *pnum,
                              int64_t *map, BlockDriverState **file)
@@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top,
      return 0;
  }
+@@ -XXX,XX +XXX,XX @@ static void coroutine_fn block_copy_task_shrink(BlockCopyTask *task,
--typedef struct BdrvVmstateCo {
+                                                 int64_t new_bytes)
--    BlockDriverState    *bs;
+ {
--    QEMUIOVector        *qiov;
+     QEMU_LOCK_GUARD(&task->s->lock);
--    int64_t             pos;
+-    if (new_bytes == task->bytes) {
--    bool                is_read;
++    if (new_bytes == task->req.bytes) {
--} BdrvVmstateCo;
+         return;
--
+     }
- int coroutine_fn
- bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
+-    assert(new_bytes > 0 && new_bytes < task->bytes);
-                    bool is_read)
++    assert(new_bytes > 0 && new_bytes < task->req.bytes);
-@@ -XXX,XX +XXX,XX @@ bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
 -    task->s->in_flight_bytes -= task->bytes - new_bytes;
 +    task->s->in_flight_bytes -= task->req.bytes - new_bytes;
      bdrv_set_dirty_bitmap(task->s->copy_bitmap,
 -                          task->offset + new_bytes, task->bytes - new_bytes);
 +                          task->req.offset + new_bytes,
 +                          task->req.bytes - new_bytes);
 -    task->bytes = new_bytes;
 -    qemu_co_queue_restart_all(&task->wait_queue);
 +    reqlist_shrink_req(&task->req, new_bytes);
  }
  static void coroutine_fn block_copy_task_end(BlockCopyTask *task, int ret)
  {
      QEMU_LOCK_GUARD(&task->s->lock);
 -    task->s->in_flight_bytes -= task->bytes;
 +    task->s->in_flight_bytes -= task->req.bytes;
      if (ret < 0) {
 -        bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->offset, task->bytes);
 +        bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->req.offset,
 +                              task->req.bytes);
      }
 -    QLIST_REMOVE(task, list);
      if (task->s->progress) {
          progress_set_remaining(task->s->progress,
                                 bdrv_get_dirty_count(task->s->copy_bitmap) +
                                 task->s->in_flight_bytes);
      }
 -    qemu_co_queue_restart_all(&task->wait_queue);
 +    reqlist_remove_req(&task->req);
  }
  void block_copy_state_free(BlockCopyState *s)
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
      ratelimit_init(&s->rate_limit);
      qemu_co_mutex_init(&s->lock);
 -    QLIST_INIT(&s->tasks);
 +    QLIST_INIT(&s->reqs);
      QLIST_INIT(&s->calls);
      return s;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_run(AioTaskPool *pool,
      aio_task_pool_wait_slot(pool);
      if (aio_task_pool_status(pool) < 0) {
 -        co_put_to_shres(task->s->mem, task->bytes);
 +        co_put_to_shres(task->s->mem, task->req.bytes);
          block_copy_task_end(task, -ECANCELED);
          g_free(task);
          return -ECANCELED;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
      BlockCopyMethod method = t->method;
      int ret;
 -    ret = block_copy_do_copy(s, t->offset, t->bytes, &method, &error_is_read);
 +    ret = block_copy_do_copy(s, t->req.offset, t->req.bytes, &method,
 +                             &error_is_read);
      WITH_QEMU_LOCK_GUARD(&s->lock) {
          if (s->method == t->method) {
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
                  t->call_state->error_is_read = error_is_read;
              }
          } else if (s->progress) {
 -            progress_work_done(s->progress, t->bytes);
 +            progress_work_done(s->progress, t->req.bytes);
          }
      }
 -    co_put_to_shres(s->mem, t->bytes);
 +    co_put_to_shres(s->mem, t->req.bytes);
      block_copy_task_end(t, ret);
      return ret;
- }
+@@ -XXX,XX +XXX,XX @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
+             trace_block_copy_skip_range(s, offset, bytes);
--static int coroutine_fn bdrv_co_rw_vmstate_entry(void *opaque)
+             break;
--{
+         }
--    BdrvVmstateCo *co = opaque;
+-        if (task->offset > offset) {
--
+-            trace_block_copy_skip_range(s, offset, task->offset - offset);
--    return bdrv_co_rw_vmstate(co->bs, co->qiov, co->pos, co->is_read);
++        if (task->req.offset > offset) {
--}
++            trace_block_copy_skip_range(s, offset, task->req.offset - offset);
--
+         }
--int bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
--                    bool is_read)
+         found_dirty = true;
--{
--    BdrvVmstateCo data = {
+-        ret = block_copy_block_status(s, task->offset, task->bytes,
--        .bs         = bs,
++        ret = block_copy_block_status(s, task->req.offset, task->req.bytes,
--        .qiov       = qiov,
+                                       &status_bytes);
--        .pos        = pos,
+         assert(ret >= 0); /* never fail */
--        .is_read    = is_read,
+-        if (status_bytes < task->bytes) {
--    };
++        if (status_bytes < task->req.bytes) {
--
+             block_copy_task_shrink(task, status_bytes);
--    return bdrv_run_co(bs, bdrv_co_rw_vmstate_entry, &data);
+         }
--}
+         if (qatomic_read(&s->skip_unallocated) &&
--
+             !(ret & BDRV_BLOCK_ALLOCATED)) {
- int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf,
+             block_copy_task_end(task, 0);
-                       int64_t pos, int size)
+-            trace_block_copy_skip_range(s, task->offset, task->bytes);
- {
++            trace_block_copy_skip_range(s, task->req.offset, task->req.bytes);
-@@ -XXX,XX +XXX,XX @@ void bdrv_aio_cancel_async(BlockAIOCB *acb)
+             offset = task_end(task);
- /**************************************************************/
+             bytes = end - offset;
- /* Coroutine block device emulation */
+             g_free(task);
+@@ -XXX,XX +XXX,XX @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
--static int coroutine_fn bdrv_flush_co_entry(void *opaque)
+             }
--{
+         }
--    return bdrv_co_flush(opaque);
--}
+-        ratelimit_calculate_delay(&s->rate_limit, task->bytes);
--
++        ratelimit_calculate_delay(&s->rate_limit, task->req.bytes);
- int coroutine_fn bdrv_co_flush(BlockDriverState *bs)
- {
+-        trace_block_copy_process(s, task->offset);
-     BdrvChild *primary_child = bdrv_primary_child(bs);
++        trace_block_copy_process(s, task->req.offset);
-@@ -XXX,XX +XXX,XX @@ early_exit:
-     return ret;
+-        co_get_from_shres(s->mem, task->bytes);
- }
++        co_get_from_shres(s->mem, task->req.bytes);
--int bdrv_flush(BlockDriverState *bs)
+         offset = task_end(task);
--{
+         bytes = end - offset;
--    return bdrv_run_co(bs, bdrv_flush_co_entry, bs);
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
--}
+                  * Check that there is no task we still need to
--
+                  * wait to complete
--typedef struct DiscardCo {
+                  */
--    BdrvChild *child;
+-                ret = block_copy_wait_one(s, call_state->offset,
--    int64_t offset;
+-                                          call_state->bytes);
--    int64_t bytes;
++                ret = reqlist_wait_one(&s->reqs, call_state->offset,
--} DiscardCo;
++                                       call_state->bytes, &s->lock);
--
+                 if (ret == 0) {
--static int coroutine_fn bdrv_pdiscard_co_entry(void *opaque)
+                     /*
--{
+                      * No pending tasks, but check again the bitmap in this
--    DiscardCo *rwco = opaque;
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
--
+                      * between this and the critical section in
--    return bdrv_co_pdiscard(rwco->child, rwco->offset, rwco->bytes);
+                      * block_copy_dirty_clusters().
--}
+                      *
--
+-                     * block_copy_wait_one return value 0 also means that it
- int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset,
++                     * reqlist_wait_one return value 0 also means that it
-                                   int64_t bytes)
+                      * didn't release the lock. So, we are still in the same
- {
+                      * critical section, not interrupted by any concurrent
-@@ -XXX,XX +XXX,XX @@ out:
+                      * access to state.
-     return ret;
+diff --git a/block/reqlist.c b/block/reqlist.c
- }
+new file mode 100644
+index XXXXXXX..XXXXXXX
--int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes)
+--- /dev/null
--{
++++ b/block/reqlist.c
--    DiscardCo rwco = {
+@@ -XXX,XX +XXX,XX @@
--        .child = child,
++/*
--        .offset = offset,
++ * reqlist API
--        .bytes = bytes,
++ *
--    };
++ * Copyright (C) 2013 Proxmox Server Solutions
--
++ * Copyright (c) 2021 Virtuozzo International GmbH.
--    return bdrv_run_co(child->bs, bdrv_pdiscard_co_entry, &rwco);
++ *
--}
++ * Authors:
--
++ *  Dietmar Maurer (dietmar@proxmox.com)
- int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf)
++ *  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
- {
++ *
-     BlockDriver *drv = bs->drv;
++ * This work is licensed under the terms of the GNU GPL, version 2 or later.
-@@ -XXX,XX +XXX,XX @@ out:
++ * See the COPYING file in the top-level directory.
++ */
-     return ret;
++
- }
++#include "qemu/osdep.h"
--
++
--typedef struct TruncateCo {
++#include "block/reqlist.h"
--    BdrvChild *child;
++
--    int64_t offset;
++void reqlist_init_req(BlockReqList *reqs, BlockReq *req, int64_t offset,
--    bool exact;
++                      int64_t bytes)
--    PreallocMode prealloc;
++{
--    BdrvRequestFlags flags;
++    assert(!reqlist_find_conflict(reqs, offset, bytes));
--    Error **errp;
++
--} TruncateCo;
++    *req = (BlockReq) {
--
++        .offset = offset,
--static int coroutine_fn bdrv_truncate_co_entry(void *opaque)
++        .bytes = bytes,
--{
++    };
--    TruncateCo *tco = opaque;
++    qemu_co_queue_init(&req->wait_queue);
--
++    QLIST_INSERT_HEAD(reqs, req, list);
--    return bdrv_co_truncate(tco->child, tco->offset, tco->exact,
++}
--                            tco->prealloc, tco->flags, tco->errp);
++
--}
++BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
--
++                                int64_t bytes)
--int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact,
++{
--                  PreallocMode prealloc, BdrvRequestFlags flags, Error **errp)
++    BlockReq *r;
--{
++
--    TruncateCo tco = {
++    QLIST_FOREACH(r, reqs, list) {
--        .child      = child,
++        if (offset + bytes > r->offset && offset < r->offset + r->bytes) {
--        .offset     = offset,
++            return r;
--        .exact      = exact,
++        }
--        .prealloc   = prealloc,
++    }
--        .flags      = flags,
++
--        .errp       = errp,
++    return NULL;
--    };
++}
--
++
--    return bdrv_run_co(child->bs, bdrv_truncate_co_entry, &tco);
++bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
--}
++                                   int64_t bytes, CoMutex *lock)
 +{
 +    BlockReq *r = reqlist_find_conflict(reqs, offset, bytes);
 +
 +    if (!r) {
 +        return false;
 +    }
 +
 +    qemu_co_queue_wait(&r->wait_queue, lock);
 +
 +    return true;
 +}
 +
 +void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes)
 +{
 +    if (new_bytes == req->bytes) {
 +        return;
 +    }
 +
 +    assert(new_bytes > 0 && new_bytes < req->bytes);
 +
 +    req->bytes = new_bytes;
 +    qemu_co_queue_restart_all(&req->wait_queue);
 +}
 +
 +void coroutine_fn reqlist_remove_req(BlockReq *req)
 +{
 +    QLIST_REMOVE(req, list);
 +    qemu_co_queue_restart_all(&req->wait_queue);
 +}
 diff --git a/MAINTAINERS b/MAINTAINERS
 index XXXXXXX..XXXXXXX 100644
 --- a/MAINTAINERS
 +++ b/MAINTAINERS
@@ -XXX,XX +XXX,XX @@ F: block/stream.c
  F: block/mirror.c
  F: qapi/job.json
  F: block/block-copy.c
 -F: include/block/block-copy.c
 +F: include/block/block-copy.h
 +F: block/reqlist.c
 +F: include/block/reqlist.h
  F: block/copy-before-write.h
  F: block/copy-before-write.c
  F: include/block/aio_task.h
 diff --git a/block/meson.build b/block/meson.build
 index XXXXXXX..XXXXXXX 100644
 --- a/block/meson.build
 +++ b/block/meson.build
@@ -XXX,XX +XXX,XX @@ block_ss.add(files(
    'qcow2.c',
    'quorum.c',
    'raw-format.c',
 +  'reqlist.c',
    'snapshot.c',
    'throttle-groups.c',
    'throttle.c',
 --
-.26.2
+.34.1

-[PULL 02/17] block/nvme: Map doorbells pages write-only
+[PULL 14/23] block/reqlist: reqlist_find_conflict(): use ranges_overlap()
-From: Philippe Mathieu-Daudé <philmd@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Per the datasheet sections 3.1.13/3.1.14:
+Let's reuse convenient helper.
   "The host should not read the doorbell registers."
-As we don't need read access, map the doorbells with write-only
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-permission. We keep a reference to this mapped address in the
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-BDRVNVMeState structure.
+Message-Id: <20220303194349.2304213-8-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  block/reqlist.c | 3 ++-
 file changed, 2 insertions(+), 1 deletion(-)
-Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+diff --git a/block/reqlist.c b/block/reqlist.c
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 Message-Id: <20200922083821.578519-3-philmd@redhat.com>
 ---
  block/nvme.c | 29 +++++++++++++++++++----------
 file changed, 19 insertions(+), 10 deletions(-)
 diff --git a/block/nvme.c b/block/nvme.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/nvme.c
+--- a/block/reqlist.c
-+++ b/block/nvme.c
++++ b/block/reqlist.c
 @@ -XXX,XX +XXX,XX @@
- #define NVME_SQ_ENTRY_BYTES 64
+  */
- #define NVME_CQ_ENTRY_BYTES 16
- #define NVME_QUEUE_SIZE 128
+ #include "qemu/osdep.h"
--#define NVME_BAR_SIZE 8192
++#include "qemu/range.h"
-+#define NVME_DOORBELL_SIZE 4096
+ #include "block/reqlist.h"
- /*
-  * We have to leave one slot empty as that is the full queue case where
+@@ -XXX,XX +XXX,XX @@ BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
-@@ -XXX,XX +XXX,XX @@ typedef struct {
+     BlockReq *r;
- /* Memory mapped registers */
- typedef volatile struct {
+     QLIST_FOREACH(r, reqs, list) {
-     NvmeBar ctrl;
+-        if (offset + bytes > r->offset && offset < r->offset + r->bytes) {
--    struct {
++        if (ranges_overlap(offset, bytes, r->offset, r->bytes)) {
--        uint32_t sq_tail;
+             return r;
 -        uint32_t cq_head;
 -    } doorbells[];
  } NVMeRegs;
  #define INDEX_ADMIN     0
@@ -XXX,XX +XXX,XX @@ struct BDRVNVMeState {
      AioContext *aio_context;
      QEMUVFIOState *vfio;
      NVMeRegs *regs;
 +    /* Memory mapped registers */
 +    volatile struct {
 +        uint32_t sq_tail;
 +        uint32_t cq_head;
 +    } *doorbells;
      /* The submission/completion queue pairs.
       * [0]: admin queue.
       * [1..]: io queues.
@@ -XXX,XX +XXX,XX @@ static NVMeQueuePair *nvme_create_queue_pair(BDRVNVMeState *s,
          error_propagate(errp, local_err);
          goto fail;
      }
 -    q->sq.doorbell = &s->regs->doorbells[idx * s->doorbell_scale].sq_tail;
 +    q->sq.doorbell = &s->doorbells[idx * s->doorbell_scale].sq_tail;
      nvme_init_queue(s, &q->cq, size, NVME_CQ_ENTRY_BYTES, &local_err);
      if (local_err) {
          error_propagate(errp, local_err);
          goto fail;
      }
 -    q->cq.doorbell = &s->regs->doorbells[idx * s->doorbell_scale].cq_head;
 +    q->cq.doorbell = &s->doorbells[idx * s->doorbell_scale].cq_head;
      return q;
  fail:
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
          goto out;
      }
 -    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, NVME_BAR_SIZE,
 +    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, sizeof(NvmeBar),
                                      PROT_READ | PROT_WRITE, errp);
      if (!s->regs) {
          ret = -EINVAL;
          goto out;
      }
 -
      /* Perform initialize sequence as described in NVMe spec "7.6.1
       * Initialization". */
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
          }
      }
-+    s->doorbells = qemu_vfio_pci_map_bar(s->vfio, 0, sizeof(NvmeBar),
-+                                         NVME_DOORBELL_SIZE, PROT_WRITE, errp);
-+    if (!s->doorbells) {
-+        ret = -EINVAL;
-+        goto out;
-+    }
-+
-     /* Set up admin queue. */
-     s->queues = g_new(NVMeQueuePair *, 1);
-     s->queues[INDEX_ADMIN] = nvme_create_queue_pair(s, aio_context, 0,
-@@ -XXX,XX +XXX,XX @@ static void nvme_close(BlockDriverState *bs)
-                            &s->irq_notifier[MSIX_SHARED_IRQ_IDX],
-                            false, NULL, NULL);
-     event_notifier_cleanup(&s->irq_notifier[MSIX_SHARED_IRQ_IDX]);
--    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->regs, 0, NVME_BAR_SIZE);
-+    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->doorbells,
-+                            sizeof(NvmeBar), NVME_DOORBELL_SIZE);
-+    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->regs, 0, sizeof(NvmeBar));
-     qemu_vfio_close(s->vfio);
-     g_free(s->device);
 --
-.26.2
+.34.1

-[PULL 17/17] util/vfio-helpers: Rework the IOVA allocator to avoid IOVA reserved regions
+[PULL 15/23] block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
-From: Eric Auger <eric.auger@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Introduce the qemu_vfio_find_fixed/temp_iova helpers which
+Add a convenient function similar with bdrv_block_status() to get
-respectively allocate IOVAs from the bottom/top parts of the
+status of dirty bitmap.
 usable IOVA range, without picking within host IOVA reserved
 windows. The allocation remains basic: if the size is too big
 for the remaining of the current usable IOVA range, we jump
 to the next one, leaving a hole in the address map.
-Signed-off-by: Eric Auger <eric.auger@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-id: 20200929085550.30926-3-eric.auger@redhat.com
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-Id: <20220303194349.2304213-9-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- util/vfio-helpers.c | 57 +++++++++++++++++++++++++++++++++++++++++----
+ include/block/dirty-bitmap.h |  2 ++
-file changed, 53 insertions(+), 4 deletions(-)
+ include/qemu/hbitmap.h       | 12 ++++++++++++
  block/dirty-bitmap.c         |  6 ++++++
  util/hbitmap.c               | 33 +++++++++++++++++++++++++++++++++
 files changed, 53 insertions(+)
-diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c
+diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
 index XXXXXXX..XXXXXXX 100644
---- a/util/vfio-helpers.c
+--- a/include/block/dirty-bitmap.h
-+++ b/util/vfio-helpers.c
++++ b/include/block/dirty-bitmap.h
-@@ -XXX,XX +XXX,XX @@ static bool qemu_vfio_verify_mappings(QEMUVFIOState *s)
+@@ -XXX,XX +XXX,XX @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
  bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
          int64_t start, int64_t end, int64_t max_dirty_count,
          int64_t *dirty_start, int64_t *dirty_count);
 +bool bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap, int64_t offset,
 +                              int64_t bytes, int64_t *count);
  BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
                                                    Error **errp);
 diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/qemu/hbitmap.h
 +++ b/include/qemu/hbitmap.h
@@ -XXX,XX +XXX,XX @@ bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
                               int64_t max_dirty_count,
                               int64_t *dirty_start, int64_t *dirty_count);
 +/*
 + * bdrv_dirty_bitmap_status:
 + * @hb: The HBitmap to operate on
 + * @start: The bit to start from
 + * @count: Number of bits to proceed
 + * @pnum: Out-parameter. How many bits has same value starting from @start
 + *
 + * Returns true if bitmap is dirty at @start, false otherwise.
 + */
 +bool hbitmap_status(const HBitmap *hb, int64_t start, int64_t count,
 +                    int64_t *pnum);
 +
  /**
   * hbitmap_iter_next:
   * @hbi: HBitmapIter to operate on.
 diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/dirty-bitmap.c
 +++ b/block/dirty-bitmap.c
@@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
                                     dirty_start, dirty_count);
  }
 +bool bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap, int64_t offset,
 +                              int64_t bytes, int64_t *count)
 +{
 +    return hbitmap_status(bitmap->bitmap, offset, bytes, count);
 +}
 +
  /**
   * bdrv_merge_dirty_bitmap: merge src into dest.
   * Ensures permissions on bitmaps are reasonable; use for public API.
 diff --git a/util/hbitmap.c b/util/hbitmap.c
 index XXXXXXX..XXXXXXX 100644
 --- a/util/hbitmap.c
 +++ b/util/hbitmap.c
@@ -XXX,XX +XXX,XX @@ bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
      return true;
  }
-+static int
++bool hbitmap_status(const HBitmap *hb, int64_t start, int64_t count,
-+qemu_vfio_find_fixed_iova(QEMUVFIOState *s, size_t size, uint64_t *iova)
++                    int64_t *pnum)
 +{
-+    int i;
++    int64_t next_dirty, next_zero;
 +
-+    for (i = 0; i < s->nb_iova_ranges; i++) {
++    assert(start >= 0);
-+        if (s->usable_iova_ranges[i].end < s->low_water_mark) {
++    assert(count > 0);
-+            continue;
++    assert(start + count <= hb->orig_size);
 +        }
 +        s->low_water_mark =
 +            MAX(s->low_water_mark, s->usable_iova_ranges[i].start);
 +
-+        if (s->usable_iova_ranges[i].end - s->low_water_mark + 1 >= size ||
++    next_dirty = hbitmap_next_dirty(hb, start, count);
-+            s->usable_iova_ranges[i].end - s->low_water_mark + 1 == 0) {
++    if (next_dirty == -1) {
-+            *iova = s->low_water_mark;
++        *pnum = count;
-+            s->low_water_mark += size;
++        return false;
 +            return 0;
 +        }
 +    }
-+    return -ENOMEM;
++
 +    if (next_dirty > start) {
 +        *pnum = next_dirty - start;
 +        return false;
 +    }
 +
 +    assert(next_dirty == start);
 +
 +    next_zero = hbitmap_next_zero(hb, start, count);
 +    if (next_zero == -1) {
 +        *pnum = count;
 +        return true;
 +    }
 +
 +    assert(next_zero > start);
 +    *pnum = next_zero - start;
 +    return false;
 +}
 +
-+static int
+ bool hbitmap_empty(const HBitmap *hb)
-+qemu_vfio_find_temp_iova(QEMUVFIOState *s, size_t size, uint64_t *iova)
+ {
-+{
+     return hb->count == 0;
 +    int i;
 +
 +    for (i = s->nb_iova_ranges - 1; i >= 0; i--) {
 +        if (s->usable_iova_ranges[i].start > s->high_water_mark) {
 +            continue;
 +        }
 +        s->high_water_mark =
 +            MIN(s->high_water_mark, s->usable_iova_ranges[i].end + 1);
 +
 +        if (s->high_water_mark - s->usable_iova_ranges[i].start + 1 >= size ||
 +            s->high_water_mark - s->usable_iova_ranges[i].start + 1 == 0) {
 +            *iova = s->high_water_mark - size;
 +            s->high_water_mark = *iova;
 +            return 0;
 +        }
 +    }
 +    return -ENOMEM;
 +}
 +
  /* Map [host, host + size) area into a contiguous IOVA address space, and store
   * the result in @iova if not NULL. The caller need to make sure the area is
   * aligned to page size, and mustn't overlap with existing mapping areas (split
@@ -XXX,XX +XXX,XX @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size,
              goto out;
          }
          if (!temporary) {
 -            iova0 = s->low_water_mark;
 +            if (qemu_vfio_find_fixed_iova(s, size, &iova0)) {
 +                ret = -ENOMEM;
 +                goto out;
 +            }
 +
              mapping = qemu_vfio_add_mapping(s, host, size, index + 1, iova0);
              if (!mapping) {
                  ret = -ENOMEM;
@@ -XXX,XX +XXX,XX @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size,
                  qemu_vfio_undo_mapping(s, mapping, NULL);
                  goto out;
              }
 -            s->low_water_mark += size;
              qemu_vfio_dump_mappings(s);
          } else {
 -            iova0 = s->high_water_mark - size;
 +            if (qemu_vfio_find_temp_iova(s, size, &iova0)) {
 +                ret = -ENOMEM;
 +                goto out;
 +            }
              ret = qemu_vfio_do_mapping(s, host, size, iova0);
              if (ret) {
                  goto out;
              }
 -            s->high_water_mark -= size;
          }
      }
      if (iova) {
 --
-.26.2
+.34.1

-[PULL 08/17] block/io: refactor coroutine wrappers
+[PULL 16/23] block/reqlist: add reqlist_wait_all()
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Most of our coroutine wrappers already follow this convention:
+Add function to wait for all intersecting requests.
+To be used in the further commit.
 We have 'coroutine_fn bdrv_co_<something>(<normal argument list>)' as
 the core function, and a wrapper 'bdrv_<something>(<same argument
 list>)' which does parameter packing and calls bdrv_run_co().
 The only outsiders are the bdrv_prwv_co and
 bdrv_common_block_status_above wrappers. Let's refactor them to behave
 as the others, it simplifies further conversion of coroutine wrappers.
 This patch adds an indirection layer, but it will be compensated by
 a further commit, which will drop bdrv_co_prwv together with the
 is_write logic, to keep the read and write paths separate.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+Message-Id: <20220303194349.2304213-10-vsementsov@virtuozzo.com>
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20200924185414.28642-3-vsementsov@virtuozzo.com>
 ---
- block/io.c | 60 +++++++++++++++++++++++++++++-------------------------
+ include/block/reqlist.h | 8 ++++++++
-file changed, 32 insertions(+), 28 deletions(-)
+ block/reqlist.c         | 8 ++++++++
 files changed, 16 insertions(+)
-diff --git a/block/io.c b/block/io.c
+diff --git a/include/block/reqlist.h b/include/block/reqlist.h
 index XXXXXXX..XXXXXXX 100644
---- a/block/io.c
+--- a/include/block/reqlist.h
-+++ b/block/io.c
++++ b/include/block/reqlist.h
-@@ -XXX,XX +XXX,XX @@ typedef struct RwCo {
+@@ -XXX,XX +XXX,XX @@ BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
-     BdrvRequestFlags flags;
+ bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
- } RwCo;
+                                    int64_t bytes, CoMutex *lock);
-+static int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
++/*
-+                                     QEMUIOVector *qiov, bool is_write,
++ * Wait for all intersecting requests. It just calls reqlist_wait_one() in a
-+                                     BdrvRequestFlags flags)
++ * loop, caller is responsible to stop producing new requests in this region
 + * in parallel, otherwise reqlist_wait_all() may never return.
 + */
 +void coroutine_fn reqlist_wait_all(BlockReqList *reqs, int64_t offset,
 +                                   int64_t bytes, CoMutex *lock);
 +
  /*
   * Shrink request and wake all waiting coroutines (maybe some of them are not
   * intersecting with shrunk request).
 diff --git a/block/reqlist.c b/block/reqlist.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/reqlist.c
 +++ b/block/reqlist.c
@@ -XXX,XX +XXX,XX @@ bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
      return true;
  }
 +void coroutine_fn reqlist_wait_all(BlockReqList *reqs, int64_t offset,
 +                                   int64_t bytes, CoMutex *lock)
 +{
-+    if (is_write) {
++    while (reqlist_wait_one(reqs, offset, bytes, lock)) {
-+        return bdrv_co_pwritev(child, offset, qiov->size, qiov, flags);
++        /* continue */
 +    } else {
 +        return bdrv_co_preadv(child, offset, qiov->size, qiov, flags);
 +    }
 +}
 +
- static int coroutine_fn bdrv_rw_co_entry(void *opaque)
+ void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes)
  {
-     RwCo *rwco = opaque;
+     if (new_bytes == req->bytes) {
 -    if (!rwco->is_write) {
 -        return bdrv_co_preadv(rwco->child, rwco->offset,
 -                              rwco->qiov->size, rwco->qiov,
 -                              rwco->flags);
 -    } else {
 -        return bdrv_co_pwritev(rwco->child, rwco->offset,
 -                               rwco->qiov->size, rwco->qiov,
 -                               rwco->flags);
 -    }
 +    return bdrv_co_prwv(rwco->child, rwco->offset, rwco->qiov,
 +                        rwco->is_write, rwco->flags);
  }
  /*
   * Process a vectored synchronous request using coroutines
   */
 -static int bdrv_prwv_co(BdrvChild *child, int64_t offset,
 -                        QEMUIOVector *qiov, bool is_write,
 -                        BdrvRequestFlags flags)
 +static int bdrv_prwv(BdrvChild *child, int64_t offset,
 +                     QEMUIOVector *qiov, bool is_write,
 +                     BdrvRequestFlags flags)
  {
      RwCo rwco = {
          .child = child,
@@ -XXX,XX +XXX,XX @@ int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
  {
      QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, NULL, bytes);
 -    return bdrv_prwv_co(child, offset, &qiov, true,
 -                        BDRV_REQ_ZERO_WRITE | flags);
 +    return bdrv_prwv(child, offset, &qiov, true, BDRV_REQ_ZERO_WRITE | flags);
  }
  /*
@@ -XXX,XX +XXX,XX @@ int bdrv_preadv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
  {
      int ret;
 -    ret = bdrv_prwv_co(child, offset, qiov, false, 0);
 +    ret = bdrv_prwv(child, offset, qiov, false, 0);
      if (ret < 0) {
          return ret;
      }
@@ -XXX,XX +XXX,XX @@ int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
  {
      int ret;
 -    ret = bdrv_prwv_co(child, offset, qiov, true, 0);
 +    ret = bdrv_prwv(child, offset, qiov, true, 0);
      if (ret < 0) {
          return ret;
      }
@@ -XXX,XX +XXX,XX @@ early_out:
      return ret;
  }
 -static int coroutine_fn bdrv_co_block_status_above(BlockDriverState *bs,
 -                                                   BlockDriverState *base,
 -                                                   bool want_zero,
 -                                                   int64_t offset,
 -                                                   int64_t bytes,
 -                                                   int64_t *pnum,
 -                                                   int64_t *map,
 -                                                   BlockDriverState **file)
 +static int coroutine_fn
 +bdrv_co_common_block_status_above(BlockDriverState *bs,
 +                                  BlockDriverState *base,
 +                                  bool want_zero,
 +                                  int64_t offset,
 +                                  int64_t bytes,
 +                                  int64_t *pnum,
 +                                  int64_t *map,
 +                                  BlockDriverState **file)
  {
      BlockDriverState *p;
      int ret = 0;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_block_status_above_co_entry(void *opaque)
  {
      BdrvCoBlockStatusData *data = opaque;
 -    return bdrv_co_block_status_above(data->bs, data->base,
 -                                      data->want_zero,
 -                                      data->offset, data->bytes,
 -                                      data->pnum, data->map, data->file);
 +    return bdrv_co_common_block_status_above(data->bs, data->base,
 +                                             data->want_zero,
 +                                             data->offset, data->bytes,
 +                                             data->pnum, data->map, data->file);
  }
  /*
 --
-.26.2
+.34.1

-[PULL 13/17] block/io: refactor save/load vmstate
+[PULL 17/23] block/io: introduce block driver snapshot-access API
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Like for read/write in a previous commit, drop extra indirection layer,
+Add new block driver handlers and corresponding generic wrappers.
-generate directly bdrv_readv_vmstate() and bdrv_writev_vmstate().
+It will be used to allow copy-before-write filter to provide
 reach fleecing interface in further commit.
 In future this approach may be used to allow reading qcow2 internal
 snapshots, for example to export them through NBD.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Message-Id: <20220303194349.2304213-11-vsementsov@virtuozzo.com>
-Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
+[hreitz: Rebased on block GS/IO split]
-Message-Id: <20200924185414.28642-8-vsementsov@virtuozzo.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- block/coroutines.h    | 10 +++----
+ include/block/block_int-common.h | 18 ++++++++
- include/block/block.h |  6 ++--
+ include/block/block_int-io.h     |  9 ++++
- block/io.c            | 70 ++++++++++++++++++++++---------------------
+ block/io.c                       | 72 ++++++++++++++++++++++++++++++++
-files changed, 44 insertions(+), 42 deletions(-)
+files changed, 99 insertions(+)
-diff --git a/block/coroutines.h b/block/coroutines.h
+diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
 index XXXXXXX..XXXXXXX 100644
---- a/block/coroutines.h
+--- a/include/block/block_int-common.h
-+++ b/block/coroutines.h
++++ b/include/block/block_int-common.h
-@@ -XXX,XX +XXX,XX @@ bdrv_common_block_status_above(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
-                                int64_t *map,
+         bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
-                                BlockDriverState **file);
+         int64_t *map, BlockDriverState **file);
--int coroutine_fn
++    /*
--bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
++     * Snapshot-access API.
--                   bool is_read);
++     *
--int generated_co_wrapper
++     * Block-driver may provide snapshot-access API: special functions to access
--bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
++     * some internal "snapshot". The functions are similar with normal
--                bool is_read);
++     * read/block_status/discard handler, but don't have any specific handling
-+int coroutine_fn bdrv_co_readv_vmstate(BlockDriverState *bs,
++     * in generic block-layer: no serializing, no alignment, no tracked
-+                                       QEMUIOVector *qiov, int64_t pos);
++     * requests. So, block-driver that realizes these APIs is fully responsible
-+int coroutine_fn bdrv_co_writev_vmstate(BlockDriverState *bs,
++     * for synchronization between snapshot-access API and normal IO requests.
-+                                        QEMUIOVector *qiov, int64_t pos);
++     */
++    int coroutine_fn (*bdrv_co_preadv_snapshot)(BlockDriverState *bs,
- #endif /* BLOCK_COROUTINES_INT_H */
++        int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
-diff --git a/include/block/block.h b/include/block/block.h
++    int coroutine_fn (*bdrv_co_snapshot_block_status)(BlockDriverState *bs,
 +        bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
 +        int64_t *map, BlockDriverState **file);
 +    int coroutine_fn (*bdrv_co_pdiscard_snapshot)(BlockDriverState *bs,
 +        int64_t offset, int64_t bytes);
 +
      /*
       * Invalidate any cached meta-data.
       */
 diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h
 index XXXXXXX..XXXXXXX 100644
---- a/include/block/block.h
+--- a/include/block/block_int-io.h
-+++ b/include/block/block.h
++++ b/include/block/block_int-io.h
-@@ -XXX,XX +XXX,XX @@ int path_has_protocol(const char *path);
+@@ -XXX,XX +XXX,XX @@
- int path_is_absolute(const char *path);
+  * the I/O API.
- char *path_combine(const char *base_path, const char *filename);
+  */
--int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
++int coroutine_fn bdrv_co_preadv_snapshot(BdrvChild *child,
--int bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
++    int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
-+int generated_co_wrapper
++int coroutine_fn bdrv_co_snapshot_block_status(BlockDriverState *bs,
-+bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
++    bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
-+int generated_co_wrapper
++    int64_t *map, BlockDriverState **file);
-+bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
++int coroutine_fn bdrv_co_pdiscard_snapshot(BlockDriverState *bs,
- int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf,
++    int64_t offset, int64_t bytes);
-                       int64_t pos, int size);
++
++
  int coroutine_fn bdrv_co_preadv(BdrvChild *child,
      int64_t offset, int64_t bytes, QEMUIOVector *qiov,
      BdrvRequestFlags flags);
 diff --git a/block/io.c b/block/io.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/io.c
 +++ b/block/io.c
-@@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top,
+@@ -XXX,XX +XXX,XX @@ void bdrv_cancel_in_flight(BlockDriverState *bs)
          bs->drv->bdrv_cancel_in_flight(bs);
      }
  }
++
- int coroutine_fn
++int coroutine_fn
--bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
++bdrv_co_preadv_snapshot(BdrvChild *child, int64_t offset, int64_t bytes,
--                   bool is_read)
++                        QEMUIOVector *qiov, size_t qiov_offset)
-+bdrv_co_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
++{
- {
++    BlockDriverState *bs = child->bs;
-     BlockDriver *drv = bs->drv;
++    BlockDriver *drv = bs->drv;
-     BlockDriverState *child_bs = bdrv_primary_bs(bs);
++    int ret;
-     int ret = -ENOTSUP;
++    IO_CODE();
++
 +    if (!drv) {
 +        return -ENOMEDIUM;
 +    }
 +
-     bdrv_inc_in_flight(bs);
++    if (!drv->bdrv_co_preadv_snapshot) {
++        return -ENOTSUP;
 +    if (drv->bdrv_load_vmstate) {
 +        ret = drv->bdrv_load_vmstate(bs, qiov, pos);
 +    } else if (child_bs) {
 +        ret = bdrv_co_readv_vmstate(child_bs, qiov, pos);
 +    }
 +
++    bdrv_inc_in_flight(bs);
++    ret = drv->bdrv_co_preadv_snapshot(bs, offset, bytes, qiov, qiov_offset);
 +    bdrv_dec_in_flight(bs);
 +
 +    return ret;
 +}
 +
 +int coroutine_fn
-+bdrv_co_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
++bdrv_co_snapshot_block_status(BlockDriverState *bs,
 +                              bool want_zero, int64_t offset, int64_t bytes,
 +                              int64_t *pnum, int64_t *map,
 +                              BlockDriverState **file)
 +{
 +    BlockDriver *drv = bs->drv;
-+    BlockDriverState *child_bs = bdrv_primary_bs(bs);
++    int ret;
-+    int ret = -ENOTSUP;
++    IO_CODE();
 +
-     if (!drv) {
++    if (!drv) {
 -        ret = -ENOMEDIUM;
 -    } else if (drv->bdrv_load_vmstate) {
 -        if (is_read) {
 -            ret = drv->bdrv_load_vmstate(bs, qiov, pos);
 -        } else {
 -            ret = drv->bdrv_save_vmstate(bs, qiov, pos);
 -        }
 +        return -ENOMEDIUM;
 +    }
 +
++    if (!drv->bdrv_co_snapshot_block_status) {
++        return -ENOTSUP;
++    }
++
 +    bdrv_inc_in_flight(bs);
++    ret = drv->bdrv_co_snapshot_block_status(bs, want_zero, offset, bytes,
++                                             pnum, map, file);
++    bdrv_dec_in_flight(bs);
 +
-+    if (drv->bdrv_save_vmstate) {
++    return ret;
-+        ret = drv->bdrv_save_vmstate(bs, qiov, pos);
++}
      } else if (child_bs) {
 -        ret = bdrv_co_rw_vmstate(child_bs, qiov, pos, is_read);
 +        ret = bdrv_co_writev_vmstate(child_bs, qiov, pos);
      }
      bdrv_dec_in_flight(bs);
 +
-     return ret;
++int coroutine_fn
- }
++bdrv_co_pdiscard_snapshot(BlockDriverState *bs, int64_t offset, int64_t bytes)
++{
-@@ -XXX,XX +XXX,XX @@ int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf,
++    BlockDriver *drv = bs->drv;
-                       int64_t pos, int size)
++    int ret;
- {
++    IO_CODE();
-     QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, size);
++
--    int ret;
++    if (!drv) {
-+    int ret = bdrv_writev_vmstate(bs, &qiov, pos);
++        return -ENOMEDIUM;
++    }
--    ret = bdrv_writev_vmstate(bs, &qiov, pos);
++
--    if (ret < 0) {
++    if (!drv->bdrv_co_pdiscard_snapshot) {
--        return ret;
++        return -ENOTSUP;
--    }
++    }
--
++
--    return size;
++    bdrv_inc_in_flight(bs);
--}
++    ret = drv->bdrv_co_pdiscard_snapshot(bs, offset, bytes);
--
++    bdrv_dec_in_flight(bs);
--int bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
++
--{
++    return ret;
--    return bdrv_rw_vmstate(bs, qiov, pos, false);
++}
 +    return ret < 0 ? ret : size;
  }
  int bdrv_load_vmstate(BlockDriverState *bs, uint8_t *buf,
                        int64_t pos, int size)
  {
      QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, size);
 -    int ret;
 +    int ret = bdrv_readv_vmstate(bs, &qiov, pos);
 -    ret = bdrv_readv_vmstate(bs, &qiov, pos);
 -    if (ret < 0) {
 -        return ret;
 -    }
 -
 -    return size;
 -}
 -
 -int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
 -{
 -    return bdrv_rw_vmstate(bs, qiov, pos, true);
 +    return ret < 0 ? ret : size;
  }
  /**************************************************************/
 --
-.26.2
+.34.1

-[PULL 10/17] scripts: add block-coroutine-wrapper.py
+[PULL 18/23] block: introduce snapshot-access block driver
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-We have a very frequent pattern of creating a coroutine from a function
+The new block driver simply utilizes snapshot-access API of underlying
-with several arguments:
+block node.
-  - create a structure to pack parameters
+In further patches we want to use it like this:
-  - create _entry function to call original function taking parameters
-    from struct
+[guest]                   [NBD export]
-  - do different magic to handle completion: set ret to NOT_DONE or
+   |                            |
-    EINPROGRESS or use separate bool field
+   | root                       | root
-  - fill the struct and create coroutine from _entry function with this
+   v                 file       v
-    struct as a parameter
+[copy-before-write]<------[snapshot-access]
-  - do coroutine enter and BDRV_POLL_WHILE loop
+   |           |
+   | file      | target
-Let's reduce code duplication by generating coroutine wrappers.
+   v           v
+[active-disk] [temp.img]
-This patch adds scripts/block-coroutine-wrapper.py together with some
-friends, which will generate functions with declared prototypes marked
+This way, NBD client will be able to read snapshotted state of active
-by the 'generated_co_wrapper' specifier.
+disk, when active disk is continued to be written by guest. This is
+known as "fleecing", and currently uses another scheme based on qcow2
-The usage of new code generation is as follows:
+temporary image which backing file is active-disk. New scheme comes
+with benefits - see next commit.
-. define the coroutine function somewhere
+The other possible application is exporting internal snapshots of
-        int coroutine_fn bdrv_co_NAME(...) {...}
+qcow2, like this:
-. declare in some header file
+[guest]          [NBD export]
+   |                  |
-        int generated_co_wrapper bdrv_NAME(...);
+   | root             | root
+   v       file       v
-       with same list of parameters (generated_co_wrapper is
+[qcow2]<---------[snapshot-access]
-       defined in "include/block/block.h").
+For this, we'll need to implement snapshot-access API handlers in
-. Make sure the block_gen_c declaration in block/meson.build
+qcow2 driver, and improve snapshot-access block driver (and API) to
-       mentions the file with your marker function.
+make it possible to select snapshot by name. Another thing to improve
+is size of snapshot. Now for simplicity we just use size of bs->file,
-Still, no function is now marked, this work is for the following
+which is OK for backup, but for qcow2 snapshots export we'll need to
-commit.
+imporve snapshot-access API to get size of snapshot.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Message-Id: <20220303194349.2304213-12-vsementsov@virtuozzo.com>
-Message-Id: <20200924185414.28642-5-vsementsov@virtuozzo.com>
+[hreitz: Rebased on block GS/IO split]
-[Added encoding='utf-8' to open() calls as requested by Vladimir. Fixed
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 typo and grammar issues pointed out by Eric Blake.
 --Stefan]
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/block-gen.h                      |  49 +++++++
+ qapi/block-core.json             |   4 +-
- include/block/block.h                  |  10 ++
+ include/block/block_int-common.h |   6 ++
- block/meson.build                      |   8 ++
+ block/snapshot-access.c          | 132 +++++++++++++++++++++++++++++++
- docs/devel/block-coroutine-wrapper.rst |  54 +++++++
+ MAINTAINERS                      |   1 +
- docs/devel/index.rst                   |   1 +
+ block/meson.build                |   1 +
- scripts/block-coroutine-wrapper.py     | 188 +++++++++++++++++++++++++
+files changed, 143 insertions(+), 1 deletion(-)
-files changed, 310 insertions(+)
+ create mode 100644 block/snapshot-access.c
- create mode 100644 block/block-gen.h
- create mode 100644 docs/devel/block-coroutine-wrapper.rst
+diff --git a/qapi/block-core.json b/qapi/block-core.json
- create mode 100644 scripts/block-coroutine-wrapper.py
+index XXXXXXX..XXXXXXX 100644
+--- a/qapi/block-core.json
-diff --git a/block/block-gen.h b/block/block-gen.h
++++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
  # @blkreplay: Since 4.2
  # @compress: Since 5.0
  # @copy-before-write: Since 6.2
 +# @snapshot-access: Since 7.0
  #
  # Since: 2.9
  ##
  { 'enum': 'BlockdevDriver',
    'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
              'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
 -            'file', 'ftp', 'ftps', 'gluster',
 +            'file', 'snapshot-access', 'ftp', 'ftps', 'gluster',
              {'name': 'host_cdrom', 'if': 'HAVE_HOST_BLOCK_DEVICE' },
              {'name': 'host_device', 'if': 'HAVE_HOST_BLOCK_DEVICE' },
              'http', 'https', 'iscsi',
@@ -XXX,XX +XXX,XX @@
        'rbd':        'BlockdevOptionsRbd',
        'replication': { 'type': 'BlockdevOptionsReplication',
                         'if': 'CONFIG_REPLICATION' },
 +      'snapshot-access': 'BlockdevOptionsGenericFormat',
        'ssh':        'BlockdevOptionsSsh',
        'throttle':   'BlockdevOptionsThrottle',
        'vdi':        'BlockdevOptionsGenericFormat',
 diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/block/block_int-common.h
 +++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
       * in generic block-layer: no serializing, no alignment, no tracked
       * requests. So, block-driver that realizes these APIs is fully responsible
       * for synchronization between snapshot-access API and normal IO requests.
 +     *
 +     * TODO: To be able to support qcow2's internal snapshots, this API will
 +     * need to be extended to:
 +     * - be able to select a specific snapshot
 +     * - receive the snapshot's actual length (which may differ from bs's
 +     *   length)
       */
      int coroutine_fn (*bdrv_co_preadv_snapshot)(BlockDriverState *bs,
          int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
 diff --git a/block/snapshot-access.c b/block/snapshot-access.c
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
-+++ b/block/block-gen.h
++++ b/block/snapshot-access.c
 @@ -XXX,XX +XXX,XX @@
 +/*
-+ * Block coroutine wrapping core, used by auto-generated block/block-gen.c
++ * snapshot_access block driver
 + *
-+ * Copyright (c) 2003 Fabrice Bellard
++ * Copyright (c) 2022 Virtuozzo International GmbH.
-+ * Copyright (c) 2020 Virtuozzo International GmbH
++ *
-+ *
++ * Author:
-+ * Permission is hereby granted, free of charge, to any person obtaining a copy
++ *  Sementsov-Ogievskiy Vladimir <vsementsov@virtuozzo.com>
-+ * of this software and associated documentation files (the "Software"), to deal
++ *
-+ * in the Software without restriction, including without limitation the rights
++ * This program is free software; you can redistribute it and/or modify
-+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
++ * it under the terms of the GNU General Public License as published by
-+ * copies of the Software, and to permit persons to whom the Software is
++ * the Free Software Foundation; either version 2 of the License, or
-+ * furnished to do so, subject to the following conditions:
++ * (at your option) any later version.
 + *
-+ * The above copyright notice and this permission notice shall be included in
++ * This program is distributed in the hope that it will be useful,
-+ * all copies or substantial portions of the Software.
++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
-+ *
++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
++ * GNU General Public License for more details.
-+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
++ *
-+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
++ * You should have received a copy of the GNU General Public License
-+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
++ * along with this program. If not, see <http://www.gnu.org/licenses/>.
 + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 + * THE SOFTWARE.
 + */
 +
-+#ifndef BLOCK_BLOCK_GEN_H
++#include "qemu/osdep.h"
-+#define BLOCK_BLOCK_GEN_H
++
-+
++#include "sysemu/block-backend.h"
 +#include "qemu/cutils.h"
 +#include "block/block_int.h"
 +
-+/* Base structure for argument packing structures */
++static coroutine_fn int
-+typedef struct BdrvPollCo {
++snapshot_access_co_preadv_part(BlockDriverState *bs,
-+    BlockDriverState *bs;
++                               int64_t offset, int64_t bytes,
-+    bool in_progress;
++                               QEMUIOVector *qiov, size_t qiov_offset,
-+    int ret;
++                               BdrvRequestFlags flags)
-+    Coroutine *co; /* Keep pointer here for debugging */
++{
-+} BdrvPollCo;
++    if (flags) {
-+
++        return -ENOTSUP;
-+static inline int bdrv_poll_co(BdrvPollCo *s)
++    }
-+{
++
-+    assert(!qemu_in_coroutine());
++    return bdrv_co_preadv_snapshot(bs->file, offset, bytes, qiov, qiov_offset);
-+
++}
-+    bdrv_coroutine_enter(s->bs, s->co);
++
-+    BDRV_POLL_WHILE(s->bs, s->in_progress);
++static int coroutine_fn
-+
++snapshot_access_co_block_status(BlockDriverState *bs,
-+    return s->ret;
++                                bool want_zero, int64_t offset,
-+}
++                                int64_t bytes, int64_t *pnum,
-+
++                                int64_t *map, BlockDriverState **file)
-+#endif /* BLOCK_BLOCK_GEN_H */
++{
-diff --git a/include/block/block.h b/include/block/block.h
++    return bdrv_co_snapshot_block_status(bs->file->bs, want_zero, offset,
-index XXXXXXX..XXXXXXX 100644
++                                         bytes, pnum, map, file);
---- a/include/block/block.h
++}
-+++ b/include/block/block.h
++
-@@ -XXX,XX +XXX,XX @@
++static int coroutine_fn snapshot_access_co_pdiscard(BlockDriverState *bs,
- #include "block/blockjob.h"
++                                             int64_t offset, int64_t bytes)
- #include "qemu/hbitmap.h"
++{
++    return bdrv_co_pdiscard_snapshot(bs->file->bs, offset, bytes);
-+/*
++}
-+ * generated_co_wrapper
++
-+ *
++static int coroutine_fn
-+ * Function specifier, which does nothing but mark functions to be
++snapshot_access_co_pwrite_zeroes(BlockDriverState *bs,
-+ * generated by scripts/block-coroutine-wrapper.py
++                                 int64_t offset, int64_t bytes,
-+ *
++                                 BdrvRequestFlags flags)
-+ * Read more in docs/devel/block-coroutine-wrapper.rst
++{
-+ */
++    return -ENOTSUP;
-+#define generated_co_wrapper
++}
 +
- /* block.c */
++static coroutine_fn int
- typedef struct BlockDriver BlockDriver;
++snapshot_access_co_pwritev_part(BlockDriverState *bs,
- typedef struct BdrvChild BdrvChild;
++                                int64_t offset, int64_t bytes,
 +                                QEMUIOVector *qiov, size_t qiov_offset,
 +                                BdrvRequestFlags flags)
 +{
 +    return -ENOTSUP;
 +}
 +
 +
 +static void snapshot_access_refresh_filename(BlockDriverState *bs)
 +{
 +    pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
 +            bs->file->bs->filename);
 +}
 +
 +static int snapshot_access_open(BlockDriverState *bs, QDict *options, int flags,
 +                                Error **errp)
 +{
 +    bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
 +                               BDRV_CHILD_DATA | BDRV_CHILD_PRIMARY,
 +                               false, errp);
 +    if (!bs->file) {
 +        return -EINVAL;
 +    }
 +
 +    bs->total_sectors = bs->file->bs->total_sectors;
 +
 +    return 0;
 +}
 +
 +static void snapshot_access_child_perm(BlockDriverState *bs, BdrvChild *c,
 +                                BdrvChildRole role,
 +                                BlockReopenQueue *reopen_queue,
 +                                uint64_t perm, uint64_t shared,
 +                                uint64_t *nperm, uint64_t *nshared)
 +{
 +    /*
 +     * Currently, we don't need any permissions. If bs->file provides
 +     * snapshot-access API, we can use it.
 +     */
 +    *nperm = 0;
 +    *nshared = BLK_PERM_ALL;
 +}
 +
 +BlockDriver bdrv_snapshot_access_drv = {
 +    .format_name = "snapshot-access",
 +
 +    .bdrv_open                  = snapshot_access_open,
 +
 +    .bdrv_co_preadv_part        = snapshot_access_co_preadv_part,
 +    .bdrv_co_pwritev_part       = snapshot_access_co_pwritev_part,
 +    .bdrv_co_pwrite_zeroes      = snapshot_access_co_pwrite_zeroes,
 +    .bdrv_co_pdiscard           = snapshot_access_co_pdiscard,
 +    .bdrv_co_block_status       = snapshot_access_co_block_status,
 +
 +    .bdrv_refresh_filename      = snapshot_access_refresh_filename,
 +
 +    .bdrv_child_perm            = snapshot_access_child_perm,
 +};
 +
 +static void snapshot_access_init(void)
 +{
 +    bdrv_register(&bdrv_snapshot_access_drv);
 +}
 +
 +block_init(snapshot_access_init);
 diff --git a/MAINTAINERS b/MAINTAINERS
 index XXXXXXX..XXXXXXX 100644
 --- a/MAINTAINERS
 +++ b/MAINTAINERS
@@ -XXX,XX +XXX,XX @@ F: block/reqlist.c
  F: include/block/reqlist.h
  F: block/copy-before-write.h
  F: block/copy-before-write.c
 +F: block/snapshot-access.c
  F: include/block/aio_task.h
  F: block/aio_task.c
  F: util/qemu-co-shared-resource.c
 diff --git a/block/meson.build b/block/meson.build
 index XXXXXXX..XXXXXXX 100644
 --- a/block/meson.build
 +++ b/block/meson.build
-@@ -XXX,XX +XXX,XX @@ module_block_h = custom_target('module_block.h',
+@@ -XXX,XX +XXX,XX @@ block_ss.add(files(
-                                command: [module_block_py, '@OUTPUT0@', modsrc])
+   'raw-format.c',
- block_ss.add(module_block_h)
+   'reqlist.c',
+   'snapshot.c',
-+wrapper_py = find_program('../scripts/block-coroutine-wrapper.py')
++  'snapshot-access.c',
-+block_gen_c = custom_target('block-gen.c',
+   'throttle-groups.c',
-+                            output: 'block-gen.c',
+   'throttle.c',
-+                            input: files('../include/block/block.h',
+   'vhdx-endian.c',
 +                                         'coroutines.h'),
 +                            command: [wrapper_py, '@OUTPUT@', '@INPUT@'])
 +block_ss.add(block_gen_c)
 +
  block_ss.add(files('stream.c'))
  softmmu_ss.add(files('qapi-sysemu.c'))
 diff --git a/docs/devel/block-coroutine-wrapper.rst b/docs/devel/block-coroutine-wrapper.rst
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/docs/devel/block-coroutine-wrapper.rst
@@ -XXX,XX +XXX,XX @@
 +=======================
 +block-coroutine-wrapper
 +=======================
 +
 +A lot of functions in QEMU block layer (see ``block/*``) can only be
 +called in coroutine context. Such functions are normally marked by the
 +coroutine_fn specifier. Still, sometimes we need to call them from
 +non-coroutine context; for this we need to start a coroutine, run the
 +needed function from it and wait for the coroutine to finish in a
 +BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one
 +void* argument. So for each coroutine_fn function which needs a
 +non-coroutine interface, we should define a structure to pack the
 +parameters, define a separate function to unpack the parameters and
 +call the original function and finally define a new interface function
 +with same list of arguments as original one, which will pack the
 +parameters into a struct, create a coroutine, run it and wait in
 +BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand,
 +so we have a script to generate them.
 +
 +Usage
 +=====
 +
 +Assume we have defined the ``coroutine_fn`` function
 +``bdrv_co_foo(<some args>)`` and need a non-coroutine interface for it,
 +called ``bdrv_foo(<same args>)``. In this case the script can help. To
 +trigger the generation:
 +
 +1. You need ``bdrv_foo`` declaration somewhere (for example, in
 +   ``block/coroutines.h``) with the ``generated_co_wrapper`` mark,
 +   like this:
 +
 +.. code-block:: c
 +
 +    int generated_co_wrapper bdrv_foo(<some args>);
 +
 +2. You need to feed this declaration to block-coroutine-wrapper script.
 +   For this, add the .h (or .c) file with the declaration to the
 +   ``input: files(...)`` list of ``block_gen_c`` target declaration in
 +   ``block/meson.build``
 +
 +You are done. During the build, coroutine wrappers will be generated in
 +``<BUILD_DIR>/block/block-gen.c``.
 +
 +Links
 +=====
 +
 +1. The script location is ``scripts/block-coroutine-wrapper.py``.
 +
 +2. Generic place for private ``generated_co_wrapper`` declarations is
 +   ``block/coroutines.h``, for public declarations:
 +   ``include/block/block.h``
 +
 +3. The core API of generated coroutine wrappers is placed in
 +   (not generated) ``block/block-gen.h``
 diff --git a/docs/devel/index.rst b/docs/devel/index.rst
 index XXXXXXX..XXXXXXX 100644
 --- a/docs/devel/index.rst
 +++ b/docs/devel/index.rst
@@ -XXX,XX +XXX,XX @@ Contents:
     reset
     s390-dasd-ipl
     clocks
 +   block-coroutine-wrapper
 diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/scripts/block-coroutine-wrapper.py
@@ -XXX,XX +XXX,XX @@
 +#! /usr/bin/env python3
 +"""Generate coroutine wrappers for block subsystem.
 +
 +The program parses one or several concatenated c files from stdin,
 +searches for functions with the 'generated_co_wrapper' specifier
 +and generates corresponding wrappers on stdout.
 +
 +Usage: block-coroutine-wrapper.py generated-file.c FILE.[ch]...
 +
 +Copyright (c) 2020 Virtuozzo International GmbH.
 +
 +This program is free software; you can redistribute it and/or modify
 +it under the terms of the GNU General Public License as published by
 +the Free Software Foundation; either version 2 of the License, or
 +(at your option) any later version.
 +
 +This program is distributed in the hope that it will be useful,
 +but WITHOUT ANY WARRANTY; without even the implied warranty of
 +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 +GNU General Public License for more details.
 +
 +You should have received a copy of the GNU General Public License
 +along with this program.  If not, see <http://www.gnu.org/licenses/>.
 +"""
 +
 +import sys
 +import re
 +import subprocess
 +import json
 +from typing import Iterator
 +
 +
 +def prettify(code: str) -> str:
 +    """Prettify code using clang-format if available"""
 +
 +    try:
 +        style = json.dumps({
 +            'IndentWidth': 4,
 +            'BraceWrapping': {'AfterFunction': True},
 +            'BreakBeforeBraces': 'Custom',
 +            'SortIncludes': False,
 +            'MaxEmptyLinesToKeep': 2,
 +        })
 +        p = subprocess.run(['clang-format', f'-style={style}'], check=True,
 +                           encoding='utf-8', input=code,
 +                           stdout=subprocess.PIPE)
 +        return p.stdout
 +    except FileNotFoundError:
 +        return code
 +
 +
 +def gen_header():
 +    copyright = re.sub('^.*Copyright', 'Copyright', __doc__, flags=re.DOTALL)
 +    copyright = re.sub('^(?=.)', ' * ', copyright.strip(), flags=re.MULTILINE)
 +    copyright = re.sub('^$', ' *', copyright, flags=re.MULTILINE)
 +    return f"""\
 +/*
 + * File is generated by scripts/block-coroutine-wrapper.py
 + *
 +{copyright}
 + */
 +
 +#include "qemu/osdep.h"
 +#include "block/coroutines.h"
 +#include "block/block-gen.h"
 +#include "block/block_int.h"\
 +"""
 +
 +
 +class ParamDecl:
 +    param_re = re.compile(r'(?P<decl>'
 +                          r'(?P<type>.*[ *])'
 +                          r'(?P<name>[a-z][a-z0-9_]*)'
 +                          r')')
 +
 +    def __init__(self, param_decl: str) -> None:
 +        m = self.param_re.match(param_decl.strip())
 +        if m is None:
 +            raise ValueError(f'Wrong parameter declaration: "{param_decl}"')
 +        self.decl = m.group('decl')
 +        self.type = m.group('type')
 +        self.name = m.group('name')
 +
 +
 +class FuncDecl:
 +    def __init__(self, return_type: str, name: str, args: str) -> None:
 +        self.return_type = return_type.strip()
 +        self.name = name.strip()
 +        self.args = [ParamDecl(arg.strip()) for arg in args.split(',')]
 +
 +    def gen_list(self, format: str) -> str:
 +        return ', '.join(format.format_map(arg.__dict__) for arg in self.args)
 +
 +    def gen_block(self, format: str) -> str:
 +        return '\n'.join(format.format_map(arg.__dict__) for arg in self.args)
 +
 +
 +# Match wrappers declared with a generated_co_wrapper mark
 +func_decl_re = re.compile(r'^int\s*generated_co_wrapper\s*'
 +                          r'(?P<wrapper_name>[a-z][a-z0-9_]*)'
 +                          r'\((?P<args>[^)]*)\);$', re.MULTILINE)
 +
 +
 +def func_decl_iter(text: str) -> Iterator:
 +    for m in func_decl_re.finditer(text):
 +        yield FuncDecl(return_type='int',
 +                       name=m.group('wrapper_name'),
 +                       args=m.group('args'))
 +
 +
 +def snake_to_camel(func_name: str) -> str:
 +    """
 +    Convert underscore names like 'some_function_name' to camel-case like
 +    'SomeFunctionName'
 +    """
 +    words = func_name.split('_')
 +    words = [w[0].upper() + w[1:] for w in words]
 +    return ''.join(words)
 +
 +
 +def gen_wrapper(func: FuncDecl) -> str:
 +    assert func.name.startswith('bdrv_')
 +    assert not func.name.startswith('bdrv_co_')
 +    assert func.return_type == 'int'
 +    assert func.args[0].type in ['BlockDriverState *', 'BdrvChild *']
 +
 +    name = 'bdrv_co_' + func.name[5:]
 +    bs = 'bs' if func.args[0].type == 'BlockDriverState *' else 'child->bs'
 +    struct_name = snake_to_camel(name)
 +
 +    return f"""\
 +/*
 + * Wrappers for {name}
 + */
 +
 +typedef struct {struct_name} {{
 +    BdrvPollCo poll_state;
 +{ func.gen_block('    {decl};') }
 +}} {struct_name};
 +
 +static void coroutine_fn {name}_entry(void *opaque)
 +{{
 +    {struct_name} *s = opaque;
 +
 +    s->poll_state.ret = {name}({ func.gen_list('s->{name}') });
 +    s->poll_state.in_progress = false;
 +
 +    aio_wait_kick();
 +}}
 +
 +int {func.name}({ func.gen_list('{decl}') })
 +{{
 +    if (qemu_in_coroutine()) {{
 +        return {name}({ func.gen_list('{name}') });
 +    }} else {{
 +        {struct_name} s = {{
 +            .poll_state.bs = {bs},
 +            .poll_state.in_progress = true,
 +
 +{ func.gen_block('            .{name} = {name},') }
 +        }};
 +
 +        s.poll_state.co = qemu_coroutine_create({name}_entry, &s);
 +
 +        return bdrv_poll_co(&s.poll_state);
 +    }}
 +}}"""
 +
 +
 +def gen_wrappers(input_code: str) -> str:
 +    res = ''
 +    for func in func_decl_iter(input_code):
 +        res += '\n\n\n'
 +        res += gen_wrapper(func)
 +
 +    return prettify(res)  # prettify to wrap long lines
 +
 +
 +if __name__ == '__main__':
 +    if len(sys.argv) < 3:
 +        exit(f'Usage: {sys.argv[0]} OUT_FILE.c IN_FILE.[ch]...')
 +
 +    with open(sys.argv[1], 'w', encoding='utf-8') as f_out:
 +        f_out.write(gen_header())
 +        for fname in sys.argv[2:]:
 +            with open(fname, encoding='utf-8') as f_in:
 +                f_out.write(gen_wrappers(f_in.read()))
 +                f_out.write('\n')
 --
-.26.2
+.34.1

-[PULL 07/17] block: return error-code from bdrv_invalidate_cache
+[PULL 19/23] block: copy-before-write: realize snapshot-access API
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-This is the only coroutine wrapper from block.c and block/io.c which
+Current scheme of image fleecing looks like this:
-doesn't return a value, so let's convert it to the common behavior, to
-simplify moving to generated coroutine wrappers in a further commit.
+[guest]                    [NBD export]
+  |                              |
-Also, bdrv_invalidate_cache is a void function, returning error only
+  |root                          | root
-through **errp parameter, which is considered to be bad practice, as
+  v                              v
-it forces callers to define and propagate local_err variable, so
+[copy-before-write] -----> [temp.qcow2]
-conversion is good anyway.
+  |                 target  |
+  |file                     |backing
-This patch leaves the conversion of .bdrv_co_invalidate_cache() driver
+  v                         |
-callbacks and bdrv_invalidate_cache_all() for another day.
+[active disk] <-------------+
  - On guest writes copy-before-write filter copies old data from active
    disk to temp.qcow2. So fleecing client (NBD export) when reads
    changed regions from temp.qcow2 image and unchanged from active disk
    through backing link.
 This patch makes possible new image fleecing scheme:
 [guest]                   [NBD export]
    |                            |
    | root                       | root
    v                 file       v
 [copy-before-write]<------[snapshot-access]
    |           |
    | file      | target
    v           v
 [active-disk] [temp.img]
  - copy-before-write does CBW operations and also provides
    snapshot-access API. The API may be accessed through
    snapshot-access driver.
 Benefits of new scheme:
 . Access control: if remote client try to read data that not covered
    by original dirty bitmap used on copy-before-write open, client gets
    -EACCES.
 . Discard support: if remote client do DISCARD, this additionally to
    discarding data in temp.img informs block-copy process to not copy
    these clusters. Next read from discarded area will return -EACCES.
    This is significant thing: when fleecing user reads data that was
    not yet copied to temp.img, we can avoid copying it on further guest
    write.
 . Synchronisation between client reads and block-copy write is more
    efficient. In old scheme we just rely on BDRV_REQ_SERIALISING flag
    used for writes to temp.qcow2. New scheme is less blocking:
      - fleecing reads are never blocked: if data region is untouched or
        in-flight, we just read from active-disk, otherwise we read from
        temp.img
      - writes to temp.img are not blocked by fleecing reads
      - still, guest writes of-course are blocked by in-flight fleecing
        reads, that currently read from active-disk - it's the minimum
        necessary blocking
 . Temporary image may be of any format, as we don't rely on backing
    feature.
 . Permission relation are simplified. With old scheme we have to share
    write permission on target child of copy-before-write, otherwise
    backing link conflicts with copy-before-write file child write
    permissions. With new scheme we don't have backing link, and
    copy-before-write node may have unshared access to temporary node.
    (Not realized in this commit, will be in future).
 . Having control on fleecing reads we'll be able to implement
    alternative behavior on failed copy-before-write operations.
    Currently we just break guest request (that's a historical behavior
    of backup). But in some scenarios it's a bad behavior: better
    is to drop the backup as failed but don't break guest request.
    With new scheme we can simply unset some bits in a bitmap on CBW
    failure and further fleecing reads will -EACCES, or something like
    this. (Not implemented in this commit, will be in future)
    Additional application for this is implementing timeout for CBW
    operations.
 Iotest 257 output is updated, as two more bitmaps now live in
 copy-before-write filter.
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-Id: <20220303194349.2304213-13-vsementsov@virtuozzo.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
 Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
 Message-Id: <20200924185414.28642-2-vsementsov@virtuozzo.com>
 ---
- include/block/block.h |  2 +-
+ block/copy-before-write.c  | 212 ++++++++++++++++++++++++++++++++++-
- block.c               | 32 ++++++++++++++++++--------------
+ tests/qemu-iotests/257.out | 224 +++++++++++++++++++++++++++++++++++++
-files changed, 19 insertions(+), 15 deletions(-)
+files changed, 435 insertions(+), 1 deletion(-)
-diff --git a/include/block/block.h b/include/block/block.h
+diff --git a/block/copy-before-write.c b/block/copy-before-write.c
 index XXXXXXX..XXXXXXX 100644
---- a/include/block/block.h
+--- a/block/copy-before-write.c
-+++ b/include/block/block.h
++++ b/block/copy-before-write.c
-@@ -XXX,XX +XXX,XX @@ void bdrv_aio_cancel_async(BlockAIOCB *acb);
+@@ -XXX,XX +XXX,XX @@
- int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf);
+ #include "block/block-copy.h"
- /* Invalidate any cached metadata used by image formats */
+ #include "block/copy-before-write.h"
--void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp);
++#include "block/reqlist.h"
-+int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp);
- void bdrv_invalidate_cache_all(Error **errp);
+ #include "qapi/qapi-visit-block-core.h"
- int bdrv_inactivate_all(void);
+ typedef struct BDRVCopyBeforeWriteState {
-diff --git a/block.c b/block.c
+     BlockCopyState *bcs;
-index XXXXXXX..XXXXXXX 100644
+     BdrvChild *target;
---- a/block.c
++
-+++ b/block.c
++    /*
-@@ -XXX,XX +XXX,XX @@ void bdrv_init_with_whitelist(void)
++     * @lock: protects access to @access_bitmap, @done_bitmap and
-     bdrv_init();
++     * @frozen_read_reqs
 +     */
 +    CoMutex lock;
 +
 +    /*
 +     * @access_bitmap: represents areas allowed for reading by fleecing user.
 +     * Reading from non-dirty areas leads to -EACCES.
 +     */
 +    BdrvDirtyBitmap *access_bitmap;
 +
 +    /*
 +     * @done_bitmap: represents areas that was successfully copied to @target by
 +     * copy-before-write operations.
 +     */
 +    BdrvDirtyBitmap *done_bitmap;
 +
 +    /*
 +     * @frozen_read_reqs: current read requests for fleecing user in bs->file
 +     * node. These areas must not be rewritten by guest.
 +     */
 +    BlockReqList frozen_read_reqs;
  } BDRVCopyBeforeWriteState;
  static coroutine_fn int cbw_co_preadv(
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int cbw_co_preadv(
      return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);
  }
--static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
++/*
--                                                  Error **errp)
++ * Do copy-before-write operation.
-+static int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
++ *
-+                                                 Error **errp)
++ * On failure guest request must be failed too.
- {
++ *
-     BdrvChild *child, *parent;
++ * On success, we also wait for all in-flight fleecing read requests in source
-     uint64_t perm, shared_perm;
++ * node, and it's guaranteed that after cbw_do_copy_before_write() successful
-@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
++ * return there are no such requests and they will never appear.
-     BdrvDirtyBitmap *bm;
++ */
+ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
-     if (!bs->drv)  {
+         uint64_t offset, uint64_t bytes, BdrvRequestFlags flags)
--        return;
+ {
-+        return -ENOMEDIUM;
+     BDRVCopyBeforeWriteState *s = bs->opaque;
-     }
++    int ret;
+     uint64_t off, end;
-     QLIST_FOREACH(child, &bs->children, next) {
+     int64_t cluster_size = block_copy_cluster_size(s->bcs);
-         bdrv_co_invalidate_cache(child->bs, &local_err);
-         if (local_err) {
+@@ -XXX,XX +XXX,XX @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
-             error_propagate(errp, local_err);
+     off = QEMU_ALIGN_DOWN(offset, cluster_size);
--            return;
+     end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
-+            return -EINVAL;
-         }
+-    return block_copy(s->bcs, off, end - off, true);
-     }
++    ret = block_copy(s->bcs, off, end - off, true);
++    if (ret < 0) {
-@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
++        return ret;
-         ret = bdrv_check_perm(bs, NULL, perm, shared_perm, NULL, NULL, errp);
++    }
-         if (ret < 0) {
++
-             bs->open_flags |= BDRV_O_INACTIVE;
++    WITH_QEMU_LOCK_GUARD(&s->lock) {
--            return;
++        bdrv_set_dirty_bitmap(s->done_bitmap, off, end - off);
-+            return ret;
++        reqlist_wait_all(&s->frozen_read_reqs, off, end - off, &s->lock);
-         }
++    }
          bdrv_set_perm(bs, perm, shared_perm);
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
              if (local_err) {
                  bs->open_flags |= BDRV_O_INACTIVE;
                  error_propagate(errp, local_err);
 -                return;
 +                return -EINVAL;
              }
          }
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
          if (ret < 0) {
              bs->open_flags |= BDRV_O_INACTIVE;
              error_setg_errno(errp, -ret, "Could not refresh total sector count");
 -            return;
 +            return ret;
          }
      }
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
              if (local_err) {
                  bs->open_flags |= BDRV_O_INACTIVE;
                  error_propagate(errp, local_err);
 -                return;
 +                return -EINVAL;
              }
          }
      }
 +
 +    return 0;
  }
- typedef struct InvalidateCacheCo {
+ static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs,
-     BlockDriverState *bs;
+@@ -XXX,XX +XXX,XX @@ static int coroutine_fn cbw_co_flush(BlockDriverState *bs)
-     Error **errp;
+     return bdrv_co_flush(bs->file->bs);
-     bool done;
+ }
 +/*
 + * If @offset not accessible - return NULL.
 + *
 + * Otherwise, set @pnum to some bytes that accessible from @file (@file is set
 + * to bs->file or to s->target). Return newly allocated BlockReq object that
 + * should be than passed to cbw_snapshot_read_unlock().
 + *
 + * It's guaranteed that guest writes will not interact in the region until
 + * cbw_snapshot_read_unlock() called.
 + */
 +static BlockReq *cbw_snapshot_read_lock(BlockDriverState *bs,
 +                                        int64_t offset, int64_t bytes,
 +                                        int64_t *pnum, BdrvChild **file)
 +{
 +    BDRVCopyBeforeWriteState *s = bs->opaque;
 +    BlockReq *req = g_new(BlockReq, 1);
 +    bool done;
 +
 +    QEMU_LOCK_GUARD(&s->lock);
 +
 +    if (bdrv_dirty_bitmap_next_zero(s->access_bitmap, offset, bytes) != -1) {
 +        g_free(req);
 +        return NULL;
 +    }
 +
 +    done = bdrv_dirty_bitmap_status(s->done_bitmap, offset, bytes, pnum);
 +    if (done) {
 +        /*
 +         * Special invalid BlockReq, that is handled in
 +         * cbw_snapshot_read_unlock(). We don't need to lock something to read
 +         * from s->target.
 +         */
 +        *req = (BlockReq) {.offset = -1, .bytes = -1};
 +        *file = s->target;
 +    } else {
 +        reqlist_init_req(&s->frozen_read_reqs, req, offset, bytes);
 +        *file = bs->file;
 +    }
 +
 +    return req;
 +}
 +
 +static void cbw_snapshot_read_unlock(BlockDriverState *bs, BlockReq *req)
 +{
 +    BDRVCopyBeforeWriteState *s = bs->opaque;
 +
 +    if (req->offset == -1 && req->bytes == -1) {
 +        g_free(req);
 +        return;
 +    }
 +
 +    QEMU_LOCK_GUARD(&s->lock);
 +
 +    reqlist_remove_req(req);
 +    g_free(req);
 +}
 +
 +static coroutine_fn int
 +cbw_co_preadv_snapshot(BlockDriverState *bs, int64_t offset, int64_t bytes,
 +                       QEMUIOVector *qiov, size_t qiov_offset)
 +{
 +    BlockReq *req;
 +    BdrvChild *file;
 +    int ret;
- } InvalidateCacheCo;
++
++    /* TODO: upgrade to async loop using AioTask */
- static void coroutine_fn bdrv_invalidate_cache_co_entry(void *opaque)
++    while (bytes) {
- {
++        int64_t cur_bytes;
-     InvalidateCacheCo *ico = opaque;
++
--    bdrv_co_invalidate_cache(ico->bs, ico->errp);
++        req = cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &file);
-+    ico->ret = bdrv_co_invalidate_cache(ico->bs, ico->errp);
++        if (!req) {
-     ico->done = true;
++            return -EACCES;
-     aio_wait_kick();
++        }
 +
 +        ret = bdrv_co_preadv_part(file, offset, cur_bytes,
 +                                  qiov, qiov_offset, 0);
 +        cbw_snapshot_read_unlock(bs, req);
 +        if (ret < 0) {
 +            return ret;
 +        }
 +
 +        bytes -= cur_bytes;
 +        offset += cur_bytes;
 +        qiov_offset += cur_bytes;
 +    }
 +
 +    return 0;
 +}
 +
 +static int coroutine_fn
 +cbw_co_snapshot_block_status(BlockDriverState *bs,
 +                             bool want_zero, int64_t offset, int64_t bytes,
 +                             int64_t *pnum, int64_t *map,
 +                             BlockDriverState **file)
 +{
 +    BDRVCopyBeforeWriteState *s = bs->opaque;
 +    BlockReq *req;
 +    int ret;
 +    int64_t cur_bytes;
 +    BdrvChild *child;
 +
 +    req = cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &child);
 +    if (!req) {
 +        return -EACCES;
 +    }
 +
 +    ret = bdrv_block_status(child->bs, offset, cur_bytes, pnum, map, file);
 +    if (child == s->target) {
 +        /*
 +         * We refer to s->target only for areas that we've written to it.
 +         * And we can not report unallocated blocks in s->target: this will
 +         * break generic block-status-above logic, that will go to
 +         * copy-before-write filtered child in this case.
 +         */
 +        assert(ret & BDRV_BLOCK_ALLOCATED);
 +    }
 +
 +    cbw_snapshot_read_unlock(bs, req);
 +
 +    return ret;
 +}
 +
 +static int coroutine_fn cbw_co_pdiscard_snapshot(BlockDriverState *bs,
 +                                                 int64_t offset, int64_t bytes)
 +{
 +    BDRVCopyBeforeWriteState *s = bs->opaque;
 +
 +    WITH_QEMU_LOCK_GUARD(&s->lock) {
 +        bdrv_reset_dirty_bitmap(s->access_bitmap, offset, bytes);
 +    }
 +
 +    block_copy_reset(s->bcs, offset, bytes);
 +
 +    return bdrv_co_pdiscard(s->target, offset, bytes);
 +}
 +
  static void cbw_refresh_filename(BlockDriverState *bs)
  {
      pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
  {
      BDRVCopyBeforeWriteState *s = bs->opaque;
      BdrvDirtyBitmap *bitmap = NULL;
 +    int64_t cluster_size;
      bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
                                 BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
          return -EINVAL;
      }
 +    cluster_size = block_copy_cluster_size(s->bcs);
 +
 +    s->done_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
 +    if (!s->done_bitmap) {
 +        return -EINVAL;
 +    }
 +    bdrv_disable_dirty_bitmap(s->done_bitmap);
 +
 +    /* s->access_bitmap starts equal to bcs bitmap */
 +    s->access_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
 +    if (!s->access_bitmap) {
 +        return -EINVAL;
 +    }
 +    bdrv_disable_dirty_bitmap(s->access_bitmap);
 +    bdrv_dirty_bitmap_merge_internal(s->access_bitmap,
 +                                     block_copy_dirty_bitmap(s->bcs), NULL,
 +                                     true);
 +
 +    qemu_co_mutex_init(&s->lock);
 +    QLIST_INIT(&s->frozen_read_reqs);
 +
      return 0;
  }
--void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
+@@ -XXX,XX +XXX,XX @@ static void cbw_close(BlockDriverState *bs)
-+int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
+ {
- {
+     BDRVCopyBeforeWriteState *s = bs->opaque;
-     Coroutine *co;
-     InvalidateCacheCo ico = {
++    bdrv_release_dirty_bitmap(s->access_bitmap);
-@@ -XXX,XX +XXX,XX @@ void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
++    bdrv_release_dirty_bitmap(s->done_bitmap);
-         bdrv_coroutine_enter(bs, co);
++
-         BDRV_POLL_WHILE(bs, !ico.done);
+     block_copy_state_free(s->bcs);
-     }
+     s->bcs = NULL;
 +
 +    return ico.ret;
  }
+@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_cbw_filter = {
- void bdrv_invalidate_cache_all(Error **errp)
+     .bdrv_co_pdiscard           = cbw_co_pdiscard,
- {
+     .bdrv_co_flush              = cbw_co_flush,
-     BlockDriverState *bs;
--    Error *local_err = NULL;
++    .bdrv_co_preadv_snapshot       = cbw_co_preadv_snapshot,
-     BdrvNextIterator it;
++    .bdrv_co_pdiscard_snapshot     = cbw_co_pdiscard_snapshot,
++    .bdrv_co_snapshot_block_status = cbw_co_snapshot_block_status,
-     for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
++
-         AioContext *aio_context = bdrv_get_aio_context(bs);
+     .bdrv_refresh_filename      = cbw_refresh_filename,
-+        int ret;
+     .bdrv_child_perm            = cbw_child_perm,
-         aio_context_acquire(aio_context);
+diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
--        bdrv_invalidate_cache(bs, &local_err);
+index XXXXXXX..XXXXXXX 100644
-+        ret = bdrv_invalidate_cache(bs, errp);
+--- a/tests/qemu-iotests/257.out
-         aio_context_release(aio_context);
++++ b/tests/qemu-iotests/257.out
--        if (local_err) {
+@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
--            error_propagate(errp, local_err);
+ {"return": ""}
-+        if (ret < 0) {
+ {
-             bdrv_next_cleanup(&it);
+   "bitmaps": {
-             return;
++    "backup-top": [
-         }
++      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
  {"return": ""}
  {
    "bitmaps": {
 +    "backup-top": [
 +      {
 +        "busy": false,
 +        "count": 67108864,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      },
 +      {
 +        "busy": false,
 +        "count": 458752,
 +        "granularity": 65536,
 +        "persistent": false,
 +        "recording": false
 +      }
 +    ],
      "drive0": [
        {
          "busy": false,
 --
-.26.2
+.34.1

-[PULL 15/17] docs: add 'io_uring' option to 'aio' param in qemu-options.hx
+[PULL 20/23] iotests/image-fleecing: add test-case for fleecing format node
-From: Stefano Garzarella <sgarzare@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-When we added io_uring AIO engine, we forgot to update qemu-options.hx,
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-so qemu(1) man page and qemu help were outdated.
+Reviewed-by: Hanna Reitz <hreitz@redhat.com>
 Message-Id: <20220303194349.2304213-14-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/tests/image-fleecing     | 64 +++++++++++++-----
  tests/qemu-iotests/tests/image-fleecing.out | 74 ++++++++++++++++++++-
 files changed, 119 insertions(+), 19 deletions(-)
-Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
+diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+index XXXXXXX..XXXXXXX 100755
-Reviewed-by: Julia Suvorova <jusual@redhat.com>
+--- a/tests/qemu-iotests/tests/image-fleecing
-Reviewed-by: Pankaj Gupta <pankaj.gupta.linux@gmail.com>
++++ b/tests/qemu-iotests/tests/image-fleecing
-Message-Id: <20200924151511.131471-1-sgarzare@redhat.com>
+@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000',  '32k'), # Right-end of partial-left [1]
----
+              ('0xdc', '32M',       '32k'), # Left-end of partial-right [2]
- qemu-options.hx | 10 ++++++----
+              ('0xcd', '0x3ff0000', '64k')] # patterns[3]
-file changed, 6 insertions(+), 4 deletions(-)
+-def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
-diff --git a/qemu-options.hx b/qemu-options.hx
++def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
 +            fleece_img_path, nbd_sock_path, vm):
      log('--- Setting up images ---')
      log('')
      assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
 -    assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
 +    if use_snapshot_access_filter:
 +        assert use_cbw
 +        assert qemu_img('create', '-f', 'raw', fleece_img_path, '64M') == 0
 +    else:
 +        assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
      for p in patterns:
          qemu_io('-f', iotests.imgfmt,
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
      log('')
 -    # create tmp_node backed by src_node
 -    log(vm.qmp('blockdev-add', {
 -        'driver': 'qcow2',
 -        'node-name': tmp_node,
 -        'file': {
 +    if use_snapshot_access_filter:
 +        log(vm.qmp('blockdev-add', {
 +            'node-name': tmp_node,
              'driver': 'file',
              'filename': fleece_img_path,
 -        },
 -        'backing': src_node,
 -    }))
 +        }))
 +    else:
 +        # create tmp_node backed by src_node
 +        log(vm.qmp('blockdev-add', {
 +            'driver': 'qcow2',
 +            'node-name': tmp_node,
 +            'file': {
 +                'driver': 'file',
 +                'filename': fleece_img_path,
 +            },
 +            'backing': src_node,
 +        }))
      # Establish CBW from source to fleecing node
      if use_cbw:
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
          }))
          log(vm.qmp('qom-set', path=qom_path, property='drive', value='fl-cbw'))
 +
 +        if use_snapshot_access_filter:
 +            log(vm.qmp('blockdev-add', {
 +                'driver': 'snapshot-access',
 +                'node-name': 'fl-access',
 +                'file': 'fl-cbw',
 +            }))
      else:
          log(vm.qmp('blockdev-backup',
                     job_id='fleecing',
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
                     target=tmp_node,
                     sync='none'))
 +    export_node = 'fl-access' if use_snapshot_access_filter else tmp_node
 +
      log('')
      log('--- Setting up NBD Export ---')
      log('')
 -    nbd_uri = 'nbd+unix:///%s?socket=%s' % (tmp_node, nbd_sock_path)
 +    nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
      log(vm.qmp('nbd-server-start',
                 {'addr': {'type': 'unix',
                           'data': {'path': nbd_sock_path}}}))
 -    log(vm.qmp('nbd-server-add', device=tmp_node))
 +    log(vm.qmp('nbd-server-add', device=export_node))
      log('')
      log('--- Sanity Check ---')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
      log('--- Cleanup ---')
      log('')
 +    log(vm.qmp('nbd-server-stop'))
 +
      if use_cbw:
 +        if use_snapshot_access_filter:
 +            log(vm.qmp('blockdev-del', node_name='fl-access'))
          log(vm.qmp('qom-set', path=qom_path, property='drive', value=src_node))
          log(vm.qmp('blockdev-del', node_name='fl-cbw'))
      else:
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
          assert e is not None
          log(e, filters=[iotests.filter_qmp_event])
 -    log(vm.qmp('nbd-server-stop'))
      log(vm.qmp('blockdev-del', node_name=tmp_node))
      vm.shutdown()
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
      log('Done')
 -def test(use_cbw):
 +def test(use_cbw, use_snapshot_access_filter):
      with iotests.FilePath('base.img') as base_img_path, \
           iotests.FilePath('fleece.img') as fleece_img_path, \
           iotests.FilePath('nbd.sock',
                            base_dir=iotests.sock_dir) as nbd_sock_path, \
           iotests.VM() as vm:
 -        do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm)
 +        do_test(use_cbw, use_snapshot_access_filter, base_img_path,
 +                fleece_img_path, nbd_sock_path, vm)
  log('=== Test backup(sync=none) based fleecing ===\n')
 -test(False)
 +test(False, False)
 +
 +log('=== Test cbw-filter based fleecing ===\n')
 +test(True, False)
 -log('=== Test filter based fleecing ===\n')
 -test(True)
 +log('=== Test fleecing-format based fleecing ===\n')
 +test(True, True)
 diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
 index XXXXXXX..XXXXXXX 100644
---- a/qemu-options.hx
+--- a/tests/qemu-iotests/tests/image-fleecing.out
-+++ b/qemu-options.hx
++++ b/tests/qemu-iotests/tests/image-fleecing.out
-@@ -XXX,XX +XXX,XX @@ SRST
+@@ -XXX,XX +XXX,XX @@ read -P0 0x3fe0000 64k
-             The path to the image file in the local filesystem
+ --- Cleanup ---
-         ``aio``
--            Specifies the AIO backend (threads/native, default: threads)
++{"return": {}}
-+            Specifies the AIO backend (threads/native/io_uring,
+ {"return": {}}
-+            default: threads)
+ {"data": {"device": "fleecing", "len": 67108864, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+ {"return": {}}
-         ``locking``
++
-             Specifies whether the image file is protected with Linux OFD
++--- Confirming writes ---
-@@ -XXX,XX +XXX,XX @@ DEF("drive", HAS_ARG, QEMU_OPTION_drive,
++
-     "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n"
++read -P0xab 0 64k
-     "       [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n"
++read -P0xad 0x00f8000 64k
-     "       [,snapshot=on|off][,rerror=ignore|stop|report]\n"
++read -P0x1d 0x2008000 64k
--    "       [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n"
++read -P0xea 0x3fe0000 64k
-+    "       [,werror=ignore|stop|report|enospc][,id=name]\n"
++read -P0xd5 0x108000 32k
-+    "       [,aio=threads|native|io_uring]\n"
++read -P0xdc 32M 32k
-     "       [,readonly=on|off][,copy-on-read=on|off]\n"
++read -P0xcd 0x3ff0000 64k
-     "       [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n"
++
-     "       [[,bps=b]|[[,bps_rd=r][,bps_wr=w]]]\n"
++Done
-@@ -XXX,XX +XXX,XX @@ SRST
++=== Test cbw-filter based fleecing ===
-         The default mode is ``cache=writeback``.
++
++--- Setting up images ---
-     ``aio=aio``
++
--        aio is "threads", or "native" and selects between pthread based
++Done
--        disk I/O and native Linux AIO.
++
-+        aio is "threads", "native", or "io_uring" and selects between pthread
++--- Launching VM ---
-+        based disk I/O, native Linux AIO, or Linux io_uring API.
++
++Done
-     ``format=format``
++
-         Specify which disk format will be used rather than detecting the
++--- Setting up Fleecing Graph ---
 +
 +{"return": {}}
 +{"return": {}}
 +{"return": {}}
 +
 +--- Setting up NBD Export ---
 +
 +{"return": {}}
 +{"return": {}}
 +
 +--- Sanity Check ---
 +
 +read -P0x5d 0 64k
 +read -P0xd5 1M 64k
 +read -P0xdc 32M 64k
 +read -P0xcd 0x3ff0000 64k
 +read -P0 0x00f8000 32k
 +read -P0 0x2010000 32k
 +read -P0 0x3fe0000 64k
 +
 +--- Testing COW ---
 +
 +write -P0xab 0 64k
 +{"return": ""}
 +write -P0xad 0x00f8000 64k
 +{"return": ""}
 +write -P0x1d 0x2008000 64k
 +{"return": ""}
 +write -P0xea 0x3fe0000 64k
 +{"return": ""}
 +
 +--- Verifying Data ---
 +
 +read -P0x5d 0 64k
 +read -P0xd5 1M 64k
 +read -P0xdc 32M 64k
 +read -P0xcd 0x3ff0000 64k
 +read -P0 0x00f8000 32k
 +read -P0 0x2010000 32k
 +read -P0 0x3fe0000 64k
 +
 +--- Cleanup ---
 +
 +{"return": {}}
 +{"return": {}}
 +{"return": {}}
  {"return": {}}
  --- Confirming writes ---
@@ -XXX,XX +XXX,XX @@ read -P0xdc 32M 32k
  read -P0xcd 0x3ff0000 64k
  Done
 -=== Test filter based fleecing ===
 +=== Test fleecing-format based fleecing ===
  --- Setting up images ---
@@ -XXX,XX +XXX,XX @@ Done
  {"return": {}}
  {"return": {}}
  {"return": {}}
 +{"return": {}}
  --- Setting up NBD Export ---
@@ -XXX,XX +XXX,XX @@ read -P0 0x3fe0000 64k
  {"return": {}}
  {"return": {}}
  {"return": {}}
 +{"return": {}}
  --- Confirming writes ---
 --
-.26.2
+.34.1

-[PULL 06/17] block/nvme: Replace magic value by SCALE_MS definition
+[PULL 21/23] iotests.py: add qemu_io_pipe_and_status()
-From: Philippe Mathieu-Daudé <philmd@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Use self-explicit SCALE_MS definition instead of magic value
+Add helper that returns both status and output, to be used in the
-(missed in similar commit e4f310fe7f5).
+following commit
-Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Message-Id: <20220303194349.2304213-15-vsementsov@virtuozzo.com>
-Message-Id: <20200922083821.578519-7-philmd@redhat.com>
+Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
- block/nvme.c | 2 +-
+ tests/qemu-iotests/iotests.py | 3 +++
-file changed, 1 insertion(+), 1 deletion(-)
+file changed, 3 insertions(+)
-diff --git a/block/nvme.c b/block/nvme.c
+diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
 index XXXXXXX..XXXXXXX 100644
---- a/block/nvme.c
+--- a/tests/qemu-iotests/iotests.py
-+++ b/block/nvme.c
++++ b/tests/qemu-iotests/iotests.py
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
+@@ -XXX,XX +XXX,XX @@ def qemu_io(*args):
-                            CC_EN_MASK);
+     '''Run qemu-io and return the stdout data'''
-     /* Wait for CSTS.RDY = 1. */
+     return qemu_tool_pipe_and_status('qemu-io', qemu_io_wrap_args(args))[0]
-     now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
--    deadline = now + timeout_ms * 1000000;
++def qemu_io_pipe_and_status(*args):
-+    deadline = now + timeout_ms * SCALE_MS;
++    return qemu_tool_pipe_and_status('qemu-io', qemu_io_wrap_args(args))
-     while (!NVME_CSTS_RDY(le32_to_cpu(regs->csts))) {
++
-         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
+ def qemu_io_log(*args):
-             error_setg(errp, "Timeout while waiting for device to start (%"
+     result = qemu_io(*args)
      log(result, filters=[filter_testfiles, filter_qemu_io])
 --
-.26.2
+.34.1

-[PULL 04/17] block/nvme: Drop NVMeRegs structure, directly use NvmeBar
+[PULL 22/23] iotests/image-fleecing: add test case with bitmap
-From: Philippe Mathieu-Daudé <philmd@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-NVMeRegs only contains NvmeBar. Simplify the code by using NvmeBar
+Note that reads zero areas (not dirty in the bitmap) fails, that's
-directly.
+correct.
-This triggers a checkpatch.pl error:
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
 Message-Id: <20220303194349.2304213-16-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/tests/image-fleecing     | 38 +++++++---
  tests/qemu-iotests/tests/image-fleecing.out | 84 +++++++++++++++++++++
 files changed, 113 insertions(+), 9 deletions(-)
-  ERROR: Use of volatile is usually wrong, please add a comment
+diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
-  #30: FILE: block/nvme.c:691:
+index XXXXXXX..XXXXXXX 100755
-  +    volatile NvmeBar *regs;
+--- a/tests/qemu-iotests/tests/image-fleecing
++++ b/tests/qemu-iotests/tests/image-fleecing
-This is a false positive as in our case we are using I/O registers,
+@@ -XXX,XX +XXX,XX @@
-so the 'volatile' use is justified.
+ # Creator/Owner: John Snow <jsnow@redhat.com>
-Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+ import iotests
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+-from iotests import log, qemu_img, qemu_io, qemu_io_silent
-Message-Id: <20200922083821.578519-5-philmd@redhat.com>
++from iotests import log, qemu_img, qemu_io, qemu_io_silent, \
----
++    qemu_io_pipe_and_status
- block/nvme.c | 23 +++++++++--------------
-file changed, 9 insertions(+), 14 deletions(-)
+ iotests.script_initialize(
+-    supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk', 'vhdx', 'raw'],
-diff --git a/block/nvme.c b/block/nvme.c
++    supported_fmts=['qcow2'],
      supported_platforms=['linux'],
      required_fmts=['copy-before-write'],
 +    unsupported_imgopts=['compat']
  )
  patterns = [('0x5d', '0',         '64k'),
@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000',  '32k'), # Right-end of partial-left [1]
               ('0xcd', '0x3ff0000', '64k')] # patterns[3]
  def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
 -            fleece_img_path, nbd_sock_path, vm):
 +            fleece_img_path, nbd_sock_path, vm,
 +            bitmap=False):
      log('--- Setting up images ---')
      log('')
      assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
 +    if bitmap:
 +        assert qemu_img('bitmap', '--add', base_img_path, 'bitmap0') == 0
 +
      if use_snapshot_access_filter:
          assert use_cbw
          assert qemu_img('create', '-f', 'raw', fleece_img_path, '64M') == 0
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      # Establish CBW from source to fleecing node
      if use_cbw:
 -        log(vm.qmp('blockdev-add', {
 +        fl_cbw = {
              'driver': 'copy-before-write',
              'node-name': 'fl-cbw',
              'file': src_node,
              'target': tmp_node
 -        }))
 +        }
 +
 +        if bitmap:
 +            fl_cbw['bitmap'] = {'node': src_node, 'name': 'bitmap0'}
 +
 +        log(vm.qmp('blockdev-add', fl_cbw))
          log(vm.qmp('qom-set', path=qom_path, property='drive', value='fl-cbw'))
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      for p in patterns + zeroes:
          cmd = 'read -P%s %s %s' % p
          log(cmd)
 -        assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0
 +        out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
 +                                           nbd_uri)
 +        if ret != 0:
 +            print(out)
      log('')
      log('--- Testing COW ---')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      for p in patterns + zeroes:
          cmd = 'read -P%s %s %s' % p
          log(cmd)
 -        assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0
 +        out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
 +                                           nbd_uri)
 +        if ret != 0:
 +            print(out)
      log('')
      log('--- Cleanup ---')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      log('Done')
 -def test(use_cbw, use_snapshot_access_filter):
 +def test(use_cbw, use_snapshot_access_filter, bitmap=False):
      with iotests.FilePath('base.img') as base_img_path, \
           iotests.FilePath('fleece.img') as fleece_img_path, \
           iotests.FilePath('nbd.sock',
                            base_dir=iotests.sock_dir) as nbd_sock_path, \
           iotests.VM() as vm:
          do_test(use_cbw, use_snapshot_access_filter, base_img_path,
 -                fleece_img_path, nbd_sock_path, vm)
 +                fleece_img_path, nbd_sock_path, vm, bitmap=bitmap)
  log('=== Test backup(sync=none) based fleecing ===\n')
@@ -XXX,XX +XXX,XX @@ test(True, False)
  log('=== Test fleecing-format based fleecing ===\n')
  test(True, True)
 +
 +log('=== Test fleecing-format based fleecing with bitmap ===\n')
 +test(True, True, bitmap=True)
 diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
 index XXXXXXX..XXXXXXX 100644
---- a/block/nvme.c
+--- a/tests/qemu-iotests/tests/image-fleecing.out
-+++ b/block/nvme.c
++++ b/tests/qemu-iotests/tests/image-fleecing.out
-@@ -XXX,XX +XXX,XX @@ typedef struct {
+@@ -XXX,XX +XXX,XX @@ read -P0 0x00f8000 32k
-     QEMUBH      *completion_bh;
+ read -P0 0x2010000 32k
- } NVMeQueuePair;
+ read -P0 0x3fe0000 64k
--/* Memory mapped registers */
++--- Cleanup ---
--typedef volatile struct {
++
--    NvmeBar ctrl;
++{"return": {}}
--} NVMeRegs;
++{"return": {}}
--
++{"return": {}}
- #define INDEX_ADMIN     0
++{"return": {}}
- #define INDEX_IO(n)     (1 + n)
++{"return": {}}
++
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++--- Confirming writes ---
-     uint64_t timeout_ms;
++
-     uint64_t deadline, now;
++read -P0xab 0 64k
-     Error *local_err = NULL;
++read -P0xad 0x00f8000 64k
--    NVMeRegs *regs;
++read -P0x1d 0x2008000 64k
-+    volatile NvmeBar *regs = NULL;
++read -P0xea 0x3fe0000 64k
++read -P0xd5 0x108000 32k
-     qemu_co_mutex_init(&s->dma_map_lock);
++read -P0xdc 32M 32k
-     qemu_co_queue_init(&s->dma_flush_queue);
++read -P0xcd 0x3ff0000 64k
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++
-     /* Perform initialize sequence as described in NVMe spec "7.6.1
++Done
-      * Initialization". */
++=== Test fleecing-format based fleecing with bitmap ===
++
--    cap = le64_to_cpu(regs->ctrl.cap);
++--- Setting up images ---
-+    cap = le64_to_cpu(regs->cap);
++
-     if (!(cap & (1ULL << 37))) {
++Done
-         error_setg(errp, "Device doesn't support NVMe command set");
++
-         ret = -EINVAL;
++--- Launching VM ---
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++
-     timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
++Done
++
-     /* Reset device to get a clean state. */
++--- Setting up Fleecing Graph ---
--    regs->ctrl.cc = cpu_to_le32(le32_to_cpu(regs->ctrl.cc) & 0xFE);
++
-+    regs->cc = cpu_to_le32(le32_to_cpu(regs->cc) & 0xFE);
++{"return": {}}
-     /* Wait for CSTS.RDY = 0. */
++{"return": {}}
-     deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCALE_MS;
++{"return": {}}
--    while (le32_to_cpu(regs->ctrl.csts) & 0x1) {
++{"return": {}}
-+    while (le32_to_cpu(regs->csts) & 0x1) {
++
-         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
++--- Setting up NBD Export ---
-             error_setg(errp, "Timeout while waiting for device to reset (%"
++
-                              PRId64 " ms)",
++{"return": {}}
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++{"return": {}}
-     }
++
-     s->nr_queues = 1;
++--- Sanity Check ---
-     QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000);
++
--    regs->ctrl.aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
++read -P0x5d 0 64k
--    regs->ctrl.asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
++read -P0xd5 1M 64k
--    regs->ctrl.acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
++read -P0xdc 32M 64k
-+    regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
++read -P0xcd 0x3ff0000 64k
-+    regs->asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
++read -P0 0x00f8000 32k
-+    regs->acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
++read failed: Invalid argument
++
-     /* After setting up all control registers we can enable device now. */
++read -P0 0x2010000 32k
--    regs->ctrl.cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
++read failed: Invalid argument
-+    regs->cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
++
-                               (ctz32(NVME_SQ_ENTRY_BYTES) << 16) |
++read -P0 0x3fe0000 64k
-x1);
++read failed: Invalid argument
-     /* Wait for CSTS.RDY = 1. */
++
-     now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
++
-     deadline = now + timeout_ms * 1000000;
++--- Testing COW ---
--    while (!(le32_to_cpu(regs->ctrl.csts) & 0x1)) {
++
-+    while (!(le32_to_cpu(regs->csts) & 0x1)) {
++write -P0xab 0 64k
-         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
++{"return": ""}
-             error_setg(errp, "Timeout while waiting for device to start (%"
++write -P0xad 0x00f8000 64k
-                              PRId64 " ms)",
++{"return": ""}
 +write -P0x1d 0x2008000 64k
 +{"return": ""}
 +write -P0xea 0x3fe0000 64k
 +{"return": ""}
 +
 +--- Verifying Data ---
 +
 +read -P0x5d 0 64k
 +read -P0xd5 1M 64k
 +read -P0xdc 32M 64k
 +read -P0xcd 0x3ff0000 64k
 +read -P0 0x00f8000 32k
 +read failed: Invalid argument
 +
 +read -P0 0x2010000 32k
 +read failed: Invalid argument
 +
 +read -P0 0x3fe0000 64k
 +read failed: Invalid argument
 +
 +
  --- Cleanup ---
  {"return": {}}
 --
-.26.2
+.34.1

-[PULL 03/17] block/nvme: Reduce I/O registers scope
+[PULL 23/23] iotests/image-fleecing: test push backup with fleecing
-From: Philippe Mathieu-Daudé <philmd@redhat.com>
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-We only access the I/O register in nvme_init().
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Remove the reference in BDRVNVMeState and reduce its scope.
+Message-Id: <20220303194349.2304213-17-vsementsov@virtuozzo.com>
 Signed-off-by: Hanna Reitz <hreitz@redhat.com>
 ---
  tests/qemu-iotests/tests/image-fleecing     | 125 +++++++++++++++-----
  tests/qemu-iotests/tests/image-fleecing.out |  63 ++++++++++
 files changed, 156 insertions(+), 32 deletions(-)
-Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
+diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+index XXXXXXX..XXXXXXX 100755
-Message-Id: <20200922083821.578519-4-philmd@redhat.com>
+--- a/tests/qemu-iotests/tests/image-fleecing
----
++++ b/tests/qemu-iotests/tests/image-fleecing
- block/nvme.c | 29 ++++++++++++++++-------------
+@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000',  '32k'), # Right-end of partial-left [1]
-file changed, 16 insertions(+), 13 deletions(-)
+              ('0xdc', '32M',       '32k'), # Left-end of partial-right [2]
+              ('0xcd', '0x3ff0000', '64k')] # patterns[3]
-diff --git a/block/nvme.c b/block/nvme.c
 -def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
 -            fleece_img_path, nbd_sock_path, vm,
 +def do_test(vm, use_cbw, use_snapshot_access_filter, base_img_path,
 +            fleece_img_path, nbd_sock_path=None,
 +            target_img_path=None,
              bitmap=False):
 +    push_backup = target_img_path is not None
 +    assert (nbd_sock_path is not None) != push_backup
 +    if push_backup:
 +        assert use_cbw
 +
      log('--- Setting up images ---')
      log('')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      else:
          assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
 +    if push_backup:
 +        assert qemu_img('create', '-f', 'qcow2', target_img_path, '64M') == 0
 +
      for p in patterns:
          qemu_io('-f', iotests.imgfmt,
                  '-c', 'write -P%s %s %s' % p, base_img_path)
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      export_node = 'fl-access' if use_snapshot_access_filter else tmp_node
 -    log('')
 -    log('--- Setting up NBD Export ---')
 -    log('')
 +    if push_backup:
 +        log('')
 +        log('--- Starting actual backup ---')
 +        log('')
 -    nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
 -    log(vm.qmp('nbd-server-start',
 -               {'addr': {'type': 'unix',
 -                         'data': {'path': nbd_sock_path}}}))
 +        log(vm.qmp('blockdev-add', **{
 +            'driver': iotests.imgfmt,
 +            'node-name': 'target',
 +            'file': {
 +                'driver': 'file',
 +                'filename': target_img_path
 +            }
 +        }))
 +        log(vm.qmp('blockdev-backup', device=export_node,
 +                   sync='full', target='target',
 +                   job_id='push-backup', speed=1))
 +    else:
 +        log('')
 +        log('--- Setting up NBD Export ---')
 +        log('')
 -    log(vm.qmp('nbd-server-add', device=export_node))
 +        nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
 +        log(vm.qmp('nbd-server-start',
 +                   {'addr': { 'type': 'unix',
 +                              'data': { 'path': nbd_sock_path } } }))
 -    log('')
 -    log('--- Sanity Check ---')
 -    log('')
 +        log(vm.qmp('nbd-server-add', device=export_node))
 -    for p in patterns + zeroes:
 -        cmd = 'read -P%s %s %s' % p
 -        log(cmd)
 -        out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
 -                                           nbd_uri)
 -        if ret != 0:
 -            print(out)
 +        log('')
 +        log('--- Sanity Check ---')
 +        log('')
 +
 +        for p in patterns + zeroes:
 +            cmd = 'read -P%s %s %s' % p
 +            log(cmd)
 +            out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
 +                                               nbd_uri)
 +            if ret != 0:
 +                print(out)
      log('')
      log('--- Testing COW ---')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
          log(cmd)
          log(vm.hmp_qemu_io(qom_path, cmd, qdev=True))
 +    if push_backup:
 +        # Check that previous operations were done during backup, not after
 +        # If backup is already finished, it's possible that it was finished
 +        # even before hmp qemu_io write, and we didn't actually test
 +        # copy-before-write operation. This should not happen, as we use
 +        # speed=1. But worth checking.
 +        result = vm.qmp('query-block-jobs')
 +        assert len(result['return']) == 1
 +
 +        result = vm.qmp('block-job-set-speed', device='push-backup', speed=0)
 +        assert result == {'return': {}}
 +
 +        log(vm.event_wait(name='BLOCK_JOB_COMPLETED',
 +                          match={'data': {'device': 'push-backup'}}),
 +            filters=[iotests.filter_qmp_event])
 +        log(vm.qmp('blockdev-del', node_name='target'))
 +
      log('')
      log('--- Verifying Data ---')
      log('')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      for p in patterns + zeroes:
          cmd = 'read -P%s %s %s' % p
          log(cmd)
 -        out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
 -                                           nbd_uri)
 +        args = ['-r', '-c', cmd]
 +        if push_backup:
 +            args += [target_img_path]
 +        else:
 +            args += ['-f', 'raw', nbd_uri]
 +        out, ret = qemu_io_pipe_and_status(*args)
          if ret != 0:
              print(out)
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      log('--- Cleanup ---')
      log('')
 -    log(vm.qmp('nbd-server-stop'))
 +    if not push_backup:
 +        log(vm.qmp('nbd-server-stop'))
      if use_cbw:
          if use_snapshot_access_filter:
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
      log('Done')
 -def test(use_cbw, use_snapshot_access_filter, bitmap=False):
 +def test(use_cbw, use_snapshot_access_filter,
 +         nbd_sock_path=None, target_img_path=None, bitmap=False):
      with iotests.FilePath('base.img') as base_img_path, \
           iotests.FilePath('fleece.img') as fleece_img_path, \
 -         iotests.FilePath('nbd.sock',
 -                          base_dir=iotests.sock_dir) as nbd_sock_path, \
           iotests.VM() as vm:
 -        do_test(use_cbw, use_snapshot_access_filter, base_img_path,
 -                fleece_img_path, nbd_sock_path, vm, bitmap=bitmap)
 +        do_test(vm, use_cbw, use_snapshot_access_filter, base_img_path,
 +                fleece_img_path, nbd_sock_path, target_img_path,
 +                bitmap=bitmap)
 +
 +def test_pull(use_cbw, use_snapshot_access_filter, bitmap=False):
 +    with iotests.FilePath('nbd.sock',
 +                          base_dir=iotests.sock_dir) as nbd_sock_path:
 +        test(use_cbw, use_snapshot_access_filter, nbd_sock_path, None,
 +             bitmap=bitmap)
 +
 +def test_push():
 +    with iotests.FilePath('target.img') as target_img_path:
 +        test(True, True, None, target_img_path)
  log('=== Test backup(sync=none) based fleecing ===\n')
 -test(False, False)
 +test_pull(False, False)
  log('=== Test cbw-filter based fleecing ===\n')
 -test(True, False)
 +test_pull(True, False)
  log('=== Test fleecing-format based fleecing ===\n')
 -test(True, True)
 +test_pull(True, True)
  log('=== Test fleecing-format based fleecing with bitmap ===\n')
 -test(True, True, bitmap=True)
 +test_pull(True, True, bitmap=True)
 +
 +log('=== Test push backup with fleecing ===\n')
 +test_push()
 diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
 index XXXXXXX..XXXXXXX 100644
---- a/block/nvme.c
+--- a/tests/qemu-iotests/tests/image-fleecing.out
-+++ b/block/nvme.c
++++ b/tests/qemu-iotests/tests/image-fleecing.out
-@@ -XXX,XX +XXX,XX @@ enum {
+@@ -XXX,XX +XXX,XX @@ read -P0xdc 32M 32k
- struct BDRVNVMeState {
+ read -P0xcd 0x3ff0000 64k
-     AioContext *aio_context;
-     QEMUVFIOState *vfio;
+ Done
--    NVMeRegs *regs;
++=== Test push backup with fleecing ===
-     /* Memory mapped registers */
++
-     volatile struct {
++--- Setting up images ---
-         uint32_t sq_tail;
++
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++Done
-     uint64_t timeout_ms;
++
-     uint64_t deadline, now;
++--- Launching VM ---
-     Error *local_err = NULL;
++
-+    NVMeRegs *regs;
++Done
++
-     qemu_co_mutex_init(&s->dma_map_lock);
++--- Setting up Fleecing Graph ---
-     qemu_co_queue_init(&s->dma_flush_queue);
++
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++{"return": {}}
-         goto out;
++{"return": {}}
-     }
++{"return": {}}
++{"return": {}}
--    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, sizeof(NvmeBar),
++
--                                    PROT_READ | PROT_WRITE, errp);
++--- Starting actual backup ---
--    if (!s->regs) {
++
-+    regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, sizeof(NvmeBar),
++{"return": {}}
-+                                 PROT_READ | PROT_WRITE, errp);
++{"return": {}}
-+    if (!regs) {
++
-         ret = -EINVAL;
++--- Testing COW ---
-         goto out;
++
-     }
++write -P0xab 0 64k
-     /* Perform initialize sequence as described in NVMe spec "7.6.1
++{"return": ""}
-      * Initialization". */
++write -P0xad 0x00f8000 64k
++{"return": ""}
--    cap = le64_to_cpu(s->regs->ctrl.cap);
++write -P0x1d 0x2008000 64k
-+    cap = le64_to_cpu(regs->ctrl.cap);
++{"return": ""}
-     if (!(cap & (1ULL << 37))) {
++write -P0xea 0x3fe0000 64k
-         error_setg(errp, "Device doesn't support NVMe command set");
++{"return": ""}
-         ret = -EINVAL;
++{"data": {"device": "push-backup", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++{"return": {}}
-     timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
++
++--- Verifying Data ---
-     /* Reset device to get a clean state. */
++
--    s->regs->ctrl.cc = cpu_to_le32(le32_to_cpu(s->regs->ctrl.cc) & 0xFE);
++read -P0x5d 0 64k
-+    regs->ctrl.cc = cpu_to_le32(le32_to_cpu(regs->ctrl.cc) & 0xFE);
++read -P0xd5 1M 64k
-     /* Wait for CSTS.RDY = 0. */
++read -P0xdc 32M 64k
-     deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCALE_MS;
++read -P0xcd 0x3ff0000 64k
--    while (le32_to_cpu(s->regs->ctrl.csts) & 0x1) {
++read -P0 0x00f8000 32k
-+    while (le32_to_cpu(regs->ctrl.csts) & 0x1) {
++read -P0 0x2010000 32k
-         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
++read -P0 0x3fe0000 64k
-             error_setg(errp, "Timeout while waiting for device to reset (%"
++
-                              PRId64 " ms)",
++--- Cleanup ---
-@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
++
-     }
++{"return": {}}
-     s->nr_queues = 1;
++{"return": {}}
-     QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000);
++{"return": {}}
--    s->regs->ctrl.aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
++{"return": {}}
--    s->regs->ctrl.asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
++
--    s->regs->ctrl.acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
++--- Confirming writes ---
-+    regs->ctrl.aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
++
-+    regs->ctrl.asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
++read -P0xab 0 64k
-+    regs->ctrl.acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
++read -P0xad 0x00f8000 64k
++read -P0x1d 0x2008000 64k
-     /* After setting up all control registers we can enable device now. */
++read -P0xea 0x3fe0000 64k
--    s->regs->ctrl.cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
++read -P0xd5 0x108000 32k
-+    regs->ctrl.cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
++read -P0xdc 32M 32k
-                               (ctz32(NVME_SQ_ENTRY_BYTES) << 16) |
++read -P0xcd 0x3ff0000 64k
-x1);
++
-     /* Wait for CSTS.RDY = 1. */
++Done
      now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
      deadline = now + timeout_ms * 1000000;
 -    while (!(le32_to_cpu(s->regs->ctrl.csts) & 0x1)) {
 +    while (!(le32_to_cpu(regs->ctrl.csts) & 0x1)) {
          if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
              error_setg(errp, "Timeout while waiting for device to start (%"
                               PRId64 " ms)",
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
          ret = -EIO;
      }
  out:
 +    if (regs) {
 +        qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)regs, 0, sizeof(NvmeBar));
 +    }
 +
      /* Cleaning up is done in nvme_file_open() upon error. */
      return ret;
  }
@@ -XXX,XX +XXX,XX @@ static void nvme_close(BlockDriverState *bs)
      event_notifier_cleanup(&s->irq_notifier[MSIX_SHARED_IRQ_IDX]);
      qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->doorbells,
                              sizeof(NvmeBar), NVME_DOORBELL_SIZE);
 -    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->regs, 0, sizeof(NvmeBar));
      qemu_vfio_close(s->vfio);
      g_free(s->device);
 --
-.26.2
+.34.1

The following changes since commit b150cb8f67bf491a49a1cb1c7da151eeacbdbcc9:

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2020-09-29 13:18:54 +0100)

are available in the Git repository at:

https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to bc47831ff28d6f5830c9c8d74220131dc54c5253:

util/vfio-helpers: Rework the IOVA allocator to avoid IOVA reserved regions (2020-09-30 10:23:05 +0100)

----------------------------------------------------------------
Pull request

Note I have switched from GitHub to GitLab.

----------------------------------------------------------------

Eric Auger (2):
  util/vfio-helpers: Collect IOVA reserved regions
  util/vfio-helpers: Rework the IOVA allocator to avoid IOVA reserved
    regions

Philippe Mathieu-Daudé (6):
  util/vfio-helpers: Pass page protections to qemu_vfio_pci_map_bar()
  block/nvme: Map doorbells pages write-only
  block/nvme: Reduce I/O registers scope
  block/nvme: Drop NVMeRegs structure, directly use NvmeBar
  block/nvme: Use register definitions from 'block/nvme.h'
  block/nvme: Replace magic value by SCALE_MS definition

Stefano Garzarella (1):
  docs: add 'io_uring' option to 'aio' param in qemu-options.hx

Vladimir Sementsov-Ogievskiy (8):
  block: return error-code from bdrv_invalidate_cache
  block/io: refactor coroutine wrappers
  block: declare some coroutine functions in block/coroutines.h
  scripts: add block-coroutine-wrapper.py
  block: generate coroutine-wrapper code
  block: drop bdrv_prwv
  block/io: refactor save/load vmstate
  include/block/block.h: drop non-ascii quotation mark

block/block-gen.h                      |  49 ++++
 block/coroutines.h                     |  65 +++++
 include/block/block.h                  |  36 ++-
 include/qemu/vfio-helpers.h            |   2 +-
 block.c                                |  97 +------
 block/io.c                             | 339 ++++---------------------
 block/nvme.c                           |  73 +++---
 tests/test-bdrv-drain.c                |   2 +-
 util/vfio-helpers.c                    | 133 +++++++++-
 block/meson.build                      |   8 +
 docs/devel/block-coroutine-wrapper.rst |  54 ++++
 docs/devel/index.rst                   |   1 +
 qemu-options.hx                        |  10 +-
 scripts/block-coroutine-wrapper.py     | 188 ++++++++++++++
 14 files changed, 629 insertions(+), 428 deletions(-)
 create mode 100644 block/block-gen.h
 create mode 100644 block/coroutines.h
 create mode 100644 docs/devel/block-coroutine-wrapper.rst
 create mode 100644 scripts/block-coroutine-wrapper.py

-- 
2.26.2

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Pages are currently mapped READ/WRITE. To be able to use different
protections, add a new argument to qemu_vfio_pci_map_bar().

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200922083821.578519-2-philmd@redhat.com>
---
 include/qemu/vfio-helpers.h | 2 +-
 block/nvme.c                | 3 ++-
 util/vfio-helpers.c         | 4 ++--
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/include/qemu/vfio-helpers.h b/include/qemu/vfio-helpers.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/vfio-helpers.h
+++ b/include/qemu/vfio-helpers.h
@@ -XXX,XX +XXX,XX @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size,
 int qemu_vfio_dma_reset_temporary(QEMUVFIOState *s);
 void qemu_vfio_dma_unmap(QEMUVFIOState *s, void *host);
 void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index,
-                            uint64_t offset, uint64_t size,
+                            uint64_t offset, uint64_t size, int prot,
                             Error **errp);
 void qemu_vfio_pci_unmap_bar(QEMUVFIOState *s, int index, void *bar,
                              uint64_t offset, uint64_t size);
diff --git a/block/nvme.c b/block/nvme.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
         goto out;
     }
 
-    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, NVME_BAR_SIZE, errp);
+    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, NVME_BAR_SIZE,
+                                    PROT_READ | PROT_WRITE, errp);
     if (!s->regs) {
         ret = -EINVAL;
         goto out;
diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c
index XXXXXXX..XXXXXXX 100644
--- a/util/vfio-helpers.c
+++ b/util/vfio-helpers.c
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_pci_init_bar(QEMUVFIOState *s, int index, Error **errp)
  * Map a PCI bar area.
  */
 void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index,
-                            uint64_t offset, uint64_t size,
+                            uint64_t offset, uint64_t size, int prot,
                             Error **errp)
 {
     void *p;
     assert_bar_index_valid(s, index);
     p = mmap(NULL, MIN(size, s->bar_region_info[index].size - offset),
-             PROT_READ | PROT_WRITE, MAP_SHARED,
+             prot, MAP_SHARED,
              s->device, s->bar_region_info[index].offset + offset);
     if (p == MAP_FAILED) {
         error_setg_errno(errp, errno, "Failed to map BAR region");
-- 
2.26.2

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Per the datasheet sections 3.1.13/3.1.14:
  "The host should not read the doorbell registers."

As we don't need read access, map the doorbells with write-only
permission. We keep a reference to this mapped address in the
BDRVNVMeState structure.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200922083821.578519-3-philmd@redhat.com>
---
 block/nvme.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/block/nvme.c b/block/nvme.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -XXX,XX +XXX,XX @@
 #define NVME_SQ_ENTRY_BYTES 64
 #define NVME_CQ_ENTRY_BYTES 16
 #define NVME_QUEUE_SIZE 128
-#define NVME_BAR_SIZE 8192
+#define NVME_DOORBELL_SIZE 4096
 
 /*
  * We have to leave one slot empty as that is the full queue case where
@@ -XXX,XX +XXX,XX @@ typedef struct {
 /* Memory mapped registers */
 typedef volatile struct {
     NvmeBar ctrl;
-    struct {
-        uint32_t sq_tail;
-        uint32_t cq_head;
-    } doorbells[];
 } NVMeRegs;
 
 #define INDEX_ADMIN     0
@@ -XXX,XX +XXX,XX @@ struct BDRVNVMeState {
     AioContext *aio_context;
     QEMUVFIOState *vfio;
     NVMeRegs *regs;
+    /* Memory mapped registers */
+    volatile struct {
+        uint32_t sq_tail;
+        uint32_t cq_head;
+    } *doorbells;
     /* The submission/completion queue pairs.
      * [0]: admin queue.
      * [1..]: io queues.
@@ -XXX,XX +XXX,XX @@ static NVMeQueuePair *nvme_create_queue_pair(BDRVNVMeState *s,
         error_propagate(errp, local_err);
         goto fail;
     }
-    q->sq.doorbell = &s->regs->doorbells[idx * s->doorbell_scale].sq_tail;
+    q->sq.doorbell = &s->doorbells[idx * s->doorbell_scale].sq_tail;
 
     nvme_init_queue(s, &q->cq, size, NVME_CQ_ENTRY_BYTES, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
         goto fail;
     }
-    q->cq.doorbell = &s->regs->doorbells[idx * s->doorbell_scale].cq_head;
+    q->cq.doorbell = &s->doorbells[idx * s->doorbell_scale].cq_head;
 
     return q;
 fail:
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
         goto out;
     }
 
-    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, NVME_BAR_SIZE,
+    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, sizeof(NvmeBar),
                                     PROT_READ | PROT_WRITE, errp);
     if (!s->regs) {
         ret = -EINVAL;
         goto out;
     }
-
     /* Perform initialize sequence as described in NVMe spec "7.6.1
      * Initialization". */
 
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
         }
     }
 
+    s->doorbells = qemu_vfio_pci_map_bar(s->vfio, 0, sizeof(NvmeBar),
+                                         NVME_DOORBELL_SIZE, PROT_WRITE, errp);
+    if (!s->doorbells) {
+        ret = -EINVAL;
+        goto out;
+    }
+
     /* Set up admin queue. */
     s->queues = g_new(NVMeQueuePair *, 1);
     s->queues[INDEX_ADMIN] = nvme_create_queue_pair(s, aio_context, 0,
@@ -XXX,XX +XXX,XX @@ static void nvme_close(BlockDriverState *bs)
                            &s->irq_notifier[MSIX_SHARED_IRQ_IDX],
                            false, NULL, NULL);
     event_notifier_cleanup(&s->irq_notifier[MSIX_SHARED_IRQ_IDX]);
-    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->regs, 0, NVME_BAR_SIZE);
+    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->doorbells,
+                            sizeof(NvmeBar), NVME_DOORBELL_SIZE);
+    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->regs, 0, sizeof(NvmeBar));
     qemu_vfio_close(s->vfio);
 
     g_free(s->device);
-- 
2.26.2

From: Philippe Mathieu-Daudé <philmd@redhat.com>

We only access the I/O register in nvme_init().
Remove the reference in BDRVNVMeState and reduce its scope.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200922083821.578519-4-philmd@redhat.com>
---
 block/nvme.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/block/nvme.c b/block/nvme.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -XXX,XX +XXX,XX @@ enum {
 struct BDRVNVMeState {
     AioContext *aio_context;
     QEMUVFIOState *vfio;
-    NVMeRegs *regs;
     /* Memory mapped registers */
     volatile struct {
         uint32_t sq_tail;
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     uint64_t timeout_ms;
     uint64_t deadline, now;
     Error *local_err = NULL;
+    NVMeRegs *regs;
 
     qemu_co_mutex_init(&s->dma_map_lock);
     qemu_co_queue_init(&s->dma_flush_queue);
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
         goto out;
     }
 
-    s->regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, sizeof(NvmeBar),
-                                    PROT_READ | PROT_WRITE, errp);
-    if (!s->regs) {
+    regs = qemu_vfio_pci_map_bar(s->vfio, 0, 0, sizeof(NvmeBar),
+                                 PROT_READ | PROT_WRITE, errp);
+    if (!regs) {
         ret = -EINVAL;
         goto out;
     }
     /* Perform initialize sequence as described in NVMe spec "7.6.1
      * Initialization". */
 
-    cap = le64_to_cpu(s->regs->ctrl.cap);
+    cap = le64_to_cpu(regs->ctrl.cap);
     if (!(cap & (1ULL << 37))) {
         error_setg(errp, "Device doesn't support NVMe command set");
         ret = -EINVAL;
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
 
     /* Reset device to get a clean state. */
-    s->regs->ctrl.cc = cpu_to_le32(le32_to_cpu(s->regs->ctrl.cc) & 0xFE);
+    regs->ctrl.cc = cpu_to_le32(le32_to_cpu(regs->ctrl.cc) & 0xFE);
     /* Wait for CSTS.RDY = 0. */
     deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCALE_MS;
-    while (le32_to_cpu(s->regs->ctrl.csts) & 0x1) {
+    while (le32_to_cpu(regs->ctrl.csts) & 0x1) {
         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
             error_setg(errp, "Timeout while waiting for device to reset (%"
                              PRId64 " ms)",
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     }
     s->nr_queues = 1;
     QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000);
-    s->regs->ctrl.aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
-    s->regs->ctrl.asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
-    s->regs->ctrl.acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
+    regs->ctrl.aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
+    regs->ctrl.asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
+    regs->ctrl.acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
 
     /* After setting up all control registers we can enable device now. */
-    s->regs->ctrl.cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
+    regs->ctrl.cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
                               (ctz32(NVME_SQ_ENTRY_BYTES) << 16) |
                               0x1);
     /* Wait for CSTS.RDY = 1. */
     now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
     deadline = now + timeout_ms * 1000000;
-    while (!(le32_to_cpu(s->regs->ctrl.csts) & 0x1)) {
+    while (!(le32_to_cpu(regs->ctrl.csts) & 0x1)) {
         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
             error_setg(errp, "Timeout while waiting for device to start (%"
                              PRId64 " ms)",
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
         ret = -EIO;
     }
 out:
+    if (regs) {
+        qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)regs, 0, sizeof(NvmeBar));
+    }
+
     /* Cleaning up is done in nvme_file_open() upon error. */
     return ret;
 }
@@ -XXX,XX +XXX,XX @@ static void nvme_close(BlockDriverState *bs)
     event_notifier_cleanup(&s->irq_notifier[MSIX_SHARED_IRQ_IDX]);
     qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->doorbells,
                             sizeof(NvmeBar), NVME_DOORBELL_SIZE);
-    qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->regs, 0, sizeof(NvmeBar));
     qemu_vfio_close(s->vfio);
 
     g_free(s->device);
-- 
2.26.2

From: Philippe Mathieu-Daudé <philmd@redhat.com>

NVMeRegs only contains NvmeBar. Simplify the code by using NvmeBar
directly.

This triggers a checkpatch.pl error:

ERROR: Use of volatile is usually wrong, please add a comment
  #30: FILE: block/nvme.c:691:
  +    volatile NvmeBar *regs;

This is a false positive as in our case we are using I/O registers,
so the 'volatile' use is justified.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200922083821.578519-5-philmd@redhat.com>
---
 block/nvme.c | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/block/nvme.c b/block/nvme.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -XXX,XX +XXX,XX @@ typedef struct {
     QEMUBH      *completion_bh;
 } NVMeQueuePair;
 
-/* Memory mapped registers */
-typedef volatile struct {
-    NvmeBar ctrl;
-} NVMeRegs;
-
 #define INDEX_ADMIN     0
 #define INDEX_IO(n)     (1 + n)
 
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     uint64_t timeout_ms;
     uint64_t deadline, now;
     Error *local_err = NULL;
-    NVMeRegs *regs;
+    volatile NvmeBar *regs = NULL;
 
     qemu_co_mutex_init(&s->dma_map_lock);
     qemu_co_queue_init(&s->dma_flush_queue);
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     /* Perform initialize sequence as described in NVMe spec "7.6.1
      * Initialization". */
 
-    cap = le64_to_cpu(regs->ctrl.cap);
+    cap = le64_to_cpu(regs->cap);
     if (!(cap & (1ULL << 37))) {
         error_setg(errp, "Device doesn't support NVMe command set");
         ret = -EINVAL;
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
 
     /* Reset device to get a clean state. */
-    regs->ctrl.cc = cpu_to_le32(le32_to_cpu(regs->ctrl.cc) & 0xFE);
+    regs->cc = cpu_to_le32(le32_to_cpu(regs->cc) & 0xFE);
     /* Wait for CSTS.RDY = 0. */
     deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCALE_MS;
-    while (le32_to_cpu(regs->ctrl.csts) & 0x1) {
+    while (le32_to_cpu(regs->csts) & 0x1) {
         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
             error_setg(errp, "Timeout while waiting for device to reset (%"
                              PRId64 " ms)",
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     }
     s->nr_queues = 1;
     QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000);
-    regs->ctrl.aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
-    regs->ctrl.asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
-    regs->ctrl.acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
+    regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
+    regs->asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
+    regs->acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
 
     /* After setting up all control registers we can enable device now. */
-    regs->ctrl.cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
+    regs->cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
                               (ctz32(NVME_SQ_ENTRY_BYTES) << 16) |
                               0x1);
     /* Wait for CSTS.RDY = 1. */
     now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
     deadline = now + timeout_ms * 1000000;
-    while (!(le32_to_cpu(regs->ctrl.csts) & 0x1)) {
+    while (!(le32_to_cpu(regs->csts) & 0x1)) {
         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
             error_setg(errp, "Timeout while waiting for device to start (%"
                              PRId64 " ms)",
-- 
2.26.2

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Use the NVMe register definitions from "block/nvme.h" which
ease a bit reviewing the code while matching the datasheet.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200922083821.578519-6-philmd@redhat.com>
---
 block/nvme.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/block/nvme.c b/block/nvme.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
      * Initialization". */
 
     cap = le64_to_cpu(regs->cap);
-    if (!(cap & (1ULL << 37))) {
+    if (!NVME_CAP_CSS(cap)) {
         error_setg(errp, "Device doesn't support NVMe command set");
         ret = -EINVAL;
         goto out;
     }
 
-    s->page_size = MAX(4096, 1 << (12 + ((cap >> 48) & 0xF)));
-    s->doorbell_scale = (4 << (((cap >> 32) & 0xF))) / sizeof(uint32_t);
+    s->page_size = MAX(4096, 1 << NVME_CAP_MPSMIN(cap));
+    s->doorbell_scale = (4 << NVME_CAP_DSTRD(cap)) / sizeof(uint32_t);
     bs->bl.opt_mem_alignment = s->page_size;
-    timeout_ms = MIN(500 * ((cap >> 24) & 0xFF), 30000);
+    timeout_ms = MIN(500 * NVME_CAP_TO(cap), 30000);
 
     /* Reset device to get a clean state. */
     regs->cc = cpu_to_le32(le32_to_cpu(regs->cc) & 0xFE);
     /* Wait for CSTS.RDY = 0. */
     deadline = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + timeout_ms * SCALE_MS;
-    while (le32_to_cpu(regs->csts) & 0x1) {
+    while (NVME_CSTS_RDY(le32_to_cpu(regs->csts))) {
         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
             error_setg(errp, "Timeout while waiting for device to reset (%"
                              PRId64 " ms)",
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     }
     s->nr_queues = 1;
     QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000);
-    regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE);
+    regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << AQA_ACQS_SHIFT) |
+                            (NVME_QUEUE_SIZE << AQA_ASQS_SHIFT));
     regs->asq = cpu_to_le64(s->queues[INDEX_ADMIN]->sq.iova);
     regs->acq = cpu_to_le64(s->queues[INDEX_ADMIN]->cq.iova);
 
     /* After setting up all control registers we can enable device now. */
-    regs->cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << 20) |
-                              (ctz32(NVME_SQ_ENTRY_BYTES) << 16) |
-                              0x1);
+    regs->cc = cpu_to_le32((ctz32(NVME_CQ_ENTRY_BYTES) << CC_IOCQES_SHIFT) |
+                           (ctz32(NVME_SQ_ENTRY_BYTES) << CC_IOSQES_SHIFT) |
+                           CC_EN_MASK);
     /* Wait for CSTS.RDY = 1. */
     now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
     deadline = now + timeout_ms * 1000000;
-    while (!(le32_to_cpu(regs->csts) & 0x1)) {
+    while (!NVME_CSTS_RDY(le32_to_cpu(regs->csts))) {
         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
             error_setg(errp, "Timeout while waiting for device to start (%"
                              PRId64 " ms)",
-- 
2.26.2

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Use self-explicit SCALE_MS definition instead of magic value
(missed in similar commit e4f310fe7f5).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200922083821.578519-7-philmd@redhat.com>
---
 block/nvme.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/nvme.c b/block/nvme.c
index XXXXXXX..XXXXXXX 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -XXX,XX +XXX,XX @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
                            CC_EN_MASK);
     /* Wait for CSTS.RDY = 1. */
     now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
-    deadline = now + timeout_ms * 1000000;
+    deadline = now + timeout_ms * SCALE_MS;
     while (!NVME_CSTS_RDY(le32_to_cpu(regs->csts))) {
         if (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) > deadline) {
             error_setg(errp, "Timeout while waiting for device to start (%"
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

This is the only coroutine wrapper from block.c and block/io.c which
doesn't return a value, so let's convert it to the common behavior, to
simplify moving to generated coroutine wrappers in a further commit.

Also, bdrv_invalidate_cache is a void function, returning error only
through **errp parameter, which is considered to be bad practice, as
it forces callers to define and propagate local_err variable, so
conversion is good anyway.

This patch leaves the conversion of .bdrv_co_invalidate_cache() driver
callbacks and bdrv_invalidate_cache_all() for another day.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200924185414.28642-2-vsementsov@virtuozzo.com>
---
 include/block/block.h |  2 +-
 block.c               | 32 ++++++++++++++++++--------------
 2 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ void bdrv_aio_cancel_async(BlockAIOCB *acb);
 int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf);
 
 /* Invalidate any cached metadata used by image formats */
-void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp);
+int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp);
 void bdrv_invalidate_cache_all(Error **errp);
 int bdrv_inactivate_all(void);
 
diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ void bdrv_init_with_whitelist(void)
     bdrv_init();
 }
 
-static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
-                                                  Error **errp)
+static int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
+                                                 Error **errp)
 {
     BdrvChild *child, *parent;
     uint64_t perm, shared_perm;
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
     BdrvDirtyBitmap *bm;
 
     if (!bs->drv)  {
-        return;
+        return -ENOMEDIUM;
     }
 
     QLIST_FOREACH(child, &bs->children, next) {
         bdrv_co_invalidate_cache(child->bs, &local_err);
         if (local_err) {
             error_propagate(errp, local_err);
-            return;
+            return -EINVAL;
         }
     }
 
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
         ret = bdrv_check_perm(bs, NULL, perm, shared_perm, NULL, NULL, errp);
         if (ret < 0) {
             bs->open_flags |= BDRV_O_INACTIVE;
-            return;
+            return ret;
         }
         bdrv_set_perm(bs, perm, shared_perm);
 
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
             if (local_err) {
                 bs->open_flags |= BDRV_O_INACTIVE;
                 error_propagate(errp, local_err);
-                return;
+                return -EINVAL;
             }
         }
 
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
         if (ret < 0) {
             bs->open_flags |= BDRV_O_INACTIVE;
             error_setg_errno(errp, -ret, "Could not refresh total sector count");
-            return;
+            return ret;
         }
     }
 
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
             if (local_err) {
                 bs->open_flags |= BDRV_O_INACTIVE;
                 error_propagate(errp, local_err);
-                return;
+                return -EINVAL;
             }
         }
     }
+
+    return 0;
 }
 
 typedef struct InvalidateCacheCo {
     BlockDriverState *bs;
     Error **errp;
     bool done;
+    int ret;
 } InvalidateCacheCo;
 
 static void coroutine_fn bdrv_invalidate_cache_co_entry(void *opaque)
 {
     InvalidateCacheCo *ico = opaque;
-    bdrv_co_invalidate_cache(ico->bs, ico->errp);
+    ico->ret = bdrv_co_invalidate_cache(ico->bs, ico->errp);
     ico->done = true;
     aio_wait_kick();
 }
 
-void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
+int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
 {
     Coroutine *co;
     InvalidateCacheCo ico = {
@@ -XXX,XX +XXX,XX @@ void bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
         bdrv_coroutine_enter(bs, co);
         BDRV_POLL_WHILE(bs, !ico.done);
     }
+
+    return ico.ret;
 }
 
 void bdrv_invalidate_cache_all(Error **errp)
 {
     BlockDriverState *bs;
-    Error *local_err = NULL;
     BdrvNextIterator it;
 
     for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
         AioContext *aio_context = bdrv_get_aio_context(bs);
+        int ret;
 
         aio_context_acquire(aio_context);
-        bdrv_invalidate_cache(bs, &local_err);
+        ret = bdrv_invalidate_cache(bs, errp);
         aio_context_release(aio_context);
-        if (local_err) {
-            error_propagate(errp, local_err);
+        if (ret < 0) {
             bdrv_next_cleanup(&it);
             return;
         }
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Most of our coroutine wrappers already follow this convention:

We have 'coroutine_fn bdrv_co_<something>(<normal argument list>)' as
the core function, and a wrapper 'bdrv_<something>(<same argument
list>)' which does parameter packing and calls bdrv_run_co().

The only outsiders are the bdrv_prwv_co and
bdrv_common_block_status_above wrappers. Let's refactor them to behave
as the others, it simplifies further conversion of coroutine wrappers.

This patch adds an indirection layer, but it will be compensated by
a further commit, which will drop bdrv_co_prwv together with the
is_write logic, to keep the read and write paths separate.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200924185414.28642-3-vsementsov@virtuozzo.com>
---
 block/io.c | 60 +++++++++++++++++++++++++++++-------------------------
 1 file changed, 32 insertions(+), 28 deletions(-)

diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ typedef struct RwCo {
     BdrvRequestFlags flags;
 } RwCo;
 
+static int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
+                                     QEMUIOVector *qiov, bool is_write,
+                                     BdrvRequestFlags flags)
+{
+    if (is_write) {
+        return bdrv_co_pwritev(child, offset, qiov->size, qiov, flags);
+    } else {
+        return bdrv_co_preadv(child, offset, qiov->size, qiov, flags);
+    }
+}
+
 static int coroutine_fn bdrv_rw_co_entry(void *opaque)
 {
     RwCo *rwco = opaque;
 
-    if (!rwco->is_write) {
-        return bdrv_co_preadv(rwco->child, rwco->offset,
-                              rwco->qiov->size, rwco->qiov,
-                              rwco->flags);
-    } else {
-        return bdrv_co_pwritev(rwco->child, rwco->offset,
-                               rwco->qiov->size, rwco->qiov,
-                               rwco->flags);
-    }
+    return bdrv_co_prwv(rwco->child, rwco->offset, rwco->qiov,
+                        rwco->is_write, rwco->flags);
 }
 
 /*
  * Process a vectored synchronous request using coroutines
  */
-static int bdrv_prwv_co(BdrvChild *child, int64_t offset,
-                        QEMUIOVector *qiov, bool is_write,
-                        BdrvRequestFlags flags)
+static int bdrv_prwv(BdrvChild *child, int64_t offset,
+                     QEMUIOVector *qiov, bool is_write,
+                     BdrvRequestFlags flags)
 {
     RwCo rwco = {
         .child = child,
@@ -XXX,XX +XXX,XX @@ int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
 {
     QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, NULL, bytes);
 
-    return bdrv_prwv_co(child, offset, &qiov, true,
-                        BDRV_REQ_ZERO_WRITE | flags);
+    return bdrv_prwv(child, offset, &qiov, true, BDRV_REQ_ZERO_WRITE | flags);
 }
 
 /*
@@ -XXX,XX +XXX,XX @@ int bdrv_preadv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
 {
     int ret;
 
-    ret = bdrv_prwv_co(child, offset, qiov, false, 0);
+    ret = bdrv_prwv(child, offset, qiov, false, 0);
     if (ret < 0) {
         return ret;
     }
@@ -XXX,XX +XXX,XX @@ int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
 {
     int ret;
 
-    ret = bdrv_prwv_co(child, offset, qiov, true, 0);
+    ret = bdrv_prwv(child, offset, qiov, true, 0);
     if (ret < 0) {
         return ret;
     }
@@ -XXX,XX +XXX,XX @@ early_out:
     return ret;
 }
 
-static int coroutine_fn bdrv_co_block_status_above(BlockDriverState *bs,
-                                                   BlockDriverState *base,
-                                                   bool want_zero,
-                                                   int64_t offset,
-                                                   int64_t bytes,
-                                                   int64_t *pnum,
-                                                   int64_t *map,
-                                                   BlockDriverState **file)
+static int coroutine_fn
+bdrv_co_common_block_status_above(BlockDriverState *bs,
+                                  BlockDriverState *base,
+                                  bool want_zero,
+                                  int64_t offset,
+                                  int64_t bytes,
+                                  int64_t *pnum,
+                                  int64_t *map,
+                                  BlockDriverState **file)
 {
     BlockDriverState *p;
     int ret = 0;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_block_status_above_co_entry(void *opaque)
 {
     BdrvCoBlockStatusData *data = opaque;
 
-    return bdrv_co_block_status_above(data->bs, data->base,
-                                      data->want_zero,
-                                      data->offset, data->bytes,
-                                      data->pnum, data->map, data->file);
+    return bdrv_co_common_block_status_above(data->bs, data->base,
+                                             data->want_zero,
+                                             data->offset, data->bytes,
+                                             data->pnum, data->map, data->file);
 }
 
 /*
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We are going to keep coroutine-wrappers code (structure-packing
parameters, BDRV_POLL wrapper functions) in separate auto-generated
files. So, we'll need a header with declaration of original _co_
functions, for those which are static now. As well, we'll need
declarations for wrapper functions. Do these declarations now, as a
preparation step.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200924185414.28642-4-vsementsov@virtuozzo.com>
---
 block/coroutines.h | 67 ++++++++++++++++++++++++++++++++++++++++++++++
 block.c            |  8 +++---
 block/io.c         | 34 +++++++++++------------
 3 files changed, 88 insertions(+), 21 deletions(-)
 create mode 100644 block/coroutines.h

diff --git a/block/coroutines.h b/block/coroutines.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/block/coroutines.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * Block layer I/O functions
+ *
+ * Copyright (c) 2003 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef BLOCK_COROUTINES_INT_H
+#define BLOCK_COROUTINES_INT_H
+
+#include "block/block_int.h"
+
+int coroutine_fn bdrv_co_check(BlockDriverState *bs,
+                               BdrvCheckResult *res, BdrvCheckMode fix);
+int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp);
+
+int coroutine_fn
+bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
+             bool is_write, BdrvRequestFlags flags);
+int
+bdrv_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
+          bool is_write, BdrvRequestFlags flags);
+
+int coroutine_fn
+bdrv_co_common_block_status_above(BlockDriverState *bs,
+                                  BlockDriverState *base,
+                                  bool want_zero,
+                                  int64_t offset,
+                                  int64_t bytes,
+                                  int64_t *pnum,
+                                  int64_t *map,
+                                  BlockDriverState **file);
+int
+bdrv_common_block_status_above(BlockDriverState *bs,
+                               BlockDriverState *base,
+                               bool want_zero,
+                               int64_t offset,
+                               int64_t bytes,
+                               int64_t *pnum,
+                               int64_t *map,
+                               BlockDriverState **file);
+
+int coroutine_fn
+bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
+                   bool is_read);
+int
+bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
+                bool is_read);
+
+#endif /* BLOCK_COROUTINES_INT_H */
diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@
 #include "qemu/timer.h"
 #include "qemu/cutils.h"
 #include "qemu/id.h"
+#include "block/coroutines.h"
 
 #ifdef CONFIG_BSD
 #include <sys/ioctl.h>
@@ -XXX,XX +XXX,XX @@ static void bdrv_delete(BlockDriverState *bs)
  * free of errors) or -errno when an internal error occurred. The results of the
  * check are stored in res.
  */
-static int coroutine_fn bdrv_co_check(BlockDriverState *bs,
-                                      BdrvCheckResult *res, BdrvCheckMode fix)
+int coroutine_fn bdrv_co_check(BlockDriverState *bs,
+                               BdrvCheckResult *res, BdrvCheckMode fix)
 {
     if (bs->drv == NULL) {
         return -ENOMEDIUM;
@@ -XXX,XX +XXX,XX @@ void bdrv_init_with_whitelist(void)
     bdrv_init();
 }
 
-static int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs,
-                                                 Error **errp)
+int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
 {
     BdrvChild *child, *parent;
     uint64_t perm, shared_perm;
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@
 #include "block/blockjob.h"
 #include "block/blockjob_int.h"
 #include "block/block_int.h"
+#include "block/coroutines.h"
 #include "qemu/cutils.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
@@ -XXX,XX +XXX,XX @@ typedef struct RwCo {
     BdrvRequestFlags flags;
 } RwCo;
 
-static int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
-                                     QEMUIOVector *qiov, bool is_write,
-                                     BdrvRequestFlags flags)
+int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
+                              QEMUIOVector *qiov, bool is_write,
+                              BdrvRequestFlags flags)
 {
     if (is_write) {
         return bdrv_co_pwritev(child, offset, qiov->size, qiov, flags);
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_rw_co_entry(void *opaque)
 /*
  * Process a vectored synchronous request using coroutines
  */
-static int bdrv_prwv(BdrvChild *child, int64_t offset,
-                     QEMUIOVector *qiov, bool is_write,
-                     BdrvRequestFlags flags)
+int bdrv_prwv(BdrvChild *child, int64_t offset,
+              QEMUIOVector *qiov, bool is_write,
+              BdrvRequestFlags flags)
 {
     RwCo rwco = {
         .child = child,
@@ -XXX,XX +XXX,XX @@ early_out:
     return ret;
 }
 
-static int coroutine_fn
+int coroutine_fn
 bdrv_co_common_block_status_above(BlockDriverState *bs,
                                   BlockDriverState *base,
                                   bool want_zero,
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_block_status_above_co_entry(void *opaque)
  *
  * See bdrv_co_block_status_above() for details.
  */
-static int bdrv_common_block_status_above(BlockDriverState *bs,
-                                          BlockDriverState *base,
-                                          bool want_zero, int64_t offset,
-                                          int64_t bytes, int64_t *pnum,
-                                          int64_t *map,
-                                          BlockDriverState **file)
+int bdrv_common_block_status_above(BlockDriverState *bs,
+                                   BlockDriverState *base,
+                                   bool want_zero, int64_t offset,
+                                   int64_t bytes, int64_t *pnum,
+                                   int64_t *map,
+                                   BlockDriverState **file)
 {
     BdrvCoBlockStatusData data = {
         .bs = bs,
@@ -XXX,XX +XXX,XX @@ typedef struct BdrvVmstateCo {
     bool                is_read;
 } BdrvVmstateCo;
 
-static int coroutine_fn
+int coroutine_fn
 bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
                    bool is_read)
 {
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_rw_vmstate_entry(void *opaque)
     return bdrv_co_rw_vmstate(co->bs, co->qiov, co->pos, co->is_read);
 }
 
-static inline int
-bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
-                bool is_read)
+int bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
+                    bool is_read)
 {
     BdrvVmstateCo data = {
         .bs         = bs,
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We have a very frequent pattern of creating a coroutine from a function
with several arguments:

- create a structure to pack parameters
  - create _entry function to call original function taking parameters
    from struct
  - do different magic to handle completion: set ret to NOT_DONE or
    EINPROGRESS or use separate bool field
  - fill the struct and create coroutine from _entry function with this
    struct as a parameter
  - do coroutine enter and BDRV_POLL_WHILE loop

Let's reduce code duplication by generating coroutine wrappers.

This patch adds scripts/block-coroutine-wrapper.py together with some
friends, which will generate functions with declared prototypes marked
by the 'generated_co_wrapper' specifier.

The usage of new code generation is as follows:

1. define the coroutine function somewhere

int coroutine_fn bdrv_co_NAME(...) {...}

2. declare in some header file

int generated_co_wrapper bdrv_NAME(...);

with same list of parameters (generated_co_wrapper is
       defined in "include/block/block.h").

3. Make sure the block_gen_c declaration in block/meson.build
       mentions the file with your marker function.

Still, no function is now marked, this work is for the following
commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200924185414.28642-5-vsementsov@virtuozzo.com>
[Added encoding='utf-8' to open() calls as requested by Vladimir. Fixed
typo and grammar issues pointed out by Eric Blake.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/block-gen.h                      |  49 +++++++
 include/block/block.h                  |  10 ++
 block/meson.build                      |   8 ++
 docs/devel/block-coroutine-wrapper.rst |  54 +++++++
 docs/devel/index.rst                   |   1 +
 scripts/block-coroutine-wrapper.py     | 188 +++++++++++++++++++++++++
 6 files changed, 310 insertions(+)
 create mode 100644 block/block-gen.h
 create mode 100644 docs/devel/block-coroutine-wrapper.rst
 create mode 100644 scripts/block-coroutine-wrapper.py

diff --git a/block/block-gen.h b/block/block-gen.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/block/block-gen.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * Block coroutine wrapping core, used by auto-generated block/block-gen.c
+ *
+ * Copyright (c) 2003 Fabrice Bellard
+ * Copyright (c) 2020 Virtuozzo International GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef BLOCK_BLOCK_GEN_H
+#define BLOCK_BLOCK_GEN_H
+
+#include "block/block_int.h"
+
+/* Base structure for argument packing structures */
+typedef struct BdrvPollCo {
+    BlockDriverState *bs;
+    bool in_progress;
+    int ret;
+    Coroutine *co; /* Keep pointer here for debugging */
+} BdrvPollCo;
+
+static inline int bdrv_poll_co(BdrvPollCo *s)
+{
+    assert(!qemu_in_coroutine());
+
+    bdrv_coroutine_enter(s->bs, s->co);
+    BDRV_POLL_WHILE(s->bs, s->in_progress);
+
+    return s->ret;
+}
+
+#endif /* BLOCK_BLOCK_GEN_H */
diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@
 #include "block/blockjob.h"
 #include "qemu/hbitmap.h"
 
+/*
+ * generated_co_wrapper
+ *
+ * Function specifier, which does nothing but mark functions to be
+ * generated by scripts/block-coroutine-wrapper.py
+ *
+ * Read more in docs/devel/block-coroutine-wrapper.rst
+ */
+#define generated_co_wrapper
+
 /* block.c */
 typedef struct BlockDriver BlockDriver;
 typedef struct BdrvChild BdrvChild;
diff --git a/block/meson.build b/block/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -XXX,XX +XXX,XX @@ module_block_h = custom_target('module_block.h',
                                command: [module_block_py, '@OUTPUT0@', modsrc])
 block_ss.add(module_block_h)
 
+wrapper_py = find_program('../scripts/block-coroutine-wrapper.py')
+block_gen_c = custom_target('block-gen.c',
+                            output: 'block-gen.c',
+                            input: files('../include/block/block.h',
+                                         'coroutines.h'),
+                            command: [wrapper_py, '@OUTPUT@', '@INPUT@'])
+block_ss.add(block_gen_c)
+
 block_ss.add(files('stream.c'))
 
 softmmu_ss.add(files('qapi-sysemu.c'))
diff --git a/docs/devel/block-coroutine-wrapper.rst b/docs/devel/block-coroutine-wrapper.rst
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/docs/devel/block-coroutine-wrapper.rst
@@ -XXX,XX +XXX,XX @@
+=======================
+block-coroutine-wrapper
+=======================
+
+A lot of functions in QEMU block layer (see ``block/*``) can only be
+called in coroutine context. Such functions are normally marked by the
+coroutine_fn specifier. Still, sometimes we need to call them from
+non-coroutine context; for this we need to start a coroutine, run the
+needed function from it and wait for the coroutine to finish in a
+BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one
+void* argument. So for each coroutine_fn function which needs a
+non-coroutine interface, we should define a structure to pack the
+parameters, define a separate function to unpack the parameters and
+call the original function and finally define a new interface function
+with same list of arguments as original one, which will pack the
+parameters into a struct, create a coroutine, run it and wait in
+BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand,
+so we have a script to generate them.
+
+Usage
+=====
+
+Assume we have defined the ``coroutine_fn`` function
+``bdrv_co_foo(<some args>)`` and need a non-coroutine interface for it,
+called ``bdrv_foo(<same args>)``. In this case the script can help. To
+trigger the generation:
+
+1. You need ``bdrv_foo`` declaration somewhere (for example, in
+   ``block/coroutines.h``) with the ``generated_co_wrapper`` mark,
+   like this:
+
+.. code-block:: c
+
+    int generated_co_wrapper bdrv_foo(<some args>);
+
+2. You need to feed this declaration to block-coroutine-wrapper script.
+   For this, add the .h (or .c) file with the declaration to the
+   ``input: files(...)`` list of ``block_gen_c`` target declaration in
+   ``block/meson.build``
+
+You are done. During the build, coroutine wrappers will be generated in
+``<BUILD_DIR>/block/block-gen.c``.
+
+Links
+=====
+
+1. The script location is ``scripts/block-coroutine-wrapper.py``.
+
+2. Generic place for private ``generated_co_wrapper`` declarations is
+   ``block/coroutines.h``, for public declarations:
+   ``include/block/block.h``
+
+3. The core API of generated coroutine wrappers is placed in
+   (not generated) ``block/block-gen.h``
diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index XXXXXXX..XXXXXXX 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -XXX,XX +XXX,XX @@ Contents:
    reset
    s390-dasd-ipl
    clocks
+   block-coroutine-wrapper
diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/scripts/block-coroutine-wrapper.py
@@ -XXX,XX +XXX,XX @@
+#! /usr/bin/env python3
+"""Generate coroutine wrappers for block subsystem.
+
+The program parses one or several concatenated c files from stdin,
+searches for functions with the 'generated_co_wrapper' specifier
+and generates corresponding wrappers on stdout.
+
+Usage: block-coroutine-wrapper.py generated-file.c FILE.[ch]...
+
+Copyright (c) 2020 Virtuozzo International GmbH.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import sys
+import re
+import subprocess
+import json
+from typing import Iterator
+
+
+def prettify(code: str) -> str:
+    """Prettify code using clang-format if available"""
+
+    try:
+        style = json.dumps({
+            'IndentWidth': 4,
+            'BraceWrapping': {'AfterFunction': True},
+            'BreakBeforeBraces': 'Custom',
+            'SortIncludes': False,
+            'MaxEmptyLinesToKeep': 2,
+        })
+        p = subprocess.run(['clang-format', f'-style={style}'], check=True,
+                           encoding='utf-8', input=code,
+                           stdout=subprocess.PIPE)
+        return p.stdout
+    except FileNotFoundError:
+        return code
+
+
+def gen_header():
+    copyright = re.sub('^.*Copyright', 'Copyright', __doc__, flags=re.DOTALL)
+    copyright = re.sub('^(?=.)', ' * ', copyright.strip(), flags=re.MULTILINE)
+    copyright = re.sub('^$', ' *', copyright, flags=re.MULTILINE)
+    return f"""\
+/*
+ * File is generated by scripts/block-coroutine-wrapper.py
+ *
+{copyright}
+ */
+
+#include "qemu/osdep.h"
+#include "block/coroutines.h"
+#include "block/block-gen.h"
+#include "block/block_int.h"\
+"""
+
+
+class ParamDecl:
+    param_re = re.compile(r'(?P<decl>'
+                          r'(?P<type>.*[ *])'
+                          r'(?P<name>[a-z][a-z0-9_]*)'
+                          r')')
+
+    def __init__(self, param_decl: str) -> None:
+        m = self.param_re.match(param_decl.strip())
+        if m is None:
+            raise ValueError(f'Wrong parameter declaration: "{param_decl}"')
+        self.decl = m.group('decl')
+        self.type = m.group('type')
+        self.name = m.group('name')
+
+
+class FuncDecl:
+    def __init__(self, return_type: str, name: str, args: str) -> None:
+        self.return_type = return_type.strip()
+        self.name = name.strip()
+        self.args = [ParamDecl(arg.strip()) for arg in args.split(',')]
+
+    def gen_list(self, format: str) -> str:
+        return ', '.join(format.format_map(arg.__dict__) for arg in self.args)
+
+    def gen_block(self, format: str) -> str:
+        return '\n'.join(format.format_map(arg.__dict__) for arg in self.args)
+
+
+# Match wrappers declared with a generated_co_wrapper mark
+func_decl_re = re.compile(r'^int\s*generated_co_wrapper\s*'
+                          r'(?P<wrapper_name>[a-z][a-z0-9_]*)'
+                          r'$(?P<args>[^)]*)$;$', re.MULTILINE)
+
+
+def func_decl_iter(text: str) -> Iterator:
+    for m in func_decl_re.finditer(text):
+        yield FuncDecl(return_type='int',
+                       name=m.group('wrapper_name'),
+                       args=m.group('args'))
+
+
+def snake_to_camel(func_name: str) -> str:
+    """
+    Convert underscore names like 'some_function_name' to camel-case like
+    'SomeFunctionName'
+    """
+    words = func_name.split('_')
+    words = [w[0].upper() + w[1:] for w in words]
+    return ''.join(words)
+
+
+def gen_wrapper(func: FuncDecl) -> str:
+    assert func.name.startswith('bdrv_')
+    assert not func.name.startswith('bdrv_co_')
+    assert func.return_type == 'int'
+    assert func.args[0].type in ['BlockDriverState *', 'BdrvChild *']
+
+    name = 'bdrv_co_' + func.name[5:]
+    bs = 'bs' if func.args[0].type == 'BlockDriverState *' else 'child->bs'
+    struct_name = snake_to_camel(name)
+
+    return f"""\
+/*
+ * Wrappers for {name}
+ */
+
+typedef struct {struct_name} {{
+    BdrvPollCo poll_state;
+{ func.gen_block('    {decl};') }
+}} {struct_name};
+
+static void coroutine_fn {name}_entry(void *opaque)
+{{
+    {struct_name} *s = opaque;
+
+    s->poll_state.ret = {name}({ func.gen_list('s->{name}') });
+    s->poll_state.in_progress = false;
+
+    aio_wait_kick();
+}}
+
+int {func.name}({ func.gen_list('{decl}') })
+{{
+    if (qemu_in_coroutine()) {{
+        return {name}({ func.gen_list('{name}') });
+    }} else {{
+        {struct_name} s = {{
+            .poll_state.bs = {bs},
+            .poll_state.in_progress = true,
+
+{ func.gen_block('            .{name} = {name},') }
+        }};
+
+        s.poll_state.co = qemu_coroutine_create({name}_entry, &s);
+
+        return bdrv_poll_co(&s.poll_state);
+    }}
+}}"""
+
+
+def gen_wrappers(input_code: str) -> str:
+    res = ''
+    for func in func_decl_iter(input_code):
+        res += '\n\n\n'
+        res += gen_wrapper(func)
+
+    return prettify(res)  # prettify to wrap long lines
+
+
+if __name__ == '__main__':
+    if len(sys.argv) < 3:
+        exit(f'Usage: {sys.argv[0]} OUT_FILE.c IN_FILE.[ch]...')
+
+    with open(sys.argv[1], 'w', encoding='utf-8') as f_out:
+        f_out.write(gen_header())
+        for fname in sys.argv[2:]:
+            with open(fname, encoding='utf-8') as f_in:
+                f_out.write(gen_wrappers(f_in.read()))
+                f_out.write('\n')
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Use code generation implemented in previous commit to generated
coroutine wrappers in block.c and block/io.c

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200924185414.28642-6-vsementsov@virtuozzo.com>
---
 block/coroutines.h    |   6 +-
 include/block/block.h |  16 ++--
 block.c               |  73 ---------------
 block/io.c            | 212 ------------------------------------------
 4 files changed, 13 insertions(+), 294 deletions(-)

diff --git a/block/coroutines.h b/block/coroutines.h
index XXXXXXX..XXXXXXX 100644
--- a/block/coroutines.h
+++ b/block/coroutines.h
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp);
 int coroutine_fn
 bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
              bool is_write, BdrvRequestFlags flags);
-int
+int generated_co_wrapper
 bdrv_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
           bool is_write, BdrvRequestFlags flags);
 
@@ -XXX,XX +XXX,XX @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
                                   int64_t *pnum,
                                   int64_t *map,
                                   BlockDriverState **file);
-int
+int generated_co_wrapper
 bdrv_common_block_status_above(BlockDriverState *bs,
                                BlockDriverState *base,
                                bool want_zero,
@@ -XXX,XX +XXX,XX @@ bdrv_common_block_status_above(BlockDriverState *bs,
 int coroutine_fn
 bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
                    bool is_read);
-int
+int generated_co_wrapper
 bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
                 bool is_read);
 
diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ void bdrv_refresh_filename(BlockDriverState *bs);
 int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact,
                                   PreallocMode prealloc, BdrvRequestFlags flags,
                                   Error **errp);
-int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact,
-                  PreallocMode prealloc, BdrvRequestFlags flags, Error **errp);
+int generated_co_wrapper
+bdrv_truncate(BdrvChild *child, int64_t offset, bool exact,
+              PreallocMode prealloc, BdrvRequestFlags flags, Error **errp);
 
 int64_t bdrv_nb_sectors(BlockDriverState *bs);
 int64_t bdrv_getlength(BlockDriverState *bs);
@@ -XXX,XX +XXX,XX @@ typedef enum {
     BDRV_FIX_ERRORS   = 2,
 } BdrvCheckMode;
 
-int bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix);
+int generated_co_wrapper bdrv_check(BlockDriverState *bs, BdrvCheckResult *res,
+                                    BdrvCheckMode fix);
 
 /* The units of offset and total_work_size may be chosen arbitrarily by the
  * block driver; total_work_size may change during the course of the amendment
@@ -XXX,XX +XXX,XX @@ void bdrv_aio_cancel_async(BlockAIOCB *acb);
 int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf);
 
 /* Invalidate any cached metadata used by image formats */
-int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp);
+int generated_co_wrapper bdrv_invalidate_cache(BlockDriverState *bs,
+                                               Error **errp);
 void bdrv_invalidate_cache_all(Error **errp);
 int bdrv_inactivate_all(void);
 
 /* Ensure contents are flushed to disk.  */
-int bdrv_flush(BlockDriverState *bs);
+int generated_co_wrapper bdrv_flush(BlockDriverState *bs);
 int coroutine_fn bdrv_co_flush(BlockDriverState *bs);
 int bdrv_flush_all(void);
 void bdrv_close_all(void);
@@ -XXX,XX +XXX,XX @@ void bdrv_drain_all(void);
     AIO_WAIT_WHILE(bdrv_get_aio_context(bs_),              \
                    cond); })
 
-int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
+int generated_co_wrapper bdrv_pdiscard(BdrvChild *child, int64_t offset,
+                                       int64_t bytes);
 int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes);
 int bdrv_has_zero_init_1(BlockDriverState *bs);
 int bdrv_has_zero_init(BlockDriverState *bs);
diff --git a/block.c b/block.c
index XXXXXXX..XXXXXXX 100644
--- a/block.c
+++ b/block.c
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_check(BlockDriverState *bs,
     return bs->drv->bdrv_co_check(bs, res, fix);
 }
 
-typedef struct CheckCo {
-    BlockDriverState *bs;
-    BdrvCheckResult *res;
-    BdrvCheckMode fix;
-    int ret;
-} CheckCo;
-
-static void coroutine_fn bdrv_check_co_entry(void *opaque)
-{
-    CheckCo *cco = opaque;
-    cco->ret = bdrv_co_check(cco->bs, cco->res, cco->fix);
-    aio_wait_kick();
-}
-
-int bdrv_check(BlockDriverState *bs,
-               BdrvCheckResult *res, BdrvCheckMode fix)
-{
-    Coroutine *co;
-    CheckCo cco = {
-        .bs = bs,
-        .res = res,
-        .ret = -EINPROGRESS,
-        .fix = fix,
-    };
-
-    if (qemu_in_coroutine()) {
-        /* Fast-path if already in coroutine context */
-        bdrv_check_co_entry(&cco);
-    } else {
-        co = qemu_coroutine_create(bdrv_check_co_entry, &cco);
-        bdrv_coroutine_enter(bs, co);
-        BDRV_POLL_WHILE(bs, cco.ret == -EINPROGRESS);
-    }
-
-    return cco.ret;
-}
-
 /*
  * Return values:
  * 0        - success
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp)
     return 0;
 }
 
-typedef struct InvalidateCacheCo {
-    BlockDriverState *bs;
-    Error **errp;
-    bool done;
-    int ret;
-} InvalidateCacheCo;
-
-static void coroutine_fn bdrv_invalidate_cache_co_entry(void *opaque)
-{
-    InvalidateCacheCo *ico = opaque;
-    ico->ret = bdrv_co_invalidate_cache(ico->bs, ico->errp);
-    ico->done = true;
-    aio_wait_kick();
-}
-
-int bdrv_invalidate_cache(BlockDriverState *bs, Error **errp)
-{
-    Coroutine *co;
-    InvalidateCacheCo ico = {
-        .bs = bs,
-        .done = false,
-        .errp = errp
-    };
-
-    if (qemu_in_coroutine()) {
-        /* Fast-path if already in coroutine context */
-        bdrv_invalidate_cache_co_entry(&ico);
-    } else {
-        co = qemu_coroutine_create(bdrv_invalidate_cache_co_entry, &ico);
-        bdrv_coroutine_enter(bs, co);
-        BDRV_POLL_WHILE(bs, !ico.done);
-    }
-
-    return ico.ret;
-}
-
 void bdrv_invalidate_cache_all(Error **errp)
 {
     BlockDriverState *bs;
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ static int bdrv_check_byte_request(BlockDriverState *bs, int64_t offset,
     return 0;
 }
 
-typedef int coroutine_fn BdrvRequestEntry(void *opaque);
-typedef struct BdrvRunCo {
-    BdrvRequestEntry *entry;
-    void *opaque;
-    int ret;
-    bool done;
-    Coroutine *co; /* Coroutine, running bdrv_run_co_entry, for debugging */
-} BdrvRunCo;
-
-static void coroutine_fn bdrv_run_co_entry(void *opaque)
-{
-    BdrvRunCo *arg = opaque;
-
-    arg->ret = arg->entry(arg->opaque);
-    arg->done = true;
-    aio_wait_kick();
-}
-
-static int bdrv_run_co(BlockDriverState *bs, BdrvRequestEntry *entry,
-                       void *opaque)
-{
-    if (qemu_in_coroutine()) {
-        /* Fast-path if already in coroutine context */
-        return entry(opaque);
-    } else {
-        BdrvRunCo s = { .entry = entry, .opaque = opaque };
-
-        s.co = qemu_coroutine_create(bdrv_run_co_entry, &s);
-        bdrv_coroutine_enter(bs, s.co);
-
-        BDRV_POLL_WHILE(bs, !s.done);
-
-        return s.ret;
-    }
-}
-
-typedef struct RwCo {
-    BdrvChild *child;
-    int64_t offset;
-    QEMUIOVector *qiov;
-    bool is_write;
-    BdrvRequestFlags flags;
-} RwCo;
-
 int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
                               QEMUIOVector *qiov, bool is_write,
                               BdrvRequestFlags flags)
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
     }
 }
 
-static int coroutine_fn bdrv_rw_co_entry(void *opaque)
-{
-    RwCo *rwco = opaque;
-
-    return bdrv_co_prwv(rwco->child, rwco->offset, rwco->qiov,
-                        rwco->is_write, rwco->flags);
-}
-
-/*
- * Process a vectored synchronous request using coroutines
- */
-int bdrv_prwv(BdrvChild *child, int64_t offset,
-              QEMUIOVector *qiov, bool is_write,
-              BdrvRequestFlags flags)
-{
-    RwCo rwco = {
-        .child = child,
-        .offset = offset,
-        .qiov = qiov,
-        .is_write = is_write,
-        .flags = flags,
-    };
-
-    return bdrv_run_co(child->bs, bdrv_rw_co_entry, &rwco);
-}
-
 int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
                        int bytes, BdrvRequestFlags flags)
 {
@@ -XXX,XX +XXX,XX @@ int bdrv_flush_all(void)
     return result;
 }
 
-
-typedef struct BdrvCoBlockStatusData {
-    BlockDriverState *bs;
-    BlockDriverState *base;
-    bool want_zero;
-    int64_t offset;
-    int64_t bytes;
-    int64_t *pnum;
-    int64_t *map;
-    BlockDriverState **file;
-} BdrvCoBlockStatusData;
-
 /*
  * Returns the allocation status of the specified sectors.
  * Drivers not implementing the functionality are assumed to not support
@@ -XXX,XX +XXX,XX @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
     return ret;
 }
 
-/* Coroutine wrapper for bdrv_block_status_above() */
-static int coroutine_fn bdrv_block_status_above_co_entry(void *opaque)
-{
-    BdrvCoBlockStatusData *data = opaque;
-
-    return bdrv_co_common_block_status_above(data->bs, data->base,
-                                             data->want_zero,
-                                             data->offset, data->bytes,
-                                             data->pnum, data->map, data->file);
-}
-
-/*
- * Synchronous wrapper around bdrv_co_block_status_above().
- *
- * See bdrv_co_block_status_above() for details.
- */
-int bdrv_common_block_status_above(BlockDriverState *bs,
-                                   BlockDriverState *base,
-                                   bool want_zero, int64_t offset,
-                                   int64_t bytes, int64_t *pnum,
-                                   int64_t *map,
-                                   BlockDriverState **file)
-{
-    BdrvCoBlockStatusData data = {
-        .bs = bs,
-        .base = base,
-        .want_zero = want_zero,
-        .offset = offset,
-        .bytes = bytes,
-        .pnum = pnum,
-        .map = map,
-        .file = file,
-    };
-
-    return bdrv_run_co(bs, bdrv_block_status_above_co_entry, &data);
-}
-
 int bdrv_block_status_above(BlockDriverState *bs, BlockDriverState *base,
                             int64_t offset, int64_t bytes, int64_t *pnum,
                             int64_t *map, BlockDriverState **file)
@@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top,
     return 0;
 }
 
-typedef struct BdrvVmstateCo {
-    BlockDriverState    *bs;
-    QEMUIOVector        *qiov;
-    int64_t             pos;
-    bool                is_read;
-} BdrvVmstateCo;
-
 int coroutine_fn
 bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
                    bool is_read)
@@ -XXX,XX +XXX,XX @@ bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
     return ret;
 }
 
-static int coroutine_fn bdrv_co_rw_vmstate_entry(void *opaque)
-{
-    BdrvVmstateCo *co = opaque;
-
-    return bdrv_co_rw_vmstate(co->bs, co->qiov, co->pos, co->is_read);
-}
-
-int bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
-                    bool is_read)
-{
-    BdrvVmstateCo data = {
-        .bs         = bs,
-        .qiov       = qiov,
-        .pos        = pos,
-        .is_read    = is_read,
-    };
-
-    return bdrv_run_co(bs, bdrv_co_rw_vmstate_entry, &data);
-}
-
 int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf,
                       int64_t pos, int size)
 {
@@ -XXX,XX +XXX,XX @@ void bdrv_aio_cancel_async(BlockAIOCB *acb)
 /**************************************************************/
 /* Coroutine block device emulation */
 
-static int coroutine_fn bdrv_flush_co_entry(void *opaque)
-{
-    return bdrv_co_flush(opaque);
-}
-
 int coroutine_fn bdrv_co_flush(BlockDriverState *bs)
 {
     BdrvChild *primary_child = bdrv_primary_child(bs);
@@ -XXX,XX +XXX,XX @@ early_exit:
     return ret;
 }
 
-int bdrv_flush(BlockDriverState *bs)
-{
-    return bdrv_run_co(bs, bdrv_flush_co_entry, bs);
-}
-
-typedef struct DiscardCo {
-    BdrvChild *child;
-    int64_t offset;
-    int64_t bytes;
-} DiscardCo;
-
-static int coroutine_fn bdrv_pdiscard_co_entry(void *opaque)
-{
-    DiscardCo *rwco = opaque;
-
-    return bdrv_co_pdiscard(rwco->child, rwco->offset, rwco->bytes);
-}
-
 int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset,
                                   int64_t bytes)
 {
@@ -XXX,XX +XXX,XX @@ out:
     return ret;
 }
 
-int bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes)
-{
-    DiscardCo rwco = {
-        .child = child,
-        .offset = offset,
-        .bytes = bytes,
-    };
-
-    return bdrv_run_co(child->bs, bdrv_pdiscard_co_entry, &rwco);
-}
-
 int bdrv_co_ioctl(BlockDriverState *bs, int req, void *buf)
 {
     BlockDriver *drv = bs->drv;
@@ -XXX,XX +XXX,XX @@ out:
 
     return ret;
 }
-
-typedef struct TruncateCo {
-    BdrvChild *child;
-    int64_t offset;
-    bool exact;
-    PreallocMode prealloc;
-    BdrvRequestFlags flags;
-    Error **errp;
-} TruncateCo;
-
-static int coroutine_fn bdrv_truncate_co_entry(void *opaque)
-{
-    TruncateCo *tco = opaque;
-
-    return bdrv_co_truncate(tco->child, tco->offset, tco->exact,
-                            tco->prealloc, tco->flags, tco->errp);
-}
-
-int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact,
-                  PreallocMode prealloc, BdrvRequestFlags flags, Error **errp)
-{
-    TruncateCo tco = {
-        .child      = child,
-        .offset     = offset,
-        .exact      = exact,
-        .prealloc   = prealloc,
-        .flags      = flags,
-        .errp       = errp,
-    };
-
-    return bdrv_run_co(child->bs, bdrv_truncate_co_entry, &tco);
-}
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Now that we are not maintaining boilerplate code for coroutine
wrappers, there is no more sense in keeping the extra indirection layer
of bdrv_prwv().  Let's drop it and instead generate pure bdrv_preadv()
and bdrv_pwritev().

Currently, bdrv_pwritev() and bdrv_preadv() are returning bytes on
success, auto generated functions will instead return zero, as their
_co_ prototype. Still, it's simple to make the conversion safe: the
only external user of bdrv_pwritev() is test-bdrv-drain, and it is
comfortable enough with bdrv_co_pwritev() instead. So prototypes are
moved to local block/coroutines.h. Next, the only internal use is
bdrv_pread() and bdrv_pwrite(), which are modified to return bytes on
success.

Of course, it would be great to convert bdrv_pread() and bdrv_pwrite()
to return 0 on success. But this requires audit (and probably
conversion) of all their users, let's leave it for another day
refactoring.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200924185414.28642-7-vsementsov@virtuozzo.com>
---
 block/coroutines.h      | 10 ++++-----
 include/block/block.h   |  2 --
 block/io.c              | 49 ++++++++---------------------------------
 tests/test-bdrv-drain.c |  2 +-
 4 files changed, 15 insertions(+), 48 deletions(-)

diff --git a/block/coroutines.h b/block/coroutines.h
index XXXXXXX..XXXXXXX 100644
--- a/block/coroutines.h
+++ b/block/coroutines.h
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_check(BlockDriverState *bs,
                                BdrvCheckResult *res, BdrvCheckMode fix);
 int coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, Error **errp);
 
-int coroutine_fn
-bdrv_co_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
-             bool is_write, BdrvRequestFlags flags);
 int generated_co_wrapper
-bdrv_prwv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov,
-          bool is_write, BdrvRequestFlags flags);
+bdrv_preadv(BdrvChild *child, int64_t offset, unsigned int bytes,
+            QEMUIOVector *qiov, BdrvRequestFlags flags);
+int generated_co_wrapper
+bdrv_pwritev(BdrvChild *child, int64_t offset, unsigned int bytes,
+             QEMUIOVector *qiov, BdrvRequestFlags flags);
 
 int coroutine_fn
 bdrv_co_common_block_status_above(BlockDriverState *bs,
diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
                        int bytes, BdrvRequestFlags flags);
 int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags);
 int bdrv_pread(BdrvChild *child, int64_t offset, void *buf, int bytes);
-int bdrv_preadv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov);
 int bdrv_pwrite(BdrvChild *child, int64_t offset, const void *buf, int bytes);
-int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov);
 int bdrv_pwrite_sync(BdrvChild *child, int64_t offset,
                      const void *buf, int count);
 /*
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ static int bdrv_check_byte_request(BlockDriverState *bs, int64_t offset,
     return 0;
 }
 
-int coroutine_fn bdrv_co_prwv(BdrvChild *child, int64_t offset,
-                              QEMUIOVector *qiov, bool is_write,
-                              BdrvRequestFlags flags)
-{
-    if (is_write) {
-        return bdrv_co_pwritev(child, offset, qiov->size, qiov, flags);
-    } else {
-        return bdrv_co_preadv(child, offset, qiov->size, qiov, flags);
-    }
-}
-
 int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
                        int bytes, BdrvRequestFlags flags)
 {
-    QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, NULL, bytes);
-
-    return bdrv_prwv(child, offset, &qiov, true, BDRV_REQ_ZERO_WRITE | flags);
+    return bdrv_pwritev(child, offset, bytes, NULL,
+                        BDRV_REQ_ZERO_WRITE | flags);
 }
 
 /*
@@ -XXX,XX +XXX,XX @@ int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags)
     }
 }
 
-/* return < 0 if error. See bdrv_pwrite() for the return codes */
-int bdrv_preadv(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
-{
-    int ret;
-
-    ret = bdrv_prwv(child, offset, qiov, false, 0);
-    if (ret < 0) {
-        return ret;
-    }
-
-    return qiov->size;
-}
-
 /* See bdrv_pwrite() for the return codes */
 int bdrv_pread(BdrvChild *child, int64_t offset, void *buf, int bytes)
 {
+    int ret;
     QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
 
     if (bytes < 0) {
         return -EINVAL;
     }
 
-    return bdrv_preadv(child, offset, &qiov);
-}
+    ret = bdrv_preadv(child, offset, bytes, &qiov,  0);
 
-int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
-{
-    int ret;
-
-    ret = bdrv_prwv(child, offset, qiov, true, 0);
-    if (ret < 0) {
-        return ret;
-    }
-
-    return qiov->size;
+    return ret < 0 ? ret : bytes;
 }
 
 /* Return no. of bytes on success or < 0 on error. Important errors are:
@@ -XXX,XX +XXX,XX @@ int bdrv_pwritev(BdrvChild *child, int64_t offset, QEMUIOVector *qiov)
 */
 int bdrv_pwrite(BdrvChild *child, int64_t offset, const void *buf, int bytes)
 {
+    int ret;
     QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, bytes);
 
     if (bytes < 0) {
         return -EINVAL;
     }
 
-    return bdrv_pwritev(child, offset, &qiov);
+    ret = bdrv_pwritev(child, offset, bytes, &qiov, 0);
+
+    return ret < 0 ? ret : bytes;
 }
 
 /*
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index XXXXXXX..XXXXXXX 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_replace_test_co_preadv(BlockDriverState *bs,
         }
         s->io_co = NULL;
 
-        ret = bdrv_preadv(bs->backing, offset, qiov);
+        ret = bdrv_co_preadv(bs->backing, offset, bytes, qiov, 0);
         s->has_read = true;
 
         /* Wake up drain_co if it runs */
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Like for read/write in a previous commit, drop extra indirection layer,
generate directly bdrv_readv_vmstate() and bdrv_writev_vmstate().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200924185414.28642-8-vsementsov@virtuozzo.com>
---
 block/coroutines.h    | 10 +++----
 include/block/block.h |  6 ++--
 block/io.c            | 70 ++++++++++++++++++++++---------------------
 3 files changed, 44 insertions(+), 42 deletions(-)

diff --git a/block/coroutines.h b/block/coroutines.h
index XXXXXXX..XXXXXXX 100644
--- a/block/coroutines.h
+++ b/block/coroutines.h
@@ -XXX,XX +XXX,XX @@ bdrv_common_block_status_above(BlockDriverState *bs,
                                int64_t *map,
                                BlockDriverState **file);
 
-int coroutine_fn
-bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
-                   bool is_read);
-int generated_co_wrapper
-bdrv_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
-                bool is_read);
+int coroutine_fn bdrv_co_readv_vmstate(BlockDriverState *bs,
+                                       QEMUIOVector *qiov, int64_t pos);
+int coroutine_fn bdrv_co_writev_vmstate(BlockDriverState *bs,
+                                        QEMUIOVector *qiov, int64_t pos);
 
 #endif /* BLOCK_COROUTINES_INT_H */
diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ int path_has_protocol(const char *path);
 int path_is_absolute(const char *path);
 char *path_combine(const char *base_path, const char *filename);
 
-int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
-int bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
+int generated_co_wrapper
+bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
+int generated_co_wrapper
+bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
 int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf,
                       int64_t pos, int size);
 
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ int bdrv_is_allocated_above(BlockDriverState *top,
 }
 
 int coroutine_fn
-bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,
-                   bool is_read)
+bdrv_co_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
 {
     BlockDriver *drv = bs->drv;
     BlockDriverState *child_bs = bdrv_primary_bs(bs);
     int ret = -ENOTSUP;
 
+    if (!drv) {
+        return -ENOMEDIUM;
+    }
+
     bdrv_inc_in_flight(bs);
 
+    if (drv->bdrv_load_vmstate) {
+        ret = drv->bdrv_load_vmstate(bs, qiov, pos);
+    } else if (child_bs) {
+        ret = bdrv_co_readv_vmstate(child_bs, qiov, pos);
+    }
+
+    bdrv_dec_in_flight(bs);
+
+    return ret;
+}
+
+int coroutine_fn
+bdrv_co_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
+{
+    BlockDriver *drv = bs->drv;
+    BlockDriverState *child_bs = bdrv_primary_bs(bs);
+    int ret = -ENOTSUP;
+
     if (!drv) {
-        ret = -ENOMEDIUM;
-    } else if (drv->bdrv_load_vmstate) {
-        if (is_read) {
-            ret = drv->bdrv_load_vmstate(bs, qiov, pos);
-        } else {
-            ret = drv->bdrv_save_vmstate(bs, qiov, pos);
-        }
+        return -ENOMEDIUM;
+    }
+
+    bdrv_inc_in_flight(bs);
+
+    if (drv->bdrv_save_vmstate) {
+        ret = drv->bdrv_save_vmstate(bs, qiov, pos);
     } else if (child_bs) {
-        ret = bdrv_co_rw_vmstate(child_bs, qiov, pos, is_read);
+        ret = bdrv_co_writev_vmstate(child_bs, qiov, pos);
     }
 
     bdrv_dec_in_flight(bs);
+
     return ret;
 }
 
@@ -XXX,XX +XXX,XX @@ int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf,
                       int64_t pos, int size)
 {
     QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, size);
-    int ret;
+    int ret = bdrv_writev_vmstate(bs, &qiov, pos);
 
-    ret = bdrv_writev_vmstate(bs, &qiov, pos);
-    if (ret < 0) {
-        return ret;
-    }
-
-    return size;
-}
-
-int bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
-{
-    return bdrv_rw_vmstate(bs, qiov, pos, false);
+    return ret < 0 ? ret : size;
 }
 
 int bdrv_load_vmstate(BlockDriverState *bs, uint8_t *buf,
                       int64_t pos, int size)
 {
     QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, buf, size);
-    int ret;
+    int ret = bdrv_readv_vmstate(bs, &qiov, pos);
 
-    ret = bdrv_readv_vmstate(bs, &qiov, pos);
-    if (ret < 0) {
-        return ret;
-    }
-
-    return size;
-}
-
-int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)
-{
-    return bdrv_rw_vmstate(bs, qiov, pos, true);
+    return ret < 0 ? ret : size;
 }
 
 /**************************************************************/
-- 
2.26.2

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

This is the only non-ascii character in the file and it doesn't really
needed here. Let's use normal "'" symbol for consistency with the rest
11 occurrences of "'" in the file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/block/block.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ enum BdrvChildRoleBits {
     BDRV_CHILD_FILTERED     = (1 << 2),
 
     /*
-     * Child from which to read all data that isn’t allocated in the
+     * Child from which to read all data that isn't allocated in the
      * parent (i.e., the backing child); such data is copied to the
      * parent through COW (and optionally COR).
      * This field is mutually exclusive with DATA, METADATA, and
-- 
2.26.2

From: Stefano Garzarella <sgarzare@redhat.com>

When we added io_uring AIO engine, we forgot to update qemu-options.hx,
so qemu(1) man page and qemu help were outdated.

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Julia Suvorova <jusual@redhat.com>
Reviewed-by: Pankaj Gupta <pankaj.gupta.linux@gmail.com>
Message-Id: <20200924151511.131471-1-sgarzare@redhat.com>
---
 qemu-options.hx | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/qemu-options.hx b/qemu-options.hx
index XXXXXXX..XXXXXXX 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -XXX,XX +XXX,XX @@ SRST
             The path to the image file in the local filesystem
 
         ``aio``
-            Specifies the AIO backend (threads/native, default: threads)
+            Specifies the AIO backend (threads/native/io_uring,
+            default: threads)
 
         ``locking``
             Specifies whether the image file is protected with Linux OFD
@@ -XXX,XX +XXX,XX @@ DEF("drive", HAS_ARG, QEMU_OPTION_drive,
     "-drive [file=file][,if=type][,bus=n][,unit=m][,media=d][,index=i]\n"
     "       [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n"
     "       [,snapshot=on|off][,rerror=ignore|stop|report]\n"
-    "       [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n"
+    "       [,werror=ignore|stop|report|enospc][,id=name]\n"
+    "       [,aio=threads|native|io_uring]\n"
     "       [,readonly=on|off][,copy-on-read=on|off]\n"
     "       [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n"
     "       [[,bps=b]|[[,bps_rd=r][,bps_wr=w]]]\n"
@@ -XXX,XX +XXX,XX @@ SRST
         The default mode is ``cache=writeback``.
 
     ``aio=aio``
-        aio is "threads", or "native" and selects between pthread based
-        disk I/O and native Linux AIO.
+        aio is "threads", "native", or "io_uring" and selects between pthread
+        based disk I/O, native Linux AIO, or Linux io_uring API.
 
     ``format=format``
         Specify which disk format will be used rather than detecting the
-- 
2.26.2

From: Eric Auger <eric.auger@redhat.com>

The IOVA allocator currently ignores host reserved regions.
As a result some chosen IOVAs may collide with some of them,
resulting in VFIO MAP_DMA errors later on. This happens on ARM
where the MSI reserved window quickly is encountered:
[0x8000000, 0x8100000]. since 5.4 kernel, VFIO returns the usable
IOVA regions. So let's enumerate them in the prospect to avoid
them, later on.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Message-id: 20200929085550.30926-2-eric.auger@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/vfio-helpers.c | 72 +++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 70 insertions(+), 2 deletions(-)

diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c
index XXXXXXX..XXXXXXX 100644
--- a/util/vfio-helpers.c
+++ b/util/vfio-helpers.c
@@ -XXX,XX +XXX,XX @@ typedef struct {
     uint64_t iova;
 } IOVAMapping;
 
+struct IOVARange {
+    uint64_t start;
+    uint64_t end;
+};
+
 struct QEMUVFIOState {
     QemuMutex lock;
 
@@ -XXX,XX +XXX,XX @@ struct QEMUVFIOState {
     int device;
     RAMBlockNotifier ram_notifier;
     struct vfio_region_info config_region_info, bar_region_info[6];
+    struct IOVARange *usable_iova_ranges;
+    uint8_t nb_iova_ranges;
 
     /* These fields are protected by @lock */
     /* VFIO's IO virtual address space is managed by splitting into a few
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_pci_write_config(QEMUVFIOState *s, void *buf, int size, int
     return ret == size ? 0 : -errno;
 }
 
+static void collect_usable_iova_ranges(QEMUVFIOState *s, void *buf)
+{
+    struct vfio_iommu_type1_info *info = (struct vfio_iommu_type1_info *)buf;
+    struct vfio_info_cap_header *cap = (void *)buf + info->cap_offset;
+    struct vfio_iommu_type1_info_cap_iova_range *cap_iova_range;
+    int i;
+
+    while (cap->id != VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE) {
+        if (!cap->next) {
+            return;
+        }
+        cap = (struct vfio_info_cap_header *)(buf + cap->next);
+    }
+
+    cap_iova_range = (struct vfio_iommu_type1_info_cap_iova_range *)cap;
+
+    s->nb_iova_ranges = cap_iova_range->nr_iovas;
+    if (s->nb_iova_ranges > 1) {
+        s->usable_iova_ranges =
+            g_realloc(s->usable_iova_ranges,
+                      s->nb_iova_ranges * sizeof(struct IOVARange));
+    }
+
+    for (i = 0; i < s->nb_iova_ranges; i++) {
+        s->usable_iova_ranges[i].start = cap_iova_range->iova_ranges[i].start;
+        s->usable_iova_ranges[i].end = cap_iova_range->iova_ranges[i].end;
+    }
+}
+
 static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
                               Error **errp)
 {
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
     int i;
     uint16_t pci_cmd;
     struct vfio_group_status group_status = { .argsz = sizeof(group_status) };
-    struct vfio_iommu_type1_info iommu_info = { .argsz = sizeof(iommu_info) };
+    struct vfio_iommu_type1_info *iommu_info = NULL;
+    size_t iommu_info_size = sizeof(*iommu_info);
     struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
     char *group_file = NULL;
 
+    s->usable_iova_ranges = NULL;
+
     /* Create a new container */
     s->container = open("/dev/vfio/vfio", O_RDWR);
 
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
         goto fail;
     }
 
+    iommu_info = g_malloc0(iommu_info_size);
+    iommu_info->argsz = iommu_info_size;
+
     /* Get additional IOMMU info */
-    if (ioctl(s->container, VFIO_IOMMU_GET_INFO, &iommu_info)) {
+    if (ioctl(s->container, VFIO_IOMMU_GET_INFO, iommu_info)) {
         error_setg_errno(errp, errno, "Failed to get IOMMU info");
         ret = -errno;
         goto fail;
     }
 
+    /*
+     * if the kernel does not report usable IOVA regions, choose
+     * the legacy [QEMU_VFIO_IOVA_MIN, QEMU_VFIO_IOVA_MAX -1] region
+     */
+    s->nb_iova_ranges = 1;
+    s->usable_iova_ranges = g_new0(struct IOVARange, 1);
+    s->usable_iova_ranges[0].start = QEMU_VFIO_IOVA_MIN;
+    s->usable_iova_ranges[0].end = QEMU_VFIO_IOVA_MAX - 1;
+
+    if (iommu_info->argsz > iommu_info_size) {
+        iommu_info_size = iommu_info->argsz;
+        iommu_info = g_realloc(iommu_info, iommu_info_size);
+        if (ioctl(s->container, VFIO_IOMMU_GET_INFO, iommu_info)) {
+            ret = -errno;
+            goto fail;
+        }
+        collect_usable_iova_ranges(s, iommu_info);
+    }
+
     s->device = ioctl(s->group, VFIO_GROUP_GET_DEVICE_FD, device);
 
     if (s->device < 0) {
@@ -XXX,XX +XXX,XX @@ static int qemu_vfio_init_pci(QEMUVFIOState *s, const char *device,
     if (ret) {
         goto fail;
     }
+    g_free(iommu_info);
     return 0;
 fail:
+    g_free(s->usable_iova_ranges);
+    s->usable_iova_ranges = NULL;
+    s->nb_iova_ranges = 0;
+    g_free(iommu_info);
     close(s->group);
 fail_container:
     close(s->container);
@@ -XXX,XX +XXX,XX @@ void qemu_vfio_close(QEMUVFIOState *s)
         qemu_vfio_undo_mapping(s, &s->mappings[i], NULL);
     }
     ram_block_notifier_remove(&s->ram_notifier);
+    g_free(s->usable_iova_ranges);
+    s->nb_iova_ranges = 0;
     qemu_vfio_reset(s);
     close(s->device);
     close(s->group);
-- 
2.26.2

From: Eric Auger <eric.auger@redhat.com>

Introduce the qemu_vfio_find_fixed/temp_iova helpers which
respectively allocate IOVAs from the bottom/top parts of the
usable IOVA range, without picking within host IOVA reserved
windows. The allocation remains basic: if the size is too big
for the remaining of the current usable IOVA range, we jump
to the next one, leaving a hole in the address map.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Message-id: 20200929085550.30926-3-eric.auger@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 util/vfio-helpers.c | 57 +++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 53 insertions(+), 4 deletions(-)

diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c
index XXXXXXX..XXXXXXX 100644
--- a/util/vfio-helpers.c
+++ b/util/vfio-helpers.c
@@ -XXX,XX +XXX,XX @@ static bool qemu_vfio_verify_mappings(QEMUVFIOState *s)
     return true;
 }
 
+static int
+qemu_vfio_find_fixed_iova(QEMUVFIOState *s, size_t size, uint64_t *iova)
+{
+    int i;
+
+    for (i = 0; i < s->nb_iova_ranges; i++) {
+        if (s->usable_iova_ranges[i].end < s->low_water_mark) {
+            continue;
+        }
+        s->low_water_mark =
+            MAX(s->low_water_mark, s->usable_iova_ranges[i].start);
+
+        if (s->usable_iova_ranges[i].end - s->low_water_mark + 1 >= size ||
+            s->usable_iova_ranges[i].end - s->low_water_mark + 1 == 0) {
+            *iova = s->low_water_mark;
+            s->low_water_mark += size;
+            return 0;
+        }
+    }
+    return -ENOMEM;
+}
+
+static int
+qemu_vfio_find_temp_iova(QEMUVFIOState *s, size_t size, uint64_t *iova)
+{
+    int i;
+
+    for (i = s->nb_iova_ranges - 1; i >= 0; i--) {
+        if (s->usable_iova_ranges[i].start > s->high_water_mark) {
+            continue;
+        }
+        s->high_water_mark =
+            MIN(s->high_water_mark, s->usable_iova_ranges[i].end + 1);
+
+        if (s->high_water_mark - s->usable_iova_ranges[i].start + 1 >= size ||
+            s->high_water_mark - s->usable_iova_ranges[i].start + 1 == 0) {
+            *iova = s->high_water_mark - size;
+            s->high_water_mark = *iova;
+            return 0;
+        }
+    }
+    return -ENOMEM;
+}
+
 /* Map [host, host + size) area into a contiguous IOVA address space, and store
  * the result in @iova if not NULL. The caller need to make sure the area is
  * aligned to page size, and mustn't overlap with existing mapping areas (split
@@ -XXX,XX +XXX,XX @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size,
             goto out;
         }
         if (!temporary) {
-            iova0 = s->low_water_mark;
+            if (qemu_vfio_find_fixed_iova(s, size, &iova0)) {
+                ret = -ENOMEM;
+                goto out;
+            }
+
             mapping = qemu_vfio_add_mapping(s, host, size, index + 1, iova0);
             if (!mapping) {
                 ret = -ENOMEM;
@@ -XXX,XX +XXX,XX @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size,
                 qemu_vfio_undo_mapping(s, mapping, NULL);
                 goto out;
             }
-            s->low_water_mark += size;
             qemu_vfio_dump_mappings(s);
         } else {
-            iova0 = s->high_water_mark - size;
+            if (qemu_vfio_find_temp_iova(s, size, &iova0)) {
+                ret = -ENOMEM;
+                goto out;
+            }
             ret = qemu_vfio_do_mapping(s, host, size, iova0);
             if (ret) {
                 goto out;
             }
-            s->high_water_mark -= size;
         }
     }
     if (iova) {
-- 
2.26.2

The following changes since commit 9d662a6b22a0838a85c5432385f35db2488a33a5:

Merge remote-tracking branch 'remotes/legoater/tags/pull-ppc-20220305' into staging (2022-03-05 18:03:15 +0000)

are available in the Git repository at:

https://gitlab.com/hreitz/qemu.git tags/pull-block-2022-03-07

for you to fetch changes up to 743da0b401cdc3ee94bc519975e339a3cdbe0ad1:

iotests/image-fleecing: test push backup with fleecing (2022-03-07 09:33:31 +0100)

----------------------------------------------------------------
Block patches for 7.0-rc0:
- New fleecing backup scheme
- iotest fixes
- Fixes for the curl block driver
- Fix for the preallocate block driver
- IDE fix for zero-length TRIM requests

----------------------------------------------------------------
Hanna Reitz (2):
  ide: Increment BB in-flight counter for TRIM BH
  iotests: Write test output to TEST_DIR

Peter Maydell (2):
  block/curl.c: Set error message string if curl_init_state() fails
  block/curl.c: Check error return from curl_easy_setopt()

Thomas Huth (2):
  tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'
  tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP mode

Vladimir Sementsov-Ogievskiy (17):
  block: fix preallocate filter: don't do unaligned preallocate requests
  block/block-copy: move copy_bitmap initialization to
    block_copy_state_new()
  block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
  block/block-copy: block_copy_state_new(): add bitmap parameter
  block/copy-before-write: add bitmap open parameter
  block/block-copy: add block_copy_reset()
  block: intoduce reqlist
  block/reqlist: reqlist_find_conflict(): use ranges_overlap()
  block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
  block/reqlist: add reqlist_wait_all()
  block/io: introduce block driver snapshot-access API
  block: introduce snapshot-access block driver
  block: copy-before-write: realize snapshot-access API
  iotests/image-fleecing: add test-case for fleecing format node
  iotests.py: add qemu_io_pipe_and_status()
  iotests/image-fleecing: add test case with bitmap
  iotests/image-fleecing: test push backup with fleecing

-- 
2.34.1

When we still have an AIOCB registered for DMA operations, we try to
settle the respective operation by draining the BlockBackend associated
with the IDE device.

However, this assumes that every DMA operation is associated with an
increment of the BlockBackend’s in-flight counter (e.g. through some
ongoing I/O operation), so that draining the BB until its in-flight
counter reaches 0 will settle all DMA operations.  That is not the case:
For TRIM, the guest can issue a zero-length operation that will not
result in any I/O operation forwarded to the BlockBackend, and also not
increment the in-flight counter in any other way.  In such a case,
blk_drain() will be a no-op if no other operations are in flight.

It is clear that if blk_drain() is a no-op, the value of
s->bus->dma->aiocb will not change between checking it in the `if`
condition and asserting that it is NULL after blk_drain().

The particular problem is that ide_issue_trim() creates a BH
(ide_trim_bh_cb()) to settle the TRIM request: iocb->common.cb() is
ide_dma_cb(), which will either create a new request, or find the
transfer to be done and call ide_set_inactive(), which clears
s->bus->dma->aiocb.  Therefore, the blk_drain() must wait for
ide_trim_bh_cb() to run, which currently it will not always do.

To fix this issue, we increment the BlockBackend's in-flight counter
when the TRIM operation begins (in ide_issue_trim(), when the
ide_trim_bh_cb() BH is created) and decrement it when ide_trim_bh_cb()
is done.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2029980
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220120142259.120189-1-hreitz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Tested-by: John Snow <jsnow@redhat.com>
---
 hw/ide/core.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/hw/ide/core.c b/hw/ide/core.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -XXX,XX +XXX,XX @@ static const AIOCBInfo trim_aiocb_info = {
 static void ide_trim_bh_cb(void *opaque)
 {
     TrimAIOCB *iocb = opaque;
+    BlockBackend *blk = iocb->s->blk;
 
     iocb->common.cb(iocb->common.opaque, iocb->ret);
 
     qemu_bh_delete(iocb->bh);
     iocb->bh = NULL;
     qemu_aio_unref(iocb);
+
+    /* Paired with an increment in ide_issue_trim() */
+    blk_dec_in_flight(blk);
 }
 
 static void ide_issue_trim_cb(void *opaque, int ret)
@@ -XXX,XX +XXX,XX @@ BlockAIOCB *ide_issue_trim(
     IDEState *s = opaque;
     TrimAIOCB *iocb;
 
+    /* Paired with a decrement in ide_trim_bh_cb() */
+    blk_inc_in_flight(s->blk);
+
     iocb = blk_aio_get(&trim_aiocb_info, s->blk, cb, cb_opaque);
     iocb->s = s;
     iocb->bh = qemu_bh_new(ide_trim_bh_cb, iocb);
-- 
2.34.1

From: Peter Maydell <peter.maydell@linaro.org>

In curl_open(), the 'out' label assumes that the state->errmsg string
has been set (either by curl_easy_perform() or by manually copying a
string into it); however if curl_init_state() fails we will jump to
that label without setting the string.  Add the missing error string
setup.

(We can't be specific about the cause of failure: the documentation
of curl_easy_init() just says "If this function returns NULL,
something went wrong".)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20220222152341.850419-2-peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 block/curl.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
     // Get file size
 
     if (curl_init_state(s, state) < 0) {
+        pstrcpy(state->errmsg, CURL_ERROR_SIZE,
+                "curl library initialization failed.");
         goto out;
     }
 
-- 
2.34.1

From: Peter Maydell <peter.maydell@linaro.org>

Coverity points out that we aren't checking the return value
from curl_easy_setopt() for any of the calls to it we make
in block/curl.c.

Some of these options are documented as always succeeding (e.g.
CURLOPT_VERBOSE) but others have documented failure cases (e.g.
CURLOPT_URL).  For consistency we check every call, even the ones
that theoretically cannot fail.

Fixes: Coverity CID 1459336, 1459482, 1460331
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20220222152341.850419-3-peter.maydell@linaro.org>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 block/curl.c | 90 +++++++++++++++++++++++++++++++++-------------------
 1 file changed, 57 insertions(+), 33 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static int curl_init_state(BDRVCURLState *s, CURLState *state)
         if (!state->curl) {
             return -EIO;
         }
-        curl_easy_setopt(state->curl, CURLOPT_URL, s->url);
-        curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
-                         (long) s->sslverify);
-        curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYHOST,
-                         s->sslverify ? 2L : 0L);
+        if (curl_easy_setopt(state->curl, CURLOPT_URL, s->url) ||
+            curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
+                             (long) s->sslverify) ||
+            curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYHOST,
+                             s->sslverify ? 2L : 0L)) {
+            goto err;
+        }
         if (s->cookie) {
-            curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie);
+            if (curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie)) {
+                goto err;
+            }
+        }
+        if (curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, (long)s->timeout) ||
+            curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
+                             (void *)curl_read_cb) ||
+            curl_easy_setopt(state->curl, CURLOPT_WRITEDATA, (void *)state) ||
+            curl_easy_setopt(state->curl, CURLOPT_PRIVATE, (void *)state) ||
+            curl_easy_setopt(state->curl, CURLOPT_AUTOREFERER, 1) ||
+            curl_easy_setopt(state->curl, CURLOPT_FOLLOWLOCATION, 1) ||
+            curl_easy_setopt(state->curl, CURLOPT_NOSIGNAL, 1) ||
+            curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg) ||
+            curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1)) {
+            goto err;
         }
-        curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, (long)s->timeout);
-        curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
-                         (void *)curl_read_cb);
-        curl_easy_setopt(state->curl, CURLOPT_WRITEDATA, (void *)state);
-        curl_easy_setopt(state->curl, CURLOPT_PRIVATE, (void *)state);
-        curl_easy_setopt(state->curl, CURLOPT_AUTOREFERER, 1);
-        curl_easy_setopt(state->curl, CURLOPT_FOLLOWLOCATION, 1);
-        curl_easy_setopt(state->curl, CURLOPT_NOSIGNAL, 1);
-        curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg);
-        curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1);
-
         if (s->username) {
-            curl_easy_setopt(state->curl, CURLOPT_USERNAME, s->username);
+            if (curl_easy_setopt(state->curl, CURLOPT_USERNAME, s->username)) {
+                goto err;
+            }
         }
         if (s->password) {
-            curl_easy_setopt(state->curl, CURLOPT_PASSWORD, s->password);
+            if (curl_easy_setopt(state->curl, CURLOPT_PASSWORD, s->password)) {
+                goto err;
+            }
         }
         if (s->proxyusername) {
-            curl_easy_setopt(state->curl,
-                             CURLOPT_PROXYUSERNAME, s->proxyusername);
+            if (curl_easy_setopt(state->curl,
+                                 CURLOPT_PROXYUSERNAME, s->proxyusername)) {
+                goto err;
+            }
         }
         if (s->proxypassword) {
-            curl_easy_setopt(state->curl,
-                             CURLOPT_PROXYPASSWORD, s->proxypassword);
+            if (curl_easy_setopt(state->curl,
+                                 CURLOPT_PROXYPASSWORD, s->proxypassword)) {
+                goto err;
+            }
         }
 
         /* Restrict supported protocols to avoid security issues in the more
@@ -XXX,XX +XXX,XX @@ static int curl_init_state(BDRVCURLState *s, CURLState *state)
          * Restricting protocols is only supported from 7.19.4 upwards.
          */
 #if LIBCURL_VERSION_NUM >= 0x071304
-        curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, PROTOCOLS);
-        curl_easy_setopt(state->curl, CURLOPT_REDIR_PROTOCOLS, PROTOCOLS);
+        if (curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, PROTOCOLS) ||
+            curl_easy_setopt(state->curl, CURLOPT_REDIR_PROTOCOLS, PROTOCOLS)) {
+            goto err;
+        }
 #endif
 
 #ifdef DEBUG_VERBOSE
-        curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1);
+        if (curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1)) {
+            goto err;
+        }
 #endif
     }
 
     state->s = s;
 
     return 0;
+
+err:
+    curl_easy_cleanup(state->curl);
+    state->curl = NULL;
+    return -EIO;
 }
 
 /* Called with s->mutex held.  */
@@ -XXX,XX +XXX,XX @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
     }
 
     s->accept_range = false;
-    curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1);
-    curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION,
-                     curl_header_cb);
-    curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s);
+    if (curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1) ||
+        curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION, curl_header_cb) ||
+        curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s)) {
+        pstrcpy(state->errmsg, CURL_ERROR_SIZE,
+                "curl library initialization failed.");
+        goto out;
+    }
     if (curl_easy_perform(state->curl))
         goto out;
     if (curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &d)) {
@@ -XXX,XX +XXX,XX @@ static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb)
 
     snprintf(state->range, 127, "%" PRIu64 "-%" PRIu64, start, end);
     trace_curl_setup_preadv(acb->bytes, start, state->range);
-    curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
-
-    if (curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
+    if (curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range) ||
+        curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
         state->acb[0] = NULL;
         acb->ret = -EIO;
 
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

There is a bug in handling BDRV_REQ_NO_WAIT flag: we still may wait in
wait_serialising_requests() if request is unaligned. And this is
possible for the only user of this flag (preallocate filter) if
underlying file is unaligned to its request_alignment on start.

So, we have to fix preallocate filter to do only aligned preallocate
requests.

Next, we should fix generic block/io.c somehow. Keeping in mind that
preallocate is the only user of BDRV_REQ_NO_WAIT and that we have to
fix its behavior now, it seems more safe to just assert that we never
use BDRV_REQ_NO_WAIT with unaligned requests and add corresponding
comment. Let's do so.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Message-Id: <20220215121609.38570-1-vsementsov@virtuozzo.com>
[hreitz: Rebased on block GS/IO split]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/block-common.h |  3 ++-
 block/io.c                   |  4 ++++
 block/preallocate.c          | 15 ++++++++++++---
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/include/block/block-common.h b/include/block/block-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block-common.h
+++ b/include/block/block-common.h
@@ -XXX,XX +XXX,XX @@ typedef enum {
 
     /*
      * If we need to wait for other requests, just fail immediately. Used
-     * only together with BDRV_REQ_SERIALISING.
+     * only together with BDRV_REQ_SERIALISING. Used only with requests aligned
+     * to request_alignment (corresponding assertions are in block/io.c).
      */
     BDRV_REQ_NO_WAIT = 0x400,
 
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_do_zero_pwritev(BdrvChild *child,
 
     padding = bdrv_init_padding(bs, offset, bytes, &pad);
     if (padding) {
+        assert(!(flags & BDRV_REQ_NO_WAIT));
         bdrv_make_request_serialising(req, align);
 
         bdrv_padding_rmw_read(child, req, &pad, true);
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child,
          * serialize the request to prevent interactions of the
          * widened region with other transactions.
          */
+        assert(!(flags & BDRV_REQ_NO_WAIT));
         bdrv_make_request_serialising(&req, align);
         bdrv_padding_rmw_read(child, &req, &pad, false);
     }
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_co_copy_range_internal(
     /* TODO We can support BDRV_REQ_NO_FALLBACK here */
     assert(!(read_flags & BDRV_REQ_NO_FALLBACK));
     assert(!(write_flags & BDRV_REQ_NO_FALLBACK));
+    assert(!(read_flags & BDRV_REQ_NO_WAIT));
+    assert(!(write_flags & BDRV_REQ_NO_WAIT));
 
     if (!dst || !dst->bs || !bdrv_is_inserted(dst->bs)) {
         return -ENOMEDIUM;
diff --git a/block/preallocate.c b/block/preallocate.c
index XXXXXXX..XXXXXXX 100644
--- a/block/preallocate.c
+++ b/block/preallocate.c
@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn handle_write(BlockDriverState *bs, int64_t offset,
     int64_t end = offset + bytes;
     int64_t prealloc_start, prealloc_end;
     int ret;
+    uint32_t file_align = bs->file->bs->bl.request_alignment;
+    uint32_t prealloc_align = MAX(s->opts.prealloc_align, file_align);
+
+    assert(QEMU_IS_ALIGNED(prealloc_align, file_align));
 
     if (!has_prealloc_perms(bs)) {
         /* We don't have state neither should try to recover it */
@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn handle_write(BlockDriverState *bs, int64_t offset,
 
     /* Now we want new preallocation, as request writes beyond s->file_end. */
 
-    prealloc_start = want_merge_zero ? MIN(offset, s->file_end) : s->file_end;
-    prealloc_end = QEMU_ALIGN_UP(end + s->opts.prealloc_size,
-                                 s->opts.prealloc_align);
+    prealloc_start = QEMU_ALIGN_UP(
+            want_merge_zero ? MIN(offset, s->file_end) : s->file_end,
+            file_align);
+    prealloc_end = QEMU_ALIGN_UP(
+            MAX(prealloc_start, end) + s->opts.prealloc_size,
+            prealloc_align);
+
+    want_merge_zero = want_merge_zero && (prealloc_start <= offset);
 
     ret = bdrv_co_pwrite_zeroes(
             bs->file, prealloc_start, prealloc_end - prealloc_start,
-- 
2.34.1

From: Thomas Huth <thuth@redhat.com>

iotest 040 already has some checks for the availability of the 'throttle'
driver, but some new code has been added in the course of time that
depends on 'throttle' but does not check for its availability. Add
a check to the TestCommitWithFilters class so that this iotest now
also passes again if 'throttle' has not been enabled in the QEMU
binaries.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220223123127.3206042-1-thuth@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/040 | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/040
+++ b/tests/qemu-iotests/040
@@ -XXX,XX +XXX,XX @@ class TestCommitWithFilters(iotests.QMPTestCase):
                              pattern_file)
             self.assertFalse('Pattern verification failed' in result)
 
+    @iotests.skip_if_unsupported(['throttle'])
     def setUp(self):
         qemu_img('create', '-f', iotests.imgfmt, self.img0, '64M')
         qemu_img('create', '-f', iotests.imgfmt, self.img1, '64M')
-- 
2.34.1

From: Thomas Huth <thuth@redhat.com>

In TAP mode, the stdout is reserved for the TAP protocol, so we
have to make sure to mark other lines with a comment '#' character
at the beginning to avoid that the TAP parser at the other end
gets confused.

To test this condition, run "configure" for example with:

--block-drv-rw-whitelist=copy-before-write,qcow2,raw,file,host_device,blkdebug,null-co,copy-on-read

so that iotest 041 will report that some tests are not run due to
the missing "quorum" driver. Without this change, "make check-block"
fails since the meson tap parser gets confused by these messages.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220223124353.3273898-1-thuth@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/testrunner.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -XXX,XX +XXX,XX @@ def run_test(self, test: str,
                                  description=res.description)
 
         if res.casenotrun:
-            print(res.casenotrun)
+            if self.tap:
+                print('#' + res.casenotrun.replace('\n', '\n#'))
+            else:
+                print(res.casenotrun)
 
         return res
 
-- 
2.34.1

Drop the use of OUTPUT_DIR (test/qemu-iotests under the build
directory), and instead write test output files (.out.bad, .notrun, and
.casenotrun) to TEST_DIR.

With this, the same test can be run concurrently without the separate
instances interfering, because they will need separate TEST_DIRs anyway.
Running the same test separately is useful when running the iotests with
various format/protocol combinations in parallel, or when you just want
to aggressively exercise a single test (e.g. when it fails only
sporadically).

Putting this output into TEST_DIR means that it will stick around for
inspection after the test run is done (though running the same test in
the same TEST_DIR will overwrite it, just as it used to be); but given
that TEST_DIR is a scratch directory, it should be clear that users can
delete all of its content at any point.  (And if TEST_DIR is on tmpfs,
it will just disappear on shutdown.)  Contrarily, alternative approaches
that would put these output files into OUTPUT_DIR with some prefix to
differentiate between separate test runs might easily lead to cluttering
OUTPUT_DIR.

(This change means OUTPUT_DIR is no longer written to by the iotests, so
we can drop its usage altogether.)

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220221172909.762858-1-hreitz@redhat.com>
[hreitz: Simplified `Path(os.path.join(x, y))` to `Path(x, y)`, as
         suggested by Vladimir; and rebased on 9086c7639822b6
         ("tests/qemu-iotests: Rework the checks and spots using GNU
         sed")]
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/common.rc     |  6 +++---
 tests/qemu-iotests/iotests.py    |  5 ++---
 tests/qemu-iotests/testenv.py    |  5 +----
 tests/qemu-iotests/testrunner.py | 14 ++++++++------
 4 files changed, 14 insertions(+), 16 deletions(-)

diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -XXX,XX +XXX,XX @@
 # bail out, setting up .notrun file
 _notrun()
 {
-    echo "$*" >"$OUTPUT_DIR/$seq.notrun"
+    echo "$*" >"$TEST_DIR/$seq.notrun"
     echo "$seq not run: $*"
     status=0
     exit
@@ -XXX,XX +XXX,XX @@ _img_info()
 #
 _casenotrun()
 {
-    echo "    [case not run] $*" >>"$OUTPUT_DIR/$seq.casenotrun"
+    echo "    [case not run] $*" >>"$TEST_DIR/$seq.casenotrun"
 }
 
 # just plain bail out
 #
 _fail()
 {
-    echo "$*" | tee -a "$OUTPUT_DIR/$seq.full"
+    echo "$*" | tee -a "$TEST_DIR/$seq.full"
     echo "(see $seq.full for details)"
     status=1
     exit 1
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -XXX,XX +XXX,XX @@
 
 imgfmt = os.environ.get('IMGFMT', 'raw')
 imgproto = os.environ.get('IMGPROTO', 'file')
-output_dir = os.environ.get('OUTPUT_DIR', '.')
 
 try:
     test_dir = os.environ['TEST_DIR']
@@ -XXX,XX +XXX,XX @@ def notrun(reason):
     # Each test in qemu-iotests has a number ("seq")
     seq = os.path.basename(sys.argv[0])
 
-    with open('%s/%s.notrun' % (output_dir, seq), 'w', encoding='utf-8') \
+    with open('%s/%s.notrun' % (test_dir, seq), 'w', encoding='utf-8') \
             as outfile:
         outfile.write(reason + '\n')
     logger.warning("%s not run: %s", seq, reason)
@@ -XXX,XX +XXX,XX @@ def case_notrun(reason):
     # Each test in qemu-iotests has a number ("seq")
     seq = os.path.basename(sys.argv[0])
 
-    with open('%s/%s.casenotrun' % (output_dir, seq), 'a', encoding='utf-8') \
+    with open('%s/%s.casenotrun' % (test_dir, seq), 'a', encoding='utf-8') \
             as outfile:
         outfile.write('    [case not run] ' + reason + '\n')
 
diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/testenv.py
+++ b/tests/qemu-iotests/testenv.py
@@ -XXX,XX +XXX,XX @@ class TestEnv(ContextManager['TestEnv']):
     # pylint: disable=too-many-instance-attributes
 
     env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR',
-                     'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
+                     'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG',
                      'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG',
                      'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS',
                      'QEMU_IO_OPTIONS', 'QEMU_IO_OPTIONS_NO_FMT',
@@ -XXX,XX +XXX,XX @@ def init_directories(self) -> None:
              TEST_DIR
              SOCK_DIR
              SAMPLE_IMG_DIR
-             OUTPUT_DIR
         """
 
         # Path where qemu goodies live in this source tree.
@@ -XXX,XX +XXX,XX @@ def init_directories(self) -> None:
                                         os.path.join(self.source_iotests,
                                                      'sample_images'))
 
-        self.output_dir = os.getcwd()  # OUTPUT_DIR
-
     def init_binaries(self) -> None:
         """Init binary path variables:
              PYTHON (for bash tests)
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/testrunner.py
+++ b/tests/qemu-iotests/testrunner.py
@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
         """
 
         f_test = Path(test)
-        f_bad = Path(f_test.name + '.out.bad')
-        f_notrun = Path(f_test.name + '.notrun')
-        f_casenotrun = Path(f_test.name + '.casenotrun')
         f_reference = Path(self.find_reference(test))
 
         if not f_test.exists():
@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
                               description='No qualified output '
                                           f'(expected {f_reference})')
 
-        for p in (f_bad, f_notrun, f_casenotrun):
-            silent_unlink(p)
-
         args = [str(f_test.resolve())]
         env = self.env.prepare_subprocess(args)
         if mp:
@@ -XXX,XX +XXX,XX @@ def do_run_test(self, test: str, mp: bool) -> TestResult:
                 env[d] = os.path.join(env[d], f_test.name)
                 Path(env[d]).mkdir(parents=True, exist_ok=True)
 
+        test_dir = env['TEST_DIR']
+        f_bad = Path(test_dir, f_test.name + '.out.bad')
+        f_notrun = Path(test_dir, f_test.name + '.notrun')
+        f_casenotrun = Path(test_dir, f_test.name + '.casenotrun')
+
+        for p in (f_notrun, f_casenotrun):
+            silent_unlink(p)
+
         t0 = time.time()
         with f_bad.open('w', encoding="utf-8") as f:
             with subprocess.Popen(args, cwd=str(f_test.parent), env=env,
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We are going to complicate bitmap initialization in the further
commit. And in future, backup job will be able to work without filter
(when source is immutable), so we'll need same bitmap initialization in
copy-before-write filter and in backup job. So, it's reasonable to do
it in block-copy.

Note that for now cbw_open() is the only caller of
block_copy_state_new().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-2-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 block/block-copy.c        | 1 +
 block/copy-before-write.c | 4 ----
 2 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/block/block-copy.c b/block/block-copy.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
         return NULL;
     }
     bdrv_disable_dirty_bitmap(copy_bitmap);
+    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
 
     /*
      * If source is in backing chain of target assume that target is going to be
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index XXXXXXX..XXXXXXX 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
                     Error **errp)
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
-    BdrvDirtyBitmap *copy_bitmap;
 
     bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
                                BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
         return -EINVAL;
     }
 
-    copy_bitmap = block_copy_dirty_bitmap(s->bcs);
-    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
-
     return 0;
 }
 
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

That simplifies handling failure in existing code and in further new
usage of bdrv_merge_dirty_bitmap().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/dirty-bitmap.h    | 2 +-
 block/dirty-bitmap.c            | 9 +++++++--
 block/monitor/bitmap-qmp-cmds.c | 5 +----
 3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -XXX,XX +XXX,XX @@ void bdrv_dirty_bitmap_set_persistence(BdrvDirtyBitmap *bitmap,
                                        bool persistent);
 void bdrv_dirty_bitmap_set_inconsistent(BdrvDirtyBitmap *bitmap);
 void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, bool busy);
-void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
+bool bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
                              HBitmap **backup, Error **errp);
 void bdrv_dirty_bitmap_skip_store(BdrvDirtyBitmap *bitmap, bool skip);
 bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index XXXXXXX..XXXXXXX 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
  * Ensures permissions on bitmaps are reasonable; use for public API.
  *
  * @backup: If provided, make a copy of dest here prior to merge.
+ *
+ * Returns true on success, false on failure. In case of failure bitmaps are
+ * untouched.
  */
-void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
+bool bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
                              HBitmap **backup, Error **errp)
 {
-    bool ret;
+    bool ret = false;
 
     bdrv_dirty_bitmaps_lock(dest->bs);
     if (src->bs != dest->bs) {
@@ -XXX,XX +XXX,XX @@ out:
     if (src->bs != dest->bs) {
         bdrv_dirty_bitmaps_unlock(src->bs);
     }
+
+    return ret;
 }
 
 /**
diff --git a/block/monitor/bitmap-qmp-cmds.c b/block/monitor/bitmap-qmp-cmds.c
index XXXXXXX..XXXXXXX 100644
--- a/block/monitor/bitmap-qmp-cmds.c
+++ b/block/monitor/bitmap-qmp-cmds.c
@@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target,
     BlockDriverState *bs;
     BdrvDirtyBitmap *dst, *src, *anon;
     BlockDirtyBitmapMergeSourceList *lst;
-    Error *local_err = NULL;
 
     GLOBAL_STATE_CODE();
 
@@ -XXX,XX +XXX,XX @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target,
             abort();
         }
 
-        bdrv_merge_dirty_bitmap(anon, src, NULL, &local_err);
-        if (local_err) {
-            error_propagate(errp, local_err);
+        if (!bdrv_merge_dirty_bitmap(anon, src, NULL, errp)) {
             dst = NULL;
             goto out;
         }
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

This will be used in the following commit to bring "incremental" mode
to copy-before-write filter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-4-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/block-copy.h |  1 +
 block/block-copy.c         | 14 +++++++++++++-
 block/copy-before-write.c  |  2 +-
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState BlockCopyState;
 typedef struct BlockCopyCallState BlockCopyCallState;
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
+                                     const BdrvDirtyBitmap *bitmap,
                                      Error **errp);
 
 /* Function should be called prior any actual copy request */
diff --git a/block/block-copy.c b/block/block-copy.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -XXX,XX +XXX,XX @@ static int64_t block_copy_calculate_cluster_size(BlockDriverState *target,
 }
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
+                                     const BdrvDirtyBitmap *bitmap,
                                      Error **errp)
 {
+    ERRP_GUARD();
     BlockCopyState *s;
     int64_t cluster_size;
     BdrvDirtyBitmap *copy_bitmap;
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
         return NULL;
     }
     bdrv_disable_dirty_bitmap(copy_bitmap);
-    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
+    if (bitmap) {
+        if (!bdrv_merge_dirty_bitmap(copy_bitmap, bitmap, NULL, errp)) {
+            error_prepend(errp, "Failed to merge bitmap '%s' to internal "
+                          "copy-bitmap: ", bdrv_dirty_bitmap_name(bitmap));
+            bdrv_release_dirty_bitmap(copy_bitmap);
+            return NULL;
+        }
+    } else {
+        bdrv_set_dirty_bitmap(copy_bitmap, 0,
+                              bdrv_dirty_bitmap_size(copy_bitmap));
+    }
 
     /*
      * If source is in backing chain of target assume that target is going to be
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index XXXXXXX..XXXXXXX 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
             ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
              bs->file->bs->supported_zero_flags);
 
-    s->bcs = block_copy_state_new(bs->file, s->target, errp);
+    s->bcs = block_copy_state_new(bs->file, s->target, NULL, errp);
     if (!s->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         return -EINVAL;
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

This brings "incremental" mode to copy-before-write filter: user can
specify bitmap so that filter will copy only "dirty" areas.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220303194349.2304213-5-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 qapi/block-core.json      | 10 +++++++-
 block/copy-before-write.c | 51 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 59 insertions(+), 2 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
 #
 # @target: The target for copy-before-write operations.
 #
+# @bitmap: If specified, copy-before-write filter will do
+#          copy-before-write operations only for dirty regions of the
+#          bitmap. Bitmap size must be equal to length of file and
+#          target child of the filter. Note also, that bitmap is used
+#          only to initialize internal bitmap of the process, so further
+#          modifications (or removing) of specified bitmap doesn't
+#          influence the filter. (Since 7.0)
+#
 # Since: 6.2
 ##
 { 'struct': 'BlockdevOptionsCbw',
   'base': 'BlockdevOptionsGenericFormat',
-  'data': { 'target': 'BlockdevRef' } }
+  'data': { 'target': 'BlockdevRef', '*bitmap': 'BlockDirtyBitmap' } }
 
 ##
 # @BlockdevOptions:
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index XXXXXXX..XXXXXXX 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -XXX,XX +XXX,XX @@
 
 #include "block/copy-before-write.h"
 
+#include "qapi/qapi-visit-block-core.h"
+
 typedef struct BDRVCopyBeforeWriteState {
     BlockCopyState *bcs;
     BdrvChild *target;
@@ -XXX,XX +XXX,XX @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
     }
 }
 
+static bool cbw_parse_bitmap_option(QDict *options, BdrvDirtyBitmap **bitmap,
+                                    Error **errp)
+{
+    QDict *bitmap_qdict = NULL;
+    BlockDirtyBitmap *bmp_param = NULL;
+    Visitor *v = NULL;
+    bool ret = false;
+
+    *bitmap = NULL;
+
+    qdict_extract_subqdict(options, &bitmap_qdict, "bitmap.");
+    if (!qdict_size(bitmap_qdict)) {
+        ret = true;
+        goto out;
+    }
+
+    v = qobject_input_visitor_new_flat_confused(bitmap_qdict, errp);
+    if (!v) {
+        goto out;
+    }
+
+    visit_type_BlockDirtyBitmap(v, NULL, &bmp_param, errp);
+    if (!bmp_param) {
+        goto out;
+    }
+
+    *bitmap = block_dirty_bitmap_lookup(bmp_param->node, bmp_param->name, NULL,
+                                        errp);
+    if (!*bitmap) {
+        goto out;
+    }
+
+    ret = true;
+
+out:
+    qapi_free_BlockDirtyBitmap(bmp_param);
+    visit_free(v);
+    qobject_unref(bitmap_qdict);
+
+    return ret;
+}
+
 static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
                     Error **errp)
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
+    BdrvDirtyBitmap *bitmap = NULL;
 
     bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
                                BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
         return -EINVAL;
     }
 
+    if (!cbw_parse_bitmap_option(options, &bitmap, errp)) {
+        return -EINVAL;
+    }
+
     bs->total_sectors = bs->file->bs->total_sectors;
     bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
             (BDRV_REQ_FUA & bs->file->bs->supported_write_flags);
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
             ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
              bs->file->bs->supported_zero_flags);
 
-    s->bcs = block_copy_state_new(bs->file, s->target, NULL, errp);
+    s->bcs = block_copy_state_new(bs->file, s->target, bitmap, errp);
     if (!s->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         return -EINVAL;
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Split block_copy_reset() out of block_copy_reset_unallocated() to be
used separately later.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-6-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/block-copy.h |  1 +
 block/block-copy.c         | 21 +++++++++++++--------
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -XXX,XX +XXX,XX @@ void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
 
 void block_copy_state_free(BlockCopyState *s);
 
+void block_copy_reset(BlockCopyState *s, int64_t offset, int64_t bytes);
 int64_t block_copy_reset_unallocated(BlockCopyState *s,
                                      int64_t offset, int64_t *count);
 
diff --git a/block/block-copy.c b/block/block-copy.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -XXX,XX +XXX,XX @@ static int block_copy_is_cluster_allocated(BlockCopyState *s, int64_t offset,
     }
 }
 
+void block_copy_reset(BlockCopyState *s, int64_t offset, int64_t bytes)
+{
+    QEMU_LOCK_GUARD(&s->lock);
+
+    bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
+    if (s->progress) {
+        progress_set_remaining(s->progress,
+                               bdrv_get_dirty_count(s->copy_bitmap) +
+                               s->in_flight_bytes);
+    }
+}
+
 /*
  * Reset bits in copy_bitmap starting at offset if they represent unallocated
  * data in the image. May reset subsequent contiguous bits.
@@ -XXX,XX +XXX,XX @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
     bytes = clusters * s->cluster_size;
 
     if (!ret) {
-        qemu_co_mutex_lock(&s->lock);
-        bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
-        if (s->progress) {
-            progress_set_remaining(s->progress,
-                                   bdrv_get_dirty_count(s->copy_bitmap) +
-                                   s->in_flight_bytes);
-        }
-        qemu_co_mutex_unlock(&s->lock);
+        block_copy_reset(s, offset, bytes);
     }
 
     *count = bytes;
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Split intersecting-requests functionality out of block-copy to be
reused in copy-before-write filter.

Note: while being here, fix tiny typo in MAINTAINERS.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-7-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/reqlist.h |  67 +++++++++++++++++++++++
 block/block-copy.c      | 116 +++++++++++++---------------------------
 block/reqlist.c         |  76 ++++++++++++++++++++++++++
 MAINTAINERS             |   4 +-
 block/meson.build       |   1 +
 5 files changed, 184 insertions(+), 80 deletions(-)
 create mode 100644 include/block/reqlist.h
 create mode 100644 block/reqlist.c

diff --git a/include/block/reqlist.h b/include/block/reqlist.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/include/block/reqlist.h
@@ -XXX,XX +XXX,XX @@
+/*
+ * reqlist API
+ *
+ * Copyright (C) 2013 Proxmox Server Solutions
+ * Copyright (c) 2021 Virtuozzo International GmbH.
+ *
+ * Authors:
+ *  Dietmar Maurer (dietmar@proxmox.com)
+ *  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef REQLIST_H
+#define REQLIST_H
+
+#include "qemu/coroutine.h"
+
+/*
+ * The API is not thread-safe and shouldn't be. The struct is public to be part
+ * of other structures and protected by third-party locks, see
+ * block/block-copy.c for example.
+ */
+
+typedef struct BlockReq {
+    int64_t offset;
+    int64_t bytes;
+
+    CoQueue wait_queue; /* coroutines blocked on this req */
+    QLIST_ENTRY(BlockReq) list;
+} BlockReq;
+
+typedef QLIST_HEAD(, BlockReq) BlockReqList;
+
+/*
+ * Initialize new request and add it to the list. Caller must be sure that
+ * there are no conflicting requests in the list.
+ */
+void reqlist_init_req(BlockReqList *reqs, BlockReq *req, int64_t offset,
+                      int64_t bytes);
+/* Search for request in the list intersecting with @offset/@bytes area. */
+BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
+                                int64_t bytes);
+
+/*
+ * If there are no intersecting requests return false. Otherwise, wait for the
+ * first found intersecting request to finish and return true.
+ *
+ * @lock is passed to qemu_co_queue_wait()
+ * False return value proves that lock was released at no point.
+ */
+bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
+                                   int64_t bytes, CoMutex *lock);
+
+/*
+ * Shrink request and wake all waiting coroutines (maybe some of them are not
+ * intersecting with shrunk request).
+ */
+void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes);
+
+/*
+ * Remove request and wake all waiting coroutines. Do not release any memory.
+ */
+void coroutine_fn reqlist_remove_req(BlockReq *req);
+
+#endif /* REQLIST_H */
diff --git a/block/block-copy.c b/block/block-copy.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -XXX,XX +XXX,XX @@
 #include "trace.h"
 #include "qapi/error.h"
 #include "block/block-copy.h"
+#include "block/reqlist.h"
 #include "sysemu/block-backend.h"
 #include "qemu/units.h"
 #include "qemu/coroutine.h"
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyTask {
      */
     BlockCopyState *s;
     BlockCopyCallState *call_state;
-    int64_t offset;
     /*
      * @method can also be set again in the while loop of
      * block_copy_dirty_clusters(), but it is never accessed concurrently
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyTask {
     BlockCopyMethod method;
 
     /*
-     * Fields whose state changes throughout the execution
-     * Protected by lock in BlockCopyState.
+     * Generally, req is protected by lock in BlockCopyState, Still req.offset
+     * is only set on task creation, so may be read concurrently after creation.
+     * req.bytes is changed at most once, and need only protecting the case of
+     * parallel read while updating @bytes value in block_copy_task_shrink().
      */
-    CoQueue wait_queue; /* coroutines blocked on this task */
-    /*
-     * Only protect the case of parallel read while updating @bytes
-     * value in block_copy_task_shrink().
-     */
-    int64_t bytes;
-    QLIST_ENTRY(BlockCopyTask) list;
+    BlockReq req;
 } BlockCopyTask;
 
 static int64_t task_end(BlockCopyTask *task)
 {
-    return task->offset + task->bytes;
+    return task->req.offset + task->req.bytes;
 }
 
 typedef struct BlockCopyState {
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState {
     CoMutex lock;
     int64_t in_flight_bytes;
     BlockCopyMethod method;
-    QLIST_HEAD(, BlockCopyTask) tasks; /* All tasks from all block-copy calls */
+    BlockReqList reqs;
     QLIST_HEAD(, BlockCopyCallState) calls;
     /*
      * skip_unallocated:
@@ -XXX,XX +XXX,XX @@ typedef struct BlockCopyState {
     RateLimit rate_limit;
 } BlockCopyState;
 
-/* Called with lock held */
-static BlockCopyTask *find_conflicting_task(BlockCopyState *s,
-                                            int64_t offset, int64_t bytes)
-{
-    BlockCopyTask *t;
-
-    QLIST_FOREACH(t, &s->tasks, list) {
-        if (offset + bytes > t->offset && offset < t->offset + t->bytes) {
-            return t;
-        }
-    }
-
-    return NULL;
-}
-
-/*
- * If there are no intersecting tasks return false. Otherwise, wait for the
- * first found intersecting tasks to finish and return true.
- *
- * Called with lock held. May temporary release the lock.
- * Return value of 0 proves that lock was NOT released.
- */
-static bool coroutine_fn block_copy_wait_one(BlockCopyState *s, int64_t offset,
-                                             int64_t bytes)
-{
-    BlockCopyTask *task = find_conflicting_task(s, offset, bytes);
-
-    if (!task) {
-        return false;
-    }
-
-    qemu_co_queue_wait(&task->wait_queue, &s->lock);
-
-    return true;
-}
-
 /* Called with lock held */
 static int64_t block_copy_chunk_size(BlockCopyState *s)
 {
@@ -XXX,XX +XXX,XX @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state,
     bytes = QEMU_ALIGN_UP(bytes, s->cluster_size);
 
     /* region is dirty, so no existent tasks possible in it */
-    assert(!find_conflicting_task(s, offset, bytes));
+    assert(!reqlist_find_conflict(&s->reqs, offset, bytes));
 
     bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
     s->in_flight_bytes += bytes;
@@ -XXX,XX +XXX,XX @@ block_copy_task_create(BlockCopyState *s, BlockCopyCallState *call_state,
         .task.func = block_copy_task_entry,
         .s = s,
         .call_state = call_state,
-        .offset = offset,
-        .bytes = bytes,
         .method = s->method,
     };
-    qemu_co_queue_init(&task->wait_queue);
-    QLIST_INSERT_HEAD(&s->tasks, task, list);
+    reqlist_init_req(&s->reqs, &task->req, offset, bytes);
 
     return task;
 }
@@ -XXX,XX +XXX,XX @@ static void coroutine_fn block_copy_task_shrink(BlockCopyTask *task,
                                                 int64_t new_bytes)
 {
     QEMU_LOCK_GUARD(&task->s->lock);
-    if (new_bytes == task->bytes) {
+    if (new_bytes == task->req.bytes) {
         return;
     }
 
-    assert(new_bytes > 0 && new_bytes < task->bytes);
+    assert(new_bytes > 0 && new_bytes < task->req.bytes);
 
-    task->s->in_flight_bytes -= task->bytes - new_bytes;
+    task->s->in_flight_bytes -= task->req.bytes - new_bytes;
     bdrv_set_dirty_bitmap(task->s->copy_bitmap,
-                          task->offset + new_bytes, task->bytes - new_bytes);
+                          task->req.offset + new_bytes,
+                          task->req.bytes - new_bytes);
 
-    task->bytes = new_bytes;
-    qemu_co_queue_restart_all(&task->wait_queue);
+    reqlist_shrink_req(&task->req, new_bytes);
 }
 
 static void coroutine_fn block_copy_task_end(BlockCopyTask *task, int ret)
 {
     QEMU_LOCK_GUARD(&task->s->lock);
-    task->s->in_flight_bytes -= task->bytes;
+    task->s->in_flight_bytes -= task->req.bytes;
     if (ret < 0) {
-        bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->offset, task->bytes);
+        bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->req.offset,
+                              task->req.bytes);
     }
-    QLIST_REMOVE(task, list);
     if (task->s->progress) {
         progress_set_remaining(task->s->progress,
                                bdrv_get_dirty_count(task->s->copy_bitmap) +
                                task->s->in_flight_bytes);
     }
-    qemu_co_queue_restart_all(&task->wait_queue);
+    reqlist_remove_req(&task->req);
 }
 
 void block_copy_state_free(BlockCopyState *s)
@@ -XXX,XX +XXX,XX @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
 
     ratelimit_init(&s->rate_limit);
     qemu_co_mutex_init(&s->lock);
-    QLIST_INIT(&s->tasks);
+    QLIST_INIT(&s->reqs);
     QLIST_INIT(&s->calls);
 
     return s;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_run(AioTaskPool *pool,
 
     aio_task_pool_wait_slot(pool);
     if (aio_task_pool_status(pool) < 0) {
-        co_put_to_shres(task->s->mem, task->bytes);
+        co_put_to_shres(task->s->mem, task->req.bytes);
         block_copy_task_end(task, -ECANCELED);
         g_free(task);
         return -ECANCELED;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
     BlockCopyMethod method = t->method;
     int ret;
 
-    ret = block_copy_do_copy(s, t->offset, t->bytes, &method, &error_is_read);
+    ret = block_copy_do_copy(s, t->req.offset, t->req.bytes, &method,
+                             &error_is_read);
 
     WITH_QEMU_LOCK_GUARD(&s->lock) {
         if (s->method == t->method) {
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
                 t->call_state->error_is_read = error_is_read;
             }
         } else if (s->progress) {
-            progress_work_done(s->progress, t->bytes);
+            progress_work_done(s->progress, t->req.bytes);
         }
     }
-    co_put_to_shres(s->mem, t->bytes);
+    co_put_to_shres(s->mem, t->req.bytes);
     block_copy_task_end(t, ret);
 
     return ret;
@@ -XXX,XX +XXX,XX @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
             trace_block_copy_skip_range(s, offset, bytes);
             break;
         }
-        if (task->offset > offset) {
-            trace_block_copy_skip_range(s, offset, task->offset - offset);
+        if (task->req.offset > offset) {
+            trace_block_copy_skip_range(s, offset, task->req.offset - offset);
         }
 
         found_dirty = true;
 
-        ret = block_copy_block_status(s, task->offset, task->bytes,
+        ret = block_copy_block_status(s, task->req.offset, task->req.bytes,
                                       &status_bytes);
         assert(ret >= 0); /* never fail */
-        if (status_bytes < task->bytes) {
+        if (status_bytes < task->req.bytes) {
             block_copy_task_shrink(task, status_bytes);
         }
         if (qatomic_read(&s->skip_unallocated) &&
             !(ret & BDRV_BLOCK_ALLOCATED)) {
             block_copy_task_end(task, 0);
-            trace_block_copy_skip_range(s, task->offset, task->bytes);
+            trace_block_copy_skip_range(s, task->req.offset, task->req.bytes);
             offset = task_end(task);
             bytes = end - offset;
             g_free(task);
@@ -XXX,XX +XXX,XX @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
             }
         }
 
-        ratelimit_calculate_delay(&s->rate_limit, task->bytes);
+        ratelimit_calculate_delay(&s->rate_limit, task->req.bytes);
 
-        trace_block_copy_process(s, task->offset);
+        trace_block_copy_process(s, task->req.offset);
 
-        co_get_from_shres(s->mem, task->bytes);
+        co_get_from_shres(s->mem, task->req.bytes);
 
         offset = task_end(task);
         bytes = end - offset;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
                  * Check that there is no task we still need to
                  * wait to complete
                  */
-                ret = block_copy_wait_one(s, call_state->offset,
-                                          call_state->bytes);
+                ret = reqlist_wait_one(&s->reqs, call_state->offset,
+                                       call_state->bytes, &s->lock);
                 if (ret == 0) {
                     /*
                      * No pending tasks, but check again the bitmap in this
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
                      * between this and the critical section in
                      * block_copy_dirty_clusters().
                      *
-                     * block_copy_wait_one return value 0 also means that it
+                     * reqlist_wait_one return value 0 also means that it
                      * didn't release the lock. So, we are still in the same
                      * critical section, not interrupted by any concurrent
                      * access to state.
diff --git a/block/reqlist.c b/block/reqlist.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/block/reqlist.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * reqlist API
+ *
+ * Copyright (C) 2013 Proxmox Server Solutions
+ * Copyright (c) 2021 Virtuozzo International GmbH.
+ *
+ * Authors:
+ *  Dietmar Maurer (dietmar@proxmox.com)
+ *  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+
+#include "block/reqlist.h"
+
+void reqlist_init_req(BlockReqList *reqs, BlockReq *req, int64_t offset,
+                      int64_t bytes)
+{
+    assert(!reqlist_find_conflict(reqs, offset, bytes));
+
+    *req = (BlockReq) {
+        .offset = offset,
+        .bytes = bytes,
+    };
+    qemu_co_queue_init(&req->wait_queue);
+    QLIST_INSERT_HEAD(reqs, req, list);
+}
+
+BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
+                                int64_t bytes)
+{
+    BlockReq *r;
+
+    QLIST_FOREACH(r, reqs, list) {
+        if (offset + bytes > r->offset && offset < r->offset + r->bytes) {
+            return r;
+        }
+    }
+
+    return NULL;
+}
+
+bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
+                                   int64_t bytes, CoMutex *lock)
+{
+    BlockReq *r = reqlist_find_conflict(reqs, offset, bytes);
+
+    if (!r) {
+        return false;
+    }
+
+    qemu_co_queue_wait(&r->wait_queue, lock);
+
+    return true;
+}
+
+void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes)
+{
+    if (new_bytes == req->bytes) {
+        return;
+    }
+
+    assert(new_bytes > 0 && new_bytes < req->bytes);
+
+    req->bytes = new_bytes;
+    qemu_co_queue_restart_all(&req->wait_queue);
+}
+
+void coroutine_fn reqlist_remove_req(BlockReq *req)
+{
+    QLIST_REMOVE(req, list);
+    qemu_co_queue_restart_all(&req->wait_queue);
+}
diff --git a/MAINTAINERS b/MAINTAINERS
index XXXXXXX..XXXXXXX 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -XXX,XX +XXX,XX @@ F: block/stream.c
 F: block/mirror.c
 F: qapi/job.json
 F: block/block-copy.c
-F: include/block/block-copy.c
+F: include/block/block-copy.h
+F: block/reqlist.c
+F: include/block/reqlist.h
 F: block/copy-before-write.h
 F: block/copy-before-write.c
 F: include/block/aio_task.h
diff --git a/block/meson.build b/block/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -XXX,XX +XXX,XX @@ block_ss.add(files(
   'qcow2.c',
   'quorum.c',
   'raw-format.c',
+  'reqlist.c',
   'snapshot.c',
   'throttle-groups.c',
   'throttle.c',
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Add a convenient function similar with bdrv_block_status() to get
status of dirty bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-9-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/dirty-bitmap.h |  2 ++
 include/qemu/hbitmap.h       | 12 ++++++++++++
 block/dirty-bitmap.c         |  6 ++++++
 util/hbitmap.c               | 33 +++++++++++++++++++++++++++++++++
 4 files changed, 53 insertions(+)

diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -XXX,XX +XXX,XX @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
 bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
         int64_t start, int64_t end, int64_t max_dirty_count,
         int64_t *dirty_start, int64_t *dirty_count);
+bool bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap, int64_t offset,
+                              int64_t bytes, int64_t *count);
 BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
                                                   Error **errp);
 
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index XXXXXXX..XXXXXXX 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -XXX,XX +XXX,XX @@ bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
                              int64_t max_dirty_count,
                              int64_t *dirty_start, int64_t *dirty_count);
 
+/*
+ * bdrv_dirty_bitmap_status:
+ * @hb: The HBitmap to operate on
+ * @start: The bit to start from
+ * @count: Number of bits to proceed
+ * @pnum: Out-parameter. How many bits has same value starting from @start
+ *
+ * Returns true if bitmap is dirty at @start, false otherwise.
+ */
+bool hbitmap_status(const HBitmap *hb, int64_t start, int64_t count,
+                    int64_t *pnum);
+
 /**
  * hbitmap_iter_next:
  * @hbi: HBitmapIter to operate on.
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index XXXXXXX..XXXXXXX 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -XXX,XX +XXX,XX @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
                                    dirty_start, dirty_count);
 }
 
+bool bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap, int64_t offset,
+                              int64_t bytes, int64_t *count)
+{
+    return hbitmap_status(bitmap->bitmap, offset, bytes, count);
+}
+
 /**
  * bdrv_merge_dirty_bitmap: merge src into dest.
  * Ensures permissions on bitmaps are reasonable; use for public API.
diff --git a/util/hbitmap.c b/util/hbitmap.c
index XXXXXXX..XXXXXXX 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -XXX,XX +XXX,XX @@ bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
     return true;
 }
 
+bool hbitmap_status(const HBitmap *hb, int64_t start, int64_t count,
+                    int64_t *pnum)
+{
+    int64_t next_dirty, next_zero;
+
+    assert(start >= 0);
+    assert(count > 0);
+    assert(start + count <= hb->orig_size);
+
+    next_dirty = hbitmap_next_dirty(hb, start, count);
+    if (next_dirty == -1) {
+        *pnum = count;
+        return false;
+    }
+
+    if (next_dirty > start) {
+        *pnum = next_dirty - start;
+        return false;
+    }
+
+    assert(next_dirty == start);
+
+    next_zero = hbitmap_next_zero(hb, start, count);
+    if (next_zero == -1) {
+        *pnum = count;
+        return true;
+    }
+
+    assert(next_zero > start);
+    *pnum = next_zero - start;
+    return false;
+}
+
 bool hbitmap_empty(const HBitmap *hb)
 {
     return hb->count == 0;
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Add function to wait for all intersecting requests.
To be used in the further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-10-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/reqlist.h | 8 ++++++++
 block/reqlist.c         | 8 ++++++++
 2 files changed, 16 insertions(+)

diff --git a/include/block/reqlist.h b/include/block/reqlist.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/reqlist.h
+++ b/include/block/reqlist.h
@@ -XXX,XX +XXX,XX @@ BlockReq *reqlist_find_conflict(BlockReqList *reqs, int64_t offset,
 bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
                                    int64_t bytes, CoMutex *lock);
 
+/*
+ * Wait for all intersecting requests. It just calls reqlist_wait_one() in a
+ * loop, caller is responsible to stop producing new requests in this region
+ * in parallel, otherwise reqlist_wait_all() may never return.
+ */
+void coroutine_fn reqlist_wait_all(BlockReqList *reqs, int64_t offset,
+                                   int64_t bytes, CoMutex *lock);
+
 /*
  * Shrink request and wake all waiting coroutines (maybe some of them are not
  * intersecting with shrunk request).
diff --git a/block/reqlist.c b/block/reqlist.c
index XXXXXXX..XXXXXXX 100644
--- a/block/reqlist.c
+++ b/block/reqlist.c
@@ -XXX,XX +XXX,XX @@ bool coroutine_fn reqlist_wait_one(BlockReqList *reqs, int64_t offset,
     return true;
 }
 
+void coroutine_fn reqlist_wait_all(BlockReqList *reqs, int64_t offset,
+                                   int64_t bytes, CoMutex *lock)
+{
+    while (reqlist_wait_one(reqs, offset, bytes, lock)) {
+        /* continue */
+    }
+}
+
 void coroutine_fn reqlist_shrink_req(BlockReq *req, int64_t new_bytes)
 {
     if (new_bytes == req->bytes) {
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Add new block driver handlers and corresponding generic wrappers.
It will be used to allow copy-before-write filter to provide
reach fleecing interface in further commit.

In future this approach may be used to allow reading qcow2 internal
snapshots, for example to export them through NBD.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-11-vsementsov@virtuozzo.com>
[hreitz: Rebased on block GS/IO split]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 include/block/block_int-common.h | 18 ++++++++
 include/block/block_int-io.h     |  9 ++++
 block/io.c                       | 72 ++++++++++++++++++++++++++++++++
 3 files changed, 99 insertions(+)

diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
         bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
         int64_t *map, BlockDriverState **file);
 
+    /*
+     * Snapshot-access API.
+     *
+     * Block-driver may provide snapshot-access API: special functions to access
+     * some internal "snapshot". The functions are similar with normal
+     * read/block_status/discard handler, but don't have any specific handling
+     * in generic block-layer: no serializing, no alignment, no tracked
+     * requests. So, block-driver that realizes these APIs is fully responsible
+     * for synchronization between snapshot-access API and normal IO requests.
+     */
+    int coroutine_fn (*bdrv_co_preadv_snapshot)(BlockDriverState *bs,
+        int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
+    int coroutine_fn (*bdrv_co_snapshot_block_status)(BlockDriverState *bs,
+        bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
+        int64_t *map, BlockDriverState **file);
+    int coroutine_fn (*bdrv_co_pdiscard_snapshot)(BlockDriverState *bs,
+        int64_t offset, int64_t bytes);
+
     /*
      * Invalidate any cached meta-data.
      */
diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-io.h
+++ b/include/block/block_int-io.h
@@ -XXX,XX +XXX,XX @@
  * the I/O API.
  */
 
+int coroutine_fn bdrv_co_preadv_snapshot(BdrvChild *child,
+    int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
+int coroutine_fn bdrv_co_snapshot_block_status(BlockDriverState *bs,
+    bool want_zero, int64_t offset, int64_t bytes, int64_t *pnum,
+    int64_t *map, BlockDriverState **file);
+int coroutine_fn bdrv_co_pdiscard_snapshot(BlockDriverState *bs,
+    int64_t offset, int64_t bytes);
+
+
 int coroutine_fn bdrv_co_preadv(BdrvChild *child,
     int64_t offset, int64_t bytes, QEMUIOVector *qiov,
     BdrvRequestFlags flags);
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ void bdrv_cancel_in_flight(BlockDriverState *bs)
         bs->drv->bdrv_cancel_in_flight(bs);
     }
 }
+
+int coroutine_fn
+bdrv_co_preadv_snapshot(BdrvChild *child, int64_t offset, int64_t bytes,
+                        QEMUIOVector *qiov, size_t qiov_offset)
+{
+    BlockDriverState *bs = child->bs;
+    BlockDriver *drv = bs->drv;
+    int ret;
+    IO_CODE();
+
+    if (!drv) {
+        return -ENOMEDIUM;
+    }
+
+    if (!drv->bdrv_co_preadv_snapshot) {
+        return -ENOTSUP;
+    }
+
+    bdrv_inc_in_flight(bs);
+    ret = drv->bdrv_co_preadv_snapshot(bs, offset, bytes, qiov, qiov_offset);
+    bdrv_dec_in_flight(bs);
+
+    return ret;
+}
+
+int coroutine_fn
+bdrv_co_snapshot_block_status(BlockDriverState *bs,
+                              bool want_zero, int64_t offset, int64_t bytes,
+                              int64_t *pnum, int64_t *map,
+                              BlockDriverState **file)
+{
+    BlockDriver *drv = bs->drv;
+    int ret;
+    IO_CODE();
+
+    if (!drv) {
+        return -ENOMEDIUM;
+    }
+
+    if (!drv->bdrv_co_snapshot_block_status) {
+        return -ENOTSUP;
+    }
+
+    bdrv_inc_in_flight(bs);
+    ret = drv->bdrv_co_snapshot_block_status(bs, want_zero, offset, bytes,
+                                             pnum, map, file);
+    bdrv_dec_in_flight(bs);
+
+    return ret;
+}
+
+int coroutine_fn
+bdrv_co_pdiscard_snapshot(BlockDriverState *bs, int64_t offset, int64_t bytes)
+{
+    BlockDriver *drv = bs->drv;
+    int ret;
+    IO_CODE();
+
+    if (!drv) {
+        return -ENOMEDIUM;
+    }
+
+    if (!drv->bdrv_co_pdiscard_snapshot) {
+        return -ENOTSUP;
+    }
+
+    bdrv_inc_in_flight(bs);
+    ret = drv->bdrv_co_pdiscard_snapshot(bs, offset, bytes);
+    bdrv_dec_in_flight(bs);
+
+    return ret;
+}
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

The new block driver simply utilizes snapshot-access API of underlying
block node.

In further patches we want to use it like this:

This way, NBD client will be able to read snapshotted state of active
disk, when active disk is continued to be written by guest. This is
known as "fleecing", and currently uses another scheme based on qcow2
temporary image which backing file is active-disk. New scheme comes
with benefits - see next commit.

The other possible application is exporting internal snapshots of
qcow2, like this:

[guest]          [NBD export]
   |                  |
   | root             | root
   v       file       v
[qcow2]<---------[snapshot-access]

For this, we'll need to implement snapshot-access API handlers in
qcow2 driver, and improve snapshot-access block driver (and API) to
make it possible to select snapshot by name. Another thing to improve
is size of snapshot. Now for simplicity we just use size of bs->file,
which is OK for backup, but for qcow2 snapshots export we'll need to
imporve snapshot-access API to get size of snapshot.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220303194349.2304213-12-vsementsov@virtuozzo.com>
[hreitz: Rebased on block GS/IO split]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 qapi/block-core.json             |   4 +-
 include/block/block_int-common.h |   6 ++
 block/snapshot-access.c          | 132 +++++++++++++++++++++++++++++++
 MAINTAINERS                      |   1 +
 block/meson.build                |   1 +
 5 files changed, 143 insertions(+), 1 deletion(-)
 create mode 100644 block/snapshot-access.c

diff --git a/qapi/block-core.json b/qapi/block-core.json
index XXXXXXX..XXXXXXX 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -XXX,XX +XXX,XX @@
 # @blkreplay: Since 4.2
 # @compress: Since 5.0
 # @copy-before-write: Since 6.2
+# @snapshot-access: Since 7.0
 #
 # Since: 2.9
 ##
 { 'enum': 'BlockdevDriver',
   'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
             'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
-            'file', 'ftp', 'ftps', 'gluster',
+            'file', 'snapshot-access', 'ftp', 'ftps', 'gluster',
             {'name': 'host_cdrom', 'if': 'HAVE_HOST_BLOCK_DEVICE' },
             {'name': 'host_device', 'if': 'HAVE_HOST_BLOCK_DEVICE' },
             'http', 'https', 'iscsi',
@@ -XXX,XX +XXX,XX @@
       'rbd':        'BlockdevOptionsRbd',
       'replication': { 'type': 'BlockdevOptionsReplication',
                        'if': 'CONFIG_REPLICATION' },
+      'snapshot-access': 'BlockdevOptionsGenericFormat',
       'ssh':        'BlockdevOptionsSsh',
       'throttle':   'BlockdevOptionsThrottle',
       'vdi':        'BlockdevOptionsGenericFormat',
diff --git a/include/block/block_int-common.h b/include/block/block_int-common.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block_int-common.h
+++ b/include/block/block_int-common.h
@@ -XXX,XX +XXX,XX @@ struct BlockDriver {
      * in generic block-layer: no serializing, no alignment, no tracked
      * requests. So, block-driver that realizes these APIs is fully responsible
      * for synchronization between snapshot-access API and normal IO requests.
+     *
+     * TODO: To be able to support qcow2's internal snapshots, this API will
+     * need to be extended to:
+     * - be able to select a specific snapshot
+     * - receive the snapshot's actual length (which may differ from bs's
+     *   length)
      */
     int coroutine_fn (*bdrv_co_preadv_snapshot)(BlockDriverState *bs,
         int64_t offset, int64_t bytes, QEMUIOVector *qiov, size_t qiov_offset);
diff --git a/block/snapshot-access.c b/block/snapshot-access.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/block/snapshot-access.c
@@ -XXX,XX +XXX,XX @@
+/*
+ * snapshot_access block driver
+ *
+ * Copyright (c) 2022 Virtuozzo International GmbH.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir <vsementsov@virtuozzo.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+
+#include "sysemu/block-backend.h"
+#include "qemu/cutils.h"
+#include "block/block_int.h"
+
+static coroutine_fn int
+snapshot_access_co_preadv_part(BlockDriverState *bs,
+                               int64_t offset, int64_t bytes,
+                               QEMUIOVector *qiov, size_t qiov_offset,
+                               BdrvRequestFlags flags)
+{
+    if (flags) {
+        return -ENOTSUP;
+    }
+
+    return bdrv_co_preadv_snapshot(bs->file, offset, bytes, qiov, qiov_offset);
+}
+
+static int coroutine_fn
+snapshot_access_co_block_status(BlockDriverState *bs,
+                                bool want_zero, int64_t offset,
+                                int64_t bytes, int64_t *pnum,
+                                int64_t *map, BlockDriverState **file)
+{
+    return bdrv_co_snapshot_block_status(bs->file->bs, want_zero, offset,
+                                         bytes, pnum, map, file);
+}
+
+static int coroutine_fn snapshot_access_co_pdiscard(BlockDriverState *bs,
+                                             int64_t offset, int64_t bytes)
+{
+    return bdrv_co_pdiscard_snapshot(bs->file->bs, offset, bytes);
+}
+
+static int coroutine_fn
+snapshot_access_co_pwrite_zeroes(BlockDriverState *bs,
+                                 int64_t offset, int64_t bytes,
+                                 BdrvRequestFlags flags)
+{
+    return -ENOTSUP;
+}
+
+static coroutine_fn int
+snapshot_access_co_pwritev_part(BlockDriverState *bs,
+                                int64_t offset, int64_t bytes,
+                                QEMUIOVector *qiov, size_t qiov_offset,
+                                BdrvRequestFlags flags)
+{
+    return -ENOTSUP;
+}
+
+
+static void snapshot_access_refresh_filename(BlockDriverState *bs)
+{
+    pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
+            bs->file->bs->filename);
+}
+
+static int snapshot_access_open(BlockDriverState *bs, QDict *options, int flags,
+                                Error **errp)
+{
+    bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
+                               BDRV_CHILD_DATA | BDRV_CHILD_PRIMARY,
+                               false, errp);
+    if (!bs->file) {
+        return -EINVAL;
+    }
+
+    bs->total_sectors = bs->file->bs->total_sectors;
+
+    return 0;
+}
+
+static void snapshot_access_child_perm(BlockDriverState *bs, BdrvChild *c,
+                                BdrvChildRole role,
+                                BlockReopenQueue *reopen_queue,
+                                uint64_t perm, uint64_t shared,
+                                uint64_t *nperm, uint64_t *nshared)
+{
+    /*
+     * Currently, we don't need any permissions. If bs->file provides
+     * snapshot-access API, we can use it.
+     */
+    *nperm = 0;
+    *nshared = BLK_PERM_ALL;
+}
+
+BlockDriver bdrv_snapshot_access_drv = {
+    .format_name = "snapshot-access",
+
+    .bdrv_open                  = snapshot_access_open,
+
+    .bdrv_co_preadv_part        = snapshot_access_co_preadv_part,
+    .bdrv_co_pwritev_part       = snapshot_access_co_pwritev_part,
+    .bdrv_co_pwrite_zeroes      = snapshot_access_co_pwrite_zeroes,
+    .bdrv_co_pdiscard           = snapshot_access_co_pdiscard,
+    .bdrv_co_block_status       = snapshot_access_co_block_status,
+
+    .bdrv_refresh_filename      = snapshot_access_refresh_filename,
+
+    .bdrv_child_perm            = snapshot_access_child_perm,
+};
+
+static void snapshot_access_init(void)
+{
+    bdrv_register(&bdrv_snapshot_access_drv);
+}
+
+block_init(snapshot_access_init);
diff --git a/MAINTAINERS b/MAINTAINERS
index XXXXXXX..XXXXXXX 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -XXX,XX +XXX,XX @@ F: block/reqlist.c
 F: include/block/reqlist.h
 F: block/copy-before-write.h
 F: block/copy-before-write.c
+F: block/snapshot-access.c
 F: include/block/aio_task.h
 F: block/aio_task.c
 F: util/qemu-co-shared-resource.c
diff --git a/block/meson.build b/block/meson.build
index XXXXXXX..XXXXXXX 100644
--- a/block/meson.build
+++ b/block/meson.build
@@ -XXX,XX +XXX,XX @@ block_ss.add(files(
   'raw-format.c',
   'reqlist.c',
   'snapshot.c',
+  'snapshot-access.c',
   'throttle-groups.c',
   'throttle.c',
   'vhdx-endian.c',
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Current scheme of image fleecing looks like this:

- On guest writes copy-before-write filter copies old data from active
   disk to temp.qcow2. So fleecing client (NBD export) when reads
   changed regions from temp.qcow2 image and unchanged from active disk
   through backing link.

This patch makes possible new image fleecing scheme:

- copy-before-write does CBW operations and also provides
   snapshot-access API. The API may be accessed through
   snapshot-access driver.

Benefits of new scheme:

1. Access control: if remote client try to read data that not covered
   by original dirty bitmap used on copy-before-write open, client gets
   -EACCES.

2. Discard support: if remote client do DISCARD, this additionally to
   discarding data in temp.img informs block-copy process to not copy
   these clusters. Next read from discarded area will return -EACCES.
   This is significant thing: when fleecing user reads data that was
   not yet copied to temp.img, we can avoid copying it on further guest
   write.

3. Synchronisation between client reads and block-copy write is more
   efficient. In old scheme we just rely on BDRV_REQ_SERIALISING flag
   used for writes to temp.qcow2. New scheme is less blocking:
     - fleecing reads are never blocked: if data region is untouched or
       in-flight, we just read from active-disk, otherwise we read from
       temp.img
     - writes to temp.img are not blocked by fleecing reads
     - still, guest writes of-course are blocked by in-flight fleecing
       reads, that currently read from active-disk - it's the minimum
       necessary blocking

4. Temporary image may be of any format, as we don't rely on backing
   feature.

5. Permission relation are simplified. With old scheme we have to share
   write permission on target child of copy-before-write, otherwise
   backing link conflicts with copy-before-write file child write
   permissions. With new scheme we don't have backing link, and
   copy-before-write node may have unshared access to temporary node.
   (Not realized in this commit, will be in future).

6. Having control on fleecing reads we'll be able to implement
   alternative behavior on failed copy-before-write operations.
   Currently we just break guest request (that's a historical behavior
   of backup). But in some scenarios it's a bad behavior: better
   is to drop the backup as failed but don't break guest request.
   With new scheme we can simply unset some bits in a bitmap on CBW
   failure and further fleecing reads will -EACCES, or something like
   this. (Not implemented in this commit, will be in future)
   Additional application for this is implementing timeout for CBW
   operations.

Iotest 257 output is updated, as two more bitmaps now live in
copy-before-write filter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220303194349.2304213-13-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 block/copy-before-write.c  | 212 ++++++++++++++++++++++++++++++++++-
 tests/qemu-iotests/257.out | 224 +++++++++++++++++++++++++++++++++++++
 2 files changed, 435 insertions(+), 1 deletion(-)

diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index XXXXXXX..XXXXXXX 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -XXX,XX +XXX,XX @@
 #include "block/block-copy.h"
 
 #include "block/copy-before-write.h"
+#include "block/reqlist.h"
 
 #include "qapi/qapi-visit-block-core.h"
 
 typedef struct BDRVCopyBeforeWriteState {
     BlockCopyState *bcs;
     BdrvChild *target;
+
+    /*
+     * @lock: protects access to @access_bitmap, @done_bitmap and
+     * @frozen_read_reqs
+     */
+    CoMutex lock;
+
+    /*
+     * @access_bitmap: represents areas allowed for reading by fleecing user.
+     * Reading from non-dirty areas leads to -EACCES.
+     */
+    BdrvDirtyBitmap *access_bitmap;
+
+    /*
+     * @done_bitmap: represents areas that was successfully copied to @target by
+     * copy-before-write operations.
+     */
+    BdrvDirtyBitmap *done_bitmap;
+
+    /*
+     * @frozen_read_reqs: current read requests for fleecing user in bs->file
+     * node. These areas must not be rewritten by guest.
+     */
+    BlockReqList frozen_read_reqs;
 } BDRVCopyBeforeWriteState;
 
 static coroutine_fn int cbw_co_preadv(
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int cbw_co_preadv(
     return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);
 }
 
+/*
+ * Do copy-before-write operation.
+ *
+ * On failure guest request must be failed too.
+ *
+ * On success, we also wait for all in-flight fleecing read requests in source
+ * node, and it's guaranteed that after cbw_do_copy_before_write() successful
+ * return there are no such requests and they will never appear.
+ */
 static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
         uint64_t offset, uint64_t bytes, BdrvRequestFlags flags)
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
+    int ret;
     uint64_t off, end;
     int64_t cluster_size = block_copy_cluster_size(s->bcs);
 
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
     off = QEMU_ALIGN_DOWN(offset, cluster_size);
     end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
 
-    return block_copy(s->bcs, off, end - off, true);
+    ret = block_copy(s->bcs, off, end - off, true);
+    if (ret < 0) {
+        return ret;
+    }
+
+    WITH_QEMU_LOCK_GUARD(&s->lock) {
+        bdrv_set_dirty_bitmap(s->done_bitmap, off, end - off);
+        reqlist_wait_all(&s->frozen_read_reqs, off, end - off, &s->lock);
+    }
+
+    return 0;
 }
 
 static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs,
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn cbw_co_flush(BlockDriverState *bs)
     return bdrv_co_flush(bs->file->bs);
 }
 
+/*
+ * If @offset not accessible - return NULL.
+ *
+ * Otherwise, set @pnum to some bytes that accessible from @file (@file is set
+ * to bs->file or to s->target). Return newly allocated BlockReq object that
+ * should be than passed to cbw_snapshot_read_unlock().
+ *
+ * It's guaranteed that guest writes will not interact in the region until
+ * cbw_snapshot_read_unlock() called.
+ */
+static BlockReq *cbw_snapshot_read_lock(BlockDriverState *bs,
+                                        int64_t offset, int64_t bytes,
+                                        int64_t *pnum, BdrvChild **file)
+{
+    BDRVCopyBeforeWriteState *s = bs->opaque;
+    BlockReq *req = g_new(BlockReq, 1);
+    bool done;
+
+    QEMU_LOCK_GUARD(&s->lock);
+
+    if (bdrv_dirty_bitmap_next_zero(s->access_bitmap, offset, bytes) != -1) {
+        g_free(req);
+        return NULL;
+    }
+
+    done = bdrv_dirty_bitmap_status(s->done_bitmap, offset, bytes, pnum);
+    if (done) {
+        /*
+         * Special invalid BlockReq, that is handled in
+         * cbw_snapshot_read_unlock(). We don't need to lock something to read
+         * from s->target.
+         */
+        *req = (BlockReq) {.offset = -1, .bytes = -1};
+        *file = s->target;
+    } else {
+        reqlist_init_req(&s->frozen_read_reqs, req, offset, bytes);
+        *file = bs->file;
+    }
+
+    return req;
+}
+
+static void cbw_snapshot_read_unlock(BlockDriverState *bs, BlockReq *req)
+{
+    BDRVCopyBeforeWriteState *s = bs->opaque;
+
+    if (req->offset == -1 && req->bytes == -1) {
+        g_free(req);
+        return;
+    }
+
+    QEMU_LOCK_GUARD(&s->lock);
+
+    reqlist_remove_req(req);
+    g_free(req);
+}
+
+static coroutine_fn int
+cbw_co_preadv_snapshot(BlockDriverState *bs, int64_t offset, int64_t bytes,
+                       QEMUIOVector *qiov, size_t qiov_offset)
+{
+    BlockReq *req;
+    BdrvChild *file;
+    int ret;
+
+    /* TODO: upgrade to async loop using AioTask */
+    while (bytes) {
+        int64_t cur_bytes;
+
+        req = cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &file);
+        if (!req) {
+            return -EACCES;
+        }
+
+        ret = bdrv_co_preadv_part(file, offset, cur_bytes,
+                                  qiov, qiov_offset, 0);
+        cbw_snapshot_read_unlock(bs, req);
+        if (ret < 0) {
+            return ret;
+        }
+
+        bytes -= cur_bytes;
+        offset += cur_bytes;
+        qiov_offset += cur_bytes;
+    }
+
+    return 0;
+}
+
+static int coroutine_fn
+cbw_co_snapshot_block_status(BlockDriverState *bs,
+                             bool want_zero, int64_t offset, int64_t bytes,
+                             int64_t *pnum, int64_t *map,
+                             BlockDriverState **file)
+{
+    BDRVCopyBeforeWriteState *s = bs->opaque;
+    BlockReq *req;
+    int ret;
+    int64_t cur_bytes;
+    BdrvChild *child;
+
+    req = cbw_snapshot_read_lock(bs, offset, bytes, &cur_bytes, &child);
+    if (!req) {
+        return -EACCES;
+    }
+
+    ret = bdrv_block_status(child->bs, offset, cur_bytes, pnum, map, file);
+    if (child == s->target) {
+        /*
+         * We refer to s->target only for areas that we've written to it.
+         * And we can not report unallocated blocks in s->target: this will
+         * break generic block-status-above logic, that will go to
+         * copy-before-write filtered child in this case.
+         */
+        assert(ret & BDRV_BLOCK_ALLOCATED);
+    }
+
+    cbw_snapshot_read_unlock(bs, req);
+
+    return ret;
+}
+
+static int coroutine_fn cbw_co_pdiscard_snapshot(BlockDriverState *bs,
+                                                 int64_t offset, int64_t bytes)
+{
+    BDRVCopyBeforeWriteState *s = bs->opaque;
+
+    WITH_QEMU_LOCK_GUARD(&s->lock) {
+        bdrv_reset_dirty_bitmap(s->access_bitmap, offset, bytes);
+    }
+
+    block_copy_reset(s->bcs, offset, bytes);
+
+    return bdrv_co_pdiscard(s->target, offset, bytes);
+}
+
 static void cbw_refresh_filename(BlockDriverState *bs)
 {
     pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
     BdrvDirtyBitmap *bitmap = NULL;
+    int64_t cluster_size;
 
     bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
                                BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -XXX,XX +XXX,XX @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
         return -EINVAL;
     }
 
+    cluster_size = block_copy_cluster_size(s->bcs);
+
+    s->done_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
+    if (!s->done_bitmap) {
+        return -EINVAL;
+    }
+    bdrv_disable_dirty_bitmap(s->done_bitmap);
+
+    /* s->access_bitmap starts equal to bcs bitmap */
+    s->access_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
+    if (!s->access_bitmap) {
+        return -EINVAL;
+    }
+    bdrv_disable_dirty_bitmap(s->access_bitmap);
+    bdrv_dirty_bitmap_merge_internal(s->access_bitmap,
+                                     block_copy_dirty_bitmap(s->bcs), NULL,
+                                     true);
+
+    qemu_co_mutex_init(&s->lock);
+    QLIST_INIT(&s->frozen_read_reqs);
+
     return 0;
 }
 
@@ -XXX,XX +XXX,XX @@ static void cbw_close(BlockDriverState *bs)
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
 
+    bdrv_release_dirty_bitmap(s->access_bitmap);
+    bdrv_release_dirty_bitmap(s->done_bitmap);
+
     block_copy_state_free(s->bcs);
     s->bcs = NULL;
 }
@@ -XXX,XX +XXX,XX @@ BlockDriver bdrv_cbw_filter = {
     .bdrv_co_pdiscard           = cbw_co_pdiscard,
     .bdrv_co_flush              = cbw_co_flush,
 
+    .bdrv_co_preadv_snapshot       = cbw_co_preadv_snapshot,
+    .bdrv_co_pdiscard_snapshot     = cbw_co_pdiscard_snapshot,
+    .bdrv_co_snapshot_block_status = cbw_co_snapshot_block_status,
+
     .bdrv_refresh_filename      = cbw_refresh_filename,
 
     .bdrv_child_perm            = cbw_child_perm,
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
@@ -XXX,XX +XXX,XX @@ write -P0x67 0x3fe0000 0x20000
 {"return": ""}
 {
   "bitmaps": {
+    "backup-top": [
+      {
+        "busy": false,
+        "count": 67108864,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      },
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "persistent": false,
+        "recording": false
+      }
+    ],
     "drive0": [
       {
         "busy": false,
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-14-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/tests/image-fleecing     | 64 +++++++++++++-----
 tests/qemu-iotests/tests/image-fleecing.out | 74 ++++++++++++++++++++-
 2 files changed, 119 insertions(+), 19 deletions(-)

diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/tests/image-fleecing
+++ b/tests/qemu-iotests/tests/image-fleecing
@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000',  '32k'), # Right-end of partial-left [1]
              ('0xdc', '32M',       '32k'), # Left-end of partial-right [2]
              ('0xcd', '0x3ff0000', '64k')] # patterns[3]
 
-def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
+def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
+            fleece_img_path, nbd_sock_path, vm):
     log('--- Setting up images ---')
     log('')
 
     assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
-    assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
+    if use_snapshot_access_filter:
+        assert use_cbw
+        assert qemu_img('create', '-f', 'raw', fleece_img_path, '64M') == 0
+    else:
+        assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
 
     for p in patterns:
         qemu_io('-f', iotests.imgfmt,
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
     log('')
 
 
-    # create tmp_node backed by src_node
-    log(vm.qmp('blockdev-add', {
-        'driver': 'qcow2',
-        'node-name': tmp_node,
-        'file': {
+    if use_snapshot_access_filter:
+        log(vm.qmp('blockdev-add', {
+            'node-name': tmp_node,
             'driver': 'file',
             'filename': fleece_img_path,
-        },
-        'backing': src_node,
-    }))
+        }))
+    else:
+        # create tmp_node backed by src_node
+        log(vm.qmp('blockdev-add', {
+            'driver': 'qcow2',
+            'node-name': tmp_node,
+            'file': {
+                'driver': 'file',
+                'filename': fleece_img_path,
+            },
+            'backing': src_node,
+        }))
 
     # Establish CBW from source to fleecing node
     if use_cbw:
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
         }))
 
         log(vm.qmp('qom-set', path=qom_path, property='drive', value='fl-cbw'))
+
+        if use_snapshot_access_filter:
+            log(vm.qmp('blockdev-add', {
+                'driver': 'snapshot-access',
+                'node-name': 'fl-access',
+                'file': 'fl-cbw',
+            }))
     else:
         log(vm.qmp('blockdev-backup',
                    job_id='fleecing',
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
                    target=tmp_node,
                    sync='none'))
 
+    export_node = 'fl-access' if use_snapshot_access_filter else tmp_node
+
     log('')
     log('--- Setting up NBD Export ---')
     log('')
 
-    nbd_uri = 'nbd+unix:///%s?socket=%s' % (tmp_node, nbd_sock_path)
+    nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
     log(vm.qmp('nbd-server-start',
                {'addr': {'type': 'unix',
                          'data': {'path': nbd_sock_path}}}))
 
-    log(vm.qmp('nbd-server-add', device=tmp_node))
+    log(vm.qmp('nbd-server-add', device=export_node))
 
     log('')
     log('--- Sanity Check ---')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
     log('--- Cleanup ---')
     log('')
 
+    log(vm.qmp('nbd-server-stop'))
+
     if use_cbw:
+        if use_snapshot_access_filter:
+            log(vm.qmp('blockdev-del', node_name='fl-access'))
         log(vm.qmp('qom-set', path=qom_path, property='drive', value=src_node))
         log(vm.qmp('blockdev-del', node_name='fl-cbw'))
     else:
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
         assert e is not None
         log(e, filters=[iotests.filter_qmp_event])
 
-    log(vm.qmp('nbd-server-stop'))
     log(vm.qmp('blockdev-del', node_name=tmp_node))
     vm.shutdown()
 
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
     log('Done')
 
 
-def test(use_cbw):
+def test(use_cbw, use_snapshot_access_filter):
     with iotests.FilePath('base.img') as base_img_path, \
          iotests.FilePath('fleece.img') as fleece_img_path, \
          iotests.FilePath('nbd.sock',
                           base_dir=iotests.sock_dir) as nbd_sock_path, \
          iotests.VM() as vm:
-        do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm)
+        do_test(use_cbw, use_snapshot_access_filter, base_img_path,
+                fleece_img_path, nbd_sock_path, vm)
 
 
 log('=== Test backup(sync=none) based fleecing ===\n')
-test(False)
+test(False, False)
+
+log('=== Test cbw-filter based fleecing ===\n')
+test(True, False)
 
-log('=== Test filter based fleecing ===\n')
-test(True)
+log('=== Test fleecing-format based fleecing ===\n')
+test(True, True)
diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/tests/image-fleecing.out
+++ b/tests/qemu-iotests/tests/image-fleecing.out
@@ -XXX,XX +XXX,XX @@ read -P0 0x3fe0000 64k
 
 --- Cleanup ---
 
+{"return": {}}
 {"return": {}}
 {"data": {"device": "fleecing", "len": 67108864, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
 {"return": {}}
+
+--- Confirming writes ---
+
+read -P0xab 0 64k
+read -P0xad 0x00f8000 64k
+read -P0x1d 0x2008000 64k
+read -P0xea 0x3fe0000 64k
+read -P0xd5 0x108000 32k
+read -P0xdc 32M 32k
+read -P0xcd 0x3ff0000 64k
+
+Done
+=== Test cbw-filter based fleecing ===
+
+--- Setting up images ---
+
+Done
+
+--- Launching VM ---
+
+Done
+
+--- Setting up Fleecing Graph ---
+
+{"return": {}}
+{"return": {}}
+{"return": {}}
+
+--- Setting up NBD Export ---
+
+{"return": {}}
+{"return": {}}
+
+--- Sanity Check ---
+
+read -P0x5d 0 64k
+read -P0xd5 1M 64k
+read -P0xdc 32M 64k
+read -P0xcd 0x3ff0000 64k
+read -P0 0x00f8000 32k
+read -P0 0x2010000 32k
+read -P0 0x3fe0000 64k
+
+--- Testing COW ---
+
+write -P0xab 0 64k
+{"return": ""}
+write -P0xad 0x00f8000 64k
+{"return": ""}
+write -P0x1d 0x2008000 64k
+{"return": ""}
+write -P0xea 0x3fe0000 64k
+{"return": ""}
+
+--- Verifying Data ---
+
+read -P0x5d 0 64k
+read -P0xd5 1M 64k
+read -P0xdc 32M 64k
+read -P0xcd 0x3ff0000 64k
+read -P0 0x00f8000 32k
+read -P0 0x2010000 32k
+read -P0 0x3fe0000 64k
+
+--- Cleanup ---
+
+{"return": {}}
+{"return": {}}
+{"return": {}}
 {"return": {}}
 
 --- Confirming writes ---
@@ -XXX,XX +XXX,XX @@ read -P0xdc 32M 32k
 read -P0xcd 0x3ff0000 64k
 
 Done
-=== Test filter based fleecing ===
+=== Test fleecing-format based fleecing ===
 
 --- Setting up images ---
 
@@ -XXX,XX +XXX,XX @@ Done
 {"return": {}}
 {"return": {}}
 {"return": {}}
+{"return": {}}
 
 --- Setting up NBD Export ---
 
@@ -XXX,XX +XXX,XX @@ read -P0 0x3fe0000 64k
 {"return": {}}
 {"return": {}}
 {"return": {}}
+{"return": {}}
 
 --- Confirming writes ---
 
-- 
2.34.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Note that reads zero areas (not dirty in the bitmap) fails, that's
correct.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220303194349.2304213-16-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/tests/image-fleecing     | 38 +++++++---
 tests/qemu-iotests/tests/image-fleecing.out | 84 +++++++++++++++++++++
 2 files changed, 113 insertions(+), 9 deletions(-)

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20220303194349.2304213-17-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
---
 tests/qemu-iotests/tests/image-fleecing     | 125 +++++++++++++++-----
 tests/qemu-iotests/tests/image-fleecing.out |  63 ++++++++++
 2 files changed, 156 insertions(+), 32 deletions(-)

diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/tests/image-fleecing
+++ b/tests/qemu-iotests/tests/image-fleecing
@@ -XXX,XX +XXX,XX @@ remainder = [('0xd5', '0x108000',  '32k'), # Right-end of partial-left [1]
              ('0xdc', '32M',       '32k'), # Left-end of partial-right [2]
              ('0xcd', '0x3ff0000', '64k')] # patterns[3]
 
-def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
-            fleece_img_path, nbd_sock_path, vm,
+def do_test(vm, use_cbw, use_snapshot_access_filter, base_img_path,
+            fleece_img_path, nbd_sock_path=None,
+            target_img_path=None,
             bitmap=False):
+    push_backup = target_img_path is not None
+    assert (nbd_sock_path is not None) != push_backup
+    if push_backup:
+        assert use_cbw
+
     log('--- Setting up images ---')
     log('')
 
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
     else:
         assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0
 
+    if push_backup:
+        assert qemu_img('create', '-f', 'qcow2', target_img_path, '64M') == 0
+
     for p in patterns:
         qemu_io('-f', iotests.imgfmt,
                 '-c', 'write -P%s %s %s' % p, base_img_path)
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
 
     export_node = 'fl-access' if use_snapshot_access_filter else tmp_node
 
-    log('')
-    log('--- Setting up NBD Export ---')
-    log('')
+    if push_backup:
+        log('')
+        log('--- Starting actual backup ---')
+        log('')
 
-    nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
-    log(vm.qmp('nbd-server-start',
-               {'addr': {'type': 'unix',
-                         'data': {'path': nbd_sock_path}}}))
+        log(vm.qmp('blockdev-add', **{
+            'driver': iotests.imgfmt,
+            'node-name': 'target',
+            'file': {
+                'driver': 'file',
+                'filename': target_img_path
+            }
+        }))
+        log(vm.qmp('blockdev-backup', device=export_node,
+                   sync='full', target='target',
+                   job_id='push-backup', speed=1))
+    else:
+        log('')
+        log('--- Setting up NBD Export ---')
+        log('')
 
-    log(vm.qmp('nbd-server-add', device=export_node))
+        nbd_uri = 'nbd+unix:///%s?socket=%s' % (export_node, nbd_sock_path)
+        log(vm.qmp('nbd-server-start',
+                   {'addr': { 'type': 'unix',
+                              'data': { 'path': nbd_sock_path } } }))
 
-    log('')
-    log('--- Sanity Check ---')
-    log('')
+        log(vm.qmp('nbd-server-add', device=export_node))
 
-    for p in patterns + zeroes:
-        cmd = 'read -P%s %s %s' % p
-        log(cmd)
-        out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
-                                           nbd_uri)
-        if ret != 0:
-            print(out)
+        log('')
+        log('--- Sanity Check ---')
+        log('')
+
+        for p in patterns + zeroes:
+            cmd = 'read -P%s %s %s' % p
+            log(cmd)
+            out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
+                                               nbd_uri)
+            if ret != 0:
+                print(out)
 
     log('')
     log('--- Testing COW ---')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
         log(cmd)
         log(vm.hmp_qemu_io(qom_path, cmd, qdev=True))
 
+    if push_backup:
+        # Check that previous operations were done during backup, not after
+        # If backup is already finished, it's possible that it was finished
+        # even before hmp qemu_io write, and we didn't actually test
+        # copy-before-write operation. This should not happen, as we use
+        # speed=1. But worth checking.
+        result = vm.qmp('query-block-jobs')
+        assert len(result['return']) == 1
+
+        result = vm.qmp('block-job-set-speed', device='push-backup', speed=0)
+        assert result == {'return': {}}
+
+        log(vm.event_wait(name='BLOCK_JOB_COMPLETED',
+                          match={'data': {'device': 'push-backup'}}),
+            filters=[iotests.filter_qmp_event])
+        log(vm.qmp('blockdev-del', node_name='target'))
+
     log('')
     log('--- Verifying Data ---')
     log('')
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
     for p in patterns + zeroes:
         cmd = 'read -P%s %s %s' % p
         log(cmd)
-        out, ret = qemu_io_pipe_and_status('-r', '-f', 'raw', '-c', cmd,
-                                           nbd_uri)
+        args = ['-r', '-c', cmd]
+        if push_backup:
+            args += [target_img_path]
+        else:
+            args += ['-f', 'raw', nbd_uri]
+        out, ret = qemu_io_pipe_and_status(*args)
         if ret != 0:
             print(out)
 
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
     log('--- Cleanup ---')
     log('')
 
-    log(vm.qmp('nbd-server-stop'))
+    if not push_backup:
+        log(vm.qmp('nbd-server-stop'))
 
     if use_cbw:
         if use_snapshot_access_filter:
@@ -XXX,XX +XXX,XX @@ def do_test(use_cbw, use_snapshot_access_filter, base_img_path,
     log('Done')
 
 
-def test(use_cbw, use_snapshot_access_filter, bitmap=False):
+def test(use_cbw, use_snapshot_access_filter,
+         nbd_sock_path=None, target_img_path=None, bitmap=False):
     with iotests.FilePath('base.img') as base_img_path, \
          iotests.FilePath('fleece.img') as fleece_img_path, \
-         iotests.FilePath('nbd.sock',
-                          base_dir=iotests.sock_dir) as nbd_sock_path, \
          iotests.VM() as vm:
-        do_test(use_cbw, use_snapshot_access_filter, base_img_path,
-                fleece_img_path, nbd_sock_path, vm, bitmap=bitmap)
+        do_test(vm, use_cbw, use_snapshot_access_filter, base_img_path,
+                fleece_img_path, nbd_sock_path, target_img_path,
+                bitmap=bitmap)
+
+def test_pull(use_cbw, use_snapshot_access_filter, bitmap=False):
+    with iotests.FilePath('nbd.sock',
+                          base_dir=iotests.sock_dir) as nbd_sock_path:
+        test(use_cbw, use_snapshot_access_filter, nbd_sock_path, None,
+             bitmap=bitmap)
+
+def test_push():
+    with iotests.FilePath('target.img') as target_img_path:
+        test(True, True, None, target_img_path)
 
 
 log('=== Test backup(sync=none) based fleecing ===\n')
-test(False, False)
+test_pull(False, False)
 
 log('=== Test cbw-filter based fleecing ===\n')
-test(True, False)
+test_pull(True, False)
 
 log('=== Test fleecing-format based fleecing ===\n')
-test(True, True)
+test_pull(True, True)
 
 log('=== Test fleecing-format based fleecing with bitmap ===\n')
-test(True, True, bitmap=True)
+test_pull(True, True, bitmap=True)
+
+log('=== Test push backup with fleecing ===\n')
+test_push()
diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/tests/image-fleecing.out
+++ b/tests/qemu-iotests/tests/image-fleecing.out
@@ -XXX,XX +XXX,XX @@ read -P0xdc 32M 32k
 read -P0xcd 0x3ff0000 64k
 
 Done
+=== Test push backup with fleecing ===
+
+--- Setting up images ---
+
+Done
+
+--- Launching VM ---
+
+Done
+
+--- Setting up Fleecing Graph ---
+
+{"return": {}}
+{"return": {}}
+{"return": {}}
+{"return": {}}
+
+--- Starting actual backup ---
+
+{"return": {}}
+{"return": {}}
+
+--- Testing COW ---
+
+write -P0xab 0 64k
+{"return": ""}
+write -P0xad 0x00f8000 64k
+{"return": ""}
+write -P0x1d 0x2008000 64k
+{"return": ""}
+write -P0xea 0x3fe0000 64k
+{"return": ""}
+{"data": {"device": "push-backup", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"return": {}}
+
+--- Verifying Data ---
+
+read -P0x5d 0 64k
+read -P0xd5 1M 64k
+read -P0xdc 32M 64k
+read -P0xcd 0x3ff0000 64k
+read -P0 0x00f8000 32k
+read -P0 0x2010000 32k
+read -P0 0x3fe0000 64k
+
+--- Cleanup ---
+
+{"return": {}}
+{"return": {}}
+{"return": {}}
+{"return": {}}
+
+--- Confirming writes ---
+
+read -P0xab 0 64k
+read -P0xad 0x00f8000 64k
+read -P0x1d 0x2008000 64k
+read -P0xea 0x3fe0000 64k
+read -P0xd5 0x108000 32k
+read -P0xdc 32M 32k
+read -P0xcd 0x3ff0000 64k
+
+Done
-- 
2.34.1