Series comparison

-[Qemu-devel] [PULL 00/16] Block patches
+[Qemu-devel] [PULL v2 0/8] Block patches
-The following changes since commit dd25f97c66a75d1508f1d4c6478ed2c95bec428f:
+The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:
-  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190913' into staging (2019-09-16 10:15:15 +0100)
+  Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)
 are available in the Git repository at:
-  https://github.com/XanClic/qemu.git tags/pull-block-2019-09-16
+  https://github.com/XanClic/qemu.git tags/pull-block-2019-06-24
-for you to fetch changes up to 1825cc0783ccf0ec5d9f0b225a99b340bdd4c68f:
+for you to fetch changes up to ab5d4a30f7f3803ca5106b370969c1b7b54136f8:
-  qemu-iotests: Add test for bz #1745922 (2019-09-16 15:37:12 +0200)
+  iotests: Fix 205 for concurrent runs (2019-06-24 16:01:40 +0200)
 ----------------------------------------------------------------
 Block patches:
-- Fix for block jobs when used with I/O threads
+- The SSH block driver now uses libssh instead of libssh2
-- Fix for a corruption when using qcow2's LUKS encryption mode
+- The VMDK block driver gets read-only support for the seSparse
-- cURL fix
+  subformat
-- check-block.sh cleanups (for make check)
+- Various fixes
-- Refactoring
 ---
 v2:
 - Squashed Pino's fix for pre-0.8 libssh into the libssh patch
 ----------------------------------------------------------------
-Max Reitz (7):
+Anton Nefedov (1):
-  curl: Keep pointer to the CURLState in CURLSocket
+  iotest 134: test cluster-misaligned encrypted write
   curl: Keep *socket until the end of curl_sock_cb()
   curl: Check completion in curl_multi_do()
   curl: Pass CURLSocket to curl_multi_do()
   curl: Report only ready sockets
   curl: Handle success in multi_check_completion
   curl: Check curl_multi_add_handle()'s return code
-Maxim Levitsky (3):
+Klaus Birkelund Jensen (1):
-  block/qcow2: Fix corruption introduced by commit 8ac0f15f335
+  nvme: do not advertise support for unsupported arbitration mechanism
   block/qcow2: refactor encryption code
   qemu-iotests: Add test for bz #1745922
-Nir Soffer (2):
+Max Reitz (1):
-  block: Use QEMU_IS_ALIGNED
+  iotests: Fix 205 for concurrent runs
   block: Remove unused masks
-Sergio Lopez (1):
+Pino Toscano (1):
-  blockjob: update nodes head while removing all bdrv
+  ssh: switch from libssh2 to libssh
-Thomas Huth (2):
+Sam Eiderman (3):
-  tests/qemu-iotests/check: Replace "tests" with "iotests" in final
+  vmdk: Fix comment regarding max l1_size coverage
-    status text
+  vmdk: Reduce the max bound for L1 table size
-  tests/Makefile: Do not print the name of the check-block.sh shell
+  vmdk: Add read-only support for seSparse snapshots
     script
 Vladimir Sementsov-Ogievskiy (1):
-  tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache
+  blockdev: enable non-root nodes for transaction drive-backup source
- tests/Makefile.include             |   2 +-
+ configure                                     |  65 +-
- block/qcow2.h                      |   8 +-
+ block/Makefile.objs                           |   6 +-
- include/block/block.h              |   2 -
+ block/ssh.c                                   | 652 ++++++++++--------
- block/bochs.c                      |   4 +-
+ block/vmdk.c                                  | 372 +++++++++-
- block/cloop.c                      |   4 +-
+ blockdev.c                                    |   2 +-
- block/curl.c                       | 133 ++++++++++-------------
+ hw/block/nvme.c                               |   1 -
- block/dmg.c                        |   4 +-
+ .travis.yml                                   |   4 +-
- block/io.c                         |   8 +-
+ block/trace-events                            |  14 +-
- block/qcow2-cluster.c              |  40 +++----
+ docs/qemu-block-drivers.texi                  |   2 +-
- block/qcow2-threads.c              |  63 ++++++++---
+ .../dockerfiles/debian-win32-cross.docker     |   1 -
- block/qcow2.c                      |   9 +-
+ .../dockerfiles/debian-win64-cross.docker     |   1 -
- block/vvfat.c                      |   8 +-
+ tests/docker/dockerfiles/fedora.docker        |   4 +-
- blockjob.c                         |  17 ++-
+ tests/docker/dockerfiles/ubuntu.docker        |   2 +-
- migration/block.c                  |   2 +-
+ tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
- qemu-img.c                         |   2 +-
+ tests/qemu-iotests/059.out                    |   2 +-
- tests/qemu-iotests/026.out.nocache | 168 ++++++++++++++---------------
+ tests/qemu-iotests/134                        |   9 +
- tests/qemu-iotests/263             |  91 ++++++++++++++++
+ tests/qemu-iotests/134.out                    |  10 +
- tests/qemu-iotests/263.out         |  40 +++++++
+ tests/qemu-iotests/205                        |   2 +-
- tests/qemu-iotests/check           |   8 +-
+ tests/qemu-iotests/207                        |  54 +-
- tests/qemu-iotests/group           |   1 +
+ tests/qemu-iotests/207.out                    |   2 +-
-files changed, 380 insertions(+), 234 deletions(-)
+files changed, 823 insertions(+), 384 deletions(-)
  create mode 100755 tests/qemu-iotests/263
  create mode 100644 tests/qemu-iotests/263.out
 --
 .21.0

-[Qemu-devel] [PULL 01/16] block: Use QEMU_IS_ALIGNED
+Deleted patch
-From: Nir Soffer <nirsof@gmail.com>
-Replace instances of:
-    (n & (BDRV_SECTOR_SIZE - 1)) == 0
-And:
-   (n & ~BDRV_SECTOR_MASK) == 0
-With:
-    QEMU_IS_ALIGNED(n, BDRV_SECTOR_SIZE)
-Which reveals the intent of the code better, and makes it easier to
-locate the code checking alignment.
-Signed-off-by: Nir Soffer <nsoffer@redhat.com>
-Message-id: 20190827185913.27427-2-nsoffer@redhat.com
-Reviewed-by: John Snow <jsnow@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- block/bochs.c         | 4 ++--
- block/cloop.c         | 4 ++--
- block/dmg.c           | 4 ++--
- block/io.c            | 8 ++++----
- block/qcow2-cluster.c | 4 ++--
- block/qcow2.c         | 4 ++--
- block/vvfat.c         | 8 ++++----
- qemu-img.c            | 2 +-
-files changed, 19 insertions(+), 19 deletions(-)
-diff --git a/block/bochs.c b/block/bochs.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/bochs.c
-+++ b/block/bochs.c
-@@ -XXX,XX +XXX,XX @@ bochs_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
-     QEMUIOVector local_qiov;
-     int ret;
--    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-     qemu_iovec_init(&local_qiov, qiov->niov);
-     qemu_co_mutex_lock(&s->lock);
-diff --git a/block/cloop.c b/block/cloop.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/cloop.c
-+++ b/block/cloop.c
-@@ -XXX,XX +XXX,XX @@ cloop_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
-     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
-     int ret, i;
--    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-     qemu_co_mutex_lock(&s->lock);
-diff --git a/block/dmg.c b/block/dmg.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/dmg.c
-+++ b/block/dmg.c
-@@ -XXX,XX +XXX,XX @@ dmg_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
-     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
-     int ret, i;
--    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-     qemu_co_mutex_lock(&s->lock);
-diff --git a/block/io.c b/block/io.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/io.c
-+++ b/block/io.c
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_driver_preadv(BlockDriverState *bs,
-     sector_num = offset >> BDRV_SECTOR_BITS;
-     nb_sectors = bytes >> BDRV_SECTOR_BITS;
--    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-     assert(bytes <= BDRV_REQUEST_MAX_BYTES);
-     assert(drv->bdrv_co_readv);
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_driver_pwritev(BlockDriverState *bs,
-     sector_num = offset >> BDRV_SECTOR_BITS;
-     nb_sectors = bytes >> BDRV_SECTOR_BITS;
--    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-     assert(bytes <= BDRV_REQUEST_MAX_BYTES);
-     assert(drv->bdrv_co_writev);
-diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2-cluster.c
-+++ b/block/qcow2-cluster.c
-@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
- {
-     if (bytes && bs->encrypted) {
-         BDRVQcow2State *s = bs->opaque;
--        assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
--        assert((bytes & ~BDRV_SECTOR_MASK) == 0);
-+        assert(QEMU_IS_ALIGNED(offset_in_cluster, BDRV_SECTOR_SIZE));
-+        assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-         assert(s->crypto);
-         if (qcow2_co_encrypt(bs, cluster_offset,
-                              src_cluster_offset + offset_in_cluster,
-diff --git a/block/qcow2.c b/block/qcow2.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2.c
-+++ b/block/qcow2.c
-@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
-                     goto fail;
-                 }
--                assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--                assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+                assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+                assert(QEMU_IS_ALIGNED(cur_bytes, BDRV_SECTOR_SIZE));
-                 if (qcow2_co_decrypt(bs, cluster_offset, offset,
-                                      cluster_data, cur_bytes) < 0) {
-                     ret = -EIO;
-diff --git a/block/vvfat.c b/block/vvfat.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/vvfat.c
-+++ b/block/vvfat.c
-@@ -XXX,XX +XXX,XX @@ vvfat_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
-     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
-     void *buf;
--    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-     buf = g_try_malloc(bytes);
-     if (bytes && buf == NULL) {
-@@ -XXX,XX +XXX,XX @@ vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
-     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
-     void *buf;
--    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
--    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
-+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-     buf = g_try_malloc(bytes);
-     if (bytes && buf == NULL) {
-diff --git a/qemu-img.c b/qemu-img.c
-index XXXXXXX..XXXXXXX 100644
---- a/qemu-img.c
-+++ b/qemu-img.c
-@@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv)
-             int64_t sval;
-             sval = cvtnum(optarg);
--            if (sval < 0 || sval & (BDRV_SECTOR_SIZE - 1) ||
-+            if (sval < 0 || !QEMU_IS_ALIGNED(sval, BDRV_SECTOR_SIZE) ||
-                 sval / BDRV_SECTOR_SIZE > MAX_BUF_SECTORS) {
-                 error_report("Invalid buffer size for sparse output specified. "
-                     "Valid sizes are multiples of %llu up to %llu. Select "
---
-.21.0

-[Qemu-devel] [PULL 02/16] block: Remove unused masks
+Deleted patch
-From: Nir Soffer <nirsof@gmail.com>
-Replace confusing usage:
-    ~BDRV_SECTOR_MASK
-With more clear:
-    (BDRV_SECTOR_SIZE - 1)
-Remove BDRV_SECTOR_MASK and the unused BDRV_BLOCK_OFFSET_MASK which was
-it's last user.
-Signed-off-by: Nir Soffer <nsoffer@redhat.com>
-Message-id: 20190827185913.27427-3-nsoffer@redhat.com
-Reviewed-by: Juan Quintela <quintela@redhat.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- include/block/block.h | 2 --
- migration/block.c     | 2 +-
-files changed, 1 insertion(+), 3 deletions(-)
-diff --git a/include/block/block.h b/include/block/block.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/block/block.h
-+++ b/include/block/block.h
-@@ -XXX,XX +XXX,XX @@ typedef struct HDGeometry {
- #define BDRV_SECTOR_BITS   9
- #define BDRV_SECTOR_SIZE   (1ULL << BDRV_SECTOR_BITS)
--#define BDRV_SECTOR_MASK   ~(BDRV_SECTOR_SIZE - 1)
- #define BDRV_REQUEST_MAX_SECTORS MIN(SIZE_MAX >> BDRV_SECTOR_BITS, \
-                                      INT_MAX >> BDRV_SECTOR_BITS)
-@@ -XXX,XX +XXX,XX @@ typedef struct HDGeometry {
- #define BDRV_BLOCK_ALLOCATED    0x10
- #define BDRV_BLOCK_EOF          0x20
- #define BDRV_BLOCK_RECURSE      0x40
--#define BDRV_BLOCK_OFFSET_MASK  BDRV_SECTOR_MASK
- typedef QSIMPLEQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopenQueue;
-diff --git a/migration/block.c b/migration/block.c
-index XXXXXXX..XXXXXXX 100644
---- a/migration/block.c
-+++ b/migration/block.c
-@@ -XXX,XX +XXX,XX @@ static int block_load(QEMUFile *f, void *opaque, int version_id)
-     do {
-         addr = qemu_get_be64(f);
--        flags = addr & ~BDRV_SECTOR_MASK;
-+        flags = addr & (BDRV_SECTOR_SIZE - 1);
-         addr >>= BDRV_SECTOR_BITS;
-         if (flags & BLK_MIG_FLAG_DEVICE_BLOCK) {
---
-.21.0

-[Qemu-devel] [PULL 03/16] tests/qemu-iotests/check: Replace "tests" with "iotests" in final status text
+Deleted patch
-From: Thomas Huth <thuth@redhat.com>
-When running "make check -j8" or something similar, the iotests are
-running in parallel with the other tests. So when they are printing
-out "Passed all xx tests" or a similar status message at the end,
-it might not be quite clear that this message belongs to the iotests,
-since the output might be mixed with the other tests. Thus change the
-word "tests" here to "iotests" instead to avoid confusion.
-Signed-off-by: Thomas Huth <thuth@redhat.com>
-Message-id: 20190906113920.11271-1-thuth@redhat.com
-Reviewed-by: John Snow <jsnow@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- tests/qemu-iotests/check | 8 ++++----
-file changed, 4 insertions(+), 4 deletions(-)
-diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
-index XXXXXXX..XXXXXXX 100755
---- a/tests/qemu-iotests/check
-+++ b/tests/qemu-iotests/check
-@@ -XXX,XX +XXX,XX @@ END        { if (NR > 0) {
-         if [ ! -z "$n_bad" -a $n_bad != 0 ]
-         then
-             echo "Failures:$bad"
--            echo "Failed $n_bad of $try tests"
-+            echo "Failed $n_bad of $try iotests"
-             echo "Failures:$bad" | fmt >>check.log
--            echo "Failed $n_bad of $try tests" >>check.log
-+            echo "Failed $n_bad of $try iotests" >>check.log
-         else
--            echo "Passed all $try tests"
--            echo "Passed all $try tests" >>check.log
-+            echo "Passed all $try iotests"
-+            echo "Passed all $try iotests" >>check.log
-         fi
-         needwrap=false
-     fi
---
-.21.0

-[Qemu-devel] [PULL 13/16] blockjob: update nodes head while removing all bdrv
+[Qemu-devel] [PULL v2 1/8] nvme: do not advertise support for unsupported arbitration mechanism
-From: Sergio Lopez <slp@redhat.com>
+From: Klaus Birkelund Jensen <klaus@birkelund.eu>
-block_job_remove_all_bdrv() iterates through job->nodes, calling
+The device mistakenly reports that the Weighted Round Robin with Urgent
-bdrv_root_unref_child() for each entry. The call to the latter may
+Priority Class arbitration mechanism is supported.
 reach child_job_[can_]set_aio_ctx(), which will also attempt to
 traverse job->nodes, potentially finding entries that where freed
 on previous iterations.
-To avoid this situation, update job->nodes head on each iteration to
+It is not.
 ensure that already freed entries are no longer linked to the list.
-RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
+Signed-off-by: Klaus Birkelund Jensen <klaus.jensen@cnexlabs.com>
-Signed-off-by: Sergio Lopez <slp@redhat.com>
+Message-id: 20190606092530.14206-1-klaus@birkelund.eu
-Cc: qemu-stable@nongnu.org
+Acked-by: Maxim Levitsky <mlevitsk@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 Message-id: 20190911100316.32282-1-mreitz@redhat.com
 Reviewed-by: Sergio Lopez <slp@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- blockjob.c | 17 +++++++++++++----
+ hw/block/nvme.c | 1 -
-file changed, 13 insertions(+), 4 deletions(-)
+file changed, 1 deletion(-)
-diff --git a/blockjob.c b/blockjob.c
+diff --git a/hw/block/nvme.c b/hw/block/nvme.c
 index XXXXXXX..XXXXXXX 100644
---- a/blockjob.c
+--- a/hw/block/nvme.c
-+++ b/blockjob.c
++++ b/hw/block/nvme.c
-@@ -XXX,XX +XXX,XX @@ static const BdrvChildRole child_job = {
+@@ -XXX,XX +XXX,XX @@ static void nvme_realize(PCIDevice *pci_dev, Error **errp)
+     n->bar.cap = 0;
- void block_job_remove_all_bdrv(BlockJob *job)
+     NVME_CAP_SET_MQES(n->bar.cap, 0x7ff);
- {
+     NVME_CAP_SET_CQR(n->bar.cap, 1);
--    GSList *l;
+-    NVME_CAP_SET_AMS(n->bar.cap, 1);
--    for (l = job->nodes; l; l = l->next) {
+     NVME_CAP_SET_TO(n->bar.cap, 0xf);
-+    /*
+     NVME_CAP_SET_CSS(n->bar.cap, 1);
-+     * bdrv_root_unref_child() may reach child_job_[can_]set_aio_ctx(),
+     NVME_CAP_SET_MPSMAX(n->bar.cap, 4);
 +     * which will also traverse job->nodes, so consume the list one by
 +     * one to make sure that such a concurrent access does not attempt
 +     * to process an already freed BdrvChild.
 +     */
 +    while (job->nodes) {
 +        GSList *l = job->nodes;
          BdrvChild *c = l->data;
 +
 +        job->nodes = l->next;
 +
          bdrv_op_unblock_all(c->bs, job->blocker);
          bdrv_root_unref_child(c);
 +
 +        g_slist_free_1(l);
      }
 -    g_slist_free(job->nodes);
 -    job->nodes = NULL;
  }
  bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs)
 --
 .21.0

-[Qemu-devel] [PULL 05/16] tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache
+[Qemu-devel] [PULL v2 2/8] blockdev: enable non-root nodes for transaction drive-backup source
 From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-qemu-io now prefixes its error and warnings with "qemu-io:".
+We forget to enable it for transaction .prepare, while it is already
-b9986b08787019e fixed a lot of iotests output but forget about
+enabled in do_drive_backup since commit a2d665c1bc362
-.out.nocache. Fix it too.
+    "blockdev: loosen restrictions on drive-backup source node"
-Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()")
 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-id: 20190816153015.447957-2-vsementsov@virtuozzo.com
+Message-id: 20190618140804.59214-1-vsementsov@virtuozzo.com
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/026.out.nocache | 168 ++++++++++++++---------------
+ blockdev.c | 2 +-
-file changed, 84 insertions(+), 84 deletions(-)
+file changed, 1 insertion(+), 1 deletion(-)
-diff --git a/tests/qemu-iotests/026.out.nocache b/tests/qemu-iotests/026.out.nocache
+diff --git a/blockdev.c b/blockdev.c
 index XXXXXXX..XXXXXXX 100644
---- a/tests/qemu-iotests/026.out.nocache
+--- a/blockdev.c
-+++ b/tests/qemu-iotests/026.out.nocache
++++ b/blockdev.c
-@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
+@@ -XXX,XX +XXX,XX @@ static void drive_backup_prepare(BlkActionState *common, Error **errp)
- Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
+     assert(common->action->type == TRANSACTION_ACTION_KIND_DRIVE_BACKUP);
+     backup = common->action->u.drive_backup.data;
- Event: l1_update; errno: 5; imm: off; once: off; write
--Failed to flush the L2 table cache: Input/output error
+-    bs = qmp_get_root_bs(backup->device, errp);
--Failed to flush the refcount block cache: Input/output error
++    bs = bdrv_lookup_bs(backup->device, backup->device, errp);
-+qemu-io: Failed to flush the L2 table cache: Input/output error
+     if (!bs) {
-+qemu-io: Failed to flush the refcount block cache: Input/output error
+         return;
- write failed: Input/output error
+     }
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l1_update; errno: 5; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l1_update; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l1_update; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_update; errno: 5; imm: off; once: off; write
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  wrote 131072/131072 bytes at offset 0
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_update; errno: 5; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  wrote 131072/131072 bytes at offset 0
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_update; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  wrote 131072/131072 bytes at offset 0
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_update; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  wrote 131072/131072 bytes at offset 0
 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_alloc_write; errno: 5; imm: off; once: off; write
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_alloc_write; errno: 5; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_alloc_write; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l2_alloc_write; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: write_aio; errno: 5; imm: off; once: off; write
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: write_aio; errno: 5; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: write_aio; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: write_aio; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_load; errno: 5; imm: off; once: off; write
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_load; errno: 5; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_load; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_load; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_update_part; errno: 5; imm: off; once: off; write
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_update_part; errno: 5; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_update_part; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_update_part; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc; errno: 5; imm: off; once: off; write
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc; errno: 5; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_write; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write -b
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l1_grow_write_table; errno: 5; imm: off; once: off
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l1_grow_write_table; errno: 28; imm: off; once: off
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
  No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l1_grow_activate_table; errno: 5; imm: off; once: off
 -Failed to flush the L2 table cache: Input/output error
 -Failed to flush the refcount block cache: Input/output error
 +qemu-io: Failed to flush the L2 table cache: Input/output error
 +qemu-io: Failed to flush the refcount block cache: Input/output error
  write failed: Input/output error
 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
  Event: l1_grow_activate_table; errno: 28; imm: off; once: off
 -Failed to flush the L2 table cache: No space left on device
 -Failed to flush the refcount block cache: No space left on device
 +qemu-io: Failed to flush the L2 table cache: No space left on device
 +qemu-io: Failed to flush the refcount block cache: No space left on device
  write failed: No space left on device
 leaked clusters were found on the image.
 --
 .21.0

-[Qemu-devel] [PULL 16/16] qemu-iotests: Add test for bz #1745922
+[Qemu-devel] [PULL v2 3/8] iotest 134: test cluster-misaligned encrypted write
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Anton Nefedov <anton.nefedov@virtuozzo.com>
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+COW (even empty/zero) areas require encryption too
-Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Message-id: 20190915203655.21638-4-mlevitsk@redhat.com
+Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
 Reviewed-by: Eric Blake <eblake@redhat.com>
 Reviewed-by: Max Reitz <mreitz@redhat.com>
+Reviewed-by: Alberto Garcia <berto@igalia.com>
+Message-id: 20190516143028.81155-1-anton.nefedov@virtuozzo.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/qemu-iotests/263     | 91 ++++++++++++++++++++++++++++++++++++++
+ tests/qemu-iotests/134     |  9 +++++++++
- tests/qemu-iotests/263.out | 40 +++++++++++++++++
+ tests/qemu-iotests/134.out | 10 ++++++++++
- tests/qemu-iotests/group   |  1 +
+files changed, 19 insertions(+)
 files changed, 132 insertions(+)
  create mode 100755 tests/qemu-iotests/263
  create mode 100644 tests/qemu-iotests/263.out
-diff --git a/tests/qemu-iotests/263 b/tests/qemu-iotests/263
+diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134
-new file mode 100755
+index XXXXXXX..XXXXXXX 100755
-index XXXXXXX..XXXXXXX
+--- a/tests/qemu-iotests/134
---- /dev/null
++++ b/tests/qemu-iotests/134
-+++ b/tests/qemu-iotests/263
+@@ -XXX,XX +XXX,XX @@ echo
-@@ -XXX,XX +XXX,XX @@
+ echo "== reading whole image =="
-+#!/usr/bin/env bash
+ $QEMU_IO --object $SECRET -c "read 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
-+#
-+# Test encrypted write that crosses cluster boundary of two unallocated clusters
++echo
-+# Based on 188
++echo "== rewriting cluster part =="
-+#
++$QEMU_IO --object $SECRET -c "write -P 0xb 512 512" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
 +# Copyright (C) 2019 Red Hat, Inc.
 +#
 +# This program is free software; you can redistribute it and/or modify
 +# it under the terms of the GNU General Public License as published by
 +# the Free Software Foundation; either version 2 of the License, or
 +# (at your option) any later version.
 +#
 +# This program is distributed in the hope that it will be useful,
 +# but WITHOUT ANY WARRANTY; without even the implied warranty of
 +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 +# GNU General Public License for more details.
 +#
 +# You should have received a copy of the GNU General Public License
 +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
 +#
 +
 +# creator
 +owner=mlevitsk@redhat.com
 +
 +seq=`basename $0`
 +echo "QA output created by $seq"
 +
 +status=1    # failure is the default!
 +
 +_cleanup()
 +{
 +    _cleanup_test_img
 +}
 +trap "_cleanup; exit \$status" 0 1 2 3 15
 +
 +# get standard environment, filters and checks
 +. ./common.rc
 +. ./common.filter
 +
 +_supported_fmt qcow2
 +_supported_proto generic
 +_supported_os Linux
 +
 +
 +size=1M
 +
 +SECRET="secret,id=sec0,data=astrochicken"
 +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
 +
 +
 +_run_test()
 +{
 +    echo "== reading the whole image =="
 +    $QEMU_IO --object $SECRET -c "read -P 0 0 $size" --image-opts "$1" | _filter_qemu_io | _filter_testdir
 +
 +    echo
 +    echo "== write two 512 byte sectors on a cluster boundary =="
 +    $QEMU_IO --object $SECRET -c "write -P 0xAA 0xFE00 0x400" --image-opts "$1" | _filter_qemu_io | _filter_testdir
 +
 +    echo
 +    echo "== verify that the rest of the image is not changed =="
 +    $QEMU_IO --object $SECRET -c "read -P 0x00 0x00000 0xFE00" --image-opts "$1" | _filter_qemu_io | _filter_testdir
 +    $QEMU_IO --object $SECRET -c "read -P 0xAA 0x0FE00 0x400" --image-opts "$1" | _filter_qemu_io | _filter_testdir
 +    $QEMU_IO --object $SECRET -c "read -P 0x00 0x10200 0xEFE00" --image-opts "$1" | _filter_qemu_io | _filter_testdir
 +
 +}
 +
 +
 +echo
-+echo "testing LUKS qcow2 encryption"
++echo "== verify pattern =="
-+echo
++$QEMU_IO --object $SECRET -c "read -P 0 0 512"  --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
 +$QEMU_IO --object $SECRET -c "read -P 0xb 512 512"  --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
 +
-+_make_test_img --object $SECRET -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10,cluster_size=64K" $size
+ echo
-+_run_test "driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
+ echo "== rewriting whole image =="
-+_cleanup_test_img
+ $QEMU_IO --object $SECRET -c "write -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
 diff --git a/tests/qemu-iotests/134.out b/tests/qemu-iotests/134.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/134.out
 +++ b/tests/qemu-iotests/134.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.
  read 134217728/134217728 bytes at offset 0
 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +== rewriting cluster part ==
 +wrote 512/512 bytes at offset 512
 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
-+echo
++== verify pattern ==
-+echo "testing legacy AES qcow2 encryption"
++read 512/512 bytes at offset 0
-+echo
++512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 512/512 bytes at offset 512
 +512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
-+
+ == rewriting whole image ==
-+_make_test_img --object $SECRET -o "encrypt.format=aes,encrypt.key-secret=sec0,cluster_size=64K" $size
+ wrote 134217728/134217728 bytes at offset 0
-+_run_test "driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
+MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +_cleanup_test_img
 +
 +
 +
 +# success, all done
 +echo "*** done"
 +rm -f $seq.full
 +status=0
 diff --git a/tests/qemu-iotests/263.out b/tests/qemu-iotests/263.out
 new file mode 100644
 index XXXXXXX..XXXXXXX
 --- /dev/null
 +++ b/tests/qemu-iotests/263.out
@@ -XXX,XX +XXX,XX @@
 +QA output created by 263
 +
 +testing LUKS qcow2 encryption
 +
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
 +== reading the whole image ==
 +read 1048576/1048576 bytes at offset 0
 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== write two 512 byte sectors on a cluster boundary ==
 +wrote 1024/1024 bytes at offset 65024
 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== verify that the rest of the image is not changed ==
 +read 65024/65024 bytes at offset 0
 +63.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 1024/1024 bytes at offset 65024
 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 982528/982528 bytes at offset 66048
 +959.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +testing legacy AES qcow2 encryption
 +
 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=aes encrypt.key-secret=sec0
 +== reading the whole image ==
 +read 1048576/1048576 bytes at offset 0
 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== write two 512 byte sectors on a cluster boundary ==
 +wrote 1024/1024 bytes at offset 65024
 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +
 +== verify that the rest of the image is not changed ==
 +read 65024/65024 bytes at offset 0
 +63.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 1024/1024 bytes at offset 65024
 +1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +read 982528/982528 bytes at offset 66048
 +959.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 +*** done
 diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/group
 +++ b/tests/qemu-iotests/group
@@ -XXX,XX +XXX,XX @@
 rw
 rw quick
 rw quick migration
 +263 rw quick
 rw auto quick
 rw quick
 --
 .21.0

-[Qemu-devel] [PULL 14/16] block/qcow2: Fix corruption introduced by commit 8ac0f15f335
+[Qemu-devel] [PULL v2 4/8] vmdk: Fix comment regarding max l1_size coverage
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Sam Eiderman <shmuel.eiderman@oracle.com>
-This fixes subtle corruption introduced by luks threaded encryption
+Commit b0651b8c246d ("vmdk: Move l1_size check into vmdk_add_extent")
-in commit 8ac0f15f335
+extended the l1_size check from VMDK4 to VMDK3 but did not update the
 default coverage in the moved comment.
-Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1745922
+The previous vmdk4 calculation:
-The corruption happens when we do a write that
+    (512 * 1024 * 1024) * 512(l2 entries) * 65536(grain) = 16PB
    * writes to two or more unallocated clusters at once
    * doesn't fully cover the first sector
    * doesn't fully cover the last sector
    * uses luks encryption
-In this case, when allocating the new clusters we COW both areas
+The added vmdk3 calculation:
 prior to the write and after the write, and we encrypt them.
-The above mentioned commit accidentally made it so we encrypt the
+    (512 * 1024 * 1024) * 4096(l2 entries) * 512(grain) = 1PB
 second COW area using the physical cluster offset of the first area.
-The problem is that offset_in_cluster in do_perform_cow_encrypt
+Adding the calculation of vmdk3 to the comment.
 can be larger that the cluster size, thus cluster_offset
 will no longer point to the start of the cluster at which encrypted
 area starts.
-Next patch in this series will refactor the code to avoid all these
+In any case, VMware does not offer virtual disks more than 2TB for
-assumptions.
+vmdk4/vmdk3 or 64TB for the new undocumented seSparse format which is
 not implemented yet in qemu.
-In the bugreport that was triggered by rebasing a luks image to new,
+Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
-zero filled base, which lot of such writes, and causes some files
+Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
-with zero areas to contain garbage there instead.
+Reviewed-by: Liran Alon <liran.alon@oracle.com>
-But as described above it can happen elsewhere as well
+Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
+Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+Message-id: 20190620091057.47441-2-shmuel.eiderman@oracle.com
-Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Reviewed-by: yuchenlin <yuchenlin@synology.com>
 Message-id: 20190915203655.21638-2-mlevitsk@redhat.com
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/qcow2-cluster.c | 7 ++++---
+ block/vmdk.c | 11 ++++++++---
-file changed, 4 insertions(+), 3 deletions(-)
+file changed, 8 insertions(+), 3 deletions(-)
-diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
+diff --git a/block/vmdk.c b/block/vmdk.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2-cluster.c
+--- a/block/vmdk.c
-+++ b/block/qcow2-cluster.c
++++ b/block/vmdk.c
-@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
+@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
-         assert(QEMU_IS_ALIGNED(offset_in_cluster, BDRV_SECTOR_SIZE));
+         return -EFBIG;
-         assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
+     }
-         assert(s->crypto);
+     if (l1_size > 512 * 1024 * 1024) {
--        if (qcow2_co_encrypt(bs, cluster_offset,
+-        /* Although with big capacity and small l1_entry_sectors, we can get a
--                             src_cluster_offset + offset_in_cluster,
++        /*
--                             buffer, bytes) < 0) {
++         * Although with big capacity and small l1_entry_sectors, we can get a
-+        if (qcow2_co_encrypt(bs,
+          * big l1_size, we don't want unbounded value to allocate the table.
-+                start_of_cluster(s, cluster_offset + offset_in_cluster),
+-         * Limit it to 512M, which is 16PB for default cluster and L2 table
-+                src_cluster_offset + offset_in_cluster,
+-         * size */
-+                buffer, bytes) < 0) {
++         * Limit it to 512M, which is:
-             return false;
++         *     16PB - for default "Hosted Sparse Extent" (VMDK4)
-         }
++         *            cluster size: 64KB, L2 table size: 512 entries
 +         *     1PB  - for default "ESXi Host Sparse Extent" (VMDK3/vmfsSparse)
 +         *            cluster size: 512B, L2 table size: 4096 entries
 +         */
          error_setg(errp, "L1 size too big");
          return -EFBIG;
      }
 --
 .21.0

-[Qemu-devel] [PULL 04/16] tests/Makefile: Do not print the name of the check-block.sh shell script
+[Qemu-devel] [PULL v2 5/8] vmdk: Reduce the max bound for L1 table size
-From: Thomas Huth <thuth@redhat.com>
+From: Sam Eiderman <shmuel.eiderman@oracle.com>
-The check script is already printing out which iotest is currently
+M of L1 entries is a very loose bound, only 32M are required to store
-running, so printing out the name of the check-block.sh shell script
+the maximal supported VMDK file size of 2TB.
 looks superfluous here.
-Signed-off-by: Thomas Huth <thuth@redhat.com>
+Fixed qemu-iotest 59# - now failure occures before on impossible L1
-Message-id: 20190906113534.10907-1-thuth@redhat.com
+table size.
-Acked-by: John Snow <jsnow@redhat.com>
 Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
 Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
 Reviewed-by: Liran Alon <liran.alon@oracle.com>
 Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
 Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
 Message-id: 20190620091057.47441-3-shmuel.eiderman@oracle.com
 Reviewed-by: Max Reitz <mreitz@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- tests/Makefile.include | 2 +-
+ block/vmdk.c               | 13 +++++++------
-file changed, 1 insertion(+), 1 deletion(-)
+ tests/qemu-iotests/059.out |  2 +-
 files changed, 8 insertions(+), 7 deletions(-)
-diff --git a/tests/Makefile.include b/tests/Makefile.include
+diff --git a/block/vmdk.c b/block/vmdk.c
 index XXXXXXX..XXXXXXX 100644
---- a/tests/Makefile.include
+--- a/block/vmdk.c
-+++ b/tests/Makefile.include
++++ b/block/vmdk.c
-@@ -XXX,XX +XXX,XX @@ QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu
+@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
- check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) \
+         error_setg(errp, "Invalid granularity, image may be corrupt");
-         qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \
+         return -EFBIG;
-         $(patsubst %,%/all,$(filter %-softmmu,$(TARGET_DIRS)))
+     }
--    $<
+-    if (l1_size > 512 * 1024 * 1024) {
-+    @$<
++    if (l1_size > 32 * 1024 * 1024) {
+         /*
- .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
+          * Although with big capacity and small l1_entry_sectors, we can get a
- $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json
+          * big l1_size, we don't want unbounded value to allocate the table.
 -         * Limit it to 512M, which is:
 -         *     16PB - for default "Hosted Sparse Extent" (VMDK4)
 -         *            cluster size: 64KB, L2 table size: 512 entries
 -         *     1PB  - for default "ESXi Host Sparse Extent" (VMDK3/vmfsSparse)
 -         *            cluster size: 512B, L2 table size: 4096 entries
 +         * Limit it to 32M, which is enough to store:
 +         *     8TB  - for both VMDK3 & VMDK4 with
 +         *            minimal cluster size: 512B
 +         *            minimal L2 table size: 512 entries
 +         *            8 TB is still more than the maximal value supported for
 +         *            VMDK3 & VMDK4 which is 2TB.
           */
          error_setg(errp, "L1 size too big");
          return -EFBIG;
 diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/059.out
 +++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
 x140000000     0x10000         0x50000         TEST_DIR/t-s003.vmdk
  === Testing afl image with a very large capacity ===
 -qemu-img: Can't get image size 'TEST_DIR/afl9.IMGFMT': File too large
 +qemu-img: Could not open 'TEST_DIR/afl9.IMGFMT': L1 size too big
  *** done
 --
 .21.0

-[Qemu-devel] [PULL 06/16] curl: Keep pointer to the CURLState in CURLSocket
+Deleted patch
-A follow-up patch will make curl_multi_do() and curl_multi_read() take a
-CURLSocket instead of the CURLState.  They still need the latter,
-though, so add a pointer to it to the former.
-Cc: qemu-stable@nongnu.org
-Signed-off-by: Max Reitz <mreitz@redhat.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
-Message-id: 20190910124136.10565-2-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- block/curl.c | 3 +++
-file changed, 3 insertions(+)
-diff --git a/block/curl.c b/block/curl.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
-+++ b/block/curl.c
-@@ -XXX,XX +XXX,XX @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
- #define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5
- struct BDRVCURLState;
-+struct CURLState;
- static bool libcurl_initialized;
-@@ -XXX,XX +XXX,XX @@ typedef struct CURLAIOCB {
- typedef struct CURLSocket {
-     int fd;
-+    struct CURLState *state;
-     QLIST_ENTRY(CURLSocket) next;
- } CURLSocket;
-@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
-     if (!socket) {
-         socket = g_new0(CURLSocket, 1);
-         socket->fd = fd;
-+        socket->state = state;
-         QLIST_INSERT_HEAD(&state->sockets, socket, next);
-     }
-     socket = NULL;
---
-.21.0

-[Qemu-devel] [PULL 07/16] curl: Keep *socket until the end of curl_sock_cb()
+Deleted patch
-This does not really change anything, but it makes the code a bit easier
-to follow once we use @socket as the opaque pointer for
-aio_set_fd_handler().
-Cc: qemu-stable@nongnu.org
-Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190910124136.10565-3-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- block/curl.c | 10 +++++-----
-file changed, 5 insertions(+), 5 deletions(-)
-diff --git a/block/curl.c b/block/curl.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
-+++ b/block/curl.c
-@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
-     QLIST_FOREACH(socket, &state->sockets, next) {
-         if (socket->fd == fd) {
--            if (action == CURL_POLL_REMOVE) {
--                QLIST_REMOVE(socket, next);
--                g_free(socket);
--            }
-             break;
-         }
-     }
-@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
-         socket->state = state;
-         QLIST_INSERT_HEAD(&state->sockets, socket, next);
-     }
--    socket = NULL;
-     trace_curl_sock_cb(action, (int)fd);
-     switch (action) {
-@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
-             break;
-     }
-+    if (action == CURL_POLL_REMOVE) {
-+        QLIST_REMOVE(socket, next);
-+        g_free(socket);
-+    }
-+
-     return 0;
- }
---
-.21.0

-[Qemu-devel] [PULL 08/16] curl: Check completion in curl_multi_do()
+Deleted patch
-While it is more likely that transfers complete after some file
-descriptor has data ready to read, we probably should not rely on it.
-Better be safe than sorry and call curl_multi_check_completion() in
-curl_multi_do(), too, just like it is done in curl_multi_read().
-With this change, curl_multi_do() and curl_multi_read() are actually the
-same, so drop curl_multi_read() and use curl_multi_do() as the sole FD
-handler.
-Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190910124136.10565-4-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- block/curl.c | 14 ++------------
-file changed, 2 insertions(+), 12 deletions(-)
-diff --git a/block/curl.c b/block/curl.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
-+++ b/block/curl.c
-@@ -XXX,XX +XXX,XX @@ typedef struct BDRVCURLState {
- static void curl_clean_state(CURLState *s);
- static void curl_multi_do(void *arg);
--static void curl_multi_read(void *arg);
- #ifdef NEED_CURL_TIMER_CALLBACK
- /* Called from curl_multi_do_locked, with s->mutex held.  */
-@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
-     switch (action) {
-         case CURL_POLL_IN:
-             aio_set_fd_handler(s->aio_context, fd, false,
--                               curl_multi_read, NULL, NULL, state);
-+                               curl_multi_do, NULL, NULL, state);
-             break;
-         case CURL_POLL_OUT:
-             aio_set_fd_handler(s->aio_context, fd, false,
-@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
-             break;
-         case CURL_POLL_INOUT:
-             aio_set_fd_handler(s->aio_context, fd, false,
--                               curl_multi_read, curl_multi_do, NULL, state);
-+                               curl_multi_do, curl_multi_do, NULL, state);
-             break;
-         case CURL_POLL_REMOVE:
-             aio_set_fd_handler(s->aio_context, fd, false,
-@@ -XXX,XX +XXX,XX @@ static void curl_multi_do(void *arg)
- {
-     CURLState *s = (CURLState *)arg;
--    qemu_mutex_lock(&s->s->mutex);
--    curl_multi_do_locked(s);
--    qemu_mutex_unlock(&s->s->mutex);
--}
--
--static void curl_multi_read(void *arg)
--{
--    CURLState *s = (CURLState *)arg;
--
-     qemu_mutex_lock(&s->s->mutex);
-     curl_multi_do_locked(s);
-     curl_multi_check_completion(s->s);
---
-.21.0

-[Qemu-devel] [PULL 15/16] block/qcow2: refactor encryption code
+[Qemu-devel] [PULL v2 6/8] vmdk: Add read-only support for seSparse snapshots
-From: Maxim Levitsky <mlevitsk@redhat.com>
+From: Sam Eiderman <shmuel.eiderman@oracle.com>
-* Change the qcow2_co_{encrypt|decrypt} to just receive full host and
+Until ESXi 6.5 VMware used the vmfsSparse format for snapshots (VMDK3 in
-  guest offsets and use this function directly instead of calling
+QEMU).
-  do_perform_cow_encrypt (which is removed by that patch).
+This format was lacking in the following:
-* Adjust qcow2_co_encdec to take full host and guest offsets as well.
+    * Grain directory (L1) and grain table (L2) entries were 32-bit,
-* Document the qcow2_co_{encrypt|decrypt} arguments
+      allowing access to only 2TB (slightly less) of data.
-  to prevent the bug fixed in former commit from hopefully
+    * The grain size (default) was 512 bytes - leading to data
-  happening again.
+      fragmentation and many grain tables.
+    * For space reclamation purposes, it was necessary to find all the
-Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
+      grains which are not pointed to by any grain table - so a reverse
-Message-id: 20190915203655.21638-3-mlevitsk@redhat.com
+      mapping of "offset of grain in vmdk" to "grain table" must be
-Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+      constructed - which takes large amounts of CPU/RAM.
-[mreitz: Let perform_cow() return the error value returned by
-         qcow2_co_encrypt(), as proposed by Vladimir]
+The format specification can be found in VMware's documentation:
 https://www.vmware.com/support/developer/vddk/vmdk_50_technote.pdf
 In ESXi 6.5, to support snapshot files larger than 2TB, a new format was
 introduced: SESparse (Space Efficient).
 This format fixes the above issues:
     * All entries are now 64-bit.
     * The grain size (default) is 4KB.
     * Grain directory and grain tables are now located at the beginning
       of the file.
       + seSparse format reserves space for all grain tables.
       + Grain tables can be addressed using an index.
       + Grains are located in the end of the file and can also be
         addressed with an index.
       - seSparse vmdks of large disks (64TB) have huge preallocated
         headers - mainly due to L2 tables, even for empty snapshots.
     * The header contains a reverse mapping ("backmap") of "offset of
       grain in vmdk" to "grain table" and a bitmap ("free bitmap") which
       specifies for each grain - whether it is allocated or not.
       Using these data structures we can implement space reclamation
       efficiently.
     * Due to the fact that the header now maintains two mappings:
         * The regular one (grain directory & grain tables)
         * A reverse one (backmap and free bitmap)
       These data structures can lose consistency upon crash and result
       in a corrupted VMDK.
       Therefore, a journal is also added to the VMDK and is replayed
       when the VMware reopens the file after a crash.
 Since ESXi 6.7 - SESparse is the only snapshot format available.
 Unfortunately, VMware does not provide documentation regarding the new
 seSparse format.
 This commit is based on black-box research of the seSparse format.
 Various in-guest block operations and their effect on the snapshot file
 were tested.
 The only VMware provided source of information (regarding the underlying
 implementation) was a log file on the ESXi:
     /var/log/hostd.log
 Whenever an seSparse snapshot is created - the log is being populated
 with seSparse records.
 Relevant log records are of the form:
 [...] Const Header:
 [...]  constMagic     = 0xcafebabe
 [...]  version        = 2.1
 [...]  capacity       = 204800
 [...]  grainSize      = 8
 [...]  grainTableSize = 64
 [...]  flags          = 0
 [...] Extents:
 [...]  Header         : <1 : 1>
 [...]  JournalHdr     : <2 : 2>
 [...]  Journal        : <2048 : 2048>
 [...]  GrainDirectory : <4096 : 2048>
 [...]  GrainTables    : <6144 : 2048>
 [...]  FreeBitmap     : <8192 : 2048>
 [...]  BackMap        : <10240 : 2048>
 [...]  Grain          : <12288 : 204800>
 [...] Volatile Header:
 [...] volatileMagic     = 0xcafecafe
 [...] FreeGTNumber      = 0
 [...] nextTxnSeqNumber  = 0
 [...] replayJournal     = 0
 The sizes that are seen in the log file are in sectors.
 Extents are of the following format: <offset : size>
 This commit is a strict implementation which enforces:
     * magics
     * version number 2.1
     * grain size of 8 sectors  (4KB)
     * grain table size of 64 sectors
     * zero flags
     * extent locations
 Additionally, this commit proivdes only a subset of the functionality
 offered by seSparse's format:
     * Read-only
     * No journal replay
     * No space reclamation
     * No unmap support
 Hence, journal header, journal, free bitmap and backmap extents are
 unused, only the "classic" (L1 -> L2 -> data) grain access is
 implemented.
 However there are several differences in the grain access itself.
 Grain directory (L1):
     * Grain directory entries are indexes (not offsets) to grain
       tables.
     * Valid grain directory entries have their highest nibble set to
 x1.
     * Since grain tables are always located in the beginning of the
       file - the index can fit into 32 bits - so we can use its low
       part if it's valid.
 Grain table (L2):
     * Grain table entries are indexes (not offsets) to grains.
     * If the highest nibble of the entry is:
 x0:
             The grain in not allocated.
             The rest of the bytes are 0.
 x1:
             The grain is unmapped - guest sees a zero grain.
             The rest of the bits point to the previously mapped grain,
             see 0x3 case.
 x2:
             The grain is zero.
 x3:
             The grain is allocated - to get the index calculate:
             ((entry & 0x0fff000000000000) >> 48) |
             ((entry & 0x0000ffffffffffff) << 12)
     * The difference between 0x1 and 0x2 is that 0x1 is an unallocated
       grain which results from the guest using sg_unmap to unmap the
       grain - but the grain itself still exists in the grain extent - a
       space reclamation procedure should delete it.
       Unmapping a zero grain has no effect (0x2 will not change to 0x1)
       but unmapping an unallocated grain will (0x0 to 0x1) - naturally.
 In order to implement seSparse some fields had to be changed to support
 both 32-bit and 64-bit entry sizes.
 Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
 Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
 Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
 Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
 Message-id: 20190620091057.47441-4-shmuel.eiderman@oracle.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/qcow2.h         |  8 +++---
+ block/vmdk.c | 358 ++++++++++++++++++++++++++++++++++++++++++++++++---
- block/qcow2-cluster.c | 41 +++++++++-------------------
+file changed, 342 insertions(+), 16 deletions(-)
- block/qcow2-threads.c | 63 +++++++++++++++++++++++++++++++++----------
- block/qcow2.c         |  5 ++--
+diff --git a/block/vmdk.c b/block/vmdk.c
 files changed, 69 insertions(+), 48 deletions(-)
 diff --git a/block/qcow2.h b/block/qcow2.h
 index XXXXXXX..XXXXXXX 100644
---- a/block/qcow2.h
+--- a/block/vmdk.c
-+++ b/block/qcow2.h
++++ b/block/vmdk.c
-@@ -XXX,XX +XXX,XX @@ ssize_t coroutine_fn
+@@ -XXX,XX +XXX,XX @@ typedef struct {
- qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size,
+     uint16_t compressAlgorithm;
-                     const void *src, size_t src_size);
+ } QEMU_PACKED VMDK4Header;
- int coroutine_fn
--qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
++typedef struct VMDKSESparseConstHeader {
--                 uint64_t offset, void *buf, size_t len);
++    uint64_t magic;
-+qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
++    uint64_t version;
-+                 uint64_t guest_offset, void *buf, size_t len);
++    uint64_t capacity;
- int coroutine_fn
++    uint64_t grain_size;
--qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
++    uint64_t grain_table_size;
--                 uint64_t offset, void *buf, size_t len);
++    uint64_t flags;
-+qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
++    uint64_t reserved1;
-+                 uint64_t guest_offset, void *buf, size_t len);
++    uint64_t reserved2;
++    uint64_t reserved3;
- #endif
++    uint64_t reserved4;
-diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
++    uint64_t volatile_header_offset;
-index XXXXXXX..XXXXXXX 100644
++    uint64_t volatile_header_size;
---- a/block/qcow2-cluster.c
++    uint64_t journal_header_offset;
-+++ b/block/qcow2-cluster.c
++    uint64_t journal_header_size;
-@@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_read(BlockDriverState *bs,
++    uint64_t journal_offset;
 +    uint64_t journal_size;
 +    uint64_t grain_dir_offset;
 +    uint64_t grain_dir_size;
 +    uint64_t grain_tables_offset;
 +    uint64_t grain_tables_size;
 +    uint64_t free_bitmap_offset;
 +    uint64_t free_bitmap_size;
 +    uint64_t backmap_offset;
 +    uint64_t backmap_size;
 +    uint64_t grains_offset;
 +    uint64_t grains_size;
 +    uint8_t pad[304];
 +} QEMU_PACKED VMDKSESparseConstHeader;
 +
 +typedef struct VMDKSESparseVolatileHeader {
 +    uint64_t magic;
 +    uint64_t free_gt_number;
 +    uint64_t next_txn_seq_number;
 +    uint64_t replay_journal;
 +    uint8_t pad[480];
 +} QEMU_PACKED VMDKSESparseVolatileHeader;
 +
  #define L2_CACHE_SIZE 16
  typedef struct VmdkExtent {
@@ -XXX,XX +XXX,XX @@ typedef struct VmdkExtent {
      bool compressed;
      bool has_marker;
      bool has_zero_grain;
 +    bool sesparse;
 +    uint64_t sesparse_l2_tables_offset;
 +    uint64_t sesparse_clusters_offset;
 +    int32_t entry_size;
      int version;
      int64_t sectors;
      int64_t end_sector;
      int64_t flat_start_offset;
      int64_t l1_table_offset;
      int64_t l1_backup_table_offset;
 -    uint32_t *l1_table;
 +    void *l1_table;
      uint32_t *l1_backup_table;
      unsigned int l1_size;
      uint32_t l1_entry_sectors;
      unsigned int l2_size;
 -    uint32_t *l2_cache;
 +    void *l2_cache;
      uint32_t l2_cache_offsets[L2_CACHE_SIZE];
      uint32_t l2_cache_counts[L2_CACHE_SIZE];
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
           *            minimal L2 table size: 512 entries
           *            8 TB is still more than the maximal value supported for
           *            VMDK3 & VMDK4 which is 2TB.
 +         *     64TB - for "ESXi seSparse Extent"
 +         *            minimal cluster size: 512B (default is 4KB)
 +         *            L2 table size: 4096 entries (const).
 +         *            64TB is more than the maximal value supported for
 +         *            seSparse VMDKs (which is slightly less than 64TB)
           */
          error_setg(errp, "L1 size too big");
          return -EFBIG;
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
      extent->l2_size = l2_size;
      extent->cluster_sectors = flat ? sectors : cluster_sectors;
      extent->next_cluster_sector = ROUND_UP(nb_sectors, cluster_sectors);
 +    extent->entry_size = sizeof(uint32_t);
      if (s->num_extents > 1) {
          extent->end_sector = (*(extent - 1)).end_sector + extent->sectors;
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
      int i;
      /* read the L1 table */
 -    l1_size = extent->l1_size * sizeof(uint32_t);
 +    l1_size = extent->l1_size * extent->entry_size;
      extent->l1_table = g_try_malloc(l1_size);
      if (l1_size && extent->l1_table == NULL) {
          return -ENOMEM;
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
          goto fail_l1;
      }
      for (i = 0; i < extent->l1_size; i++) {
 -        le32_to_cpus(&extent->l1_table[i]);
 +        if (extent->entry_size == sizeof(uint64_t)) {
 +            le64_to_cpus((uint64_t *)extent->l1_table + i);
 +        } else {
 +            assert(extent->entry_size == sizeof(uint32_t));
 +            le32_to_cpus((uint32_t *)extent->l1_table + i);
 +        }
      }
      if (extent->l1_backup_table_offset) {
 +        assert(!extent->sesparse);
          extent->l1_backup_table = g_try_malloc(l1_size);
          if (l1_size && extent->l1_backup_table == NULL) {
              ret = -ENOMEM;
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
      }
      extent->l2_cache =
 -        g_new(uint32_t, extent->l2_size * L2_CACHE_SIZE);
 +        g_malloc(extent->entry_size * extent->l2_size * L2_CACHE_SIZE);
      return 0;
+  fail_l1b:
+     g_free(extent->l1_backup_table);
+@@ -XXX,XX +XXX,XX @@ static int vmdk_open_vmfs_sparse(BlockDriverState *bs,
+     return ret;
  }
--static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
++#define SESPARSE_CONST_HEADER_MAGIC UINT64_C(0x00000000cafebabe)
--                                                uint64_t src_cluster_offset,
++#define SESPARSE_VOLATILE_HEADER_MAGIC UINT64_C(0x00000000cafecafe)
--                                                uint64_t cluster_offset,
++
--                                                unsigned offset_in_cluster,
++/* Strict checks - format not officially documented */
--                                                uint8_t *buffer,
++static int check_se_sparse_const_header(VMDKSESparseConstHeader *header,
--                                                unsigned bytes)
++                                        Error **errp)
--{
++{
--    if (bytes && bs->encrypted) {
++    header->magic = le64_to_cpu(header->magic);
--        BDRVQcow2State *s = bs->opaque;
++    header->version = le64_to_cpu(header->version);
--        assert(QEMU_IS_ALIGNED(offset_in_cluster, BDRV_SECTOR_SIZE));
++    header->grain_size = le64_to_cpu(header->grain_size);
--        assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
++    header->grain_table_size = le64_to_cpu(header->grain_table_size);
--        assert(s->crypto);
++    header->flags = le64_to_cpu(header->flags);
--        if (qcow2_co_encrypt(bs,
++    header->reserved1 = le64_to_cpu(header->reserved1);
--                start_of_cluster(s, cluster_offset + offset_in_cluster),
++    header->reserved2 = le64_to_cpu(header->reserved2);
--                src_cluster_offset + offset_in_cluster,
++    header->reserved3 = le64_to_cpu(header->reserved3);
--                buffer, bytes) < 0) {
++    header->reserved4 = le64_to_cpu(header->reserved4);
--            return false;
++
--        }
++    header->volatile_header_offset =
--    }
++        le64_to_cpu(header->volatile_header_offset);
--    return true;
++    header->volatile_header_size = le64_to_cpu(header->volatile_header_size);
--}
++
--
++    header->journal_header_offset = le64_to_cpu(header->journal_header_offset);
- static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
++    header->journal_header_size = le64_to_cpu(header->journal_header_size);
-                                              uint64_t cluster_offset,
++
-                                              unsigned offset_in_cluster,
++    header->journal_offset = le64_to_cpu(header->journal_offset);
-@@ -XXX,XX +XXX,XX @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta *m)
++    header->journal_size = le64_to_cpu(header->journal_size);
++
-     /* Encrypt the data if necessary before writing it */
++    header->grain_dir_offset = le64_to_cpu(header->grain_dir_offset);
-     if (bs->encrypted) {
++    header->grain_dir_size = le64_to_cpu(header->grain_dir_size);
--        if (!do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
++
--                                    start->offset, start_buffer,
++    header->grain_tables_offset = le64_to_cpu(header->grain_tables_offset);
--                                    start->nb_bytes) ||
++    header->grain_tables_size = le64_to_cpu(header->grain_tables_size);
--            !do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
++
--                                    end->offset, end_buffer, end->nb_bytes)) {
++    header->free_bitmap_offset = le64_to_cpu(header->free_bitmap_offset);
--            ret = -EIO;
++    header->free_bitmap_size = le64_to_cpu(header->free_bitmap_size);
-+        ret = qcow2_co_encrypt(bs,
++
-+                               m->alloc_offset + start->offset,
++    header->backmap_offset = le64_to_cpu(header->backmap_offset);
-+                               m->offset + start->offset,
++    header->backmap_size = le64_to_cpu(header->backmap_size);
-+                               start_buffer, start->nb_bytes);
++
-+        if (ret < 0) {
++    header->grains_offset = le64_to_cpu(header->grains_offset);
-+            goto fail;
++    header->grains_size = le64_to_cpu(header->grains_size);
 +
 +    if (header->magic != SESPARSE_CONST_HEADER_MAGIC) {
 +        error_setg(errp, "Bad const header magic: 0x%016" PRIx64,
 +                   header->magic);
 +        return -EINVAL;
 +    }
 +
 +    if (header->version != 0x0000000200000001) {
 +        error_setg(errp, "Unsupported version: 0x%016" PRIx64,
 +                   header->version);
 +        return -ENOTSUP;
 +    }
 +
 +    if (header->grain_size != 8) {
 +        error_setg(errp, "Unsupported grain size: %" PRIu64,
 +                   header->grain_size);
 +        return -ENOTSUP;
 +    }
 +
 +    if (header->grain_table_size != 64) {
 +        error_setg(errp, "Unsupported grain table size: %" PRIu64,
 +                   header->grain_table_size);
 +        return -ENOTSUP;
 +    }
 +
 +    if (header->flags != 0) {
 +        error_setg(errp, "Unsupported flags: 0x%016" PRIx64,
 +                   header->flags);
 +        return -ENOTSUP;
 +    }
 +
 +    if (header->reserved1 != 0 || header->reserved2 != 0 ||
 +        header->reserved3 != 0 || header->reserved4 != 0) {
 +        error_setg(errp, "Unsupported reserved bits:"
 +                   " 0x%016" PRIx64 " 0x%016" PRIx64
 +                   " 0x%016" PRIx64 " 0x%016" PRIx64,
 +                   header->reserved1, header->reserved2,
 +                   header->reserved3, header->reserved4);
 +        return -ENOTSUP;
 +    }
 +
 +    /* check that padding is 0 */
 +    if (!buffer_is_zero(header->pad, sizeof(header->pad))) {
 +        error_setg(errp, "Unsupported non-zero const header padding");
 +        return -ENOTSUP;
 +    }
 +
 +    return 0;
 +}
 +
 +static int check_se_sparse_volatile_header(VMDKSESparseVolatileHeader *header,
 +                                           Error **errp)
 +{
 +    header->magic = le64_to_cpu(header->magic);
 +    header->free_gt_number = le64_to_cpu(header->free_gt_number);
 +    header->next_txn_seq_number = le64_to_cpu(header->next_txn_seq_number);
 +    header->replay_journal = le64_to_cpu(header->replay_journal);
 +
 +    if (header->magic != SESPARSE_VOLATILE_HEADER_MAGIC) {
 +        error_setg(errp, "Bad volatile header magic: 0x%016" PRIx64,
 +                   header->magic);
 +        return -EINVAL;
 +    }
 +
 +    if (header->replay_journal) {
 +        error_setg(errp, "Image is dirty, Replaying journal not supported");
 +        return -ENOTSUP;
 +    }
 +
 +    /* check that padding is 0 */
 +    if (!buffer_is_zero(header->pad, sizeof(header->pad))) {
 +        error_setg(errp, "Unsupported non-zero volatile header padding");
 +        return -ENOTSUP;
 +    }
 +
 +    return 0;
 +}
 +
 +static int vmdk_open_se_sparse(BlockDriverState *bs,
 +                               BdrvChild *file,
 +                               int flags, Error **errp)
 +{
 +    int ret;
 +    VMDKSESparseConstHeader const_header;
 +    VMDKSESparseVolatileHeader volatile_header;
 +    VmdkExtent *extent;
 +
 +    ret = bdrv_apply_auto_read_only(bs,
 +            "No write support for seSparse images available", errp);
 +    if (ret < 0) {
 +        return ret;
 +    }
 +
 +    assert(sizeof(const_header) == SECTOR_SIZE);
 +
 +    ret = bdrv_pread(file, 0, &const_header, sizeof(const_header));
 +    if (ret < 0) {
 +        bdrv_refresh_filename(file->bs);
 +        error_setg_errno(errp, -ret,
 +                         "Could not read const header from file '%s'",
 +                         file->bs->filename);
 +        return ret;
 +    }
 +
 +    /* check const header */
 +    ret = check_se_sparse_const_header(&const_header, errp);
 +    if (ret < 0) {
 +        return ret;
 +    }
 +
 +    assert(sizeof(volatile_header) == SECTOR_SIZE);
 +
 +    ret = bdrv_pread(file,
 +                     const_header.volatile_header_offset * SECTOR_SIZE,
 +                     &volatile_header, sizeof(volatile_header));
 +    if (ret < 0) {
 +        bdrv_refresh_filename(file->bs);
 +        error_setg_errno(errp, -ret,
 +                         "Could not read volatile header from file '%s'",
 +                         file->bs->filename);
 +        return ret;
 +    }
 +
 +    /* check volatile header */
 +    ret = check_se_sparse_volatile_header(&volatile_header, errp);
 +    if (ret < 0) {
 +        return ret;
 +    }
 +
 +    ret = vmdk_add_extent(bs, file, false,
 +                          const_header.capacity,
 +                          const_header.grain_dir_offset * SECTOR_SIZE,
 +                          0,
 +                          const_header.grain_dir_size *
 +                          SECTOR_SIZE / sizeof(uint64_t),
 +                          const_header.grain_table_size *
 +                          SECTOR_SIZE / sizeof(uint64_t),
 +                          const_header.grain_size,
 +                          &extent,
 +                          errp);
 +    if (ret < 0) {
 +        return ret;
 +    }
 +
 +    extent->sesparse = true;
 +    extent->sesparse_l2_tables_offset = const_header.grain_tables_offset;
 +    extent->sesparse_clusters_offset = const_header.grains_offset;
 +    extent->entry_size = sizeof(uint64_t);
 +
 +    ret = vmdk_init_tables(bs, extent, errp);
 +    if (ret) {
 +        /* free extent allocated by vmdk_add_extent */
 +        vmdk_free_last_extent(bs);
 +    }
 +
 +    return ret;
 +}
 +
  static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
                                 QDict *options, Error **errp);
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
           * RW [size in sectors] SPARSE "file-name.vmdk"
           * RW [size in sectors] VMFS "file-name.vmdk"
           * RW [size in sectors] VMFSSPARSE "file-name.vmdk"
 +         * RW [size in sectors] SESPARSE "file-name.vmdk"
           */
          flat_offset = -1;
          matches = sscanf(p, "%10s %" SCNd64 " %10s \"%511[^\n\r\"]\" %" SCNd64,
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
          if (sectors <= 0 ||
              (strcmp(type, "FLAT") && strcmp(type, "SPARSE") &&
 -             strcmp(type, "VMFS") && strcmp(type, "VMFSSPARSE")) ||
 +             strcmp(type, "VMFS") && strcmp(type, "VMFSSPARSE") &&
 +             strcmp(type, "SESPARSE")) ||
              (strcmp(access, "RW"))) {
              continue;
          }
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
                  return ret;
              }
              extent = &s->extents[s->num_extents - 1];
 +        } else if (!strcmp(type, "SESPARSE")) {
 +            ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
 +            if (ret) {
 +                bdrv_unref_child(bs, extent_file);
 +                return ret;
 +            }
 +            extent = &s->extents[s->num_extents - 1];
          } else {
              error_setg(errp, "Unsupported extent type '%s'", type);
              bdrv_unref_child(bs, extent_file);
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
      if (strcmp(ct, "monolithicFlat") &&
          strcmp(ct, "vmfs") &&
          strcmp(ct, "vmfsSparse") &&
 +        strcmp(ct, "seSparse") &&
          strcmp(ct, "twoGbMaxExtentSparse") &&
          strcmp(ct, "twoGbMaxExtentFlat")) {
          error_setg(errp, "Unsupported image type '%s'", ct);
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
  {
      unsigned int l1_index, l2_offset, l2_index;
      int min_index, i, j;
 -    uint32_t min_count, *l2_table;
 +    uint32_t min_count;
 +    void *l2_table;
      bool zeroed = false;
      int64_t ret;
      int64_t cluster_sector;
 +    unsigned int l2_size_bytes = extent->l2_size * extent->entry_size;
      if (m_data) {
          m_data->valid = 0;
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
      if (l1_index >= extent->l1_size) {
          return VMDK_ERROR;
      }
 -    l2_offset = extent->l1_table[l1_index];
 +    if (extent->sesparse) {
 +        uint64_t l2_offset_u64;
 +
 +        assert(extent->entry_size == sizeof(uint64_t));
 +
 +        l2_offset_u64 = ((uint64_t *)extent->l1_table)[l1_index];
 +        if (l2_offset_u64 == 0) {
 +            l2_offset = 0;
 +        } else if ((l2_offset_u64 & 0xffffffff00000000) != 0x1000000000000000) {
 +            /*
 +             * Top most nibble is 0x1 if grain table is allocated.
 +             * strict check - top most 4 bytes must be 0x10000000 since max
 +             * supported size is 64TB for disk - so no more than 64TB / 16MB
 +             * grain directories which is smaller than uint32,
 +             * where 16MB is the only supported default grain table coverage.
 +             */
 +            return VMDK_ERROR;
 +        } else {
 +            l2_offset_u64 = l2_offset_u64 & 0x00000000ffffffff;
 +            l2_offset_u64 = extent->sesparse_l2_tables_offset +
 +                l2_offset_u64 * l2_size_bytes / SECTOR_SIZE;
 +            if (l2_offset_u64 > 0x00000000ffffffff) {
 +                return VMDK_ERROR;
 +            }
 +            l2_offset = (unsigned int)(l2_offset_u64);
 +        }
-+
++    } else {
-+        ret = qcow2_co_encrypt(bs,
++        assert(extent->entry_size == sizeof(uint32_t));
-+                               m->alloc_offset + end->offset,
++        l2_offset = ((uint32_t *)extent->l1_table)[l1_index];
-+                               m->offset + end->offset,
++    }
-+                               end_buffer, end->nb_bytes);
+     if (!l2_offset) {
-+        if (ret < 0) {
+         return VMDK_UNALLOC;
-             goto fail;
+     }
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
                      extent->l2_cache_counts[j] >>= 1;
                  }
              }
 -            l2_table = extent->l2_cache + (i * extent->l2_size);
 +            l2_table = (char *)extent->l2_cache + (i * l2_size_bytes);
              goto found;
          }
      }
-diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
+@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
-index XXXXXXX..XXXXXXX 100644
+             min_index = i;
---- a/block/qcow2-threads.c
+         }
-+++ b/block/qcow2-threads.c
+     }
-@@ -XXX,XX +XXX,XX @@ static int qcow2_encdec_pool_func(void *opaque)
+-    l2_table = extent->l2_cache + (min_index * extent->l2_size);
- }
++    l2_table = (char *)extent->l2_cache + (min_index * l2_size_bytes);
+     BLKDBG_EVENT(extent->file, BLKDBG_L2_LOAD);
- static int coroutine_fn
+     if (bdrv_pread(extent->file,
--qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
+                 (int64_t)l2_offset * 512,
--                  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc func)
+                 l2_table,
-+qcow2_co_encdec(BlockDriverState *bs, uint64_t host_offset,
+-                extent->l2_size * sizeof(uint32_t)
-+                uint64_t guest_offset, void *buf, size_t len,
+-            ) != extent->l2_size * sizeof(uint32_t)) {
-+                Qcow2EncDecFunc func)
++                l2_size_bytes
- {
++            ) != l2_size_bytes) {
-     BDRVQcow2State *s = bs->opaque;
+         return VMDK_ERROR;
-     Qcow2EncDecData arg = {
+     }
-         .block = s->crypto,
--        .offset = s->crypt_physical_offset ?
+@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
--                      file_cluster_offset + offset_into_cluster(s, offset) :
+     extent->l2_cache_counts[min_index] = 1;
--                      offset,
+  found:
-+        .offset = s->crypt_physical_offset ? host_offset : guest_offset,
+     l2_index = ((offset >> 9) / extent->cluster_sectors) % extent->l2_size;
-         .buf = buf,
+-    cluster_sector = le32_to_cpu(l2_table[l2_index]);
-         .len = len,
-         .func = func,
+-    if (extent->has_zero_grain && cluster_sector == VMDK_GTE_ZEROED) {
-     };
+-        zeroed = true;
++    if (extent->sesparse) {
--    return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
++        cluster_sector = le64_to_cpu(((uint64_t *)l2_table)[l2_index]);
-+    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
++        switch (cluster_sector & 0xf000000000000000) {
-+    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
++        case 0x0000000000000000:
-+    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
++            /* unallocated grain */
-+    assert(s->crypto);
++            if (cluster_sector != 0) {
-+
++                return VMDK_ERROR;
-+    return len == 0 ? 0 : qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
++            }
- }
++            break;
++        case 0x1000000000000000:
-+/*
++            /* scsi-unmapped grain - fallthrough */
-+ * qcow2_co_encrypt()
++        case 0x2000000000000000:
-+ *
++            /* zero grain */
-+ * Encrypts one or more contiguous aligned sectors
++            zeroed = true;
-+ *
++            break;
-+ * @host_offset - underlying storage offset of the first sector of the
++        case 0x3000000000000000:
-+ * data to be encrypted
++            /* allocated grain */
-+ *
++            cluster_sector = (((cluster_sector & 0x0fff000000000000) >> 48) |
-+ * @guest_offset - guest (virtual) offset of the first sector of the
++                              ((cluster_sector & 0x0000ffffffffffff) << 12));
-+ * data to be encrypted
++            cluster_sector = extent->sesparse_clusters_offset +
-+ *
++                cluster_sector * extent->cluster_sectors;
-+ * @buf - buffer with the data to encrypt, that after encryption
++            break;
-+ *        will be written to the underlying storage device at
++        default:
-+ *        @host_offset
++            return VMDK_ERROR;
-+ *
++        }
-+ * @len - length of the buffer (must be a BDRV_SECTOR_SIZE multiple)
++    } else {
-+ *
++        cluster_sector = le32_to_cpu(((uint32_t *)l2_table)[l2_index]);
-+ * Depending on the encryption method, @host_offset and/or @guest_offset
++
-+ * may be used for generating the initialization vector for
++        if (extent->has_zero_grain && cluster_sector == VMDK_GTE_ZEROED) {
-+ * encryption.
++            zeroed = true;
-+ *
++        }
-+ * Note that while the whole range must be aligned on sectors, it
+     }
-+ * does not have to be aligned on clusters and can also cross cluster
-+ * boundaries
+     if (!cluster_sector || zeroed) {
-+ */
+         if (!allocate) {
- int coroutine_fn
+             return zeroed ? VMDK_ZEROED : VMDK_UNALLOC;
--qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
+         }
--                 uint64_t offset, void *buf, size_t len)
++        assert(!extent->sesparse);
-+qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
-+                 uint64_t guest_offset, void *buf, size_t len)
+         if (extent->next_cluster_sector >= VMDK_EXTENT_MAX_SECTORS) {
- {
+             return VMDK_ERROR;
--    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
+@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
--                             qcrypto_block_encrypt);
+             m_data->l1_index = l1_index;
-+    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
+             m_data->l2_index = l2_index;
-+                           qcrypto_block_encrypt);
+             m_data->l2_offset = l2_offset;
- }
+-            m_data->l2_cache_entry = &l2_table[l2_index];
++            m_data->l2_cache_entry = ((uint32_t *)l2_table) + l2_index;
-+/*
+         }
-+ * qcow2_co_decrypt()
+     }
-+ *
+     *cluster_offset = cluster_sector << BDRV_SECTOR_BITS;
-+ * Decrypts one or more contiguous aligned sectors
+@@ -XXX,XX +XXX,XX @@ static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset,
-+ * Similar to qcow2_co_encrypt
+         if (!extent) {
-+ */
+             return -EIO;
- int coroutine_fn
+         }
--qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
++        if (extent->sesparse) {
--                 uint64_t offset, void *buf, size_t len)
++            return -ENOTSUP;
-+qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
++        }
-+                 uint64_t guest_offset, void *buf, size_t len)
+         offset_in_cluster = vmdk_find_offset_in_cluster(extent, offset);
- {
+         n_bytes = MIN(bytes, extent->cluster_sectors * BDRV_SECTOR_SIZE
--    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
+                              - offset_in_cluster);
 -                             qcrypto_block_decrypt);
 +    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
 +                           qcrypto_block_decrypt);
  }
 diff --git a/block/qcow2.c b/block/qcow2.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/qcow2.c
 +++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
                  assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
                  assert(QEMU_IS_ALIGNED(cur_bytes, BDRV_SECTOR_SIZE));
 -                if (qcow2_co_decrypt(bs, cluster_offset, offset,
 +                if (qcow2_co_decrypt(bs, cluster_offset + offset_in_cluster,
 +                                     offset,
                                       cluster_data, cur_bytes) < 0) {
                      ret = -EIO;
                      goto fail;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_part(
              qemu_iovec_to_buf(qiov, qiov_offset + bytes_done,
                                cluster_data, cur_bytes);
 -            if (qcow2_co_encrypt(bs, cluster_offset, offset,
 +            if (qcow2_co_encrypt(bs, cluster_offset + offset_in_cluster, offset,
                                   cluster_data, cur_bytes) < 0) {
                  ret = -EIO;
                  goto out_unlocked;
 --
 .21.0

-[Qemu-devel] [PULL 09/16] curl: Pass CURLSocket to curl_multi_do()
+[Qemu-devel] [PULL v2 7/8] ssh: switch from libssh2 to libssh
-curl_multi_do_locked() currently marks all sockets as ready.  That is
+From: Pino Toscano <ptoscano@redhat.com>
 not only inefficient, but in fact unsafe (the loop is).  A follow-up
 patch will change that, but to do so, curl_multi_do_locked() needs to
 know exactly which socket is ready; and that is accomplished by this
 patch here.
-Cc: qemu-stable@nongnu.org
+Rewrite the implementation of the ssh block driver to use libssh instead
-Signed-off-by: Max Reitz <mreitz@redhat.com>
+of libssh2.  The libssh library has various advantages over libssh2:
-Message-id: 20190910124136.10565-5-mreitz@redhat.com
+- easier API for authentication (for example for using ssh-agent)
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+- easier API for known_hosts handling
-Reviewed-by: John Snow <jsnow@redhat.com>
+- supports newer types of keys in known_hosts
 Use APIs/features available in libssh 0.8 conditionally, to support
 older versions (which are not recommended though).
 Adjust the iotest 207 according to the different error message, and to
 find the default key type for localhost (to properly compare the
 fingerprint with).
 Contributed-by: Max Reitz <mreitz@redhat.com>
 Adjust the various Docker/Travis scripts to use libssh when available
 instead of libssh2. The mingw/mxe testing is dropped for now, as there
 are no packages for it.
 Signed-off-by: Pino Toscano <ptoscano@redhat.com>
 Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
 Acked-by: Alex Bennée <alex.bennee@linaro.org>
 Message-id: 20190620200840.17655-1-ptoscano@redhat.com
 Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
 Message-id: 5873173.t2JhDm7DL7@lindworm.usersys.redhat.com
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/curl.c | 20 +++++++++++---------
+ configure                                     |  65 +-
-file changed, 11 insertions(+), 9 deletions(-)
+ block/Makefile.objs                           |   6 +-
  block/ssh.c                                   | 652 ++++++++++--------
  .travis.yml                                   |   4 +-
  block/trace-events                            |  14 +-
  docs/qemu-block-drivers.texi                  |   2 +-
  .../dockerfiles/debian-win32-cross.docker     |   1 -
  .../dockerfiles/debian-win64-cross.docker     |   1 -
  tests/docker/dockerfiles/fedora.docker        |   4 +-
  tests/docker/dockerfiles/ubuntu.docker        |   2 +-
  tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
  tests/qemu-iotests/207                        |  54 +-
  tests/qemu-iotests/207.out                    |   2 +-
 files changed, 449 insertions(+), 360 deletions(-)
-diff --git a/block/curl.c b/block/curl.c
+diff --git a/configure b/configure
 index XXXXXXX..XXXXXXX 100755
 --- a/configure
 +++ b/configure
@@ -XXX,XX +XXX,XX @@ auth_pam=""
  vte=""
  virglrenderer=""
  tpm=""
 -libssh2=""
 +libssh=""
  live_block_migration="yes"
  numa=""
  tcmalloc="no"
@@ -XXX,XX +XXX,XX @@ for opt do
    ;;
    --enable-tpm) tpm="yes"
    ;;
 -  --disable-libssh2) libssh2="no"
 +  --disable-libssh) libssh="no"
    ;;
 -  --enable-libssh2) libssh2="yes"
 +  --enable-libssh) libssh="yes"
    ;;
    --disable-live-block-migration) live_block_migration="no"
    ;;
@@ -XXX,XX +XXX,XX @@ disabled with --disable-FEATURE, default is enabled if available:
    coroutine-pool  coroutine freelist (better performance)
    glusterfs       GlusterFS backend
    tpm             TPM support
 -  libssh2         ssh block device support
 +  libssh          ssh block device support
    numa            libnuma support
    libxml2         for Parallels image format
    tcmalloc        tcmalloc support
@@ -XXX,XX +XXX,XX @@ EOF
  fi
  ##########################################
 -# libssh2 probe
 -min_libssh2_version=1.2.8
 -if test "$libssh2" != "no" ; then
 -  if $pkg_config --atleast-version=$min_libssh2_version libssh2; then
 -    libssh2_cflags=$($pkg_config libssh2 --cflags)
 -    libssh2_libs=$($pkg_config libssh2 --libs)
 -    libssh2=yes
 +# libssh probe
 +if test "$libssh" != "no" ; then
 +  if $pkg_config --exists libssh; then
 +    libssh_cflags=$($pkg_config libssh --cflags)
 +    libssh_libs=$($pkg_config libssh --libs)
 +    libssh=yes
    else
 -    if test "$libssh2" = "yes" ; then
 -      error_exit "libssh2 >= $min_libssh2_version required for --enable-libssh2"
 +    if test "$libssh" = "yes" ; then
 +      error_exit "libssh required for --enable-libssh"
      fi
 -    libssh2=no
 +    libssh=no
    fi
  fi
  ##########################################
 -# libssh2_sftp_fsync probe
 +# Check for libssh 0.8
 +# This is done like this instead of using the LIBSSH_VERSION_* and
 +# SSH_VERSION_* macros because some distributions in the past shipped
 +# snapshots of the future 0.8 from Git, and those snapshots did not
 +# have updated version numbers (still referring to 0.7.0).
 -if test "$libssh2" = "yes"; then
 +if test "$libssh" = "yes"; then
    cat > $TMPC <<EOF
 -#include <stdio.h>
 -#include <libssh2.h>
 -#include <libssh2_sftp.h>
 -int main(void) {
 -    LIBSSH2_SESSION *session;
 -    LIBSSH2_SFTP *sftp;
 -    LIBSSH2_SFTP_HANDLE *sftp_handle;
 -    session = libssh2_session_init ();
 -    sftp = libssh2_sftp_init (session);
 -    sftp_handle = libssh2_sftp_open (sftp, "/", 0, 0);
 -    libssh2_sftp_fsync (sftp_handle);
 -    return 0;
 -}
 +#include <libssh/libssh.h>
 +int main(void) { return ssh_get_server_publickey(NULL, NULL); }
  EOF
 -  # libssh2_cflags/libssh2_libs defined in previous test.
 -  if compile_prog "$libssh2_cflags" "$libssh2_libs" ; then
 -    QEMU_CFLAGS="-DHAS_LIBSSH2_SFTP_FSYNC $QEMU_CFLAGS"
 +  if compile_prog "$libssh_cflags" "$libssh_libs"; then
 +    libssh_cflags="-DHAVE_LIBSSH_0_8 $libssh_cflags"
    fi
  fi
@@ -XXX,XX +XXX,XX @@ echo "GlusterFS support $glusterfs"
  echo "gcov              $gcov_tool"
  echo "gcov enabled      $gcov"
  echo "TPM support       $tpm"
 -echo "libssh2 support   $libssh2"
 +echo "libssh support    $libssh"
  echo "QOM debugging     $qom_cast_debug"
  echo "Live block migration $live_block_migration"
  echo "lzo support       $lzo"
@@ -XXX,XX +XXX,XX @@ if test "$glusterfs_iocb_has_stat" = "yes" ; then
    echo "CONFIG_GLUSTERFS_IOCB_HAS_STAT=y" >> $config_host_mak
  fi
 -if test "$libssh2" = "yes" ; then
 -  echo "CONFIG_LIBSSH2=m" >> $config_host_mak
 -  echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
 -  echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
 +if test "$libssh" = "yes" ; then
 +  echo "CONFIG_LIBSSH=m" >> $config_host_mak
 +  echo "LIBSSH_CFLAGS=$libssh_cflags" >> $config_host_mak
 +  echo "LIBSSH_LIBS=$libssh_libs" >> $config_host_mak
  fi
  if test "$live_block_migration" = "yes" ; then
 diff --git a/block/Makefile.objs b/block/Makefile.objs
 index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
+--- a/block/Makefile.objs
-+++ b/block/curl.c
++++ b/block/Makefile.objs
-@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
+@@ -XXX,XX +XXX,XX @@ block-obj-$(CONFIG_CURL) += curl.o
-     switch (action) {
+ block-obj-$(CONFIG_RBD) += rbd.o
-         case CURL_POLL_IN:
+ block-obj-$(CONFIG_GLUSTERFS) += gluster.o
-             aio_set_fd_handler(s->aio_context, fd, false,
+ block-obj-$(CONFIG_VXHS) += vxhs.o
--                               curl_multi_do, NULL, NULL, state);
+-block-obj-$(CONFIG_LIBSSH2) += ssh.o
-+                               curl_multi_do, NULL, NULL, socket);
++block-obj-$(CONFIG_LIBSSH) += ssh.o
-             break;
+ block-obj-y += accounting.o dirty-bitmap.o
-         case CURL_POLL_OUT:
+ block-obj-y += write-threshold.o
-             aio_set_fd_handler(s->aio_context, fd, false,
+ block-obj-y += backup.o
--                               NULL, curl_multi_do, NULL, state);
+@@ -XXX,XX +XXX,XX @@ rbd.o-libs         := $(RBD_LIBS)
-+                               NULL, curl_multi_do, NULL, socket);
+ gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
-             break;
+ gluster.o-libs     := $(GLUSTERFS_LIBS)
-         case CURL_POLL_INOUT:
+ vxhs.o-libs        := $(VXHS_LIBS)
-             aio_set_fd_handler(s->aio_context, fd, false,
+-ssh.o-cflags       := $(LIBSSH2_CFLAGS)
--                               curl_multi_do, curl_multi_do, NULL, state);
+-ssh.o-libs         := $(LIBSSH2_LIBS)
-+                               curl_multi_do, curl_multi_do, NULL, socket);
++ssh.o-cflags       := $(LIBSSH_CFLAGS)
-             break;
++ssh.o-libs         := $(LIBSSH_LIBS)
-         case CURL_POLL_REMOVE:
+ block-obj-dmg-bz2-$(CONFIG_BZIP2) += dmg-bz2.o
-             aio_set_fd_handler(s->aio_context, fd, false,
+ block-obj-$(if $(CONFIG_DMG),m,n) += $(block-obj-dmg-bz2-y)
-@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
+ dmg-bz2.o-libs     := $(BZIP2_LIBS)
 diff --git a/block/ssh.c b/block/ssh.c
 index XXXXXXX..XXXXXXX 100644
 --- a/block/ssh.c
 +++ b/block/ssh.c
@@ -XXX,XX +XXX,XX @@
  #include "qemu/osdep.h"
 -#include <libssh2.h>
 -#include <libssh2_sftp.h>
 +#include <libssh/libssh.h>
 +#include <libssh/sftp.h>
  #include "block/block_int.h"
  #include "block/qdict.h"
@@ -XXX,XX +XXX,XX @@
  #include "trace.h"
  /*
 - * TRACE_LIBSSH2=<bitmask> enables tracing in libssh2 itself.  Note
 - * that this requires that libssh2 was specially compiled with the
 - * `./configure --enable-debug' option, so most likely you will have
 - * to compile it yourself.  The meaning of <bitmask> is described
 - * here: http://www.libssh2.org/libssh2_trace.html
 + * TRACE_LIBSSH=<level> enables tracing in libssh itself.
 + * The meaning of <level> is described here:
 + * http://api.libssh.org/master/group__libssh__log.html
   */
 -#define TRACE_LIBSSH2 0 /* or try: LIBSSH2_TRACE_SFTP */
 +#define TRACE_LIBSSH  0 /* see: SSH_LOG_* */
  typedef struct BDRVSSHState {
      /* Coroutine. */
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVSSHState {
      /* SSH connection. */
      int sock;                         /* socket */
 -    LIBSSH2_SESSION *session;         /* ssh session */
 -    LIBSSH2_SFTP *sftp;               /* sftp session */
 -    LIBSSH2_SFTP_HANDLE *sftp_handle; /* sftp remote file handle */
 +    ssh_session session;              /* ssh session */
 +    sftp_session sftp;                /* sftp session */
 +    sftp_file sftp_handle;            /* sftp remote file handle */
 -    /* See ssh_seek() function below. */
 -    int64_t offset;
 -    bool offset_op_read;
 -
 -    /* File attributes at open.  We try to keep the .filesize field
 +    /*
 +     * File attributes at open.  We try to keep the .size field
       * updated if it changes (eg by writing at the end of the file).
       */
 -    LIBSSH2_SFTP_ATTRIBUTES attrs;
 +    sftp_attributes attrs;
      InetSocketAddress *inet;
@@ -XXX,XX +XXX,XX @@ static void ssh_state_init(BDRVSSHState *s)
  {
      memset(s, 0, sizeof *s);
      s->sock = -1;
 -    s->offset = -1;
      qemu_co_mutex_init(&s->lock);
  }
- /* Called with s->mutex held.  */
+@@ -XXX,XX +XXX,XX @@ static void ssh_state_free(BDRVSSHState *s)
 -static void curl_multi_do_locked(CURLState *s)
 +static void curl_multi_do_locked(CURLSocket *ready_socket)
  {
-     CURLSocket *socket, *next_socket;
+     g_free(s->user);
-+    CURLState *s = ready_socket->state;
-     int running;
++    if (s->attrs) {
 +        sftp_attributes_free(s->attrs);
 +    }
      if (s->sftp_handle) {
 -        libssh2_sftp_close(s->sftp_handle);
 +        sftp_close(s->sftp_handle);
      }
      if (s->sftp) {
 -        libssh2_sftp_shutdown(s->sftp);
 +        sftp_free(s->sftp);
      }
      if (s->session) {
 -        libssh2_session_disconnect(s->session,
 -                                   "from qemu ssh client: "
 -                                   "user closed the connection");
 -        libssh2_session_free(s->session);
 -    }
 -    if (s->sock >= 0) {
 -        close(s->sock);
 +        ssh_disconnect(s->session);
 +        ssh_free(s->session); /* This frees s->sock */
      }
  }
@@ -XXX,XX +XXX,XX @@ session_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
      va_end(args);
      if (s->session) {
 -        char *ssh_err;
 +        const char *ssh_err;
          int ssh_err_code;
 -        /* This is not an errno.  See <libssh2.h>. */
 -        ssh_err_code = libssh2_session_last_error(s->session,
 -                                                  &ssh_err, NULL, 0);
 -        error_setg(errp, "%s: %s (libssh2 error code: %d)",
 +        /* This is not an errno.  See <libssh/libssh.h>. */
 +        ssh_err = ssh_get_error(s->session);
 +        ssh_err_code = ssh_get_error_code(s->session);
 +        error_setg(errp, "%s: %s (libssh error code: %d)",
                     msg, ssh_err, ssh_err_code);
      } else {
          error_setg(errp, "%s", msg);
@@ -XXX,XX +XXX,XX @@ sftp_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
      va_end(args);
      if (s->sftp) {
 -        char *ssh_err;
 +        const char *ssh_err;
          int ssh_err_code;
 -        unsigned long sftp_err_code;
 +        int sftp_err_code;
 -        /* This is not an errno.  See <libssh2.h>. */
 -        ssh_err_code = libssh2_session_last_error(s->session,
 -                                                  &ssh_err, NULL, 0);
 -        /* See <libssh2_sftp.h>. */
 -        sftp_err_code = libssh2_sftp_last_error((s)->sftp);
 +        /* This is not an errno.  See <libssh/libssh.h>. */
 +        ssh_err = ssh_get_error(s->session);
 +        ssh_err_code = ssh_get_error_code(s->session);
 +        /* See <libssh/sftp.h>. */
 +        sftp_err_code = sftp_get_error(s->sftp);
          error_setg(errp,
 -                   "%s: %s (libssh2 error code: %d, sftp error code: %lu)",
 +                   "%s: %s (libssh error code: %d, sftp error code: %d)",
                     msg, ssh_err, ssh_err_code, sftp_err_code);
      } else {
          error_setg(errp, "%s", msg);
@@ -XXX,XX +XXX,XX @@ sftp_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
  static void sftp_error_trace(BDRVSSHState *s, const char *op)
  {
 -    char *ssh_err;
 +    const char *ssh_err;
      int ssh_err_code;
 -    unsigned long sftp_err_code;
 +    int sftp_err_code;
 -    /* This is not an errno.  See <libssh2.h>. */
 -    ssh_err_code = libssh2_session_last_error(s->session,
 -                                              &ssh_err, NULL, 0);
 -    /* See <libssh2_sftp.h>. */
 -    sftp_err_code = libssh2_sftp_last_error((s)->sftp);
 +    /* This is not an errno.  See <libssh/libssh.h>. */
 +    ssh_err = ssh_get_error(s->session);
 +    ssh_err_code = ssh_get_error_code(s->session);
 +    /* See <libssh/sftp.h>. */
 +    sftp_err_code = sftp_get_error(s->sftp);
      trace_sftp_error(op, ssh_err, ssh_err_code, sftp_err_code);
  }
@@ -XXX,XX +XXX,XX @@ static void ssh_parse_filename(const char *filename, QDict *options,
      parse_uri(filename, options, errp);
  }
 -static int check_host_key_knownhosts(BDRVSSHState *s,
 -                                     const char *host, int port, Error **errp)
 +static int check_host_key_knownhosts(BDRVSSHState *s, Error **errp)
  {
 -    const char *home;
 -    char *knh_file = NULL;
 -    LIBSSH2_KNOWNHOSTS *knh = NULL;
 -    struct libssh2_knownhost *found;
 -    int ret, r;
 -    const char *hostkey;
 -    size_t len;
 -    int type;
 -
 -    hostkey = libssh2_session_hostkey(s->session, &len, &type);
 -    if (!hostkey) {
 +    int ret;
 +#ifdef HAVE_LIBSSH_0_8
 +    enum ssh_known_hosts_e state;
 +    int r;
 +    ssh_key pubkey;
 +    enum ssh_keytypes_e pubkey_type;
 +    unsigned char *server_hash = NULL;
 +    size_t server_hash_len;
 +    char *fingerprint = NULL;
 +
 +    state = ssh_session_is_known_server(s->session);
 +    trace_ssh_server_status(state);
 +
 +    switch (state) {
 +    case SSH_KNOWN_HOSTS_OK:
 +        /* OK */
 +        trace_ssh_check_host_key_knownhosts();
 +        break;
 +    case SSH_KNOWN_HOSTS_CHANGED:
          ret = -EINVAL;
 -        session_error_setg(errp, s, "failed to read remote host key");
 +        r = ssh_get_server_publickey(s->session, &pubkey);
 +        if (r == 0) {
 +            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA256,
 +                                       &server_hash, &server_hash_len);
 +            pubkey_type = ssh_key_type(pubkey);
 +            ssh_key_free(pubkey);
 +        }
 +        if (r == 0) {
 +            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA256,
 +                                                   server_hash,
 +                                                   server_hash_len);
 +            ssh_clean_pubkey_hash(&server_hash);
 +        }
 +        if (fingerprint) {
 +            error_setg(errp,
 +                       "host key (%s key with fingerprint %s) does not match "
 +                       "the one in known_hosts; this may be a possible attack",
 +                       ssh_key_type_to_char(pubkey_type), fingerprint);
 +            ssh_string_free_char(fingerprint);
 +        } else  {
 +            error_setg(errp,
 +                       "host key does not match the one in known_hosts; this "
 +                       "may be a possible attack");
 +        }
          goto out;
 -    }
 -
 -    knh = libssh2_knownhost_init(s->session);
 -    if (!knh) {
 +    case SSH_KNOWN_HOSTS_OTHER:
          ret = -EINVAL;
 -        session_error_setg(errp, s,
 -                           "failed to initialize known hosts support");
 +        error_setg(errp,
 +                   "host key for this server not found, another type exists");
 +        goto out;
 +    case SSH_KNOWN_HOSTS_UNKNOWN:
 +        ret = -EINVAL;
 +        error_setg(errp, "no host key was found in known_hosts");
 +        goto out;
 +    case SSH_KNOWN_HOSTS_NOT_FOUND:
 +        ret = -ENOENT;
 +        error_setg(errp, "known_hosts file not found");
 +        goto out;
 +    case SSH_KNOWN_HOSTS_ERROR:
 +        ret = -EINVAL;
 +        error_setg(errp, "error while checking the host");
 +        goto out;
 +    default:
 +        ret = -EINVAL;
 +        error_setg(errp, "error while checking for known server (%d)", state);
          goto out;
      }
 +#else /* !HAVE_LIBSSH_0_8 */
 +    int state;
 -    home = getenv("HOME");
 -    if (home) {
 -        knh_file = g_strdup_printf("%s/.ssh/known_hosts", home);
 -    } else {
 -        knh_file = g_strdup_printf("/root/.ssh/known_hosts");
 -    }
 -
 -    /* Read all known hosts from OpenSSH-style known_hosts file. */
 -    libssh2_knownhost_readfile(knh, knh_file, LIBSSH2_KNOWNHOST_FILE_OPENSSH);
 +    state = ssh_is_server_known(s->session);
 +    trace_ssh_server_status(state);
 -    r = libssh2_knownhost_checkp(knh, host, port, hostkey, len,
 -                                 LIBSSH2_KNOWNHOST_TYPE_PLAIN|
 -                                 LIBSSH2_KNOWNHOST_KEYENC_RAW,
 -                                 &found);
 -    switch (r) {
 -    case LIBSSH2_KNOWNHOST_CHECK_MATCH:
 +    switch (state) {
 +    case SSH_SERVER_KNOWN_OK:
          /* OK */
 -        trace_ssh_check_host_key_knownhosts(found->key);
 +        trace_ssh_check_host_key_knownhosts();
          break;
 -    case LIBSSH2_KNOWNHOST_CHECK_MISMATCH:
 +    case SSH_SERVER_KNOWN_CHANGED:
          ret = -EINVAL;
 -        session_error_setg(errp, s,
 -                      "host key does not match the one in known_hosts"
 -                      " (found key %s)", found->key);
 +        error_setg(errp,
 +                   "host key does not match the one in known_hosts; this "
 +                   "may be a possible attack");
          goto out;
 -    case LIBSSH2_KNOWNHOST_CHECK_NOTFOUND:
 +    case SSH_SERVER_FOUND_OTHER:
          ret = -EINVAL;
 -        session_error_setg(errp, s, "no host key was found in known_hosts");
 +        error_setg(errp,
 +                   "host key for this server not found, another type exists");
 +        goto out;
 +    case SSH_SERVER_FILE_NOT_FOUND:
 +        ret = -ENOENT;
 +        error_setg(errp, "known_hosts file not found");
          goto out;
 -    case LIBSSH2_KNOWNHOST_CHECK_FAILURE:
 +    case SSH_SERVER_NOT_KNOWN:
          ret = -EINVAL;
 -        session_error_setg(errp, s,
 -                      "failure matching the host key with known_hosts");
 +        error_setg(errp, "no host key was found in known_hosts");
 +        goto out;
 +    case SSH_SERVER_ERROR:
 +        ret = -EINVAL;
 +        error_setg(errp, "server error");
          goto out;
      default:
          ret = -EINVAL;
 -        session_error_setg(errp, s, "unknown error matching the host key"
 -                      " with known_hosts (%d)", r);
 +        error_setg(errp, "error while checking for known server (%d)", state);
          goto out;
      }
 +#endif /* !HAVE_LIBSSH_0_8 */
      /* known_hosts checking successful. */
      ret = 0;
   out:
 -    if (knh != NULL) {
 -        libssh2_knownhost_free(knh);
 -    }
 -    g_free(knh_file);
      return ret;
  }
@@ -XXX,XX +XXX,XX @@ static int compare_fingerprint(const unsigned char *fingerprint, size_t len,
  static int
  check_host_key_hash(BDRVSSHState *s, const char *hash,
 -                    int hash_type, size_t fingerprint_len, Error **errp)
 +                    enum ssh_publickey_hash_type type, Error **errp)
  {
 -    const char *fingerprint;
 -
 -    fingerprint = libssh2_hostkey_hash(s->session, hash_type);
 -    if (!fingerprint) {
 +    int r;
 +    ssh_key pubkey;
 +    unsigned char *server_hash;
 +    size_t server_hash_len;
 +
 +#ifdef HAVE_LIBSSH_0_8
 +    r = ssh_get_server_publickey(s->session, &pubkey);
 +#else
 +    r = ssh_get_publickey(s->session, &pubkey);
 +#endif
 +    if (r != SSH_OK) {
          session_error_setg(errp, s, "failed to read remote host key");
          return -EINVAL;
      }
 -    if(compare_fingerprint((unsigned char *) fingerprint, fingerprint_len,
 -                           hash) != 0) {
 +    r = ssh_get_publickey_hash(pubkey, type, &server_hash, &server_hash_len);
 +    ssh_key_free(pubkey);
 +    if (r != 0) {
 +        session_error_setg(errp, s,
 +                           "failed reading the hash of the server SSH key");
 +        return -EINVAL;
 +    }
 +
 +    r = compare_fingerprint(server_hash, server_hash_len, hash);
 +    ssh_clean_pubkey_hash(&server_hash);
 +    if (r != 0) {
          error_setg(errp, "remote host key does not match host_key_check '%s'",
                     hash);
          return -EPERM;
@@ -XXX,XX +XXX,XX @@ check_host_key_hash(BDRVSSHState *s, const char *hash,
      return 0;
  }
 -static int check_host_key(BDRVSSHState *s, const char *host, int port,
 -                          SshHostKeyCheck *hkc, Error **errp)
 +static int check_host_key(BDRVSSHState *s, SshHostKeyCheck *hkc, Error **errp)
  {
      SshHostKeyCheckMode mode;
@@ -XXX,XX +XXX,XX @@ static int check_host_key(BDRVSSHState *s, const char *host, int port,
      case SSH_HOST_KEY_CHECK_MODE_HASH:
          if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_MD5) {
              return check_host_key_hash(s, hkc->u.hash.hash,
 -                                       LIBSSH2_HOSTKEY_HASH_MD5, 16, errp);
 +                                       SSH_PUBLICKEY_HASH_MD5, errp);
          } else if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_SHA1) {
              return check_host_key_hash(s, hkc->u.hash.hash,
 -                                       LIBSSH2_HOSTKEY_HASH_SHA1, 20, errp);
 +                                       SSH_PUBLICKEY_HASH_SHA1, errp);
          }
          g_assert_not_reached();
          break;
      case SSH_HOST_KEY_CHECK_MODE_KNOWN_HOSTS:
 -        return check_host_key_knownhosts(s, host, port, errp);
 +        return check_host_key_knownhosts(s, errp);
      default:
          g_assert_not_reached();
      }
@@ -XXX,XX +XXX,XX @@ static int check_host_key(BDRVSSHState *s, const char *host, int port,
      return -EINVAL;
  }
 -static int authenticate(BDRVSSHState *s, const char *user, Error **errp)
 +static int authenticate(BDRVSSHState *s, Error **errp)
  {
      int r, ret;
 -    const char *userauthlist;
 -    LIBSSH2_AGENT *agent = NULL;
 -    struct libssh2_agent_publickey *identity;
 -    struct libssh2_agent_publickey *prev_identity = NULL;
 +    int method;
 -    userauthlist = libssh2_userauth_list(s->session, user, strlen(user));
 -    if (strstr(userauthlist, "publickey") == NULL) {
 +    /* Try to authenticate with the "none" method. */
 +    r = ssh_userauth_none(s->session, NULL);
 +    if (r == SSH_AUTH_ERROR) {
          ret = -EPERM;
 -        error_setg(errp,
 -                "remote server does not support \"publickey\" authentication");
 +        session_error_setg(errp, s, "failed to authenticate using none "
 +                                    "authentication");
          goto out;
 -    }
 -
 -    /* Connect to ssh-agent and try each identity in turn. */
 -    agent = libssh2_agent_init(s->session);
 -    if (!agent) {
 -        ret = -EINVAL;
 -        session_error_setg(errp, s, "failed to initialize ssh-agent support");
 -        goto out;
 -    }
 -    if (libssh2_agent_connect(agent)) {
 -        ret = -ECONNREFUSED;
 -        session_error_setg(errp, s, "failed to connect to ssh-agent");
 -        goto out;
 -    }
 -    if (libssh2_agent_list_identities(agent)) {
 -        ret = -EINVAL;
 -        session_error_setg(errp, s,
 -                           "failed requesting identities from ssh-agent");
 +    } else if (r == SSH_AUTH_SUCCESS) {
 +        /* Authenticated! */
 +        ret = 0;
          goto out;
      }
 -    for(;;) {
 -        r = libssh2_agent_get_identity(agent, &identity, prev_identity);
 -        if (r == 1) {           /* end of list */
 -            break;
 -        }
 -        if (r < 0) {
 +    method = ssh_userauth_list(s->session, NULL);
 +    trace_ssh_auth_methods(method);
 +
 +    /*
 +     * Try to authenticate with publickey, using the ssh-agent
 +     * if available.
 +     */
 +    if (method & SSH_AUTH_METHOD_PUBLICKEY) {
 +        r = ssh_userauth_publickey_auto(s->session, NULL, NULL);
 +        if (r == SSH_AUTH_ERROR) {
              ret = -EINVAL;
 -            session_error_setg(errp, s,
 -                               "failed to obtain identity from ssh-agent");
 +            session_error_setg(errp, s, "failed to authenticate using "
 +                                        "publickey authentication");
              goto out;
 -        }
 -        r = libssh2_agent_userauth(agent, user, identity);
 -        if (r == 0) {
 +        } else if (r == SSH_AUTH_SUCCESS) {
              /* Authenticated! */
              ret = 0;
              goto out;
          }
 -        /* Failed to authenticate with this identity, try the next one. */
 -        prev_identity = identity;
      }
      ret = -EPERM;
@@ -XXX,XX +XXX,XX @@ static int authenticate(BDRVSSHState *s, const char *user, Error **errp)
                 "and the identities held by your ssh-agent");
   out:
 -    if (agent != NULL) {
 -        /* Note: libssh2 implementation implicitly calls
 -         * libssh2_agent_disconnect if necessary.
 -         */
 -        libssh2_agent_free(agent);
 -    }
 -
      return ret;
  }
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
                            int ssh_flags, int creat_mode, Error **errp)
  {
      int r, ret;
 -    long port = 0;
 +    unsigned int port = 0;
 +    int new_sock = -1;
      if (opts->has_user) {
          s->user = g_strdup(opts->user);
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
      s->inet = opts->server;
      opts->server = NULL;
 -    if (qemu_strtol(s->inet->port, NULL, 10, &port) < 0) {
 +    if (qemu_strtoui(s->inet->port, NULL, 10, &port) < 0) {
          error_setg(errp, "Use only numeric port value");
          ret = -EINVAL;
          goto err;
      }
      /* Open the socket and connect. */
 -    s->sock = inet_connect_saddr(s->inet, errp);
 -    if (s->sock < 0) {
 +    new_sock = inet_connect_saddr(s->inet, errp);
 +    if (new_sock < 0) {
          ret = -EIO;
          goto err;
      }
 +    /*
 +     * Try to disable the Nagle algorithm on TCP sockets to reduce latency,
 +     * but do not fail if it cannot be disabled.
 +     */
 +    r = socket_set_nodelay(new_sock);
 +    if (r < 0) {
 +        warn_report("can't set TCP_NODELAY for the ssh server %s: %s",
 +                    s->inet->host, strerror(errno));
 +    }
 +
      /* Create SSH session. */
 -    s->session = libssh2_session_init();
 +    s->session = ssh_new();
      if (!s->session) {
          ret = -EINVAL;
 -        session_error_setg(errp, s, "failed to initialize libssh2 session");
 +        session_error_setg(errp, s, "failed to initialize libssh session");
          goto err;
      }
 -#if TRACE_LIBSSH2 != 0
 -    libssh2_trace(s->session, TRACE_LIBSSH2);
 -#endif
 +    /*
 +     * Make sure we are in blocking mode during the connection and
 +     * authentication phases.
 +     */
 +    ssh_set_blocking(s->session, 1);
 -    r = libssh2_session_handshake(s->session, s->sock);
 -    if (r != 0) {
 +    r = ssh_options_set(s->session, SSH_OPTIONS_USER, s->user);
 +    if (r < 0) {
 +        ret = -EINVAL;
 +        session_error_setg(errp, s,
 +                           "failed to set the user in the libssh session");
 +        goto err;
 +    }
 +
 +    r = ssh_options_set(s->session, SSH_OPTIONS_HOST, s->inet->host);
 +    if (r < 0) {
 +        ret = -EINVAL;
 +        session_error_setg(errp, s,
 +                           "failed to set the host in the libssh session");
 +        goto err;
 +    }
 +
 +    if (port > 0) {
 +        r = ssh_options_set(s->session, SSH_OPTIONS_PORT, &port);
 +        if (r < 0) {
 +            ret = -EINVAL;
 +            session_error_setg(errp, s,
 +                               "failed to set the port in the libssh session");
 +            goto err;
 +        }
 +    }
 +
 +    r = ssh_options_set(s->session, SSH_OPTIONS_COMPRESSION, "none");
 +    if (r < 0) {
 +        ret = -EINVAL;
 +        session_error_setg(errp, s,
 +                           "failed to disable the compression in the libssh "
 +                           "session");
 +        goto err;
 +    }
 +
 +    /* Read ~/.ssh/config. */
 +    r = ssh_options_parse_config(s->session, NULL);
 +    if (r < 0) {
 +        ret = -EINVAL;
 +        session_error_setg(errp, s, "failed to parse ~/.ssh/config");
 +        goto err;
 +    }
 +
 +    r = ssh_options_set(s->session, SSH_OPTIONS_FD, &new_sock);
 +    if (r < 0) {
 +        ret = -EINVAL;
 +        session_error_setg(errp, s,
 +                           "failed to set the socket in the libssh session");
 +        goto err;
 +    }
 +    /* libssh took ownership of the socket. */
 +    s->sock = new_sock;
 +    new_sock = -1;
 +
 +    /* Connect. */
 +    r = ssh_connect(s->session);
 +    if (r != SSH_OK) {
          ret = -EINVAL;
          session_error_setg(errp, s, "failed to establish SSH session");
          goto err;
      }
      /* Check the remote host's key against known_hosts. */
 -    ret = check_host_key(s, s->inet->host, port, opts->host_key_check, errp);
 +    ret = check_host_key(s, opts->host_key_check, errp);
      if (ret < 0) {
          goto err;
      }
      /* Authenticate. */
 -    ret = authenticate(s, s->user, errp);
 +    ret = authenticate(s, errp);
      if (ret < 0) {
          goto err;
      }
      /* Start SFTP. */
 -    s->sftp = libssh2_sftp_init(s->session);
 +    s->sftp = sftp_new(s->session);
      if (!s->sftp) {
 -        session_error_setg(errp, s, "failed to initialize sftp handle");
 +        session_error_setg(errp, s, "failed to create sftp handle");
 +        ret = -EINVAL;
 +        goto err;
 +    }
 +
 +    r = sftp_init(s->sftp);
 +    if (r < 0) {
 +        sftp_error_setg(errp, s, "failed to initialize sftp handle");
          ret = -EINVAL;
          goto err;
      }
      /* Open the remote file. */
      trace_ssh_connect_to_ssh(opts->path, ssh_flags, creat_mode);
 -    s->sftp_handle = libssh2_sftp_open(s->sftp, opts->path, ssh_flags,
 -                                       creat_mode);
 +    s->sftp_handle = sftp_open(s->sftp, opts->path, ssh_flags, creat_mode);
      if (!s->sftp_handle) {
 -        session_error_setg(errp, s, "failed to open remote file '%s'",
 -                           opts->path);
 +        sftp_error_setg(errp, s, "failed to open remote file '%s'",
 +                        opts->path);
          ret = -EINVAL;
          goto err;
      }
 -    r = libssh2_sftp_fstat(s->sftp_handle, &s->attrs);
 -    if (r < 0) {
 +    /* Make sure the SFTP file is handled in blocking mode. */
 +    sftp_file_set_blocking(s->sftp_handle);
 +
 +    s->attrs = sftp_fstat(s->sftp_handle);
 +    if (!s->attrs) {
          sftp_error_setg(errp, s, "failed to read file attributes");
          return -EINVAL;
      }
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
      return 0;
   err:
 +    if (s->attrs) {
 +        sftp_attributes_free(s->attrs);
 +    }
 +    s->attrs = NULL;
      if (s->sftp_handle) {
 -        libssh2_sftp_close(s->sftp_handle);
 +        sftp_close(s->sftp_handle);
      }
      s->sftp_handle = NULL;
      if (s->sftp) {
 -        libssh2_sftp_shutdown(s->sftp);
 +        sftp_free(s->sftp);
      }
      s->sftp = NULL;
      if (s->session) {
 -        libssh2_session_disconnect(s->session,
 -                                   "from qemu ssh client: "
 -                                   "error opening connection");
 -        libssh2_session_free(s->session);
 +        ssh_disconnect(s->session);
 +        ssh_free(s->session);
      }
      s->session = NULL;
 +    s->sock = -1;
 +    if (new_sock >= 0) {
 +        close(new_sock);
 +    }
      return ret;
  }
@@ -XXX,XX +XXX,XX @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
      ssh_state_init(s);
 -    ssh_flags = LIBSSH2_FXF_READ;
 +    ssh_flags = 0;
      if (bdrv_flags & BDRV_O_RDWR) {
 -        ssh_flags |= LIBSSH2_FXF_WRITE;
 +        ssh_flags |= O_RDWR;
 +    } else {
 +        ssh_flags |= O_RDONLY;
      }
      opts = ssh_parse_options(options, errp);
@@ -XXX,XX +XXX,XX @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
      }
      /* Go non-blocking. */
 -    libssh2_session_set_blocking(s->session, 0);
 +    ssh_set_blocking(s->session, 0);
      qapi_free_BlockdevOptionsSsh(opts);
      return 0;
   err:
 -    if (s->sock >= 0) {
 -        close(s->sock);
 -    }
 -    s->sock = -1;
 -
      qapi_free_BlockdevOptionsSsh(opts);
      return ret;
@@ -XXX,XX +XXX,XX @@ static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp)
  {
      ssize_t ret;
      char c[1] = { '\0' };
 -    int was_blocking = libssh2_session_get_blocking(s->session);
 +    int was_blocking = ssh_is_blocking(s->session);
      /* offset must be strictly greater than the current size so we do
       * not overwrite anything */
 -    assert(offset > 0 && offset > s->attrs.filesize);
 +    assert(offset > 0 && offset > s->attrs->size);
 -    libssh2_session_set_blocking(s->session, 1);
 +    ssh_set_blocking(s->session, 1);
 -    libssh2_sftp_seek64(s->sftp_handle, offset - 1);
 -    ret = libssh2_sftp_write(s->sftp_handle, c, 1);
 +    sftp_seek64(s->sftp_handle, offset - 1);
 +    ret = sftp_write(s->sftp_handle, c, 1);
 -    libssh2_session_set_blocking(s->session, was_blocking);
 +    ssh_set_blocking(s->session, was_blocking);
      if (ret < 0) {
          sftp_error_setg(errp, s, "Failed to grow file");
          return -EIO;
      }
 -    s->attrs.filesize = offset;
 +    s->attrs->size = offset;
      return 0;
  }
@@ -XXX,XX +XXX,XX @@ static int ssh_co_create(BlockdevCreateOptions *options, Error **errp)
      ssh_state_init(&s);
      ret = connect_to_ssh(&s, opts->location,
 -                         LIBSSH2_FXF_READ|LIBSSH2_FXF_WRITE|
 -                         LIBSSH2_FXF_CREAT|LIBSSH2_FXF_TRUNC,
 +                         O_RDWR | O_CREAT | O_TRUNC,
 , errp);
      if (ret < 0) {
          goto fail;
@@ -XXX,XX +XXX,XX @@ static int ssh_has_zero_init(BlockDriverState *bs)
      /* Assume false, unless we can positively prove it's true. */
      int has_zero_init = 0;
 -    if (s->attrs.flags & LIBSSH2_SFTP_ATTR_PERMISSIONS) {
 -        if (s->attrs.permissions & LIBSSH2_SFTP_S_IFREG) {
 -            has_zero_init = 1;
 -        }
 +    if (s->attrs->type == SSH_FILEXFER_TYPE_REGULAR) {
 +        has_zero_init = 1;
      }
      return has_zero_init;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void co_yield(BDRVSSHState *s, BlockDriverState *bs)
          .co = qemu_coroutine_self()
      };
 -    r = libssh2_session_block_directions(s->session);
 +    r = ssh_get_poll_flags(s->session);
 -    if (r & LIBSSH2_SESSION_BLOCK_INBOUND) {
 +    if (r & SSH_READ_PENDING) {
          rd_handler = restart_coroutine;
      }
 -    if (r & LIBSSH2_SESSION_BLOCK_OUTBOUND) {
 +    if (r & SSH_WRITE_PENDING) {
          wr_handler = restart_coroutine;
      }
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void co_yield(BDRVSSHState *s, BlockDriverState *bs)
      trace_ssh_co_yield_back(s->sock);
  }
 -/* SFTP has a function `libssh2_sftp_seek64' which seeks to a position
 - * in the remote file.  Notice that it just updates a field in the
 - * sftp_handle structure, so there is no network traffic and it cannot
 - * fail.
 - *
 - * However, `libssh2_sftp_seek64' does have a catastrophic effect on
 - * performance since it causes the handle to throw away all in-flight
 - * reads and buffered readahead data.  Therefore this function tries
 - * to be intelligent about when to call the underlying libssh2 function.
 - */
 -#define SSH_SEEK_WRITE 0
 -#define SSH_SEEK_READ  1
 -#define SSH_SEEK_FORCE 2
 -
 -static void ssh_seek(BDRVSSHState *s, int64_t offset, int flags)
 -{
 -    bool op_read = (flags & SSH_SEEK_READ) != 0;
 -    bool force = (flags & SSH_SEEK_FORCE) != 0;
 -
 -    if (force || op_read != s->offset_op_read || offset != s->offset) {
 -        trace_ssh_seek(offset);
 -        libssh2_sftp_seek64(s->sftp_handle, offset);
 -        s->offset = offset;
 -        s->offset_op_read = op_read;
 -    }
 -}
 -
  static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
                                   int64_t offset, size_t size,
                                   QEMUIOVector *qiov)
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
      trace_ssh_read(offset, size);
 -    ssh_seek(s, offset, SSH_SEEK_READ);
 +    trace_ssh_seek(offset);
 +    sftp_seek64(s->sftp_handle, offset);
      /* This keeps track of the current iovec element ('i'), where we
       * will write to next ('buf'), and the end of the current iovec
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
      buf = i->iov_base;
      end_of_vec = i->iov_base + i->iov_len;
 -    /* libssh2 has a hard-coded limit of 2000 bytes per request,
 -     * although it will also do readahead behind our backs.  Therefore
 -     * we may have to do repeated reads here until we have read 'size'
 -     * bytes.
 -     */
      for (got = 0; got < size; ) {
 +        size_t request_read_size;
      again:
 -        trace_ssh_read_buf(buf, end_of_vec - buf);
 -        r = libssh2_sftp_read(s->sftp_handle, buf, end_of_vec - buf);
 -        trace_ssh_read_return(r);
 +        /*
 +         * The size of SFTP packets is limited to 32K bytes, so limit
 +         * the amount of data requested to 16K, as libssh currently
 +         * does not handle multiple requests on its own.
 +         */
 +        request_read_size = MIN(end_of_vec - buf, 16384);
 +        trace_ssh_read_buf(buf, end_of_vec - buf, request_read_size);
 +        r = sftp_read(s->sftp_handle, buf, request_read_size);
 +        trace_ssh_read_return(r, sftp_get_error(s->sftp));
 -        if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
 +        if (r == SSH_AGAIN) {
              co_yield(s, bs);
              goto again;
          }
 -        if (r < 0) {
 -            sftp_error_trace(s, "read");
 -            s->offset = -1;
 -            return -EIO;
 -        }
 -        if (r == 0) {
 +        if (r == SSH_EOF || (r == 0 && sftp_get_error(s->sftp) == SSH_FX_EOF)) {
              /* EOF: Short read so pad the buffer with zeroes and return it. */
              qemu_iovec_memset(qiov, got, 0, size - got);
              return 0;
          }
 +        if (r <= 0) {
 +            sftp_error_trace(s, "read");
 +            return -EIO;
 +        }
          got += r;
          buf += r;
 -        s->offset += r;
          if (buf >= end_of_vec && got < size) {
              i++;
              buf = i->iov_base;
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
      trace_ssh_write(offset, size);
 -    ssh_seek(s, offset, SSH_SEEK_WRITE);
 +    trace_ssh_seek(offset);
 +    sftp_seek64(s->sftp_handle, offset);
      /* This keeps track of the current iovec element ('i'), where we
       * will read from next ('buf'), and the end of the current iovec
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
      end_of_vec = i->iov_base + i->iov_len;
      for (written = 0; written < size; ) {
 +        size_t request_write_size;
      again:
 -        trace_ssh_write_buf(buf, end_of_vec - buf);
 -        r = libssh2_sftp_write(s->sftp_handle, buf, end_of_vec - buf);
 -        trace_ssh_write_return(r);
 +        /*
 +         * Avoid too large data packets, as libssh currently does not
 +         * handle multiple requests on its own.
 +         */
 +        request_write_size = MIN(end_of_vec - buf, 131072);
 +        trace_ssh_write_buf(buf, end_of_vec - buf, request_write_size);
 +        r = sftp_write(s->sftp_handle, buf, request_write_size);
 +        trace_ssh_write_return(r, sftp_get_error(s->sftp));
 -        if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
 +        if (r == SSH_AGAIN) {
              co_yield(s, bs);
              goto again;
          }
          if (r < 0) {
              sftp_error_trace(s, "write");
 -            s->offset = -1;
              return -EIO;
          }
 -        /* The libssh2 API is very unclear about this.  A comment in
 -         * the code says "nothing was acked, and no EAGAIN was
 -         * received!" which apparently means that no data got sent
 -         * out, and the underlying channel didn't return any EAGAIN
 -         * indication.  I think this is a bug in either libssh2 or
 -         * OpenSSH (server-side).  In any case, forcing a seek (to
 -         * discard libssh2 internal buffers), and then trying again
 -         * works for me.
 -         */
 -        if (r == 0) {
 -            ssh_seek(s, offset + written, SSH_SEEK_WRITE|SSH_SEEK_FORCE);
 -            co_yield(s, bs);
 -            goto again;
 -        }
          written += r;
          buf += r;
 -        s->offset += r;
          if (buf >= end_of_vec && written < size) {
              i++;
              buf = i->iov_base;
              end_of_vec = i->iov_base + i->iov_len;
          }
 -        if (offset + written > s->attrs.filesize)
 -            s->attrs.filesize = offset + written;
 +        if (offset + written > s->attrs->size) {
 +            s->attrs->size = offset + written;
 +        }
      }
      return 0;
@@ -XXX,XX +XXX,XX @@ static void unsafe_flush_warning(BDRVSSHState *s, const char *what)
      }
  }
 -#ifdef HAS_LIBSSH2_SFTP_FSYNC
 +#ifdef HAVE_LIBSSH_0_8
  static coroutine_fn int ssh_flush(BDRVSSHState *s, BlockDriverState *bs)
  {
      int r;
-@@ -XXX,XX +XXX,XX @@ static void curl_multi_do_locked(CURLState *s)
+     trace_ssh_flush();
++
- static void curl_multi_do(void *arg)
++    if (!sftp_extension_supported(s->sftp, "fsync@openssh.com", "1")) {
 +        unsafe_flush_warning(s, "OpenSSH >= 6.3");
 +        return 0;
 +    }
   again:
 -    r = libssh2_sftp_fsync(s->sftp_handle);
 -    if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
 +    r = sftp_fsync(s->sftp_handle);
 +    if (r == SSH_AGAIN) {
          co_yield(s, bs);
          goto again;
      }
 -    if (r == LIBSSH2_ERROR_SFTP_PROTOCOL &&
 -        libssh2_sftp_last_error(s->sftp) == LIBSSH2_FX_OP_UNSUPPORTED) {
 -        unsafe_flush_warning(s, "OpenSSH >= 6.3");
 -        return 0;
 -    }
      if (r < 0) {
          sftp_error_trace(s, "fsync");
          return -EIO;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_co_flush(BlockDriverState *bs)
      return ret;
  }
 -#else /* !HAS_LIBSSH2_SFTP_FSYNC */
 +#else /* !HAVE_LIBSSH_0_8 */
  static coroutine_fn int ssh_co_flush(BlockDriverState *bs)
  {
--    CURLState *s = (CURLState *)arg;
+     BDRVSSHState *s = bs->opaque;
-+    CURLSocket *socket = arg;
-+    BDRVCURLState *s = socket->state->s;
+-    unsafe_flush_warning(s, "libssh2 >= 1.4.4");
++    unsafe_flush_warning(s, "libssh >= 0.8.0");
--    qemu_mutex_lock(&s->s->mutex);
+     return 0;
 -    curl_multi_do_locked(s);
 -    curl_multi_check_completion(s->s);
 -    qemu_mutex_unlock(&s->s->mutex);
 +    qemu_mutex_lock(&s->mutex);
 +    curl_multi_do_locked(socket);
 +    curl_multi_check_completion(s);
 +    qemu_mutex_unlock(&s->mutex);
  }
- static void curl_multi_timeout_do(void *arg)
+-#endif /* !HAS_LIBSSH2_SFTP_FSYNC */
 +#endif /* !HAVE_LIBSSH_0_8 */
  static int64_t ssh_getlength(BlockDriverState *bs)
  {
      BDRVSSHState *s = bs->opaque;
      int64_t length;
 -    /* Note we cannot make a libssh2 call here. */
 -    length = (int64_t) s->attrs.filesize;
 +    /* Note we cannot make a libssh call here. */
 +    length = (int64_t) s->attrs->size;
      trace_ssh_getlength(length);
      return length;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn ssh_co_truncate(BlockDriverState *bs, int64_t offset,
          return -ENOTSUP;
      }
 -    if (offset < s->attrs.filesize) {
 +    if (offset < s->attrs->size) {
          error_setg(errp, "ssh driver does not support shrinking files");
          return -ENOTSUP;
      }
 -    if (offset == s->attrs.filesize) {
 +    if (offset == s->attrs->size) {
          return 0;
      }
@@ -XXX,XX +XXX,XX @@ static void bdrv_ssh_init(void)
  {
      int r;
 -    r = libssh2_init(0);
 +    r = ssh_init();
      if (r != 0) {
 -        fprintf(stderr, "libssh2 initialization failed, %d\n", r);
 +        fprintf(stderr, "libssh initialization failed, %d\n", r);
          exit(EXIT_FAILURE);
      }
 +#if TRACE_LIBSSH != 0
 +    ssh_set_log_level(TRACE_LIBSSH);
 +#endif
 +
      bdrv_register(&bdrv_ssh);
  }
 diff --git a/.travis.yml b/.travis.yml
 index XXXXXXX..XXXXXXX 100644
 --- a/.travis.yml
 +++ b/.travis.yml
@@ -XXX,XX +XXX,XX @@ addons:
        - libseccomp-dev
        - libspice-protocol-dev
        - libspice-server-dev
 -      - libssh2-1-dev
 +      - libssh-dev
        - liburcu-dev
        - libusb-1.0-0-dev
        - libvte-2.91-dev
@@ -XXX,XX +XXX,XX @@ matrix:
              - libseccomp-dev
              - libspice-protocol-dev
              - libspice-server-dev
 -            - libssh2-1-dev
 +            - libssh-dev
              - liburcu-dev
              - libusb-1.0-0-dev
              - libvte-2.91-dev
 diff --git a/block/trace-events b/block/trace-events
 index XXXXXXX..XXXXXXX 100644
 --- a/block/trace-events
 +++ b/block/trace-events
@@ -XXX,XX +XXX,XX @@ nbd_client_connect_success(const char *export_name) "export '%s'"
  # ssh.c
  ssh_restart_coroutine(void *co) "co=%p"
  ssh_flush(void) "fsync"
 -ssh_check_host_key_knownhosts(const char *key) "host key OK: %s"
 +ssh_check_host_key_knownhosts(void) "host key OK"
  ssh_connect_to_ssh(char *path, int flags, int mode) "opening file %s flags=0x%x creat_mode=0%o"
  ssh_co_yield(int sock, void *rd_handler, void *wr_handler) "s->sock=%d rd_handler=%p wr_handler=%p"
  ssh_co_yield_back(int sock) "s->sock=%d - back"
  ssh_getlength(int64_t length) "length=%" PRIi64
  ssh_co_create_opts(uint64_t size) "total_size=%" PRIu64
  ssh_read(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
 -ssh_read_buf(void *buf, size_t size) "sftp_read buf=%p size=%zu"
 -ssh_read_return(ssize_t ret) "sftp_read returned %zd"
 +ssh_read_buf(void *buf, size_t size, size_t actual_size) "sftp_read buf=%p size=%zu (actual size=%zu)"
 +ssh_read_return(ssize_t ret, int sftp_err) "sftp_read returned %zd (sftp error=%d)"
  ssh_write(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
 -ssh_write_buf(void *buf, size_t size) "sftp_write buf=%p size=%zu"
 -ssh_write_return(ssize_t ret) "sftp_write returned %zd"
 +ssh_write_buf(void *buf, size_t size, size_t actual_size) "sftp_write buf=%p size=%zu (actual size=%zu)"
 +ssh_write_return(ssize_t ret, int sftp_err) "sftp_write returned %zd (sftp error=%d)"
  ssh_seek(int64_t offset) "seeking to offset=%" PRIi64
 +ssh_auth_methods(int methods) "auth methods=0x%x"
 +ssh_server_status(int status) "server status=%d"
  # curl.c
  curl_timer_cb(long timeout_ms) "timer callback timeout_ms %ld"
@@ -XXX,XX +XXX,XX @@ sheepdog_snapshot_create(const char *sn_name, const char *id) "%s %s"
  sheepdog_snapshot_create_inode(const char *name, uint32_t snap, uint32_t vdi) "s->inode: name %s snap_id 0x%" PRIx32 " vdi 0x%" PRIx32
  # ssh.c
 -sftp_error(const char *op, const char *ssh_err, int ssh_err_code, unsigned long sftp_err_code) "%s failed: %s (libssh2 error code: %d, sftp error code: %lu)"
 +sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
 diff --git a/docs/qemu-block-drivers.texi b/docs/qemu-block-drivers.texi
 index XXXXXXX..XXXXXXX 100644
 --- a/docs/qemu-block-drivers.texi
 +++ b/docs/qemu-block-drivers.texi
@@ -XXX,XX +XXX,XX @@ print a warning when @code{fsync} is not supported:
  warning: ssh server @code{ssh.example.com:22} does not support fsync
 -With sufficiently new versions of libssh2 and OpenSSH, @code{fsync} is
 +With sufficiently new versions of libssh and OpenSSH, @code{fsync} is
  supported.
  @node disk_images_nvme
 diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/docker/dockerfiles/debian-win32-cross.docker
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/docker/dockerfiles/debian-win32-cross.docker
 +++ b/tests/docker/dockerfiles/debian-win32-cross.docker
@@ -XXX,XX +XXX,XX @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
          mxe-$TARGET-w64-mingw32.shared-curl \
          mxe-$TARGET-w64-mingw32.shared-glib \
          mxe-$TARGET-w64-mingw32.shared-libgcrypt \
 -        mxe-$TARGET-w64-mingw32.shared-libssh2 \
          mxe-$TARGET-w64-mingw32.shared-libusb1 \
          mxe-$TARGET-w64-mingw32.shared-lzo \
          mxe-$TARGET-w64-mingw32.shared-nettle \
 diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/docker/dockerfiles/debian-win64-cross.docker
 +++ b/tests/docker/dockerfiles/debian-win64-cross.docker
@@ -XXX,XX +XXX,XX @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
          mxe-$TARGET-w64-mingw32.shared-curl \
          mxe-$TARGET-w64-mingw32.shared-glib \
          mxe-$TARGET-w64-mingw32.shared-libgcrypt \
 -        mxe-$TARGET-w64-mingw32.shared-libssh2 \
          mxe-$TARGET-w64-mingw32.shared-libusb1 \
          mxe-$TARGET-w64-mingw32.shared-lzo \
          mxe-$TARGET-w64-mingw32.shared-nettle \
 diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/docker/dockerfiles/fedora.docker
 +++ b/tests/docker/dockerfiles/fedora.docker
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
      libpng-devel \
      librbd-devel \
      libseccomp-devel \
 -    libssh2-devel \
 +    libssh-devel \
      libubsan \
      libusbx-devel \
      libxml2-devel \
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
      mingw32-gtk3 \
      mingw32-libjpeg-turbo \
      mingw32-libpng \
 -    mingw32-libssh2 \
      mingw32-libtasn1 \
      mingw32-nettle \
      mingw32-pixman \
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
      mingw64-gtk3 \
      mingw64-libjpeg-turbo \
      mingw64-libpng \
 -    mingw64-libssh2 \
      mingw64-libtasn1 \
      mingw64-nettle \
      mingw64-pixman \
 diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/docker/dockerfiles/ubuntu.docker
 +++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES flex bison \
      libsnappy-dev \
      libspice-protocol-dev \
      libspice-server-dev \
 -    libssh2-1-dev \
 +    libssh-dev \
      libusb-1.0-0-dev \
      libusbredirhost-dev \
      libvdeplug-dev \
 diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/docker/dockerfiles/ubuntu1804.docker
 +++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES flex bison \
      libsnappy-dev \
      libspice-protocol-dev \
      libspice-server-dev \
 -    libssh2-1-dev \
 +    libssh-dev \
      libusb-1.0-0-dev \
      libusbredirhost-dev \
      libvdeplug-dev \
 diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
 index XXXXXXX..XXXXXXX 100755
 --- a/tests/qemu-iotests/207
 +++ b/tests/qemu-iotests/207
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
      iotests.img_info_log(remote_path)
 -    md5_key = subprocess.check_output(
 -        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
 -        'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
 -        shell=True).rstrip().decode('ascii')
 +    keys = subprocess.check_output(
 +        'ssh-keyscan 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
 +        'cut -d" " -f3',
 +        shell=True).rstrip().decode('ascii').split('\n')
 +
 +    # Mappings of base64 representations to digests
 +    md5_keys = {}
 +    sha1_keys = {}
 +
 +    for key in keys:
 +        md5_keys[key] = subprocess.check_output(
 +            'echo %s | base64 -d | md5sum -b | cut -d" " -f1' % key,
 +            shell=True).rstrip().decode('ascii')
 +
 +        sha1_keys[key] = subprocess.check_output(
 +            'echo %s | base64 -d | sha1sum -b | cut -d" " -f1' % key,
 +            shell=True).rstrip().decode('ascii')
      vm.launch()
 +
 +    # Find correct key first
 +    matching_key = None
 +    for key in keys:
 +        result = vm.qmp('blockdev-add',
 +                        driver='ssh', node_name='node0', path=disk_path,
 +                        server={
 +                             'host': '127.0.0.1',
 +                             'port': '22',
 +                        }, host_key_check={
 +                             'mode': 'hash',
 +                             'type': 'md5',
 +                             'hash': md5_keys[key],
 +                        })
 +
 +        if 'error' not in result:
 +            vm.qmp('blockdev-del', node_name='node0')
 +            matching_key = key
 +            break
 +
 +    if matching_key is None:
 +        vm.shutdown()
 +        iotests.notrun('Did not find a key that fits 127.0.0.1')
 +
      blockdev_create(vm, { 'driver': 'ssh',
                            'location': {
                                'path': disk_path,
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
                                'host-key-check': {
                                    'mode': 'hash',
                                    'type': 'md5',
 -                                  'hash': md5_key,
 +                                  'hash': md5_keys[matching_key],
                                }
                            },
                            'size': 8388608 })
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
      iotests.img_info_log(remote_path)
 -    sha1_key = subprocess.check_output(
 -        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
 -        'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
 -        shell=True).rstrip().decode('ascii')
 -
      vm.launch()
      blockdev_create(vm, { 'driver': 'ssh',
                            'location': {
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
                                'host-key-check': {
                                    'mode': 'hash',
                                    'type': 'sha1',
 -                                  'hash': sha1_key,
 +                                  'hash': sha1_keys[matching_key],
                                }
                            },
                            'size': 4194304 })
 diff --git a/tests/qemu-iotests/207.out b/tests/qemu-iotests/207.out
 index XXXXXXX..XXXXXXX 100644
 --- a/tests/qemu-iotests/207.out
 +++ b/tests/qemu-iotests/207.out
@@ -XXX,XX +XXX,XX @@ virtual size: 4 MiB (4194304 bytes)
  {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "ssh", "location": {"host-key-check": {"mode": "none"}, "path": "/this/is/not/an/existing/path", "server": {"host": "127.0.0.1", "port": "22"}}, "size": 4194304}}}
  {"return": {}}
 -Job failed: failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31)
 +Job failed: failed to open remote file '/this/is/not/an/existing/path': SFTP server: No such file (libssh error code: 1, sftp error code: 2)
  {"execute": "job-dismiss", "arguments": {"id": "job0"}}
  {"return": {}}
 --
 .21.0

-[Qemu-devel] [PULL 10/16] curl: Report only ready sockets
+Deleted patch
-Instead of reporting all sockets to cURL, only report the one that has
-caused curl_multi_do_locked() to be called.  This lets us get rid of the
-QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are
-only safe when the current element is removed in each iteration.  If it
-possible for the list to be concurrently modified, we cannot guarantee
-that only the current element will be removed.  Therefore, we must not
-use QLIST_FOREACH_SAFE() here.
-Fixes: ff5ca1664af85b24a4180d595ea6873fd3deac57
-Cc: qemu-stable@nongnu.org
-Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190910124136.10565-6-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
-Reviewed-by: John Snow <jsnow@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- block/curl.c | 17 ++++++-----------
-file changed, 6 insertions(+), 11 deletions(-)
-diff --git a/block/curl.c b/block/curl.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
-+++ b/block/curl.c
-@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
- }
- /* Called with s->mutex held.  */
--static void curl_multi_do_locked(CURLSocket *ready_socket)
-+static void curl_multi_do_locked(CURLSocket *socket)
- {
--    CURLSocket *socket, *next_socket;
--    CURLState *s = ready_socket->state;
-+    BDRVCURLState *s = socket->state->s;
-     int running;
-     int r;
--    if (!s->s->multi) {
-+    if (!s->multi) {
-         return;
-     }
--    /* Need to use _SAFE because curl_multi_socket_action() may trigger
--     * curl_sock_cb() which might modify this list */
--    QLIST_FOREACH_SAFE(socket, &s->sockets, next, next_socket) {
--        do {
--            r = curl_multi_socket_action(s->s->multi, socket->fd, 0, &running);
--        } while (r == CURLM_CALL_MULTI_PERFORM);
--    }
-+    do {
-+        r = curl_multi_socket_action(s->multi, socket->fd, 0, &running);
-+    } while (r == CURLM_CALL_MULTI_PERFORM);
- }
- static void curl_multi_do(void *arg)
---
-.21.0

-[Qemu-devel] [PULL 11/16] curl: Handle success in multi_check_completion
+Deleted patch
-Background: As of cURL 7.59.0, it verifies that several functions are
-not called from within a callback.  Among these functions is
-curl_multi_add_handle().
-curl_read_cb() is a callback from cURL and not a coroutine.  Waking up
-acb->co will lead to entering it then and there, which means the current
-request will settle and the caller (if it runs in the same coroutine)
-may then issue the next request.  In such a case, we will enter
-curl_setup_preadv() effectively from within curl_read_cb().
-Calling curl_multi_add_handle() will then fail and the new request will
-not be processed.
-Fix this by not letting curl_read_cb() wake up acb->co.  Instead, leave
-the whole business of settling the AIOCB objects to
-curl_multi_check_completion() (which is called from our timer callback
-and our FD handler, so not from any cURL callbacks).
-Reported-by: Natalie Gavrielov <ngavrilo@redhat.com>
-Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1740193
-Cc: qemu-stable@nongnu.org
-Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190910124136.10565-7-mreitz@redhat.com
-Reviewed-by: John Snow <jsnow@redhat.com>
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
-Signed-off-by: Max Reitz <mreitz@redhat.com>
----
- block/curl.c | 69 ++++++++++++++++++++++------------------------------
-file changed, 29 insertions(+), 40 deletions(-)
-diff --git a/block/curl.c b/block/curl.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/curl.c
-+++ b/block/curl.c
-@@ -XXX,XX +XXX,XX @@ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
- {
-     CURLState *s = ((CURLState*)opaque);
-     size_t realsize = size * nmemb;
--    int i;
-     trace_curl_read_cb(realsize);
-@@ -XXX,XX +XXX,XX @@ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
-     memcpy(s->orig_buf + s->buf_off, ptr, realsize);
-     s->buf_off += realsize;
--    for(i=0; i<CURL_NUM_ACB; i++) {
--        CURLAIOCB *acb = s->acb[i];
--
--        if (!acb)
--            continue;
--
--        if ((s->buf_off >= acb->end)) {
--            size_t request_length = acb->bytes;
--
--            qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start,
--                                acb->end - acb->start);
--
--            if (acb->end - acb->start < request_length) {
--                size_t offset = acb->end - acb->start;
--                qemu_iovec_memset(acb->qiov, offset, 0,
--                                  request_length - offset);
--            }
--
--            acb->ret = 0;
--            s->acb[i] = NULL;
--            qemu_mutex_unlock(&s->s->mutex);
--            aio_co_wake(acb->co);
--            qemu_mutex_lock(&s->s->mutex);
--        }
--    }
--
- read_end:
-     /* curl will error out if we do not return this value */
-     return size * nmemb;
-@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
-             break;
-         if (msg->msg == CURLMSG_DONE) {
-+            int i;
-             CURLState *state = NULL;
-+            bool error = msg->data.result != CURLE_OK;
-+
-             curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE,
-                               (char **)&state);
--            /* ACBs for successful messages get completed in curl_read_cb */
--            if (msg->data.result != CURLE_OK) {
--                int i;
-+            if (error) {
-                 static int errcount = 100;
-                 /* Don't lose the original error message from curl, since
-@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
-                         error_report("curl: further errors suppressed");
-                     }
-                 }
-+            }
--                for (i = 0; i < CURL_NUM_ACB; i++) {
--                    CURLAIOCB *acb = state->acb[i];
-+            for (i = 0; i < CURL_NUM_ACB; i++) {
-+                CURLAIOCB *acb = state->acb[i];
--                    if (acb == NULL) {
--                        continue;
--                    }
-+                if (acb == NULL) {
-+                    continue;
-+                }
-+
-+                if (!error) {
-+                    /* Assert that we have read all data */
-+                    assert(state->buf_off >= acb->end);
-+
-+                    qemu_iovec_from_buf(acb->qiov, 0,
-+                                        state->orig_buf + acb->start,
-+                                        acb->end - acb->start);
--                    acb->ret = -EIO;
--                    state->acb[i] = NULL;
--                    qemu_mutex_unlock(&s->mutex);
--                    aio_co_wake(acb->co);
--                    qemu_mutex_lock(&s->mutex);
-+                    if (acb->end - acb->start < acb->bytes) {
-+                        size_t offset = acb->end - acb->start;
-+                        qemu_iovec_memset(acb->qiov, offset, 0,
-+                                          acb->bytes - offset);
-+                    }
-                 }
-+
-+                acb->ret = error ? -EIO : 0;
-+                state->acb[i] = NULL;
-+                qemu_mutex_unlock(&s->mutex);
-+                aio_co_wake(acb->co);
-+                qemu_mutex_lock(&s->mutex);
-             }
-             curl_clean_state(state);
---
-.21.0

-[Qemu-devel] [PULL 12/16] curl: Check curl_multi_add_handle()'s return code
+[Qemu-devel] [PULL v2 8/8] iotests: Fix 205 for concurrent runs
-If we had done that all along, debugging would have been much simpler.
+Tests should place their files into the test directory.  This includes
-(Also, I/O errors are better than hangs.)
+Unix sockets.  205 currently fails to do so, which prevents it from
 being run concurrently.
 Signed-off-by: Max Reitz <mreitz@redhat.com>
-Message-id: 20190910124136.10565-8-mreitz@redhat.com
+Message-id: 20190618210238.9524-1-mreitz@redhat.com
-Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
+Reviewed-by: Eric Blake <eblake@redhat.com>
 Reviewed-by: John Snow <jsnow@redhat.com>
 Signed-off-by: Max Reitz <mreitz@redhat.com>
 ---
- block/curl.c | 8 +++++++-
+ tests/qemu-iotests/205 | 2 +-
-file changed, 7 insertions(+), 1 deletion(-)
+file changed, 1 insertion(+), 1 deletion(-)
-diff --git a/block/curl.c b/block/curl.c
+diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205
-index XXXXXXX..XXXXXXX 100644
+index XXXXXXX..XXXXXXX 100755
---- a/block/curl.c
+--- a/tests/qemu-iotests/205
-+++ b/block/curl.c
++++ b/tests/qemu-iotests/205
-@@ -XXX,XX +XXX,XX @@ static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb)
+@@ -XXX,XX +XXX,XX @@ import iotests
-     trace_curl_setup_preadv(acb->bytes, start, state->range);
+ import time
-     curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
+ from iotests import qemu_img_create, qemu_io, filter_qemu_io, QemuIoInteractive
--    curl_multi_add_handle(s->multi, state->curl);
+-nbd_sock = 'nbd_sock'
-+    if (curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
++nbd_sock = os.path.join(iotests.test_dir, 'nbd_sock')
-+        state->acb[0] = NULL;
+ nbd_uri = 'nbd+unix:///exp?socket=' + nbd_sock
-+        acb->ret = -EIO;
+ disk = os.path.join(iotests.test_dir, 'disk')
-+
 +        curl_clean_state(state);
 +        goto out;
 +    }
      /* Tell curl it needs to kick things off */
      curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);
 --
 .21.0

The following changes since commit dd25f97c66a75d1508f1d4c6478ed2c95bec428f:

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190913' into staging (2019-09-16 10:15:15 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2019-09-16

for you to fetch changes up to 1825cc0783ccf0ec5d9f0b225a99b340bdd4c68f:

qemu-iotests: Add test for bz #1745922 (2019-09-16 15:37:12 +0200)

----------------------------------------------------------------
Block patches:
- Fix for block jobs when used with I/O threads
- Fix for a corruption when using qcow2's LUKS encryption mode
- cURL fix
- check-block.sh cleanups (for make check)
- Refactoring

----------------------------------------------------------------
Max Reitz (7):
  curl: Keep pointer to the CURLState in CURLSocket
  curl: Keep *socket until the end of curl_sock_cb()
  curl: Check completion in curl_multi_do()
  curl: Pass CURLSocket to curl_multi_do()
  curl: Report only ready sockets
  curl: Handle success in multi_check_completion
  curl: Check curl_multi_add_handle()'s return code

Maxim Levitsky (3):
  block/qcow2: Fix corruption introduced by commit 8ac0f15f335
  block/qcow2: refactor encryption code
  qemu-iotests: Add test for bz #1745922

Nir Soffer (2):
  block: Use QEMU_IS_ALIGNED
  block: Remove unused masks

Sergio Lopez (1):
  blockjob: update nodes head while removing all bdrv

Thomas Huth (2):
  tests/qemu-iotests/check: Replace "tests" with "iotests" in final
    status text
  tests/Makefile: Do not print the name of the check-block.sh shell
    script

Vladimir Sementsov-Ogievskiy (1):
  tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache

-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Replace instances of:

(n & (BDRV_SECTOR_SIZE - 1)) == 0

And:

(n & ~BDRV_SECTOR_MASK) == 0

With:

QEMU_IS_ALIGNED(n, BDRV_SECTOR_SIZE)

Which reveals the intent of the code better, and makes it easier to
locate the code checking alignment.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827185913.27427-2-nsoffer@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/bochs.c         | 4 ++--
 block/cloop.c         | 4 ++--
 block/dmg.c           | 4 ++--
 block/io.c            | 8 ++++----
 block/qcow2-cluster.c | 4 ++--
 block/qcow2.c         | 4 ++--
 block/vvfat.c         | 8 ++++----
 qemu-img.c            | 2 +-
 8 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/block/bochs.c b/block/bochs.c
index XXXXXXX..XXXXXXX 100644
--- a/block/bochs.c
+++ b/block/bochs.c
@@ -XXX,XX +XXX,XX @@ bochs_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     QEMUIOVector local_qiov;
     int ret;
 
-    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
 
     qemu_iovec_init(&local_qiov, qiov->niov);
     qemu_co_mutex_lock(&s->lock);
diff --git a/block/cloop.c b/block/cloop.c
index XXXXXXX..XXXXXXX 100644
--- a/block/cloop.c
+++ b/block/cloop.c
@@ -XXX,XX +XXX,XX @@ cloop_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
     int ret, i;
 
-    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
 
     qemu_co_mutex_lock(&s->lock);
 
diff --git a/block/dmg.c b/block/dmg.c
index XXXXXXX..XXXXXXX 100644
--- a/block/dmg.c
+++ b/block/dmg.c
@@ -XXX,XX +XXX,XX @@ dmg_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
     int ret, i;
 
-    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
 
     qemu_co_mutex_lock(&s->lock);
 
diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_driver_preadv(BlockDriverState *bs,
     sector_num = offset >> BDRV_SECTOR_BITS;
     nb_sectors = bytes >> BDRV_SECTOR_BITS;
 
-    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
     assert(bytes <= BDRV_REQUEST_MAX_BYTES);
     assert(drv->bdrv_co_readv);
 
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn bdrv_driver_pwritev(BlockDriverState *bs,
     sector_num = offset >> BDRV_SECTOR_BITS;
     nb_sectors = bytes >> BDRV_SECTOR_BITS;
 
-    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
     assert(bytes <= BDRV_REQUEST_MAX_BYTES);
 
     assert(drv->bdrv_co_writev);
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
 {
     if (bytes && bs->encrypted) {
         BDRVQcow2State *s = bs->opaque;
-        assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
-        assert((bytes & ~BDRV_SECTOR_MASK) == 0);
+        assert(QEMU_IS_ALIGNED(offset_in_cluster, BDRV_SECTOR_SIZE));
+        assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
         assert(s->crypto);
         if (qcow2_co_encrypt(bs, cluster_offset,
                              src_cluster_offset + offset_in_cluster,
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
                     goto fail;
                 }
 
-                assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-                assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+                assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+                assert(QEMU_IS_ALIGNED(cur_bytes, BDRV_SECTOR_SIZE));
                 if (qcow2_co_decrypt(bs, cluster_offset, offset,
                                      cluster_data, cur_bytes) < 0) {
                     ret = -EIO;
diff --git a/block/vvfat.c b/block/vvfat.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vvfat.c
+++ b/block/vvfat.c
@@ -XXX,XX +XXX,XX @@ vvfat_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
     void *buf;
 
-    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
 
     buf = g_try_malloc(bytes);
     if (bytes && buf == NULL) {
@@ -XXX,XX +XXX,XX @@ vvfat_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
     int nb_sectors = bytes >> BDRV_SECTOR_BITS;
     void *buf;
 
-    assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
-    assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
 
     buf = g_try_malloc(bytes);
     if (bytes && buf == NULL) {
diff --git a/qemu-img.c b/qemu-img.c
index XXXXXXX..XXXXXXX 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -XXX,XX +XXX,XX @@ static int img_convert(int argc, char **argv)
             int64_t sval;
 
             sval = cvtnum(optarg);
-            if (sval < 0 || sval & (BDRV_SECTOR_SIZE - 1) ||
+            if (sval < 0 || !QEMU_IS_ALIGNED(sval, BDRV_SECTOR_SIZE) ||
                 sval / BDRV_SECTOR_SIZE > MAX_BUF_SECTORS) {
                 error_report("Invalid buffer size for sparse output specified. "
                     "Valid sizes are multiples of %llu up to %llu. Select "
-- 
2.21.0

From: Nir Soffer <nirsof@gmail.com>

Replace confusing usage:

~BDRV_SECTOR_MASK

With more clear:

(BDRV_SECTOR_SIZE - 1)

Remove BDRV_SECTOR_MASK and the unused BDRV_BLOCK_OFFSET_MASK which was
it's last user.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-id: 20190827185913.27427-3-nsoffer@redhat.com
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 include/block/block.h | 2 --
 migration/block.c     | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -XXX,XX +XXX,XX @@ typedef struct HDGeometry {
 
 #define BDRV_SECTOR_BITS   9
 #define BDRV_SECTOR_SIZE   (1ULL << BDRV_SECTOR_BITS)
-#define BDRV_SECTOR_MASK   ~(BDRV_SECTOR_SIZE - 1)
 
 #define BDRV_REQUEST_MAX_SECTORS MIN(SIZE_MAX >> BDRV_SECTOR_BITS, \
                                      INT_MAX >> BDRV_SECTOR_BITS)
@@ -XXX,XX +XXX,XX @@ typedef struct HDGeometry {
 #define BDRV_BLOCK_ALLOCATED    0x10
 #define BDRV_BLOCK_EOF          0x20
 #define BDRV_BLOCK_RECURSE      0x40
-#define BDRV_BLOCK_OFFSET_MASK  BDRV_SECTOR_MASK
 
 typedef QSIMPLEQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopenQueue;
 
diff --git a/migration/block.c b/migration/block.c
index XXXXXXX..XXXXXXX 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -XXX,XX +XXX,XX @@ static int block_load(QEMUFile *f, void *opaque, int version_id)
     do {
         addr = qemu_get_be64(f);
 
-        flags = addr & ~BDRV_SECTOR_MASK;
+        flags = addr & (BDRV_SECTOR_SIZE - 1);
         addr >>= BDRV_SECTOR_BITS;
 
         if (flags & BLK_MIG_FLAG_DEVICE_BLOCK) {
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

When running "make check -j8" or something similar, the iotests are
running in parallel with the other tests. So when they are printing
out "Passed all xx tests" or a similar status message at the end,
it might not be quite clear that this message belongs to the iotests,
since the output might be mixed with the other tests. Thus change the
word "tests" here to "iotests" instead to avoid confusion.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190906113920.11271-1-thuth@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/check | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -XXX,XX +XXX,XX @@ END        { if (NR > 0) {
         if [ ! -z "$n_bad" -a $n_bad != 0 ]
         then
             echo "Failures:$bad"
-            echo "Failed $n_bad of $try tests"
+            echo "Failed $n_bad of $try iotests"
             echo "Failures:$bad" | fmt >>check.log
-            echo "Failed $n_bad of $try tests" >>check.log
+            echo "Failed $n_bad of $try iotests" >>check.log
         else
-            echo "Passed all $try tests"
-            echo "Passed all $try tests" >>check.log
+            echo "Passed all $try iotests"
+            echo "Passed all $try iotests" >>check.log
         fi
         needwrap=false
     fi
-- 
2.21.0

From: Thomas Huth <thuth@redhat.com>

The check script is already printing out which iotest is currently
running, so printing out the name of the check-block.sh shell script
looks superfluous here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190906113534.10907-1-thuth@redhat.com
Acked-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index XXXXXXX..XXXXXXX 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -XXX,XX +XXX,XX @@ QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu
 check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) \
 		qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) \
 		$(patsubst %,%/all,$(filter %-softmmu,$(TARGET_DIRS)))
-	$<
+	@$<
 
 .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y))
 $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json
-- 
2.21.0

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

qemu-io now prefixes its error and warnings with "qemu-io:".
36b9986b08787019e fixed a lot of iotests output but forget about
026.out.nocache. Fix it too.

Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()")
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190816153015.447957-2-vsementsov@virtuozzo.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/026.out.nocache | 168 ++++++++++++++---------------
 1 file changed, 84 insertions(+), 84 deletions(-)

diff --git a/tests/qemu-iotests/026.out.nocache b/tests/qemu-iotests/026.out.nocache
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/026.out.nocache
+++ b/tests/qemu-iotests/026.out.nocache
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_update; errno: 5; imm: off; once: off; write 
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 
 1 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_update; errno: 5; imm: off; once: off; write -b
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 
 1 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_update; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 1 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_update; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 1 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
 
 Event: l2_update; errno: 5; imm: off; once: off; write 
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 wrote 131072/131072 bytes at offset 0
 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l2_update; errno: 5; imm: off; once: off; write -b
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 wrote 131072/131072 bytes at offset 0
 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
 
 Event: l2_update; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 wrote 131072/131072 bytes at offset 0
 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l2_update; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 wrote 131072/131072 bytes at offset 0
 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l2_alloc_write; errno: 5; imm: off; once: off; write 
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l2_alloc_write; errno: 5; imm: off; once: off; write -b
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 
 1 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l2_alloc_write; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l2_alloc_write; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 1 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: write_aio; errno: 5; imm: off; once: off; write 
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: write_aio; errno: 5; imm: off; once: off; write -b
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: write_aio; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: write_aio; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_load; errno: 5; imm: off; once: off; write 
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_load; errno: 5; imm: off; once: off; write -b
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_load; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_load; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_update_part; errno: 5; imm: off; once: off; write 
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_update_part; errno: 5; imm: off; once: off; write -b
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_update_part; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_update_part; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc; errno: 5; imm: off; once: off; write 
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc; errno: 5; imm: off; once: off; write -b
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 55 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 251 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 10 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 23 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 10 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 23 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write 
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 10 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ This means waste of disk space, but no harm to data.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write -b
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 23 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_grow_write_table; errno: 5; imm: off; once: off
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_grow_write_table; errno: 28; imm: off; once: off
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_grow_activate_table; errno: 5; imm: off; once: off
-Failed to flush the L2 table cache: Input/output error
-Failed to flush the refcount block cache: Input/output error
+qemu-io: Failed to flush the L2 table cache: Input/output error
+qemu-io: Failed to flush the refcount block cache: Input/output error
 write failed: Input/output error
 
 96 leaked clusters were found on the image.
@@ -XXX,XX +XXX,XX @@ No errors were found on the image.
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
 
 Event: l1_grow_activate_table; errno: 28; imm: off; once: off
-Failed to flush the L2 table cache: No space left on device
-Failed to flush the refcount block cache: No space left on device
+qemu-io: Failed to flush the L2 table cache: No space left on device
+qemu-io: Failed to flush the refcount block cache: No space left on device
 write failed: No space left on device
 
 96 leaked clusters were found on the image.
-- 
2.21.0

A follow-up patch will make curl_multi_do() and curl_multi_read() take a
CURLSocket instead of the CURLState.  They still need the latter,
though, so add a pointer to it to the former.

Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190910124136.10565-2-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/curl.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
 #define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5
 
 struct BDRVCURLState;
+struct CURLState;
 
 static bool libcurl_initialized;
 
@@ -XXX,XX +XXX,XX @@ typedef struct CURLAIOCB {
 
 typedef struct CURLSocket {
     int fd;
+    struct CURLState *state;
     QLIST_ENTRY(CURLSocket) next;
 } CURLSocket;
 
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     if (!socket) {
         socket = g_new0(CURLSocket, 1);
         socket->fd = fd;
+        socket->state = state;
         QLIST_INSERT_HEAD(&state->sockets, socket, next);
     }
     socket = NULL;
-- 
2.21.0

This does not really change anything, but it makes the code a bit easier
to follow once we use @socket as the opaque pointer for
aio_set_fd_handler().

Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190910124136.10565-3-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/curl.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
 
     QLIST_FOREACH(socket, &state->sockets, next) {
         if (socket->fd == fd) {
-            if (action == CURL_POLL_REMOVE) {
-                QLIST_REMOVE(socket, next);
-                g_free(socket);
-            }
             break;
         }
     }
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
         socket->state = state;
         QLIST_INSERT_HEAD(&state->sockets, socket, next);
     }
-    socket = NULL;
 
     trace_curl_sock_cb(action, (int)fd);
     switch (action) {
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
             break;
     }
 
+    if (action == CURL_POLL_REMOVE) {
+        QLIST_REMOVE(socket, next);
+        g_free(socket);
+    }
+
     return 0;
 }
 
-- 
2.21.0

While it is more likely that transfers complete after some file
descriptor has data ready to read, we probably should not rely on it.
Better be safe than sorry and call curl_multi_check_completion() in
curl_multi_do(), too, just like it is done in curl_multi_read().

With this change, curl_multi_do() and curl_multi_read() are actually the
same, so drop curl_multi_read() and use curl_multi_do() as the sole FD
handler.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190910124136.10565-4-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/curl.c | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVCURLState {
 
 static void curl_clean_state(CURLState *s);
 static void curl_multi_do(void *arg);
-static void curl_multi_read(void *arg);
 
 #ifdef NEED_CURL_TIMER_CALLBACK
 /* Called from curl_multi_do_locked, with s->mutex held.  */
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     switch (action) {
         case CURL_POLL_IN:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_read, NULL, NULL, state);
+                               curl_multi_do, NULL, NULL, state);
             break;
         case CURL_POLL_OUT:
             aio_set_fd_handler(s->aio_context, fd, false,
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
             break;
         case CURL_POLL_INOUT:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_read, curl_multi_do, NULL, state);
+                               curl_multi_do, curl_multi_do, NULL, state);
             break;
         case CURL_POLL_REMOVE:
             aio_set_fd_handler(s->aio_context, fd, false,
@@ -XXX,XX +XXX,XX @@ static void curl_multi_do(void *arg)
 {
     CURLState *s = (CURLState *)arg;
 
-    qemu_mutex_lock(&s->s->mutex);
-    curl_multi_do_locked(s);
-    qemu_mutex_unlock(&s->s->mutex);
-}
-
-static void curl_multi_read(void *arg)
-{
-    CURLState *s = (CURLState *)arg;
-
     qemu_mutex_lock(&s->s->mutex);
     curl_multi_do_locked(s);
     curl_multi_check_completion(s->s);
-- 
2.21.0

curl_multi_do_locked() currently marks all sockets as ready.  That is
not only inefficient, but in fact unsafe (the loop is).  A follow-up
patch will change that, but to do so, curl_multi_do_locked() needs to
know exactly which socket is ready; and that is accomplished by this
patch here.

Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190910124136.10565-5-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/curl.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     switch (action) {
         case CURL_POLL_IN:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_do, NULL, NULL, state);
+                               curl_multi_do, NULL, NULL, socket);
             break;
         case CURL_POLL_OUT:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               NULL, curl_multi_do, NULL, state);
+                               NULL, curl_multi_do, NULL, socket);
             break;
         case CURL_POLL_INOUT:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_do, curl_multi_do, NULL, state);
+                               curl_multi_do, curl_multi_do, NULL, socket);
             break;
         case CURL_POLL_REMOVE:
             aio_set_fd_handler(s->aio_context, fd, false,
@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
 }
 
 /* Called with s->mutex held.  */
-static void curl_multi_do_locked(CURLState *s)
+static void curl_multi_do_locked(CURLSocket *ready_socket)
 {
     CURLSocket *socket, *next_socket;
+    CURLState *s = ready_socket->state;
     int running;
     int r;
 
@@ -XXX,XX +XXX,XX @@ static void curl_multi_do_locked(CURLState *s)
 
 static void curl_multi_do(void *arg)
 {
-    CURLState *s = (CURLState *)arg;
+    CURLSocket *socket = arg;
+    BDRVCURLState *s = socket->state->s;
 
-    qemu_mutex_lock(&s->s->mutex);
-    curl_multi_do_locked(s);
-    curl_multi_check_completion(s->s);
-    qemu_mutex_unlock(&s->s->mutex);
+    qemu_mutex_lock(&s->mutex);
+    curl_multi_do_locked(socket);
+    curl_multi_check_completion(s);
+    qemu_mutex_unlock(&s->mutex);
 }
 
 static void curl_multi_timeout_do(void *arg)
-- 
2.21.0

Instead of reporting all sockets to cURL, only report the one that has
caused curl_multi_do_locked() to be called.  This lets us get rid of the
QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are
only safe when the current element is removed in each iteration.  If it
possible for the list to be concurrently modified, we cannot guarantee
that only the current element will be removed.  Therefore, we must not
use QLIST_FOREACH_SAFE() here.

Fixes: ff5ca1664af85b24a4180d595ea6873fd3deac57
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190910124136.10565-6-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/curl.c | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
 }
 
 /* Called with s->mutex held.  */
-static void curl_multi_do_locked(CURLSocket *ready_socket)
+static void curl_multi_do_locked(CURLSocket *socket)
 {
-    CURLSocket *socket, *next_socket;
-    CURLState *s = ready_socket->state;
+    BDRVCURLState *s = socket->state->s;
     int running;
     int r;
 
-    if (!s->s->multi) {
+    if (!s->multi) {
         return;
     }
 
-    /* Need to use _SAFE because curl_multi_socket_action() may trigger
-     * curl_sock_cb() which might modify this list */
-    QLIST_FOREACH_SAFE(socket, &s->sockets, next, next_socket) {
-        do {
-            r = curl_multi_socket_action(s->s->multi, socket->fd, 0, &running);
-        } while (r == CURLM_CALL_MULTI_PERFORM);
-    }
+    do {
+        r = curl_multi_socket_action(s->multi, socket->fd, 0, &running);
+    } while (r == CURLM_CALL_MULTI_PERFORM);
 }
 
 static void curl_multi_do(void *arg)
-- 
2.21.0

Background: As of cURL 7.59.0, it verifies that several functions are
not called from within a callback.  Among these functions is
curl_multi_add_handle().

curl_read_cb() is a callback from cURL and not a coroutine.  Waking up
acb->co will lead to entering it then and there, which means the current
request will settle and the caller (if it runs in the same coroutine)
may then issue the next request.  In such a case, we will enter
curl_setup_preadv() effectively from within curl_read_cb().

Calling curl_multi_add_handle() will then fail and the new request will
not be processed.

Fix this by not letting curl_read_cb() wake up acb->co.  Instead, leave
the whole business of settling the AIOCB objects to
curl_multi_check_completion() (which is called from our timer callback
and our FD handler, so not from any cURL callbacks).

Reported-by: Natalie Gavrielov <ngavrilo@redhat.com>
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1740193
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190910124136.10565-7-mreitz@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/curl.c | 69 ++++++++++++++++++++++------------------------------
 1 file changed, 29 insertions(+), 40 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
 {
     CURLState *s = ((CURLState*)opaque);
     size_t realsize = size * nmemb;
-    int i;
 
     trace_curl_read_cb(realsize);
 
@@ -XXX,XX +XXX,XX @@ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
     memcpy(s->orig_buf + s->buf_off, ptr, realsize);
     s->buf_off += realsize;
 
-    for(i=0; i<CURL_NUM_ACB; i++) {
-        CURLAIOCB *acb = s->acb[i];
-
-        if (!acb)
-            continue;
-
-        if ((s->buf_off >= acb->end)) {
-            size_t request_length = acb->bytes;
-
-            qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start,
-                                acb->end - acb->start);
-
-            if (acb->end - acb->start < request_length) {
-                size_t offset = acb->end - acb->start;
-                qemu_iovec_memset(acb->qiov, offset, 0,
-                                  request_length - offset);
-            }
-
-            acb->ret = 0;
-            s->acb[i] = NULL;
-            qemu_mutex_unlock(&s->s->mutex);
-            aio_co_wake(acb->co);
-            qemu_mutex_lock(&s->s->mutex);
-        }
-    }
-
 read_end:
     /* curl will error out if we do not return this value */
     return size * nmemb;
@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
             break;
 
         if (msg->msg == CURLMSG_DONE) {
+            int i;
             CURLState *state = NULL;
+            bool error = msg->data.result != CURLE_OK;
+
             curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE,
                               (char **)&state);
 
-            /* ACBs for successful messages get completed in curl_read_cb */
-            if (msg->data.result != CURLE_OK) {
-                int i;
+            if (error) {
                 static int errcount = 100;
 
                 /* Don't lose the original error message from curl, since
@@ -XXX,XX +XXX,XX @@ static void curl_multi_check_completion(BDRVCURLState *s)
                         error_report("curl: further errors suppressed");
                     }
                 }
+            }
 
-                for (i = 0; i < CURL_NUM_ACB; i++) {
-                    CURLAIOCB *acb = state->acb[i];
+            for (i = 0; i < CURL_NUM_ACB; i++) {
+                CURLAIOCB *acb = state->acb[i];
 
-                    if (acb == NULL) {
-                        continue;
-                    }
+                if (acb == NULL) {
+                    continue;
+                }
+
+                if (!error) {
+                    /* Assert that we have read all data */
+                    assert(state->buf_off >= acb->end);
+
+                    qemu_iovec_from_buf(acb->qiov, 0,
+                                        state->orig_buf + acb->start,
+                                        acb->end - acb->start);
 
-                    acb->ret = -EIO;
-                    state->acb[i] = NULL;
-                    qemu_mutex_unlock(&s->mutex);
-                    aio_co_wake(acb->co);
-                    qemu_mutex_lock(&s->mutex);
+                    if (acb->end - acb->start < acb->bytes) {
+                        size_t offset = acb->end - acb->start;
+                        qemu_iovec_memset(acb->qiov, offset, 0,
+                                          acb->bytes - offset);
+                    }
                 }
+
+                acb->ret = error ? -EIO : 0;
+                state->acb[i] = NULL;
+                qemu_mutex_unlock(&s->mutex);
+                aio_co_wake(acb->co);
+                qemu_mutex_lock(&s->mutex);
             }
 
             curl_clean_state(state);
-- 
2.21.0

If we had done that all along, debugging would have been much simpler.
(Also, I/O errors are better than hangs.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190910124136.10565-8-mreitz@redhat.com
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/curl.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/block/curl.c b/block/curl.c
index XXXXXXX..XXXXXXX 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -XXX,XX +XXX,XX @@ static void curl_setup_preadv(BlockDriverState *bs, CURLAIOCB *acb)
     trace_curl_setup_preadv(acb->bytes, start, state->range);
     curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
 
-    curl_multi_add_handle(s->multi, state->curl);
+    if (curl_multi_add_handle(s->multi, state->curl) != CURLM_OK) {
+        state->acb[0] = NULL;
+        acb->ret = -EIO;
+
+        curl_clean_state(state);
+        goto out;
+    }
 
     /* Tell curl it needs to kick things off */
     curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);
-- 
2.21.0

From: Sergio Lopez <slp@redhat.com>

block_job_remove_all_bdrv() iterates through job->nodes, calling
bdrv_root_unref_child() for each entry. The call to the latter may
reach child_job_[can_]set_aio_ctx(), which will also attempt to
traverse job->nodes, potentially finding entries that where freed
on previous iterations.

To avoid this situation, update job->nodes head on each iteration to
ensure that already freed entries are no longer linked to the list.

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631
Signed-off-by: Sergio Lopez <slp@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190911100316.32282-1-mreitz@redhat.com
Reviewed-by: Sergio Lopez <slp@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 blockjob.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/blockjob.c b/blockjob.c
index XXXXXXX..XXXXXXX 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -XXX,XX +XXX,XX @@ static const BdrvChildRole child_job = {
 
 void block_job_remove_all_bdrv(BlockJob *job)
 {
-    GSList *l;
-    for (l = job->nodes; l; l = l->next) {
+    /*
+     * bdrv_root_unref_child() may reach child_job_[can_]set_aio_ctx(),
+     * which will also traverse job->nodes, so consume the list one by
+     * one to make sure that such a concurrent access does not attempt
+     * to process an already freed BdrvChild.
+     */
+    while (job->nodes) {
+        GSList *l = job->nodes;
         BdrvChild *c = l->data;
+
+        job->nodes = l->next;
+
         bdrv_op_unblock_all(c->bs, job->blocker);
         bdrv_root_unref_child(c);
+
+        g_slist_free_1(l);
     }
-    g_slist_free(job->nodes);
-    job->nodes = NULL;
 }
 
 bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs)
-- 
2.21.0

From: Maxim Levitsky <mlevitsk@redhat.com>

This fixes subtle corruption introduced by luks threaded encryption
in commit 8ac0f15f335

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1745922

The corruption happens when we do a write that
   * writes to two or more unallocated clusters at once
   * doesn't fully cover the first sector
   * doesn't fully cover the last sector
   * uses luks encryption

In this case, when allocating the new clusters we COW both areas
prior to the write and after the write, and we encrypt them.

The above mentioned commit accidentally made it so we encrypt the
second COW area using the physical cluster offset of the first area.

The problem is that offset_in_cluster in do_perform_cow_encrypt
can be larger that the cluster size, thus cluster_offset
will no longer point to the start of the cluster at which encrypted
area starts.

Next patch in this series will refactor the code to avoid all these
assumptions.

In the bugreport that was triggered by rebasing a luks image to new,
zero filled base, which lot of such writes, and causes some files
with zero areas to contain garbage there instead.
But as described above it can happen elsewhere as well

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190915203655.21638-2-mlevitsk@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qcow2-cluster.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -XXX,XX +XXX,XX @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
         assert(QEMU_IS_ALIGNED(offset_in_cluster, BDRV_SECTOR_SIZE));
         assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
         assert(s->crypto);
-        if (qcow2_co_encrypt(bs, cluster_offset,
-                             src_cluster_offset + offset_in_cluster,
-                             buffer, bytes) < 0) {
+        if (qcow2_co_encrypt(bs,
+                start_of_cluster(s, cluster_offset + offset_in_cluster),
+                src_cluster_offset + offset_in_cluster,
+                buffer, bytes) < 0) {
             return false;
         }
     }
-- 
2.21.0

From: Maxim Levitsky <mlevitsk@redhat.com>

* Change the qcow2_co_{encrypt|decrypt} to just receive full host and
  guest offsets and use this function directly instead of calling
  do_perform_cow_encrypt (which is removed by that patch).

* Adjust qcow2_co_encdec to take full host and guest offsets as well.

* Document the qcow2_co_{encrypt|decrypt} arguments
  to prevent the bug fixed in former commit from hopefully
  happening again.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20190915203655.21638-3-mlevitsk@redhat.com
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[mreitz: Let perform_cow() return the error value returned by
         qcow2_co_encrypt(), as proposed by Vladimir]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/qcow2.h         |  8 +++---
 block/qcow2-cluster.c | 41 +++++++++-------------------
 block/qcow2-threads.c | 63 +++++++++++++++++++++++++++++++++----------
 block/qcow2.c         |  5 ++--
 4 files changed, 69 insertions(+), 48 deletions(-)

diff --git a/block/qcow2.h b/block/qcow2.h
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -XXX,XX +XXX,XX @@ ssize_t coroutine_fn
 qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size,
                     const void *src, size_t src_size);
 int coroutine_fn
-qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len);
+qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len);
 int coroutine_fn
-qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len);
+qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len);
 
 #endif
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn do_perform_cow_read(BlockDriverState *bs,
     return 0;
 }
 
-static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
-                                                uint64_t src_cluster_offset,
-                                                uint64_t cluster_offset,
-                                                unsigned offset_in_cluster,
-                                                uint8_t *buffer,
-                                                unsigned bytes)
-{
-    if (bytes && bs->encrypted) {
-        BDRVQcow2State *s = bs->opaque;
-        assert(QEMU_IS_ALIGNED(offset_in_cluster, BDRV_SECTOR_SIZE));
-        assert(QEMU_IS_ALIGNED(bytes, BDRV_SECTOR_SIZE));
-        assert(s->crypto);
-        if (qcow2_co_encrypt(bs,
-                start_of_cluster(s, cluster_offset + offset_in_cluster),
-                src_cluster_offset + offset_in_cluster,
-                buffer, bytes) < 0) {
-            return false;
-        }
-    }
-    return true;
-}
-
 static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
                                              uint64_t cluster_offset,
                                              unsigned offset_in_cluster,
@@ -XXX,XX +XXX,XX @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta *m)
 
     /* Encrypt the data if necessary before writing it */
     if (bs->encrypted) {
-        if (!do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
-                                    start->offset, start_buffer,
-                                    start->nb_bytes) ||
-            !do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
-                                    end->offset, end_buffer, end->nb_bytes)) {
-            ret = -EIO;
+        ret = qcow2_co_encrypt(bs,
+                               m->alloc_offset + start->offset,
+                               m->offset + start->offset,
+                               start_buffer, start->nb_bytes);
+        if (ret < 0) {
+            goto fail;
+        }
+
+        ret = qcow2_co_encrypt(bs,
+                               m->alloc_offset + end->offset,
+                               m->offset + end->offset,
+                               end_buffer, end->nb_bytes);
+        if (ret < 0) {
             goto fail;
         }
     }
diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2-threads.c
+++ b/block/qcow2-threads.c
@@ -XXX,XX +XXX,XX @@ static int qcow2_encdec_pool_func(void *opaque)
 }
 
 static int coroutine_fn
-qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
-                  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc func)
+qcow2_co_encdec(BlockDriverState *bs, uint64_t host_offset,
+                uint64_t guest_offset, void *buf, size_t len,
+                Qcow2EncDecFunc func)
 {
     BDRVQcow2State *s = bs->opaque;
     Qcow2EncDecData arg = {
         .block = s->crypto,
-        .offset = s->crypt_physical_offset ?
-                      file_cluster_offset + offset_into_cluster(s, offset) :
-                      offset,
+        .offset = s->crypt_physical_offset ? host_offset : guest_offset,
         .buf = buf,
         .len = len,
         .func = func,
     };
 
-    return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
+    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
+    assert(s->crypto);
+
+    return len == 0 ? 0 : qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
 }
 
+/*
+ * qcow2_co_encrypt()
+ *
+ * Encrypts one or more contiguous aligned sectors
+ *
+ * @host_offset - underlying storage offset of the first sector of the
+ * data to be encrypted
+ *
+ * @guest_offset - guest (virtual) offset of the first sector of the
+ * data to be encrypted
+ *
+ * @buf - buffer with the data to encrypt, that after encryption
+ *        will be written to the underlying storage device at
+ *        @host_offset
+ *
+ * @len - length of the buffer (must be a BDRV_SECTOR_SIZE multiple)
+ *
+ * Depending on the encryption method, @host_offset and/or @guest_offset
+ * may be used for generating the initialization vector for
+ * encryption.
+ *
+ * Note that while the whole range must be aligned on sectors, it
+ * does not have to be aligned on clusters and can also cross cluster
+ * boundaries
+ */
 int coroutine_fn
-qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len)
+qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len)
 {
-    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
-                             qcrypto_block_encrypt);
+    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
+                           qcrypto_block_encrypt);
 }
 
+/*
+ * qcow2_co_decrypt()
+ *
+ * Decrypts one or more contiguous aligned sectors
+ * Similar to qcow2_co_encrypt
+ */
 int coroutine_fn
-qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len)
+qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len)
 {
-    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
-                             qcrypto_block_decrypt);
+    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
+                           qcrypto_block_decrypt);
 }
diff --git a/block/qcow2.c b/block/qcow2.c
index XXXXXXX..XXXXXXX 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
 
                 assert(QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE));
                 assert(QEMU_IS_ALIGNED(cur_bytes, BDRV_SECTOR_SIZE));
-                if (qcow2_co_decrypt(bs, cluster_offset, offset,
+                if (qcow2_co_decrypt(bs, cluster_offset + offset_in_cluster,
+                                     offset,
                                      cluster_data, cur_bytes) < 0) {
                     ret = -EIO;
                     goto fail;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int qcow2_co_pwritev_part(
             qemu_iovec_to_buf(qiov, qiov_offset + bytes_done,
                               cluster_data, cur_bytes);
 
-            if (qcow2_co_encrypt(bs, cluster_offset, offset,
+            if (qcow2_co_encrypt(bs, cluster_offset + offset_in_cluster, offset,
                                  cluster_data, cur_bytes) < 0) {
                 ret = -EIO;
                 goto out_unlocked;
-- 
2.21.0

From: Maxim Levitsky <mlevitsk@redhat.com>

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190915203655.21638-4-mlevitsk@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/263     | 91 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/263.out | 40 +++++++++++++++++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 132 insertions(+)
 create mode 100755 tests/qemu-iotests/263
 create mode 100644 tests/qemu-iotests/263.out

diff --git a/tests/qemu-iotests/263 b/tests/qemu-iotests/263
new file mode 100755
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/263
@@ -XXX,XX +XXX,XX @@
+#!/usr/bin/env bash
+#
+# Test encrypted write that crosses cluster boundary of two unallocated clusters
+# Based on 188
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mlevitsk@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto generic
+_supported_os Linux
+
+
+size=1M
+
+SECRET="secret,id=sec0,data=astrochicken"
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+
+_run_test()
+{
+	echo "== reading the whole image =="
+	$QEMU_IO --object $SECRET -c "read -P 0 0 $size" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+
+	echo
+	echo "== write two 512 byte sectors on a cluster boundary =="
+	$QEMU_IO --object $SECRET -c "write -P 0xAA 0xFE00 0x400" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+
+	echo
+	echo "== verify that the rest of the image is not changed =="
+	$QEMU_IO --object $SECRET -c "read -P 0x00 0x00000 0xFE00" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+	$QEMU_IO --object $SECRET -c "read -P 0xAA 0x0FE00 0x400" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+	$QEMU_IO --object $SECRET -c "read -P 0x00 0x10200 0xEFE00" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+
+}
+
+
+echo
+echo "testing LUKS qcow2 encryption"
+echo
+
+_make_test_img --object $SECRET -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10,cluster_size=64K" $size
+_run_test "driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
+_cleanup_test_img
+
+echo
+echo "testing legacy AES qcow2 encryption"
+echo
+
+
+_make_test_img --object $SECRET -o "encrypt.format=aes,encrypt.key-secret=sec0,cluster_size=64K" $size
+_run_test "driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
+_cleanup_test_img
+
+
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/263.out b/tests/qemu-iotests/263.out
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/tests/qemu-iotests/263.out
@@ -XXX,XX +XXX,XX @@
+QA output created by 263
+
+testing LUKS qcow2 encryption
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+== reading the whole image ==
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== write two 512 byte sectors on a cluster boundary ==
+wrote 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== verify that the rest of the image is not changed ==
+read 65024/65024 bytes at offset 0
+63.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 982528/982528 bytes at offset 66048
+959.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+testing legacy AES qcow2 encryption
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=aes encrypt.key-secret=sec0
+== reading the whole image ==
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== write two 512 byte sectors on a cluster boundary ==
+wrote 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== verify that the rest of the image is not changed ==
+read 65024/65024 bytes at offset 0
+63.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 982528/982528 bytes at offset 66048
+959.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -XXX,XX +XXX,XX @@
 257 rw
 258 rw quick
 262 rw quick migration
+263 rw quick
 265 rw auto quick
 266 rw quick
-- 
2.21.0

The following changes since commit 474f3938d79ab36b9231c9ad3b5a9314c2aeacde:

Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-jun-21-2019' into staging (2019-06-21 15:40:50 +0100)

are available in the Git repository at:

https://github.com/XanClic/qemu.git tags/pull-block-2019-06-24

for you to fetch changes up to ab5d4a30f7f3803ca5106b370969c1b7b54136f8:

iotests: Fix 205 for concurrent runs (2019-06-24 16:01:40 +0200)

----------------------------------------------------------------
Block patches:
- The SSH block driver now uses libssh instead of libssh2
- The VMDK block driver gets read-only support for the seSparse
  subformat
- Various fixes

---

v2:
- Squashed Pino's fix for pre-0.8 libssh into the libssh patch

----------------------------------------------------------------
Anton Nefedov (1):
  iotest 134: test cluster-misaligned encrypted write

Klaus Birkelund Jensen (1):
  nvme: do not advertise support for unsupported arbitration mechanism

Max Reitz (1):
  iotests: Fix 205 for concurrent runs

Pino Toscano (1):
  ssh: switch from libssh2 to libssh

Sam Eiderman (3):
  vmdk: Fix comment regarding max l1_size coverage
  vmdk: Reduce the max bound for L1 table size
  vmdk: Add read-only support for seSparse snapshots

Vladimir Sementsov-Ogievskiy (1):
  blockdev: enable non-root nodes for transaction drive-backup source

-- 
2.21.0

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

We forget to enable it for transaction .prepare, while it is already
enabled in do_drive_backup since commit a2d665c1bc362
    "blockdev: loosen restrictions on drive-backup source node"

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190618140804.59214-1-vsementsov@virtuozzo.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 blockdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/blockdev.c b/blockdev.c
index XXXXXXX..XXXXXXX 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -XXX,XX +XXX,XX @@ static void drive_backup_prepare(BlkActionState *common, Error **errp)
     assert(common->action->type == TRANSACTION_ACTION_KIND_DRIVE_BACKUP);
     backup = common->action->u.drive_backup.data;
 
-    bs = qmp_get_root_bs(backup->device, errp);
+    bs = bdrv_lookup_bs(backup->device, backup->device, errp);
     if (!bs) {
         return;
     }
-- 
2.21.0

From: Anton Nefedov <anton.nefedov@virtuozzo.com>

COW (even empty/zero) areas require encryption too

Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20190516143028.81155-1-anton.nefedov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/134     |  9 +++++++++
 tests/qemu-iotests/134.out | 10 ++++++++++
 2 files changed, 19 insertions(+)

diff --git a/tests/qemu-iotests/134 b/tests/qemu-iotests/134
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/134
+++ b/tests/qemu-iotests/134
@@ -XXX,XX +XXX,XX @@ echo
 echo "== reading whole image =="
 $QEMU_IO --object $SECRET -c "read 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
 
+echo
+echo "== rewriting cluster part =="
+$QEMU_IO --object $SECRET -c "write -P 0xb 512 512" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
+
+echo
+echo "== verify pattern =="
+$QEMU_IO --object $SECRET -c "read -P 0 0 512"  --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
+$QEMU_IO --object $SECRET -c "read -P 0xb 512 512"  --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
+
 echo
 echo "== rewriting whole image =="
 $QEMU_IO --object $SECRET -c "write -P 0xa 0 $size" --image-opts $IMGSPEC | _filter_qemu_io | _filter_testdir
diff --git a/tests/qemu-iotests/134.out b/tests/qemu-iotests/134.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/134.out
+++ b/tests/qemu-iotests/134.out
@@ -XXX,XX +XXX,XX @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.
 read 134217728/134217728 bytes at offset 0
 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
+== rewriting cluster part ==
+wrote 512/512 bytes at offset 512
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== verify pattern ==
+read 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 512/512 bytes at offset 512
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
 == rewriting whole image ==
 wrote 134217728/134217728 bytes at offset 0
 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-- 
2.21.0

From: Sam Eiderman <shmuel.eiderman@oracle.com>

Commit b0651b8c246d ("vmdk: Move l1_size check into vmdk_add_extent")
extended the l1_size check from VMDK4 to VMDK3 but did not update the
default coverage in the moved comment.

The previous vmdk4 calculation:

(512 * 1024 * 1024) * 512(l2 entries) * 65536(grain) = 16PB

The added vmdk3 calculation:

(512 * 1024 * 1024) * 4096(l2 entries) * 512(grain) = 1PB

Adding the calculation of vmdk3 to the comment.

In any case, VMware does not offer virtual disks more than 2TB for
vmdk4/vmdk3 or 64TB for the new undocumented seSparse format which is
not implemented yet in qemu.

Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
Reviewed-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
Message-id: 20190620091057.47441-2-shmuel.eiderman@oracle.com
Reviewed-by: yuchenlin <yuchenlin@synology.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
         return -EFBIG;
     }
     if (l1_size > 512 * 1024 * 1024) {
-        /* Although with big capacity and small l1_entry_sectors, we can get a
+        /*
+         * Although with big capacity and small l1_entry_sectors, we can get a
          * big l1_size, we don't want unbounded value to allocate the table.
-         * Limit it to 512M, which is 16PB for default cluster and L2 table
-         * size */
+         * Limit it to 512M, which is:
+         *     16PB - for default "Hosted Sparse Extent" (VMDK4)
+         *            cluster size: 64KB, L2 table size: 512 entries
+         *     1PB  - for default "ESXi Host Sparse Extent" (VMDK3/vmfsSparse)
+         *            cluster size: 512B, L2 table size: 4096 entries
+         */
         error_setg(errp, "L1 size too big");
         return -EFBIG;
     }
-- 
2.21.0

From: Sam Eiderman <shmuel.eiderman@oracle.com>

512M of L1 entries is a very loose bound, only 32M are required to store
the maximal supported VMDK file size of 2TB.

Fixed qemu-iotest 59# - now failure occures before on impossible L1
table size.

Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
Reviewed-by: Liran Alon <liran.alon@oracle.com>
Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
Message-id: 20190620091057.47441-3-shmuel.eiderman@oracle.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c               | 13 +++++++------
 tests/qemu-iotests/059.out |  2 +-
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
         error_setg(errp, "Invalid granularity, image may be corrupt");
         return -EFBIG;
     }
-    if (l1_size > 512 * 1024 * 1024) {
+    if (l1_size > 32 * 1024 * 1024) {
         /*
          * Although with big capacity and small l1_entry_sectors, we can get a
          * big l1_size, we don't want unbounded value to allocate the table.
-         * Limit it to 512M, which is:
-         *     16PB - for default "Hosted Sparse Extent" (VMDK4)
-         *            cluster size: 64KB, L2 table size: 512 entries
-         *     1PB  - for default "ESXi Host Sparse Extent" (VMDK3/vmfsSparse)
-         *            cluster size: 512B, L2 table size: 4096 entries
+         * Limit it to 32M, which is enough to store:
+         *     8TB  - for both VMDK3 & VMDK4 with
+         *            minimal cluster size: 512B
+         *            minimal L2 table size: 512 entries
+         *            8 TB is still more than the maximal value supported for
+         *            VMDK3 & VMDK4 which is 2TB.
          */
         error_setg(errp, "L1 size too big");
         return -EFBIG;
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -XXX,XX +XXX,XX @@ Offset          Length          Mapped to       File
 0x140000000     0x10000         0x50000         TEST_DIR/t-s003.vmdk
 
 === Testing afl image with a very large capacity ===
-qemu-img: Can't get image size 'TEST_DIR/afl9.IMGFMT': File too large
+qemu-img: Could not open 'TEST_DIR/afl9.IMGFMT': L1 size too big
 *** done
-- 
2.21.0

From: Sam Eiderman <shmuel.eiderman@oracle.com>

Until ESXi 6.5 VMware used the vmfsSparse format for snapshots (VMDK3 in
QEMU).

This format was lacking in the following:

* Grain directory (L1) and grain table (L2) entries were 32-bit,
      allowing access to only 2TB (slightly less) of data.
    * The grain size (default) was 512 bytes - leading to data
      fragmentation and many grain tables.
    * For space reclamation purposes, it was necessary to find all the
      grains which are not pointed to by any grain table - so a reverse
      mapping of "offset of grain in vmdk" to "grain table" must be
      constructed - which takes large amounts of CPU/RAM.

The format specification can be found in VMware's documentation:
https://www.vmware.com/support/developer/vddk/vmdk_50_technote.pdf

In ESXi 6.5, to support snapshot files larger than 2TB, a new format was
introduced: SESparse (Space Efficient).

This format fixes the above issues:

* All entries are now 64-bit.
    * The grain size (default) is 4KB.
    * Grain directory and grain tables are now located at the beginning
      of the file.
      + seSparse format reserves space for all grain tables.
      + Grain tables can be addressed using an index.
      + Grains are located in the end of the file and can also be
        addressed with an index.
      - seSparse vmdks of large disks (64TB) have huge preallocated
        headers - mainly due to L2 tables, even for empty snapshots.
    * The header contains a reverse mapping ("backmap") of "offset of
      grain in vmdk" to "grain table" and a bitmap ("free bitmap") which
      specifies for each grain - whether it is allocated or not.
      Using these data structures we can implement space reclamation
      efficiently.
    * Due to the fact that the header now maintains two mappings:
        * The regular one (grain directory & grain tables)
        * A reverse one (backmap and free bitmap)
      These data structures can lose consistency upon crash and result
      in a corrupted VMDK.
      Therefore, a journal is also added to the VMDK and is replayed
      when the VMware reopens the file after a crash.

Since ESXi 6.7 - SESparse is the only snapshot format available.

Unfortunately, VMware does not provide documentation regarding the new
seSparse format.

This commit is based on black-box research of the seSparse format.
Various in-guest block operations and their effect on the snapshot file
were tested.

The only VMware provided source of information (regarding the underlying
implementation) was a log file on the ESXi:

/var/log/hostd.log

Whenever an seSparse snapshot is created - the log is being populated
with seSparse records.

Relevant log records are of the form:

[...] Const Header:
[...]  constMagic     = 0xcafebabe
[...]  version        = 2.1
[...]  capacity       = 204800
[...]  grainSize      = 8
[...]  grainTableSize = 64
[...]  flags          = 0
[...] Extents:
[...]  Header         : <1 : 1>
[...]  JournalHdr     : <2 : 2>
[...]  Journal        : <2048 : 2048>
[...]  GrainDirectory : <4096 : 2048>
[...]  GrainTables    : <6144 : 2048>
[...]  FreeBitmap     : <8192 : 2048>
[...]  BackMap        : <10240 : 2048>
[...]  Grain          : <12288 : 204800>
[...] Volatile Header:
[...] volatileMagic     = 0xcafecafe
[...] FreeGTNumber      = 0
[...] nextTxnSeqNumber  = 0
[...] replayJournal     = 0

The sizes that are seen in the log file are in sectors.
Extents are of the following format: <offset : size>

This commit is a strict implementation which enforces:
    * magics
    * version number 2.1
    * grain size of 8 sectors  (4KB)
    * grain table size of 64 sectors
    * zero flags
    * extent locations

Additionally, this commit proivdes only a subset of the functionality
offered by seSparse's format:
    * Read-only
    * No journal replay
    * No space reclamation
    * No unmap support

Hence, journal header, journal, free bitmap and backmap extents are
unused, only the "classic" (L1 -> L2 -> data) grain access is
implemented.

However there are several differences in the grain access itself.
Grain directory (L1):
    * Grain directory entries are indexes (not offsets) to grain
      tables.
    * Valid grain directory entries have their highest nibble set to
      0x1.
    * Since grain tables are always located in the beginning of the
      file - the index can fit into 32 bits - so we can use its low
      part if it's valid.
Grain table (L2):
    * Grain table entries are indexes (not offsets) to grains.
    * If the highest nibble of the entry is:
        0x0:
            The grain in not allocated.
            The rest of the bytes are 0.
        0x1:
            The grain is unmapped - guest sees a zero grain.
            The rest of the bits point to the previously mapped grain,
            see 0x3 case.
        0x2:
            The grain is zero.
        0x3:
            The grain is allocated - to get the index calculate:
            ((entry & 0x0fff000000000000) >> 48) |
            ((entry & 0x0000ffffffffffff) << 12)
    * The difference between 0x1 and 0x2 is that 0x1 is an unallocated
      grain which results from the guest using sg_unmap to unmap the
      grain - but the grain itself still exists in the grain extent - a
      space reclamation procedure should delete it.
      Unmapping a zero grain has no effect (0x2 will not change to 0x1)
      but unmapping an unallocated grain will (0x0 to 0x1) - naturally.

In order to implement seSparse some fields had to be changed to support
both 32-bit and 64-bit entry sizes.

Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
Reviewed-by: Arbel Moshe <arbel.moshe@oracle.com>
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
Message-id: 20190620091057.47441-4-shmuel.eiderman@oracle.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/vmdk.c | 358 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 342 insertions(+), 16 deletions(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index XXXXXXX..XXXXXXX 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -XXX,XX +XXX,XX @@ typedef struct {
     uint16_t compressAlgorithm;
 } QEMU_PACKED VMDK4Header;
 
+typedef struct VMDKSESparseConstHeader {
+    uint64_t magic;
+    uint64_t version;
+    uint64_t capacity;
+    uint64_t grain_size;
+    uint64_t grain_table_size;
+    uint64_t flags;
+    uint64_t reserved1;
+    uint64_t reserved2;
+    uint64_t reserved3;
+    uint64_t reserved4;
+    uint64_t volatile_header_offset;
+    uint64_t volatile_header_size;
+    uint64_t journal_header_offset;
+    uint64_t journal_header_size;
+    uint64_t journal_offset;
+    uint64_t journal_size;
+    uint64_t grain_dir_offset;
+    uint64_t grain_dir_size;
+    uint64_t grain_tables_offset;
+    uint64_t grain_tables_size;
+    uint64_t free_bitmap_offset;
+    uint64_t free_bitmap_size;
+    uint64_t backmap_offset;
+    uint64_t backmap_size;
+    uint64_t grains_offset;
+    uint64_t grains_size;
+    uint8_t pad[304];
+} QEMU_PACKED VMDKSESparseConstHeader;
+
+typedef struct VMDKSESparseVolatileHeader {
+    uint64_t magic;
+    uint64_t free_gt_number;
+    uint64_t next_txn_seq_number;
+    uint64_t replay_journal;
+    uint8_t pad[480];
+} QEMU_PACKED VMDKSESparseVolatileHeader;
+
 #define L2_CACHE_SIZE 16
 
 typedef struct VmdkExtent {
@@ -XXX,XX +XXX,XX @@ typedef struct VmdkExtent {
     bool compressed;
     bool has_marker;
     bool has_zero_grain;
+    bool sesparse;
+    uint64_t sesparse_l2_tables_offset;
+    uint64_t sesparse_clusters_offset;
+    int32_t entry_size;
     int version;
     int64_t sectors;
     int64_t end_sector;
     int64_t flat_start_offset;
     int64_t l1_table_offset;
     int64_t l1_backup_table_offset;
-    uint32_t *l1_table;
+    void *l1_table;
     uint32_t *l1_backup_table;
     unsigned int l1_size;
     uint32_t l1_entry_sectors;
 
     unsigned int l2_size;
-    uint32_t *l2_cache;
+    void *l2_cache;
     uint32_t l2_cache_offsets[L2_CACHE_SIZE];
     uint32_t l2_cache_counts[L2_CACHE_SIZE];
 
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
          *            minimal L2 table size: 512 entries
          *            8 TB is still more than the maximal value supported for
          *            VMDK3 & VMDK4 which is 2TB.
+         *     64TB - for "ESXi seSparse Extent"
+         *            minimal cluster size: 512B (default is 4KB)
+         *            L2 table size: 4096 entries (const).
+         *            64TB is more than the maximal value supported for
+         *            seSparse VMDKs (which is slightly less than 64TB)
          */
         error_setg(errp, "L1 size too big");
         return -EFBIG;
@@ -XXX,XX +XXX,XX @@ static int vmdk_add_extent(BlockDriverState *bs,
     extent->l2_size = l2_size;
     extent->cluster_sectors = flat ? sectors : cluster_sectors;
     extent->next_cluster_sector = ROUND_UP(nb_sectors, cluster_sectors);
+    extent->entry_size = sizeof(uint32_t);
 
     if (s->num_extents > 1) {
         extent->end_sector = (*(extent - 1)).end_sector + extent->sectors;
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
     int i;
 
     /* read the L1 table */
-    l1_size = extent->l1_size * sizeof(uint32_t);
+    l1_size = extent->l1_size * extent->entry_size;
     extent->l1_table = g_try_malloc(l1_size);
     if (l1_size && extent->l1_table == NULL) {
         return -ENOMEM;
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
         goto fail_l1;
     }
     for (i = 0; i < extent->l1_size; i++) {
-        le32_to_cpus(&extent->l1_table[i]);
+        if (extent->entry_size == sizeof(uint64_t)) {
+            le64_to_cpus((uint64_t *)extent->l1_table + i);
+        } else {
+            assert(extent->entry_size == sizeof(uint32_t));
+            le32_to_cpus((uint32_t *)extent->l1_table + i);
+        }
     }
 
     if (extent->l1_backup_table_offset) {
+        assert(!extent->sesparse);
         extent->l1_backup_table = g_try_malloc(l1_size);
         if (l1_size && extent->l1_backup_table == NULL) {
             ret = -ENOMEM;
@@ -XXX,XX +XXX,XX @@ static int vmdk_init_tables(BlockDriverState *bs, VmdkExtent *extent,
     }
 
     extent->l2_cache =
-        g_new(uint32_t, extent->l2_size * L2_CACHE_SIZE);
+        g_malloc(extent->entry_size * extent->l2_size * L2_CACHE_SIZE);
     return 0;
  fail_l1b:
     g_free(extent->l1_backup_table);
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_vmfs_sparse(BlockDriverState *bs,
     return ret;
 }
 
+#define SESPARSE_CONST_HEADER_MAGIC UINT64_C(0x00000000cafebabe)
+#define SESPARSE_VOLATILE_HEADER_MAGIC UINT64_C(0x00000000cafecafe)
+
+/* Strict checks - format not officially documented */
+static int check_se_sparse_const_header(VMDKSESparseConstHeader *header,
+                                        Error **errp)
+{
+    header->magic = le64_to_cpu(header->magic);
+    header->version = le64_to_cpu(header->version);
+    header->grain_size = le64_to_cpu(header->grain_size);
+    header->grain_table_size = le64_to_cpu(header->grain_table_size);
+    header->flags = le64_to_cpu(header->flags);
+    header->reserved1 = le64_to_cpu(header->reserved1);
+    header->reserved2 = le64_to_cpu(header->reserved2);
+    header->reserved3 = le64_to_cpu(header->reserved3);
+    header->reserved4 = le64_to_cpu(header->reserved4);
+
+    header->volatile_header_offset =
+        le64_to_cpu(header->volatile_header_offset);
+    header->volatile_header_size = le64_to_cpu(header->volatile_header_size);
+
+    header->journal_header_offset = le64_to_cpu(header->journal_header_offset);
+    header->journal_header_size = le64_to_cpu(header->journal_header_size);
+
+    header->journal_offset = le64_to_cpu(header->journal_offset);
+    header->journal_size = le64_to_cpu(header->journal_size);
+
+    header->grain_dir_offset = le64_to_cpu(header->grain_dir_offset);
+    header->grain_dir_size = le64_to_cpu(header->grain_dir_size);
+
+    header->grain_tables_offset = le64_to_cpu(header->grain_tables_offset);
+    header->grain_tables_size = le64_to_cpu(header->grain_tables_size);
+
+    header->free_bitmap_offset = le64_to_cpu(header->free_bitmap_offset);
+    header->free_bitmap_size = le64_to_cpu(header->free_bitmap_size);
+
+    header->backmap_offset = le64_to_cpu(header->backmap_offset);
+    header->backmap_size = le64_to_cpu(header->backmap_size);
+
+    header->grains_offset = le64_to_cpu(header->grains_offset);
+    header->grains_size = le64_to_cpu(header->grains_size);
+
+    if (header->magic != SESPARSE_CONST_HEADER_MAGIC) {
+        error_setg(errp, "Bad const header magic: 0x%016" PRIx64,
+                   header->magic);
+        return -EINVAL;
+    }
+
+    if (header->version != 0x0000000200000001) {
+        error_setg(errp, "Unsupported version: 0x%016" PRIx64,
+                   header->version);
+        return -ENOTSUP;
+    }
+
+    if (header->grain_size != 8) {
+        error_setg(errp, "Unsupported grain size: %" PRIu64,
+                   header->grain_size);
+        return -ENOTSUP;
+    }
+
+    if (header->grain_table_size != 64) {
+        error_setg(errp, "Unsupported grain table size: %" PRIu64,
+                   header->grain_table_size);
+        return -ENOTSUP;
+    }
+
+    if (header->flags != 0) {
+        error_setg(errp, "Unsupported flags: 0x%016" PRIx64,
+                   header->flags);
+        return -ENOTSUP;
+    }
+
+    if (header->reserved1 != 0 || header->reserved2 != 0 ||
+        header->reserved3 != 0 || header->reserved4 != 0) {
+        error_setg(errp, "Unsupported reserved bits:"
+                   " 0x%016" PRIx64 " 0x%016" PRIx64
+                   " 0x%016" PRIx64 " 0x%016" PRIx64,
+                   header->reserved1, header->reserved2,
+                   header->reserved3, header->reserved4);
+        return -ENOTSUP;
+    }
+
+    /* check that padding is 0 */
+    if (!buffer_is_zero(header->pad, sizeof(header->pad))) {
+        error_setg(errp, "Unsupported non-zero const header padding");
+        return -ENOTSUP;
+    }
+
+    return 0;
+}
+
+static int check_se_sparse_volatile_header(VMDKSESparseVolatileHeader *header,
+                                           Error **errp)
+{
+    header->magic = le64_to_cpu(header->magic);
+    header->free_gt_number = le64_to_cpu(header->free_gt_number);
+    header->next_txn_seq_number = le64_to_cpu(header->next_txn_seq_number);
+    header->replay_journal = le64_to_cpu(header->replay_journal);
+
+    if (header->magic != SESPARSE_VOLATILE_HEADER_MAGIC) {
+        error_setg(errp, "Bad volatile header magic: 0x%016" PRIx64,
+                   header->magic);
+        return -EINVAL;
+    }
+
+    if (header->replay_journal) {
+        error_setg(errp, "Image is dirty, Replaying journal not supported");
+        return -ENOTSUP;
+    }
+
+    /* check that padding is 0 */
+    if (!buffer_is_zero(header->pad, sizeof(header->pad))) {
+        error_setg(errp, "Unsupported non-zero volatile header padding");
+        return -ENOTSUP;
+    }
+
+    return 0;
+}
+
+static int vmdk_open_se_sparse(BlockDriverState *bs,
+                               BdrvChild *file,
+                               int flags, Error **errp)
+{
+    int ret;
+    VMDKSESparseConstHeader const_header;
+    VMDKSESparseVolatileHeader volatile_header;
+    VmdkExtent *extent;
+
+    ret = bdrv_apply_auto_read_only(bs,
+            "No write support for seSparse images available", errp);
+    if (ret < 0) {
+        return ret;
+    }
+
+    assert(sizeof(const_header) == SECTOR_SIZE);
+
+    ret = bdrv_pread(file, 0, &const_header, sizeof(const_header));
+    if (ret < 0) {
+        bdrv_refresh_filename(file->bs);
+        error_setg_errno(errp, -ret,
+                         "Could not read const header from file '%s'",
+                         file->bs->filename);
+        return ret;
+    }
+
+    /* check const header */
+    ret = check_se_sparse_const_header(&const_header, errp);
+    if (ret < 0) {
+        return ret;
+    }
+
+    assert(sizeof(volatile_header) == SECTOR_SIZE);
+
+    ret = bdrv_pread(file,
+                     const_header.volatile_header_offset * SECTOR_SIZE,
+                     &volatile_header, sizeof(volatile_header));
+    if (ret < 0) {
+        bdrv_refresh_filename(file->bs);
+        error_setg_errno(errp, -ret,
+                         "Could not read volatile header from file '%s'",
+                         file->bs->filename);
+        return ret;
+    }
+
+    /* check volatile header */
+    ret = check_se_sparse_volatile_header(&volatile_header, errp);
+    if (ret < 0) {
+        return ret;
+    }
+
+    ret = vmdk_add_extent(bs, file, false,
+                          const_header.capacity,
+                          const_header.grain_dir_offset * SECTOR_SIZE,
+                          0,
+                          const_header.grain_dir_size *
+                          SECTOR_SIZE / sizeof(uint64_t),
+                          const_header.grain_table_size *
+                          SECTOR_SIZE / sizeof(uint64_t),
+                          const_header.grain_size,
+                          &extent,
+                          errp);
+    if (ret < 0) {
+        return ret;
+    }
+
+    extent->sesparse = true;
+    extent->sesparse_l2_tables_offset = const_header.grain_tables_offset;
+    extent->sesparse_clusters_offset = const_header.grains_offset;
+    extent->entry_size = sizeof(uint64_t);
+
+    ret = vmdk_init_tables(bs, extent, errp);
+    if (ret) {
+        /* free extent allocated by vmdk_add_extent */
+        vmdk_free_last_extent(bs);
+    }
+
+    return ret;
+}
+
 static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
                                QDict *options, Error **errp);
 
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
          * RW [size in sectors] SPARSE "file-name.vmdk"
          * RW [size in sectors] VMFS "file-name.vmdk"
          * RW [size in sectors] VMFSSPARSE "file-name.vmdk"
+         * RW [size in sectors] SESPARSE "file-name.vmdk"
          */
         flat_offset = -1;
         matches = sscanf(p, "%10s %" SCNd64 " %10s \"%511[^\n\r\"]\" %" SCNd64,
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
 
         if (sectors <= 0 ||
             (strcmp(type, "FLAT") && strcmp(type, "SPARSE") &&
-             strcmp(type, "VMFS") && strcmp(type, "VMFSSPARSE")) ||
+             strcmp(type, "VMFS") && strcmp(type, "VMFSSPARSE") &&
+             strcmp(type, "SESPARSE")) ||
             (strcmp(access, "RW"))) {
             continue;
         }
@@ -XXX,XX +XXX,XX @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
                 return ret;
             }
             extent = &s->extents[s->num_extents - 1];
+        } else if (!strcmp(type, "SESPARSE")) {
+            ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
+            if (ret) {
+                bdrv_unref_child(bs, extent_file);
+                return ret;
+            }
+            extent = &s->extents[s->num_extents - 1];
         } else {
             error_setg(errp, "Unsupported extent type '%s'", type);
             bdrv_unref_child(bs, extent_file);
@@ -XXX,XX +XXX,XX @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
     if (strcmp(ct, "monolithicFlat") &&
         strcmp(ct, "vmfs") &&
         strcmp(ct, "vmfsSparse") &&
+        strcmp(ct, "seSparse") &&
         strcmp(ct, "twoGbMaxExtentSparse") &&
         strcmp(ct, "twoGbMaxExtentFlat")) {
         error_setg(errp, "Unsupported image type '%s'", ct);
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
 {
     unsigned int l1_index, l2_offset, l2_index;
     int min_index, i, j;
-    uint32_t min_count, *l2_table;
+    uint32_t min_count;
+    void *l2_table;
     bool zeroed = false;
     int64_t ret;
     int64_t cluster_sector;
+    unsigned int l2_size_bytes = extent->l2_size * extent->entry_size;
 
     if (m_data) {
         m_data->valid = 0;
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
     if (l1_index >= extent->l1_size) {
         return VMDK_ERROR;
     }
-    l2_offset = extent->l1_table[l1_index];
+    if (extent->sesparse) {
+        uint64_t l2_offset_u64;
+
+        assert(extent->entry_size == sizeof(uint64_t));
+
+        l2_offset_u64 = ((uint64_t *)extent->l1_table)[l1_index];
+        if (l2_offset_u64 == 0) {
+            l2_offset = 0;
+        } else if ((l2_offset_u64 & 0xffffffff00000000) != 0x1000000000000000) {
+            /*
+             * Top most nibble is 0x1 if grain table is allocated.
+             * strict check - top most 4 bytes must be 0x10000000 since max
+             * supported size is 64TB for disk - so no more than 64TB / 16MB
+             * grain directories which is smaller than uint32,
+             * where 16MB is the only supported default grain table coverage.
+             */
+            return VMDK_ERROR;
+        } else {
+            l2_offset_u64 = l2_offset_u64 & 0x00000000ffffffff;
+            l2_offset_u64 = extent->sesparse_l2_tables_offset +
+                l2_offset_u64 * l2_size_bytes / SECTOR_SIZE;
+            if (l2_offset_u64 > 0x00000000ffffffff) {
+                return VMDK_ERROR;
+            }
+            l2_offset = (unsigned int)(l2_offset_u64);
+        }
+    } else {
+        assert(extent->entry_size == sizeof(uint32_t));
+        l2_offset = ((uint32_t *)extent->l1_table)[l1_index];
+    }
     if (!l2_offset) {
         return VMDK_UNALLOC;
     }
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
                     extent->l2_cache_counts[j] >>= 1;
                 }
             }
-            l2_table = extent->l2_cache + (i * extent->l2_size);
+            l2_table = (char *)extent->l2_cache + (i * l2_size_bytes);
             goto found;
         }
     }
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
             min_index = i;
         }
     }
-    l2_table = extent->l2_cache + (min_index * extent->l2_size);
+    l2_table = (char *)extent->l2_cache + (min_index * l2_size_bytes);
     BLKDBG_EVENT(extent->file, BLKDBG_L2_LOAD);
     if (bdrv_pread(extent->file,
                 (int64_t)l2_offset * 512,
                 l2_table,
-                extent->l2_size * sizeof(uint32_t)
-            ) != extent->l2_size * sizeof(uint32_t)) {
+                l2_size_bytes
+            ) != l2_size_bytes) {
         return VMDK_ERROR;
     }
 
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
     extent->l2_cache_counts[min_index] = 1;
  found:
     l2_index = ((offset >> 9) / extent->cluster_sectors) % extent->l2_size;
-    cluster_sector = le32_to_cpu(l2_table[l2_index]);
 
-    if (extent->has_zero_grain && cluster_sector == VMDK_GTE_ZEROED) {
-        zeroed = true;
+    if (extent->sesparse) {
+        cluster_sector = le64_to_cpu(((uint64_t *)l2_table)[l2_index]);
+        switch (cluster_sector & 0xf000000000000000) {
+        case 0x0000000000000000:
+            /* unallocated grain */
+            if (cluster_sector != 0) {
+                return VMDK_ERROR;
+            }
+            break;
+        case 0x1000000000000000:
+            /* scsi-unmapped grain - fallthrough */
+        case 0x2000000000000000:
+            /* zero grain */
+            zeroed = true;
+            break;
+        case 0x3000000000000000:
+            /* allocated grain */
+            cluster_sector = (((cluster_sector & 0x0fff000000000000) >> 48) |
+                              ((cluster_sector & 0x0000ffffffffffff) << 12));
+            cluster_sector = extent->sesparse_clusters_offset +
+                cluster_sector * extent->cluster_sectors;
+            break;
+        default:
+            return VMDK_ERROR;
+        }
+    } else {
+        cluster_sector = le32_to_cpu(((uint32_t *)l2_table)[l2_index]);
+
+        if (extent->has_zero_grain && cluster_sector == VMDK_GTE_ZEROED) {
+            zeroed = true;
+        }
     }
 
     if (!cluster_sector || zeroed) {
         if (!allocate) {
             return zeroed ? VMDK_ZEROED : VMDK_UNALLOC;
         }
+        assert(!extent->sesparse);
 
         if (extent->next_cluster_sector >= VMDK_EXTENT_MAX_SECTORS) {
             return VMDK_ERROR;
@@ -XXX,XX +XXX,XX @@ static int get_cluster_offset(BlockDriverState *bs,
             m_data->l1_index = l1_index;
             m_data->l2_index = l2_index;
             m_data->l2_offset = l2_offset;
-            m_data->l2_cache_entry = &l2_table[l2_index];
+            m_data->l2_cache_entry = ((uint32_t *)l2_table) + l2_index;
         }
     }
     *cluster_offset = cluster_sector << BDRV_SECTOR_BITS;
@@ -XXX,XX +XXX,XX @@ static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset,
         if (!extent) {
             return -EIO;
         }
+        if (extent->sesparse) {
+            return -ENOTSUP;
+        }
         offset_in_cluster = vmdk_find_offset_in_cluster(extent, offset);
         n_bytes = MIN(bytes, extent->cluster_sectors * BDRV_SECTOR_SIZE
                              - offset_in_cluster);
-- 
2.21.0

From: Pino Toscano <ptoscano@redhat.com>

Rewrite the implementation of the ssh block driver to use libssh instead
of libssh2.  The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support
older versions (which are not recommended though).

Adjust the iotest 207 according to the different error message, and to
find the default key type for localhost (to properly compare the
fingerprint with).
Contributed-by: Max Reitz <mreitz@redhat.com>

Adjust the various Docker/Travis scripts to use libssh when available
instead of libssh2. The mingw/mxe testing is dropped for now, as there
are no packages for it.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20190620200840.17655-1-ptoscano@redhat.com
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 5873173.t2JhDm7DL7@lindworm.usersys.redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 configure                                     |  65 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 652 ++++++++++--------
 .travis.yml                                   |   4 +-
 block/trace-events                            |  14 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |  54 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 449 insertions(+), 360 deletions(-)

diff --git a/configure b/configure
index XXXXXXX..XXXXXXX 100755
--- a/configure
+++ b/configure
@@ -XXX,XX +XXX,XX @@ auth_pam=""
 vte=""
 virglrenderer=""
 tpm=""
-libssh2=""
+libssh=""
 live_block_migration="yes"
 numa=""
 tcmalloc="no"
@@ -XXX,XX +XXX,XX @@ for opt do
   ;;
   --enable-tpm) tpm="yes"
   ;;
-  --disable-libssh2) libssh2="no"
+  --disable-libssh) libssh="no"
   ;;
-  --enable-libssh2) libssh2="yes"
+  --enable-libssh) libssh="yes"
   ;;
   --disable-live-block-migration) live_block_migration="no"
   ;;
@@ -XXX,XX +XXX,XX @@ disabled with --disable-FEATURE, default is enabled if available:
   coroutine-pool  coroutine freelist (better performance)
   glusterfs       GlusterFS backend
   tpm             TPM support
-  libssh2         ssh block device support
+  libssh          ssh block device support
   numa            libnuma support
   libxml2         for Parallels image format
   tcmalloc        tcmalloc support
@@ -XXX,XX +XXX,XX @@ EOF
 fi
 
 ##########################################
-# libssh2 probe
-min_libssh2_version=1.2.8
-if test "$libssh2" != "no" ; then
-  if $pkg_config --atleast-version=$min_libssh2_version libssh2; then
-    libssh2_cflags=$($pkg_config libssh2 --cflags)
-    libssh2_libs=$($pkg_config libssh2 --libs)
-    libssh2=yes
+# libssh probe
+if test "$libssh" != "no" ; then
+  if $pkg_config --exists libssh; then
+    libssh_cflags=$($pkg_config libssh --cflags)
+    libssh_libs=$($pkg_config libssh --libs)
+    libssh=yes
   else
-    if test "$libssh2" = "yes" ; then
-      error_exit "libssh2 >= $min_libssh2_version required for --enable-libssh2"
+    if test "$libssh" = "yes" ; then
+      error_exit "libssh required for --enable-libssh"
     fi
-    libssh2=no
+    libssh=no
   fi
 fi
 
 ##########################################
-# libssh2_sftp_fsync probe
+# Check for libssh 0.8
+# This is done like this instead of using the LIBSSH_VERSION_* and
+# SSH_VERSION_* macros because some distributions in the past shipped
+# snapshots of the future 0.8 from Git, and those snapshots did not
+# have updated version numbers (still referring to 0.7.0).
 
-if test "$libssh2" = "yes"; then
+if test "$libssh" = "yes"; then
   cat > $TMPC <<EOF
-#include <stdio.h>
-#include <libssh2.h>
-#include <libssh2_sftp.h>
-int main(void) {
-    LIBSSH2_SESSION *session;
-    LIBSSH2_SFTP *sftp;
-    LIBSSH2_SFTP_HANDLE *sftp_handle;
-    session = libssh2_session_init ();
-    sftp = libssh2_sftp_init (session);
-    sftp_handle = libssh2_sftp_open (sftp, "/", 0, 0);
-    libssh2_sftp_fsync (sftp_handle);
-    return 0;
-}
+#include <libssh/libssh.h>
+int main(void) { return ssh_get_server_publickey(NULL, NULL); }
 EOF
-  # libssh2_cflags/libssh2_libs defined in previous test.
-  if compile_prog "$libssh2_cflags" "$libssh2_libs" ; then
-    QEMU_CFLAGS="-DHAS_LIBSSH2_SFTP_FSYNC $QEMU_CFLAGS"
+  if compile_prog "$libssh_cflags" "$libssh_libs"; then
+    libssh_cflags="-DHAVE_LIBSSH_0_8 $libssh_cflags"
   fi
 fi
 
@@ -XXX,XX +XXX,XX @@ echo "GlusterFS support $glusterfs"
 echo "gcov              $gcov_tool"
 echo "gcov enabled      $gcov"
 echo "TPM support       $tpm"
-echo "libssh2 support   $libssh2"
+echo "libssh support    $libssh"
 echo "QOM debugging     $qom_cast_debug"
 echo "Live block migration $live_block_migration"
 echo "lzo support       $lzo"
@@ -XXX,XX +XXX,XX @@ if test "$glusterfs_iocb_has_stat" = "yes" ; then
   echo "CONFIG_GLUSTERFS_IOCB_HAS_STAT=y" >> $config_host_mak
 fi
 
-if test "$libssh2" = "yes" ; then
-  echo "CONFIG_LIBSSH2=m" >> $config_host_mak
-  echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
-  echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
+if test "$libssh" = "yes" ; then
+  echo "CONFIG_LIBSSH=m" >> $config_host_mak
+  echo "LIBSSH_CFLAGS=$libssh_cflags" >> $config_host_mak
+  echo "LIBSSH_LIBS=$libssh_libs" >> $config_host_mak
 fi
 
 if test "$live_block_migration" = "yes" ; then
diff --git a/block/Makefile.objs b/block/Makefile.objs
index XXXXXXX..XXXXXXX 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -XXX,XX +XXX,XX @@ block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
 block-obj-$(CONFIG_VXHS) += vxhs.o
-block-obj-$(CONFIG_LIBSSH2) += ssh.o
+block-obj-$(CONFIG_LIBSSH) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 block-obj-y += backup.o
@@ -XXX,XX +XXX,XX @@ rbd.o-libs         := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs     := $(GLUSTERFS_LIBS)
 vxhs.o-libs        := $(VXHS_LIBS)
-ssh.o-cflags       := $(LIBSSH2_CFLAGS)
-ssh.o-libs         := $(LIBSSH2_LIBS)
+ssh.o-cflags       := $(LIBSSH_CFLAGS)
+ssh.o-libs         := $(LIBSSH_LIBS)
 block-obj-dmg-bz2-$(CONFIG_BZIP2) += dmg-bz2.o
 block-obj-$(if $(CONFIG_DMG),m,n) += $(block-obj-dmg-bz2-y)
 dmg-bz2.o-libs     := $(BZIP2_LIBS)
diff --git a/block/ssh.c b/block/ssh.c
index XXXXXXX..XXXXXXX 100644
--- a/block/ssh.c
+++ b/block/ssh.c
@@ -XXX,XX +XXX,XX @@
 
 #include "qemu/osdep.h"
 
-#include <libssh2.h>
-#include <libssh2_sftp.h>
+#include <libssh/libssh.h>
+#include <libssh/sftp.h>
 
 #include "block/block_int.h"
 #include "block/qdict.h"
@@ -XXX,XX +XXX,XX @@
 #include "trace.h"
 
 /*
- * TRACE_LIBSSH2=<bitmask> enables tracing in libssh2 itself.  Note
- * that this requires that libssh2 was specially compiled with the
- * `./configure --enable-debug' option, so most likely you will have
- * to compile it yourself.  The meaning of <bitmask> is described
- * here: http://www.libssh2.org/libssh2_trace.html
+ * TRACE_LIBSSH=<level> enables tracing in libssh itself.
+ * The meaning of <level> is described here:
+ * http://api.libssh.org/master/group__libssh__log.html
  */
-#define TRACE_LIBSSH2 0 /* or try: LIBSSH2_TRACE_SFTP */
+#define TRACE_LIBSSH  0 /* see: SSH_LOG_* */
 
 typedef struct BDRVSSHState {
     /* Coroutine. */
@@ -XXX,XX +XXX,XX @@ typedef struct BDRVSSHState {
 
     /* SSH connection. */
     int sock;                         /* socket */
-    LIBSSH2_SESSION *session;         /* ssh session */
-    LIBSSH2_SFTP *sftp;               /* sftp session */
-    LIBSSH2_SFTP_HANDLE *sftp_handle; /* sftp remote file handle */
+    ssh_session session;              /* ssh session */
+    sftp_session sftp;                /* sftp session */
+    sftp_file sftp_handle;            /* sftp remote file handle */
 
-    /* See ssh_seek() function below. */
-    int64_t offset;
-    bool offset_op_read;
-
-    /* File attributes at open.  We try to keep the .filesize field
+    /*
+     * File attributes at open.  We try to keep the .size field
      * updated if it changes (eg by writing at the end of the file).
      */
-    LIBSSH2_SFTP_ATTRIBUTES attrs;
+    sftp_attributes attrs;
 
     InetSocketAddress *inet;
 
@@ -XXX,XX +XXX,XX @@ static void ssh_state_init(BDRVSSHState *s)
 {
     memset(s, 0, sizeof *s);
     s->sock = -1;
-    s->offset = -1;
     qemu_co_mutex_init(&s->lock);
 }
 
@@ -XXX,XX +XXX,XX @@ static void ssh_state_free(BDRVSSHState *s)
 {
     g_free(s->user);
 
+    if (s->attrs) {
+        sftp_attributes_free(s->attrs);
+    }
     if (s->sftp_handle) {
-        libssh2_sftp_close(s->sftp_handle);
+        sftp_close(s->sftp_handle);
     }
     if (s->sftp) {
-        libssh2_sftp_shutdown(s->sftp);
+        sftp_free(s->sftp);
     }
     if (s->session) {
-        libssh2_session_disconnect(s->session,
-                                   "from qemu ssh client: "
-                                   "user closed the connection");
-        libssh2_session_free(s->session);
-    }
-    if (s->sock >= 0) {
-        close(s->sock);
+        ssh_disconnect(s->session);
+        ssh_free(s->session); /* This frees s->sock */
     }
 }
 
@@ -XXX,XX +XXX,XX @@ session_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
     va_end(args);
 
     if (s->session) {
-        char *ssh_err;
+        const char *ssh_err;
         int ssh_err_code;
 
-        /* This is not an errno.  See <libssh2.h>. */
-        ssh_err_code = libssh2_session_last_error(s->session,
-                                                  &ssh_err, NULL, 0);
-        error_setg(errp, "%s: %s (libssh2 error code: %d)",
+        /* This is not an errno.  See <libssh/libssh.h>. */
+        ssh_err = ssh_get_error(s->session);
+        ssh_err_code = ssh_get_error_code(s->session);
+        error_setg(errp, "%s: %s (libssh error code: %d)",
                    msg, ssh_err, ssh_err_code);
     } else {
         error_setg(errp, "%s", msg);
@@ -XXX,XX +XXX,XX @@ sftp_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
     va_end(args);
 
     if (s->sftp) {
-        char *ssh_err;
+        const char *ssh_err;
         int ssh_err_code;
-        unsigned long sftp_err_code;
+        int sftp_err_code;
 
-        /* This is not an errno.  See <libssh2.h>. */
-        ssh_err_code = libssh2_session_last_error(s->session,
-                                                  &ssh_err, NULL, 0);
-        /* See <libssh2_sftp.h>. */
-        sftp_err_code = libssh2_sftp_last_error((s)->sftp);
+        /* This is not an errno.  See <libssh/libssh.h>. */
+        ssh_err = ssh_get_error(s->session);
+        ssh_err_code = ssh_get_error_code(s->session);
+        /* See <libssh/sftp.h>. */
+        sftp_err_code = sftp_get_error(s->sftp);
 
         error_setg(errp,
-                   "%s: %s (libssh2 error code: %d, sftp error code: %lu)",
+                   "%s: %s (libssh error code: %d, sftp error code: %d)",
                    msg, ssh_err, ssh_err_code, sftp_err_code);
     } else {
         error_setg(errp, "%s", msg);
@@ -XXX,XX +XXX,XX @@ sftp_error_setg(Error **errp, BDRVSSHState *s, const char *fs, ...)
 
 static void sftp_error_trace(BDRVSSHState *s, const char *op)
 {
-    char *ssh_err;
+    const char *ssh_err;
     int ssh_err_code;
-    unsigned long sftp_err_code;
+    int sftp_err_code;
 
-    /* This is not an errno.  See <libssh2.h>. */
-    ssh_err_code = libssh2_session_last_error(s->session,
-                                              &ssh_err, NULL, 0);
-    /* See <libssh2_sftp.h>. */
-    sftp_err_code = libssh2_sftp_last_error((s)->sftp);
+    /* This is not an errno.  See <libssh/libssh.h>. */
+    ssh_err = ssh_get_error(s->session);
+    ssh_err_code = ssh_get_error_code(s->session);
+    /* See <libssh/sftp.h>. */
+    sftp_err_code = sftp_get_error(s->sftp);
 
     trace_sftp_error(op, ssh_err, ssh_err_code, sftp_err_code);
 }
@@ -XXX,XX +XXX,XX @@ static void ssh_parse_filename(const char *filename, QDict *options,
     parse_uri(filename, options, errp);
 }
 
-static int check_host_key_knownhosts(BDRVSSHState *s,
-                                     const char *host, int port, Error **errp)
+static int check_host_key_knownhosts(BDRVSSHState *s, Error **errp)
 {
-    const char *home;
-    char *knh_file = NULL;
-    LIBSSH2_KNOWNHOSTS *knh = NULL;
-    struct libssh2_knownhost *found;
-    int ret, r;
-    const char *hostkey;
-    size_t len;
-    int type;
-
-    hostkey = libssh2_session_hostkey(s->session, &len, &type);
-    if (!hostkey) {
+    int ret;
+#ifdef HAVE_LIBSSH_0_8
+    enum ssh_known_hosts_e state;
+    int r;
+    ssh_key pubkey;
+    enum ssh_keytypes_e pubkey_type;
+    unsigned char *server_hash = NULL;
+    size_t server_hash_len;
+    char *fingerprint = NULL;
+
+    state = ssh_session_is_known_server(s->session);
+    trace_ssh_server_status(state);
+
+    switch (state) {
+    case SSH_KNOWN_HOSTS_OK:
+        /* OK */
+        trace_ssh_check_host_key_knownhosts();
+        break;
+    case SSH_KNOWN_HOSTS_CHANGED:
         ret = -EINVAL;
-        session_error_setg(errp, s, "failed to read remote host key");
+        r = ssh_get_server_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA256,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA256,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts; this may be a possible attack",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp,
+                       "host key does not match the one in known_hosts; this "
+                       "may be a possible attack");
+        }
         goto out;
-    }
-
-    knh = libssh2_knownhost_init(s->session);
-    if (!knh) {
+    case SSH_KNOWN_HOSTS_OTHER:
         ret = -EINVAL;
-        session_error_setg(errp, s,
-                           "failed to initialize known hosts support");
+        error_setg(errp,
+                   "host key for this server not found, another type exists");
+        goto out;
+    case SSH_KNOWN_HOSTS_UNKNOWN:
+        ret = -EINVAL;
+        error_setg(errp, "no host key was found in known_hosts");
+        goto out;
+    case SSH_KNOWN_HOSTS_NOT_FOUND:
+        ret = -ENOENT;
+        error_setg(errp, "known_hosts file not found");
+        goto out;
+    case SSH_KNOWN_HOSTS_ERROR:
+        ret = -EINVAL;
+        error_setg(errp, "error while checking the host");
+        goto out;
+    default:
+        ret = -EINVAL;
+        error_setg(errp, "error while checking for known server (%d)", state);
         goto out;
     }
+#else /* !HAVE_LIBSSH_0_8 */
+    int state;
 
-    home = getenv("HOME");
-    if (home) {
-        knh_file = g_strdup_printf("%s/.ssh/known_hosts", home);
-    } else {
-        knh_file = g_strdup_printf("/root/.ssh/known_hosts");
-    }
-
-    /* Read all known hosts from OpenSSH-style known_hosts file. */
-    libssh2_knownhost_readfile(knh, knh_file, LIBSSH2_KNOWNHOST_FILE_OPENSSH);
+    state = ssh_is_server_known(s->session);
+    trace_ssh_server_status(state);
 
-    r = libssh2_knownhost_checkp(knh, host, port, hostkey, len,
-                                 LIBSSH2_KNOWNHOST_TYPE_PLAIN|
-                                 LIBSSH2_KNOWNHOST_KEYENC_RAW,
-                                 &found);
-    switch (r) {
-    case LIBSSH2_KNOWNHOST_CHECK_MATCH:
+    switch (state) {
+    case SSH_SERVER_KNOWN_OK:
         /* OK */
-        trace_ssh_check_host_key_knownhosts(found->key);
+        trace_ssh_check_host_key_knownhosts();
         break;
-    case LIBSSH2_KNOWNHOST_CHECK_MISMATCH:
+    case SSH_SERVER_KNOWN_CHANGED:
         ret = -EINVAL;
-        session_error_setg(errp, s,
-                      "host key does not match the one in known_hosts"
-                      " (found key %s)", found->key);
+        error_setg(errp,
+                   "host key does not match the one in known_hosts; this "
+                   "may be a possible attack");
         goto out;
-    case LIBSSH2_KNOWNHOST_CHECK_NOTFOUND:
+    case SSH_SERVER_FOUND_OTHER:
         ret = -EINVAL;
-        session_error_setg(errp, s, "no host key was found in known_hosts");
+        error_setg(errp,
+                   "host key for this server not found, another type exists");
+        goto out;
+    case SSH_SERVER_FILE_NOT_FOUND:
+        ret = -ENOENT;
+        error_setg(errp, "known_hosts file not found");
         goto out;
-    case LIBSSH2_KNOWNHOST_CHECK_FAILURE:
+    case SSH_SERVER_NOT_KNOWN:
         ret = -EINVAL;
-        session_error_setg(errp, s,
-                      "failure matching the host key with known_hosts");
+        error_setg(errp, "no host key was found in known_hosts");
+        goto out;
+    case SSH_SERVER_ERROR:
+        ret = -EINVAL;
+        error_setg(errp, "server error");
         goto out;
     default:
         ret = -EINVAL;
-        session_error_setg(errp, s, "unknown error matching the host key"
-                      " with known_hosts (%d)", r);
+        error_setg(errp, "error while checking for known server (%d)", state);
         goto out;
     }
+#endif /* !HAVE_LIBSSH_0_8 */
 
     /* known_hosts checking successful. */
     ret = 0;
 
  out:
-    if (knh != NULL) {
-        libssh2_knownhost_free(knh);
-    }
-    g_free(knh_file);
     return ret;
 }
 
@@ -XXX,XX +XXX,XX @@ static int compare_fingerprint(const unsigned char *fingerprint, size_t len,
 
 static int
 check_host_key_hash(BDRVSSHState *s, const char *hash,
-                    int hash_type, size_t fingerprint_len, Error **errp)
+                    enum ssh_publickey_hash_type type, Error **errp)
 {
-    const char *fingerprint;
-
-    fingerprint = libssh2_hostkey_hash(s->session, hash_type);
-    if (!fingerprint) {
+    int r;
+    ssh_key pubkey;
+    unsigned char *server_hash;
+    size_t server_hash_len;
+
+#ifdef HAVE_LIBSSH_0_8
+    r = ssh_get_server_publickey(s->session, &pubkey);
+#else
+    r = ssh_get_publickey(s->session, &pubkey);
+#endif
+    if (r != SSH_OK) {
         session_error_setg(errp, s, "failed to read remote host key");
         return -EINVAL;
     }
 
-    if(compare_fingerprint((unsigned char *) fingerprint, fingerprint_len,
-                           hash) != 0) {
+    r = ssh_get_publickey_hash(pubkey, type, &server_hash, &server_hash_len);
+    ssh_key_free(pubkey);
+    if (r != 0) {
+        session_error_setg(errp, s,
+                           "failed reading the hash of the server SSH key");
+        return -EINVAL;
+    }
+
+    r = compare_fingerprint(server_hash, server_hash_len, hash);
+    ssh_clean_pubkey_hash(&server_hash);
+    if (r != 0) {
         error_setg(errp, "remote host key does not match host_key_check '%s'",
                    hash);
         return -EPERM;
@@ -XXX,XX +XXX,XX @@ check_host_key_hash(BDRVSSHState *s, const char *hash,
     return 0;
 }
 
-static int check_host_key(BDRVSSHState *s, const char *host, int port,
-                          SshHostKeyCheck *hkc, Error **errp)
+static int check_host_key(BDRVSSHState *s, SshHostKeyCheck *hkc, Error **errp)
 {
     SshHostKeyCheckMode mode;
 
@@ -XXX,XX +XXX,XX @@ static int check_host_key(BDRVSSHState *s, const char *host, int port,
     case SSH_HOST_KEY_CHECK_MODE_HASH:
         if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_MD5) {
             return check_host_key_hash(s, hkc->u.hash.hash,
-                                       LIBSSH2_HOSTKEY_HASH_MD5, 16, errp);
+                                       SSH_PUBLICKEY_HASH_MD5, errp);
         } else if (hkc->u.hash.type == SSH_HOST_KEY_CHECK_HASH_TYPE_SHA1) {
             return check_host_key_hash(s, hkc->u.hash.hash,
-                                       LIBSSH2_HOSTKEY_HASH_SHA1, 20, errp);
+                                       SSH_PUBLICKEY_HASH_SHA1, errp);
         }
         g_assert_not_reached();
         break;
     case SSH_HOST_KEY_CHECK_MODE_KNOWN_HOSTS:
-        return check_host_key_knownhosts(s, host, port, errp);
+        return check_host_key_knownhosts(s, errp);
     default:
         g_assert_not_reached();
     }
@@ -XXX,XX +XXX,XX @@ static int check_host_key(BDRVSSHState *s, const char *host, int port,
     return -EINVAL;
 }
 
-static int authenticate(BDRVSSHState *s, const char *user, Error **errp)
+static int authenticate(BDRVSSHState *s, Error **errp)
 {
     int r, ret;
-    const char *userauthlist;
-    LIBSSH2_AGENT *agent = NULL;
-    struct libssh2_agent_publickey *identity;
-    struct libssh2_agent_publickey *prev_identity = NULL;
+    int method;
 
-    userauthlist = libssh2_userauth_list(s->session, user, strlen(user));
-    if (strstr(userauthlist, "publickey") == NULL) {
+    /* Try to authenticate with the "none" method. */
+    r = ssh_userauth_none(s->session, NULL);
+    if (r == SSH_AUTH_ERROR) {
         ret = -EPERM;
-        error_setg(errp,
-                "remote server does not support \"publickey\" authentication");
+        session_error_setg(errp, s, "failed to authenticate using none "
+                                    "authentication");
         goto out;
-    }
-
-    /* Connect to ssh-agent and try each identity in turn. */
-    agent = libssh2_agent_init(s->session);
-    if (!agent) {
-        ret = -EINVAL;
-        session_error_setg(errp, s, "failed to initialize ssh-agent support");
-        goto out;
-    }
-    if (libssh2_agent_connect(agent)) {
-        ret = -ECONNREFUSED;
-        session_error_setg(errp, s, "failed to connect to ssh-agent");
-        goto out;
-    }
-    if (libssh2_agent_list_identities(agent)) {
-        ret = -EINVAL;
-        session_error_setg(errp, s,
-                           "failed requesting identities from ssh-agent");
+    } else if (r == SSH_AUTH_SUCCESS) {
+        /* Authenticated! */
+        ret = 0;
         goto out;
     }
 
-    for(;;) {
-        r = libssh2_agent_get_identity(agent, &identity, prev_identity);
-        if (r == 1) {           /* end of list */
-            break;
-        }
-        if (r < 0) {
+    method = ssh_userauth_list(s->session, NULL);
+    trace_ssh_auth_methods(method);
+
+    /*
+     * Try to authenticate with publickey, using the ssh-agent
+     * if available.
+     */
+    if (method & SSH_AUTH_METHOD_PUBLICKEY) {
+        r = ssh_userauth_publickey_auto(s->session, NULL, NULL);
+        if (r == SSH_AUTH_ERROR) {
             ret = -EINVAL;
-            session_error_setg(errp, s,
-                               "failed to obtain identity from ssh-agent");
+            session_error_setg(errp, s, "failed to authenticate using "
+                                        "publickey authentication");
             goto out;
-        }
-        r = libssh2_agent_userauth(agent, user, identity);
-        if (r == 0) {
+        } else if (r == SSH_AUTH_SUCCESS) {
             /* Authenticated! */
             ret = 0;
             goto out;
         }
-        /* Failed to authenticate with this identity, try the next one. */
-        prev_identity = identity;
     }
 
     ret = -EPERM;
@@ -XXX,XX +XXX,XX @@ static int authenticate(BDRVSSHState *s, const char *user, Error **errp)
                "and the identities held by your ssh-agent");
 
  out:
-    if (agent != NULL) {
-        /* Note: libssh2 implementation implicitly calls
-         * libssh2_agent_disconnect if necessary.
-         */
-        libssh2_agent_free(agent);
-    }
-
     return ret;
 }
 
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
                           int ssh_flags, int creat_mode, Error **errp)
 {
     int r, ret;
-    long port = 0;
+    unsigned int port = 0;
+    int new_sock = -1;
 
     if (opts->has_user) {
         s->user = g_strdup(opts->user);
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
     s->inet = opts->server;
     opts->server = NULL;
 
-    if (qemu_strtol(s->inet->port, NULL, 10, &port) < 0) {
+    if (qemu_strtoui(s->inet->port, NULL, 10, &port) < 0) {
         error_setg(errp, "Use only numeric port value");
         ret = -EINVAL;
         goto err;
     }
 
     /* Open the socket and connect. */
-    s->sock = inet_connect_saddr(s->inet, errp);
-    if (s->sock < 0) {
+    new_sock = inet_connect_saddr(s->inet, errp);
+    if (new_sock < 0) {
         ret = -EIO;
         goto err;
     }
 
+    /*
+     * Try to disable the Nagle algorithm on TCP sockets to reduce latency,
+     * but do not fail if it cannot be disabled.
+     */
+    r = socket_set_nodelay(new_sock);
+    if (r < 0) {
+        warn_report("can't set TCP_NODELAY for the ssh server %s: %s",
+                    s->inet->host, strerror(errno));
+    }
+
     /* Create SSH session. */
-    s->session = libssh2_session_init();
+    s->session = ssh_new();
     if (!s->session) {
         ret = -EINVAL;
-        session_error_setg(errp, s, "failed to initialize libssh2 session");
+        session_error_setg(errp, s, "failed to initialize libssh session");
         goto err;
     }
 
-#if TRACE_LIBSSH2 != 0
-    libssh2_trace(s->session, TRACE_LIBSSH2);
-#endif
+    /*
+     * Make sure we are in blocking mode during the connection and
+     * authentication phases.
+     */
+    ssh_set_blocking(s->session, 1);
 
-    r = libssh2_session_handshake(s->session, s->sock);
-    if (r != 0) {
+    r = ssh_options_set(s->session, SSH_OPTIONS_USER, s->user);
+    if (r < 0) {
+        ret = -EINVAL;
+        session_error_setg(errp, s,
+                           "failed to set the user in the libssh session");
+        goto err;
+    }
+
+    r = ssh_options_set(s->session, SSH_OPTIONS_HOST, s->inet->host);
+    if (r < 0) {
+        ret = -EINVAL;
+        session_error_setg(errp, s,
+                           "failed to set the host in the libssh session");
+        goto err;
+    }
+
+    if (port > 0) {
+        r = ssh_options_set(s->session, SSH_OPTIONS_PORT, &port);
+        if (r < 0) {
+            ret = -EINVAL;
+            session_error_setg(errp, s,
+                               "failed to set the port in the libssh session");
+            goto err;
+        }
+    }
+
+    r = ssh_options_set(s->session, SSH_OPTIONS_COMPRESSION, "none");
+    if (r < 0) {
+        ret = -EINVAL;
+        session_error_setg(errp, s,
+                           "failed to disable the compression in the libssh "
+                           "session");
+        goto err;
+    }
+
+    /* Read ~/.ssh/config. */
+    r = ssh_options_parse_config(s->session, NULL);
+    if (r < 0) {
+        ret = -EINVAL;
+        session_error_setg(errp, s, "failed to parse ~/.ssh/config");
+        goto err;
+    }
+
+    r = ssh_options_set(s->session, SSH_OPTIONS_FD, &new_sock);
+    if (r < 0) {
+        ret = -EINVAL;
+        session_error_setg(errp, s,
+                           "failed to set the socket in the libssh session");
+        goto err;
+    }
+    /* libssh took ownership of the socket. */
+    s->sock = new_sock;
+    new_sock = -1;
+
+    /* Connect. */
+    r = ssh_connect(s->session);
+    if (r != SSH_OK) {
         ret = -EINVAL;
         session_error_setg(errp, s, "failed to establish SSH session");
         goto err;
     }
 
     /* Check the remote host's key against known_hosts. */
-    ret = check_host_key(s, s->inet->host, port, opts->host_key_check, errp);
+    ret = check_host_key(s, opts->host_key_check, errp);
     if (ret < 0) {
         goto err;
     }
 
     /* Authenticate. */
-    ret = authenticate(s, s->user, errp);
+    ret = authenticate(s, errp);
     if (ret < 0) {
         goto err;
     }
 
     /* Start SFTP. */
-    s->sftp = libssh2_sftp_init(s->session);
+    s->sftp = sftp_new(s->session);
     if (!s->sftp) {
-        session_error_setg(errp, s, "failed to initialize sftp handle");
+        session_error_setg(errp, s, "failed to create sftp handle");
+        ret = -EINVAL;
+        goto err;
+    }
+
+    r = sftp_init(s->sftp);
+    if (r < 0) {
+        sftp_error_setg(errp, s, "failed to initialize sftp handle");
         ret = -EINVAL;
         goto err;
     }
 
     /* Open the remote file. */
     trace_ssh_connect_to_ssh(opts->path, ssh_flags, creat_mode);
-    s->sftp_handle = libssh2_sftp_open(s->sftp, opts->path, ssh_flags,
-                                       creat_mode);
+    s->sftp_handle = sftp_open(s->sftp, opts->path, ssh_flags, creat_mode);
     if (!s->sftp_handle) {
-        session_error_setg(errp, s, "failed to open remote file '%s'",
-                           opts->path);
+        sftp_error_setg(errp, s, "failed to open remote file '%s'",
+                        opts->path);
         ret = -EINVAL;
         goto err;
     }
 
-    r = libssh2_sftp_fstat(s->sftp_handle, &s->attrs);
-    if (r < 0) {
+    /* Make sure the SFTP file is handled in blocking mode. */
+    sftp_file_set_blocking(s->sftp_handle);
+
+    s->attrs = sftp_fstat(s->sftp_handle);
+    if (!s->attrs) {
         sftp_error_setg(errp, s, "failed to read file attributes");
         return -EINVAL;
     }
@@ -XXX,XX +XXX,XX @@ static int connect_to_ssh(BDRVSSHState *s, BlockdevOptionsSsh *opts,
     return 0;
 
  err:
+    if (s->attrs) {
+        sftp_attributes_free(s->attrs);
+    }
+    s->attrs = NULL;
     if (s->sftp_handle) {
-        libssh2_sftp_close(s->sftp_handle);
+        sftp_close(s->sftp_handle);
     }
     s->sftp_handle = NULL;
     if (s->sftp) {
-        libssh2_sftp_shutdown(s->sftp);
+        sftp_free(s->sftp);
     }
     s->sftp = NULL;
     if (s->session) {
-        libssh2_session_disconnect(s->session,
-                                   "from qemu ssh client: "
-                                   "error opening connection");
-        libssh2_session_free(s->session);
+        ssh_disconnect(s->session);
+        ssh_free(s->session);
     }
     s->session = NULL;
+    s->sock = -1;
+    if (new_sock >= 0) {
+        close(new_sock);
+    }
 
     return ret;
 }
@@ -XXX,XX +XXX,XX @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
 
     ssh_state_init(s);
 
-    ssh_flags = LIBSSH2_FXF_READ;
+    ssh_flags = 0;
     if (bdrv_flags & BDRV_O_RDWR) {
-        ssh_flags |= LIBSSH2_FXF_WRITE;
+        ssh_flags |= O_RDWR;
+    } else {
+        ssh_flags |= O_RDONLY;
     }
 
     opts = ssh_parse_options(options, errp);
@@ -XXX,XX +XXX,XX @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
     }
 
     /* Go non-blocking. */
-    libssh2_session_set_blocking(s->session, 0);
+    ssh_set_blocking(s->session, 0);
 
     qapi_free_BlockdevOptionsSsh(opts);
 
     return 0;
 
  err:
-    if (s->sock >= 0) {
-        close(s->sock);
-    }
-    s->sock = -1;
-
     qapi_free_BlockdevOptionsSsh(opts);
 
     return ret;
@@ -XXX,XX +XXX,XX @@ static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp)
 {
     ssize_t ret;
     char c[1] = { '\0' };
-    int was_blocking = libssh2_session_get_blocking(s->session);
+    int was_blocking = ssh_is_blocking(s->session);
 
     /* offset must be strictly greater than the current size so we do
      * not overwrite anything */
-    assert(offset > 0 && offset > s->attrs.filesize);
+    assert(offset > 0 && offset > s->attrs->size);
 
-    libssh2_session_set_blocking(s->session, 1);
+    ssh_set_blocking(s->session, 1);
 
-    libssh2_sftp_seek64(s->sftp_handle, offset - 1);
-    ret = libssh2_sftp_write(s->sftp_handle, c, 1);
+    sftp_seek64(s->sftp_handle, offset - 1);
+    ret = sftp_write(s->sftp_handle, c, 1);
 
-    libssh2_session_set_blocking(s->session, was_blocking);
+    ssh_set_blocking(s->session, was_blocking);
 
     if (ret < 0) {
         sftp_error_setg(errp, s, "Failed to grow file");
         return -EIO;
     }
 
-    s->attrs.filesize = offset;
+    s->attrs->size = offset;
     return 0;
 }
 
@@ -XXX,XX +XXX,XX @@ static int ssh_co_create(BlockdevCreateOptions *options, Error **errp)
     ssh_state_init(&s);
 
     ret = connect_to_ssh(&s, opts->location,
-                         LIBSSH2_FXF_READ|LIBSSH2_FXF_WRITE|
-                         LIBSSH2_FXF_CREAT|LIBSSH2_FXF_TRUNC,
+                         O_RDWR | O_CREAT | O_TRUNC,
                          0644, errp);
     if (ret < 0) {
         goto fail;
@@ -XXX,XX +XXX,XX @@ static int ssh_has_zero_init(BlockDriverState *bs)
     /* Assume false, unless we can positively prove it's true. */
     int has_zero_init = 0;
 
-    if (s->attrs.flags & LIBSSH2_SFTP_ATTR_PERMISSIONS) {
-        if (s->attrs.permissions & LIBSSH2_SFTP_S_IFREG) {
-            has_zero_init = 1;
-        }
+    if (s->attrs->type == SSH_FILEXFER_TYPE_REGULAR) {
+        has_zero_init = 1;
     }
 
     return has_zero_init;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void co_yield(BDRVSSHState *s, BlockDriverState *bs)
         .co = qemu_coroutine_self()
     };
 
-    r = libssh2_session_block_directions(s->session);
+    r = ssh_get_poll_flags(s->session);
 
-    if (r & LIBSSH2_SESSION_BLOCK_INBOUND) {
+    if (r & SSH_READ_PENDING) {
         rd_handler = restart_coroutine;
     }
-    if (r & LIBSSH2_SESSION_BLOCK_OUTBOUND) {
+    if (r & SSH_WRITE_PENDING) {
         wr_handler = restart_coroutine;
     }
 
@@ -XXX,XX +XXX,XX @@ static coroutine_fn void co_yield(BDRVSSHState *s, BlockDriverState *bs)
     trace_ssh_co_yield_back(s->sock);
 }
 
-/* SFTP has a function `libssh2_sftp_seek64' which seeks to a position
- * in the remote file.  Notice that it just updates a field in the
- * sftp_handle structure, so there is no network traffic and it cannot
- * fail.
- *
- * However, `libssh2_sftp_seek64' does have a catastrophic effect on
- * performance since it causes the handle to throw away all in-flight
- * reads and buffered readahead data.  Therefore this function tries
- * to be intelligent about when to call the underlying libssh2 function.
- */
-#define SSH_SEEK_WRITE 0
-#define SSH_SEEK_READ  1
-#define SSH_SEEK_FORCE 2
-
-static void ssh_seek(BDRVSSHState *s, int64_t offset, int flags)
-{
-    bool op_read = (flags & SSH_SEEK_READ) != 0;
-    bool force = (flags & SSH_SEEK_FORCE) != 0;
-
-    if (force || op_read != s->offset_op_read || offset != s->offset) {
-        trace_ssh_seek(offset);
-        libssh2_sftp_seek64(s->sftp_handle, offset);
-        s->offset = offset;
-        s->offset_op_read = op_read;
-    }
-}
-
 static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
                                  int64_t offset, size_t size,
                                  QEMUIOVector *qiov)
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
 
     trace_ssh_read(offset, size);
 
-    ssh_seek(s, offset, SSH_SEEK_READ);
+    trace_ssh_seek(offset);
+    sftp_seek64(s->sftp_handle, offset);
 
     /* This keeps track of the current iovec element ('i'), where we
      * will write to next ('buf'), and the end of the current iovec
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_read(BDRVSSHState *s, BlockDriverState *bs,
     buf = i->iov_base;
     end_of_vec = i->iov_base + i->iov_len;
 
-    /* libssh2 has a hard-coded limit of 2000 bytes per request,
-     * although it will also do readahead behind our backs.  Therefore
-     * we may have to do repeated reads here until we have read 'size'
-     * bytes.
-     */
     for (got = 0; got < size; ) {
+        size_t request_read_size;
     again:
-        trace_ssh_read_buf(buf, end_of_vec - buf);
-        r = libssh2_sftp_read(s->sftp_handle, buf, end_of_vec - buf);
-        trace_ssh_read_return(r);
+        /*
+         * The size of SFTP packets is limited to 32K bytes, so limit
+         * the amount of data requested to 16K, as libssh currently
+         * does not handle multiple requests on its own.
+         */
+        request_read_size = MIN(end_of_vec - buf, 16384);
+        trace_ssh_read_buf(buf, end_of_vec - buf, request_read_size);
+        r = sftp_read(s->sftp_handle, buf, request_read_size);
+        trace_ssh_read_return(r, sftp_get_error(s->sftp));
 
-        if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
+        if (r == SSH_AGAIN) {
             co_yield(s, bs);
             goto again;
         }
-        if (r < 0) {
-            sftp_error_trace(s, "read");
-            s->offset = -1;
-            return -EIO;
-        }
-        if (r == 0) {
+        if (r == SSH_EOF || (r == 0 && sftp_get_error(s->sftp) == SSH_FX_EOF)) {
             /* EOF: Short read so pad the buffer with zeroes and return it. */
             qemu_iovec_memset(qiov, got, 0, size - got);
             return 0;
         }
+        if (r <= 0) {
+            sftp_error_trace(s, "read");
+            return -EIO;
+        }
 
         got += r;
         buf += r;
-        s->offset += r;
         if (buf >= end_of_vec && got < size) {
             i++;
             buf = i->iov_base;
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
 
     trace_ssh_write(offset, size);
 
-    ssh_seek(s, offset, SSH_SEEK_WRITE);
+    trace_ssh_seek(offset);
+    sftp_seek64(s->sftp_handle, offset);
 
     /* This keeps track of the current iovec element ('i'), where we
      * will read from next ('buf'), and the end of the current iovec
@@ -XXX,XX +XXX,XX @@ static int ssh_write(BDRVSSHState *s, BlockDriverState *bs,
     end_of_vec = i->iov_base + i->iov_len;
 
     for (written = 0; written < size; ) {
+        size_t request_write_size;
     again:
-        trace_ssh_write_buf(buf, end_of_vec - buf);
-        r = libssh2_sftp_write(s->sftp_handle, buf, end_of_vec - buf);
-        trace_ssh_write_return(r);
+        /*
+         * Avoid too large data packets, as libssh currently does not
+         * handle multiple requests on its own.
+         */
+        request_write_size = MIN(end_of_vec - buf, 131072);
+        trace_ssh_write_buf(buf, end_of_vec - buf, request_write_size);
+        r = sftp_write(s->sftp_handle, buf, request_write_size);
+        trace_ssh_write_return(r, sftp_get_error(s->sftp));
 
-        if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
+        if (r == SSH_AGAIN) {
             co_yield(s, bs);
             goto again;
         }
         if (r < 0) {
             sftp_error_trace(s, "write");
-            s->offset = -1;
             return -EIO;
         }
-        /* The libssh2 API is very unclear about this.  A comment in
-         * the code says "nothing was acked, and no EAGAIN was
-         * received!" which apparently means that no data got sent
-         * out, and the underlying channel didn't return any EAGAIN
-         * indication.  I think this is a bug in either libssh2 or
-         * OpenSSH (server-side).  In any case, forcing a seek (to
-         * discard libssh2 internal buffers), and then trying again
-         * works for me.
-         */
-        if (r == 0) {
-            ssh_seek(s, offset + written, SSH_SEEK_WRITE|SSH_SEEK_FORCE);
-            co_yield(s, bs);
-            goto again;
-        }
 
         written += r;
         buf += r;
-        s->offset += r;
         if (buf >= end_of_vec && written < size) {
             i++;
             buf = i->iov_base;
             end_of_vec = i->iov_base + i->iov_len;
         }
 
-        if (offset + written > s->attrs.filesize)
-            s->attrs.filesize = offset + written;
+        if (offset + written > s->attrs->size) {
+            s->attrs->size = offset + written;
+        }
     }
 
     return 0;
@@ -XXX,XX +XXX,XX @@ static void unsafe_flush_warning(BDRVSSHState *s, const char *what)
     }
 }
 
-#ifdef HAS_LIBSSH2_SFTP_FSYNC
+#ifdef HAVE_LIBSSH_0_8
 
 static coroutine_fn int ssh_flush(BDRVSSHState *s, BlockDriverState *bs)
 {
     int r;
 
     trace_ssh_flush();
+
+    if (!sftp_extension_supported(s->sftp, "fsync@openssh.com", "1")) {
+        unsafe_flush_warning(s, "OpenSSH >= 6.3");
+        return 0;
+    }
  again:
-    r = libssh2_sftp_fsync(s->sftp_handle);
-    if (r == LIBSSH2_ERROR_EAGAIN || r == LIBSSH2_ERROR_TIMEOUT) {
+    r = sftp_fsync(s->sftp_handle);
+    if (r == SSH_AGAIN) {
         co_yield(s, bs);
         goto again;
     }
-    if (r == LIBSSH2_ERROR_SFTP_PROTOCOL &&
-        libssh2_sftp_last_error(s->sftp) == LIBSSH2_FX_OP_UNSUPPORTED) {
-        unsafe_flush_warning(s, "OpenSSH >= 6.3");
-        return 0;
-    }
     if (r < 0) {
         sftp_error_trace(s, "fsync");
         return -EIO;
@@ -XXX,XX +XXX,XX @@ static coroutine_fn int ssh_co_flush(BlockDriverState *bs)
     return ret;
 }
 
-#else /* !HAS_LIBSSH2_SFTP_FSYNC */
+#else /* !HAVE_LIBSSH_0_8 */
 
 static coroutine_fn int ssh_co_flush(BlockDriverState *bs)
 {
     BDRVSSHState *s = bs->opaque;
 
-    unsafe_flush_warning(s, "libssh2 >= 1.4.4");
+    unsafe_flush_warning(s, "libssh >= 0.8.0");
     return 0;
 }
 
-#endif /* !HAS_LIBSSH2_SFTP_FSYNC */
+#endif /* !HAVE_LIBSSH_0_8 */
 
 static int64_t ssh_getlength(BlockDriverState *bs)
 {
     BDRVSSHState *s = bs->opaque;
     int64_t length;
 
-    /* Note we cannot make a libssh2 call here. */
-    length = (int64_t) s->attrs.filesize;
+    /* Note we cannot make a libssh call here. */
+    length = (int64_t) s->attrs->size;
     trace_ssh_getlength(length);
 
     return length;
@@ -XXX,XX +XXX,XX @@ static int coroutine_fn ssh_co_truncate(BlockDriverState *bs, int64_t offset,
         return -ENOTSUP;
     }
 
-    if (offset < s->attrs.filesize) {
+    if (offset < s->attrs->size) {
         error_setg(errp, "ssh driver does not support shrinking files");
         return -ENOTSUP;
     }
 
-    if (offset == s->attrs.filesize) {
+    if (offset == s->attrs->size) {
         return 0;
     }
 
@@ -XXX,XX +XXX,XX @@ static void bdrv_ssh_init(void)
 {
     int r;
 
-    r = libssh2_init(0);
+    r = ssh_init();
     if (r != 0) {
-        fprintf(stderr, "libssh2 initialization failed, %d\n", r);
+        fprintf(stderr, "libssh initialization failed, %d\n", r);
         exit(EXIT_FAILURE);
     }
 
+#if TRACE_LIBSSH != 0
+    ssh_set_log_level(TRACE_LIBSSH);
+#endif
+
     bdrv_register(&bdrv_ssh);
 }
 
diff --git a/.travis.yml b/.travis.yml
index XXXXXXX..XXXXXXX 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -XXX,XX +XXX,XX @@ addons:
       - libseccomp-dev
       - libspice-protocol-dev
       - libspice-server-dev
-      - libssh2-1-dev
+      - libssh-dev
       - liburcu-dev
       - libusb-1.0-0-dev
       - libvte-2.91-dev
@@ -XXX,XX +XXX,XX @@ matrix:
             - libseccomp-dev
             - libspice-protocol-dev
             - libspice-server-dev
-            - libssh2-1-dev
+            - libssh-dev
             - liburcu-dev
             - libusb-1.0-0-dev
             - libvte-2.91-dev
diff --git a/block/trace-events b/block/trace-events
index XXXXXXX..XXXXXXX 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -XXX,XX +XXX,XX @@ nbd_client_connect_success(const char *export_name) "export '%s'"
 # ssh.c
 ssh_restart_coroutine(void *co) "co=%p"
 ssh_flush(void) "fsync"
-ssh_check_host_key_knownhosts(const char *key) "host key OK: %s"
+ssh_check_host_key_knownhosts(void) "host key OK"
 ssh_connect_to_ssh(char *path, int flags, int mode) "opening file %s flags=0x%x creat_mode=0%o"
 ssh_co_yield(int sock, void *rd_handler, void *wr_handler) "s->sock=%d rd_handler=%p wr_handler=%p"
 ssh_co_yield_back(int sock) "s->sock=%d - back"
 ssh_getlength(int64_t length) "length=%" PRIi64
 ssh_co_create_opts(uint64_t size) "total_size=%" PRIu64
 ssh_read(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
-ssh_read_buf(void *buf, size_t size) "sftp_read buf=%p size=%zu"
-ssh_read_return(ssize_t ret) "sftp_read returned %zd"
+ssh_read_buf(void *buf, size_t size, size_t actual_size) "sftp_read buf=%p size=%zu (actual size=%zu)"
+ssh_read_return(ssize_t ret, int sftp_err) "sftp_read returned %zd (sftp error=%d)"
 ssh_write(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
-ssh_write_buf(void *buf, size_t size) "sftp_write buf=%p size=%zu"
-ssh_write_return(ssize_t ret) "sftp_write returned %zd"
+ssh_write_buf(void *buf, size_t size, size_t actual_size) "sftp_write buf=%p size=%zu (actual size=%zu)"
+ssh_write_return(ssize_t ret, int sftp_err) "sftp_write returned %zd (sftp error=%d)"
 ssh_seek(int64_t offset) "seeking to offset=%" PRIi64
+ssh_auth_methods(int methods) "auth methods=0x%x"
+ssh_server_status(int status) "server status=%d"
 
 # curl.c
 curl_timer_cb(long timeout_ms) "timer callback timeout_ms %ld"
@@ -XXX,XX +XXX,XX @@ sheepdog_snapshot_create(const char *sn_name, const char *id) "%s %s"
 sheepdog_snapshot_create_inode(const char *name, uint32_t snap, uint32_t vdi) "s->inode: name %s snap_id 0x%" PRIx32 " vdi 0x%" PRIx32
 
 # ssh.c
-sftp_error(const char *op, const char *ssh_err, int ssh_err_code, unsigned long sftp_err_code) "%s failed: %s (libssh2 error code: %d, sftp error code: %lu)"
+sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
diff --git a/docs/qemu-block-drivers.texi b/docs/qemu-block-drivers.texi
index XXXXXXX..XXXXXXX 100644
--- a/docs/qemu-block-drivers.texi
+++ b/docs/qemu-block-drivers.texi
@@ -XXX,XX +XXX,XX @@ print a warning when @code{fsync} is not supported:
 
 warning: ssh server @code{ssh.example.com:22} does not support fsync
 
-With sufficiently new versions of libssh2 and OpenSSH, @code{fsync} is
+With sufficiently new versions of libssh and OpenSSH, @code{fsync} is
 supported.
 
 @node disk_images_nvme
diff --git a/tests/docker/dockerfiles/debian-win32-cross.docker b/tests/docker/dockerfiles/debian-win32-cross.docker
index XXXXXXX..XXXXXXX 100644
--- a/tests/docker/dockerfiles/debian-win32-cross.docker
+++ b/tests/docker/dockerfiles/debian-win32-cross.docker
@@ -XXX,XX +XXX,XX @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         mxe-$TARGET-w64-mingw32.shared-curl \
         mxe-$TARGET-w64-mingw32.shared-glib \
         mxe-$TARGET-w64-mingw32.shared-libgcrypt \
-        mxe-$TARGET-w64-mingw32.shared-libssh2 \
         mxe-$TARGET-w64-mingw32.shared-libusb1 \
         mxe-$TARGET-w64-mingw32.shared-lzo \
         mxe-$TARGET-w64-mingw32.shared-nettle \
diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker
index XXXXXXX..XXXXXXX 100644
--- a/tests/docker/dockerfiles/debian-win64-cross.docker
+++ b/tests/docker/dockerfiles/debian-win64-cross.docker
@@ -XXX,XX +XXX,XX @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         mxe-$TARGET-w64-mingw32.shared-curl \
         mxe-$TARGET-w64-mingw32.shared-glib \
         mxe-$TARGET-w64-mingw32.shared-libgcrypt \
-        mxe-$TARGET-w64-mingw32.shared-libssh2 \
         mxe-$TARGET-w64-mingw32.shared-libusb1 \
         mxe-$TARGET-w64-mingw32.shared-lzo \
         mxe-$TARGET-w64-mingw32.shared-nettle \
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index XXXXXXX..XXXXXXX 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
     libpng-devel \
     librbd-devel \
     libseccomp-devel \
-    libssh2-devel \
+    libssh-devel \
     libubsan \
     libusbx-devel \
     libxml2-devel \
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
     mingw32-gtk3 \
     mingw32-libjpeg-turbo \
     mingw32-libpng \
-    mingw32-libssh2 \
     mingw32-libtasn1 \
     mingw32-nettle \
     mingw32-pixman \
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES \
     mingw64-gtk3 \
     mingw64-libjpeg-turbo \
     mingw64-libpng \
-    mingw64-libssh2 \
     mingw64-libtasn1 \
     mingw64-nettle \
     mingw64-pixman \
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index XXXXXXX..XXXXXXX 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES flex bison \
     libsnappy-dev \
     libspice-protocol-dev \
     libspice-server-dev \
-    libssh2-1-dev \
+    libssh-dev \
     libusb-1.0-0-dev \
     libusbredirhost-dev \
     libvdeplug-dev \
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
index XXXXXXX..XXXXXXX 100644
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -XXX,XX +XXX,XX @@ ENV PACKAGES flex bison \
     libsnappy-dev \
     libspice-protocol-dev \
     libspice-server-dev \
-    libssh2-1-dev \
+    libssh-dev \
     libusb-1.0-0-dev \
     libusbredirhost-dev \
     libvdeplug-dev \
diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
index XXXXXXX..XXXXXXX 100755
--- a/tests/qemu-iotests/207
+++ b/tests/qemu-iotests/207
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
 
     iotests.img_info_log(remote_path)
 
-    md5_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
-        'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
-        shell=True).rstrip().decode('ascii')
+    keys = subprocess.check_output(
+        'ssh-keyscan 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'cut -d" " -f3',
+        shell=True).rstrip().decode('ascii').split('\n')
+
+    # Mappings of base64 representations to digests
+    md5_keys = {}
+    sha1_keys = {}
+
+    for key in keys:
+        md5_keys[key] = subprocess.check_output(
+            'echo %s | base64 -d | md5sum -b | cut -d" " -f1' % key,
+            shell=True).rstrip().decode('ascii')
+
+        sha1_keys[key] = subprocess.check_output(
+            'echo %s | base64 -d | sha1sum -b | cut -d" " -f1' % key,
+            shell=True).rstrip().decode('ascii')
 
     vm.launch()
+
+    # Find correct key first
+    matching_key = None
+    for key in keys:
+        result = vm.qmp('blockdev-add',
+                        driver='ssh', node_name='node0', path=disk_path,
+                        server={
+                             'host': '127.0.0.1',
+                             'port': '22',
+                        }, host_key_check={
+                             'mode': 'hash',
+                             'type': 'md5',
+                             'hash': md5_keys[key],
+                        })
+
+        if 'error' not in result:
+            vm.qmp('blockdev-del', node_name='node0')
+            matching_key = key
+            break
+
+    if matching_key is None:
+        vm.shutdown()
+        iotests.notrun('Did not find a key that fits 127.0.0.1')
+
     blockdev_create(vm, { 'driver': 'ssh',
                           'location': {
                               'path': disk_path,
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
                               'host-key-check': {
                                   'mode': 'hash',
                                   'type': 'md5',
-                                  'hash': md5_key,
+                                  'hash': md5_keys[matching_key],
                               }
                           },
                           'size': 8388608 })
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
 
     iotests.img_info_log(remote_path)
 
-    sha1_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
-        'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
-        shell=True).rstrip().decode('ascii')
-
     vm.launch()
     blockdev_create(vm, { 'driver': 'ssh',
                           'location': {
@@ -XXX,XX +XXX,XX @@ with iotests.FilePath('t.img') as disk_path, \
                               'host-key-check': {
                                   'mode': 'hash',
                                   'type': 'sha1',
-                                  'hash': sha1_key,
+                                  'hash': sha1_keys[matching_key],
                               }
                           },
                           'size': 4194304 })
diff --git a/tests/qemu-iotests/207.out b/tests/qemu-iotests/207.out
index XXXXXXX..XXXXXXX 100644
--- a/tests/qemu-iotests/207.out
+++ b/tests/qemu-iotests/207.out
@@ -XXX,XX +XXX,XX @@ virtual size: 4 MiB (4194304 bytes)
 
 {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "ssh", "location": {"host-key-check": {"mode": "none"}, "path": "/this/is/not/an/existing/path", "server": {"host": "127.0.0.1", "port": "22"}}, "size": 4194304}}}
 {"return": {}}
-Job failed: failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31)
+Job failed: failed to open remote file '/this/is/not/an/existing/path': SFTP server: No such file (libssh error code: 1, sftp error code: 2)
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
 
-- 
2.21.0