Series comparison

-[Qemu-devel] [PULL for-v3.0 0/1] Block patches
+[PULL 0/2] Block patches
-The following changes since commit ea6abffa8a08d832feb759d359d5b935e3087cf7:
+The following changes since commit 346ed3151f1c43e72c40cb55b392a1d4cface62c:
-  Update version for v3.0.0-rc1 release (2018-07-17 18:15:19 +0100)
+  Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20200206.0' into staging (2020-02-07 11:52:15 +0000)
 are available in the Git repository at:
-  git://github.com/stefanha/qemu.git tags/block-pull-request
+  https://github.com/stefanha/qemu.git tags/block-pull-request
-for you to fetch changes up to 6fccbb475bc6effc313ee9481726a1748b6dae57:
+for you to fetch changes up to 11a18c84db4a71497d3d40769688a01b6f64b2ad:
-  throttle-groups: fix hang when group member leaves (2018-07-19 13:08:26 +0100)
+  hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host (2020-02-07 16:49:39 +0000)
 ----------------------------------------------------------------
 Pull request
-This fix prevents hangs when a drive leaves a throttling group.
 ----------------------------------------------------------------
-Stefan Hajnoczi (1):
+Philippe Mathieu-Daudé (1):
-  throttle-groups: fix hang when group member leaves
+  hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host
- block/throttle-groups.c | 4 ++++
+Vladimir Sementsov-Ogievskiy (1):
-file changed, 4 insertions(+)
+  block: fix crash on zero-length unaligned write and read
  block/io.c        | 28 +++++++++++++++++++++++++++-
  hw/core/machine.c |  3 ++-
 files changed, 29 insertions(+), 2 deletions(-)
 --
-.17.1
+.24.1

-New patch
+[PULL 1/2] block: fix crash on zero-length unaligned write and read
+From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Commit 7a3f542fbd "block/io: refactor padding" occasionally dropped
+aligning for zero-length request: bdrv_init_padding() blindly return
+false if bytes == 0, like there is nothing to align.
+This leads the following command to crash:
+./qemu-io --image-opts -c 'write 1 0' \
+  driver=blkdebug,align=512,image.driver=null-co,image.size=512
+>> qemu-io: block/io.c:1955: bdrv_aligned_pwritev: Assertion
+    `(offset & (align - 1)) == 0' failed.
+>> Aborted (core dumped)
+Prior to 7a3f542fbd we does aligning of such zero requests. Instead of
+recovering this behavior let's just do nothing on such requests as it
+is useless.
+Note that driver may have special meaning of zero-length reqeusts, like
+qcow2_co_pwritev_compressed_part, so we can't skip any zero-length
+operation. But for unaligned ones, we can't pass it to driver anyway.
+This commit also fixes crash in iotest 80 running with -nocache:
+./check -nocache -qcow2 80
+which crashes on same assertion due to trying to read empty extra data
+in qcow2_do_read_snapshots().
+Cc: qemu-stable@nongnu.org # v4.2
+Fixes: 7a3f542fbd
+Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
+Reviewed-by: Max Reitz <mreitz@redhat.com>
+Message-id: 20200206164245.17781-1-vsementsov@virtuozzo.com
+Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+---
+ block/io.c | 28 +++++++++++++++++++++++++++-
+file changed, 27 insertions(+), 1 deletion(-)
+diff --git a/block/io.c b/block/io.c
+index XXXXXXX..XXXXXXX 100644
+--- a/block/io.c
++++ b/block/io.c
+@@ -XXX,XX +XXX,XX @@ static bool bdrv_init_padding(BlockDriverState *bs,
+         pad->tail = align - pad->tail;
+     }
+-    if ((!pad->head && !pad->tail) || !bytes) {
++    if (!pad->head && !pad->tail) {
+         return false;
+     }
++    assert(bytes); /* Nothing good in aligning zero-length requests */
++
+     sum = pad->head + bytes + pad->tail;
+     pad->buf_len = (sum > align && pad->head && pad->tail) ? 2 * align : align;
+     pad->buf = qemu_blockalign(bs, pad->buf_len);
+@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_preadv_part(BdrvChild *child,
+         return ret;
+     }
++    if (bytes == 0 && !QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)) {
++        /*
++         * Aligning zero request is nonsense. Even if driver has special meaning
++         * of zero-length (like qcow2_co_pwritev_compressed_part), we can't pass
++         * it to driver due to request_alignment.
++         *
++         * Still, no reason to return an error if someone do unaligned
++         * zero-length read occasionally.
++         */
++        return 0;
++    }
++
+     bdrv_inc_in_flight(bs);
+     /* Don't do copy-on-read if we read data before write operation */
+@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child,
+         return -ENOTSUP;
+     }
++    if (bytes == 0 && !QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)) {
++        /*
++         * Aligning zero request is nonsense. Even if driver has special meaning
++         * of zero-length (like qcow2_co_pwritev_compressed_part), we can't pass
++         * it to driver due to request_alignment.
++         *
++         * Still, no reason to return an error if someone do unaligned
++         * zero-length write occasionally.
++         */
++        return 0;
++    }
++
+     bdrv_inc_in_flight(bs);
+     /*
+      * Align write if necessary by performing a read-modify-write cycle.
+--
+.24.1

-[Qemu-devel] [PULL for-v3.0 1/1] throttle-groups: fix hang when group member leaves
+[PULL 2/2] hw/core: Allow setting 'virtio-blk-device.scsi' property on OSX host
-Throttle groups consist of members sharing one throttling state
+From: Philippe Mathieu-Daudé <philmd@redhat.com>
 (including bps/iops limits).  Round-robin scheduling is used to ensure
 fairness.  If a group member already has a timer pending then other
 groups members do not schedule their own timers.  The next group member
 will have its turn when the existing timer expires.
-A hang may occur when a group member leaves while it had a timer
+Commit ed65fd1a2750 ("virtio-blk: switch off scsi-passthrough by
-scheduled.  Although the code carefully removes the group member from
+default") changed the default value of the 'scsi' property of
-the round-robin list, it does not schedule the next member.  Therefore
+virtio-blk, which is only available on Linux hosts. It also added
-remaining members continue to wait for the removed member's timer to
+an unconditional compat entry for 2.4 or earlier machines.
 expire.
-This patch schedules the next request if a timer is pending.
+Trying to set this property on a pre-2.5 machine on OSX, we get:
 Unfortunately the actual bug is a race condition that I've been unable
 to capture in a test case.
-Sometimes drive2 hangs when drive1 is removed from the throttling group:
+   Unexpected error in object_property_find() at qom/object.c:1201:
    qemu-system-x86_64: -device virtio-blk-pci,id=scsi0,drive=drive0: can't apply global virtio-blk-device.scsi=true: Property '.scsi' not found
-  $ qemu ... -drive if=none,id=drive1,cache=none,format=qcow2,file=data1.qcow2,iops=100,group=foo \
+Fix this error by marking the property optional.
              -device virtio-blk-pci,id=virtio-blk-pci0,drive=drive1 \
              -drive if=none,id=drive2,cache=none,format=qcow2,file=data2.qcow2,iops=10,group=foo \
              -device virtio-blk-pci,id=virtio-blk-pci1,drive=drive2
   (guest-console1)# fio -filename /dev/vda 4k-seq-read.job
   (guest-console2)# fio -filename /dev/vdb 4k-seq-read.job
   (qmp) {"execute": "block_set_io_throttle", "arguments": {"device": "drive1","bps": 0,"bps_rd": 0,"bps_wr": 0,"iops": 0,"iops_rd": 0,"iops_wr": 0}}
-Reported-by: Nini Gu <ngu@redhat.com>
+Fixes: ed65fd1a27 ("virtio-blk: switch off scsi-passthrough by default")
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Suggested-by: Cornelia Huck <cohuck@redhat.com>
-Message-id: 20180704145410.794-1-stefanha@redhat.com
+Reviewed-by: Cornelia Huck <cohuck@redhat.com>
-RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1535914
+Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
-Cc: Alberto Garcia <berto@igalia.com>
+Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
 Message-id: 20200207001404.1739-1-philmd@redhat.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- block/throttle-groups.c | 4 ++++
+ hw/core/machine.c | 3 ++-
-file changed, 4 insertions(+)
+file changed, 2 insertions(+), 1 deletion(-)
-diff --git a/block/throttle-groups.c b/block/throttle-groups.c
+diff --git a/hw/core/machine.c b/hw/core/machine.c
 index XXXXXXX..XXXXXXX 100644
---- a/block/throttle-groups.c
+--- a/hw/core/machine.c
-+++ b/block/throttle-groups.c
++++ b/hw/core/machine.c
-@@ -XXX,XX +XXX,XX @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm)
+@@ -XXX,XX +XXX,XX @@ GlobalProperty hw_compat_2_5[] = {
+ const size_t hw_compat_2_5_len = G_N_ELEMENTS(hw_compat_2_5);
-     qemu_mutex_lock(&tg->lock);
-     for (i = 0; i < 2; i++) {
+ GlobalProperty hw_compat_2_4[] = {
-+        if (timer_pending(tgm->throttle_timers.timers[i])) {
+-    { "virtio-blk-device", "scsi", "true" },
-+            tg->any_timer_armed[i] = false;
++    /* Optional because the 'scsi' property is Linux-only */
-+            schedule_next_request(tgm, i);
++    { "virtio-blk-device", "scsi", "true", .optional = true },
-+        }
+     { "e1000", "extra_mac_registers", "off" },
-         if (tg->tokens[i] == tgm) {
+     { "virtio-pci", "x-disable-pcie", "on" },
-             token = throttle_group_next_tgm(tgm);
+     { "virtio-pci", "migrate-extra", "off" },
              /* Take care of the case where this is the last tgm in the group */
 --
-.17.1
+.24.1

Throttle groups consist of members sharing one throttling state
(including bps/iops limits).  Round-robin scheduling is used to ensure
fairness.  If a group member already has a timer pending then other
groups members do not schedule their own timers.  The next group member
will have its turn when the existing timer expires.

A hang may occur when a group member leaves while it had a timer
scheduled.  Although the code carefully removes the group member from
the round-robin list, it does not schedule the next member.  Therefore
remaining members continue to wait for the removed member's timer to
expire.

This patch schedules the next request if a timer is pending.
Unfortunately the actual bug is a race condition that I've been unable
to capture in a test case.

Sometimes drive2 hangs when drive1 is removed from the throttling group:

$ qemu ... -drive if=none,id=drive1,cache=none,format=qcow2,file=data1.qcow2,iops=100,group=foo \
             -device virtio-blk-pci,id=virtio-blk-pci0,drive=drive1 \
             -drive if=none,id=drive2,cache=none,format=qcow2,file=data2.qcow2,iops=10,group=foo \
             -device virtio-blk-pci,id=virtio-blk-pci1,drive=drive2
  (guest-console1)# fio -filename /dev/vda 4k-seq-read.job
  (guest-console2)# fio -filename /dev/vdb 4k-seq-read.job
  (qmp) {"execute": "block_set_io_throttle", "arguments": {"device": "drive1","bps": 0,"bps_rd": 0,"bps_wr": 0,"iops": 0,"iops_rd": 0,"iops_wr": 0}}

Reported-by: Nini Gu <ngu@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20180704145410.794-1-stefanha@redhat.com
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1535914
Cc: Alberto Garcia <berto@igalia.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/throttle-groups.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/block/throttle-groups.c b/block/throttle-groups.c
index XXXXXXX..XXXXXXX 100644
--- a/block/throttle-groups.c
+++ b/block/throttle-groups.c
@@ -XXX,XX +XXX,XX @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm)
 
     qemu_mutex_lock(&tg->lock);
     for (i = 0; i < 2; i++) {
+        if (timer_pending(tgm->throttle_timers.timers[i])) {
+            tg->any_timer_armed[i] = false;
+            schedule_next_request(tgm, i);
+        }
         if (tg->tokens[i] == tgm) {
             token = throttle_group_next_tgm(tgm);
             /* Take care of the case where this is the last tgm in the group */
-- 
2.17.1

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

Commit 7a3f542fbd "block/io: refactor padding" occasionally dropped
aligning for zero-length request: bdrv_init_padding() blindly return
false if bytes == 0, like there is nothing to align.

This leads the following command to crash:

./qemu-io --image-opts -c 'write 1 0' \
  driver=blkdebug,align=512,image.driver=null-co,image.size=512

>> qemu-io: block/io.c:1955: bdrv_aligned_pwritev: Assertion
    `(offset & (align - 1)) == 0' failed.
>> Aborted (core dumped)

Prior to 7a3f542fbd we does aligning of such zero requests. Instead of
recovering this behavior let's just do nothing on such requests as it
is useless.

Note that driver may have special meaning of zero-length reqeusts, like
qcow2_co_pwritev_compressed_part, so we can't skip any zero-length
operation. But for unaligned ones, we can't pass it to driver anyway.

This commit also fixes crash in iotest 80 running with -nocache:

./check -nocache -qcow2 80

which crashes on same assertion due to trying to read empty extra data
in qcow2_do_read_snapshots().

Cc: qemu-stable@nongnu.org # v4.2
Fixes: 7a3f542fbd
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20200206164245.17781-1-vsementsov@virtuozzo.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/io.c | 28 +++++++++++++++++++++++++++-
 1 file changed, 27 insertions(+), 1 deletion(-)

diff --git a/block/io.c b/block/io.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io.c
+++ b/block/io.c
@@ -XXX,XX +XXX,XX @@ static bool bdrv_init_padding(BlockDriverState *bs,
         pad->tail = align - pad->tail;
     }
 
-    if ((!pad->head && !pad->tail) || !bytes) {
+    if (!pad->head && !pad->tail) {
         return false;
     }
 
+    assert(bytes); /* Nothing good in aligning zero-length requests */
+
     sum = pad->head + bytes + pad->tail;
     pad->buf_len = (sum > align && pad->head && pad->tail) ? 2 * align : align;
     pad->buf = qemu_blockalign(bs, pad->buf_len);
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_preadv_part(BdrvChild *child,
         return ret;
     }
 
+    if (bytes == 0 && !QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)) {
+        /*
+         * Aligning zero request is nonsense. Even if driver has special meaning
+         * of zero-length (like qcow2_co_pwritev_compressed_part), we can't pass
+         * it to driver due to request_alignment.
+         *
+         * Still, no reason to return an error if someone do unaligned
+         * zero-length read occasionally.
+         */
+        return 0;
+    }
+
     bdrv_inc_in_flight(bs);
 
     /* Don't do copy-on-read if we read data before write operation */
@@ -XXX,XX +XXX,XX @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child,
         return -ENOTSUP;
     }
 
+    if (bytes == 0 && !QEMU_IS_ALIGNED(offset, bs->bl.request_alignment)) {
+        /*
+         * Aligning zero request is nonsense. Even if driver has special meaning
+         * of zero-length (like qcow2_co_pwritev_compressed_part), we can't pass
+         * it to driver due to request_alignment.
+         *
+         * Still, no reason to return an error if someone do unaligned
+         * zero-length write occasionally.
+         */
+        return 0;
+    }
+
     bdrv_inc_in_flight(bs);
     /*
      * Align write if necessary by performing a read-modify-write cycle.
-- 
2.24.1

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Commit ed65fd1a2750 ("virtio-blk: switch off scsi-passthrough by
default") changed the default value of the 'scsi' property of
virtio-blk, which is only available on Linux hosts. It also added
an unconditional compat entry for 2.4 or earlier machines.

Trying to set this property on a pre-2.5 machine on OSX, we get:

Unexpected error in object_property_find() at qom/object.c:1201:
   qemu-system-x86_64: -device virtio-blk-pci,id=scsi0,drive=drive0: can't apply global virtio-blk-device.scsi=true: Property '.scsi' not found

Fix this error by marking the property optional.

Fixes: ed65fd1a27 ("virtio-blk: switch off scsi-passthrough by default")
Suggested-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Message-id: 20200207001404.1739-1-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/core/machine.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -XXX,XX +XXX,XX @@ GlobalProperty hw_compat_2_5[] = {
 const size_t hw_compat_2_5_len = G_N_ELEMENTS(hw_compat_2_5);
 
 GlobalProperty hw_compat_2_4[] = {
-    { "virtio-blk-device", "scsi", "true" },
+    /* Optional because the 'scsi' property is Linux-only */
+    { "virtio-blk-device", "scsi", "true", .optional = true },
     { "e1000", "extra_mac_registers", "off" },
     { "virtio-pci", "x-disable-pcie", "on" },
     { "virtio-pci", "migrate-extra", "off" },
-- 
2.24.1