Series comparison

-[Qemu-devel] [PULL 0/7] Block patches
+[PULL for-6.1 0/3] Block patches
-The following changes since commit 0ab4537f08e09b13788db67efd760592fb7db769:
+The following changes since commit 3521ade3510eb5cefb2e27a101667f25dad89935:
-  Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2018-03-07-1' into staging (2018-03-08 12:56:39 +0000)
+  Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-07-29' into staging (2021-07-29 13:17:20 +0100)
 are available in the Git repository at:
-  git://github.com/stefanha/qemu.git tags/block-pull-request
+  https://gitlab.com/stefanha/qemu.git tags/block-pull-request
-for you to fetch changes up to 4486e89c219c0d1b9bd8dfa0b1dd5b0d51ff2268:
+for you to fetch changes up to cc8eecd7f105a1dff5876adeb238a14696061a4a:
-  vl: introduce vm_shutdown() (2018-03-08 17:38:51 +0000)
+  MAINTAINERS: Added myself as a reviewer for the NVMe Block Driver (2021-07-29 17:17:34 +0100)
 ----------------------------------------------------------------
 Pull request
 The main fix here is for io_uring. Spurious -EAGAIN errors can happen and the
 request needs to be resubmitted.
 The MAINTAINERS changes carry no risk and we might as well include them in QEMU
 .1.
 ----------------------------------------------------------------
-----------------------------------------------------------------
+Fabian Ebner (1):
   block/io_uring: resubmit when result is -EAGAIN
-Deepa Srinivasan (1):
+Philippe Mathieu-Daudé (1):
-  block: Fix qemu crash when using scsi-block
+  MAINTAINERS: Added myself as a reviewer for the NVMe Block Driver
-Fam Zheng (1):
+Stefano Garzarella (1):
-  README: Fix typo 'git-publish'
+  MAINTAINERS: add Stefano Garzarella as io_uring reviewer
-Sergio Lopez (1):
+ MAINTAINERS      |  2 ++
-  virtio-blk: dataplane: Don't batch notifications if EVENT_IDX is
+ block/io_uring.c | 16 +++++++++++++++-
-    present
+files changed, 17 insertions(+), 1 deletion(-)
 Stefan Hajnoczi (4):
   block: add aio_wait_bh_oneshot()
   virtio-blk: fix race between .ioeventfd_stop() and vq handler
   virtio-scsi: fix race between .ioeventfd_stop() and vq handler
   vl: introduce vm_shutdown()
  include/block/aio-wait.h        | 13 +++++++++++
  include/sysemu/iothread.h       |  1 -
  include/sysemu/sysemu.h         |  1 +
  block/block-backend.c           | 51 ++++++++++++++++++++---------------------
  cpus.c                          | 16 ++++++++++---
  hw/block/dataplane/virtio-blk.c | 39 +++++++++++++++++++++++--------
  hw/scsi/virtio-scsi-dataplane.c |  9 ++++----
  iothread.c                      | 31 -------------------------
  util/aio-wait.c                 | 31 +++++++++++++++++++++++++
  vl.c                            | 13 +++--------
  README                          |  2 +-
 files changed, 122 insertions(+), 85 deletions(-)
 --
-.14.3
+.31.1

-[Qemu-devel] [PULL 1/7] block: Fix qemu crash when using scsi-block
+Deleted patch
-From: Deepa Srinivasan <deepa.srinivasan@oracle.com>
-Starting qemu with the following arguments causes qemu to segfault:
-... -device lsi,id=lsi0 -drive file=iscsi:<...>,format=raw,if=none,node-name=
-iscsi1 -device scsi-block,bus=lsi0.0,id=<...>,drive=iscsi1
-This patch fixes blk_aio_ioctl() so it does not pass stack addresses to
-blk_aio_ioctl_entry() which may be invoked after blk_aio_ioctl() returns. More
-details about the bug follow.
-blk_aio_ioctl() invokes blk_aio_prwv() with blk_aio_ioctl_entry as the
-coroutine parameter. blk_aio_prwv() ultimately calls aio_co_enter().
-When blk_aio_ioctl() is executed from within a coroutine context (e.g.
-iscsi_bh_cb()), aio_co_enter() adds the coroutine (blk_aio_ioctl_entry) to
-the current coroutine's wakeup queue. blk_aio_ioctl() then returns.
-When blk_aio_ioctl_entry() executes later, it accesses an invalid pointer:
-....
-    BlkRwCo *rwco = &acb->rwco;
-    rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset,
-                             rwco->qiov->iov[0].iov_base);  <--- qiov is
-                                                                 invalid here
-...
-In the case when blk_aio_ioctl() is called from a non-coroutine context,
-blk_aio_ioctl_entry() executes immediately. But if bdrv_co_ioctl() calls
-qemu_coroutine_yield(), blk_aio_ioctl() will return. When the coroutine
-execution is complete, control returns to blk_aio_ioctl_entry() after the call
-to blk_co_ioctl(). There is no invalid reference after this point, but the
-function is still holding on to invalid pointers.
-The fix is to change blk_aio_prwv() to accept a void pointer for the IO buffer
-rather than a QEMUIOVector. blk_aio_prwv() passes this through in BlkRwCo and the
-coroutine function casts it to QEMUIOVector or uses the void pointer directly.
-Signed-off-by: Deepa Srinivasan <deepa.srinivasan@oracle.com>
-Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
-Reviewed-by: Mark Kanda <mark.kanda@oracle.com>
-Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
----
- block/block-backend.c | 51 +++++++++++++++++++++++++--------------------------
-file changed, 25 insertions(+), 26 deletions(-)
-diff --git a/block/block-backend.c b/block/block-backend.c
-index XXXXXXX..XXXXXXX 100644
---- a/block/block-backend.c
-+++ b/block/block-backend.c
-@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_pwritev(BlockBackend *blk, int64_t offset,
- typedef struct BlkRwCo {
-     BlockBackend *blk;
-     int64_t offset;
--    QEMUIOVector *qiov;
-+    void *iobuf;
-     int ret;
-     BdrvRequestFlags flags;
- } BlkRwCo;
-@@ -XXX,XX +XXX,XX @@ typedef struct BlkRwCo {
- static void blk_read_entry(void *opaque)
- {
-     BlkRwCo *rwco = opaque;
-+    QEMUIOVector *qiov = rwco->iobuf;
--    rwco->ret = blk_co_preadv(rwco->blk, rwco->offset, rwco->qiov->size,
--                              rwco->qiov, rwco->flags);
-+    rwco->ret = blk_co_preadv(rwco->blk, rwco->offset, qiov->size,
-+                              qiov, rwco->flags);
- }
- static void blk_write_entry(void *opaque)
- {
-     BlkRwCo *rwco = opaque;
-+    QEMUIOVector *qiov = rwco->iobuf;
--    rwco->ret = blk_co_pwritev(rwco->blk, rwco->offset, rwco->qiov->size,
--                               rwco->qiov, rwco->flags);
-+    rwco->ret = blk_co_pwritev(rwco->blk, rwco->offset, qiov->size,
-+                               qiov, rwco->flags);
- }
- static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf,
-@@ -XXX,XX +XXX,XX @@ static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf,
-     rwco = (BlkRwCo) {
-         .blk    = blk,
-         .offset = offset,
--        .qiov   = &qiov,
-+        .iobuf  = &qiov,
-         .flags  = flags,
-         .ret    = NOT_DONE,
-     };
-@@ -XXX,XX +XXX,XX @@ static void blk_aio_complete_bh(void *opaque)
- }
- static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, int64_t offset, int bytes,
--                                QEMUIOVector *qiov, CoroutineEntry co_entry,
-+                                void *iobuf, CoroutineEntry co_entry,
-                                 BdrvRequestFlags flags,
-                                 BlockCompletionFunc *cb, void *opaque)
- {
-@@ -XXX,XX +XXX,XX @@ static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, int64_t offset, int bytes,
-     acb->rwco = (BlkRwCo) {
-         .blk    = blk,
-         .offset = offset,
--        .qiov   = qiov,
-+        .iobuf  = iobuf,
-         .flags  = flags,
-         .ret    = NOT_DONE,
-     };
-@@ -XXX,XX +XXX,XX @@ static void blk_aio_read_entry(void *opaque)
- {
-     BlkAioEmAIOCB *acb = opaque;
-     BlkRwCo *rwco = &acb->rwco;
-+    QEMUIOVector *qiov = rwco->iobuf;
--    assert(rwco->qiov->size == acb->bytes);
-+    assert(qiov->size == acb->bytes);
-     rwco->ret = blk_co_preadv(rwco->blk, rwco->offset, acb->bytes,
--                              rwco->qiov, rwco->flags);
-+                              qiov, rwco->flags);
-     blk_aio_complete(acb);
- }
-@@ -XXX,XX +XXX,XX @@ static void blk_aio_write_entry(void *opaque)
- {
-     BlkAioEmAIOCB *acb = opaque;
-     BlkRwCo *rwco = &acb->rwco;
-+    QEMUIOVector *qiov = rwco->iobuf;
--    assert(!rwco->qiov || rwco->qiov->size == acb->bytes);
-+    assert(!qiov || qiov->size == acb->bytes);
-     rwco->ret = blk_co_pwritev(rwco->blk, rwco->offset, acb->bytes,
--                               rwco->qiov, rwco->flags);
-+                               qiov, rwco->flags);
-     blk_aio_complete(acb);
- }
-@@ -XXX,XX +XXX,XX @@ int blk_co_ioctl(BlockBackend *blk, unsigned long int req, void *buf)
- static void blk_ioctl_entry(void *opaque)
- {
-     BlkRwCo *rwco = opaque;
-+    QEMUIOVector *qiov = rwco->iobuf;
-+
-     rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset,
--                             rwco->qiov->iov[0].iov_base);
-+                             qiov->iov[0].iov_base);
- }
- int blk_ioctl(BlockBackend *blk, unsigned long int req, void *buf)
-@@ -XXX,XX +XXX,XX @@ static void blk_aio_ioctl_entry(void *opaque)
-     BlkAioEmAIOCB *acb = opaque;
-     BlkRwCo *rwco = &acb->rwco;
--    rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset,
--                             rwco->qiov->iov[0].iov_base);
-+    rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset, rwco->iobuf);
-+
-     blk_aio_complete(acb);
- }
- BlockAIOCB *blk_aio_ioctl(BlockBackend *blk, unsigned long int req, void *buf,
-                           BlockCompletionFunc *cb, void *opaque)
- {
--    QEMUIOVector qiov;
--    struct iovec iov;
--
--    iov = (struct iovec) {
--        .iov_base = buf,
--        .iov_len = 0,
--    };
--    qemu_iovec_init_external(&qiov, &iov, 1);
--
--    return blk_aio_prwv(blk, req, 0, &qiov, blk_aio_ioctl_entry, 0, cb, opaque);
-+    return blk_aio_prwv(blk, req, 0, buf, blk_aio_ioctl_entry, 0, cb, opaque);
- }
- int blk_co_pdiscard(BlockBackend *blk, int64_t offset, int bytes)
-@@ -XXX,XX +XXX,XX @@ int blk_truncate(BlockBackend *blk, int64_t offset, PreallocMode prealloc,
- static void blk_pdiscard_entry(void *opaque)
- {
-     BlkRwCo *rwco = opaque;
--    rwco->ret = blk_co_pdiscard(rwco->blk, rwco->offset, rwco->qiov->size);
-+    QEMUIOVector *qiov = rwco->iobuf;
-+
-+    rwco->ret = blk_co_pdiscard(rwco->blk, rwco->offset, qiov->size);
- }
- int blk_pdiscard(BlockBackend *blk, int64_t offset, int bytes)
---
-.14.3

-[Qemu-devel] [PULL 2/7] README: Fix typo 'git-publish'
+Deleted patch
-From: Fam Zheng <famz@redhat.com>
-Reported-by: Alberto Garcia <berto@igalia.com>
-Signed-off-by: Fam Zheng <famz@redhat.com>
-Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
-Message-id: 20180306024328.19195-1-famz@redhat.com
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
----
- README | 2 +-
-file changed, 1 insertion(+), 1 deletion(-)
-diff --git a/README b/README
-index XXXXXXX..XXXXXXX 100644
---- a/README
-+++ b/README
-@@ -XXX,XX +XXX,XX @@ The QEMU website is also maintained under source control.
-   git clone git://git.qemu.org/qemu-web.git
-   https://www.qemu.org/2017/02/04/the-new-qemu-website-is-up/
--A 'git-profile' utility was created to make above process less
-+A 'git-publish' utility was created to make above process less
- cumbersome, and is highly recommended for making regular contributions,
- or even just for sending consecutive patch series revisions. It also
- requires a working 'git send-email' setup, and by default doesn't
---
-.14.3

-[Qemu-devel] [PULL 3/7] virtio-blk: dataplane: Don't batch notifications if EVENT_IDX is present
+Deleted patch
-From: Sergio Lopez <slp@redhat.com>
-Commit 5b2ffbe4d99843fd8305c573a100047a8c962327 ("virtio-blk: dataplane:
-notify guest as a batch") deferred guest notification to a BH in order
-batch notifications, with purpose of avoiding flooding the guest with
-interruptions.
-This optimization came with a cost. The average latency perceived in the
-guest is increased by a few microseconds, but also when multiple IO
-operations finish at the same time, the guest won't be notified until
-all completions from each operation has been run. On the contrary,
-virtio-scsi issues the notification at the end of each completion.
-On the other hand, nowadays we have the EVENT_IDX feature that allows a
-better coordination between QEMU and the Guest OS to avoid sending
-unnecessary interruptions.
-With this change, virtio-blk/dataplane only batches notifications if the
-EVENT_IDX feature is not present.
-Some numbers obtained with fio (ioengine=sync, iodepth=1, direct=1):
- - Test specs:
-   * fio-3.4 (ioengine=sync, iodepth=1, direct=1)
-   * qemu master
-   * virtio-blk with a dedicated iothread (default poll-max-ns)
-   * backend: null_blk nr_devices=1 irqmode=2 completion_nsec=280000
-   * 8 vCPUs pinned to isolated physical cores
-   * Emulator and iothread also pinned to separate isolated cores
-   * variance between runs < 1%
- - Not patched
-   * numjobs=1:  lat_avg=327.32  irqs=29998
-   * numjobs=4:  lat_avg=337.89  irqs=29073
-   * numjobs=8:  lat_avg=342.98  irqs=28643
- - Patched:
-   * numjobs=1:  lat_avg=323.92  irqs=30262
-   * numjobs=4:  lat_avg=332.65  irqs=29520
-   * numjobs=8:  lat_avg=335.54  irqs=29323
-Signed-off-by: Sergio Lopez <slp@redhat.com>
-Message-id: 20180307114459.26636-1-slp@redhat.com
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
----
- hw/block/dataplane/virtio-blk.c | 15 +++++++++++++--
-file changed, 13 insertions(+), 2 deletions(-)
-diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
-index XXXXXXX..XXXXXXX 100644
---- a/hw/block/dataplane/virtio-blk.c
-+++ b/hw/block/dataplane/virtio-blk.c
-@@ -XXX,XX +XXX,XX @@ struct VirtIOBlockDataPlane {
-     VirtIODevice *vdev;
-     QEMUBH *bh;                     /* bh for guest notification */
-     unsigned long *batch_notify_vqs;
-+    bool batch_notifications;
-     /* Note that these EventNotifiers are assigned by value.  This is
-      * fine as long as you do not call event_notifier_cleanup on them
-@@ -XXX,XX +XXX,XX @@ struct VirtIOBlockDataPlane {
- /* Raise an interrupt to signal guest, if necessary */
- void virtio_blk_data_plane_notify(VirtIOBlockDataPlane *s, VirtQueue *vq)
- {
--    set_bit(virtio_get_queue_index(vq), s->batch_notify_vqs);
--    qemu_bh_schedule(s->bh);
-+    if (s->batch_notifications) {
-+        set_bit(virtio_get_queue_index(vq), s->batch_notify_vqs);
-+        qemu_bh_schedule(s->bh);
-+    } else {
-+        virtio_notify_irqfd(s->vdev, vq);
-+    }
- }
- static void notify_guest_bh(void *opaque)
-@@ -XXX,XX +XXX,XX @@ int virtio_blk_data_plane_start(VirtIODevice *vdev)
-     s->starting = true;
-+    if (!virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) {
-+        s->batch_notifications = true;
-+    } else {
-+        s->batch_notifications = false;
-+    }
-+
-     /* Set up guest notifier (irq) */
-     r = k->set_guest_notifiers(qbus->parent, nvqs, true);
-     if (r != 0) {
---
-.14.3

-[Qemu-devel] [PULL 4/7] block: add aio_wait_bh_oneshot()
+Deleted patch
-Sometimes it's necessary for the main loop thread to run a BH in an
-IOThread and wait for its completion.  This primitive is useful during
-startup/shutdown to synchronize and avoid race conditions.
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
-Reviewed-by: Fam Zheng <famz@redhat.com>
-Acked-by: Paolo Bonzini <pbonzini@redhat.com>
-Message-id: 20180307144205.20619-2-stefanha@redhat.com
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
----
- include/block/aio-wait.h | 13 +++++++++++++
- util/aio-wait.c          | 31 +++++++++++++++++++++++++++++++
-files changed, 44 insertions(+)
-diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h
-index XXXXXXX..XXXXXXX 100644
---- a/include/block/aio-wait.h
-+++ b/include/block/aio-wait.h
-@@ -XXX,XX +XXX,XX @@ typedef struct {
-  */
- void aio_wait_kick(AioWait *wait);
-+/**
-+ * aio_wait_bh_oneshot:
-+ * @ctx: the aio context
-+ * @cb: the BH callback function
-+ * @opaque: user data for the BH callback function
-+ *
-+ * Run a BH in @ctx and wait for it to complete.
-+ *
-+ * Must be called from the main loop thread with @ctx acquired exactly once.
-+ * Note that main loop event processing may occur.
-+ */
-+void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque);
-+
- #endif /* QEMU_AIO_WAIT */
-diff --git a/util/aio-wait.c b/util/aio-wait.c
-index XXXXXXX..XXXXXXX 100644
---- a/util/aio-wait.c
-+++ b/util/aio-wait.c
-@@ -XXX,XX +XXX,XX @@ void aio_wait_kick(AioWait *wait)
-         aio_bh_schedule_oneshot(qemu_get_aio_context(), dummy_bh_cb, NULL);
-     }
- }
-+
-+typedef struct {
-+    AioWait wait;
-+    bool done;
-+    QEMUBHFunc *cb;
-+    void *opaque;
-+} AioWaitBHData;
-+
-+/* Context: BH in IOThread */
-+static void aio_wait_bh(void *opaque)
-+{
-+    AioWaitBHData *data = opaque;
-+
-+    data->cb(data->opaque);
-+
-+    data->done = true;
-+    aio_wait_kick(&data->wait);
-+}
-+
-+void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
-+{
-+    AioWaitBHData data = {
-+        .cb = cb,
-+        .opaque = opaque,
-+    };
-+
-+    assert(qemu_get_current_aio_context() == qemu_get_aio_context());
-+
-+    aio_bh_schedule_oneshot(ctx, aio_wait_bh, &data);
-+    AIO_WAIT_WHILE(&data.wait, ctx, !data.done);
-+}
---
-.14.3

-[Qemu-devel] [PULL 7/7] vl: introduce vm_shutdown()
+[PULL for-6.1 1/3] MAINTAINERS: add Stefano Garzarella as io_uring reviewer
-Commit 00d09fdbbae5f7864ce754913efc84c12fdf9f1a ("vl: pause vcpus before
+From: Stefano Garzarella <sgarzare@redhat.com>
 stopping iothreads") and commit dce8921b2baaf95974af8176406881872067adfa
 ("iothread: Stop threads before main() quits") tried to work around the
 fact that emulation was still active during termination by stopping
 iothreads.  They suffer from race conditions:
 . virtio_scsi_handle_cmd_vq() racing with iothread_stop_all() hits the
    virtio_scsi_ctx_check() assertion failure because the BDS AioContext
    has been modified by iothread_stop_all().
 . Guest vq kick racing with main loop termination leaves a readable
    ioeventfd that is handled by the next aio_poll() when external
    clients are enabled again, resulting in unwanted emulation activity.
-This patch obsoletes those commits by fully disabling emulation activity
+I've been working with io_uring for a while so I'd like to help
-when vcpus are stopped.
+with reviews.
-Use the new vm_shutdown() function instead of pause_all_vcpus() so that
+Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
-vm change state handlers are invoked too.  Virtio devices will now stop
+Message-Id: <20210728131515.131045-1-sgarzare@redhat.com>
 their ioeventfds, preventing further emulation activity after vm_stop().
 Note that vm_stop(RUN_STATE_SHUTDOWN) cannot be used because it emits a
 QMP STOP event that may affect existing clients.
 It is no longer necessary to call replay_disable_events() directly since
 vm_shutdown() does so already.
 Drop iothread_stop_all() since it is no longer used.
 Cc: Fam Zheng <famz@redhat.com>
 Cc: Kevin Wolf <kwolf@redhat.com>
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 Reviewed-by: Fam Zheng <famz@redhat.com>
 Acked-by: Paolo Bonzini <pbonzini@redhat.com>
 Message-id: 20180307144205.20619-5-stefanha@redhat.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- include/sysemu/iothread.h |  1 -
+ MAINTAINERS | 1 +
- include/sysemu/sysemu.h   |  1 +
+file changed, 1 insertion(+)
  cpus.c                    | 16 +++++++++++++---
  iothread.c                | 31 -------------------------------
  vl.c                      | 13 +++----------
 files changed, 17 insertions(+), 45 deletions(-)
-diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h
+diff --git a/MAINTAINERS b/MAINTAINERS
 index XXXXXXX..XXXXXXX 100644
---- a/include/sysemu/iothread.h
+--- a/MAINTAINERS
-+++ b/include/sysemu/iothread.h
++++ b/MAINTAINERS
-@@ -XXX,XX +XXX,XX @@ typedef struct {
+@@ -XXX,XX +XXX,XX @@ Linux io_uring
- char *iothread_get_id(IOThread *iothread);
+ M: Aarushi Mehta <mehta.aaru20@gmail.com>
- IOThread *iothread_by_id(const char *id);
+ M: Julia Suvorova <jusual@redhat.com>
- AioContext *iothread_get_aio_context(IOThread *iothread);
+ M: Stefan Hajnoczi <stefanha@redhat.com>
--void iothread_stop_all(void);
++R: Stefano Garzarella <sgarzare@redhat.com>
- GMainContext *iothread_get_g_main_context(IOThread *iothread);
+ L: qemu-block@nongnu.org
+ S: Maintained
- /*
+ F: block/io_uring.c
 diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
 index XXXXXXX..XXXXXXX 100644
 --- a/include/sysemu/sysemu.h
 +++ b/include/sysemu/sysemu.h
@@ -XXX,XX +XXX,XX @@ void vm_start(void);
  int vm_prepare_start(void);
  int vm_stop(RunState state);
  int vm_stop_force_state(RunState state);
 +int vm_shutdown(void);
  typedef enum WakeupReason {
      /* Always keep QEMU_WAKEUP_REASON_NONE = 0 */
 diff --git a/cpus.c b/cpus.c
 index XXXXXXX..XXXXXXX 100644
 --- a/cpus.c
 +++ b/cpus.c
@@ -XXX,XX +XXX,XX @@ void cpu_synchronize_all_pre_loadvm(void)
      }
  }
 -static int do_vm_stop(RunState state)
 +static int do_vm_stop(RunState state, bool send_stop)
  {
      int ret = 0;
@@ -XXX,XX +XXX,XX @@ static int do_vm_stop(RunState state)
          pause_all_vcpus();
          runstate_set(state);
          vm_state_notify(0, state);
 -        qapi_event_send_stop(&error_abort);
 +        if (send_stop) {
 +            qapi_event_send_stop(&error_abort);
 +        }
      }
      bdrv_drain_all();
@@ -XXX,XX +XXX,XX @@ static int do_vm_stop(RunState state)
      return ret;
  }
 +/* Special vm_stop() variant for terminating the process.  Historically clients
 + * did not expect a QMP STOP event and so we need to retain compatibility.
 + */
 +int vm_shutdown(void)
 +{
 +    return do_vm_stop(RUN_STATE_SHUTDOWN, false);
 +}
 +
  static bool cpu_can_run(CPUState *cpu)
  {
      if (cpu->stop) {
@@ -XXX,XX +XXX,XX @@ int vm_stop(RunState state)
          return 0;
      }
 -    return do_vm_stop(state);
 +    return do_vm_stop(state, true);
  }
  /**
 diff --git a/iothread.c b/iothread.c
 index XXXXXXX..XXXXXXX 100644
 --- a/iothread.c
 +++ b/iothread.c
@@ -XXX,XX +XXX,XX @@ void iothread_stop(IOThread *iothread)
      qemu_thread_join(&iothread->thread);
  }
 -static int iothread_stop_iter(Object *object, void *opaque)
 -{
 -    IOThread *iothread;
 -
 -    iothread = (IOThread *)object_dynamic_cast(object, TYPE_IOTHREAD);
 -    if (!iothread) {
 -        return 0;
 -    }
 -    iothread_stop(iothread);
 -    return 0;
 -}
 -
  static void iothread_instance_init(Object *obj)
  {
      IOThread *iothread = IOTHREAD(obj);
@@ -XXX,XX +XXX,XX @@ IOThreadInfoList *qmp_query_iothreads(Error **errp)
      return head;
  }
 -void iothread_stop_all(void)
 -{
 -    Object *container = object_get_objects_root();
 -    BlockDriverState *bs;
 -    BdrvNextIterator it;
 -
 -    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
 -        AioContext *ctx = bdrv_get_aio_context(bs);
 -        if (ctx == qemu_get_aio_context()) {
 -            continue;
 -        }
 -        aio_context_acquire(ctx);
 -        bdrv_set_aio_context(bs, qemu_get_aio_context());
 -        aio_context_release(ctx);
 -    }
 -
 -    object_child_foreach(container, iothread_stop_iter, NULL);
 -}
 -
  static gpointer iothread_g_main_context_init(gpointer opaque)
  {
      AioContext *ctx;
 diff --git a/vl.c b/vl.c
 index XXXXXXX..XXXXXXX 100644
 --- a/vl.c
 +++ b/vl.c
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv, char **envp)
      os_setup_post();
      main_loop();
 -    replay_disable_events();
 -    /* The ordering of the following is delicate.  Stop vcpus to prevent new
 -     * I/O requests being queued by the guest.  Then stop IOThreads (this
 -     * includes a drain operation and completes all request processing).  At
 -     * this point emulated devices are still associated with their IOThreads
 -     * (if any) but no longer have any work to do.  Only then can we close
 -     * block devices safely because we know there is no more I/O coming.
 -     */
 -    pause_all_vcpus();
 -    iothread_stop_all();
 +    /* No more vcpu or device emulation activity beyond this point */
 +    vm_shutdown();
 +
      bdrv_close_all();
      res_free();
 --
-.14.3
+.31.1

-[Qemu-devel] [PULL 6/7] virtio-scsi: fix race between .ioeventfd_stop() and vq handler
+[PULL for-6.1 2/3] block/io_uring: resubmit when result is -EAGAIN
-If the main loop thread invokes .ioeventfd_stop() just as the vq handler
+From: Fabian Ebner <f.ebner@proxmox.com>
 function begins in the IOThread then the handler may lose the race for
 the AioContext lock.  By the time the vq handler is able to acquire the
 AioContext lock the ioeventfd has already been removed and the handler
 isn't supposed to run anymore!
-Use the new aio_wait_bh_oneshot() function to perform ioeventfd removal
+Linux SCSI can throw spurious -EAGAIN in some corner cases in its
-from within the IOThread.  This way no races with the vq handler are
+completion path, which will end up being the result in the completed
-possible.
+io_uring request.
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Resubmitting such requests should allow block jobs to complete, even
-Reviewed-by: Fam Zheng <famz@redhat.com>
+if such spurious errors are encountered.
-Acked-by: Paolo Bonzini <pbonzini@redhat.com>
-Message-id: 20180307144205.20619-4-stefanha@redhat.com
+Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
 Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
 Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
 Message-id: 20210729091029.65369-1-f.ebner@proxmox.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- hw/scsi/virtio-scsi-dataplane.c | 9 +++++----
+ block/io_uring.c | 16 +++++++++++++++-
-file changed, 5 insertions(+), 4 deletions(-)
+file changed, 15 insertions(+), 1 deletion(-)
-diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c
+diff --git a/block/io_uring.c b/block/io_uring.c
 index XXXXXXX..XXXXXXX 100644
---- a/hw/scsi/virtio-scsi-dataplane.c
+--- a/block/io_uring.c
-+++ b/hw/scsi/virtio-scsi-dataplane.c
++++ b/block/io_uring.c
-@@ -XXX,XX +XXX,XX @@ static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQueue *vq, int n,
+@@ -XXX,XX +XXX,XX @@ static void luring_process_completions(LuringState *s)
-     return 0;
+         total_bytes = ret + luringcb->total_read;
- }
+         if (ret < 0) {
--/* assumes s->ctx held */
+-            if (ret == -EINTR) {
--static void virtio_scsi_clear_aio(VirtIOSCSI *s)
++            /*
-+/* Context: BH in IOThread */
++             * Only writev/readv/fsync requests on regular files or host block
-+static void virtio_scsi_dataplane_stop_bh(void *opaque)
++             * devices are submitted. Therefore -EAGAIN is not expected but it's
- {
++             * known to happen sometimes with Linux SCSI. Submit again and hope
-+    VirtIOSCSI *s = opaque;
++             * the request completes successfully.
-     VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(s);
++             *
-     int i;
++             * For more information, see:
++             * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@kernel.dk/T/#u
-@@ -XXX,XX +XXX,XX @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev)
++             *
-     return 0;
++             * If the code is changed to submit other types of requests in the
++             * future, then this workaround may need to be extended to deal with
- fail_vrings:
++             * genuine -EAGAIN results that should not be resubmitted
--    virtio_scsi_clear_aio(s);
++             * immediately.
-+    aio_wait_bh_oneshot(s->ctx, virtio_scsi_dataplane_stop_bh, s);
++             */
-     aio_context_release(s->ctx);
++            if (ret == -EINTR || ret == -EAGAIN) {
-     for (i = 0; i < vs->conf.num_queues + 2; i++) {
+                 luring_resubmit(s, luringcb);
-         virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false);
+                 continue;
-@@ -XXX,XX +XXX,XX @@ void virtio_scsi_dataplane_stop(VirtIODevice *vdev)
+             }
      s->dataplane_stopping = true;
      aio_context_acquire(s->ctx);
 -    virtio_scsi_clear_aio(s);
 +    aio_wait_bh_oneshot(s->ctx, virtio_scsi_dataplane_stop_bh, s);
      aio_context_release(s->ctx);
      blk_drain_all(); /* ensure there are no in-flight requests */
 --
-.14.3
+.31.1

-[Qemu-devel] [PULL 5/7] virtio-blk: fix race between .ioeventfd_stop() and vq handler
+[PULL for-6.1 3/3] MAINTAINERS: Added myself as a reviewer for the NVMe Block Driver
-If the main loop thread invokes .ioeventfd_stop() just as the vq handler
+From: Philippe Mathieu-Daudé <philmd@redhat.com>
 function begins in the IOThread then the handler may lose the race for
 the AioContext lock.  By the time the vq handler is able to acquire the
 AioContext lock the ioeventfd has already been removed and the handler
 isn't supposed to run anymore!
-Use the new aio_wait_bh_oneshot() function to perform ioeventfd removal
+I'm interested in following the activity around the NVMe bdrv.
 from within the IOThread.  This way no races with the vq handler are
 possible.
-Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
+Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
-Reviewed-by: Fam Zheng <famz@redhat.com>
+Message-id: 20210728183340.2018313-1-philmd@redhat.com
 Acked-by: Paolo Bonzini <pbonzini@redhat.com>
 Message-id: 20180307144205.20619-3-stefanha@redhat.com
 Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
 ---
- hw/block/dataplane/virtio-blk.c | 24 +++++++++++++++++-------
+ MAINTAINERS | 1 +
-file changed, 17 insertions(+), 7 deletions(-)
+file changed, 1 insertion(+)
-diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
+diff --git a/MAINTAINERS b/MAINTAINERS
 index XXXXXXX..XXXXXXX 100644
---- a/hw/block/dataplane/virtio-blk.c
+--- a/MAINTAINERS
-+++ b/hw/block/dataplane/virtio-blk.c
++++ b/MAINTAINERS
-@@ -XXX,XX +XXX,XX @@ int virtio_blk_data_plane_start(VirtIODevice *vdev)
+@@ -XXX,XX +XXX,XX @@ F: block/null.c
-     return -ENOSYS;
+ NVMe Block Driver
- }
+ M: Stefan Hajnoczi <stefanha@redhat.com>
+ R: Fam Zheng <fam@euphon.net>
-+/* Stop notifications for new requests from guest.
++R: Philippe Mathieu-Daudé <philmd@redhat.com>
-+ *
+ L: qemu-block@nongnu.org
-+ * Context: BH in IOThread
+ S: Supported
-+ */
+ F: block/nvme*
 +static void virtio_blk_data_plane_stop_bh(void *opaque)
 +{
 +    VirtIOBlockDataPlane *s = opaque;
 +    unsigned i;
 +
 +    for (i = 0; i < s->conf->num_queues; i++) {
 +        VirtQueue *vq = virtio_get_queue(s->vdev, i);
 +
 +        virtio_queue_aio_set_host_notifier_handler(vq, s->ctx, NULL);
 +    }
 +}
 +
  /* Context: QEMU global mutex held */
  void virtio_blk_data_plane_stop(VirtIODevice *vdev)
  {
@@ -XXX,XX +XXX,XX @@ void virtio_blk_data_plane_stop(VirtIODevice *vdev)
      trace_virtio_blk_data_plane_stop(s);
      aio_context_acquire(s->ctx);
 -
 -    /* Stop notifications for new requests from guest */
 -    for (i = 0; i < nvqs; i++) {
 -        VirtQueue *vq = virtio_get_queue(s->vdev, i);
 -
 -        virtio_queue_aio_set_host_notifier_handler(vq, s->ctx, NULL);
 -    }
 +    aio_wait_bh_oneshot(s->ctx, virtio_blk_data_plane_stop_bh, s);
      /* Drain and switch bs back to the QEMU main loop */
      blk_set_aio_context(s->conf->conf.blk, qemu_get_aio_context());
 --
-.14.3
+.31.1

The following changes since commit 0ab4537f08e09b13788db67efd760592fb7db769:

Merge remote-tracking branch 'remotes/stefanberger/tags/pull-tpm-2018-03-07-1' into staging (2018-03-08 12:56:39 +0000)

are available in the Git repository at:

git://github.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 4486e89c219c0d1b9bd8dfa0b1dd5b0d51ff2268:

vl: introduce vm_shutdown() (2018-03-08 17:38:51 +0000)

----------------------------------------------------------------

Deepa Srinivasan (1):
  block: Fix qemu crash when using scsi-block

Fam Zheng (1):
  README: Fix typo 'git-publish'

Sergio Lopez (1):
  virtio-blk: dataplane: Don't batch notifications if EVENT_IDX is
    present

Stefan Hajnoczi (4):
  block: add aio_wait_bh_oneshot()
  virtio-blk: fix race between .ioeventfd_stop() and vq handler
  virtio-scsi: fix race between .ioeventfd_stop() and vq handler
  vl: introduce vm_shutdown()

-- 
2.14.3

From: Deepa Srinivasan <deepa.srinivasan@oracle.com>

Starting qemu with the following arguments causes qemu to segfault:
... -device lsi,id=lsi0 -drive file=iscsi:<...>,format=raw,if=none,node-name=
iscsi1 -device scsi-block,bus=lsi0.0,id=<...>,drive=iscsi1

This patch fixes blk_aio_ioctl() so it does not pass stack addresses to
blk_aio_ioctl_entry() which may be invoked after blk_aio_ioctl() returns. More
details about the bug follow.

blk_aio_ioctl() invokes blk_aio_prwv() with blk_aio_ioctl_entry as the
coroutine parameter. blk_aio_prwv() ultimately calls aio_co_enter().

When blk_aio_ioctl() is executed from within a coroutine context (e.g.
iscsi_bh_cb()), aio_co_enter() adds the coroutine (blk_aio_ioctl_entry) to
the current coroutine's wakeup queue. blk_aio_ioctl() then returns.

When blk_aio_ioctl_entry() executes later, it accesses an invalid pointer:
....
    BlkRwCo *rwco = &acb->rwco;

rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset,
                             rwco->qiov->iov[0].iov_base);  <--- qiov is
                                                                 invalid here
...

In the case when blk_aio_ioctl() is called from a non-coroutine context,
blk_aio_ioctl_entry() executes immediately. But if bdrv_co_ioctl() calls
qemu_coroutine_yield(), blk_aio_ioctl() will return. When the coroutine
execution is complete, control returns to blk_aio_ioctl_entry() after the call
to blk_co_ioctl(). There is no invalid reference after this point, but the
function is still holding on to invalid pointers.

The fix is to change blk_aio_prwv() to accept a void pointer for the IO buffer
rather than a QEMUIOVector. blk_aio_prwv() passes this through in BlkRwCo and the
coroutine function casts it to QEMUIOVector or uses the void pointer directly.

Signed-off-by: Deepa Srinivasan <deepa.srinivasan@oracle.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Mark Kanda <mark.kanda@oracle.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/block-backend.c | 51 +++++++++++++++++++++++++--------------------------
 1 file changed, 25 insertions(+), 26 deletions(-)

diff --git a/block/block-backend.c b/block/block-backend.c
index XXXXXXX..XXXXXXX 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -XXX,XX +XXX,XX @@ int coroutine_fn blk_co_pwritev(BlockBackend *blk, int64_t offset,
 typedef struct BlkRwCo {
     BlockBackend *blk;
     int64_t offset;
-    QEMUIOVector *qiov;
+    void *iobuf;
     int ret;
     BdrvRequestFlags flags;
 } BlkRwCo;
@@ -XXX,XX +XXX,XX @@ typedef struct BlkRwCo {
 static void blk_read_entry(void *opaque)
 {
     BlkRwCo *rwco = opaque;
+    QEMUIOVector *qiov = rwco->iobuf;
 
-    rwco->ret = blk_co_preadv(rwco->blk, rwco->offset, rwco->qiov->size,
-                              rwco->qiov, rwco->flags);
+    rwco->ret = blk_co_preadv(rwco->blk, rwco->offset, qiov->size,
+                              qiov, rwco->flags);
 }
 
 static void blk_write_entry(void *opaque)
 {
     BlkRwCo *rwco = opaque;
+    QEMUIOVector *qiov = rwco->iobuf;
 
-    rwco->ret = blk_co_pwritev(rwco->blk, rwco->offset, rwco->qiov->size,
-                               rwco->qiov, rwco->flags);
+    rwco->ret = blk_co_pwritev(rwco->blk, rwco->offset, qiov->size,
+                               qiov, rwco->flags);
 }
 
 static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf,
@@ -XXX,XX +XXX,XX @@ static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf,
     rwco = (BlkRwCo) {
         .blk    = blk,
         .offset = offset,
-        .qiov   = &qiov,
+        .iobuf  = &qiov,
         .flags  = flags,
         .ret    = NOT_DONE,
     };
@@ -XXX,XX +XXX,XX @@ static void blk_aio_complete_bh(void *opaque)
 }
 
 static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, int64_t offset, int bytes,
-                                QEMUIOVector *qiov, CoroutineEntry co_entry,
+                                void *iobuf, CoroutineEntry co_entry,
                                 BdrvRequestFlags flags,
                                 BlockCompletionFunc *cb, void *opaque)
 {
@@ -XXX,XX +XXX,XX @@ static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, int64_t offset, int bytes,
     acb->rwco = (BlkRwCo) {
         .blk    = blk,
         .offset = offset,
-        .qiov   = qiov,
+        .iobuf  = iobuf,
         .flags  = flags,
         .ret    = NOT_DONE,
     };
@@ -XXX,XX +XXX,XX @@ static void blk_aio_read_entry(void *opaque)
 {
     BlkAioEmAIOCB *acb = opaque;
     BlkRwCo *rwco = &acb->rwco;
+    QEMUIOVector *qiov = rwco->iobuf;
 
-    assert(rwco->qiov->size == acb->bytes);
+    assert(qiov->size == acb->bytes);
     rwco->ret = blk_co_preadv(rwco->blk, rwco->offset, acb->bytes,
-                              rwco->qiov, rwco->flags);
+                              qiov, rwco->flags);
     blk_aio_complete(acb);
 }
 
@@ -XXX,XX +XXX,XX @@ static void blk_aio_write_entry(void *opaque)
 {
     BlkAioEmAIOCB *acb = opaque;
     BlkRwCo *rwco = &acb->rwco;
+    QEMUIOVector *qiov = rwco->iobuf;
 
-    assert(!rwco->qiov || rwco->qiov->size == acb->bytes);
+    assert(!qiov || qiov->size == acb->bytes);
     rwco->ret = blk_co_pwritev(rwco->blk, rwco->offset, acb->bytes,
-                               rwco->qiov, rwco->flags);
+                               qiov, rwco->flags);
     blk_aio_complete(acb);
 }
 
@@ -XXX,XX +XXX,XX @@ int blk_co_ioctl(BlockBackend *blk, unsigned long int req, void *buf)
 static void blk_ioctl_entry(void *opaque)
 {
     BlkRwCo *rwco = opaque;
+    QEMUIOVector *qiov = rwco->iobuf;
+
     rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset,
-                             rwco->qiov->iov[0].iov_base);
+                             qiov->iov[0].iov_base);
 }
 
 int blk_ioctl(BlockBackend *blk, unsigned long int req, void *buf)
@@ -XXX,XX +XXX,XX @@ static void blk_aio_ioctl_entry(void *opaque)
     BlkAioEmAIOCB *acb = opaque;
     BlkRwCo *rwco = &acb->rwco;
 
-    rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset,
-                             rwco->qiov->iov[0].iov_base);
+    rwco->ret = blk_co_ioctl(rwco->blk, rwco->offset, rwco->iobuf);
+
     blk_aio_complete(acb);
 }
 
 BlockAIOCB *blk_aio_ioctl(BlockBackend *blk, unsigned long int req, void *buf,
                           BlockCompletionFunc *cb, void *opaque)
 {
-    QEMUIOVector qiov;
-    struct iovec iov;
-
-    iov = (struct iovec) {
-        .iov_base = buf,
-        .iov_len = 0,
-    };
-    qemu_iovec_init_external(&qiov, &iov, 1);
-
-    return blk_aio_prwv(blk, req, 0, &qiov, blk_aio_ioctl_entry, 0, cb, opaque);
+    return blk_aio_prwv(blk, req, 0, buf, blk_aio_ioctl_entry, 0, cb, opaque);
 }
 
 int blk_co_pdiscard(BlockBackend *blk, int64_t offset, int bytes)
@@ -XXX,XX +XXX,XX @@ int blk_truncate(BlockBackend *blk, int64_t offset, PreallocMode prealloc,
 static void blk_pdiscard_entry(void *opaque)
 {
     BlkRwCo *rwco = opaque;
-    rwco->ret = blk_co_pdiscard(rwco->blk, rwco->offset, rwco->qiov->size);
+    QEMUIOVector *qiov = rwco->iobuf;
+
+    rwco->ret = blk_co_pdiscard(rwco->blk, rwco->offset, qiov->size);
 }
 
 int blk_pdiscard(BlockBackend *blk, int64_t offset, int bytes)
-- 
2.14.3

From: Sergio Lopez <slp@redhat.com>

Commit 5b2ffbe4d99843fd8305c573a100047a8c962327 ("virtio-blk: dataplane:
notify guest as a batch") deferred guest notification to a BH in order
batch notifications, with purpose of avoiding flooding the guest with
interruptions.

This optimization came with a cost. The average latency perceived in the
guest is increased by a few microseconds, but also when multiple IO
operations finish at the same time, the guest won't be notified until
all completions from each operation has been run. On the contrary,
virtio-scsi issues the notification at the end of each completion.

On the other hand, nowadays we have the EVENT_IDX feature that allows a
better coordination between QEMU and the Guest OS to avoid sending
unnecessary interruptions.

With this change, virtio-blk/dataplane only batches notifications if the
EVENT_IDX feature is not present.

Some numbers obtained with fio (ioengine=sync, iodepth=1, direct=1):
 - Test specs:
   * fio-3.4 (ioengine=sync, iodepth=1, direct=1)
   * qemu master
   * virtio-blk with a dedicated iothread (default poll-max-ns)
   * backend: null_blk nr_devices=1 irqmode=2 completion_nsec=280000
   * 8 vCPUs pinned to isolated physical cores
   * Emulator and iothread also pinned to separate isolated cores
   * variance between runs < 1%

- Not patched
   * numjobs=1:  lat_avg=327.32  irqs=29998
   * numjobs=4:  lat_avg=337.89  irqs=29073
   * numjobs=8:  lat_avg=342.98  irqs=28643

- Patched:
   * numjobs=1:  lat_avg=323.92  irqs=30262
   * numjobs=4:  lat_avg=332.65  irqs=29520
   * numjobs=8:  lat_avg=335.54  irqs=29323

Signed-off-by: Sergio Lopez <slp@redhat.com>
Message-id: 20180307114459.26636-1-slp@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/dataplane/virtio-blk.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -XXX,XX +XXX,XX @@ struct VirtIOBlockDataPlane {
     VirtIODevice *vdev;
     QEMUBH *bh;                     /* bh for guest notification */
     unsigned long *batch_notify_vqs;
+    bool batch_notifications;
 
     /* Note that these EventNotifiers are assigned by value.  This is
      * fine as long as you do not call event_notifier_cleanup on them
@@ -XXX,XX +XXX,XX @@ struct VirtIOBlockDataPlane {
 /* Raise an interrupt to signal guest, if necessary */
 void virtio_blk_data_plane_notify(VirtIOBlockDataPlane *s, VirtQueue *vq)
 {
-    set_bit(virtio_get_queue_index(vq), s->batch_notify_vqs);
-    qemu_bh_schedule(s->bh);
+    if (s->batch_notifications) {
+        set_bit(virtio_get_queue_index(vq), s->batch_notify_vqs);
+        qemu_bh_schedule(s->bh);
+    } else {
+        virtio_notify_irqfd(s->vdev, vq);
+    }
 }
 
 static void notify_guest_bh(void *opaque)
@@ -XXX,XX +XXX,XX @@ int virtio_blk_data_plane_start(VirtIODevice *vdev)
 
     s->starting = true;
 
+    if (!virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) {
+        s->batch_notifications = true;
+    } else {
+        s->batch_notifications = false;
+    }
+
     /* Set up guest notifier (irq) */
     r = k->set_guest_notifiers(qbus->parent, nvqs, true);
     if (r != 0) {
-- 
2.14.3

Sometimes it's necessary for the main loop thread to run a BH in an
IOThread and wait for its completion.  This primitive is useful during
startup/shutdown to synchronize and avoid race conditions.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20180307144205.20619-2-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/block/aio-wait.h | 13 +++++++++++++
 util/aio-wait.c          | 31 +++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h
index XXXXXXX..XXXXXXX 100644
--- a/include/block/aio-wait.h
+++ b/include/block/aio-wait.h
@@ -XXX,XX +XXX,XX @@ typedef struct {
  */
 void aio_wait_kick(AioWait *wait);
 
+/**
+ * aio_wait_bh_oneshot:
+ * @ctx: the aio context
+ * @cb: the BH callback function
+ * @opaque: user data for the BH callback function
+ *
+ * Run a BH in @ctx and wait for it to complete.
+ *
+ * Must be called from the main loop thread with @ctx acquired exactly once.
+ * Note that main loop event processing may occur.
+ */
+void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque);
+
 #endif /* QEMU_AIO_WAIT */
diff --git a/util/aio-wait.c b/util/aio-wait.c
index XXXXXXX..XXXXXXX 100644
--- a/util/aio-wait.c
+++ b/util/aio-wait.c
@@ -XXX,XX +XXX,XX @@ void aio_wait_kick(AioWait *wait)
         aio_bh_schedule_oneshot(qemu_get_aio_context(), dummy_bh_cb, NULL);
     }
 }
+
+typedef struct {
+    AioWait wait;
+    bool done;
+    QEMUBHFunc *cb;
+    void *opaque;
+} AioWaitBHData;
+
+/* Context: BH in IOThread */
+static void aio_wait_bh(void *opaque)
+{
+    AioWaitBHData *data = opaque;
+
+    data->cb(data->opaque);
+
+    data->done = true;
+    aio_wait_kick(&data->wait);
+}
+
+void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
+{
+    AioWaitBHData data = {
+        .cb = cb,
+        .opaque = opaque,
+    };
+
+    assert(qemu_get_current_aio_context() == qemu_get_aio_context());
+
+    aio_bh_schedule_oneshot(ctx, aio_wait_bh, &data);
+    AIO_WAIT_WHILE(&data.wait, ctx, !data.done);
+}
-- 
2.14.3

If the main loop thread invokes .ioeventfd_stop() just as the vq handler
function begins in the IOThread then the handler may lose the race for
the AioContext lock.  By the time the vq handler is able to acquire the
AioContext lock the ioeventfd has already been removed and the handler
isn't supposed to run anymore!

Use the new aio_wait_bh_oneshot() function to perform ioeventfd removal
from within the IOThread.  This way no races with the vq handler are
possible.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20180307144205.20619-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/dataplane/virtio-blk.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -XXX,XX +XXX,XX @@ int virtio_blk_data_plane_start(VirtIODevice *vdev)
     return -ENOSYS;
 }
 
+/* Stop notifications for new requests from guest.
+ *
+ * Context: BH in IOThread
+ */
+static void virtio_blk_data_plane_stop_bh(void *opaque)
+{
+    VirtIOBlockDataPlane *s = opaque;
+    unsigned i;
+
+    for (i = 0; i < s->conf->num_queues; i++) {
+        VirtQueue *vq = virtio_get_queue(s->vdev, i);
+
+        virtio_queue_aio_set_host_notifier_handler(vq, s->ctx, NULL);
+    }
+}
+
 /* Context: QEMU global mutex held */
 void virtio_blk_data_plane_stop(VirtIODevice *vdev)
 {
@@ -XXX,XX +XXX,XX @@ void virtio_blk_data_plane_stop(VirtIODevice *vdev)
     trace_virtio_blk_data_plane_stop(s);
 
     aio_context_acquire(s->ctx);
-
-    /* Stop notifications for new requests from guest */
-    for (i = 0; i < nvqs; i++) {
-        VirtQueue *vq = virtio_get_queue(s->vdev, i);
-
-        virtio_queue_aio_set_host_notifier_handler(vq, s->ctx, NULL);
-    }
+    aio_wait_bh_oneshot(s->ctx, virtio_blk_data_plane_stop_bh, s);
 
     /* Drain and switch bs back to the QEMU main loop */
     blk_set_aio_context(s->conf->conf.blk, qemu_get_aio_context());
-- 
2.14.3

Use the new aio_wait_bh_oneshot() function to perform ioeventfd removal
from within the IOThread.  This way no races with the vq handler are
possible.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20180307144205.20619-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/scsi/virtio-scsi-dataplane.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c
index XXXXXXX..XXXXXXX 100644
--- a/hw/scsi/virtio-scsi-dataplane.c
+++ b/hw/scsi/virtio-scsi-dataplane.c
@@ -XXX,XX +XXX,XX @@ static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQueue *vq, int n,
     return 0;
 }
 
-/* assumes s->ctx held */
-static void virtio_scsi_clear_aio(VirtIOSCSI *s)
+/* Context: BH in IOThread */
+static void virtio_scsi_dataplane_stop_bh(void *opaque)
 {
+    VirtIOSCSI *s = opaque;
     VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(s);
     int i;
 
@@ -XXX,XX +XXX,XX @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev)
     return 0;
 
 fail_vrings:
-    virtio_scsi_clear_aio(s);
+    aio_wait_bh_oneshot(s->ctx, virtio_scsi_dataplane_stop_bh, s);
     aio_context_release(s->ctx);
     for (i = 0; i < vs->conf.num_queues + 2; i++) {
         virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false);
@@ -XXX,XX +XXX,XX @@ void virtio_scsi_dataplane_stop(VirtIODevice *vdev)
     s->dataplane_stopping = true;
 
     aio_context_acquire(s->ctx);
-    virtio_scsi_clear_aio(s);
+    aio_wait_bh_oneshot(s->ctx, virtio_scsi_dataplane_stop_bh, s);
     aio_context_release(s->ctx);
 
     blk_drain_all(); /* ensure there are no in-flight requests */
-- 
2.14.3

Commit 00d09fdbbae5f7864ce754913efc84c12fdf9f1a ("vl: pause vcpus before
stopping iothreads") and commit dce8921b2baaf95974af8176406881872067adfa
("iothread: Stop threads before main() quits") tried to work around the
fact that emulation was still active during termination by stopping
iothreads.  They suffer from race conditions:
1. virtio_scsi_handle_cmd_vq() racing with iothread_stop_all() hits the
   virtio_scsi_ctx_check() assertion failure because the BDS AioContext
   has been modified by iothread_stop_all().
2. Guest vq kick racing with main loop termination leaves a readable
   ioeventfd that is handled by the next aio_poll() when external
   clients are enabled again, resulting in unwanted emulation activity.

This patch obsoletes those commits by fully disabling emulation activity
when vcpus are stopped.

Use the new vm_shutdown() function instead of pause_all_vcpus() so that
vm change state handlers are invoked too.  Virtio devices will now stop
their ioeventfds, preventing further emulation activity after vm_stop().

Note that vm_stop(RUN_STATE_SHUTDOWN) cannot be used because it emits a
QMP STOP event that may affect existing clients.

It is no longer necessary to call replay_disable_events() directly since
vm_shutdown() does so already.

Drop iothread_stop_all() since it is no longer used.

Cc: Fam Zheng <famz@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20180307144205.20619-5-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/sysemu/iothread.h |  1 -
 include/sysemu/sysemu.h   |  1 +
 cpus.c                    | 16 +++++++++++++---
 iothread.c                | 31 -------------------------------
 vl.c                      | 13 +++----------
 5 files changed, 17 insertions(+), 45 deletions(-)

diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/iothread.h
+++ b/include/sysemu/iothread.h
@@ -XXX,XX +XXX,XX @@ typedef struct {
 char *iothread_get_id(IOThread *iothread);
 IOThread *iothread_by_id(const char *id);
 AioContext *iothread_get_aio_context(IOThread *iothread);
-void iothread_stop_all(void);
 GMainContext *iothread_get_g_main_context(IOThread *iothread);
 
 /*
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index XXXXXXX..XXXXXXX 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -XXX,XX +XXX,XX @@ void vm_start(void);
 int vm_prepare_start(void);
 int vm_stop(RunState state);
 int vm_stop_force_state(RunState state);
+int vm_shutdown(void);
 
 typedef enum WakeupReason {
     /* Always keep QEMU_WAKEUP_REASON_NONE = 0 */
diff --git a/cpus.c b/cpus.c
index XXXXXXX..XXXXXXX 100644
--- a/cpus.c
+++ b/cpus.c
@@ -XXX,XX +XXX,XX @@ void cpu_synchronize_all_pre_loadvm(void)
     }
 }
 
-static int do_vm_stop(RunState state)
+static int do_vm_stop(RunState state, bool send_stop)
 {
     int ret = 0;
 
@@ -XXX,XX +XXX,XX @@ static int do_vm_stop(RunState state)
         pause_all_vcpus();
         runstate_set(state);
         vm_state_notify(0, state);
-        qapi_event_send_stop(&error_abort);
+        if (send_stop) {
+            qapi_event_send_stop(&error_abort);
+        }
     }
 
     bdrv_drain_all();
@@ -XXX,XX +XXX,XX @@ static int do_vm_stop(RunState state)
     return ret;
 }
 
+/* Special vm_stop() variant for terminating the process.  Historically clients
+ * did not expect a QMP STOP event and so we need to retain compatibility.
+ */
+int vm_shutdown(void)
+{
+    return do_vm_stop(RUN_STATE_SHUTDOWN, false);
+}
+
 static bool cpu_can_run(CPUState *cpu)
 {
     if (cpu->stop) {
@@ -XXX,XX +XXX,XX @@ int vm_stop(RunState state)
         return 0;
     }
 
-    return do_vm_stop(state);
+    return do_vm_stop(state, true);
 }
 
 /**
diff --git a/iothread.c b/iothread.c
index XXXXXXX..XXXXXXX 100644
--- a/iothread.c
+++ b/iothread.c
@@ -XXX,XX +XXX,XX @@ void iothread_stop(IOThread *iothread)
     qemu_thread_join(&iothread->thread);
 }
 
-static int iothread_stop_iter(Object *object, void *opaque)
-{
-    IOThread *iothread;
-
-    iothread = (IOThread *)object_dynamic_cast(object, TYPE_IOTHREAD);
-    if (!iothread) {
-        return 0;
-    }
-    iothread_stop(iothread);
-    return 0;
-}
-
 static void iothread_instance_init(Object *obj)
 {
     IOThread *iothread = IOTHREAD(obj);
@@ -XXX,XX +XXX,XX @@ IOThreadInfoList *qmp_query_iothreads(Error **errp)
     return head;
 }
 
-void iothread_stop_all(void)
-{
-    Object *container = object_get_objects_root();
-    BlockDriverState *bs;
-    BdrvNextIterator it;
-
-    for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
-        AioContext *ctx = bdrv_get_aio_context(bs);
-        if (ctx == qemu_get_aio_context()) {
-            continue;
-        }
-        aio_context_acquire(ctx);
-        bdrv_set_aio_context(bs, qemu_get_aio_context());
-        aio_context_release(ctx);
-    }
-
-    object_child_foreach(container, iothread_stop_iter, NULL);
-}
-
 static gpointer iothread_g_main_context_init(gpointer opaque)
 {
     AioContext *ctx;
diff --git a/vl.c b/vl.c
index XXXXXXX..XXXXXXX 100644
--- a/vl.c
+++ b/vl.c
@@ -XXX,XX +XXX,XX @@ int main(int argc, char **argv, char **envp)
     os_setup_post();
 
     main_loop();
-    replay_disable_events();
 
-    /* The ordering of the following is delicate.  Stop vcpus to prevent new
-     * I/O requests being queued by the guest.  Then stop IOThreads (this
-     * includes a drain operation and completes all request processing).  At
-     * this point emulated devices are still associated with their IOThreads
-     * (if any) but no longer have any work to do.  Only then can we close
-     * block devices safely because we know there is no more I/O coming.
-     */
-    pause_all_vcpus();
-    iothread_stop_all();
+    /* No more vcpu or device emulation activity beyond this point */
+    vm_shutdown();
+
     bdrv_close_all();
 
     res_free();
-- 
2.14.3

The following changes since commit 3521ade3510eb5cefb2e27a101667f25dad89935:

Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-07-29' into staging (2021-07-29 13:17:20 +0100)

are available in the Git repository at:

https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to cc8eecd7f105a1dff5876adeb238a14696061a4a:

MAINTAINERS: Added myself as a reviewer for the NVMe Block Driver (2021-07-29 17:17:34 +0100)

----------------------------------------------------------------
Pull request

The main fix here is for io_uring. Spurious -EAGAIN errors can happen and the
request needs to be resubmitted.

The MAINTAINERS changes carry no risk and we might as well include them in QEMU
6.1.

----------------------------------------------------------------

Fabian Ebner (1):
  block/io_uring: resubmit when result is -EAGAIN

Philippe Mathieu-Daudé (1):
  MAINTAINERS: Added myself as a reviewer for the NVMe Block Driver

Stefano Garzarella (1):
  MAINTAINERS: add Stefano Garzarella as io_uring reviewer

MAINTAINERS      |  2 ++
 block/io_uring.c | 16 +++++++++++++++-
 2 files changed, 17 insertions(+), 1 deletion(-)

-- 
2.31.1

From: Fabian Ebner <f.ebner@proxmox.com>

Linux SCSI can throw spurious -EAGAIN in some corner cases in its
completion path, which will end up being the result in the completed
io_uring request.

Resubmitting such requests should allow block jobs to complete, even
if such spurious errors are encountered.

Co-authored-by: Stefan Hajnoczi <stefanha@gmail.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Message-id: 20210729091029.65369-1-f.ebner@proxmox.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/io_uring.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/block/io_uring.c b/block/io_uring.c
index XXXXXXX..XXXXXXX 100644
--- a/block/io_uring.c
+++ b/block/io_uring.c
@@ -XXX,XX +XXX,XX @@ static void luring_process_completions(LuringState *s)
         total_bytes = ret + luringcb->total_read;
 
         if (ret < 0) {
-            if (ret == -EINTR) {
+            /*
+             * Only writev/readv/fsync requests on regular files or host block
+             * devices are submitted. Therefore -EAGAIN is not expected but it's
+             * known to happen sometimes with Linux SCSI. Submit again and hope
+             * the request completes successfully.
+             *
+             * For more information, see:
+             * https://lore.kernel.org/io-uring/20210727165811.284510-3-axboe@kernel.dk/T/#u
+             *
+             * If the code is changed to submit other types of requests in the
+             * future, then this workaround may need to be extended to deal with
+             * genuine -EAGAIN results that should not be resubmitted
+             * immediately.
+             */
+            if (ret == -EINTR || ret == -EAGAIN) {
                 luring_resubmit(s, luringcb);
                 continue;
             }
-- 
2.31.1