From nobody Thu May 2 06:20:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1520601743343706.368951894595; Fri, 9 Mar 2018 05:22:23 -0800 (PST) Received: from localhost ([::1]:45243 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHyX-0002Ni-Q1 for importer@patchew.org; Fri, 09 Mar 2018 08:22:21 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45317) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHwN-000105-Ri for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1euHwM-0007OM-FK for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:07 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:48112 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1euHwD-0007L1-Ke; Fri, 09 Mar 2018 08:19:57 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 57C1B8D6FB; Fri, 9 Mar 2018 13:19:55 +0000 (UTC) Received: from localhost (ovpn-116-226.ams2.redhat.com [10.36.116.226]) by smtp.corp.redhat.com (Postfix) with ESMTP id D9D49202322A; Fri, 9 Mar 2018 13:19:54 +0000 (UTC) From: Stefan Hajnoczi To: Date: Fri, 9 Mar 2018 13:19:43 +0000 Message-Id: <20180309131949.18640-2-stefanha@redhat.com> In-Reply-To: <20180309131949.18640-1-stefanha@redhat.com> References: <20180309131949.18640-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 09 Mar 2018 13:19:55 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Fri, 09 Mar 2018 13:19:55 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'stefanha@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 1/7] block: Fix qemu crash when using scsi-block X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , qemu-block@nongnu.org, Peter Crosthwaite , "Michael S. Tsirkin" , Max Reitz , Deepa Srinivasan , Stefan Hajnoczi , Paolo Bonzini , Konrad Rzeszutek Wilk , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Deepa Srinivasan Starting qemu with the following arguments causes qemu to segfault: ... -device lsi,id=3Dlsi0 -drive file=3Discsi:<...>,format=3Draw,if=3Dnone,= node-name=3D iscsi1 -device scsi-block,bus=3Dlsi0.0,id=3D<...>,drive=3Discsi1 This patch fixes blk_aio_ioctl() so it does not pass stack addresses to blk_aio_ioctl_entry() which may be invoked after blk_aio_ioctl() returns. M= ore details about the bug follow. blk_aio_ioctl() invokes blk_aio_prwv() with blk_aio_ioctl_entry as the coroutine parameter. blk_aio_prwv() ultimately calls aio_co_enter(). When blk_aio_ioctl() is executed from within a coroutine context (e.g. iscsi_bh_cb()), aio_co_enter() adds the coroutine (blk_aio_ioctl_entry) to the current coroutine's wakeup queue. blk_aio_ioctl() then returns. When blk_aio_ioctl_entry() executes later, it accesses an invalid pointer: .... BlkRwCo *rwco =3D &acb->rwco; rwco->ret =3D blk_co_ioctl(rwco->blk, rwco->offset, rwco->qiov->iov[0].iov_base); <--- qiov is invalid he= re ... In the case when blk_aio_ioctl() is called from a non-coroutine context, blk_aio_ioctl_entry() executes immediately. But if bdrv_co_ioctl() calls qemu_coroutine_yield(), blk_aio_ioctl() will return. When the coroutine execution is complete, control returns to blk_aio_ioctl_entry() after the c= all to blk_co_ioctl(). There is no invalid reference after this point, but the function is still holding on to invalid pointers. The fix is to change blk_aio_prwv() to accept a void pointer for the IO buf= fer rather than a QEMUIOVector. blk_aio_prwv() passes this through in BlkRwCo a= nd the coroutine function casts it to QEMUIOVector or uses the void pointer direct= ly. Signed-off-by: Deepa Srinivasan Signed-off-by: Konrad Rzeszutek Wilk Reviewed-by: Mark Kanda Reviewed-by: Paolo Bonzini Signed-off-by: Stefan Hajnoczi --- block/block-backend.c | 51 +++++++++++++++++++++++++----------------------= ---- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index b3c790e2bd..f2e0a855ff 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1150,7 +1150,7 @@ int coroutine_fn blk_co_pwritev(BlockBackend *blk, in= t64_t offset, typedef struct BlkRwCo { BlockBackend *blk; int64_t offset; - QEMUIOVector *qiov; + void *iobuf; int ret; BdrvRequestFlags flags; } BlkRwCo; @@ -1158,17 +1158,19 @@ typedef struct BlkRwCo { static void blk_read_entry(void *opaque) { BlkRwCo *rwco =3D opaque; + QEMUIOVector *qiov =3D rwco->iobuf; =20 - rwco->ret =3D blk_co_preadv(rwco->blk, rwco->offset, rwco->qiov->size, - rwco->qiov, rwco->flags); + rwco->ret =3D blk_co_preadv(rwco->blk, rwco->offset, qiov->size, + qiov, rwco->flags); } =20 static void blk_write_entry(void *opaque) { BlkRwCo *rwco =3D opaque; + QEMUIOVector *qiov =3D rwco->iobuf; =20 - rwco->ret =3D blk_co_pwritev(rwco->blk, rwco->offset, rwco->qiov->size, - rwco->qiov, rwco->flags); + rwco->ret =3D blk_co_pwritev(rwco->blk, rwco->offset, qiov->size, + qiov, rwco->flags); } =20 static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf, @@ -1188,7 +1190,7 @@ static int blk_prw(BlockBackend *blk, int64_t offset,= uint8_t *buf, rwco =3D (BlkRwCo) { .blk =3D blk, .offset =3D offset, - .qiov =3D &qiov, + .iobuf =3D &qiov, .flags =3D flags, .ret =3D NOT_DONE, }; @@ -1296,7 +1298,7 @@ static void blk_aio_complete_bh(void *opaque) } =20 static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, int64_t offset, int byt= es, - QEMUIOVector *qiov, CoroutineEntry co_entr= y, + void *iobuf, CoroutineEntry co_entry, BdrvRequestFlags flags, BlockCompletionFunc *cb, void *opaque) { @@ -1308,7 +1310,7 @@ static BlockAIOCB *blk_aio_prwv(BlockBackend *blk, in= t64_t offset, int bytes, acb->rwco =3D (BlkRwCo) { .blk =3D blk, .offset =3D offset, - .qiov =3D qiov, + .iobuf =3D iobuf, .flags =3D flags, .ret =3D NOT_DONE, }; @@ -1331,10 +1333,11 @@ static void blk_aio_read_entry(void *opaque) { BlkAioEmAIOCB *acb =3D opaque; BlkRwCo *rwco =3D &acb->rwco; + QEMUIOVector *qiov =3D rwco->iobuf; =20 - assert(rwco->qiov->size =3D=3D acb->bytes); + assert(qiov->size =3D=3D acb->bytes); rwco->ret =3D blk_co_preadv(rwco->blk, rwco->offset, acb->bytes, - rwco->qiov, rwco->flags); + qiov, rwco->flags); blk_aio_complete(acb); } =20 @@ -1342,10 +1345,11 @@ static void blk_aio_write_entry(void *opaque) { BlkAioEmAIOCB *acb =3D opaque; BlkRwCo *rwco =3D &acb->rwco; + QEMUIOVector *qiov =3D rwco->iobuf; =20 - assert(!rwco->qiov || rwco->qiov->size =3D=3D acb->bytes); + assert(!qiov || qiov->size =3D=3D acb->bytes); rwco->ret =3D blk_co_pwritev(rwco->blk, rwco->offset, acb->bytes, - rwco->qiov, rwco->flags); + qiov, rwco->flags); blk_aio_complete(acb); } =20 @@ -1474,8 +1478,10 @@ int blk_co_ioctl(BlockBackend *blk, unsigned long in= t req, void *buf) static void blk_ioctl_entry(void *opaque) { BlkRwCo *rwco =3D opaque; + QEMUIOVector *qiov =3D rwco->iobuf; + rwco->ret =3D blk_co_ioctl(rwco->blk, rwco->offset, - rwco->qiov->iov[0].iov_base); + qiov->iov[0].iov_base); } =20 int blk_ioctl(BlockBackend *blk, unsigned long int req, void *buf) @@ -1488,24 +1494,15 @@ static void blk_aio_ioctl_entry(void *opaque) BlkAioEmAIOCB *acb =3D opaque; BlkRwCo *rwco =3D &acb->rwco; =20 - rwco->ret =3D blk_co_ioctl(rwco->blk, rwco->offset, - rwco->qiov->iov[0].iov_base); + rwco->ret =3D blk_co_ioctl(rwco->blk, rwco->offset, rwco->iobuf); + blk_aio_complete(acb); } =20 BlockAIOCB *blk_aio_ioctl(BlockBackend *blk, unsigned long int req, void *= buf, BlockCompletionFunc *cb, void *opaque) { - QEMUIOVector qiov; - struct iovec iov; - - iov =3D (struct iovec) { - .iov_base =3D buf, - .iov_len =3D 0, - }; - qemu_iovec_init_external(&qiov, &iov, 1); - - return blk_aio_prwv(blk, req, 0, &qiov, blk_aio_ioctl_entry, 0, cb, op= aque); + return blk_aio_prwv(blk, req, 0, buf, blk_aio_ioctl_entry, 0, cb, opaq= ue); } =20 int blk_co_pdiscard(BlockBackend *blk, int64_t offset, int bytes) @@ -1949,7 +1946,9 @@ int blk_truncate(BlockBackend *blk, int64_t offset, P= reallocMode prealloc, static void blk_pdiscard_entry(void *opaque) { BlkRwCo *rwco =3D opaque; - rwco->ret =3D blk_co_pdiscard(rwco->blk, rwco->offset, rwco->qiov->siz= e); + QEMUIOVector *qiov =3D rwco->iobuf; + + rwco->ret =3D blk_co_pdiscard(rwco->blk, rwco->offset, qiov->size); } =20 int blk_pdiscard(BlockBackend *blk, int64_t offset, int bytes) --=20 2.14.3 From nobody Thu May 2 06:20:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1520601757299187.80546926662953; Fri, 9 Mar 2018 05:22:37 -0800 (PST) Received: from localhost ([::1]:45245 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHyk-0002Zb-DS for importer@patchew.org; Fri, 09 Mar 2018 08:22:34 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45369) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHwS-00017e-LQ for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1euHwR-0007Uk-J1 for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:12 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:56288 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1euHwK-0007NH-BK; Fri, 09 Mar 2018 08:20:04 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E55A8406E8B9; Fri, 9 Mar 2018 13:20:03 +0000 (UTC) Received: from localhost (ovpn-116-226.ams2.redhat.com [10.36.116.226]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4F5B10B0F24; Fri, 9 Mar 2018 13:19:56 +0000 (UTC) From: Stefan Hajnoczi To: Date: Fri, 9 Mar 2018 13:19:44 +0000 Message-Id: <20180309131949.18640-3-stefanha@redhat.com> In-Reply-To: <20180309131949.18640-1-stefanha@redhat.com> References: <20180309131949.18640-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Fri, 09 Mar 2018 13:20:03 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Fri, 09 Mar 2018 13:20:03 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'stefanha@redhat.com' RCPT:'' Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 2/7] README: Fix typo 'git-publish' X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , qemu-block@nongnu.org, Peter Crosthwaite , "Michael S. Tsirkin" , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" From: Fam Zheng Reported-by: Alberto Garcia Signed-off-by: Fam Zheng Reviewed-by: Philippe Mathieu-Daud=C3=A9 Message-id: 20180306024328.19195-1-famz@redhat.com Signed-off-by: Stefan Hajnoczi --- README | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README b/README index 7833b97365..49a9fd09cd 100644 --- a/README +++ b/README @@ -73,7 +73,7 @@ The QEMU website is also maintained under source control. git clone git://git.qemu.org/qemu-web.git https://www.qemu.org/2017/02/04/the-new-qemu-website-is-up/ =20 -A 'git-profile' utility was created to make above process less +A 'git-publish' utility was created to make above process less cumbersome, and is highly recommended for making regular contributions, or even just for sending consecutive patch series revisions. It also requires a working 'git send-email' setup, and by default doesn't --=20 2.14.3 From nobody Thu May 2 06:20:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1520602065794222.3147147327893; Fri, 9 Mar 2018 05:27:45 -0800 (PST) Received: from localhost ([::1]:45279 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euI3f-0006gw-Q8 for importer@patchew.org; Fri, 09 Mar 2018 08:27:39 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45422) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHwV-0001BC-Np for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1euHwR-0007Up-JH for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:15 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:39170 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1euHwM-0007O1-3B; Fri, 09 Mar 2018 08:20:06 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A499883055F1; Fri, 9 Mar 2018 13:20:05 +0000 (UTC) Received: from localhost (ovpn-116-226.ams2.redhat.com [10.36.116.226]) by smtp.corp.redhat.com (Postfix) with ESMTP id 519F32166BAE; Fri, 9 Mar 2018 13:20:05 +0000 (UTC) From: Stefan Hajnoczi To: Date: Fri, 9 Mar 2018 13:19:45 +0000 Message-Id: <20180309131949.18640-4-stefanha@redhat.com> In-Reply-To: <20180309131949.18640-1-stefanha@redhat.com> References: <20180309131949.18640-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 09 Mar 2018 13:20:05 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 09 Mar 2018 13:20:05 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'stefanha@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 3/7] virtio-blk: dataplane: Don't batch notifications if EVENT_IDX is present X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , Sergio Lopez , qemu-block@nongnu.org, Peter Crosthwaite , "Michael S. Tsirkin" , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Sergio Lopez Commit 5b2ffbe4d99843fd8305c573a100047a8c962327 ("virtio-blk: dataplane: notify guest as a batch") deferred guest notification to a BH in order batch notifications, with purpose of avoiding flooding the guest with interruptions. This optimization came with a cost. The average latency perceived in the guest is increased by a few microseconds, but also when multiple IO operations finish at the same time, the guest won't be notified until all completions from each operation has been run. On the contrary, virtio-scsi issues the notification at the end of each completion. On the other hand, nowadays we have the EVENT_IDX feature that allows a better coordination between QEMU and the Guest OS to avoid sending unnecessary interruptions. With this change, virtio-blk/dataplane only batches notifications if the EVENT_IDX feature is not present. Some numbers obtained with fio (ioengine=3Dsync, iodepth=3D1, direct=3D1): - Test specs: * fio-3.4 (ioengine=3Dsync, iodepth=3D1, direct=3D1) * qemu master * virtio-blk with a dedicated iothread (default poll-max-ns) * backend: null_blk nr_devices=3D1 irqmode=3D2 completion_nsec=3D280000 * 8 vCPUs pinned to isolated physical cores * Emulator and iothread also pinned to separate isolated cores * variance between runs < 1% - Not patched * numjobs=3D1: lat_avg=3D327.32 irqs=3D29998 * numjobs=3D4: lat_avg=3D337.89 irqs=3D29073 * numjobs=3D8: lat_avg=3D342.98 irqs=3D28643 - Patched: * numjobs=3D1: lat_avg=3D323.92 irqs=3D30262 * numjobs=3D4: lat_avg=3D332.65 irqs=3D29520 * numjobs=3D8: lat_avg=3D335.54 irqs=3D29323 Signed-off-by: Sergio Lopez Message-id: 20180307114459.26636-1-slp@redhat.com Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-bl= k.c index 2cb990997e..c46253a924 100644 --- a/hw/block/dataplane/virtio-blk.c +++ b/hw/block/dataplane/virtio-blk.c @@ -34,6 +34,7 @@ struct VirtIOBlockDataPlane { VirtIODevice *vdev; QEMUBH *bh; /* bh for guest notification */ unsigned long *batch_notify_vqs; + bool batch_notifications; =20 /* Note that these EventNotifiers are assigned by value. This is * fine as long as you do not call event_notifier_cleanup on them @@ -47,8 +48,12 @@ struct VirtIOBlockDataPlane { /* Raise an interrupt to signal guest, if necessary */ void virtio_blk_data_plane_notify(VirtIOBlockDataPlane *s, VirtQueue *vq) { - set_bit(virtio_get_queue_index(vq), s->batch_notify_vqs); - qemu_bh_schedule(s->bh); + if (s->batch_notifications) { + set_bit(virtio_get_queue_index(vq), s->batch_notify_vqs); + qemu_bh_schedule(s->bh); + } else { + virtio_notify_irqfd(s->vdev, vq); + } } =20 static void notify_guest_bh(void *opaque) @@ -177,6 +182,12 @@ int virtio_blk_data_plane_start(VirtIODevice *vdev) =20 s->starting =3D true; =20 + if (!virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) { + s->batch_notifications =3D true; + } else { + s->batch_notifications =3D false; + } + /* Set up guest notifier (irq) */ r =3D k->set_guest_notifiers(qbus->parent, nvqs, true); if (r !=3D 0) { --=20 2.14.3 From nobody Thu May 2 06:20:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1520601991820342.6786587245622; Fri, 9 Mar 2018 05:26:31 -0800 (PST) Received: from localhost ([::1]:45274 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euI2Z-00063I-4F for importer@patchew.org; Fri, 09 Mar 2018 08:26:31 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45403) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHwU-00019F-5s for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1euHwT-0007Vt-7a for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:14 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:37612 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1euHwN-0007Pa-Rw; Fri, 09 Mar 2018 08:20:07 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7E0A34023112; Fri, 9 Mar 2018 13:20:07 +0000 (UTC) Received: from localhost (ovpn-116-226.ams2.redhat.com [10.36.116.226]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2BA1B202322A; Fri, 9 Mar 2018 13:20:06 +0000 (UTC) From: Stefan Hajnoczi To: Date: Fri, 9 Mar 2018 13:19:46 +0000 Message-Id: <20180309131949.18640-5-stefanha@redhat.com> In-Reply-To: <20180309131949.18640-1-stefanha@redhat.com> References: <20180309131949.18640-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 09 Mar 2018 13:20:07 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 09 Mar 2018 13:20:07 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'stefanha@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 4/7] block: add aio_wait_bh_oneshot() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , qemu-block@nongnu.org, Peter Crosthwaite , "Michael S. Tsirkin" , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Sometimes it's necessary for the main loop thread to run a BH in an IOThread and wait for its completion. This primitive is useful during startup/shutdown to synchronize and avoid race conditions. Signed-off-by: Stefan Hajnoczi Reviewed-by: Fam Zheng Acked-by: Paolo Bonzini Message-id: 20180307144205.20619-2-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- include/block/aio-wait.h | 13 +++++++++++++ util/aio-wait.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h index a48c744fa8..f7a3972200 100644 --- a/include/block/aio-wait.h +++ b/include/block/aio-wait.h @@ -113,4 +113,17 @@ typedef struct { */ void aio_wait_kick(AioWait *wait); =20 +/** + * aio_wait_bh_oneshot: + * @ctx: the aio context + * @cb: the BH callback function + * @opaque: user data for the BH callback function + * + * Run a BH in @ctx and wait for it to complete. + * + * Must be called from the main loop thread with @ctx acquired exactly onc= e. + * Note that main loop event processing may occur. + */ +void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque); + #endif /* QEMU_AIO_WAIT */ diff --git a/util/aio-wait.c b/util/aio-wait.c index a487cdb852..975afddf4c 100644 --- a/util/aio-wait.c +++ b/util/aio-wait.c @@ -38,3 +38,34 @@ void aio_wait_kick(AioWait *wait) aio_bh_schedule_oneshot(qemu_get_aio_context(), dummy_bh_cb, NULL); } } + +typedef struct { + AioWait wait; + bool done; + QEMUBHFunc *cb; + void *opaque; +} AioWaitBHData; + +/* Context: BH in IOThread */ +static void aio_wait_bh(void *opaque) +{ + AioWaitBHData *data =3D opaque; + + data->cb(data->opaque); + + data->done =3D true; + aio_wait_kick(&data->wait); +} + +void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque) +{ + AioWaitBHData data =3D { + .cb =3D cb, + .opaque =3D opaque, + }; + + assert(qemu_get_current_aio_context() =3D=3D qemu_get_aio_context()); + + aio_bh_schedule_oneshot(ctx, aio_wait_bh, &data); + AIO_WAIT_WHILE(&data.wait, ctx, !data.done); +} --=20 2.14.3 From nobody Thu May 2 06:20:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 152060182946580.39859201596903; Fri, 9 Mar 2018 05:23:49 -0800 (PST) Received: from localhost ([::1]:45246 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHzp-0003T6-46 for importer@patchew.org; Fri, 09 Mar 2018 08:23:41 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45465) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHwb-0001Gm-PE for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1euHwZ-0007d8-IS for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:21 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53010 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1euHwP-0007Sq-LS; Fri, 09 Mar 2018 08:20:09 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3D982EBFE2; Fri, 9 Mar 2018 13:20:09 +0000 (UTC) Received: from localhost (ovpn-116-226.ams2.redhat.com [10.36.116.226]) by smtp.corp.redhat.com (Postfix) with ESMTP id E0794202322A; Fri, 9 Mar 2018 13:20:08 +0000 (UTC) From: Stefan Hajnoczi To: Date: Fri, 9 Mar 2018 13:19:47 +0000 Message-Id: <20180309131949.18640-6-stefanha@redhat.com> In-Reply-To: <20180309131949.18640-1-stefanha@redhat.com> References: <20180309131949.18640-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 09 Mar 2018 13:20:09 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Fri, 09 Mar 2018 13:20:09 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'stefanha@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 5/7] virtio-blk: fix race between .ioeventfd_stop() and vq handler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , qemu-block@nongnu.org, Peter Crosthwaite , "Michael S. Tsirkin" , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If the main loop thread invokes .ioeventfd_stop() just as the vq handler function begins in the IOThread then the handler may lose the race for the AioContext lock. By the time the vq handler is able to acquire the AioContext lock the ioeventfd has already been removed and the handler isn't supposed to run anymore! Use the new aio_wait_bh_oneshot() function to perform ioeventfd removal from within the IOThread. This way no races with the vq handler are possible. Signed-off-by: Stefan Hajnoczi Reviewed-by: Fam Zheng Acked-by: Paolo Bonzini Message-id: 20180307144205.20619-3-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- hw/block/dataplane/virtio-blk.c | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-bl= k.c index c46253a924..101f32cf66 100644 --- a/hw/block/dataplane/virtio-blk.c +++ b/hw/block/dataplane/virtio-blk.c @@ -240,6 +240,22 @@ int virtio_blk_data_plane_start(VirtIODevice *vdev) return -ENOSYS; } =20 +/* Stop notifications for new requests from guest. + * + * Context: BH in IOThread + */ +static void virtio_blk_data_plane_stop_bh(void *opaque) +{ + VirtIOBlockDataPlane *s =3D opaque; + unsigned i; + + for (i =3D 0; i < s->conf->num_queues; i++) { + VirtQueue *vq =3D virtio_get_queue(s->vdev, i); + + virtio_queue_aio_set_host_notifier_handler(vq, s->ctx, NULL); + } +} + /* Context: QEMU global mutex held */ void virtio_blk_data_plane_stop(VirtIODevice *vdev) { @@ -264,13 +280,7 @@ void virtio_blk_data_plane_stop(VirtIODevice *vdev) trace_virtio_blk_data_plane_stop(s); =20 aio_context_acquire(s->ctx); - - /* Stop notifications for new requests from guest */ - for (i =3D 0; i < nvqs; i++) { - VirtQueue *vq =3D virtio_get_queue(s->vdev, i); - - virtio_queue_aio_set_host_notifier_handler(vq, s->ctx, NULL); - } + aio_wait_bh_oneshot(s->ctx, virtio_blk_data_plane_stop_bh, s); =20 /* Drain and switch bs back to the QEMU main loop */ blk_set_aio_context(s->conf->conf.blk, qemu_get_aio_context()); --=20 2.14.3 From nobody Thu May 2 06:20:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1520601899900196.73735895449238; Fri, 9 Mar 2018 05:24:59 -0800 (PST) Received: from localhost ([::1]:45261 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euI15-0004jg-4Q for importer@patchew.org; Fri, 09 Mar 2018 08:24:59 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45408) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHwU-00019i-IX for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1euHwT-0007WA-KQ for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:14 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:37376 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1euHwR-0007U8-Af; Fri, 09 Mar 2018 08:20:11 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DAE814068032; Fri, 9 Mar 2018 13:20:10 +0000 (UTC) Received: from localhost (ovpn-116-226.ams2.redhat.com [10.36.116.226]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6F46F215CDAC; Fri, 9 Mar 2018 13:20:10 +0000 (UTC) From: Stefan Hajnoczi To: Date: Fri, 9 Mar 2018 13:19:48 +0000 Message-Id: <20180309131949.18640-7-stefanha@redhat.com> In-Reply-To: <20180309131949.18640-1-stefanha@redhat.com> References: <20180309131949.18640-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 09 Mar 2018 13:20:10 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Fri, 09 Mar 2018 13:20:10 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'stefanha@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 6/7] virtio-scsi: fix race between .ioeventfd_stop() and vq handler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , qemu-block@nongnu.org, Peter Crosthwaite , "Michael S. Tsirkin" , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" If the main loop thread invokes .ioeventfd_stop() just as the vq handler function begins in the IOThread then the handler may lose the race for the AioContext lock. By the time the vq handler is able to acquire the AioContext lock the ioeventfd has already been removed and the handler isn't supposed to run anymore! Use the new aio_wait_bh_oneshot() function to perform ioeventfd removal from within the IOThread. This way no races with the vq handler are possible. Signed-off-by: Stefan Hajnoczi Reviewed-by: Fam Zheng Acked-by: Paolo Bonzini Message-id: 20180307144205.20619-4-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- hw/scsi/virtio-scsi-dataplane.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplan= e.c index 1c33322ba6..912e5005d8 100644 --- a/hw/scsi/virtio-scsi-dataplane.c +++ b/hw/scsi/virtio-scsi-dataplane.c @@ -107,9 +107,10 @@ static int virtio_scsi_vring_init(VirtIOSCSI *s, VirtQ= ueue *vq, int n, return 0; } =20 -/* assumes s->ctx held */ -static void virtio_scsi_clear_aio(VirtIOSCSI *s) +/* Context: BH in IOThread */ +static void virtio_scsi_dataplane_stop_bh(void *opaque) { + VirtIOSCSI *s =3D opaque; VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(s); int i; =20 @@ -171,7 +172,7 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev) return 0; =20 fail_vrings: - virtio_scsi_clear_aio(s); + aio_wait_bh_oneshot(s->ctx, virtio_scsi_dataplane_stop_bh, s); aio_context_release(s->ctx); for (i =3D 0; i < vs->conf.num_queues + 2; i++) { virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), i, false); @@ -207,7 +208,7 @@ void virtio_scsi_dataplane_stop(VirtIODevice *vdev) s->dataplane_stopping =3D true; =20 aio_context_acquire(s->ctx); - virtio_scsi_clear_aio(s); + aio_wait_bh_oneshot(s->ctx, virtio_scsi_dataplane_stop_bh, s); aio_context_release(s->ctx); =20 blk_drain_all(); /* ensure there are no in-flight requests */ --=20 2.14.3 From nobody Thu May 2 06:20:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 15206021592951009.6314578500428; Fri, 9 Mar 2018 05:29:19 -0800 (PST) Received: from localhost ([::1]:45285 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euI5G-0007lv-Gv for importer@patchew.org; Fri, 09 Mar 2018 08:29:18 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45434) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1euHwX-0001Cz-8C for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1euHwW-0007XK-0H for qemu-devel@nongnu.org; Fri, 09 Mar 2018 08:20:17 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:37626 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1euHwT-0007VV-1C; Fri, 09 Mar 2018 08:20:13 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A0AD2412934D; Fri, 9 Mar 2018 13:20:12 +0000 (UTC) Received: from localhost (ovpn-116-226.ams2.redhat.com [10.36.116.226]) by smtp.corp.redhat.com (Postfix) with ESMTP id 48E77202322A; Fri, 9 Mar 2018 13:20:12 +0000 (UTC) From: Stefan Hajnoczi To: Date: Fri, 9 Mar 2018 13:19:49 +0000 Message-Id: <20180309131949.18640-8-stefanha@redhat.com> In-Reply-To: <20180309131949.18640-1-stefanha@redhat.com> References: <20180309131949.18640-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 09 Mar 2018 13:20:12 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Fri, 09 Mar 2018 13:20:12 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'stefanha@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PULL 7/7] vl: introduce vm_shutdown() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Fam Zheng , qemu-block@nongnu.org, Peter Crosthwaite , "Michael S. Tsirkin" , Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Commit 00d09fdbbae5f7864ce754913efc84c12fdf9f1a ("vl: pause vcpus before stopping iothreads") and commit dce8921b2baaf95974af8176406881872067adfa ("iothread: Stop threads before main() quits") tried to work around the fact that emulation was still active during termination by stopping iothreads. They suffer from race conditions: 1. virtio_scsi_handle_cmd_vq() racing with iothread_stop_all() hits the virtio_scsi_ctx_check() assertion failure because the BDS AioContext has been modified by iothread_stop_all(). 2. Guest vq kick racing with main loop termination leaves a readable ioeventfd that is handled by the next aio_poll() when external clients are enabled again, resulting in unwanted emulation activity. This patch obsoletes those commits by fully disabling emulation activity when vcpus are stopped. Use the new vm_shutdown() function instead of pause_all_vcpus() so that vm change state handlers are invoked too. Virtio devices will now stop their ioeventfds, preventing further emulation activity after vm_stop(). Note that vm_stop(RUN_STATE_SHUTDOWN) cannot be used because it emits a QMP STOP event that may affect existing clients. It is no longer necessary to call replay_disable_events() directly since vm_shutdown() does so already. Drop iothread_stop_all() since it is no longer used. Cc: Fam Zheng Cc: Kevin Wolf Signed-off-by: Stefan Hajnoczi Reviewed-by: Fam Zheng Acked-by: Paolo Bonzini Message-id: 20180307144205.20619-5-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- include/sysemu/iothread.h | 1 - include/sysemu/sysemu.h | 1 + cpus.c | 16 +++++++++++++--- iothread.c | 31 ------------------------------- vl.c | 13 +++---------- 5 files changed, 17 insertions(+), 45 deletions(-) diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h index 799614ffd2..8a7ac2c528 100644 --- a/include/sysemu/iothread.h +++ b/include/sysemu/iothread.h @@ -45,7 +45,6 @@ typedef struct { char *iothread_get_id(IOThread *iothread); IOThread *iothread_by_id(const char *id); AioContext *iothread_get_aio_context(IOThread *iothread); -void iothread_stop_all(void); GMainContext *iothread_get_g_main_context(IOThread *iothread); =20 /* diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h index d24ad09f37..356bfdc1c1 100644 --- a/include/sysemu/sysemu.h +++ b/include/sysemu/sysemu.h @@ -56,6 +56,7 @@ void vm_start(void); int vm_prepare_start(void); int vm_stop(RunState state); int vm_stop_force_state(RunState state); +int vm_shutdown(void); =20 typedef enum WakeupReason { /* Always keep QEMU_WAKEUP_REASON_NONE =3D 0 */ diff --git a/cpus.c b/cpus.c index 9bcff7d63c..d8fe90eafe 100644 --- a/cpus.c +++ b/cpus.c @@ -993,7 +993,7 @@ void cpu_synchronize_all_pre_loadvm(void) } } =20 -static int do_vm_stop(RunState state) +static int do_vm_stop(RunState state, bool send_stop) { int ret =3D 0; =20 @@ -1002,7 +1002,9 @@ static int do_vm_stop(RunState state) pause_all_vcpus(); runstate_set(state); vm_state_notify(0, state); - qapi_event_send_stop(&error_abort); + if (send_stop) { + qapi_event_send_stop(&error_abort); + } } =20 bdrv_drain_all(); @@ -1012,6 +1014,14 @@ static int do_vm_stop(RunState state) return ret; } =20 +/* Special vm_stop() variant for terminating the process. Historically cl= ients + * did not expect a QMP STOP event and so we need to retain compatibility. + */ +int vm_shutdown(void) +{ + return do_vm_stop(RUN_STATE_SHUTDOWN, false); +} + static bool cpu_can_run(CPUState *cpu) { if (cpu->stop) { @@ -1994,7 +2004,7 @@ int vm_stop(RunState state) return 0; } =20 - return do_vm_stop(state); + return do_vm_stop(state, true); } =20 /** diff --git a/iothread.c b/iothread.c index 2ec5a3bffe..1b3463cb00 100644 --- a/iothread.c +++ b/iothread.c @@ -101,18 +101,6 @@ void iothread_stop(IOThread *iothread) qemu_thread_join(&iothread->thread); } =20 -static int iothread_stop_iter(Object *object, void *opaque) -{ - IOThread *iothread; - - iothread =3D (IOThread *)object_dynamic_cast(object, TYPE_IOTHREAD); - if (!iothread) { - return 0; - } - iothread_stop(iothread); - return 0; -} - static void iothread_instance_init(Object *obj) { IOThread *iothread =3D IOTHREAD(obj); @@ -333,25 +321,6 @@ IOThreadInfoList *qmp_query_iothreads(Error **errp) return head; } =20 -void iothread_stop_all(void) -{ - Object *container =3D object_get_objects_root(); - BlockDriverState *bs; - BdrvNextIterator it; - - for (bs =3D bdrv_first(&it); bs; bs =3D bdrv_next(&it)) { - AioContext *ctx =3D bdrv_get_aio_context(bs); - if (ctx =3D=3D qemu_get_aio_context()) { - continue; - } - aio_context_acquire(ctx); - bdrv_set_aio_context(bs, qemu_get_aio_context()); - aio_context_release(ctx); - } - - object_child_foreach(container, iothread_stop_iter, NULL); -} - static gpointer iothread_g_main_context_init(gpointer opaque) { AioContext *ctx; diff --git a/vl.c b/vl.c index dae986b352..3ef04ce991 100644 --- a/vl.c +++ b/vl.c @@ -4722,17 +4722,10 @@ int main(int argc, char **argv, char **envp) os_setup_post(); =20 main_loop(); - replay_disable_events(); =20 - /* The ordering of the following is delicate. Stop vcpus to prevent n= ew - * I/O requests being queued by the guest. Then stop IOThreads (this - * includes a drain operation and completes all request processing). = At - * this point emulated devices are still associated with their IOThrea= ds - * (if any) but no longer have any work to do. Only then can we close - * block devices safely because we know there is no more I/O coming. - */ - pause_all_vcpus(); - iothread_stop_all(); + /* No more vcpu or device emulation activity beyond this point */ + vm_shutdown(); + bdrv_close_all(); =20 res_free(); --=20 2.14.3