From nobody Sun May 5 08:23:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 15126777486540.3575666243506248; Thu, 7 Dec 2017 12:15:48 -0800 (PST) Received: from localhost ([::1]:34090 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2a6-0000GD-1E for importer@patchew.org; Thu, 07 Dec 2017 15:15:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46087) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2Y2-0007Mn-8K for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eN2Y1-0006hF-C7 for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:34 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58990) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eN2Xw-0006fS-3Y; Thu, 07 Dec 2017 15:13:28 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 50FD47970A; Thu, 7 Dec 2017 20:13:27 +0000 (UTC) Received: from localhost (ovpn-116-107.ams2.redhat.com [10.36.116.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id D963378493; Thu, 7 Dec 2017 20:13:26 +0000 (UTC) From: Stefan Hajnoczi To: Date: Thu, 7 Dec 2017 20:13:15 +0000 Message-Id: <20171207201320.19284-2-stefanha@redhat.com> In-Reply-To: <20171207201320.19284-1-stefanha@redhat.com> References: <20171207201320.19284-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 07 Dec 2017 20:13:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 1/6] block: avoid recursive AioContext acquire in bdrv_inactivate_all() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Paolo Bonzini , qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Paolo Bonzini BDRV_POLL_WHILE() does not support recursive AioContext locking. It only releases the AioContext lock once regardless of how many times the caller has acquired it. This results in a hang since the IOThread does not make progress while the AioContext is still locked. The following steps trigger the hang: $ qemu-system-x86_64 -M accel=3Dkvm -m 1G -cpu host \ -object iothread,id=3Diothread0 \ -device virtio-scsi-pci,iothread=3Diothread0 \ -drive if=3Dnone,id=3Ddrive0,file=3Dtest.img,format= =3Draw \ -device scsi-hd,drive=3Ddrive0 \ -drive if=3Dnone,id=3Ddrive1,file=3Dtest.img,format= =3Draw \ -device scsi-hd,drive=3Ddrive1 $ qemu-system-x86_64 ...same options... \ -incoming tcp::1234 (qemu) migrate tcp:127.0.0.1:1234 ...hang... Tested-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Reviewed-by: Eric Blake Reviewed-by: Paolo Bonzini --- block.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/block.c b/block.c index 9a1a0d1e73..1c37ce4554 100644 --- a/block.c +++ b/block.c @@ -4320,9 +4320,15 @@ int bdrv_inactivate_all(void) BdrvNextIterator it; int ret =3D 0; int pass; + GSList *aio_ctxs =3D NULL, *ctx; =20 for (bs =3D bdrv_first(&it); bs; bs =3D bdrv_next(&it)) { - aio_context_acquire(bdrv_get_aio_context(bs)); + AioContext *aio_context =3D bdrv_get_aio_context(bs); + + if (!g_slist_find(aio_ctxs, aio_context)) { + aio_ctxs =3D g_slist_prepend(aio_ctxs, aio_context); + aio_context_acquire(aio_context); + } } =20 /* We do two passes of inactivation. The first pass calls to drivers' @@ -4340,9 +4346,11 @@ int bdrv_inactivate_all(void) } =20 out: - for (bs =3D bdrv_first(&it); bs; bs =3D bdrv_next(&it)) { - aio_context_release(bdrv_get_aio_context(bs)); + for (ctx =3D aio_ctxs; ctx !=3D NULL; ctx =3D ctx->next) { + AioContext *aio_context =3D ctx->data; + aio_context_release(aio_context); } + g_slist_free(aio_ctxs); =20 return ret; } --=20 2.14.3 From nobody Sun May 5 08:23:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1512677755441118.10553371246874; Thu, 7 Dec 2017 12:15:55 -0800 (PST) Received: from localhost ([::1]:34092 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2a7-0000Iu-Le for importer@patchew.org; Thu, 07 Dec 2017 15:15:43 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46112) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2Y3-0007O0-Ey for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eN2Y2-0006iK-IJ for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:38288) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eN2Y0-0006gR-EI; Thu, 07 Dec 2017 15:13:32 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ABB355F7A9; Thu, 7 Dec 2017 20:13:31 +0000 (UTC) Received: from localhost (ovpn-116-107.ams2.redhat.com [10.36.116.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 62CE970105; Thu, 7 Dec 2017 20:13:28 +0000 (UTC) From: Stefan Hajnoczi To: Date: Thu, 7 Dec 2017 20:13:16 +0000 Message-Id: <20171207201320.19284-3-stefanha@redhat.com> In-Reply-To: <20171207201320.19284-1-stefanha@redhat.com> References: <20171207201320.19284-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 07 Dec 2017 20:13:31 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 2/6] docs: mark nested AioContext locking as a legacy API X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Paolo Bonzini , Stefan Hajnoczi , qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" See the patch for why nested AioContext locking is no longer allowed. Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini --- docs/devel/multiple-iothreads.txt | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/devel/multiple-iothreads.txt b/docs/devel/multiple-iothre= ads.txt index e4d340bbb7..4f9012d154 100644 --- a/docs/devel/multiple-iothreads.txt +++ b/docs/devel/multiple-iothreads.txt @@ -1,4 +1,4 @@ -Copyright (c) 2014 Red Hat Inc. +Copyright (c) 2014-2017 Red Hat Inc. =20 This work is licensed under the terms of the GNU GPL, version 2 or later. = See the COPYING file in the top-level directory. @@ -92,8 +92,9 @@ aio_context_acquire()/aio_context_release() for mutual ex= clusion. Once the context is acquired no other thread can access it or run event loop iterat= ions in this AioContext. =20 -aio_context_acquire()/aio_context_release() calls may be nested. This -means you can call them if you're not sure whether #2 applies. +Legacy code sometimes nests aio_context_acquire()/aio_context_release() ca= lls. +Do not use nesting anymore, it is incompatible with the BDRV_POLL_WHILE() = macro +used in the block layer and can lead to hangs. =20 There is currently no lock ordering rule if a thread needs to acquire mult= iple AioContexts simultaneously. Therefore, it is only safe for code holding t= he --=20 2.14.3 From nobody Sun May 5 08:23:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1512677757827418.4153075866624; Thu, 7 Dec 2017 12:15:57 -0800 (PST) Received: from localhost ([::1]:34095 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2aA-0000Kd-4Y for importer@patchew.org; Thu, 07 Dec 2017 15:15:46 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46134) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2Y5-0007R6-UU for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eN2Y5-0006mF-0s for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:37 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50446) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eN2Y2-0006hd-Ii; Thu, 07 Dec 2017 15:13:34 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B64BD80460; Thu, 7 Dec 2017 20:13:33 +0000 (UTC) Received: from localhost (ovpn-116-107.ams2.redhat.com [10.36.116.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2176860BE5; Thu, 7 Dec 2017 20:13:32 +0000 (UTC) From: Stefan Hajnoczi To: Date: Thu, 7 Dec 2017 20:13:17 +0000 Message-Id: <20171207201320.19284-4-stefanha@redhat.com> In-Reply-To: <20171207201320.19284-1-stefanha@redhat.com> References: <20171207201320.19284-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 07 Dec 2017 20:13:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 3/6] blockdev: add x-blockdev-set-iothread force boolean X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Paolo Bonzini , Stefan Hajnoczi , qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When a node is already associated with a BlockBackend the x-blockdev-set-iothread command refuses to set the IOThread. This is to prevent accidentally changing the IOThread when the nodes are in use. When the nodes are created with -drive they automatically get a BlockBackend. In that case we know nothing is using them yet and it's safe to set the IOThread. Add a force boolean to override the check. Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini --- qapi/block-core.json | 6 +++++- blockdev.c | 11 ++++++----- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 741d6c4367..a8cdbc300b 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3962,6 +3962,9 @@ # # @iothread: the name of the IOThread object or null for the main loop # +# @force: true if the node and its children should be moved when a BlockBa= ckend +# is already attached +# # Note: this command is experimental and intended for test cases that need # control over IOThreads only. # @@ -3984,4 +3987,5 @@ ## { 'command': 'x-blockdev-set-iothread', 'data' : { 'node-name': 'str', - 'iothread': 'StrOrNull' } } + 'iothread': 'StrOrNull', + '*force': 'bool' } } diff --git a/blockdev.c b/blockdev.c index f75c01f664..9c3a430cfb 100644 --- a/blockdev.c +++ b/blockdev.c @@ -4131,7 +4131,7 @@ BlockJobInfoList *qmp_query_block_jobs(Error **errp) } =20 void qmp_x_blockdev_set_iothread(const char *node_name, StrOrNull *iothrea= d, - Error **errp) + bool has_force, bool force, Error **errp) { AioContext *old_context; AioContext *new_context; @@ -4143,10 +4143,11 @@ void qmp_x_blockdev_set_iothread(const char *node_n= ame, StrOrNull *iothread, return; } =20 - /* If we want to allow more extreme test scenarios this guard could be - * removed. For now it protects against accidents. */ - if (bdrv_has_blk(bs)) { - error_setg(errp, "Node %s is in use", node_name); + /* Protects against accidents. */ + if (!(has_force && force) && bdrv_has_blk(bs)) { + error_setg(errp, "Node %s is associated with a BlockBackend and co= uld " + "be in use (use force=3Dtrue to override this che= ck)", + node_name); return; } =20 --=20 2.14.3 From nobody Sun May 5 08:23:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1512677941963545.0511879396519; Thu, 7 Dec 2017 12:19:01 -0800 (PST) Received: from localhost ([::1]:34112 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2d8-00039J-Ca for importer@patchew.org; Thu, 07 Dec 2017 15:18:50 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46196) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2YB-0007Vt-JB for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eN2YA-0006sD-PN for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:43 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50496) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eN2Y6-0006md-3b; Thu, 07 Dec 2017 15:13:38 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5841E80491; Thu, 7 Dec 2017 20:13:37 +0000 (UTC) Received: from localhost (ovpn-116-107.ams2.redhat.com [10.36.116.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 13A4D7C57B; Thu, 7 Dec 2017 20:13:34 +0000 (UTC) From: Stefan Hajnoczi To: Date: Thu, 7 Dec 2017 20:13:18 +0000 Message-Id: <20171207201320.19284-5-stefanha@redhat.com> In-Reply-To: <20171207201320.19284-1-stefanha@redhat.com> References: <20171207201320.19284-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 07 Dec 2017 20:13:37 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 4/6] iotests: add VM.add_object() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Paolo Bonzini , Stefan Hajnoczi , qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The VM.add_object() method can be used to add IOThreads or memory backend objects. Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini --- tests/qemu-iotests/iotests.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6f057904a9..44477e9295 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -197,6 +197,11 @@ class VM(qtest.QEMUQtestMachine): socket_scm_helper=3Dsocket_scm_helper) self._num_drives =3D 0 =20 + def add_object(self, opts): + self._args.append('-object') + self._args.append(opts) + return self + def add_device(self, opts): self._args.append('-device') self._args.append(opts) --=20 2.14.3 From nobody Sun May 5 08:23:26 2024 Delivered-To: importer@patchew.org Received-SPF: temperror (zoho.com: Error in retrieving data from DNS) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=temperror (zoho.com: Error in retrieving data from DNS) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1512677949967318.66256884304335; Thu, 7 Dec 2017 12:19:09 -0800 (PST) Received: from localhost ([::1]:34114 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2dB-0003U3-Mb for importer@patchew.org; Thu, 07 Dec 2017 15:18:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46245) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2YE-0007YS-0A for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eN2YC-0006ui-T8 for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:45 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50550) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eN2Y9-0006q5-Qw; Thu, 07 Dec 2017 15:13:42 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0A0E57EA91; Thu, 7 Dec 2017 20:13:41 +0000 (UTC) Received: from localhost (ovpn-116-107.ams2.redhat.com [10.36.116.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id A8BB35D9C8; Thu, 7 Dec 2017 20:13:39 +0000 (UTC) From: Stefan Hajnoczi To: Date: Thu, 7 Dec 2017 20:13:19 +0000 Message-Id: <20171207201320.19284-6-stefanha@redhat.com> In-Reply-To: <20171207201320.19284-1-stefanha@redhat.com> References: <20171207201320.19284-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 07 Dec 2017 20:13:41 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 5/6] iothread: fix iothread_stop() race condition X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Paolo Bonzini , Stefan Hajnoczi , qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_6 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" There is a small chance that iothread_stop() hangs as follows: Thread 3 (Thread 0x7f63eba5f700 (LWP 16105)): #0 0x00007f64012c09b6 in ppoll () at /lib64/libc.so.6 #1 0x000055959992eac9 in ppoll (__ss=3D0x0, __timeout=3D0x0, __nfds=3D, __fds=3D) at /usr/include/bits/poll2.h:77 #2 0x000055959992eac9 in qemu_poll_ns (fds=3D, nfds=3D, timeout=3D) at util/qemu-timer.c:322 #3 0x0000559599930711 in aio_poll (ctx=3D0x55959bdb83c0, blocking=3Dbloc= king@entry=3Dtrue) at util/aio-posix.c:629 #4 0x00005595996806fe in iothread_run (opaque=3D0x55959bd78400) at iothr= ead.c:59 #5 0x00007f640159f609 in start_thread () at /lib64/libpthread.so.0 #6 0x00007f64012cce6f in clone () at /lib64/libc.so.6 Thread 1 (Thread 0x7f640b45b280 (LWP 16103)): #0 0x00007f64015a0b6d in pthread_join () at /lib64/libpthread.so.0 #1 0x00005595999332ef in qemu_thread_join (thread=3D) at = util/qemu-thread-posix.c:547 #2 0x00005595996808ae in iothread_stop (iothread=3D) at i= othread.c:91 #3 0x000055959968094d in iothread_stop_iter (object=3D, o= paque=3D) at iothread.c:102 #4 0x0000559599857d97 in do_object_child_foreach (obj=3Dobj@entry=3D0x55= 959bdb8100, fn=3Dfn@entry=3D0x559599680930 , opaque=3Do= paque@entry=3D0x0, recurse=3Drecurse@entry=3Dfalse) at qom/object.c:852 #5 0x0000559599859477 in object_child_foreach (obj=3Dobj@entry=3D0x55959= bdb8100, fn=3Dfn@entry=3D0x559599680930 , opaque=3Dopaq= ue@entry=3D0x0) at qom/object.c:867 #6 0x0000559599680a6e in iothread_stop_all () at iothread.c:341 #7 0x000055959955b1d5 in main (argc=3D, argv=3D, envp=3D) at vl.c:4913 The relevant code from iothread_run() is: while (!atomic_read(&iothread->stopping)) { aio_poll(iothread->ctx, true); and iothread_stop(): iothread->stopping =3D true; aio_notify(iothread->ctx); ... qemu_thread_join(&iothread->thread); The following scenario can occur: 1. IOThread: while (!atomic_read(&iothread->stopping)) -> stopping=3Dfalse 2. Main loop: iothread->stopping =3D true; aio_notify(iothread->ctx); 3. IOThread: aio_poll(iothread->ctx, true); -> hang The bug is explained by the AioContext->notify_me doc comments: "If this field is 0, everything (file descriptors, bottom halves, timers) will be re-evaluated before the next blocking poll(), thus the event_notifier_set call can be skipped." The problem is that "everything" does not include checking iothread->stopping. This means iothread_run() will block in aio_poll() if aio_notify() was called just before aio_poll(). This patch fixes the hang by replacing aio_notify() with aio_bh_schedule_oneshot(). This makes aio_poll() or g_main_loop_run() to return. Implementing this properly required a new bool running flag. The new flag prevents races that are tricky if we try to use iothread->stopping. Now iothread->stopping is purely for iothread_stop() and iothread->running is purely for the iothread_run() thread. Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini --- I'm including this patch in this series because it is needed to make the test case reliable. It's unrelated to the main goal of the patch series. --- include/sysemu/iothread.h | 3 ++- iothread.c | 20 +++++++++++++++----- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/include/sysemu/iothread.h b/include/sysemu/iothread.h index 55de1715c7..799614ffd2 100644 --- a/include/sysemu/iothread.h +++ b/include/sysemu/iothread.h @@ -29,7 +29,8 @@ typedef struct { GOnce once; QemuMutex init_done_lock; QemuCond init_done_cond; /* is thread initialization done? */ - bool stopping; + bool stopping; /* has iothread_stop() been called? */ + bool running; /* should iothread_run() continue? */ int thread_id; =20 /* AioContext poll parameters */ diff --git a/iothread.c b/iothread.c index e7b93e02a3..d8b6c1fb27 100644 --- a/iothread.c +++ b/iothread.c @@ -55,7 +55,7 @@ static void *iothread_run(void *opaque) qemu_cond_signal(&iothread->init_done_cond); qemu_mutex_unlock(&iothread->init_done_lock); =20 - while (!atomic_read(&iothread->stopping)) { + while (iothread->running) { aio_poll(iothread->ctx, true); =20 if (atomic_read(&iothread->worker_context)) { @@ -78,16 +78,25 @@ static void *iothread_run(void *opaque) return NULL; } =20 +/* Runs in iothread_run() thread */ +static void iothread_stop_bh(void *opaque) +{ + IOThread *iothread =3D opaque; + + iothread->running =3D false; /* stop iothread_run() */ + + if (iothread->main_loop) { + g_main_loop_quit(iothread->main_loop); + } +} + void iothread_stop(IOThread *iothread) { if (!iothread->ctx || iothread->stopping) { return; } iothread->stopping =3D true; - aio_notify(iothread->ctx); - if (atomic_read(&iothread->main_loop)) { - g_main_loop_quit(iothread->main_loop); - } + aio_bh_schedule_oneshot(iothread->ctx, iothread_stop_bh, iothread); qemu_thread_join(&iothread->thread); } =20 @@ -134,6 +143,7 @@ static void iothread_complete(UserCreatable *obj, Error= **errp) char *name, *thread_name; =20 iothread->stopping =3D false; + iothread->running =3D true; iothread->thread_id =3D -1; iothread->ctx =3D aio_context_new(&local_error); if (!iothread->ctx) { --=20 2.14.3 From nobody Sun May 5 08:23:26 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (208.118.235.17 [208.118.235.17]) by mx.zohomail.com with SMTPS id 1512677931857669.5653558401989; Thu, 7 Dec 2017 12:18:51 -0800 (PST) Received: from localhost ([::1]:34111 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2d3-0002vn-CU for importer@patchew.org; Thu, 07 Dec 2017 15:18:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46331) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eN2YI-0007dY-NG for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eN2YF-0006yB-II for qemu-devel@nongnu.org; Thu, 07 Dec 2017 15:13:50 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58072) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eN2YC-0006uM-Kd; Thu, 07 Dec 2017 15:13:44 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DD353C04575A; Thu, 7 Dec 2017 20:13:43 +0000 (UTC) Received: from localhost (ovpn-116-107.ams2.redhat.com [10.36.116.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6A4D07B90A; Thu, 7 Dec 2017 20:13:43 +0000 (UTC) From: Stefan Hajnoczi To: Date: Thu, 7 Dec 2017 20:13:20 +0000 Message-Id: <20171207201320.19284-7-stefanha@redhat.com> In-Reply-To: <20171207201320.19284-1-stefanha@redhat.com> References: <20171207201320.19284-1-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 07 Dec 2017 20:13:43 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 6/6] qemu-iotests: add 203 savevm with IOThreads test X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Paolo Bonzini , Stefan Hajnoczi , qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This test case will prevent future regressions with savevm and IOThreads. Signed-off-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini --- tests/qemu-iotests/203 | 59 ++++++++++++++++++++++++++++++++++++++++++= ++++ tests/qemu-iotests/203.out | 6 +++++ tests/qemu-iotests/group | 1 + 3 files changed, 66 insertions(+) create mode 100755 tests/qemu-iotests/203 create mode 100644 tests/qemu-iotests/203.out diff --git a/tests/qemu-iotests/203 b/tests/qemu-iotests/203 new file mode 100755 index 0000000000..2c811917d8 --- /dev/null +++ b/tests/qemu-iotests/203 @@ -0,0 +1,59 @@ +#!/usr/bin/env python +# +# Copyright (C) 2017 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Creator/Owner: Stefan Hajnoczi +# +# Check that QMP 'migrate' with multiple drives on a single IOThread compl= etes +# successfully. This particular command triggered a hang in the source QE= MU +# process due to recursive AioContext locking in bdrv_invalidate_all() and +# BDRV_POLL_WHILE(). + +import iotests + +iotests.verify_image_format(supported_fmts=3D['qcow2']) +iotests.verify_platform(['linux']) + +with iotests.FilePath('disk0.img') as disk0_img_path, \ + iotests.FilePath('disk1.img') as disk1_img_path, \ + iotests.VM() as vm: + + img_size =3D '10M' + iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk0_img_path, = img_size) + iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk1_img_path, = img_size) + + iotests.log('Launching VM...') + (vm.add_object('iothread,id=3Diothread0') + .add_drive(disk0_img_path, 'node-name=3Ddrive0-node', interface=3D'= none') + .add_drive(disk1_img_path, 'node-name=3Ddrive1-node', interface=3D'= none') + .launch()) + + iotests.log('Setting IOThreads...') + iotests.log(vm.qmp('x-blockdev-set-iothread', + node_name=3D'drive0-node', iothread=3D'iothread0', + force=3DTrue)) + iotests.log(vm.qmp('x-blockdev-set-iothread', + node_name=3D'drive1-node', iothread=3D'iothread0', + force=3DTrue)) + + iotests.log('Starting migration...') + iotests.log(vm.qmp('migrate', uri=3D'exec:cat >/dev/null')) + while True: + vm.get_qmp_event(wait=3D60.0) + result =3D vm.qmp('query-migrate') + status =3D result.get('return', {}).get('status', None) + if status =3D=3D 'completed': + break diff --git a/tests/qemu-iotests/203.out b/tests/qemu-iotests/203.out new file mode 100644 index 0000000000..3f1ff900e4 --- /dev/null +++ b/tests/qemu-iotests/203.out @@ -0,0 +1,6 @@ +Launching VM... +Setting IOThreads... +{u'return': {}} +{u'return': {}} +Starting migration... +{u'return': {}} diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index d0ee1e2e55..93d96fb22f 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -198,3 +198,4 @@ 198 rw auto 200 rw auto 202 rw auto quick +203 rw auto --=20 2.14.3