From nobody Sun May 5 01:56:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1558631353; cv=none; d=zoho.com; s=zohoarc; b=DqaKvjZCWyNH/pY9f0CLAgsKWTiBcobYxYAye7WmKctPPjBW3ek4ASC7gOPTTdlYKLKgykzfef/nkUQ65v86pit3ADLknkNQWqJ8ChW+kCYkmNvKePN6zarKo/F3DPFafqK7QToKbrS2KIHb0SJHZKKPRSBiVoSZW3ggC0Cm6cE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558631353; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=gYIEabNUtMuo2MjtZ7m4scka5enbMhpqaYYCVVPcGA4=; b=NMydJATGo/PLIoKO58TY2tSCWXaWBhVDIhY5Jw8XabVDkcpB0SiUlL7zkW0pbZNw+Tyi9NmWDcYdTv5YBD9Mtytb1iz7KkuOaqXYOvsTvFzLHgo5PMEv6rds+/XPpgbpEnNHe+bOlnhJZmibMzSgWJ2PVYntl+8HD6kytOtabqQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558631353024402.2692109434248; Thu, 23 May 2019 10:09:13 -0700 (PDT) Received: from localhost ([127.0.0.1]:40840 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrDA-0004rp-93 for importer@patchew.org; Thu, 23 May 2019 13:09:00 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49203) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003jv-Ao for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBF-0001fp-4a for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:06 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44776) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB5-0001Xn-9b; Thu, 23 May 2019 13:06:52 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B63E03001A77; Thu, 23 May 2019 17:06:49 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id C31D210027C5; Thu, 23 May 2019 17:06:48 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:39 -0400 Message-Id: <20190523170643.20794-2-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 23 May 2019 17:06:49 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 1/5] blockdev-backup: don't check aio_context too early X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , aihua liang , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" in blockdev_backup_prepare, we check to make sure that the target is associated with a compatible aio context. However, do_blockdev_backup is called later and has some logic to move the target to a compatible aio_context. The transaction version will fail certain commands needlessly early as a result. Allow blockdev_backup_prepare to simply call do_blockdev_backup, which will ultimately decide if the contexts are compatible or not. Note: the transaction version has always disallowed this operation since its initial commit bd8baecd (2014), whereas the version of qmp_blockdev_backup at the time, from commit c29c1dd312f, tried to enforce the aio_context switch instead. It's not clear, and I can't see from the mailing list archives at the time, why the two functions take a different approach. It wasn't until later in efd7556708b (2016) that the standalone version tried to determine if it could set the context or not. Reported-by: aihua liang Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=3D1683498 Signed-off-by: John Snow Reviewed-by: Max Reitz --- blockdev.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index e856ca4be9..01a48a2a5a 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1872,10 +1872,6 @@ static void blockdev_backup_prepare(BlkActionState *= common, Error **errp) } =20 aio_context =3D bdrv_get_aio_context(bs); - if (aio_context !=3D bdrv_get_aio_context(target)) { - error_setg(errp, "Backup between two IO threads is not implemented= "); - return; - } aio_context_acquire(aio_context); state->bs =3D bs; =20 --=20 2.20.1 From nobody Sun May 5 01:56:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1558631479; cv=none; d=zoho.com; s=zohoarc; b=A0b2UCKE2x6d6ET+/+sBSz35XBGRMuY2Hb+A3doLHESdbmiAoyOAVqqBGgcwiO/VRCo6I5TUO42jibzVOIpBHWXLV3CaTCaO8UmCMO0KSJg8srGEObRC153SLgvzxgYhLERGwZwvGntKM8BPJTTXu1EpPBMmbH5K1Mb7viKxXis= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558631479; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=eIeN7ECqHLbV/bezdZ4/fVeeKUcke+BoEpijzZgwioE=; b=eX1akPm0sbuDnodybeLIwKRxv+yctLah8bM2BcxWqWSLevK5ExYB9GV4pmWQhromoncvvgqFc34xz9BxqtPWibopu3w709BQKB8mF4lH0PMyK0LIFoZsekE/YtHqFS4rbgxe190Ft9LieMIgr+bbQQxRR3cjxdzdEXEa6vfZRZo= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558631479474771.8823722297537; Thu, 23 May 2019 10:11:19 -0700 (PDT) Received: from localhost ([127.0.0.1]:40905 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrFG-0006aV-El for importer@patchew.org; Thu, 23 May 2019 13:11:10 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49198) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003js-9K for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBG-0001iC-S9 for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54824) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB7-0001Zu-Qa; Thu, 23 May 2019 13:06:54 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B7815330260; Thu, 23 May 2019 17:06:50 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D800B1001DEF; Thu, 23 May 2019 17:06:49 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:40 -0400 Message-Id: <20190523170643.20794-3-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 23 May 2019 17:06:50 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 2/5] iotests.py: do not use infinite waits X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Cap waits to 60 seconds so that iotests can fail gracefully if something goes wrong. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6bcddd8870..6e17c040dc 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -521,7 +521,7 @@ class VM(qtest.QEMUQtestMachine): output_list +=3D [key + '=3D' + obj[key]] return ','.join(output_list) =20 - def get_qmp_events_filtered(self, wait=3DTrue): + def get_qmp_events_filtered(self, wait=3D60.0): result =3D [] for ev in self.get_qmp_events(wait=3Dwait): result.append(filter_qmp_event(ev)) @@ -539,10 +539,10 @@ class VM(qtest.QEMUQtestMachine): =20 # Returns None on success, and an error string on failure def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse, - pre_finalize=3DNone): + pre_finalize=3DNone, wait=3D60.0): error =3D None while True: - for ev in self.get_qmp_events_filtered(wait=3DTrue): + for ev in self.get_qmp_events_filtered(wait=3Dwait): if ev['event'] =3D=3D 'JOB_STATUS_CHANGE': status =3D ev['data']['status'] if status =3D=3D 'aborting': @@ -647,7 +647,7 @@ class QMPTestCase(unittest.TestCase): self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filena= me[5:])), self.vm.flatten_qmp_object(reference)) =20 - def cancel_and_wait(self, drive=3D'drive0', force=3DFalse, resume=3DFa= lse): + def cancel_and_wait(self, drive=3D'drive0', force=3DFalse, resume=3DFa= lse, wait=3D60.0): '''Cancel a block job and wait for it to finish, returning the eve= nt''' result =3D self.vm.qmp('block-job-cancel', device=3Ddrive, force= =3Dforce) self.assert_qmp(result, 'return', {}) @@ -658,7 +658,7 @@ class QMPTestCase(unittest.TestCase): cancelled =3D False result =3D None while not cancelled: - for event in self.vm.get_qmp_events(wait=3DTrue): + for event in self.vm.get_qmp_events(wait=3Dwait): if event['event'] =3D=3D 'BLOCK_JOB_COMPLETED' or \ event['event'] =3D=3D 'BLOCK_JOB_CANCELLED': self.assert_qmp(event, 'data/device', drive) @@ -671,10 +671,10 @@ class QMPTestCase(unittest.TestCase): self.assert_no_active_block_jobs() return result =20 - def wait_until_completed(self, drive=3D'drive0', check_offset=3DTrue): + def wait_until_completed(self, drive=3D'drive0', check_offset=3DTrue, = wait=3D60.0): '''Wait for a block job to finish, returning the event''' while True: - for event in self.vm.get_qmp_events(wait=3DTrue): + for event in self.vm.get_qmp_events(wait=3Dwait): if event['event'] =3D=3D 'BLOCK_JOB_COMPLETED': self.assert_qmp(event, 'data/device', drive) self.assert_qmp_absent(event, 'data/error') --=20 2.20.1 From nobody Sun May 5 01:56:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1558631572; cv=none; d=zoho.com; s=zohoarc; b=YrKBuLunX3vbg83DMDeCFODZntzlYFxzApPjd1rv4HfmxYfKhnXcTT1dSZ8+i3Lhlk9VgeBjNE41amvzMLzfj5fQ83Tad22eSu7siCa9QDnZkHSoAfcaVQvNhoSDhjRWqFbfeSatwbwwFGA9rI2SPM7Z1pFbdb4h2ItI796AxmA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558631572; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=xplmF3XRLCUkmxDx9aq14FkmMDPdY/I5Z0HQw5lrSgg=; b=CNaGOCymf9wO7k6TPE6zw426X1zI0gMQFSy/2q6vmiYbg3/2OZi9P3J2wAW9Sme1u3ZVCmqC8yIbCCCtor9U89tGVjNHwsWwoXyUlA+0P49To6G9H7/TorHQo0mt2J0VGdUn4jUZzDwOl6CVKeHZH5ID/T4k2Ke/wOdCu9XxjJA= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 155863157279712.922575707642977; Thu, 23 May 2019 10:12:52 -0700 (PDT) Received: from localhost ([127.0.0.1]:40917 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrGp-0007ZF-NI for importer@patchew.org; Thu, 23 May 2019 13:12:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49194) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003jo-8G for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBF-0001fm-4k for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33704) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB7-0001Zp-Um; Thu, 23 May 2019 13:06:54 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B50187FDFD; Thu, 23 May 2019 17:06:51 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D6C8B10027C5; Thu, 23 May 2019 17:06:50 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:41 -0400 Message-Id: <20190523170643.20794-4-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 23 May 2019 17:06:51 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 3/5] QEMUMachine: add events_wait method X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Instead of event_wait which looks for a single event, add an events_wait which can look for any number of events simultaneously. However, it will still only return one at a time, whichever happens first. Signed-off-by: John Snow Reviewed-by: Max Reitz --- python/qemu/__init__.py | 69 +++++++++++++++++++++++++++++------------ 1 file changed, 49 insertions(+), 20 deletions(-) diff --git a/python/qemu/__init__.py b/python/qemu/__init__.py index 81d9657ec0..98ed8a2e28 100644 --- a/python/qemu/__init__.py +++ b/python/qemu/__init__.py @@ -402,42 +402,71 @@ class QEMUMachine(object): self._qmp.clear_events() return events =20 - def event_wait(self, name, timeout=3D60.0, match=3DNone): + @staticmethod + def event_match(event, match=3DNone): """ - Wait for specified timeout on named event in QMP; optionally filter - results by match. + Check if an event matches optional match criteria. =20 - The 'match' is checked to be a recursive subset of the 'event'; sk= ips - branch processing on match's value None - {"foo": {"bar": 1}} matches {"foo": None} - {"foo": {"bar": 1}} does not matches {"foo": {"baz": None}} + The match criteria takes the form of a matching subdict. The event= is + checked to be a superset of the subdict, recursively, with matching + values whenever those values are not None. + + Examples, with the subdict queries on the left: + - None matches any object. + - {"foo": None} matches {"foo": {"bar": 1}} + - {"foo": {"baz": None}} does not match {"foo": {"bar": 1}} + - {"foo": {"baz": 2}} matches {"foo": {"bar": 1, "baz": 2}} """ - def event_match(event, match=3DNone): - if match is None: - return True + if match is None: + return True =20 - for key in match: - if key in event: - if isinstance(event[key], dict): - if not event_match(event[key], match[key]): - return False - elif event[key] !=3D match[key]: + for key in match: + if key in event: + if isinstance(event[key], dict): + if not QEMUMachine.event_match(event[key], match[key]): return False - else: + elif event[key] !=3D match[key]: return False + else: + return False + return True =20 - return True + def event_wait(self, name, timeout=3D60.0, match=3DNone): + """ + event_wait waits for and returns a named event from QMP with a tim= eout. + + name: The event to wait for. + timeout: QEMUMonitorProtocol.pull_event timeout parameter. + match: Optional match criteria. See event_match for details. + """ + return self.events_wait([(name, match)], timeout) + + def events_wait(self, events, timeout=3D60.0): + """ + events_wait waits for and returns a named event from QMP with a ti= meout. + + events: a sequence of (name, match_criteria) tuples. + The match criteria are optional and may be None. + See event_match for details. + timeout: QEMUMonitorProtocol.pull_event timeout parameter. + """ + def _match(event): + for name, match in events: + if (event['event'] =3D=3D name and + self.event_match(event, match)): + return True + return False =20 # Search cached events for event in self._events: - if (event['event'] =3D=3D name) and event_match(event, match): + if _match(event): self._events.remove(event) return event =20 # Poll for new events while True: event =3D self._qmp.pull_event(wait=3Dtimeout) - if (event['event'] =3D=3D name) and event_match(event, match): + if _match(event): return event self._events.append(event) =20 --=20 2.20.1 From nobody Sun May 5 01:56:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1558631353; cv=none; d=zoho.com; s=zohoarc; b=mQxaPyYCVOWuSMFS8d8mdKWdkD+8KhMZrzZ5/J3Ho67d3LyrLaKHM2Q4xB4q+6/04d36ETZFcf5iIj+HsZGA6eh4ZsKdFT19+WnbwHxkaYczdNh8mC94xwXgRoEeBJ6hWNO+ngIgkUZGdNiRQZD1ewbr8A8Md/cKe+8NNd9AlS4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558631353; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=Jw0q/mopmj49eI0QdtiFB6s6Q5jmS8gmDfbvzqwvO3M=; b=PQQr3kf2CivlnaOYSw13zubDUlpF/3SLMmBo1CovjB6HM928LHBnFcnwHVe+qGABqdXgIQkze0k0JLfwb6i2kZK0k2Sb+Ra84lhT+VEqqrc3Jj/KlURfE8S96JdjUV39vjyaqPVKIoU5vaTpag624+90BOWl2Pb80whLyPkJo18= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558631353967513.6475614483778; Thu, 23 May 2019 10:09:13 -0700 (PDT) Received: from localhost ([127.0.0.1]:40843 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrDB-0004tR-Pd for importer@patchew.org; Thu, 23 May 2019 13:09:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49193) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBK-0003jn-89 for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBF-0001fi-4s for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:04 -0400 Received: from mx1.redhat.com ([209.132.183.28]:42606) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrB7-0001Zh-JW; Thu, 23 May 2019 13:06:53 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B1F363002F44; Thu, 23 May 2019 17:06:52 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D46251001DEF; Thu, 23 May 2019 17:06:51 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:42 -0400 Message-Id: <20190523170643.20794-5-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Thu, 23 May 2019 17:06:52 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 4/5] iotests.py: rewrite run_job to be pickier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Don't pull events out of the queue that don't belong to us; be choosier so that we can use this method to drive jobs that were launched by transactions that may have more jobs. Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/iotests.py | 48 +++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6e17c040dc..dc77d3fba0 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -540,27 +540,37 @@ class VM(qtest.QEMUQtestMachine): # Returns None on success, and an error string on failure def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse, pre_finalize=3DNone, wait=3D60.0): + match_device =3D {'data': {'device': job}} + match_id =3D {'data': {'id': job}} + events =3D [ + ('BLOCK_JOB_COMPLETED', match_device), + ('BLOCK_JOB_CANCELLED', match_device), + ('BLOCK_JOB_ERROR', match_device), + ('BLOCK_JOB_READY', match_device), + ('BLOCK_JOB_PENDING', match_id), + ('JOB_STATUS_CHANGE', match_id) + ] error =3D None while True: - for ev in self.get_qmp_events_filtered(wait=3Dwait): - if ev['event'] =3D=3D 'JOB_STATUS_CHANGE': - status =3D ev['data']['status'] - if status =3D=3D 'aborting': - result =3D self.qmp('query-jobs') - for j in result['return']: - if j['id'] =3D=3D job: - error =3D j['error'] - log('Job failed: %s' % (j['error'])) - elif status =3D=3D 'pending' and not auto_finalize: - if pre_finalize: - pre_finalize() - self.qmp_log('job-finalize', id=3Djob) - elif status =3D=3D 'concluded' and not auto_dismiss: - self.qmp_log('job-dismiss', id=3Djob) - elif status =3D=3D 'null': - return error - else: - log(ev) + ev =3D filter_qmp_event(self.events_wait(events)) + if ev['event'] !=3D 'JOB_STATUS_CHANGE': + log(ev) + continue + status =3D ev['data']['status'] + if status =3D=3D 'aborting': + result =3D self.qmp('query-jobs') + for j in result['return']: + if j['id'] =3D=3D job: + error =3D j['error'] + log('Job failed: %s' % (j['error'])) + elif status =3D=3D 'pending' and not auto_finalize: + if pre_finalize: + pre_finalize() + self.qmp_log('job-finalize', id=3Djob) + elif status =3D=3D 'concluded' and not auto_dismiss: + self.qmp_log('job-dismiss', id=3Djob) + elif status =3D=3D 'null': + return error =20 def node_info(self, node_name): nodes =3D self.qmp('query-named-block-nodes') --=20 2.20.1 From nobody Sun May 5 01:56:20 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1558631659; cv=none; d=zoho.com; s=zohoarc; b=JrocZJBrsip+RSVQQMN3WLZISiUVXJW1HcET0JuWDu2ULw6hQATo5PTteKPBA9/M+I2dlhiJ5YMXVGpeKYoA+Z1IxM1cvY8x3SAIp79PcbllnZdLwQD0CeRHZva6LuCSFoSVy+yoi/q1NhGhQaYBEwBEGbeP/F02KP2CsRjZ1ro= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1558631659; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=hPffs5SH2M1IqiywOb7tNDSTq73ividlyiRG9EL6RII=; b=duhX1zZCvna63b13MMAbb3r0maKZb10QArK+Fcr7/vbaY5G3l6lGH6SpSMv2FCPWKEzVZykT6/Jn02583+GmZ0s7fDQq5LaxQ382n6LZO1B0hl2LtCYZz30BsmElW7ATBe0OIavdT8ugd7hmgce/HdNcYl8nvREvE7/Py1y/sf8= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (209.51.188.17 [209.51.188.17]) by mx.zohomail.com with SMTPS id 1558631659464544.2303733145723; Thu, 23 May 2019 10:14:19 -0700 (PDT) Received: from localhost ([127.0.0.1]:40925 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrIA-0008Vq-B0 for importer@patchew.org; Thu, 23 May 2019 13:14:10 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49287) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTrBO-0003oW-CN for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTrBM-0001mX-8O for qemu-devel@nongnu.org; Thu, 23 May 2019 13:07:10 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35118) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTrBD-0001aq-6K; Thu, 23 May 2019 13:06:59 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B15703079B76; Thu, 23 May 2019 17:06:53 +0000 (UTC) Received: from probe.bos.redhat.com (dhcp-17-187.bos.redhat.com [10.18.17.187]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4A371017E2E; Thu, 23 May 2019 17:06:52 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Thu, 23 May 2019 13:06:43 -0400 Message-Id: <20190523170643.20794-6-jsnow@redhat.com> In-Reply-To: <20190523170643.20794-1-jsnow@redhat.com> References: <20190523170643.20794-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Thu, 23 May 2019 17:06:53 +0000 (UTC) Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v3 5/5] iotests: add iotest 250 for testing blockdev-backup across iothread contexts X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-stable@nongnu.org, Markus Armbruster , Max Reitz , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow Reviewed-by: Max Reitz --- tests/qemu-iotests/250 | 122 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/250.out | 119 ++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 242 insertions(+) create mode 100755 tests/qemu-iotests/250 create mode 100644 tests/qemu-iotests/250.out diff --git a/tests/qemu-iotests/250 b/tests/qemu-iotests/250 new file mode 100755 index 0000000000..c594a43205 --- /dev/null +++ b/tests/qemu-iotests/250 @@ -0,0 +1,122 @@ +#!/usr/bin/env python +# +# Test incremental/backup across iothread contexts +# +# Copyright (c) 2019 John Snow for Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# owner=3Djsnow@redhat.com + +import os +import iotests +from iotests import log + +iotests.verify_image_format(supported_fmts=3D['qcow2']) +size =3D 64 * 1024 * 1024 + +with iotests.FilePath('img0') as img0_path, \ + iotests.FilePath('img1') as img1_path, \ + iotests.FilePath('img0-full') as img0_full_path, \ + iotests.FilePath('img1-full') as img1_full_path, \ + iotests.FilePath('img0-incr') as img0_incr_path, \ + iotests.FilePath('img1-incr') as img1_incr_path, \ + iotests.VM() as vm: + + def create_target(filepath, name, size): + basename =3D os.path.basename(filepath) + nodename =3D "file_{}".format(basename) + log(vm.command('blockdev-create', job_id=3D'job1', + options=3D{ + 'driver': 'file', + 'filename': filepath, + 'size': 0, + })) + vm.run_job('job1') + log(vm.command('blockdev-add', driver=3D'file', + node_name=3Dnodename, filename=3Dfilepath)) + log(vm.command('blockdev-create', job_id=3D'job2', + options=3D{ + 'driver': iotests.imgfmt, + 'file': nodename, + 'size': size, + })) + vm.run_job('job2') + log(vm.command('blockdev-add', driver=3Diotests.imgfmt, + node_name=3Dname, + file=3Dnodename)) + + log('--- Preparing images & VM ---\n') + vm.add_object('iothread,id=3Diothread0') + vm.add_object('iothread,id=3Diothread1') + vm.add_device('virtio-scsi-pci,id=3Dscsi0,iothread=3Diothread0') + vm.add_device('virtio-scsi-pci,id=3Dscsi1,iothread=3Diothread1') + iotests.qemu_img_create('-f', iotests.imgfmt, img0_path, str(size)) + iotests.qemu_img_create('-f', iotests.imgfmt, img1_path, str(size)) + vm.add_drive(img0_path, interface=3D'none') + vm.add_device('scsi-hd,id=3Ddevice0,drive=3Ddrive0,bus=3Dscsi0.0') + vm.add_drive(img1_path, interface=3D'none') + vm.add_device('scsi-hd,id=3Ddevice1,drive=3Ddrive1,bus=3Dscsi1.0') + + log('--- Starting VM ---\n') + vm.launch() + + log('--- Create Targets & Full Backups ---\n') + create_target(img0_full_path, 'img0-full', size) + create_target(img1_full_path, 'img1-full', size) + ret =3D vm.qmp_log('transaction', indent=3D2, actions=3D[ + { 'type': 'block-dirty-bitmap-add', + 'data': { 'node': 'drive0', 'name': 'bitmap0' }}, + { 'type': 'block-dirty-bitmap-add', + 'data': { 'node': 'drive1', 'name': 'bitmap1' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive0', + 'target': 'img0-full', + 'sync': 'full', + 'job-id': 'j0' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive1', + 'target': 'img1-full', + 'sync': 'full', + 'job-id': 'j1' }} + ]) + if "error" in ret: + raise Exception(ret['error']['desc']) + vm.run_job('j0', auto_dismiss=3DTrue) + vm.run_job('j1', auto_dismiss=3DTrue) + + log('\n--- Create Targets & Incremental Backups ---\n') + create_target(img0_incr_path, 'img0-incr', size) + create_target(img1_incr_path, 'img1-incr', size) + ret =3D vm.qmp_log('transaction', indent=3D2, actions=3D[ + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive0', + 'target': 'img0-incr', + 'sync': 'incremental', + 'bitmap': 'bitmap0', + 'job-id': 'j2' }}, + { 'type': 'blockdev-backup', + 'data': { 'device': 'drive1', + 'target': 'img1-incr', + 'sync': 'incremental', + 'bitmap': 'bitmap1', + 'job-id': 'j3' }} + ]) + if "error" in ret: + raise Exception(ret['error']['desc']) + vm.run_job('j2', auto_dismiss=3DTrue) + vm.run_job('j3', auto_dismiss=3DTrue) + + log('\n--- Done ---') + vm.shutdown() diff --git a/tests/qemu-iotests/250.out b/tests/qemu-iotests/250.out new file mode 100644 index 0000000000..eec38614ec --- /dev/null +++ b/tests/qemu-iotests/250.out @@ -0,0 +1,119 @@ +--- Preparing images & VM --- + +--- Starting VM --- + +--- Create Targets & Full Backups --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "name": "bitmap0", + "node": "drive0" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "name": "bitmap1", + "node": "drive1" + }, + "type": "block-dirty-bitmap-add" + }, + { + "data": { + "device": "drive0", + "job-id": "j0", + "sync": "full", + "target": "img0-full" + }, + "type": "blockdev-backup" + }, + { + "data": { + "device": "drive1", + "job-id": "j1", + "sync": "full", + "target": "img1-full" + }, + "type": "blockdev-backup" + } + ] + } +} +{ + "return": {} +} +{"data": {"device": "j0", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} +{"data": {"device": "j1", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} + +--- Create Targets & Incremental Backups --- + +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job1"}} +{"return": {}} +{} +{} +{"execute": "job-dismiss", "arguments": {"id": "job2"}} +{"return": {}} +{} +{ + "execute": "transaction", + "arguments": { + "actions": [ + { + "data": { + "bitmap": "bitmap0", + "device": "drive0", + "job-id": "j2", + "sync": "incremental", + "target": "img0-incr" + }, + "type": "blockdev-backup" + }, + { + "data": { + "bitmap": "bitmap1", + "device": "drive1", + "job-id": "j3", + "sync": "incremental", + "target": "img1-incr" + }, + "type": "blockdev-backup" + } + ] + } +} +{ + "return": {} +} +{"data": {"device": "j2", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} +{"data": {"device": "j3", "len": 67108864, "offset": 67108864, "speed": 0,= "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microsec= onds": "USECS", "seconds": "SECS"}} + +--- Done --- diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 2758f48143..ea52e8660d 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -249,6 +249,7 @@ 247 rw auto quick 248 rw auto quick 249 rw auto quick +250 rw auto quick 252 rw auto backing quick 253 rw auto quick 255 rw auto quick --=20 2.20.1