From nobody Fri Sep 20 22:14:55 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589423224; cv=none; d=zohomail.com; s=zohoarc; b=AkM9PyC7k3RxTMhudj6tnhuN5GxFBJhGu+K4SzjR8EVXEMF9OYV43V2wPROmwfoo3BNF6lkj4O3ljI9bv0u3OIggstRQBcDr3MsBLHYtHm6Z52s+t/IgXfGlrDkVTYQuxVMrTBxXNKBWanizIUa8wt3Ine9n4f25ZS/vSLNhWdU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589423224; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=o/oQ44lobvqiuBhXiYhIXYCZQOaFhRvOrcaEPFqbvBk=; b=PVoT7nrASIu+CDYzIWVxP+TZmuHSAJhqH7EpsGFTKa8yN2ZDoN1/J2A6e4VxnpvJq7CKnLEMTNfXOV01i0PT7ovLbrAFWzYbOaxmOh9dO4WAjjsje+bo4T7lUTQ+FD76SLX8k+Z7YYq4tlHc2K07y3ncp5iQs2AxTgG6LTRwOs0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589423224830200.0965213986501; Wed, 13 May 2020 19:27:04 -0700 (PDT) Received: from localhost ([::1]:54410 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZ3aR-0001Vo-HE for importer@patchew.org; Wed, 13 May 2020 22:27:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZ3ZE-0008Dn-NW for qemu-devel@nongnu.org; Wed, 13 May 2020 22:25:48 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:26825 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jZ3ZD-00030A-4K for qemu-devel@nongnu.org; Wed, 13 May 2020 22:25:48 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-400-3Yly26KbOoqicRK756-Kpw-1; Wed, 13 May 2020 22:25:40 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CE7BC461; Thu, 14 May 2020 02:25:39 +0000 (UTC) Received: from probe.redhat.com (ovpn-113-9.rdu2.redhat.com [10.10.113.9]) by smtp.corp.redhat.com (Postfix) with ESMTP id AAE34579AD; Thu, 14 May 2020 02:25:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589423142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o/oQ44lobvqiuBhXiYhIXYCZQOaFhRvOrcaEPFqbvBk=; b=d7LgBGFxAkO0YKBNBdeTeP44AInbnPK1CpR057mU4s8j/qneNc8hm3MOoH2sdkybHYbymI rxDbSD+op9JAxvT/cdijDqTmAbbYy8ecPPIt9jI4lfNGwj6CPXfCIXk1yz9TqsJLzxUC9g TLem+/Yz/Ik21TNHDxh+itX/rxg9d0U= X-MC-Unique: 3Yly26KbOoqicRK756-Kpw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v4 1/3] qmp.py: change event_wait to use a dict Date: Wed, 13 May 2020 22:25:34 -0400 Message-Id: <20200514022536.2568-2-jsnow@redhat.com> In-Reply-To: <20200514022536.2568-1-jsnow@redhat.com> References: <20200514022536.2568-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/13 22:25:42 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" It's easier to work with than a list of tuples, because we can check the keys for membership. Signed-off-by: John Snow --- python/qemu/machine.py | 10 +++++----- tests/qemu-iotests/040 | 12 ++++++------ tests/qemu-iotests/260 | 5 +++-- tests/qemu-iotests/iotests.py | 16 ++++++++-------- 4 files changed, 22 insertions(+), 21 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index b9a98e2c86..eaedc25172 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -478,21 +478,21 @@ def event_wait(self, name, timeout=3D60.0, match=3DNo= ne): timeout: QEMUMonitorProtocol.pull_event timeout parameter. match: Optional match criteria. See event_match for details. """ - return self.events_wait([(name, match)], timeout) + return self.events_wait({name: match}, timeout) =20 def events_wait(self, events, timeout=3D60.0): """ events_wait waits for and returns a named event from QMP with a ti= meout. =20 - events: a sequence of (name, match_criteria) tuples. + events: a mapping containing {name: match_criteria}. The match criteria are optional and may be None. See event_match for details. timeout: QEMUMonitorProtocol.pull_event timeout parameter. """ def _match(event): - for name, match in events: - if event['event'] =3D=3D name and self.event_match(event, = match): - return True + name =3D event['event'] + if name in events: + return self.event_match(event, events[name]) return False =20 # Search cached events diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index 32c82b4ec6..90b59081ff 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase): =20 def run_job(self, expected_events, error_pauses_job=3DFalse): match_device =3D {'data': {'device': 'job0'}} - events =3D [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ] + events =3D { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + } =20 completed =3D False log =3D [] diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260 index 804a7addb9..729f031122 100755 --- a/tests/qemu-iotests/260 +++ b/tests/qemu-iotests/260 @@ -67,8 +67,9 @@ def test(persistent, restart): =20 vm.qmp_log('block-commit', device=3D'drive0', top=3Dtop, filters=3D[iotests.filter_qmp_testfiles]) - ev =3D vm.events_wait((('BLOCK_JOB_READY', None), - ('BLOCK_JOB_COMPLETED', None))) + ev =3D vm.events_wait({ + 'BLOCK_JOB_READY': None, + 'BLOCK_JOB_COMPLETED': None }) log(filter_qmp_event(ev)) if (ev['event'] =3D=3D 'BLOCK_JOB_COMPLETED'): vm.shutdown() diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6c0e781af7..aada94f4f9 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -635,14 +635,14 @@ def run_job(self, job, auto_finalize=3DTrue, auto_dis= miss=3DFalse, """ match_device =3D {'data': {'device': job}} match_id =3D {'data': {'id': job}} - events =3D [ - ('BLOCK_JOB_COMPLETED', match_device), - ('BLOCK_JOB_CANCELLED', match_device), - ('BLOCK_JOB_ERROR', match_device), - ('BLOCK_JOB_READY', match_device), - ('BLOCK_JOB_PENDING', match_id), - ('JOB_STATUS_CHANGE', match_id) - ] + events =3D { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + 'BLOCK_JOB_PENDING': match_id, + 'JOB_STATUS_CHANGE': match_id, + } error =3D None while True: ev =3D filter_qmp_event(self.events_wait(events, timeout=3Dwai= t)) --=20 2.21.1 From nobody Fri Sep 20 22:14:55 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589423277; cv=none; d=zohomail.com; s=zohoarc; b=bfuZUcnf8i7bRz1RlXcu2ntiM2OU1urgD+WuEiPhT+i7q/r2m6BeUGvVoy8mGTAsI6KOlseBy8ceKijpowAWjdJq6DxL3vYNEAFyYLsQl6MEpeaIB2YOWU2F1Q7nuvPz0zDCeHkF5eVpcqkicfn4J3KH4UCtqMaDb5s1z8w3rbg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589423277; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fNIOPZCAQymSh1653exouKLx0WJK8WQ9XabE6c3zeaM=; b=USwxesGnJDso+Op/lwniLA1kn+vRtIS7CTjWzaYzPYp2eohLXGSXyDqC5FopP3LLwWZj80gVCHn7lQwxuxMdgGi56jQ315UkYy0LFd/6X+uOGxW6PhYj5GKc1m66+uyI9PvPgvvgz793nNinAScMbPTVfjEnXKUwz/KJ/tCJhkM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589423277868787.128231705997; Wed, 13 May 2020 19:27:57 -0700 (PDT) Received: from localhost ([::1]:58620 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZ3bI-0003C1-Hu for importer@patchew.org; Wed, 13 May 2020 22:27:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZ3ZG-0008Fb-4w for qemu-devel@nongnu.org; Wed, 13 May 2020 22:25:50 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:36026 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jZ3ZD-00030e-Kb for qemu-devel@nongnu.org; Wed, 13 May 2020 22:25:49 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-167-XsXixpEmP6yXVWStjvkpPQ-1; Wed, 13 May 2020 22:25:44 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C1EAB1009600; Thu, 14 May 2020 02:25:43 +0000 (UTC) Received: from probe.redhat.com (ovpn-113-9.rdu2.redhat.com [10.10.113.9]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1EBAA783B3; Thu, 14 May 2020 02:25:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589423146; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fNIOPZCAQymSh1653exouKLx0WJK8WQ9XabE6c3zeaM=; b=PPAZCHOdAy4KIvP78TJe1PzsQAwZxWxcQY6ueP8lA0ziBY66UJcMUWpvYLk014d3JuY86w oXpDcE1kuC/qDpEKOz496rt27MdIGLQn+MWDAN/aYa/exxVF1Y6rqxV9Mq1KgCar+qIRxW Tgkok74dRB8YJl89vGE3hl9ovlCpHCs= X-MC-Unique: XsXixpEmP6yXVWStjvkpPQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v4 2/3] iotests: add JobRunner class Date: Wed, 13 May 2020 22:25:35 -0400 Message-Id: <20200514022536.2568-3-jsnow@redhat.com> In-Reply-To: <20200514022536.2568-1-jsnow@redhat.com> References: <20200514022536.2568-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/13 22:25:46 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The idea is that instead of increasing the arguments to job_run all the time, create a more general-purpose job runner that can be subclassed to do interesting things with. pylint note: the 'callbacks' option guards against unused warning arguments in functions designated as callbacks. It does not currently guard against "no-self-use" though; hence a once-off ignore. mypy note: QapiEvent is only a weak alias; it's fully interchangable with the type it's declared as. In the future, we may wish to tighten these types. For now, this communicates the rough shape of the type and (more importantly) the intent. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- tests/qemu-iotests/055 | 17 +-- tests/qemu-iotests/155 | 11 +- tests/qemu-iotests/255 | 9 +- tests/qemu-iotests/257 | 54 +++++---- tests/qemu-iotests/iotests.py | 206 +++++++++++++++++++++++++--------- tests/qemu-iotests/pylintrc | 11 ++ 6 files changed, 225 insertions(+), 83 deletions(-) diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055 index 4d3744b0d3..7e13585a94 100755 --- a/tests/qemu-iotests/055 +++ b/tests/qemu-iotests/055 @@ -115,19 +115,22 @@ class TestSingleDrive(iotests.QMPTestCase): self.do_test_pause('blockdev-backup', 'drive1', blockdev_target_im= g) =20 def do_test_resize_blockdev_backup(self, device, node): - def pre_finalize(): - result =3D self.vm.qmp('block_resize', device=3Ddevice, size= =3D65536) - self.assert_qmp(result, 'error/class', 'GenericError') - - result =3D self.vm.qmp('block_resize', node_name=3Dnode, size= =3D65536) - self.assert_qmp(result, 'error/class', 'GenericError') + class JobRunner(iotests.TestJobRunner): + def on_pending(self, event): + qmp =3D self._vm.qmp + result =3D qmp('block_resize', device=3Ddevice, size=3D655= 36) + self.test.assert_qmp(result, 'error/class', 'GenericError') + result =3D qmp('block_resize', node_name=3Dnode, size=3D65= 536) + self.test.assert_qmp(result, 'error/class', 'GenericError') + super().on_pending(event) =20 result =3D self.vm.qmp('blockdev-backup', job_id=3D'job0', device= =3D'drive0', target=3D'drive1', sync=3D'full', auto_finali= ze=3DFalse, auto_dismiss=3DFalse) self.assert_qmp(result, 'return', {}) =20 - self.vm.run_job('job0', auto_finalize=3DFalse, pre_finalize=3Dpre_= finalize) + runner =3D JobRunner(self.vm, 'job0', test=3Dself, auto_finalize= =3DFalse) + runner.run() =20 def test_source_resize_blockdev_backup(self): self.do_test_resize_blockdev_backup('drive0', 'source') diff --git a/tests/qemu-iotests/155 b/tests/qemu-iotests/155 index cb371d4649..1aa6cfefb8 100755 --- a/tests/qemu-iotests/155 +++ b/tests/qemu-iotests/155 @@ -163,6 +163,12 @@ class BaseClass(iotests.QMPTestCase): self.assert_qmp_absent(node, 'image/backing-image') =20 =20 +class MirrorJob(iotests.TestJobRunner): + def on_pending(self, event): + self.test.openBacking() + super().on_pending(event) + + # Class variables for controlling its behavior: # # cmd: Mirroring command to execute, either drive-mirror or blockdev-mirror @@ -188,8 +194,9 @@ class MirrorBaseClass(BaseClass): =20 self.assert_qmp(result, 'return', {}) =20 - self.vm.run_job('mirror-job', auto_finalize=3DFalse, - pre_finalize=3Dself.openBacking, auto_dismiss=3DTr= ue) + job =3D MirrorJob(self.vm, 'mirror-job', test=3Dself, + auto_finalize=3DFalse, auto_dismiss=3DTrue) + job.run() =20 def testFull(self): self.runMirror('full') diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255 index 8f08f741da..74487548fa 100755 --- a/tests/qemu-iotests/255 +++ b/tests/qemu-iotests/255 @@ -71,8 +71,13 @@ with iotests.FilePath('t.qcow2') as disk_path, \ result =3D vm.qmp_log('block-commit', job_id=3D'job0', auto_finalize= =3DFalse, device=3D'overlay', top_node=3D'mid') =20 - vm.run_job('job0', auto_finalize=3DFalse, pre_finalize=3Dstart_request= s, - auto_dismiss=3DTrue) + class JobRunner(iotests.JobRunner): + def on_pending(self, event): + start_requests() + super().on_pending(event) + + runner =3D JobRunner(vm, 'job0', auto_finalize=3DFalse, auto_dismiss= =3DTrue) + runner.run() =20 vm.shutdown() =20 diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257 index 004a433b8b..2933e2670b 100755 --- a/tests/qemu-iotests/257 +++ b/tests/qemu-iotests/257 @@ -352,30 +352,40 @@ def test_bitmap_sync(bsync_mode, msync_mode=3D'bitmap= ', failure=3DNone): job =3D backup(drive0, 1, bsync1, msync_mode, bitmap=3D"bitmap0", bitmap_mode=3Dbsync_mode) =20 - def _callback(): - """Issue writes while the job is open to test bitmap divergenc= e.""" - # Note: when `failure` is 'intermediate', this isn't called. - log('') - bitmaps =3D perform_writes(drive0, 2, filter_node_name=3D'back= up-top') - # Named bitmap (static, should be unchanged) - ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', - bitmaps=3Dbitmaps)) - # Anonymous bitmap (dynamic, shows new writes) - anonymous =3D EmulatedBitmap() - anonymous.dirty_group(2) - anonymous.compare(vm.get_bitmap(drive0.node, '', recording=3DT= rue, - bitmaps=3Dbitmaps)) =20 - # Simulate the order in which this will happen: - # group 1 gets cleared first, then group two gets written. - if ((bsync_mode =3D=3D 'on-success' and not failure) or - (bsync_mode =3D=3D 'always')): - ebitmap.clear() - ebitmap.dirty_group(2) + class JobRunner(iotests.JobRunner): + def on_pending(self, event): + """ + Issue writes while the job is open to test bitmap divergen= ce. + """ + + # Note: when `failure` is 'intermediate', this isn't calle= d. + log('') + bitmaps =3D perform_writes(drive0, 2, + filter_node_name=3D'backup-top') + # Named bitmap (static, should be unchanged) + ebitmap.compare(vm.get_bitmap(drive0.node, 'bitmap0', + bitmaps=3Dbitmaps)) + # Anonymous bitmap (dynamic, shows new writes) + anonymous =3D EmulatedBitmap() + anonymous.dirty_group(2) + anonymous.compare(vm.get_bitmap(drive0.node, '', recording= =3DTrue, + bitmaps=3Dbitmaps)) + + # Simulate the order in which this will happen: + # group 1 gets cleared first, then group two gets written. + if ((bsync_mode =3D=3D 'on-success' and not failure) or + (bsync_mode =3D=3D 'always')): + ebitmap.clear() + ebitmap.dirty_group(2) + + super().on_pending(event) + + + runner =3D JobRunner(vm, job, cancel=3D(failure =3D=3D 'simulated'= ), + auto_finalize=3DFalse, auto_dismiss=3DTrue) + runner.run() =20 - vm.run_job(job, auto_dismiss=3DTrue, auto_finalize=3DFalse, - pre_finalize=3D_callback, - cancel=3D(failure =3D=3D 'simulated')) bitmaps =3D vm.query_bitmaps() log({'bitmaps': bitmaps}, indent=3D2) log('') diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index aada94f4f9..6b9b35acb7 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -40,6 +40,7 @@ =20 # Type Aliases QMPResponse =3D Dict[str, Any] +QapiEvent =3D Dict[str, Any] =20 =20 # Use this logger for logging messages directly from the iotests module @@ -489,6 +490,141 @@ def remote_filename(path): else: raise Exception("Protocol %s not supported" % (imgproto)) =20 + +class JobRunner: + """ + JobRunner offers a job-lifetime management framework. + + It can be used as-is for a no-frills run-to-completion module, + or subclassed to gain access to per-event callbacks for + customizable behavior. + + :param vm: The VM the job is running on + :param job: Job ID of a recently created job + :param cancel: When true, cancels the job prior to finalization. + :param auto_finalize: True if the job was configured to finalize itsel= f. + :param auto_dismiss: True if the job will dismiss itself post-finaliza= tion. + """ + def __init__(self, + vm: 'VM', + job: str, + cancel: bool =3D False, + auto_finalize: bool =3D True, + auto_dismiss: bool =3D False): + self._vm =3D vm + self._id =3D job + self.cancel =3D cancel + + self._auto_finalize =3D auto_finalize + self._auto_dismiss =3D auto_dismiss + self._exited =3D False + self._error: Optional[str] =3D None + + match_device =3D {'data': {'device': self._id}} + match_id =3D {'data': {'id': self._id}} + + # Listen for these events with these parameters: + self._events =3D { + 'BLOCK_JOB_COMPLETED': match_device, + 'BLOCK_JOB_CANCELLED': match_device, + 'BLOCK_JOB_ERROR': match_device, + 'BLOCK_JOB_READY': match_device, + 'BLOCK_JOB_PENDING': match_id, + 'JOB_STATUS_CHANGE': match_id + } + + self._dispatch =3D { + 'created': self.on_create, + 'running': self.on_run, + 'paused': self.on_pause, + 'ready': self.on_ready, + 'standby': self.on_standby, + 'waiting': self.on_waiting, + 'pending': self.on_pending, + 'aborting': self.on_abort, + 'concluded': self.on_conclude, + 'null': self.on_null, + } + + # These are Job state change callbacks. + # Subclass and override these for custom workflows. + + def on_create(self, event: QapiEvent) -> None: + pass + + def on_run(self, event: QapiEvent) -> None: + pass + + def on_pause(self, event: QapiEvent) -> None: + pass + + def on_ready(self, event: QapiEvent) -> None: + self._vm.qmp_log('job-complete', id=3Dself._id) + + def on_standby(self, event: QapiEvent) -> None: + pass + + def on_waiting(self, event: QapiEvent) -> None: + pass + + def on_pending(self, event: QapiEvent) -> None: + if self._auto_finalize: + return + + if self.cancel: + self._vm.qmp_log('job-cancel', id=3Dself._id) + else: + self._vm.qmp_log('job-finalize', id=3Dself._id) + + def on_abort(self, event: QapiEvent) -> None: + result =3D self._vm.qmp('query-jobs') + for j in result['return']: + if j['id'] =3D=3D self._id: + self._error =3D j['error'] + log('Job failed: %s' % (j['error'])) + + def on_conclude(self, event: QapiEvent) -> None: + if self._auto_dismiss: + return + + self._vm.qmp_log('job-dismiss', id=3Dself._id) + + def on_null(self, event: QapiEvent) -> None: + self._exited =3D True + + # Macro events -- QAPI events. + # These are callbacks for individual top-level events. + + def on_change(self, event: QapiEvent) -> None: + status =3D event['data']['status'] + assert status in self._dispatch + self._dispatch[status](event) + + def on_block_job_event(self, event: QapiEvent) -> None: + # pylint: disable=3Dno-self-use + log(event) + + def event(self, event: QapiEvent) -> None: + assert event['event'] in self._events.keys() + if event['event'] =3D=3D 'JOB_STATUS_CHANGE': + self.on_change(event) + else: + self.on_block_job_event(event) + + def run(self, wait: float =3D 60.0) -> Optional[str]: + """ + Run the event loop for this job. + + :param wait: Timeout in seconds specifying how long to wait + for an event. Defaults to 60.0. + :return: Error string on failure, Nothing on success. + """ + while not self._exited: + raw_event =3D self._vm.events_wait(self._events, timeout=3Dwai= t) + self.event(filter_qmp_event(raw_event)) + return self._error + + class VM(qtest.QEMUQtestMachine): '''A QEMU VM''' =20 @@ -615,60 +751,21 @@ def qmp_log(self, cmd, filters=3D(), indent=3DNone, *= *kwargs): log(result, filters, indent=3Dindent) return result =20 - # Returns None on success, and an error string on failure - def run_job(self, job, auto_finalize=3DTrue, auto_dismiss=3DFalse, - pre_finalize=3DNone, cancel=3DFalse, wait=3D60.0): + def run_job(self, job, **kwargs) -> Optional[str]: """ run_job moves a job from creation through to dismissal. =20 - :param job: String. ID of recently-launched job - :param auto_finalize: Bool. True if the job was launched with - auto_finalize. Defaults to True. - :param auto_dismiss: Bool. True if the job was launched with - auto_dismiss=3DTrue. Defaults to False. - :param pre_finalize: Callback. A callable that takes no arguments = to be - invoked prior to issuing job-finalize, if any. - :param cancel: Bool. When true, cancels the job after the pre_fina= lize - callback. - :param wait: Float. Timeout value specifying how long to wait for = any - event, in seconds. Defaults to 60.0. + :param job: Job ID of a recently created job. + :param kwargs: See JobRunner.__init__() and JobRunner.run(). + + :return: Error string on failure, Nothing on success. """ - match_device =3D {'data': {'device': job}} - match_id =3D {'data': {'id': job}} - events =3D { - 'BLOCK_JOB_COMPLETED': match_device, - 'BLOCK_JOB_CANCELLED': match_device, - 'BLOCK_JOB_ERROR': match_device, - 'BLOCK_JOB_READY': match_device, - 'BLOCK_JOB_PENDING': match_id, - 'JOB_STATUS_CHANGE': match_id, - } - error =3D None - while True: - ev =3D filter_qmp_event(self.events_wait(events, timeout=3Dwai= t)) - if ev['event'] !=3D 'JOB_STATUS_CHANGE': - log(ev) - continue - status =3D ev['data']['status'] - if status =3D=3D 'aborting': - result =3D self.qmp('query-jobs') - for j in result['return']: - if j['id'] =3D=3D job: - error =3D j['error'] - log('Job failed: %s' % (j['error'])) - elif status =3D=3D 'ready': - self.qmp_log('job-complete', id=3Djob) - elif status =3D=3D 'pending' and not auto_finalize: - if pre_finalize: - pre_finalize() - if cancel: - self.qmp_log('job-cancel', id=3Djob) - else: - self.qmp_log('job-finalize', id=3Djob) - elif status =3D=3D 'concluded' and not auto_dismiss: - self.qmp_log('job-dismiss', id=3Djob) - elif status =3D=3D 'null': - return error + if 'wait' in kwargs: + run_kwargs =3D {'wait': kwargs.pop('wait')} + else: + run_kwargs =3D {} + job_runner =3D JobRunner(self, job, **kwargs) + return job_runner.run(**run_kwargs) =20 # Returns None on success, and an error string on failure def blockdev_create(self, options, job_id=3D'job0', filters=3DNone): @@ -980,6 +1077,15 @@ def case_skip(self, reason): self.skipTest(reason) =20 =20 +class TestJobRunner(JobRunner): + """ + JobRunner intended for use within a QMPTestCase. + """ + def __init__(self, *args, test: QMPTestCase, **kwargs): + super().__init__(*args, **kwargs) + self.test =3D test + + def notrun(reason): '''Skip this test suite''' # Each test in qemu-iotests has a number ("seq") diff --git a/tests/qemu-iotests/pylintrc b/tests/qemu-iotests/pylintrc index 5481afe528..df602e02b1 100644 --- a/tests/qemu-iotests/pylintrc +++ b/tests/qemu-iotests/pylintrc @@ -17,9 +17,20 @@ disable=3Dinvalid-name, too-many-lines, too-many-locals, too-many-public-methods, + too-many-instance-attributes, # These are temporary, and should be removed: missing-docstring, =20 + +[VARIABLES] + +# List of strings which can identify a callback function by name. A callba= ck +# name must start or end with one of those strings. +callbacks=3Dcb_, + _cb, + on_, + + [FORMAT] =20 # Maximum number of characters on a single line. --=20 2.21.1 From nobody Fri Sep 20 22:14:55 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1589423223; cv=none; d=zohomail.com; s=zohoarc; b=h1JA+OEQt35XOgGozPwFDJZG+2163k5egZkVOD5fybecXgSCb+5wpjGAT7L2Jt6wf+MiteuVHXRcbtjq7tv+1Vu+cTO2EvUqAfDJK4DqvPpmhgNSmFgRbBw/O0ZjObpv44aSg40FdODPg1/H64pUdRc20cgrhxRDd24V5Wfcws0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1589423223; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=WfYl0TwQMc5ZPBZYeuBGsbh0WVxyzZyrT23KcuRjoBI=; b=DuUqY2ZP9F+uWqChHZPXVHQfROau97v2AF0SGcgXp76OsDebidWRalCZhU+eBybmYqBxncWgeTZ2CmrYLEkD5L3L7lPM+lp3XLdevVvBaoh0/7YAs9v+olL/cMtxcQb2K8AGYiwg5vSpOeb+k6frVPzjHuJR40z6u/euco+5KaA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1589423223104474.73275533247056; Wed, 13 May 2020 19:27:03 -0700 (PDT) Received: from localhost ([::1]:54208 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jZ3aP-0001Qn-F3 for importer@patchew.org; Wed, 13 May 2020 22:27:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jZ3ZL-0008Sm-Gn for qemu-devel@nongnu.org; Wed, 13 May 2020 22:25:55 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:38135 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jZ3ZK-00034F-LI for qemu-devel@nongnu.org; Wed, 13 May 2020 22:25:55 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-29-cFFc1vafPAaVioojmmk4nA-1; Wed, 13 May 2020 22:25:46 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2CE211005512; Thu, 14 May 2020 02:25:45 +0000 (UTC) Received: from probe.redhat.com (ovpn-113-9.rdu2.redhat.com [10.10.113.9]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F4406B8C2; Thu, 14 May 2020 02:25:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589423153; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WfYl0TwQMc5ZPBZYeuBGsbh0WVxyzZyrT23KcuRjoBI=; b=R0EaGMCSQzo45S+I2xPvIdMf7Poe1/h+QP5kkE+PMJynZNXZX0G5sBVr7aQdLsnt7LXgbD RAHswK7GVpCKX1iZxhtS6FVJcL6obgl/tqL4BT4flgcUSQMJbCbYo98ayV+lIezJTa7asV qWWTGHc5zkapNXr7s/uiWYjoSaWYPbQ= X-MC-Unique: cFFc1vafPAaVioojmmk4nA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v4 3/3] iotests: modify test 040 to use JobRunner Date: Wed, 13 May 2020 22:25:36 -0400 Message-Id: <20200514022536.2568-4-jsnow@redhat.com> In-Reply-To: <20200514022536.2568-1-jsnow@redhat.com> References: <20200514022536.2568-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/13 22:25:46 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Instead of having somewhat reproduced it for itself. Signed-off-by: John Snow --- tests/qemu-iotests/040 | 51 +++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index 90b59081ff..e2ef3bb812 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -483,34 +483,33 @@ class TestErrorHandling(iotests.QMPTestCase): file=3D('top-dbg' if top_debug else 'top-file'), backing=3D'mid-fmt') =20 + + class TestJobRunner(iotests.JobRunner): + expected_events =3D ('BLOCK_JOB_COMPLETED', + 'BLOCK_JOB_ERROR', + 'BLOCK_JOB_READY') + + def __init__(self, *args, test, **kwargs): + super().__init__(*args, **kwargs) + self.log =3D [] + self.test =3D test + + def on_pause(self, event): + super().on_pause(event) + result =3D self._vm.qmp('block-job-resume', device=3Dself._id) + self.test.assert_qmp(result, 'return', {}) + + def on_block_job_event(self, event): + if event['event'] not in self.expected_events: + self.test.fail("Unexpected event: %s" % event) + super().on_block_job_event(event) + self.log.append(event) + def run_job(self, expected_events, error_pauses_job=3DFalse): - match_device =3D {'data': {'device': 'job0'}} - events =3D { - 'BLOCK_JOB_COMPLETED': match_device, - 'BLOCK_JOB_CANCELLED': match_device, - 'BLOCK_JOB_ERROR': match_device, - 'BLOCK_JOB_READY': match_device, - } - - completed =3D False - log =3D [] - while not completed: - ev =3D self.vm.events_wait(events, timeout=3D5.0) - if ev['event'] =3D=3D 'BLOCK_JOB_COMPLETED': - completed =3D True - elif ev['event'] =3D=3D 'BLOCK_JOB_ERROR': - if error_pauses_job: - result =3D self.vm.qmp('block-job-resume', device=3D'j= ob0') - self.assert_qmp(result, 'return', {}) - elif ev['event'] =3D=3D 'BLOCK_JOB_READY': - result =3D self.vm.qmp('block-job-complete', device=3D'job= 0') - self.assert_qmp(result, 'return', {}) - else: - self.fail("Unexpected event: %s" % ev) - log.append(iotests.filter_qmp_event(ev)) - + job =3D self.TestJobRunner(self.vm, 'job0', test=3Dself) + job.run() self.maxDiff =3D None - self.assertEqual(expected_events, log) + self.assertEqual(expected_events, job.log) =20 def event_error(self, op, action): return { --=20 2.21.1