From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635790654863649.1462769858794; Mon, 1 Nov 2021 11:17:34 -0700 (PDT) Received: from localhost ([::1]:55182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhbsC-0004sO-W7 for importer@patchew.org; Mon, 01 Nov 2021 14:17:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60082) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9E-000832-TF for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53343) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9C-0001DW-LY for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:04 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-287-oNjJJJ95Mkqf3TpWy1oIGQ-1; Mon, 01 Nov 2021 13:30:58 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6F8A78C7AC0; Mon, 1 Nov 2021 17:30:57 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6A99C19C79; Mon, 1 Nov 2021 17:30:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635787861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QhUXOoOWxzPAZji1INAECdkm15KNkMJuh/rHhC/QTMs=; b=aq6883vRWZJWwcmQGexMqHkoL/fr/Sx0DsGnWeoUnxgf3CS+JYuvsbs8RS/QmiSFc/vQva z6B6EISoSmGC23yCvIIpWy5Mc2BLk2DgSuriJ6eJK8wyDrCqYbQ1RorIfGD0jqnKVjH3Iv MlLQYMwbk9ooJJWEiIGseaqHr1ZyEDQ= X-MC-Unique: oNjJJJ95Mkqf3TpWy1oIGQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 01/22] iotests/297: Move pylint config into pylintrc Date: Mon, 1 Nov 2021 13:29:45 -0400 Message-Id: <20211101173006.656673-2-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635790655564100001 Content-Type: text/plain; charset="utf-8" Move --score=3Dn and --notes=3DXXX,FIXME into pylintrc. This pulls configuration out of code, which I think is probably a good thing in general. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-2-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 4 +--- tests/qemu-iotests/pylintrc | 16 ++++++++++++++++ 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 91ec34d9521..bc3a0ceb2aa 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -65,10 +65,8 @@ def run_linters(): print('=3D=3D=3D pylint =3D=3D=3D') sys.stdout.flush() =20 - # Todo notes are fine, but fixme's or xxx's should probably just be - # fixed (in tests, at least) env =3D os.environ.copy() - subprocess.run(('pylint-3', '--score=3Dn', '--notes=3DFIXME,XXX', *fil= es), + subprocess.run(('pylint-3', *files), env=3Denv, check=3DFalse) =20 print('=3D=3D=3D mypy =3D=3D=3D') diff --git a/tests/qemu-iotests/pylintrc b/tests/qemu-iotests/pylintrc index 8cb4e1d6a6d..32ab77b8bb9 100644 --- a/tests/qemu-iotests/pylintrc +++ b/tests/qemu-iotests/pylintrc @@ -31,6 +31,22 @@ disable=3Dinvalid-name, too-many-statements, consider-using-f-string, =20 + +[REPORTS] + +# Activate the evaluation score. +score=3Dno + + +[MISCELLANEOUS] + +# List of note tags to take in consideration, separated by a comma. +# TODO notes are fine, but FIXMEs or XXXs should probably just be +# fixed (in tests, at least). +notes=3DFIXME, + XXX, + + [FORMAT] =20 # Maximum number of characters on a single line. --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792232280484.3878417509454; Mon, 1 Nov 2021 11:43:52 -0700 (PDT) Received: from localhost ([::1]:50354 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcHf-0007jP-8H for importer@patchew.org; Mon, 01 Nov 2021 14:43:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60202) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9N-0008VB-15 for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:60919) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9L-0001HF-5Q for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:12 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-243-wDh55ZfdNkawM3RMh9oadQ-1; Mon, 01 Nov 2021 13:31:07 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5D2F58C7AC0; Mon, 1 Nov 2021 17:31:06 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B47619C79; Mon, 1 Nov 2021 17:30:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635787870; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5/CzatqaI60Qc34fOp18eWhW5bFEc2LB3IPyDH6H9j0=; b=RCtZ5idXlzXMC6Yy3g/V6s+sGX4zBYw2nhtT7/ayvlxBAvvKmeiTu4O+2FThw87mOUnUtZ AMbnZHgC4JSu3onMg6HRTzPMZ36/95X5H2Afst2fb0aO2jULPnldXruNHpicb3pm7pqy+C vgPBGt0vPdJqspI0fRHrMbu78n7tbZ0= X-MC-Unique: wDh55ZfdNkawM3RMh9oadQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 02/22] iotests/297: Split mypy configuration out into mypy.ini Date: Mon, 1 Nov 2021 13:29:46 -0400 Message-Id: <20211101173006.656673-3-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792232729100001 Content-Type: text/plain; charset="utf-8" More separation of code and configuration. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-3-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 14 +------------- tests/qemu-iotests/mypy.ini | 12 ++++++++++++ 2 files changed, 13 insertions(+), 13 deletions(-) create mode 100644 tests/qemu-iotests/mypy.ini diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index bc3a0ceb2aa..b8101e6024a 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -73,19 +73,7 @@ def run_linters(): sys.stdout.flush() =20 env['MYPYPATH'] =3D env['PYTHONPATH'] - p =3D subprocess.run(('mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - '--scripts-are-modules', - *files), + p =3D subprocess.run(('mypy', *files), env=3Denv, check=3DFalse, stdout=3Dsubprocess.PIPE, diff --git a/tests/qemu-iotests/mypy.ini b/tests/qemu-iotests/mypy.ini new file mode 100644 index 00000000000..4c0339f5589 --- /dev/null +++ b/tests/qemu-iotests/mypy.ini @@ -0,0 +1,12 @@ +[mypy] +disallow_any_generics =3D True +disallow_incomplete_defs =3D True +disallow_subclassing_any =3D True +disallow_untyped_decorators =3D True +implicit_reexport =3D False +namespace_packages =3D True +no_implicit_optional =3D True +scripts_are_modules =3D True +warn_redundant_casts =3D True +warn_unused_configs =3D True +warn_unused_ignores =3D True --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635791232203105.04709047010181; Mon, 1 Nov 2021 11:27:12 -0700 (PDT) Received: from localhost ([::1]:46088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhc1X-0001g3-6V for importer@patchew.org; Mon, 01 Nov 2021 14:27:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60348) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9b-0000pJ-QU for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:55169) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9Z-0001UV-Ge for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-269-xPgALlDPNGKBc7t5GE2Ogw-1; Mon, 01 Nov 2021 13:31:22 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A33A68C7AC0; Mon, 1 Nov 2021 17:31:20 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1DA6419C79; Mon, 1 Nov 2021 17:31:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635787885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hApTjurf9R1BxFDtLd8YKhviLeNsZfMbTTYMbG4Nqhg=; b=CMubNU2PiLTgxzDYXppu2LgCtM6YiQQo8xi/zYhi+m+p4oj79KC36HlyRSMovWz73dm7SU J2y1ZnKtIRqU78v5WLI7Dvd970sDDUKdIVGBk2MW7oJlVA+GMHWVAFS2rFQXJ7ecOjZWcc WvrkDK/y2j943jFxHyHbG7f3w0ZUub0= X-MC-Unique: xPgALlDPNGKBc7t5GE2Ogw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 03/22] iotests/297: Add get_files() function Date: Mon, 1 Nov 2021 13:29:47 -0400 Message-Id: <20211101173006.656673-4-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635791233937100001 Content-Type: text/plain; charset="utf-8" Split out file discovery into its own method to begin separating out configuration/setup and test execution. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-4-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b8101e6024a..15b54594c11 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,6 +21,7 @@ import re import shutil import subprocess import sys +from typing import List =20 import iotests =20 @@ -54,10 +55,14 @@ def is_python_file(filename): return False =20 =20 -def run_linters(): +def get_test_files() -> List[str]: named_tests =3D [f'tests/{entry}' for entry in os.listdir('tests')] check_tests =3D set(os.listdir('.') + named_tests) - set(SKIP_FILES) - files =3D [filename for filename in check_tests if is_python_file(file= name)] + return list(filter(is_python_file, check_tests)) + + +def run_linters(): + files =3D get_test_files() =20 iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792567645942.5378411709646; Mon, 1 Nov 2021 11:49:27 -0700 (PDT) Received: from localhost ([::1]:39084 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcN4-0004BA-N0 for importer@patchew.org; Mon, 01 Nov 2021 14:49:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9u-0001pK-UO for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49847) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhb9t-0001x5-2c for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:31:46 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-409-vOjJv8pdMv2a3pQeb4IwMw-1; Mon, 01 Nov 2021 13:31:40 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 35422100CC84; Mon, 1 Nov 2021 17:31:39 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E9EF19C79; Mon, 1 Nov 2021 17:31:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635787903; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1LEHiKc3lzRZrQjvkZh2caalxSLIQwUL2gZ2TvEB9NQ=; b=INs/uv8STcnlGNvN0rMlGPir4Km9qBQvDYA/h1up1GlUgq56mrVWnUzns0/KU6iqyd72gt Ib+wrUZEwjgQGFnWKe6h2PgUZAAIGcHPI+mZfjZcXDTf3vwcJ1tLWT4a5n0W7h427Upkca /m/vBRyTsuV909TUg8Drkru1PHkqWKg= X-MC-Unique: vOjJv8pdMv2a3pQeb4IwMw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 04/22] iotests/297: Create main() function Date: Mon, 1 Nov 2021 13:29:48 -0400 Message-Id: <20211101173006.656673-5-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792569507100001 Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-5-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 15b54594c11..163ebc8ebfd 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -89,8 +89,12 @@ def run_linters(): print(p.stdout) =20 =20 -for linter in ('pylint-3', 'mypy'): - if shutil.which(linter) is None: - iotests.notrun(f'{linter} not found') +def main() -> None: + for linter in ('pylint-3', 'mypy'): + if shutil.which(linter) is None: + iotests.notrun(f'{linter} not found') =20 -iotests.script_main(run_linters) + run_linters() + + +iotests.script_main(main) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635791437403697.435006712645; Mon, 1 Nov 2021 11:30:37 -0700 (PDT) Received: from localhost ([::1]:53232 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhc4q-0006g9-7X for importer@patchew.org; Mon, 01 Nov 2021 14:30:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60604) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbAB-0002VD-3p for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:32:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:31610) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbA8-0002N6-BQ for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:32:02 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-445-fhqnlSvePFq0Xj8AYUkZig-1; Mon, 01 Nov 2021 13:31:56 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 772F81926DA1; Mon, 1 Nov 2021 17:31:55 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id D15EE19C79; Mon, 1 Nov 2021 17:31:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635787919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6/p7oY25zyBjhHKMczZ+IlEp25rFpyHZc+ByQonrjBc=; b=d/XQnHEnQ4M8CsK5Wz5Ztg0QsVuzi1ItJ4191WFv0vrQ8bsD8msTtKrCV1e5RGIXGoNYCr wBxYTiqsI+8LbkNLFFgY6kQTptYlOt2au4L8qr5dd7MQh4/VdWUv1J4POsdMk/M2ePckGe ZCeAVBcbqG4NYyNIsYhusRSD75+2jT0= X-MC-Unique: fhqnlSvePFq0Xj8AYUkZig-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 05/22] iotests/297: Don't rely on distro-specific linter binaries Date: Mon, 1 Nov 2021 13:29:49 -0400 Message-Id: <20211101173006.656673-6-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635791438816100001 'pylint-3' is another Fedora-ism. Use "python3 -m pylint" or "python3 -m mypy" to access these scripts instead. This style of invocation will prefer the "correct" tool when run in a virtual environment. Note that we still check for "pylint-3" before the test begins -- this check is now "overly strict", but shouldn't cause anything that was already running correctly to start failing. This is addressed by a commit later in this series; 'iotests/297: update tool availability checks'. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-6-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 163ebc8ebfd..c1bddb9ce0e 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -71,14 +71,14 @@ def run_linters(): sys.stdout.flush() =20 env =3D os.environ.copy() - subprocess.run(('pylint-3', *files), + subprocess.run(('python3', '-m', 'pylint', *files), env=3Denv, check=3DFalse) =20 print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() =20 env['MYPYPATH'] =3D env['PYTHONPATH'] - p =3D subprocess.run(('mypy', *files), + p =3D subprocess.run(('python3', '-m', 'mypy', *files), env=3Denv, check=3DFalse, stdout=3Dsubprocess.PIPE, --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635791771418614.9323534851264; Mon, 1 Nov 2021 11:36:11 -0700 (PDT) Received: from localhost ([::1]:33810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcAE-0004cI-AY for importer@patchew.org; Mon, 01 Nov 2021 14:36:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60706) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbAi-00036W-Nv for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:32:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43913) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbAe-0002Rb-TB for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:32:36 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-577-WMXXK2BSOAibtEGf3cxfpQ-1; Mon, 01 Nov 2021 13:32:29 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0DF341926DA3; Mon, 1 Nov 2021 17:32:28 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 623CC19C79; Mon, 1 Nov 2021 17:31:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635787952; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vWL55e6rIIfWRyv5w3FFldEHbuRfFSgyRlWMR4+nSyc=; b=feAGtBHPLeoTGCIl8COKh5Rdk7fuqbl/f+WlmoafevDW4mC0ew0XM3H43N0cFlujhlljZH WclxN4rYlElx64ak/AraigSf0XYJjNaktOB+OlqgmwI84TPBBmFmy2BzfOYLvNzxu5e2Dk UBZhXayI0kLEbpbBgF8aA8U1XcwTVqY= X-MC-Unique: WMXXK2BSOAibtEGf3cxfpQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 06/22] iotests/297: Split run_linters apart into run_pylint and run_mypy Date: Mon, 1 Nov 2021 13:29:50 -0400 Message-Id: <20211101173006.656673-7-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635791771673100001 Content-Type: text/plain; charset="utf-8" Move environment setup into main(), and split the actual linter execution into run_pylint and run_mypy, respectively. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-7-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 38 ++++++++++++++++++++++++-------------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index c1bddb9ce0e..189bcaf5f94 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,7 +21,7 @@ import re import shutil import subprocess import sys -from typing import List +from typing import List, Mapping, Optional =20 import iotests =20 @@ -61,23 +61,19 @@ def get_test_files() -> List[str]: return list(filter(is_python_file, check_tests)) =20 =20 -def run_linters(): - files =3D get_test_files() +def run_pylint( + files: List[str], + env: Optional[Mapping[str, str]] =3D None, +) -> None: =20 - iotests.logger.debug('Files to be checked:') - iotests.logger.debug(', '.join(sorted(files))) - - print('=3D=3D=3D pylint =3D=3D=3D') - sys.stdout.flush() - - env =3D os.environ.copy() subprocess.run(('python3', '-m', 'pylint', *files), env=3Denv, check=3DFalse) =20 - print('=3D=3D=3D mypy =3D=3D=3D') - sys.stdout.flush() =20 - env['MYPYPATH'] =3D env['PYTHONPATH'] +def run_mypy( + files: List[str], + env: Optional[Mapping[str, str]] =3D None, +) -> None: p =3D subprocess.run(('python3', '-m', 'mypy', *files), env=3Denv, check=3DFalse, @@ -94,7 +90,21 @@ def main() -> None: if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') =20 - run_linters() + files =3D get_test_files() + + iotests.logger.debug('Files to be checked:') + iotests.logger.debug(', '.join(sorted(files))) + + env =3D os.environ.copy() + env['MYPYPATH'] =3D env['PYTHONPATH'] + + print('=3D=3D=3D pylint =3D=3D=3D') + sys.stdout.flush() + run_pylint(files, env=3Denv) + + print('=3D=3D=3D mypy =3D=3D=3D') + sys.stdout.flush() + run_mypy(files, env=3Denv) =20 =20 iotests.script_main(main) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635791639676106.97990192193015; Mon, 1 Nov 2021 11:33:59 -0700 (PDT) Received: from localhost ([::1]:58442 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhc86-00022O-Ph for importer@patchew.org; Mon, 01 Nov 2021 14:33:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbB4-0003qm-Cu for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:32:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:53396) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbB2-0002Up-4C for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:32:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-156-lzw83j65P0uijcU8xTq50g-1; Mon, 01 Nov 2021 13:32:52 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E6F798F508; Mon, 1 Nov 2021 17:32:50 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4F6A19C79; Mon, 1 Nov 2021 17:32:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635787975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vt9quG6WaMVVLqkNRaT61HzcN6diBu8/L0zD0RtXTtc=; b=dkf/ScbJ8W85RKph6DxhCH/VTDBGup/pmHGKWd6hzRLsfKSUgLoBtTdfhxtczTn3fTCVKv IyUqSVLAcb5rxnf7boLVt1ruoa8eJ8poacCsgTvvKoVbySF8R46eoRHjpUhKFIR36Wdn+S p8tRSxCWSsdIKPGDN0ksaL7pqEjsupk= X-MC-Unique: lzw83j65P0uijcU8xTq50g-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 07/22] iotests/297: refactor run_[mypy|pylint] as generic execution shim Date: Mon, 1 Nov 2021 13:29:51 -0400 Message-Id: <20211101173006.656673-8-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635791640199100001 Content-Type: text/plain; charset="utf-8" There's virtually nothing special here anymore; we can combine these into a single, rather generic function. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-8-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 42 ++++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 189bcaf5f94..d21673a2929 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -61,27 +61,29 @@ def get_test_files() -> List[str]: return list(filter(is_python_file, check_tests)) =20 =20 -def run_pylint( - files: List[str], - env: Optional[Mapping[str, str]] =3D None, +def run_linter( + tool: str, + args: List[str], + env: Optional[Mapping[str, str]] =3D None, + suppress_output: bool =3D False, ) -> None: + """ + Run a python-based linting tool. =20 - subprocess.run(('python3', '-m', 'pylint', *files), - env=3Denv, check=3DFalse) + If suppress_output is True, capture stdout/stderr of the child + process and only print that information back to stdout if the child + process's return code was non-zero. + """ + p =3D subprocess.run( + ('python3', '-m', tool, *args), + env=3Denv, + check=3DFalse, + stdout=3Dsubprocess.PIPE if suppress_output else None, + stderr=3Dsubprocess.STDOUT if suppress_output else None, + universal_newlines=3DTrue, + ) =20 - -def run_mypy( - files: List[str], - env: Optional[Mapping[str, str]] =3D None, -) -> None: - p =3D subprocess.run(('python3', '-m', 'mypy', *files), - env=3Denv, - check=3DFalse, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.STDOUT, - universal_newlines=3DTrue) - - if p.returncode !=3D 0: + if suppress_output and p.returncode !=3D 0: print(p.stdout) =20 =20 @@ -100,11 +102,11 @@ def main() -> None: =20 print('=3D=3D=3D pylint =3D=3D=3D') sys.stdout.flush() - run_pylint(files, env=3Denv) + run_linter('pylint', files, env=3Denv) =20 print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() - run_mypy(files, env=3Denv) + run_linter('mypy', files, env=3Denv, suppress_output=3DTrue) =20 =20 iotests.script_main(main) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792818125178.4817321823482; Mon, 1 Nov 2021 11:53:38 -0700 (PDT) Received: from localhost ([::1]:49204 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcR7-00033p-3E for importer@patchew.org; Mon, 01 Nov 2021 14:53:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32838) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbBd-0005aY-Aa for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:33:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54884) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbBb-0002dE-47 for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:33:33 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-224-ODFA44m9MMCgTe4zq1Mjkg-1; Mon, 01 Nov 2021 13:33:26 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 112571926DA4; Mon, 1 Nov 2021 17:33:25 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 909CD19C79; Mon, 1 Nov 2021 17:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788009; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+lRJZDdCs6ETP0b6RYTKCKDDCsMWJU1ViGY8raNiFNI=; b=hYVA46l9PUzjNjDOKsd3N31mPwvNiqJm6evZ57eshtx3zqNCfHiadaBTjhHiztVtF0wDCZ 2vXUXYhzpFMsIPU/itCugQ8PWdsR7FsPQArJobP4jJvgsD8O8QSEB7MFR6eqKokYU2sl7k GCtmdls7GWTJeFWdMyYqBY+ycK2Tt14= X-MC-Unique: ODFA44m9MMCgTe4zq1Mjkg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 08/22] iotests/297: Change run_linter() to raise an exception on failure Date: Mon, 1 Nov 2021 13:29:52 -0400 Message-Id: <20211101173006.656673-9-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792819077100001 Content-Type: text/plain; charset="utf-8" Instead of using a process return code as the python function return value (or just not returning anything at all), allow run_linter() to raise an exception instead. The responsibility for printing output on error shifts from the function itself to the caller, who will know best how to present/format that information. (Also, "suppress_output" is now a lot more accurate of a parameter name.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-9-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index d21673a2929..76d6a23f531 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -70,22 +70,18 @@ def run_linter( """ Run a python-based linting tool. =20 - If suppress_output is True, capture stdout/stderr of the child - process and only print that information back to stdout if the child - process's return code was non-zero. + :param suppress_output: If True, suppress all stdout/stderr output. + :raise CalledProcessError: If the linter process exits with failure. """ - p =3D subprocess.run( + subprocess.run( ('python3', '-m', tool, *args), env=3Denv, - check=3DFalse, + check=3DTrue, stdout=3Dsubprocess.PIPE if suppress_output else None, stderr=3Dsubprocess.STDOUT if suppress_output else None, universal_newlines=3DTrue, ) =20 - if suppress_output and p.returncode !=3D 0: - print(p.stdout) - =20 def main() -> None: for linter in ('pylint-3', 'mypy'): @@ -102,11 +98,19 @@ def main() -> None: =20 print('=3D=3D=3D pylint =3D=3D=3D') sys.stdout.flush() - run_linter('pylint', files, env=3Denv) + try: + run_linter('pylint', files, env=3Denv) + except subprocess.CalledProcessError: + # pylint failure will be caught by diffing the IO. + pass =20 print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() - run_linter('mypy', files, env=3Denv, suppress_output=3DTrue) + try: + run_linter('mypy', files, env=3Denv, suppress_output=3DTrue) + except subprocess.CalledProcessError as exc: + if exc.output: + print(exc.output) =20 =20 iotests.script_main(main) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635791634635563.4080273814894; Mon, 1 Nov 2021 11:33:54 -0700 (PDT) Received: from localhost ([::1]:58266 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhc81-0001vO-He for importer@patchew.org; Mon, 01 Nov 2021 14:33:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33026) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbBz-0006YO-Ej for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:33:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37138) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbBx-0002gj-Gx for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:33:55 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-276-7JW-t-n_O4SJujVO0Jn2KA-1; Mon, 01 Nov 2021 13:33:49 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 889B48CDE2C; Mon, 1 Nov 2021 17:33:48 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8649319C79; Mon, 1 Nov 2021 17:33:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5vAaELmkMVxlrffC32RkW0IUAAzxzTdGBv/JuBn0n9k=; b=JHpOGTQZYdaeOHoQyAHB3brtcRLwP/iEI68GVrXxYSrGC1khLqwIDHxaQPCtCgr6fppMgT eD5xUFU0FElVmLEG1Rm6UsmhHg1V4oMOFQDfvGSpv4lBixIIOS0j3SENl7SSJ1Gt+e2xOv TrcI7laG+739A6FxUpUMh4Y8ZcP0tRk= X-MC-Unique: 7JW-t-n_O4SJujVO0Jn2KA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 09/22] iotests/297: update tool availability checks Date: Mon, 1 Nov 2021 13:29:53 -0400 Message-Id: <20211101173006.656673-10-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635791635900100001 Content-Type: text/plain; charset="utf-8" As mentioned in 'iotests/297: Don't rely on distro-specific linter binaries', these checks are overly strict. Update them to be in-line with how we actually invoke the linters themselves. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-10-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 76d6a23f531..b2ad8d1cbe0 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -18,7 +18,6 @@ =20 import os import re -import shutil import subprocess import sys from typing import List, Mapping, Optional @@ -84,9 +83,11 @@ def run_linter( =20 =20 def main() -> None: - for linter in ('pylint-3', 'mypy'): - if shutil.which(linter) is None: - iotests.notrun(f'{linter} not found') + for linter in ('pylint', 'mypy'): + try: + run_linter(linter, ['--version'], suppress_output=3DTrue) + except subprocess.CalledProcessError: + iotests.notrun(f"'{linter}' not found") =20 files =3D get_test_files() =20 --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635791919583721.2225105034253; Mon, 1 Nov 2021 11:38:39 -0700 (PDT) Received: from localhost ([::1]:38556 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcCb-0007mc-KQ for importer@patchew.org; Mon, 01 Nov 2021 14:38:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33132) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbCR-0007V0-1j for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:34:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41107) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbCP-0002jQ-2W for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:34:22 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-553-AROhGOMnMcShET6HRxg4iQ-1; Mon, 01 Nov 2021 13:34:17 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2471A9126B; Mon, 1 Nov 2021 17:34:16 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5E98F19C79; Mon, 1 Nov 2021 17:33:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788060; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gV6vgoYT6MbDRHrE9Z0zLk2wxLzMrREwSAfYcvEd2Xk=; b=W3R4faf4+ZWoJxdLUil03klPvQpFxu8YZ3uG1+yX/tJUY6xLN8/q7cizYV9hFLEcnZlWdG C431taCsRmsxHV7/zktR4+xs3UXzzVu3bzkIxQ8OwXYDOl4C6WUYFfUHZsBR5NqtfwOw0D w+Yv/LSJb7EkBDPYwq++8iUernWvaw4= X-MC-Unique: AROhGOMnMcShET6HRxg4iQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 10/22] iotests/297: split test into sub-cases Date: Mon, 1 Nov 2021 13:29:54 -0400 Message-Id: <20211101173006.656673-11-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635791921716100001 Content-Type: text/plain; charset="utf-8" Take iotest 297's main() test function and split it into two sub-cases that can be skipped individually. We can also drop custom environment setup from the pylint test as it isn't needed. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-11-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 63 ++++++++++++++++++++++++++---------------- 1 file changed, 39 insertions(+), 24 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b2ad8d1cbe0..b7d9d6077b3 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -82,36 +82,51 @@ def run_linter( ) =20 =20 +def check_linter(linter: str) -> bool: + try: + run_linter(linter, ['--version'], suppress_output=3DTrue) + except subprocess.CalledProcessError: + iotests.case_notrun(f"'{linter}' not found") + return False + return True + + +def test_pylint(files: List[str]) -> None: + print('=3D=3D=3D pylint =3D=3D=3D') + sys.stdout.flush() + + if not check_linter('pylint'): + return + + run_linter('pylint', files) + + +def test_mypy(files: List[str]) -> None: + print('=3D=3D=3D mypy =3D=3D=3D') + sys.stdout.flush() + + if not check_linter('mypy'): + return + + env =3D os.environ.copy() + env['MYPYPATH'] =3D env['PYTHONPATH'] + + run_linter('mypy', files, env=3Denv, suppress_output=3DTrue) + + def main() -> None: - for linter in ('pylint', 'mypy'): - try: - run_linter(linter, ['--version'], suppress_output=3DTrue) - except subprocess.CalledProcessError: - iotests.notrun(f"'{linter}' not found") - files =3D get_test_files() =20 iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) =20 - env =3D os.environ.copy() - env['MYPYPATH'] =3D env['PYTHONPATH'] - - print('=3D=3D=3D pylint =3D=3D=3D') - sys.stdout.flush() - try: - run_linter('pylint', files, env=3Denv) - except subprocess.CalledProcessError: - # pylint failure will be caught by diffing the IO. - pass - - print('=3D=3D=3D mypy =3D=3D=3D') - sys.stdout.flush() - try: - run_linter('mypy', files, env=3Denv, suppress_output=3DTrue) - except subprocess.CalledProcessError as exc: - if exc.output: - print(exc.output) + for test in (test_pylint, test_mypy): + try: + test(files) + except subprocess.CalledProcessError as exc: + # Linter failure will be caught by diffing the IO. + if exc.output: + print(exc.output) =20 =20 iotests.script_main(main) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792278804870.0303600886114; Mon, 1 Nov 2021 11:44:38 -0700 (PDT) Received: from localhost ([::1]:53162 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcIP-0001Qg-Rv for importer@patchew.org; Mon, 01 Nov 2021 14:44:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbD5-00089J-Jj for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:35:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:39672) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbD3-0002nB-8X for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:35:03 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-201-VaohCyIsNFa2-IZP9Sk13A-1; Mon, 01 Nov 2021 13:34:57 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 996259126B; Mon, 1 Nov 2021 17:34:55 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6870719C79; Mon, 1 Nov 2021 17:34:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788100; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XXYdt0B5tOeX7vd54jdC6tJEksva/KPFUIRnDUFXHnE=; b=FyPhSDbij9CYR7pywMV/EKSjiKiNwKrTtrKI9CES/DMcWKlDEUY6E7uvsWU9F1WPYIcl8f IwISRfe0GfhkRrj7lKFvS6dzca4fh+sOxKHk0UimTZ9tOPX873RP67a0t46nFNloR6FUmT Y9J6H5y8hSZEBA6I0RP5sO5tcz4Xzs0= X-MC-Unique: VaohCyIsNFa2-IZP9Sk13A-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 11/22] iotests: split linters.py out from 297 Date: Mon, 1 Nov 2021 13:29:55 -0400 Message-Id: <20211101173006.656673-12-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792279934100001 Content-Type: text/plain; charset="utf-8" Now, 297 is just the iotests-specific incantations and linters.py is as minimal as I can think to make it. The only remaining element in here that ought to be configuration and not code is the list of skip files, but they're still numerous enough that repeating them for mypy and pylint configurations both would be ... a hassle. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-12-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/297 | 72 +++++---------------------------- tests/qemu-iotests/linters.py | 76 +++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+), 61 deletions(-) create mode 100644 tests/qemu-iotests/linters.py diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b7d9d6077b3..ee78a627359 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -17,74 +17,24 @@ # along with this program. If not, see . =20 import os -import re import subprocess import sys -from typing import List, Mapping, Optional +from typing import List =20 import iotests +import linters =20 =20 -# TODO: Empty this list! -SKIP_FILES =3D ( - '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', - '096', '118', '124', '132', '136', '139', '147', '148', '149', - '151', '152', '155', '163', '165', '194', '196', '202', - '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', - '218', '219', '224', '228', '234', '235', '236', '237', '238', - '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', - '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', - '299', '302', '303', '304', '307', - 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' -) - - -def is_python_file(filename): - if not os.path.isfile(filename): - return False - - if filename.endswith('.py'): - return True - - with open(filename, encoding=3D'utf-8') as f: - try: - first_line =3D f.readline() - return re.match('^#!.*python', first_line) is not None - except UnicodeDecodeError: # Ignore binary files - return False - - -def get_test_files() -> List[str]: - named_tests =3D [f'tests/{entry}' for entry in os.listdir('tests')] - check_tests =3D set(os.listdir('.') + named_tests) - set(SKIP_FILES) - return list(filter(is_python_file, check_tests)) - - -def run_linter( - tool: str, - args: List[str], - env: Optional[Mapping[str, str]] =3D None, - suppress_output: bool =3D False, -) -> None: - """ - Run a python-based linting tool. - - :param suppress_output: If True, suppress all stdout/stderr output. - :raise CalledProcessError: If the linter process exits with failure. - """ - subprocess.run( - ('python3', '-m', tool, *args), - env=3Denv, - check=3DTrue, - stdout=3Dsubprocess.PIPE if suppress_output else None, - stderr=3Dsubprocess.STDOUT if suppress_output else None, - universal_newlines=3DTrue, - ) +# Looking for something? +# +# List of files to exclude from linting: linters.py +# mypy configuration: mypy.ini +# pylint configuration: pylintrc =20 =20 def check_linter(linter: str) -> bool: try: - run_linter(linter, ['--version'], suppress_output=3DTrue) + linters.run_linter(linter, ['--version'], suppress_output=3DTrue) except subprocess.CalledProcessError: iotests.case_notrun(f"'{linter}' not found") return False @@ -98,7 +48,7 @@ def test_pylint(files: List[str]) -> None: if not check_linter('pylint'): return =20 - run_linter('pylint', files) + linters.run_linter('pylint', files) =20 =20 def test_mypy(files: List[str]) -> None: @@ -111,11 +61,11 @@ def test_mypy(files: List[str]) -> None: env =3D os.environ.copy() env['MYPYPATH'] =3D env['PYTHONPATH'] =20 - run_linter('mypy', files, env=3Denv, suppress_output=3DTrue) + linters.run_linter('mypy', files, env=3Denv, suppress_output=3DTrue) =20 =20 def main() -> None: - files =3D get_test_files() + files =3D linters.get_test_files() =20 iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py new file mode 100644 index 00000000000..c515c7afe36 --- /dev/null +++ b/tests/qemu-iotests/linters.py @@ -0,0 +1,76 @@ +# Copyright (C) 2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import re +import subprocess +from typing import List, Mapping, Optional + + +# TODO: Empty this list! +SKIP_FILES =3D ( + '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', + '096', '118', '124', '132', '136', '139', '147', '148', '149', + '151', '152', '155', '163', '165', '194', '196', '202', + '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', + '218', '219', '224', '228', '234', '235', '236', '237', '238', + '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', + '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', + '299', '302', '303', '304', '307', + 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' +) + + +def is_python_file(filename): + if not os.path.isfile(filename): + return False + + if filename.endswith('.py'): + return True + + with open(filename, encoding=3D'utf-8') as f: + try: + first_line =3D f.readline() + return re.match('^#!.*python', first_line) is not None + except UnicodeDecodeError: # Ignore binary files + return False + + +def get_test_files() -> List[str]: + named_tests =3D [f'tests/{entry}' for entry in os.listdir('tests')] + check_tests =3D set(os.listdir('.') + named_tests) - set(SKIP_FILES) + return list(filter(is_python_file, check_tests)) + + +def run_linter( + tool: str, + args: List[str], + env: Optional[Mapping[str, str]] =3D None, + suppress_output: bool =3D False, +) -> None: + """ + Run a python-based linting tool. + + :param suppress_output: If True, suppress all stdout/stderr output. + :raise CalledProcessError: If the linter process exits with failure. + """ + subprocess.run( + ('python3', '-m', tool, *args), + env=3Denv, + check=3DTrue, + stdout=3Dsubprocess.PIPE if suppress_output else None, + stderr=3Dsubprocess.STDOUT if suppress_output else None, + universal_newlines=3DTrue, + ) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792452888410.92402099651304; Mon, 1 Nov 2021 11:47:32 -0700 (PDT) Received: from localhost ([::1]:33394 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcLD-0007rQ-Uy for importer@patchew.org; Mon, 01 Nov 2021 14:47:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33476) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbDc-0008W8-GO for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:35:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52546) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbDT-0002wn-BC for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:35:36 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-238-8d1YomRFNEqrCtwOfmT7jg-1; Mon, 01 Nov 2021 13:35:23 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ADB1A1926DA0; Mon, 1 Nov 2021 17:35:22 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7DC9219C79; Mon, 1 Nov 2021 17:34:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788126; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UA6ZggHcuapOOdsIo2jIAlXRy8t37afT4BCttNb6dcA=; b=SSnHD2KlGxf+YCZwFAK5qgtzFMnY94rdDc+tLLAPaYfk8wDq4ae9Vyn+IjZld/KMhqLCse /vrfnXWp6vAIqBnsgFi6hmniFJ4d2ma/kcx6RgF8+IdPMkxRc8czOCxSlquadfeQhK4iRQ cA4dEJR08RnLN1359BNNeTlqo9n+C1Y= X-MC-Unique: 8d1YomRFNEqrCtwOfmT7jg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 12/22] iotests/linters: Add entry point for linting via Python CI Date: Mon, 1 Nov 2021 13:29:56 -0400 Message-Id: <20211101173006.656673-13-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, T_SPF_HELO_TEMPERROR=0.01, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792453875100001 Content-Type: text/plain; charset="utf-8" We need at least a tiny little shim here to join test file discovery with test invocation. This logic could conceivably be hosted somewhere in python/, but I felt it was strictly the least-rude thing to keep the test logic here in iotests/, even if this small function isn't itself an iotest. Note that we don't actually even need the executable bit here, we'll be relying on the ability to run this module as a script using Python CLI arguments. No chance it gets misunderstood as an actual iotest that way. (It's named, not in tests/, doesn't have the execute bit, and doesn't have an execution shebang.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-13-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/linters.py | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index c515c7afe36..46c28fdcda0 100644 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -16,6 +16,7 @@ import os import re import subprocess +import sys from typing import List, Mapping, Optional =20 =20 @@ -74,3 +75,29 @@ def run_linter( stderr=3Dsubprocess.STDOUT if suppress_output else None, universal_newlines=3DTrue, ) + + +def main() -> None: + """ + Used by the Python CI system as an entry point to run these linters. + """ + def show_usage() -> None: + print(f"Usage: {sys.argv[0]} < --mypy | --pylint >", file=3Dsys.st= derr) + sys.exit(1) + + if len(sys.argv) !=3D 2: + show_usage() + + files =3D get_test_files() + + if sys.argv[1] =3D=3D '--pylint': + run_linter('pylint', files) + elif sys.argv[1] =3D=3D '--mypy': + run_linter('mypy', files) + else: + print(f"Unrecognized argument: '{sys.argv[1]}'", file=3Dsys.stderr) + show_usage() + + +if __name__ =3D=3D '__main__': + main() --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163579267614669.8357187988787; Mon, 1 Nov 2021 11:51:16 -0700 (PDT) Received: from localhost ([::1]:42430 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcOo-0006m1-TE for importer@patchew.org; Mon, 01 Nov 2021 14:51:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33700) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbEC-0001zl-2y for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:36:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34498) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbE9-00037x-V9 for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:36:11 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-604-8Q0bovz6Oli1bAF-Ks4Y4g-1; Mon, 01 Nov 2021 13:36:06 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4EDA610A8E01; Mon, 1 Nov 2021 17:36:05 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id DEDAB19C79; Mon, 1 Nov 2021 17:35:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788169; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ns6c1OlNy9CyNvD/UazqnY1SJqx5gH9tGPM3Sc8NxdE=; b=f1T8j/Lck1y0Nkm90U1WTN5ovsWPnAJQntIT8zVJ54BCbz66SDUh5Lu6KaeOPpYPXncqdU g1gNqUUOomObaoaNAe/Lq62OugPHQ0p8V0BUv568wOfIaq9ZNkna/ZJz8jiPvmoNd5azvD wPbV2EKOz5vFG84cqCvdNiT54qMREpo= X-MC-Unique: 8Q0bovz6Oli1bAF-Ks4Y4g-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 13/22] iotests/linters: Add workaround for mypy bug #9852 Date: Mon, 1 Nov 2021 13:29:57 -0400 Message-Id: <20211101173006.656673-14-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792678266100001 Content-Type: text/plain; charset="utf-8" This one is insidious: if you write an import as "from {namespace} import {subpackage}" as mirror-top-perms (now) does, mypy will fail on every-other invocation *if* the package being imported is a typed, installed, namespace-scoped package. Upsettingly, that's exactly what 'qemu.[aqmp|qmp|machine]' et al are in the context of Python CI tests. Now, I could just edit mirror-top-perms to avoid this invocation, but since I tripped on a landmine, I might as well head it off at the pass and make sure nobody else trips on that same landmine. It seems to have something to do with the order in which files are checked as well, meaning the random order in which set(os.listdir()) produces the list of files to test will cause problems intermittently and not just strictly "every other run". This will be fixed in mypy >=3D 0.920, which is not released yet. The workaround for now is to disable incremental checking, which avoids the issue. Note: This workaround is not applied when running iotest 297 directly, because the bug does not surface there! Given the nature of CI jobs not starting with any stale cache to begin with, this really only has a half-second impact on manual runs of the Python test suite when executed directly by a developer on their local machine. The workaround may be removed when the Python package requirements can stipulate mypy 0.920 or higher, which can happen as soon as it is released. (Barring any unforseen compatibility issues that 0.920 may bring with it.) See also: https://github.com/python/mypy/issues/11010 https://github.com/python/mypy/issues/9852 Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-14-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/linters.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index 46c28fdcda0..65c4c4e8272 100644 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -93,7 +93,9 @@ def show_usage() -> None: if sys.argv[1] =3D=3D '--pylint': run_linter('pylint', files) elif sys.argv[1] =3D=3D '--mypy': - run_linter('mypy', files) + # mypy bug #9852; disable incremental checking as a workaround. + args =3D ['--no-incremental'] + files + run_linter('mypy', args) else: print(f"Unrecognized argument: '{sys.argv[1]}'", file=3Dsys.stderr) show_usage() --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635793700792476.16049399850056; Mon, 1 Nov 2021 12:08:20 -0700 (PDT) Received: from localhost ([::1]:57086 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcfL-0002Us-Or for importer@patchew.org; Mon, 01 Nov 2021 15:08:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33792) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbEW-0002X4-Qr for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:36:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57648) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbEU-0003nY-Ki for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:36:32 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-461-bfXdBPQEMPWqPP1K92fv4A-1; Mon, 01 Nov 2021 13:36:27 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C5BF48066F7; Mon, 1 Nov 2021 17:36:25 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 45BE319C79; Mon, 1 Nov 2021 17:36:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788190; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y4sSoBFZezLBtoGV5lwWO+TDpCmaxAMN92BP93BZNGs=; b=a5tx6u313ToP2UEpErFiKW2ktnLc+O11/wR++bRfA4a90U+gmBeYU0gbikwpEP2w0QJxX6 cNXHqUdM6crUkoWSICh9nRBU1Pv0obWZGLL/LuoHxaS5Cd+jSkWU/DQ6splaBDHfcmns1U JCGHTvZq0SiTWKjj7SEZQ/18ynqWBsY= X-MC-Unique: bfXdBPQEMPWqPP1K92fv4A-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 14/22] python: Add iotest linters to test suite Date: Mon, 1 Nov 2021 13:29:58 -0400 Message-Id: <20211101173006.656673-15-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635793701921100001 Content-Type: text/plain; charset="utf-8" Run mypy and pylint on the iotests files directly from the Python CI test infrastructure. This ensures that any accidental breakages to the qemu.[qmp|aqmp|machine|utils] packages will be caught by that test suite. It also ensures that these linters are run with well-known versions and test against a wide variety of python versions, which helps to find accidental cross-version python compatibility issues. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Message-id: 20211019144918.3159078-15-jsnow@redhat.com Signed-off-by: John Snow --- python/tests/iotests-mypy.sh | 4 ++++ python/tests/iotests-pylint.sh | 4 ++++ 2 files changed, 8 insertions(+) create mode 100755 python/tests/iotests-mypy.sh create mode 100755 python/tests/iotests-pylint.sh diff --git a/python/tests/iotests-mypy.sh b/python/tests/iotests-mypy.sh new file mode 100755 index 00000000000..ee764708199 --- /dev/null +++ b/python/tests/iotests-mypy.sh @@ -0,0 +1,4 @@ +#!/bin/sh -e + +cd ../tests/qemu-iotests/ +python3 -m linters --mypy diff --git a/python/tests/iotests-pylint.sh b/python/tests/iotests-pylint.sh new file mode 100755 index 00000000000..4cae03424b4 --- /dev/null +++ b/python/tests/iotests-pylint.sh @@ -0,0 +1,4 @@ +#!/bin/sh -e + +cd ../tests/qemu-iotests/ +python3 -m linters --pylint --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635793970757373.03007101015703; Mon, 1 Nov 2021 12:12:50 -0700 (PDT) Received: from localhost ([::1]:38178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcjh-0000SM-0z for importer@patchew.org; Mon, 01 Nov 2021 15:12:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33922) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbEu-0002pU-Do for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:36:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39230) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbEs-0004iw-BA for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:36:56 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-412-KIg6p9DVMZal2R4nb1uojg-1; Mon, 01 Nov 2021 13:36:50 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 953CB100CCC3; Mon, 1 Nov 2021 17:36:49 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id BD6A719C79; Mon, 1 Nov 2021 17:36:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XIgEwrqwpa6tCID+aT9FFIFonh1gWMGFzUZRm2/Z9Q0=; b=MrOvIsXvFMXpsvumVanmakPnUtu2iZ6Pb4nbWFZsFwwiLVu8DOYp2zNjiJDCUc8a+gGeSl WPnTg1VfqyniiV9F7A36S7yfkC3eS62YihqvdKH2ufZc8qyjm5G66RmJ08Xsh2uMT7e583 LVXI9j4/jvn/xFRqBO+QrISTza/kkwY= X-MC-Unique: KIg6p9DVMZal2R4nb1uojg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 15/22] python/machine: remove has_quit argument Date: Mon, 1 Nov 2021 13:29:59 -0400 Message-Id: <20211101173006.656673-16-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635793971805100001 Content-Type: text/plain; charset="utf-8" If we spy on the QMP commands instead, we don't need callers to remember to pass it. Seems like a fair trade-off. The one slightly weird bit is overloading this instance variable for wait(), where we use it to mean "don't issue the qmp 'quit' command". This means that wait() will "fail" if the QEMU process does not terminate of its own accord. In most cases, we probably did already actually issue quit -- some iotests do this -- but in some others, we may be waiting for QEMU to terminate for some other reason, such as a test wherein we tell the guest (directly) to shut down. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf Message-id: 20211026175612.4127598-2-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/machine/machine.py | 34 +++++++++++++++++++--------------- tests/qemu-iotests/040 | 7 +------ tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/255 | 2 +- 4 files changed, 22 insertions(+), 23 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 056d340e355..0bd40bc2f76 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -170,6 +170,7 @@ def __init__(self, self._console_socket: Optional[socket.socket] =3D None self._remove_files: List[str] =3D [] self._user_killed =3D False + self._quit_issued =3D False =20 def __enter__(self: _T) -> _T: return self @@ -368,6 +369,7 @@ def _post_shutdown(self) -> None: command =3D '' LOG.warning(msg, -int(exitcode), command) =20 + self._quit_issued =3D False self._user_killed =3D False self._launched =3D False =20 @@ -443,15 +445,13 @@ def _hard_shutdown(self) -> None: self._subp.kill() self._subp.wait(timeout=3D60) =20 - def _soft_shutdown(self, timeout: Optional[int], - has_quit: bool =3D False) -> None: + def _soft_shutdown(self, timeout: Optional[int]) -> None: """ Perform early cleanup, attempt to gracefully shut down the VM, and= wait for it to terminate. =20 :param timeout: Timeout in seconds for graceful shutdown. A value of None is an infinite wait. - :param has_quit: When True, don't attempt to issue 'quit' QMP comm= and =20 :raise ConnectionReset: On QMP communication errors :raise subprocess.TimeoutExpired: When timeout is exceeded waiting= for @@ -460,21 +460,19 @@ def _soft_shutdown(self, timeout: Optional[int], self._early_cleanup() =20 if self._qmp_connection: - if not has_quit: + if not self._quit_issued: # Might raise ConnectionReset - self._qmp.cmd('quit') + self.qmp('quit') =20 # May raise subprocess.TimeoutExpired self._subp.wait(timeout=3Dtimeout) =20 - def _do_shutdown(self, timeout: Optional[int], - has_quit: bool =3D False) -> None: + def _do_shutdown(self, timeout: Optional[int]) -> None: """ Attempt to shutdown the VM gracefully; fallback to a hard shutdown. =20 :param timeout: Timeout in seconds for graceful shutdown. A value of None is an infinite wait. - :param has_quit: When True, don't attempt to issue 'quit' QMP comm= and =20 :raise AbnormalShutdown: When the VM could not be shut down gracef= ully. The inner exception will likely be ConnectionReset or @@ -482,13 +480,13 @@ def _do_shutdown(self, timeout: Optional[int], may result in its own exceptions, likely subprocess.TimeoutExp= ired. """ try: - self._soft_shutdown(timeout, has_quit) + self._soft_shutdown(timeout) except Exception as exc: self._hard_shutdown() raise AbnormalShutdown("Could not perform graceful shutdown") \ from exc =20 - def shutdown(self, has_quit: bool =3D False, + def shutdown(self, hard: bool =3D False, timeout: Optional[int] =3D 30) -> None: """ @@ -498,7 +496,6 @@ def shutdown(self, has_quit: bool =3D False, If the VM has not yet been launched, or shutdown(), wait(), or kil= l() have already been called, this method does nothing. =20 - :param has_quit: When true, do not attempt to issue 'quit' QMP com= mand. :param hard: When true, do not attempt graceful shutdown, and suppress the SIGKILL warning log message. :param timeout: Optional timeout in seconds for graceful shutdown. @@ -512,7 +509,7 @@ def shutdown(self, has_quit: bool =3D False, self._user_killed =3D True self._hard_shutdown() else: - self._do_shutdown(timeout, has_quit) + self._do_shutdown(timeout) finally: self._post_shutdown() =20 @@ -529,7 +526,8 @@ def wait(self, timeout: Optional[int] =3D 30) -> None: :param timeout: Optional timeout in seconds. Default 30 seconds. A value of `None` is an infinite wait. """ - self.shutdown(has_quit=3DTrue, timeout=3Dtimeout) + self._quit_issued =3D True + self.shutdown(timeout=3Dtimeout) =20 def set_qmp_monitor(self, enabled: bool =3D True) -> None: """ @@ -574,7 +572,10 @@ def qmp(self, cmd: str, conv_keys =3D True =20 qmp_args =3D self._qmp_args(conv_keys, args) - return self._qmp.cmd(cmd, args=3Dqmp_args) + ret =3D self._qmp.cmd(cmd, args=3Dqmp_args) + if cmd =3D=3D 'quit' and 'error' not in ret and 'return' in ret: + self._quit_issued =3D True + return ret =20 def command(self, cmd: str, conv_keys: bool =3D True, @@ -585,7 +586,10 @@ def command(self, cmd: str, On failure raise an exception. """ qmp_args =3D self._qmp_args(conv_keys, args) - return self._qmp.command(cmd, **qmp_args) + ret =3D self._qmp.command(cmd, **qmp_args) + if cmd =3D=3D 'quit': + self._quit_issued =3D True + return ret =20 def get_qmp_event(self, wait: bool =3D False) -> Optional[QMPMessage]: """ diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index f3677de9dfd..6af5ab9e764 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -92,10 +92,9 @@ class TestSingleDrive(ImageCommitTestCase): self.vm.add_device('virtio-scsi') self.vm.add_device("scsi-hd,id=3Dscsi0,drive=3Ddrive0") self.vm.launch() - self.has_quit =3D False =20 def tearDown(self): - self.vm.shutdown(has_quit=3Dself.has_quit) + self.vm.shutdown() os.remove(test_img) os.remove(mid_img) os.remove(backing_img) @@ -127,8 +126,6 @@ class TestSingleDrive(ImageCommitTestCase): result =3D self.vm.qmp('quit') self.assert_qmp(result, 'return', {}) =20 - self.has_quit =3D True - # Same as above, but this time we add the filter after starting the job @iotests.skip_if_unsupported(['throttle']) def test_commit_plus_filter_and_quit(self): @@ -147,8 +144,6 @@ class TestSingleDrive(ImageCommitTestCase): result =3D self.vm.qmp('quit') self.assert_qmp(result, 'return', {}) =20 - self.has_quit =3D True - def test_device_not_found(self): result =3D self.vm.qmp('block-commit', device=3D'nonexistent', top= =3D'%s' % mid_img) self.assert_qmp(result, 'error/class', 'DeviceNotFound') diff --git a/tests/qemu-iotests/218 b/tests/qemu-iotests/218 index 325d8244fb9..4922b4d3b6f 100755 --- a/tests/qemu-iotests/218 +++ b/tests/qemu-iotests/218 @@ -187,4 +187,4 @@ with iotests.VM() as vm, \ log(vm.qmp('quit')) =20 with iotests.Timeout(5, 'Timeout waiting for VM to quit'): - vm.shutdown(has_quit=3DTrue) + vm.shutdown() diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255 index c43aa9c67ac..3d6d0e80cb5 100755 --- a/tests/qemu-iotests/255 +++ b/tests/qemu-iotests/255 @@ -123,4 +123,4 @@ with iotests.FilePath('src.qcow2') as src_path, \ vm.qmp_log('block-job-cancel', device=3D'job0') vm.qmp_log('quit') =20 - vm.shutdown(has_quit=3DTrue) + vm.shutdown() --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163579212310729.754114761553637; Mon, 1 Nov 2021 11:42:03 -0700 (PDT) Received: from localhost ([::1]:45452 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcFt-00043W-RP for importer@patchew.org; Mon, 01 Nov 2021 14:42:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33956) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbFA-0003Zi-Ci for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30024) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbF8-0004ku-AK for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:12 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-488-ugkAI2yUMmSTVEcPkpPxCw-1; Mon, 01 Nov 2021 13:37:08 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 72152100CCC1; Mon, 1 Nov 2021 17:37:07 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id CEB1B19C79; Mon, 1 Nov 2021 17:36:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788229; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XcJbuQmzJFayLt+QeueMmpxv+eATYJ/ut7+XZoj2D1c=; b=c5alevpSLA1X1bePWUHcHzovGphharMJd9dBcLr9j4KGA/oQR744SL+2cAOTvR3Ai+bOjP TKSczqVWld7S3zlxfqocK2e1ZEr81RCGtLXyj8wU+TM+hpB2kbVp0vNzN7DbhpZmJ9LCBQ SPzzqZ4HjilzRIjBiR13yY3UmTckvcE= X-MC-Unique: ugkAI2yUMmSTVEcPkpPxCw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 16/22] python/machine: Handle QMP errors on close more meticulously Date: Mon, 1 Nov 2021 13:30:00 -0400 Message-Id: <20211101173006.656673-17-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792125285100001 Content-Type: text/plain; charset="utf-8" To use the AQMP backend, Machine just needs to be a little more diligent about what happens when closing a QMP connection. The operation is no longer a freebie in the async world; it may return errors encountered in the async bottom half on incoming message receipt, etc. (AQMP's disconnect, ultimately, serves as the quiescence point where all async contexts are gathered together, and any final errors reported at that point.) Because async QMP continues to check for messages asynchronously, it's almost certainly likely that the loop will have exited due to EOF after issuing the last 'quit' command. That error will ultimately be bubbled up when attempting to close the QMP connection. The manager class here then is free to discard it -- if it was expected. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf Message-id: 20211026175612.4127598-3-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/machine/machine.py | 48 +++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 6 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 0bd40bc2f76..a0cf69786b4 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -342,9 +342,15 @@ def _post_shutdown(self) -> None: # Comprehensive reset for the failed launch case: self._early_cleanup() =20 - if self._qmp_connection: - self._qmp.close() - self._qmp_connection =3D None + try: + self._close_qmp_connection() + except Exception as err: # pylint: disable=3Dbroad-except + LOG.warning( + "Exception closing QMP connection: %s", + str(err) if str(err) else type(err).__name__ + ) + finally: + assert self._qmp_connection is None =20 self._close_qemu_log_file() =20 @@ -420,6 +426,31 @@ def _launch(self) -> None: close_fds=3DFalse) self._post_launch() =20 + def _close_qmp_connection(self) -> None: + """ + Close the underlying QMP connection, if any. + + Dutifully report errors that occurred while closing, but assume + that any error encountered indicates an abnormal termination + process and not a failure to close. + """ + if self._qmp_connection is None: + return + + try: + self._qmp.close() + except EOFError: + # EOF can occur as an Exception here when using the Async + # QMP backend. It indicates that the server closed the + # stream. If we successfully issued 'quit' at any point, + # then this was expected. If the remote went away without + # our permission, it's worth reporting that as an abnormal + # shutdown case. + if not (self._user_killed or self._quit_issued): + raise + finally: + self._qmp_connection =3D None + def _early_cleanup(self) -> None: """ Perform any cleanup that needs to happen before the VM exits. @@ -460,9 +491,14 @@ def _soft_shutdown(self, timeout: Optional[int]) -> No= ne: self._early_cleanup() =20 if self._qmp_connection: - if not self._quit_issued: - # Might raise ConnectionReset - self.qmp('quit') + try: + if not self._quit_issued: + # May raise ExecInterruptedError or StateError if the + # connection dies or has *already* died. + self.qmp('quit') + finally: + # Regardless, we want to quiesce the connection. + self._close_qmp_connection() =20 # May raise subprocess.TimeoutExpired self._subp.wait(timeout=3Dtimeout) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792317206291.6119934247041; Mon, 1 Nov 2021 11:45:17 -0700 (PDT) Received: from localhost ([::1]:54646 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcJ2-0002aa-7e for importer@patchew.org; Mon, 01 Nov 2021 14:45:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbFS-0003sO-U0 for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22748) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbFO-0004n7-Gy for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:30 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-193-kC_3Eb5YMfWjHcbebMfcxg-1; Mon, 01 Nov 2021 13:37:23 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C63DA10A8E00; Mon, 1 Nov 2021 17:37:21 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 56C6519C79; Mon, 1 Nov 2021 17:37:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nDdBRwKr9y4wuLsc2uRhUmZgkPd0FZu4pAOfVYPf8HQ=; b=eEj+NdD3EsEifbJ/4UN41VFW6HLT9sGWVd8ss0oH96Xh0HiFTFSbcq822n4ilwYAA9b0zl r9KTqyw06L1rbHVcTcJExq6kqONW/tDiBgEXkFS6C6GjRYIaTOy5now0sqHGOAgm/5QOjc AYpe7y6F2hHN5KnKowYnLYG4DavDiIA= X-MC-Unique: kC_3Eb5YMfWjHcbebMfcxg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 17/22] python/aqmp: Remove scary message Date: Mon, 1 Nov 2021 13:30:01 -0400 Message-Id: <20211101173006.656673-18-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792317928100001 Content-Type: text/plain; charset="utf-8" The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow Acked-by: Hanna Reitz Reviewed-by: Kevin Wolf Reviewed-by: Hanna Reitz Message-id: 20211026175612.4127598-4-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/__init__.py | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py index d1b0e4dc3d3..880d5b6fa7f 100644 --- a/python/qemu/aqmp/__init__.py +++ b/python/qemu/aqmp/__init__.py @@ -22,7 +22,6 @@ # the COPYING file in the top-level directory. =20 import logging -import warnings =20 from .error import AQMPError from .events import EventListener @@ -31,17 +30,6 @@ from .qmp_client import ExecInterruptedError, ExecuteError, QMPClient =20 =20 -_WMSG =3D """ - -The Asynchronous QMP library is currently in development and its API -should be considered highly fluid and subject to change. It should -not be used by any other scripts checked into the QEMU tree. - -Proceed with caution! -""" - -warnings.warn(_WMSG, FutureWarning) - # Suppress logging unless an application engages it. logging.getLogger('qemu.aqmp').addHandler(logging.NullHandler()) =20 --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635794647881715.2133581505972; Mon, 1 Nov 2021 12:24:07 -0700 (PDT) Received: from localhost ([::1]:57824 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcua-0005dy-Qu for importer@patchew.org; Mon, 01 Nov 2021 15:24:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34192) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbFc-0004SH-D2 for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:35687) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbFa-0004oo-T1 for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:40 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-19--wpvcf7DObSoCK0uDROIVw-1; Mon, 01 Nov 2021 13:37:35 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B28138066EF; Mon, 1 Nov 2021 17:37:33 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22EC719C79; Mon, 1 Nov 2021 17:37:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788258; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QZcpucreAvBk7xxm1UjkVG5FMbri5xYkSMZO8JvYPdY=; b=DwZ7lNHmOGVcVUjzPBDMYOPZwH/y5FLDjxmIinTC2AgB4FzTKl+bwpIC57mKIijRfPam7O 56FPitHOl0iFV7Kx8YhF10oefc50Bun4LvcpyLBAAHazyVZ8PFK/XDv2ieU4F9nro3pd31 9ucyud4s7l2imne9jYdODjjbFnVqjYE= X-MC-Unique: -wpvcf7DObSoCK0uDROIVw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 18/22] iotests: Accommodate async QMP Exception classes Date: Mon, 1 Nov 2021 13:30:02 -0400 Message-Id: <20211101173006.656673-19-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635794649016100001 Content-Type: text/plain; charset="utf-8" (But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf Message-id: 20211026175612.4127598-5-jsnow@redhat.com Signed-off-by: John Snow --- scripts/simplebench/bench_block_job.py | 3 ++- tests/qemu-iotests/tests/mirror-top-perms | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/scripts/simplebench/bench_block_job.py b/scripts/simplebench/b= ench_block_job.py index 4f03c121697..a403c35b08f 100755 --- a/scripts/simplebench/bench_block_job.py +++ b/scripts/simplebench/bench_block_job.py @@ -28,6 +28,7 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'pytho= n')) from qemu.machine import QEMUMachine from qemu.qmp import QMPConnectError +from qemu.aqmp import ConnectError =20 =20 def bench_block_job(cmd, cmd_args, qemu_args): @@ -49,7 +50,7 @@ def bench_block_job(cmd, cmd_args, qemu_args): vm.launch() except OSError as e: return {'error': 'popen failed: ' + str(e)} - except (QMPConnectError, socket.timeout): + except (QMPConnectError, ConnectError, socket.timeout): return {'error': 'qemu failed: ' + str(vm.get_log())} =20 try: diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests= /tests/mirror-top-perms index 3d475aa3a54..a2d5c269d7a 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -21,8 +21,9 @@ =20 import os =20 -from qemu import qmp +from qemu.aqmp import ConnectError from qemu.machine import machine +from qemu.qmp import QMPConnectError =20 import iotests from iotests import qemu_img @@ -102,7 +103,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase): self.vm_b.launch() print('ERROR: VM B launched successfully, this should not have= ' 'happened') - except qmp.QMPConnectError: + except (QMPConnectError, ConnectError): assert 'Is another process using the image' in self.vm_b.get_l= og() =20 result =3D self.vm.qmp('block-job-cancel', --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792532335135.63667410732626; Mon, 1 Nov 2021 11:48:52 -0700 (PDT) Received: from localhost ([::1]:37752 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcMV-0002m8-8s for importer@patchew.org; Mon, 01 Nov 2021 14:48:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34294) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbFq-0005DU-8q for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:56290) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbFo-0004qw-Hm for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:37:53 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-515-YPCKyWg1Nb6T_lfIrmc_Lw-1; Mon, 01 Nov 2021 13:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AF8959126B; Mon, 1 Nov 2021 17:37:47 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD62619C79; Mon, 1 Nov 2021 17:37:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LapaOwRyJl286w7mivCvTLv8Eq1zcuBnlNOO17+2BS0=; b=bHz8esM1n/n4A74RdCraoTcgTqBUk42UD3ka7Zdzy5Dn2+6s/FS4JTqaESDqEo6zhL/CIY MnxgeaBm4zbiPUuGjqKvnMyrC8uNrVfDUb3MJQylKRhk3frn6M5Gq93PnTlkHBN4vmMegm ohYgCXugYl6dDRqWtj8ZC5vMF0f/Rbk= X-MC-Unique: YPCKyWg1Nb6T_lfIrmc_Lw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 19/22] iotests: Conditionally silence certain AQMP errors Date: Mon, 1 Nov 2021 13:30:03 -0400 Message-Id: <20211101173006.656673-20-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792534045100001 Content-Type: text/plain; charset="utf-8" AQMP likes to be very chatty about errors it encounters. In general, this is good because it allows us to get good diagnostic information for otherwise complex async failures. For example, during a failed QMP connection attempt, we might see: +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Negotiation failed: EOFError +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Failed to establish session: EOFE= rror This might be nice in iotests output, because failure scenarios involving the new QMP library will be spelled out plainly in the output diffs. For tests that are intentionally causing this scenario though, filtering that log output could be a hassle. For now, add a context manager that simply lets us toggle this output off during a critical region. (Additionally, a forthcoming patch allows the use of either legacy or async QMP to be toggled with an environment variable. In this circumstance, we can't amend the iotest output to just always expect the error message, either. Just suppress it for now. More rigorous log filtering can be investigated later if/when it is deemed safe to permanently replace the legacy QMP library.) Signed-off-by: John Snow Reviewed-by: Hanna Reitz Reviewed-by: Kevin Wolf Message-id: 20211026175612.4127598-6-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 20 +++++++++++++++++++- tests/qemu-iotests/tests/mirror-top-perms | 12 ++++++++---- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index e5fff6ddcfc..e2f9d873ada 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -30,7 +30,7 @@ import subprocess import sys import time -from typing import (Any, Callable, Dict, Iterable, +from typing import (Any, Callable, Dict, Iterable, Iterator, List, Optional, Sequence, TextIO, Tuple, Type, TypeVar) import unittest =20 @@ -114,6 +114,24 @@ sample_img_dir =3D os.environ['SAMPLE_IMG_DIR'] =20 =20 +@contextmanager +def change_log_level( + logger_name: str, level: int =3D logging.CRITICAL) -> Iterator[Non= e]: + """ + Utility function for temporarily changing the log level of a logger. + + This can be used to silence errors that are expected or uninteresting. + """ + _logger =3D logging.getLogger(logger_name) + current_level =3D _logger.level + _logger.setLevel(level) + + try: + yield + finally: + _logger.setLevel(current_level) + + def unarchive_sample_image(sample, fname): sample_fname =3D os.path.join(sample_img_dir, sample + '.bz2') with bz2.open(sample_fname) as f_in, open(fname, 'wb') as f_out: diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests= /tests/mirror-top-perms index a2d5c269d7a..0a51a613f39 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -26,7 +26,7 @@ from qemu.machine import machine from qemu.qmp import QMPConnectError =20 import iotests -from iotests import qemu_img +from iotests import change_log_level, qemu_img =20 =20 image_size =3D 1 * 1024 * 1024 @@ -100,9 +100,13 @@ class TestMirrorTopPerms(iotests.QMPTestCase): self.vm_b.add_blockdev(f'file,node-name=3Ddrive0,filename=3D{sourc= e}') self.vm_b.add_device('virtio-blk,drive=3Ddrive0,share-rw=3Don') try: - self.vm_b.launch() - print('ERROR: VM B launched successfully, this should not have= ' - 'happened') + # Silence AQMP errors temporarily. + # TODO: Remove this and just allow the errors to be logged when + # AQMP fully replaces QMP. + with change_log_level('qemu.aqmp'): + self.vm_b.launch() + print('ERROR: VM B launched successfully, ' + 'this should not have happened') except (QMPConnectError, ConnectError): assert 'Is another process using the image' in self.vm_b.get_l= og() =20 --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635792236778453.62636097094; Mon, 1 Nov 2021 11:43:56 -0700 (PDT) Received: from localhost ([::1]:50756 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcHj-00083e-Pm for importer@patchew.org; Mon, 01 Nov 2021 14:43:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34406) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbGB-00069x-EF for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:38:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:58816) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbG9-0004u5-La for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:38:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-20-SKEfSxySMr6dTIDiMWQWFg-1; Mon, 01 Nov 2021 13:38:09 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5CD46802682; Mon, 1 Nov 2021 17:38:07 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id E161019C79; Mon, 1 Nov 2021 17:37:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788293; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BnywYymkfvQbwW9WfZ/X0bSZkpmKwupdobXI+tiVyVQ=; b=Mj8QpyVNipAaQFBgzsB+n0xjDa6oJ7rDVu+9eLI/aDAqp3MTXqFRlitEYdhEYk4HvD31p9 E6P+dZZbYX/GXElLYz0hxTF4kEnXHfs1kYAvZzgemlZQhcNSQRNAMopKjDeAeOIffWTW13 hgW43UEkGnXdhYiDVNCDRMNOf0feiUw= X-MC-Unique: SKEfSxySMr6dTIDiMWQWFg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 20/22] iotests/300: avoid abnormal shutdown race condition Date: Mon, 1 Nov 2021 13:30:04 -0400 Message-Id: <20211101173006.656673-21-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635792237215100006 Content-Type: text/plain; charset="utf-8" Wait for the destination VM to close itself instead of racing to shut it down first, which produces different error log messages from AQMP depending on precisely when we tried to shut it down. (For example: We may try to issue 'quit' immediately prior to the target VM closing its QMP socket, which will cause an ECONNRESET error to be logged. Waiting for the VM to exit itself avoids the race on shutdown behavior.) Reported-by: Hanna Reitz Signed-off-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Hanna Reitz Message-id: 20211026175612.4127598-7-jsnow@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/300 | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300 index 10f9f2a8da6..dbd28384ec3 100755 --- a/tests/qemu-iotests/300 +++ b/tests/qemu-iotests/300 @@ -24,8 +24,6 @@ import random import re from typing import Dict, List, Optional =20 -from qemu.machine import machine - import iotests =20 =20 @@ -461,12 +459,11 @@ class TestBlockBitmapMappingErrors(TestDirtyBitmapMig= ration): f"'{self.src_node_name}': Name is longer than 255 by= tes", log) =20 - # Expect abnormal shutdown of the destination VM because of - # the failed migration - try: - self.vm_b.shutdown() - except machine.AbnormalShutdown: - pass + # Destination VM will terminate w/ error of its own accord + # due to the failed migration. + self.vm_b.wait() + rc =3D self.vm_b.exitcode() + assert rc is not None and rc > 0 =20 def test_aliased_bitmap_name_too_long(self) -> None: # Longer than the maximum for bitmap names --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635794934748258.797546618404; Mon, 1 Nov 2021 12:28:54 -0700 (PDT) Received: from localhost ([::1]:39992 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhczF-0004JE-QK for importer@patchew.org; Mon, 01 Nov 2021 15:28:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbHN-0006tE-Cr for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:39:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:60614) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbHI-0004zU-Ev for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:39:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-105-Q63GCEUsPUiDVvzSn44gTQ-1; Mon, 01 Nov 2021 13:39:20 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EB6E11030C23; Mon, 1 Nov 2021 17:38:54 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id D925019C79; Mon, 1 Nov 2021 17:38:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wyx64pleORoLQwf1Ou3jTAQ4rFCRiKLbPxAZZhkT94I=; b=if08JlpJQa8z3W0gfJ5N0j8hv5OBC6/AfFdK/8DZaG8ZUQr/eImJSvQxguoryiK+zUBsjs PerlxLpSyp597cKKNXDMstUv6WGa1aoravZQ4bV0V1ctA+meXNJjGCqeeinEfRfg3SmCU8 AGMb3Zm4P1ZN9kgjHhtTO+Muk9nHt3Q= X-MC-Unique: Q63GCEUsPUiDVvzSn44gTQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 21/22] python/aqmp: Create sync QMP wrapper for iotests Date: Mon, 1 Nov 2021 13:30:05 -0400 Message-Id: <20211101173006.656673-22-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635794936492100001 Content-Type: text/plain; charset="utf-8" This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John Snow Acked-by: Hanna Reitz Reviewed-by: Kevin Wolf Reviewed-by: Hanna Reitz Message-id: 20211026175612.4127598-8-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/aqmp/legacy.py | 138 +++++++++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100644 python/qemu/aqmp/legacy.py diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py new file mode 100644 index 00000000000..9e7b9fb80b9 --- /dev/null +++ b/python/qemu/aqmp/legacy.py @@ -0,0 +1,138 @@ +""" +Sync QMP Wrapper + +This class pretends to be qemu.qmp.QEMUMonitorProtocol. +""" + +import asyncio +from typing import ( + Awaitable, + List, + Optional, + TypeVar, + Union, +) + +import qemu.qmp +from qemu.qmp import QMPMessage, QMPReturnValue, SocketAddrT + +from .qmp_client import QMPClient + + +# pylint: disable=3Dmissing-docstring + + +class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol): + def __init__(self, address: SocketAddrT, + server: bool =3D False, + nickname: Optional[str] =3D None): + + # pylint: disable=3Dsuper-init-not-called + self._aqmp =3D QMPClient(nickname) + self._aloop =3D asyncio.get_event_loop() + self._address =3D address + self._timeout: Optional[float] =3D None + + _T =3D TypeVar('_T') + + def _sync( + self, future: Awaitable[_T], timeout: Optional[float] =3D None + ) -> _T: + return self._aloop.run_until_complete( + asyncio.wait_for(future, timeout=3Dtimeout) + ) + + def _get_greeting(self) -> Optional[QMPMessage]: + if self._aqmp.greeting is not None: + # pylint: disable=3Dprotected-access + return self._aqmp.greeting._asdict() + return None + + # __enter__ and __exit__ need no changes + # parse_address needs no changes + + def connect(self, negotiate: bool =3D True) -> Optional[QMPMessage]: + self._aqmp.await_greeting =3D negotiate + self._aqmp.negotiate =3D negotiate + + self._sync( + self._aqmp.connect(self._address) + ) + return self._get_greeting() + + def accept(self, timeout: Optional[float] =3D 15.0) -> QMPMessage: + self._aqmp.await_greeting =3D True + self._aqmp.negotiate =3D True + + self._sync( + self._aqmp.accept(self._address), + timeout + ) + + ret =3D self._get_greeting() + assert ret is not None + return ret + + def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: + return dict( + self._sync( + # pylint: disable=3Dprotected-access + + # _raw() isn't a public API, because turning off + # automatic ID assignment is discouraged. For + # compatibility with iotests *only*, do it anyway. + self._aqmp._raw(qmp_cmd, assign_id=3DFalse), + self._timeout + ) + ) + + # Default impl of cmd() delegates to cmd_obj + + def command(self, cmd: str, **kwds: object) -> QMPReturnValue: + return self._sync( + self._aqmp.execute(cmd, kwds), + self._timeout + ) + + def pull_event(self, + wait: Union[bool, float] =3D False) -> Optional[QMPMess= age]: + if not wait: + # wait is False/0: "do not wait, do not except." + if self._aqmp.events.empty(): + return None + + # If wait is 'True', wait forever. If wait is False/0, the events + # queue must not be empty; but it still needs some real amount + # of time to complete. + timeout =3D None + if wait and isinstance(wait, float): + timeout =3D wait + + return dict( + self._sync( + self._aqmp.events.get(), + timeout + ) + ) + + def get_events(self, wait: Union[bool, float] =3D False) -> List[QMPMe= ssage]: + events =3D [dict(x) for x in self._aqmp.events.clear()] + if events: + return events + + event =3D self.pull_event(wait) + return [event] if event is not None else [] + + def clear_events(self) -> None: + self._aqmp.events.clear() + + def close(self) -> None: + self._sync( + self._aqmp.disconnect() + ) + + def settimeout(self, timeout: Optional[float]) -> None: + self._timeout =3D timeout + + def send_fd_scm(self, fd: int) -> None: + self._aqmp.send_fd_scm(fd) --=20 2.31.1 From nobody Sun Feb 8 16:34:25 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635793148838858.5520405726768; Mon, 1 Nov 2021 11:59:08 -0700 (PDT) Received: from localhost ([::1]:35936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhcWR-0004pF-KH for importer@patchew.org; Mon, 01 Nov 2021 14:59:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34690) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbHN-0006tG-Eh for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:39:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25622) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhbHK-0004ze-Jt for qemu-devel@nongnu.org; Mon, 01 Nov 2021 13:39:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-496-YSDovnKNOx6_rUrykxH7Gg-1; Mon, 01 Nov 2021 13:39:21 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E50ED102ABE5; Mon, 1 Nov 2021 17:39:07 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id CD80919C79; Mon, 1 Nov 2021 17:38:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635788364; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5GASA0WtcVteKHrOt5sqNIwhFNEB+mMyVD9X+r15K2E=; b=V9xwQcJw3KrGkTxm9pOjxwDtVjJRAik2MNNGTysGMe2ipCij09/wl2tuZMGR/xVTgAEReF RquLxAXmzrzYpOpsSHOkYOyTyMPbKvuMRUi3KpxDI9Bs/83gtJc7LhmXS6vB1cf/COS8+z lzi2Cp3MipaQc9Pobb89fGNqGg7xWh0= X-MC-Unique: YSDovnKNOx6_rUrykxH7Gg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PULL 22/22] python, iotests: replace qmp with aqmp Date: Mon, 1 Nov 2021 13:30:06 -0400 Message-Id: <20211101173006.656673-23-jsnow@redhat.com> In-Reply-To: <20211101173006.656673-1-jsnow@redhat.com> References: <20211101173006.656673-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) DKIMWL_WL_HIGH=-0.734, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Thomas Huth , Vladimir Sementsov-Ogievskiy , Daniel Berrange , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Alex=20Benn=C3=A9e?= , Markus Armbruster , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635793150286100001 Content-Type: text/plain; charset="utf-8" Swap out the synchronous QEMUMonitorProtocol from qemu.qmp with the sync wrapper from qemu.aqmp instead. Add an escape hatch in the form of the environment variable QEMU_PYTHON_LEGACY_QMP which allows you to cajole QEMUMachine into using the old implementation, proving that both implementations work concurrently. Signed-off-by: John Snow Reviewed-by: Kevin Wolf Reviewed-by: Hanna Reitz Message-id: 20211026175612.4127598-9-jsnow@redhat.com Signed-off-by: John Snow --- python/qemu/machine/machine.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index a0cf69786b4..a487c397459 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -41,7 +41,6 @@ ) =20 from qemu.qmp import ( # pylint: disable=3Dimport-error - QEMUMonitorProtocol, QMPMessage, QMPReturnValue, SocketAddrT, @@ -50,6 +49,12 @@ from . import console_socket =20 =20 +if os.environ.get('QEMU_PYTHON_LEGACY_QMP'): + from qemu.qmp import QEMUMonitorProtocol +else: + from qemu.aqmp.legacy import QEMUMonitorProtocol + + LOG =3D logging.getLogger(__name__) =20 =20 --=20 2.31.1