From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765536103732.734880758323; Wed, 15 Sep 2021 21:12:16 -0700 (PDT) Received: from localhost ([::1]:42496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQikw-0000g4-4P for importer@patchew.org; Thu, 16 Sep 2021 00:12:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34850) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQiix-0006hK-Na for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60155) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQiit-0000NL-SY for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:09 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-514-Drp_mdV2PMagNN7m9xbfLA-1; Thu, 16 Sep 2021 00:10:03 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 05D32824FA6; Thu, 16 Sep 2021 04:10:03 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7967D19C79; Thu, 16 Sep 2021 04:10:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765406; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0p40qM0IN9RsRuikNIh7r24vKS+fPI8oVNzJ8H4Ia6Y=; b=gtivDyddz5dFKN4olyGcqDAYD2/PdWB/MrrRVCZ6HVJoiWGQhaaSkjtfCtbvEGYyNmdRYq GKDfzI6Tz8F92UKusk/R6OxP003s8+felPhZaVdctSreNPsG0Xsvn8WMdXnW7LPcJ8ljmI 7f8avAHelVIhiCPsz8FZ1GTZR0nNsvk= X-MC-Unique: Drp_mdV2PMagNN7m9xbfLA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 01/16] python: Update for pylint 2.10 Date: Thu, 16 Sep 2021 00:09:40 -0400 Message-Id: <20210916040955.628560-2-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765537543100001 Content-Type: text/plain; charset="utf-8" A few new annoyances. Of note is the new warning for an unspecified encoding when opening a text file, which actually does indicate a potentially real problem; see https://www.python.org/dev/peps/pep-0597/#motivation Use LC_CTYPE to determine an encoding to use for interpreting QEMU's terminal output. Note that Python states: "language code and encoding may be None if their values cannot be determined" -- use a platform default as a backup. Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e --- python/qemu/machine/machine.py | 9 ++++++++- python/setup.cfg | 1 + 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index a7081b1845..51b6e79a13 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -19,6 +19,7 @@ =20 import errno from itertools import chain +import locale import logging import os import shutil @@ -290,8 +291,14 @@ def get_pid(self) -> Optional[int]: return self._subp.pid =20 def _load_io_log(self) -> None: + # Assume that the output encoding of QEMU's terminal output + # is defined by our locale. If indeterminate, use a platform defau= lt. + _, encoding =3D locale.getlocale() + if encoding is None: + encoding =3D locale.getpreferredencoding(do_setlocale=3DFalse) if self._qemu_log_path is not None: - with open(self._qemu_log_path, "r") as iolog: + with open(self._qemu_log_path, "r", + encoding=3Dencoding) as iolog: self._iolog =3D iolog.read() =20 @property diff --git a/python/setup.cfg b/python/setup.cfg index 83909c1c97..0f0cab098f 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -104,6 +104,7 @@ good-names=3Di, [pylint.similarities] # Ignore imports when computing similarities. ignore-imports=3Dyes +ignore-signatures=3Dyes =20 # Minimum lines number of a similarity. # TODO: Remove after we opt in to Pylint 2.8.3. See commit msg. --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765715882985.8452984006859; Wed, 15 Sep 2021 21:15:15 -0700 (PDT) Received: from localhost ([::1]:48944 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQinq-00050F-Qt for importer@patchew.org; Thu, 16 Sep 2021 00:15:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34968) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij0-0006ks-Ep for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20794) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQiiw-0000Qd-0A for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:14 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-550-kal3aM32POOIOb-GO2tzng-1; Thu, 16 Sep 2021 00:10:06 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 089D2824FA6; Thu, 16 Sep 2021 04:10:05 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 536DB1972E; Thu, 16 Sep 2021 04:10:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qb+BprQ6x64UEBCxXFcet53dsFSAz3TUduL2t+PB6Uc=; b=gO2wzCh8vKCKtlO+pzgNc9zUUuVdLon0VsWYXXcFQk7PjakxvA5CiDKGYoV1M3acazz4Zc oIcych0zjhA4SqBYebNMr6Cw5Q2+saejpj/Jjz6F+8dAPSJYdl9XEteQu1e76p/XRvKoZG ev1+QrMX4VtXiavMYJacvkAcBjgUEXc= X-MC-Unique: kal3aM32POOIOb-GO2tzng-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 02/16] iotests/mirror-top-perms: Adjust imports Date: Thu, 16 Sep 2021 00:09:41 -0400 Message-Id: <20210916040955.628560-3-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765716695100001 Content-Type: text/plain; charset="utf-8" We need to import things from the qemu namespace; importing the namespace alone doesn't bring the submodules with it -- unless someone else (like iotests.py) imports them too. Adjust the imports. Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/tests/mirror-top-perms | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests= /tests/mirror-top-perms index 2fc8dd66e0..de18182590 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -25,7 +25,8 @@ from iotests import qemu_img =20 # Import qemu after iotests.py has amended sys.path # pylint: disable=3Dwrong-import-order -import qemu +from qemu import qmp +from qemu.machine import machine =20 =20 image_size =3D 1 * 1024 * 1024 @@ -47,7 +48,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase): def tearDown(self): try: self.vm.shutdown() - except qemu.machine.machine.AbnormalShutdown: + except machine.AbnormalShutdown: pass =20 if self.vm_b is not None: @@ -102,7 +103,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase): self.vm_b.launch() print('ERROR: VM B launched successfully, this should not have= ' 'happened') - except qemu.qmp.QMPConnectError: + except qmp.QMPConnectError: assert 'Is another process using the image' in self.vm_b.get_l= og() =20 result =3D self.vm.qmp('block-job-cancel', --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765555564510.1765161478355; Wed, 15 Sep 2021 21:12:35 -0700 (PDT) Received: from localhost ([::1]:42982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQilG-00010N-IA for importer@patchew.org; Thu, 16 Sep 2021 00:12:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34970) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij0-0006kw-G5 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41491) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQiix-0000RX-6Z for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:14 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-405-D2YOiTzeOeyvE3tyX7ep4g-1; Thu, 16 Sep 2021 00:10:08 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A4DEE1084686; Thu, 16 Sep 2021 04:10:07 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id CA02D19C79; Thu, 16 Sep 2021 04:10:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765410; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YLoSI5TVb5BhA+3TAWXlCgywUmuQZyCj+Q7q7kuq3aA=; b=EWn5sWeGARt3b0ZZhFHmdjPvxgwMVEJqko4qNgx+4B0Jl472F3ErPlrkg7C8UaUQc383Gf ccmaUFNcNruibfXYHZNaCUYs+Dx85kdso80cz0f5vSMSZVPGUPUm0naLZnTFgAMsJDo13r 25QvFnn+i+qHZkbkoyg2tze9VryC2l8= X-MC-Unique: D2YOiTzeOeyvE3tyX7ep4g-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 03/16] iotests/migrate-bitmaps-postcopy-test: declare instance variables Date: Thu, 16 Sep 2021 00:09:42 -0400 Message-Id: <20210916040955.628560-4-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765557842100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> --- tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test b/tests= /qemu-iotests/tests/migrate-bitmaps-postcopy-test index 00ebb5c251..9c00ae61c8 100755 --- a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test +++ b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test @@ -115,6 +115,9 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): self.vm_a_events =3D [] self.vm_b_events =3D [] =20 + self.discards1_sha256: str + self.all_discards_sha256: str + def start_postcopy(self): """ Run migration until RESUME event on target. Return this event.= """ for i in range(nb_bitmaps): --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765763774562.0245573914417; Wed, 15 Sep 2021 21:16:03 -0700 (PDT) Received: from localhost ([::1]:50094 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQioc-0005mO-PF for importer@patchew.org; Thu, 16 Sep 2021 00:16:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34984) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij1-0006oS-G4 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:26043) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQiiy-0000Ta-PF for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-92-acDPZEKnNN6boe9mJmwqKQ-1; Thu, 16 Sep 2021 00:10:10 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7C282801E72; Thu, 16 Sep 2021 04:10:09 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id E312619C79; Thu, 16 Sep 2021 04:10:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765412; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FToxD/qVgRUWi38kDsj7ZMrg72iH2FFhHpz6YePmeqw=; b=AmS2gfyRcpFKZq9y93IiUT8rncWVT6Ci4+t47ZNCuJdyR/OgolwVVE93UPKj40S56RLW5V SvK1mdQ8TwH/mFRA5VMJKyyOwCXQs0AmlDMaRCQSIeEGRW141tj223i2ddvrSA/LuwMRGh IV0G3+/DnVqUJYuiOaQZR/DsIPUq8oc= X-MC-Unique: acDPZEKnNN6boe9mJmwqKQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 04/16] iotests/migrate-bitmaps-test: delint Date: Thu, 16 Sep 2021 00:09:43 -0400 Message-Id: <20210916040955.628560-5-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765765903100001 Content-Type: text/plain; charset="utf-8" Mostly uninteresting stuff. Move the test injections under a function named main() so that the variables used during that process aren't in the global scope. Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/tests/migrate-bitmaps-test | 50 +++++++++++-------- 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-test b/tests/qemu-iot= ests/tests/migrate-bitmaps-test index dc431c35b3..c23df3d75c 100755 --- a/tests/qemu-iotests/tests/migrate-bitmaps-test +++ b/tests/qemu-iotests/tests/migrate-bitmaps-test @@ -19,10 +19,11 @@ # along with this program. If not, see . # =20 -import os import itertools import operator +import os import re + import iotests from iotests import qemu_img, qemu_img_create, Timeout =20 @@ -224,25 +225,6 @@ def inject_test_case(klass, suffix, method, *args, **k= wargs): setattr(klass, 'test_' + method + suffix, lambda self: mc(self)) =20 =20 -for cmb in list(itertools.product((True, False), repeat=3D5)): - name =3D ('_' if cmb[0] else '_not_') + 'persistent_' - name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap_' - name +=3D '_online' if cmb[2] else '_offline' - name +=3D '_shared' if cmb[3] else '_nonshared' - if cmb[4]: - name +=3D '__pre_shutdown' - - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', - *list(cmb)) - -for cmb in list(itertools.product((True, False), repeat=3D2)): - name =3D ('_' if cmb[0] else '_not_') + 'persistent_' - name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap' - - inject_test_case(TestDirtyBitmapMigration, name, - 'do_test_migration_resume_source', *list(cmb)) - - class TestDirtyBitmapBackingMigration(iotests.QMPTestCase): def setUp(self): qemu_img_create('-f', iotests.imgfmt, base_a, size) @@ -304,6 +286,30 @@ class TestDirtyBitmapBackingMigration(iotests.QMPTestC= ase): self.assert_qmp(result, 'return', {}) =20 =20 +def main() -> None: + for cmb in list(itertools.product((True, False), repeat=3D5)): + name =3D ('_' if cmb[0] else '_not_') + 'persistent_' + name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap_' + name +=3D '_online' if cmb[2] else '_offline' + name +=3D '_shared' if cmb[3] else '_nonshared' + if cmb[4]: + name +=3D '__pre_shutdown' + + inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migratio= n', + *list(cmb)) + + for cmb in list(itertools.product((True, False), repeat=3D2)): + name =3D ('_' if cmb[0] else '_not_') + 'persistent_' + name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap' + + inject_test_case(TestDirtyBitmapMigration, name, + 'do_test_migration_resume_source', *list(cmb)) + + iotests.main( + supported_fmts=3D['qcow2'], + supported_protocols=3D['file'] + ) + + if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2'], - supported_protocols=3D['file']) + main() --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765987966610.6717390524497; Wed, 15 Sep 2021 21:19:47 -0700 (PDT) Received: from localhost ([::1]:57100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQisC-00028Q-4G for importer@patchew.org; Thu, 16 Sep 2021 00:19:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35030) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij7-000704-03 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:54639) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij2-0000Wk-1n for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:20 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-393-Krg-4ZdMOlubqCAQGYreWQ-1; Thu, 16 Sep 2021 00:10:12 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 50F6F801B3D; Thu, 16 Sep 2021 04:10:11 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id ABC5319C79; Thu, 16 Sep 2021 04:10:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765415; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=US60V94xYr0YwASCa+Z1LSb/8isp127gJbojQVNlda4=; b=HBg5G/LldPp5ELhhk+V9QyrhPw0Tj2JWdBUcDlUDAP3MOCC9z0R5yTs6cbHN84ZxI6YWT5 em8/4R2eO/wN71TCrpuFeDQbxsBplPjCHgx6JEuJ2fRaCNxVhuHiMV9NQ651IQ2AXLa7O7 SpW52d+6vtwM6/WqUOGnFCzPb+mZP+A= X-MC-Unique: Krg-4ZdMOlubqCAQGYreWQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 05/16] iotests/297: modify is_python_file to work from any CWD Date: Thu, 16 Sep 2021 00:09:44 -0400 Message-Id: <20210916040955.628560-6-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765990224100001 Content-Type: text/plain; charset="utf-8" Add a directory argument to is_python_file to allow it to work correctly no matter what CWD we happen to run it from. This is done in anticipation of running the iotests from another directory (./python/). Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b04cba5366..3e86f788fc 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -39,14 +39,16 @@ SKIP_FILES =3D ( ) =20 =20 -def is_python_file(filename): - if not os.path.isfile(filename): +def is_python_file(filename: str, directory: str =3D '.') -> bool: + filepath =3D os.path.join(directory, filename) + + if not os.path.isfile(filepath): return False =20 if filename.endswith('.py'): return True =20 - with open(filename, encoding=3D'utf-8') as f: + with open(filepath, encoding=3D'utf-8') as f: try: first_line =3D f.readline() return re.match('^#!.*python', first_line) is not None --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631766180733679.3328582720375; Wed, 15 Sep 2021 21:23:00 -0700 (PDT) Received: from localhost ([::1]:34590 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQivL-00063q-F9 for importer@patchew.org; Thu, 16 Sep 2021 00:22:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij7-00071q-Fv for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:30982) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij4-0000aR-QI for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:21 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-27-7m7qt3g5O5q2c0-Nrm-KlQ-1; Thu, 16 Sep 2021 00:10:17 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2C389362FB; Thu, 16 Sep 2021 04:10:16 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 92AD719C79; Thu, 16 Sep 2021 04:10:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765418; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZiahqh99EnnpzzYlCIOhDSsZBcXqmlZj6Mfz46bLbw=; b=IJojKF7NLz0UbtJSwrLgRO2XK1MznDHWoOvwMBBYcVbUlZJFOYnw1mUIHoj/bgXms9GQb9 E7SUvmx4Zdh/D00iJI3NQLM/OvLO7o/eNsgjY5dZJ3L+VAltzgM/t5u0hKVJLJpBN60AD5 UkTvEHfaUETSF/cTew8xNHhV1Ndokws= X-MC-Unique: 7m7qt3g5O5q2c0-Nrm-KlQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 06/16] iotests/297: Add get_files() function Date: Thu, 16 Sep 2021 00:09:45 -0400 Message-Id: <20210916040955.628560-7-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766181489100001 Content-Type: text/plain; charset="utf-8" Split out file discovery into its own method to begin separating out the "environment setup" and "test execution" phases. Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 3e86f788fc..b3a56a3a29 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,6 +21,7 @@ import re import shutil import subprocess import sys +from typing import List =20 import iotests =20 @@ -56,10 +57,15 @@ def is_python_file(filename: str, directory: str =3D '.= ') -> bool: return False =20 =20 +def get_test_files(directory: str =3D '.') -> List[str]: + named_test_dir =3D os.path.join(directory, 'tests') + named_tests =3D [f"tests/{entry}" for entry in os.listdir(named_test_d= ir)] + check_tests =3D set(os.listdir(directory) + named_tests) - set(SKIP_FI= LES) + return list(filter(lambda f: is_python_file(f, directory), check_tests= )) + + def run_linters(): - named_tests =3D [f'tests/{entry}' for entry in os.listdir('tests')] - check_tests =3D set(os.listdir('.') + named_tests) - set(SKIP_FILES) - files =3D [filename for filename in check_tests if is_python_file(file= name)] + files =3D get_test_files() =20 iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631766317758145.79604161792201; Wed, 15 Sep 2021 21:25:17 -0700 (PDT) Received: from localhost ([::1]:39286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQixX-0000la-Bf for importer@patchew.org; Thu, 16 Sep 2021 00:25:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35066) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijA-0007Am-D4 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36648) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQij8-0000fi-QG for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:24 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-526-CJcbzZiMNIyBwkubNS3grA-1; Thu, 16 Sep 2021 00:10:19 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3486410066F3; Thu, 16 Sep 2021 04:10:18 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8905519C79; Thu, 16 Sep 2021 04:10:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765422; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u0a1iac4nTpG7FZF61Ypmp0dRtpDg9EXplY2A87ngkw=; b=HqygU3VMBNqMUAXWKD+Uiw/bNGM3e1SAfspsGZ65Gg9Kt+uWKd4Ikw3B+hN3vDLTFjBEmP 1Tbqz7aMCvLHe1VqRiAPopLn4eHvrJZzIP8pyhXN64Xur+0Nkm8eps2O9ZcCpWoQ4tYXxe 46jGIBcWQyLm6iPJXLjKmoXDvyqe6Nk= X-MC-Unique: CJcbzZiMNIyBwkubNS3grA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 07/16] iotests/297: Don't rely on distro-specific linter binaries Date: Thu, 16 Sep 2021 00:09:46 -0400 Message-Id: <20210916040955.628560-8-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766319348100001 'pylint-3' is another Fedora-ism. Use "python3 -m pylint" or "python3 -m mypy" to access these scripts instead. This style of invocation will prefer the "correct" tool when run in a virtual environment. Note that we still check for "pylint-3" before the test begins -- this check is now "overly strict", but shouldn't cause anything that was already running correctly to start failing. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Philippe Mathieu-Daud=C3=A9 Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 45 ++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b3a56a3a29..01dd8147d5 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -82,8 +82,11 @@ def run_linters(): env['PYTHONPATH'] +=3D os.pathsep + qemu_module_path except KeyError: env['PYTHONPATH'] =3D qemu_module_path - subprocess.run(('pylint-3', '--score=3Dn', '--notes=3DFIXME,XXX', *fil= es), - env=3Denv, check=3DFalse) + subprocess.run( + ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), + env=3Denv, + check=3DFalse, + ) =20 print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() @@ -94,23 +97,27 @@ def run_linters(): # must not both define the __main__ module). env['MYPYPATH'] =3D env['PYTHONPATH'] for filename in files: - p =3D subprocess.run(('mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - filename), - env=3Denv, - check=3DFalse, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.STDOUT, - universal_newlines=3DTrue) + p =3D subprocess.run( + ( + 'python3', '-m', 'mypy', + '--warn-unused-configs', + '--disallow-subclassing-any', + '--disallow-any-generics', + '--disallow-incomplete-defs', + '--disallow-untyped-decorators', + '--no-implicit-optional', + '--warn-redundant-casts', + '--warn-unused-ignores', + '--no-implicit-reexport', + '--namespace-packages', + filename, + ), + env=3Denv, + check=3DFalse, + stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT, + universal_newlines=3DTrue + ) =20 if p.returncode !=3D 0: print(p.stdout) --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631766194302186.5570599982841; Wed, 15 Sep 2021 21:23:14 -0700 (PDT) Received: from localhost ([::1]:34922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQivZ-0006Hr-9S for importer@patchew.org; Thu, 16 Sep 2021 00:23:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35164) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijF-0007UQ-OR for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:52509) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijB-0000iS-66 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:29 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-241-pNMsioN3Psmw8mR4MYvrzA-1; Thu, 16 Sep 2021 00:10:21 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 11524800FF4; Thu, 16 Sep 2021 04:10:20 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D59219C79; Thu, 16 Sep 2021 04:10:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rkx+rShWvh0w/DkMe0tSVGEM2kXq31yF7p46TKPe7c0=; b=duUEyFLHfDBRUon/z/i/fo4Yvrr38vPmZadm+ClLBx9yx20beKO0pVBusqUdgCF/delRyR PgbMPLATSCSKvZAdJhrmyUQ6LIZJo9wNRDMuwE+2JAk7jVvS3gtQmoGcLeV5OmolgncRTt qmThBXvoHRFdXmrz32ODfyCYsM+p6Z4= X-MC-Unique: pNMsioN3Psmw8mR4MYvrzA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 08/16] iotests/297: Create main() function Date: Thu, 16 Sep 2021 00:09:47 -0400 Message-Id: <20210916040955.628560-9-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766196058100001 Content-Type: text/plain; charset="utf-8" Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/297 | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 01dd8147d5..6df952808d 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -123,8 +123,12 @@ def run_linters(): print(p.stdout) =20 =20 -for linter in ('pylint-3', 'mypy'): - if shutil.which(linter) is None: - iotests.notrun(f'{linter} not found') +def main() -> None: + for linter in ('pylint-3', 'mypy'): + if shutil.which(linter) is None: + iotests.notrun(f'{linter} not found') =20 -iotests.script_main(run_linters) + run_linters() + + +iotests.script_main(main) --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631766515140307.36047065846583; Wed, 15 Sep 2021 21:28:35 -0700 (PDT) Received: from localhost ([::1]:45318 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQj0k-0004wC-4o for importer@patchew.org; Thu, 16 Sep 2021 00:28:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijE-0007P9-F5 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:59667) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijB-0000ib-Bu for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-46-KB3Z0vhkPJWRIaGuBnIAcQ-1; Thu, 16 Sep 2021 00:10:22 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DDAD01808305; Thu, 16 Sep 2021 04:10:21 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 54EFF19C79; Thu, 16 Sep 2021 04:10:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1hzmYJCwUpJlt7OQgJoE7M+hEI680jZoYzlYJoPZIGU=; b=PmcczAA5cG6ISPYAGGMurELmhlOcAybl/Wn8HmXrE6/epH2JBKbtLkB+heyFcP7vCOh2hx pbbRidhfCl2lJDnLJlqz+3DYZsmRgQOaIZRLsac4bcLB41UEjAUe0MA5Rb0TKVgthJITxw nBcnZ7Q9BbfKNlHvB4pePAQ1wqh122E= X-MC-Unique: KB3Z0vhkPJWRIaGuBnIAcQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 09/16] iotests/297: Separate environment setup from test execution Date: Thu, 16 Sep 2021 00:09:48 -0400 Message-Id: <20210916040955.628560-10-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766516422100001 Content-Type: text/plain; charset="utf-8" Move environment setup into main(), leaving pure test execution behind in run_linters(). Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 6df952808d..08d2b87108 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,7 +21,7 @@ import re import shutil import subprocess import sys -from typing import List +from typing import List, Mapping, Optional =20 import iotests =20 @@ -64,24 +64,16 @@ def get_test_files(directory: str =3D '.') -> List[str]: return list(filter(lambda f: is_python_file(f, directory), check_tests= )) =20 =20 -def run_linters(): - files =3D get_test_files() - - iotests.logger.debug('Files to be checked:') - iotests.logger.debug(', '.join(sorted(files))) +def run_linters( + files: List[str], + env: Optional[Mapping[str, str]] =3D None, +) -> None: =20 print('=3D=3D=3D pylint =3D=3D=3D') sys.stdout.flush() =20 # Todo notes are fine, but fixme's or xxx's should probably just be # fixed (in tests, at least) - env =3D os.environ.copy() - qemu_module_path =3D os.path.join(os.path.dirname(__file__), - '..', '..', 'python') - try: - env['PYTHONPATH'] +=3D os.pathsep + qemu_module_path - except KeyError: - env['PYTHONPATH'] =3D qemu_module_path subprocess.run( ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), env=3Denv, @@ -95,7 +87,6 @@ def run_linters(): # will interpret all given files as belonging together (i.e., they # may not both define the same classes, etc.; most notably, they # must not both define the __main__ module). - env['MYPYPATH'] =3D env['PYTHONPATH'] for filename in files: p =3D subprocess.run( ( @@ -128,7 +119,22 @@ def main() -> None: if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') =20 - run_linters() + files =3D get_test_files() + + iotests.logger.debug('Files to be checked:') + iotests.logger.debug(', '.join(sorted(files))) + + env =3D os.environ.copy() + qemu_module_path =3D os.path.join(os.path.dirname(__file__), + '..', '..', 'python') + try: + env['PYTHONPATH'] +=3D os.pathsep + qemu_module_path + except KeyError: + env['PYTHONPATH'] =3D qemu_module_path + + env['MYPYPATH'] =3D env['PYTHONPATH'] + + run_linters(files, env=3Denv) =20 =20 iotests.script_main(main) --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765984596134.0243107061185; Wed, 15 Sep 2021 21:19:44 -0700 (PDT) Received: from localhost ([::1]:56996 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQisB-00023w-94 for importer@patchew.org; Thu, 16 Sep 2021 00:19:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35146) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijE-0007PR-HW for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34318) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijC-0000kI-OK for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-216-gOvhuztqPVKCQfr9A8AjHQ-1; Thu, 16 Sep 2021 00:10:24 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A3275824FA7; Thu, 16 Sep 2021 04:10:23 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F64619C79; Thu, 16 Sep 2021 04:10:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C5bnYvaVaZR3Rc4sd0Frx86BC+9YsKCE/Qfue57FWLY=; b=ge5+of7E0aVieBOIjxQa974Nw6mHeZxTMj4Nq1sZTQSLoOhoT9VSyewzZNVWxqHVwYsrsm bUcaguHsWbSov8hgAdJQYu+LO+Q6eF+ni9tnlaI1xMVeEE7zDrmQUzY/cy2EdCIirFIgVx kUN+c0VGwo3KLdA1+ZHSfwT2+0i8IvQ= X-MC-Unique: gOvhuztqPVKCQfr9A8AjHQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 10/16] iotests/297: Add 'directory' argument to run_linters Date: Thu, 16 Sep 2021 00:09:49 -0400 Message-Id: <20210916040955.628560-11-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765986230100001 Content-Type: text/plain; charset="utf-8" Allow run_linters to work well if it's executed from a different directory. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 08d2b87108..e05c99972e 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -66,6 +66,7 @@ def get_test_files(directory: str =3D '.') -> List[str]: =20 def run_linters( files: List[str], + directory: str =3D '.', env: Optional[Mapping[str, str]] =3D None, ) -> None: =20 @@ -76,6 +77,7 @@ def run_linters( # fixed (in tests, at least) subprocess.run( ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), + cwd=3Ddirectory, env=3Denv, check=3DFalse, ) @@ -103,6 +105,7 @@ def run_linters( '--namespace-packages', filename, ), + cwd=3Ddirectory, env=3Denv, check=3DFalse, stdout=3Dsubprocess.PIPE, --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765733873903.4151121866212; Wed, 15 Sep 2021 21:15:33 -0700 (PDT) Received: from localhost ([::1]:49804 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQio8-0005aU-UC for importer@patchew.org; Thu, 16 Sep 2021 00:15:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35170) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijG-0007Y0-Lk for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:41747) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijF-0000lx-0F for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:30 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-292-EzWfdtZEOUWlKr3F3evKwg-1; Thu, 16 Sep 2021 00:10:26 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 79C09362F8; Thu, 16 Sep 2021 04:10:25 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 07E511972E; Thu, 16 Sep 2021 04:10:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765428; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EXSMN2cISA4KkMPi9qx2K1SzmOnDUsccxCFDZczj9FU=; b=SHmdimpeVMwiB3nU0wI9El3R7/qYWwmuTDuYfUzCwbuvJnFU0NxmfmXzxCt+XhNQqZ0LmR peZk3pv/VXFthvMs4gmZJFoVRMwl67rk0XKsgTQh2ecSfwzFaa6RHH5v4US62GYuwa26pS Cbox0OleDKaZ6p/ShfhLtJxcz9JapK8= X-MC-Unique: EzWfdtZEOUWlKr3F3evKwg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 11/16] iotests/297: return error code from run_linters() Date: Thu, 16 Sep 2021 00:09:50 -0400 Message-Id: <20210916040955.628560-12-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765735828100001 Content-Type: text/plain; charset="utf-8" This turns run_linters() into a bit of a hybrid test; returning non-zero on failed execution while also printing diffable information. This is done for the benefit of the avocado simple test runner, which will soon be attempting to execute this test from a different environment. (Note: universal_newlines is added to the pylint invocation for type consistency with the mypy run -- it's not strictly necessary, but it avoids some typing errors caused by our re-use of the 'p' variable.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Based-On: <20210915175318.853225-1-hreitz@redhat.com> --- tests/qemu-iotests/297 | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index e05c99972e..f9ddfb53a0 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -68,19 +68,22 @@ def run_linters( files: List[str], directory: str =3D '.', env: Optional[Mapping[str, str]] =3D None, -) -> None: +) -> int: + ret =3D 0 =20 print('=3D=3D=3D pylint =3D=3D=3D') sys.stdout.flush() =20 # Todo notes are fine, but fixme's or xxx's should probably just be # fixed (in tests, at least) - subprocess.run( + p =3D subprocess.run( ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), cwd=3Ddirectory, env=3Denv, check=3DFalse, + universal_newlines=3DTrue, ) + ret +=3D p.returncode =20 print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() @@ -113,9 +116,12 @@ def run_linters( universal_newlines=3DTrue ) =20 + ret +=3D p.returncode if p.returncode !=3D 0: print(p.stdout) =20 + return ret + =20 def main() -> None: for linter in ('pylint-3', 'mypy'): --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631765989005530.0984021223799; Wed, 15 Sep 2021 21:19:49 -0700 (PDT) Received: from localhost ([::1]:57300 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQisF-0002I2-Tk for importer@patchew.org; Thu, 16 Sep 2021 00:19:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35380) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijT-0007sL-Rl for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:51007) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijM-0000sO-H6 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:43 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-5-f5g7-JEyOtGdO7t7Xt3B2Q-1; Thu, 16 Sep 2021 00:10:28 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 58859801B3D; Thu, 16 Sep 2021 04:10:27 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF0E819C79; Thu, 16 Sep 2021 04:10:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765436; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T206XK0zbs2PAHZtEWEeelvImDHyMsx6iLFvbZRT79Y=; b=C8w0HKr/dBszIj5ohWDQtVqtK7ESVagViLVXiaTnn+rLw3JzjARVv90+xXjwmQExBCTUcb 7p1MuNn/VxsqU3O4lrNkpCAtLSAkY2D+DAY7hT8Ffd+7Y7tzrp8iVNxO4ElSaV4JkvLOFY 62NvFrrEoU8pMHig8yBRfNRp8B6EHbY= X-MC-Unique: f5g7-JEyOtGdO7t7Xt3B2Q-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 12/16] iotests/297: split linters.py off from 297 Date: Thu, 16 Sep 2021 00:09:51 -0400 Message-Id: <20210916040955.628560-13-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631765990282100003 Content-Type: text/plain; charset="utf-8" Split the linter execution itself out from 297, leaving just environment setup in 297. This is done so that non-iotest code can invoke the linters without needing to worry about imports of unpackaged iotest code. Eventually, it should be possible to replace linters.py with mypy.ini and pylintrc files that instruct these tools how to run properly in this directory, but ... not yet, and not in this series. Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> --- tests/qemu-iotests/297 | 105 ++---------------------------- tests/qemu-iotests/linters.py | 117 ++++++++++++++++++++++++++++++++++ 2 files changed, 121 insertions(+), 101 deletions(-) create mode 100755 tests/qemu-iotests/linters.py diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index f9ddfb53a0..3d29af5b78 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -17,110 +17,13 @@ # along with this program. If not, see . =20 import os -import re import shutil -import subprocess -import sys -from typing import List, Mapping, Optional =20 import iotests +import linters =20 =20 -# TODO: Empty this list! -SKIP_FILES =3D ( - '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', - '096', '118', '124', '132', '136', '139', '147', '148', '149', - '151', '152', '155', '163', '165', '194', '196', '202', - '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', - '218', '219', '224', '228', '234', '235', '236', '237', '238', - '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', - '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', - '299', '302', '303', '304', '307', - 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' -) - - -def is_python_file(filename: str, directory: str =3D '.') -> bool: - filepath =3D os.path.join(directory, filename) - - if not os.path.isfile(filepath): - return False - - if filename.endswith('.py'): - return True - - with open(filepath, encoding=3D'utf-8') as f: - try: - first_line =3D f.readline() - return re.match('^#!.*python', first_line) is not None - except UnicodeDecodeError: # Ignore binary files - return False - - -def get_test_files(directory: str =3D '.') -> List[str]: - named_test_dir =3D os.path.join(directory, 'tests') - named_tests =3D [f"tests/{entry}" for entry in os.listdir(named_test_d= ir)] - check_tests =3D set(os.listdir(directory) + named_tests) - set(SKIP_FI= LES) - return list(filter(lambda f: is_python_file(f, directory), check_tests= )) - - -def run_linters( - files: List[str], - directory: str =3D '.', - env: Optional[Mapping[str, str]] =3D None, -) -> int: - ret =3D 0 - - print('=3D=3D=3D pylint =3D=3D=3D') - sys.stdout.flush() - - # Todo notes are fine, but fixme's or xxx's should probably just be - # fixed (in tests, at least) - p =3D subprocess.run( - ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), - cwd=3Ddirectory, - env=3Denv, - check=3DFalse, - universal_newlines=3DTrue, - ) - ret +=3D p.returncode - - print('=3D=3D=3D mypy =3D=3D=3D') - sys.stdout.flush() - - # We have to call mypy separately for each file. Otherwise, it - # will interpret all given files as belonging together (i.e., they - # may not both define the same classes, etc.; most notably, they - # must not both define the __main__ module). - for filename in files: - p =3D subprocess.run( - ( - 'python3', '-m', 'mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - filename, - ), - cwd=3Ddirectory, - env=3Denv, - check=3DFalse, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.STDOUT, - universal_newlines=3DTrue - ) - - ret +=3D p.returncode - if p.returncode !=3D 0: - print(p.stdout) - - return ret +# Looking for the list of excluded tests? See linters.py ! =20 =20 def main() -> None: @@ -128,7 +31,7 @@ def main() -> None: if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') =20 - files =3D get_test_files() + files =3D linters.get_test_files() =20 iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) @@ -143,7 +46,7 @@ def main() -> None: =20 env['MYPYPATH'] =3D env['PYTHONPATH'] =20 - run_linters(files, env=3Denv) + linters.run_linters(files, env=3Denv) =20 =20 iotests.script_main(main) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py new file mode 100755 index 0000000000..e263b7cbee --- /dev/null +++ b/tests/qemu-iotests/linters.py @@ -0,0 +1,117 @@ +# Copyright (C) 2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import re +import subprocess +import sys +from typing import List, Mapping, Optional + + +# TODO: Empty this list! +SKIP_FILES =3D ( + '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', + '096', '118', '124', '132', '136', '139', '147', '148', '149', + '151', '152', '155', '163', '165', '194', '196', '202', + '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', + '218', '219', '224', '228', '234', '235', '236', '237', '238', + '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', + '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', + '299', '302', '303', '304', '307', + 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' +) + + +def is_python_file(filename: str, directory: str =3D '.') -> bool: + filepath =3D os.path.join(directory, filename) + + if not os.path.isfile(filepath): + return False + + if filename.endswith('.py'): + return True + + with open(filepath, encoding=3D'utf-8') as f: + try: + first_line =3D f.readline() + return re.match('^#!.*python', first_line) is not None + except UnicodeDecodeError: # Ignore binary files + return False + + +def get_test_files(directory: str =3D '.') -> List[str]: + named_test_dir =3D os.path.join(directory, 'tests') + named_tests =3D [f"tests/{entry}" for entry in os.listdir(named_test_d= ir)] + check_tests =3D set(os.listdir(directory) + named_tests) - set(SKIP_FI= LES) + return list(filter(lambda f: is_python_file(f, directory), check_tests= )) + + +def run_linters( + files: List[str], + directory: str =3D '.', + env: Optional[Mapping[str, str]] =3D None, +) -> int: + ret =3D 0 + + print('=3D=3D=3D pylint =3D=3D=3D') + sys.stdout.flush() + + # Todo notes are fine, but fixme's or xxx's should probably just be + # fixed (in tests, at least) + p =3D subprocess.run( + ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), + cwd=3Ddirectory, + env=3Denv, + check=3DFalse, + universal_newlines=3DTrue, + ) + ret +=3D p.returncode + + print('=3D=3D=3D mypy =3D=3D=3D') + sys.stdout.flush() + + # We have to call mypy separately for each file. Otherwise, it + # will interpret all given files as belonging together (i.e., they + # may not both define the same classes, etc.; most notably, they + # must not both define the __main__ module). + for filename in files: + p =3D subprocess.run( + ( + 'python3', '-m', 'mypy', + '--warn-unused-configs', + '--disallow-subclassing-any', + '--disallow-any-generics', + '--disallow-incomplete-defs', + '--disallow-untyped-decorators', + '--no-implicit-optional', + '--warn-redundant-casts', + '--warn-unused-ignores', + '--no-implicit-reexport', + '--namespace-packages', + filename, + ), + cwd=3Ddirectory, + env=3Denv, + check=3DFalse, + stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT, + universal_newlines=3DTrue + ) + + ret +=3D p.returncode + if p.returncode !=3D 0: + print(p.stdout) + + return ret --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631766310366324.3025150985575; Wed, 15 Sep 2021 21:25:10 -0700 (PDT) Received: from localhost ([::1]:39126 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQixR-0000fA-0A for importer@patchew.org; Thu, 16 Sep 2021 00:25:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35246) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijM-0007lW-64 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42614) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijJ-0000pP-IP for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:35 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-376-jSIcBQwiMymmPL3MHC1I3w-1; Thu, 16 Sep 2021 00:10:31 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4A4DB1808308; Thu, 16 Sep 2021 04:10:30 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 929CA19C79; Thu, 16 Sep 2021 04:10:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765432; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6qGz2/izmlQnfErNqQxxkT87ZCkZ22d1hoSgLEdHEfs=; b=QqgRCAPAPCEtNxfUPi5p7yVF2WgdU1UFbr7sAiaqTQPcEZw78MnUyP+z4dBpb/m1EIOxox gLo7KSoKVdHrGE66SugJXoWTBCRvuOL3jmOtySM78o6rTNHqA1ukPrCogsmqfXyENYHkBb g5NiBk0yoWiUY1koRS+okEv9Judg7w0= X-MC-Unique: jSIcBQwiMymmPL3MHC1I3w-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 13/16] iotests/linters: Add entry point for Python CI linters Date: Thu, 16 Sep 2021 00:09:52 -0400 Message-Id: <20210916040955.628560-14-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766311116100001 Content-Type: text/plain; charset="utf-8" Add a main() function to linters.py so that the Python CI infrastructure has something it can run. Now, linters.py represents an invocation of the linting scripts that more resembles a "normal" execution of pylint/mypy, like you'd expect to use if 'qemu' was a bona-fide package you obtained from PyPI. 297, by contrast, now represents the iotests-specific configuration bits you need to get it to function correctly as a part of iotests, and with 'qemu' as a namespace package that isn't "installed" to the current environment, but just lives elsewhere in our source tree. By doing this, we will able to run the same linting configuration from the Python CI tests without calling iotest logging functions or messing around with PYTHONPATH / MYPYPATH. iotest 297 continues to operate in a standalone fashion for now -- presumably, it's convenient for block maintainers and contributors to run in this manner. We can either remove this functionality at a later date if everyone is happy with the Python CI, or we can opt to continue to maintain it. Please let me know how you feel. See the following commit for how this is used from the Python packaging sid= e. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/linters.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index e263b7cbee..4df062a973 100755 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -115,3 +115,16 @@ def run_linters( print(p.stdout) =20 return ret + + +def main() -> int: + """ + Used by the Python CI system as an entry point to run these linters. + """ + directory =3D os.path.dirname(os.path.realpath(__file__)) + files =3D get_test_files(directory) + return run_linters(files, directory) + + +if __name__ =3D=3D '__main__': + sys.exit(main()) --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1631766677988110.16059896465958; Wed, 15 Sep 2021 21:31:17 -0700 (PDT) Received: from localhost ([::1]:50612 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQj3M-0008TX-OZ for importer@patchew.org; Thu, 16 Sep 2021 00:31:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijR-0007q7-J6 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29090) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijP-0000tq-B1 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:41 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-572-qciGnQlpO1uo-la-Jh5Ftg-1; Thu, 16 Sep 2021 00:10:33 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0ADD41084689; Thu, 16 Sep 2021 04:10:32 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 843AD19C79; Thu, 16 Sep 2021 04:10:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765437; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YQD+xvNddz3IgxRD7bdkK21oFfXzQ6rrTLfD3Du0hJg=; b=Su8fbtlKWgA87un4OHz9cdPcwAj9JUC/FdPu+KK9Pw28Q0qdQpO8Mg5Ptz19BS/p3niSS/ EYD7cIloVZpGYT7hatbvxMTktPsioAExtkoToYMjIAnQcUBDuj43zcb/w3ztaGeCkmcaT8 trQe87XQzZoKFIF8zrVFXRLJBJrLRYk= X-MC-Unique: qciGnQlpO1uo-la-Jh5Ftg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 14/16] iotests/linters: Add workaround for mypy bug #9852 Date: Thu, 16 Sep 2021 00:09:53 -0400 Message-Id: <20210916040955.628560-15-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766679526100001 Content-Type: text/plain; charset="utf-8" This one is insidious: if you use the invocation "from {namespace} import {subpackage}" as mirror-top-perms does, mypy will fail on every-other invocation *if* the package being imported is a package. Now, I could just edit mirror-top-perms to avoid this invocation, but since I tripped on a landmine, I might as well head it off at the pass and make sure nobody else trips on the same landmine. It seems to have something to do with the order in which files are checked as well, meaning the random order in which set(os.listdir()) produces the list of files to test will cause problems intermittently. mypy >=3D 0.920 isn't released yet, so add this workaround for now. See also: https://github.com/python/mypy/issues/11010 https://github.com/python/mypy/issues/9852 Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz --- tests/qemu-iotests/linters.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index 4df062a973..9c97324e87 100755 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -100,6 +100,9 @@ def run_linters( '--warn-unused-ignores', '--no-implicit-reexport', '--namespace-packages', + # Until we can use mypy >=3D 0.920, see + # https://github.com/python/mypy/issues/9852 + '--no-incremental', filename, ), cwd=3Ddirectory, --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163176651086457.52764408897565; Wed, 15 Sep 2021 21:28:30 -0700 (PDT) Received: from localhost ([::1]:45008 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQj0f-0004k6-Q1 for importer@patchew.org; Thu, 16 Sep 2021 00:28:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35292) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijO-0007pa-Dl for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21076) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijM-0000sH-Jp for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:38 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-412-6002h18RMF-O5ojMXfbqcg-1; Thu, 16 Sep 2021 00:10:35 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AD1B5824FAE; Thu, 16 Sep 2021 04:10:33 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 516E719C79; Thu, 16 Sep 2021 04:10:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XLSUoDkaQux3bt2HK7o/hFB8uu+8oqTXxlx/NpP+lUk=; b=FUrvXtAh43sYWGq63uzHx02EhiP2TkICDfVEgy7Ci8XOIBNQuOkdeGTa1snwNKQVPjNpw/ 1OUyterIKZLoEOW7XhgFbU9MYOLwcUIcLA0rNgktFEVGthmX77i8YxA7lpVSLvdigICtNe 39TPF5stLVhpkGX/r4iTP2aGqujSmXM= X-MC-Unique: 6002h18RMF-O5ojMXfbqcg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 15/16] python: Add iotest linters to test suite Date: Thu, 16 Sep 2021 00:09:54 -0400 Message-Id: <20210916040955.628560-16-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766512066100001 Content-Type: text/plain; charset="utf-8" As a convenience, since iotests is an extremely prominent user of the qemu.qmp and qemu.machine packages and already implements a linting regime, run those tests as well so that it's very hard to miss regressions caused by changes to the python library. Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> --- python/tests/iotests.sh | 4 ++++ 1 file changed, 4 insertions(+) create mode 100755 python/tests/iotests.sh diff --git a/python/tests/iotests.sh b/python/tests/iotests.sh new file mode 100755 index 0000000000..70324540cf --- /dev/null +++ b/python/tests/iotests.sh @@ -0,0 +1,4 @@ +#!/bin/sh -e + +cd ../tests/qemu-iotests/ +python3 -m linters --=20 2.31.1 From nobody Sun May 5 10:55:33 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163176668173798.96877693220733; Wed, 15 Sep 2021 21:31:21 -0700 (PDT) Received: from localhost ([::1]:50712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQj3Q-00005g-IO for importer@patchew.org; Thu, 16 Sep 2021 00:31:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35384) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijT-0007sO-U1 for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43464) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQijQ-0000vJ-6f for qemu-devel@nongnu.org; Thu, 16 Sep 2021 00:10:42 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-598-G0lfBaVYNaOKN0dbcco-KQ-1; Thu, 16 Sep 2021 00:10:36 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 50BE7362F9; Thu, 16 Sep 2021 04:10:35 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id E67F2196E5; Thu, 16 Sep 2021 04:10:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631765439; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=an1qGz9el9qAinieV8+Owr2ua9KInO64MhvEmIIhIW4=; b=S594C3d24qlyrJbjtmCvtXhkPDEQEGDdc0ifNSHFzaK/5A7+oyOnK/AUqg89nL0OYpFZNm NUAGfrsal4mTpNcGz3Ui31tKMBfRZqF1OTNZ+es0VuPSJ3udDQPFz8PQYj5jMRqvYvRbw9 r9Uue3b4k0rVWtnMHwNMS7sJqc7Cyso= X-MC-Unique: G0lfBaVYNaOKN0dbcco-KQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 16/16] iotests/linters: check mypy files all at once Date: Thu, 16 Sep 2021 00:09:55 -0400 Message-Id: <20210916040955.628560-17-jsnow@redhat.com> In-Reply-To: <20210916040955.628560-1-jsnow@redhat.com> References: <20210916040955.628560-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1631766683737100001 Content-Type: text/plain; charset="utf-8" We can circumvent the '__main__' redefinition problem by passing --scripts-are-modules. Take mypy out of the loop per-filename and check everything in one go: it's quite a bit faster. Signed-off-by: John Snow Based-On: <20210915175318.853225-1-hreitz@redhat.com> Reviewed-by: Hanna Reitz --- tests/qemu-iotests/linters.py | 62 ++++++++++++++++------------------- 1 file changed, 29 insertions(+), 33 deletions(-) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index 9c97324e87..ac9f77c5ac 100755 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -82,40 +82,36 @@ def run_linters( print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() =20 - # We have to call mypy separately for each file. Otherwise, it - # will interpret all given files as belonging together (i.e., they - # may not both define the same classes, etc.; most notably, they - # must not both define the __main__ module). - for filename in files: - p =3D subprocess.run( - ( - 'python3', '-m', 'mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - # Until we can use mypy >=3D 0.920, see - # https://github.com/python/mypy/issues/9852 - '--no-incremental', - filename, - ), - cwd=3Ddirectory, - env=3Denv, - check=3DFalse, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.STDOUT, - universal_newlines=3DTrue - ) + p =3D subprocess.run( + ( + 'python3', '-m', 'mypy', + '--warn-unused-configs', + '--disallow-subclassing-any', + '--disallow-any-generics', + '--disallow-incomplete-defs', + '--disallow-untyped-decorators', + '--no-implicit-optional', + '--warn-redundant-casts', + '--warn-unused-ignores', + '--no-implicit-reexport', + '--namespace-packages', + # Until we can use mypy >=3D 0.920, see + # https://github.com/python/mypy/issues/9852 + '--no-incremental', + '--scripts-are-modules', + *files, + ), + cwd=3Ddirectory, + env=3Denv, + check=3DFalse, + stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT, + universal_newlines=3DTrue + ) =20 - ret +=3D p.returncode - if p.returncode !=3D 0: - print(p.stdout) + ret +=3D p.returncode + if p.returncode !=3D 0: + print(p.stdout) =20 return ret =20 --=20 2.31.1