From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802499689732.0298570381675; Tue, 20 Jul 2021 10:34:59 -0700 (PDT) Received: from localhost ([::1]:49296 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tdy-0008Fq-LC for importer@patchew.org; Tue, 20 Jul 2021 13:34:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcp-0005QV-0I for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26283) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcm-0005WS-36 for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:46 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-475-MmNYuNfZPHWUMnE60mVz5A-1; Tue, 20 Jul 2021 13:33:41 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5AB66186840D; Tue, 20 Jul 2021 17:33:40 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BC2969CB4; Tue, 20 Jul 2021 17:33:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802422; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4ezZIRIYN/niKc/i4bU6bpgQNCe62H0HHoAlID5axx0=; b=QPYYlu+oR2duoIxBO45jxERyxOlCDZQd+0Qp4VxkIxpQp80pSxEnAK0+tpH+4XHQxpkTje wOE0ziYJxGlrgMy9N5I4XhU9smMXYQjiFiQScmQeSolyLI837dBmSD5nx3c+UUX481K+Cn cA+Q3tWJmXYVQ1cRTfv3YLZTHfTBmJ4= X-MC-Unique: MmNYuNfZPHWUMnE60mVz5A-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 01/17] iotests: use with-statement for open() calls Date: Tue, 20 Jul 2021 13:33:20 -0400 Message-Id: <20210720173336.1876937-2-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802501484100001 Content-Type: text/plain; charset="utf-8" Silences a new pylint warning. The dangers of *not* doing this are somewhat unclear; I believe the file object gets garbage collected eventually, but possibly the way in which it happens is non-deterministic. Maybe this is a valid warning, but if there are consequences of not doing it, I am not aware of them at present. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/iotests.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 89663dac06d..beadf5c821c 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -1075,7 +1075,8 @@ def notrun(reason): # Each test in qemu-iotests has a number ("seq") seq =3D os.path.basename(sys.argv[0]) =20 - open('%s/%s.notrun' % (output_dir, seq), 'w').write(reason + '\n') + with open('%s/%s.notrun' % (output_dir, seq), 'w') as outfile: + outfile.write(reason + '\n') logger.warning("%s not run: %s", seq, reason) sys.exit(0) =20 @@ -1088,8 +1089,8 @@ def case_notrun(reason): # Each test in qemu-iotests has a number ("seq") seq =3D os.path.basename(sys.argv[0]) =20 - open('%s/%s.casenotrun' % (output_dir, seq), 'a').write( - ' [case not run] ' + reason + '\n') + with open('%s/%s.casenotrun' % (output_dir, seq), 'a') as outfile: + outfile.write(' [case not run] ' + reason + '\n') =20 def _verify_image_format(supported_fmts: Sequence[str] =3D (), unsupported_fmts: Sequence[str] =3D ()) -> None: --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802653007333.607871732719; Tue, 20 Jul 2021 10:37:33 -0700 (PDT) Received: from localhost ([::1]:60078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tgR-00076m-U9 for importer@patchew.org; Tue, 20 Jul 2021 13:37:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55556) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcq-0005T4-Su for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tco-0005Z2-HC for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:48 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-401-WvtN4DlKN5Sy88PNE665uQ-1; Tue, 20 Jul 2021 13:33:42 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7F52D10C1ADC; Tue, 20 Jul 2021 17:33:41 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8233A69CB4; Tue, 20 Jul 2021 17:33:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wm+w/Q7w1N/7b7PVF5TuJJNEZKONVEp1ktCTYUmc+m0=; b=YSrvhW1+gSFMIBBW0x1a/9udUM4bKDjH86lA4Fds7urw2uwR1E9znvbilR3OY1q0+nnl/Q 4ToEsKv1FWIaG8QR6cZdXDTzbA0yMPiaWW0rmq4M1A/eSUPwRrki48Gz4zgUK4iq4hEJnL P7GMkAveFR+emVYvGuSdMgJUf+GLGpE= X-MC-Unique: WvtN4DlKN5Sy88PNE665uQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 02/17] iotests: use subprocess.DEVNULL instead of open("/dev/null") Date: Tue, 20 Jul 2021 13:33:21 -0400 Message-Id: <20210720173336.1876937-3-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802653307100001 Content-Type: text/plain; charset="utf-8" Avoids a warning from pylint not to use open() outside of a with-statement, and is ... probably more portable anyway. Not that I think we care too much about running tests *on* Windows, but... eh. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index beadf5c821c..64f9b688f55 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -219,18 +219,18 @@ def qemu_io_silent(*args): default_args =3D qemu_io_args =20 args =3D default_args + list(args) - exitcode =3D subprocess.call(args, stdout=3Dopen('/dev/null', 'w')) - if exitcode < 0: + result =3D subprocess.run(args, stdout=3Dsubprocess.DEVNULL, check=3DF= alse) + if result.returncode < 0: sys.stderr.write('qemu-io received signal %i: %s\n' % - (-exitcode, ' '.join(args))) - return exitcode + (-result.returncode, ' '.join(args))) + return result.returncode =20 def qemu_io_silent_check(*args): '''Run qemu-io and return the true if subprocess returned 0''' args =3D qemu_io_args + list(args) - exitcode =3D subprocess.call(args, stdout=3Dopen('/dev/null', 'w'), - stderr=3Dsubprocess.STDOUT) - return exitcode =3D=3D 0 + result =3D subprocess.run(args, stdout=3Dsubprocess.DEVNULL, + stderr=3Dsubprocess.STDOUT, check=3DFalse) + return result.returncode =3D=3D 0 =20 class QemuIoInteractive: def __init__(self, *args): --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802624375689.0867676105025; Tue, 20 Jul 2021 10:37:04 -0700 (PDT) Received: from localhost ([::1]:57882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tfz-0005ew-9D for importer@patchew.org; Tue, 20 Jul 2021 13:37:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55568) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcr-0005UW-8k for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29029) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcp-0005aF-Qu for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:49 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-538-W0CUG3dJPKGD3xmhMK69eQ-1; Tue, 20 Jul 2021 13:33:43 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A0BC7C73A1; Tue, 20 Jul 2021 17:33:42 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id A5EE469CB4; Tue, 20 Jul 2021 17:33:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oMuaF/gFmbTMw4UE38lpiG/uex5PNFomKE7sd81fMpQ=; b=CuPltIJFgbxRO11YwhC1aApcLJecYrfSm6IKF3tqtu2IWHK5hn/g/5JnPKkxgWAPUKNucN EUPlK++Vm0kwQ8de+bKEVUh+mUsVbrEfhZuJap1vTByYm4mGc3LtOHYdXC387aUsrtauRc 647pC8F9RU7P1zC/zHz0k4BtY3y1H0I= X-MC-Unique: W0CUG3dJPKGD3xmhMK69eQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 03/17] iotests/mirror-top-perms: Adjust import paths Date: Tue, 20 Jul 2021 13:33:22 -0400 Message-Id: <20210720173336.1876937-4-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802625957100001 Content-Type: text/plain; charset="utf-8" Technically AbnormalShutdown isn't exported via 'qemu.machine.AbnormalShutdown', but instead from 'qemu.machine.machine.AbnormalShutdown'. Odd, I know, and it's on the list to fix. For now, change the imports here. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/tests/mirror-top-perms | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests= /tests/mirror-top-perms index 451a0666f81..de181825901 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -25,7 +25,8 @@ from iotests import qemu_img =20 # Import qemu after iotests.py has amended sys.path # pylint: disable=3Dwrong-import-order -import qemu +from qemu import qmp +from qemu.machine import machine =20 =20 image_size =3D 1 * 1024 * 1024 @@ -47,7 +48,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase): def tearDown(self): try: self.vm.shutdown() - except qemu.machine.AbnormalShutdown: + except machine.AbnormalShutdown: pass =20 if self.vm_b is not None: @@ -102,7 +103,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase): self.vm_b.launch() print('ERROR: VM B launched successfully, this should not have= ' 'happened') - except qemu.qmp.QMPConnectError: + except qmp.QMPConnectError: assert 'Is another process using the image' in self.vm_b.get_l= og() =20 result =3D self.vm.qmp('block-job-cancel', --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802515746400.4609823207494; Tue, 20 Jul 2021 10:35:15 -0700 (PDT) Received: from localhost ([::1]:50776 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5teE-0000mr-IU for importer@patchew.org; Tue, 20 Jul 2021 13:35:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55534) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcq-0005RF-8A for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31788) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tco-0005Z7-Q2 for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:47 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-592-LaaaR3RWOjuCue1fuCiT1Q-1; Tue, 20 Jul 2021 13:33:44 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E294E80365C; Tue, 20 Jul 2021 17:33:43 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id C6E9A19C87; Tue, 20 Jul 2021 17:33:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DvehwYVn4BrbzkKJJNDGLCGkc9w1pu9UIFjbBIVv/h0=; b=Mat/ebKCkgkKMSX4nHS2Gafq1UNoFwLOMLt1sYF1q1mtd8fhkuaczA+g8aFDOILxA5duM0 RV9RjZgfJhwegnmwiIOlqKQ9oA7hsaPA8f+6hLTkie4QVId30eVgrhGBSykMcCSjAKs5EB XeolxZmt+GUxj8LVzwjX9uqfCe5Drr4= X-MC-Unique: LaaaR3RWOjuCue1fuCiT1Q-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 04/17] iotests/migrate-bitmaps-postcopy-test: declare instance variables Date: Tue, 20 Jul 2021 13:33:23 -0400 Message-Id: <20210720173336.1876937-5-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802516757100003 Content-Type: text/plain; charset="utf-8" Signed-off-by: John Snow --- tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test b/tests= /qemu-iotests/tests/migrate-bitmaps-postcopy-test index 584062b4128..78dc19e3e3d 100755 --- a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test +++ b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test @@ -115,6 +115,9 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestC= ase): self.vm_a_events =3D [] self.vm_b_events =3D [] =20 + self.discards1_sha256: str + self.all_discards_sha256: str + def start_postcopy(self): """ Run migration until RESUME event on target. Return this event.= """ for i in range(nb_bitmaps): --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802863209839.2007538348328; Tue, 20 Jul 2021 10:41:03 -0700 (PDT) Received: from localhost ([::1]:38960 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tjq-0003Pk-1B for importer@patchew.org; Tue, 20 Jul 2021 13:41:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55634) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tct-0005ag-Oh for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48604) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcq-0005aR-Be for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:51 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-438-gq-i_jekM2SSQvArjpe7SA-1; Tue, 20 Jul 2021 13:33:46 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2AF8D804146; Tue, 20 Jul 2021 17:33:45 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15F0C369A; Tue, 20 Jul 2021 17:33:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b/hTiJ13ihlaGBQEKy2Oz1UnjdMQKX0pyCmERH5kftQ=; b=iYLZgya7auz86Lt95qagyDNTfUtZNZR3k0Y6NUe4gnnFqO66mNy3+86QYiDF6N9h3Ga8zp 5HIHpi3mh2kSxExkH/sujR/lRfdxFYvjIZ/osNXw0kkWiXJlJZW1SLZOOepJ50vK9InieV KWfg5vaX1cn/vAP/QSw0SVoAmD/re3Y= X-MC-Unique: gq-i_jekM2SSQvArjpe7SA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 05/17] iotests/migrate-bitmaps-test: delint Date: Tue, 20 Jul 2021 13:33:24 -0400 Message-Id: <20210720173336.1876937-6-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802864889100001 Content-Type: text/plain; charset="utf-8" Mostly uninteresting stuff. Move the test injections under a function named main() so that the variables used during that process aren't in the global scope. Signed-off-by: John Snow --- tests/qemu-iotests/tests/migrate-bitmaps-test | 70 +++++++++++-------- 1 file changed, 39 insertions(+), 31 deletions(-) diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-test b/tests/qemu-iot= ests/tests/migrate-bitmaps-test index a5c7bc83e0e..fd36ee4a245 100755 --- a/tests/qemu-iotests/tests/migrate-bitmaps-test +++ b/tests/qemu-iotests/tests/migrate-bitmaps-test @@ -19,12 +19,12 @@ # along with this program. If not, see . # =20 -import os -import iotests -import time import itertools import operator +import os import re + +import iotests from iotests import qemu_img, qemu_img_create, Timeout =20 =20 @@ -62,9 +62,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): params['persistent'] =3D True =20 result =3D vm.qmp('block-dirty-bitmap-add', **params) - self.assert_qmp(result, 'return', {}); + self.assert_qmp(result, 'return', {}) =20 - def get_bitmap_hash(self, vm): + @classmethod + def get_bitmap_hash(cls, vm): result =3D vm.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap0') return result['return']['sha256'] @@ -73,10 +74,10 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): result =3D vm.qmp('x-debug-block-dirty-bitmap-sha256', node=3D'drive0', name=3D'bitmap0') if sha256: - self.assert_qmp(result, 'return/sha256', sha256); + self.assert_qmp(result, 'return/sha256', sha256) else: self.assert_qmp(result, 'error/desc', - "Dirty bitmap 'bitmap0' not found"); + "Dirty bitmap 'bitmap0' not found") =20 def do_test_migration_resume_source(self, persistent, migrate_bitmaps): granularity =3D 512 @@ -106,7 +107,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): break while True: result =3D self.vm_a.qmp('query-status') - if (result['return']['status'] =3D=3D 'postmigrate'): + if result['return']['status'] =3D=3D 'postmigrate': break =20 # test that bitmap is still here @@ -216,25 +217,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): =20 def inject_test_case(klass, name, method, *args, **kwargs): mc =3D operator.methodcaller(method, *args, **kwargs) - setattr(klass, 'test_' + method + name, lambda self: mc(self)) - -for cmb in list(itertools.product((True, False), repeat=3D5)): - name =3D ('_' if cmb[0] else '_not_') + 'persistent_' - name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap_' - name +=3D '_online' if cmb[2] else '_offline' - name +=3D '_shared' if cmb[3] else '_nonshared' - if (cmb[4]): - name +=3D '__pre_shutdown' - - inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', - *list(cmb)) - -for cmb in list(itertools.product((True, False), repeat=3D2)): - name =3D ('_' if cmb[0] else '_not_') + 'persistent_' - name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap' - - inject_test_case(TestDirtyBitmapMigration, name, - 'do_test_migration_resume_source', *list(cmb)) + setattr(klass, 'test_' + method + name, mc) =20 =20 class TestDirtyBitmapBackingMigration(iotests.QMPTestCase): @@ -270,7 +253,8 @@ class TestDirtyBitmapBackingMigration(iotests.QMPTestCa= se): self.assert_qmp(result, 'return', {}) =20 # Check that the bitmaps are there - for node in self.vm.qmp('query-named-block-nodes', flat=3DTrue)['r= eturn']: + nodes =3D self.vm.qmp('query-named-block-nodes', flat=3DTrue)['ret= urn'] + for node in nodes: if 'node0' in node['node-name']: self.assert_qmp(node, 'dirty-bitmaps[0]/name', 'bmap0') =20 @@ -287,7 +271,7 @@ class TestDirtyBitmapBackingMigration(iotests.QMPTestCa= se): """ Continue the source after migration. """ - result =3D self.vm.qmp('migrate', uri=3Df'exec: cat > /dev/null') + result =3D self.vm.qmp('migrate', uri=3D'exec: cat > /dev/null') self.assert_qmp(result, 'return', {}) =20 with Timeout(10, 'Migration timeout'): @@ -297,6 +281,30 @@ class TestDirtyBitmapBackingMigration(iotests.QMPTestC= ase): self.assert_qmp(result, 'return', {}) =20 =20 +def main(): + for cmb in list(itertools.product((True, False), repeat=3D5)): + name =3D ('_' if cmb[0] else '_not_') + 'persistent_' + name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap_' + name +=3D '_online' if cmb[2] else '_offline' + name +=3D '_shared' if cmb[3] else '_nonshared' + if cmb[4]: + name +=3D '__pre_shutdown' + + inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migratio= n', + *list(cmb)) + + for cmb in list(itertools.product((True, False), repeat=3D2)): + name =3D ('_' if cmb[0] else '_not_') + 'persistent_' + name +=3D ('_' if cmb[1] else '_not_') + 'migbitmap' + + inject_test_case(TestDirtyBitmapMigration, name, + 'do_test_migration_resume_source', *list(cmb)) + + iotests.main( + supported_fmts=3D['qcow2'], + supported_protocols=3D['file'] + ) + + if __name__ =3D=3D '__main__': - iotests.main(supported_fmts=3D['qcow2'], - supported_protocols=3D['file']) + main() --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802503752751.676662116076; Tue, 20 Jul 2021 10:35:03 -0700 (PDT) Received: from localhost ([::1]:49772 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5te2-00007D-Dg for importer@patchew.org; Tue, 20 Jul 2021 13:35:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55630) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tct-0005aI-LT for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34103) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcr-0005bE-70 for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:51 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-434-GvFcGVv2OX-4HJFBw3Rdwg-1; Tue, 20 Jul 2021 13:33:47 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7AA56C73A0; Tue, 20 Jul 2021 17:33:46 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C93219CBA; Tue, 20 Jul 2021 17:33:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802428; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9NOMct6FJQakwxhlOCli39bFxd9cuz5i4j2IxsliyYE=; b=C/9iSF6Eae13bM6fGKKVACoVrK1bJnqgQmlxIEQT1l90k67dc1SPmdfDA0TjUZ7F484LM5 bXLO6BGkuDTclZtSwd4dW80BAGVpVuolFfauH5WRLe38NVNSqlJON5xncYXXRr+KC7prnR eoxcfKiTXEGobAij6W4NsQBPzP5W254= X-MC-Unique: GvFcGVv2OX-4HJFBw3Rdwg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 06/17] iotests/297: modify is_python_file to work from any CWD Date: Tue, 20 Jul 2021 13:33:25 -0400 Message-Id: <20210720173336.1876937-7-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802505966100001 Content-Type: text/plain; charset="utf-8" Add a directory argument to is_python_file to allow it to work correctly no matter what CWD we happen to run it from. This is done in anticipation of running the iotests from another directory (./python/). Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 433b7323368..493dda17fb6 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -39,14 +39,16 @@ SKIP_FILES =3D ( ) =20 =20 -def is_python_file(filename): - if not os.path.isfile(filename): +def is_python_file(filename: str, directory: str =3D '.') -> bool: + filepath =3D os.path.join(directory, filename) + + if not os.path.isfile(filepath): return False =20 if filename.endswith('.py'): return True =20 - with open(filename) as f: + with open(filepath) as f: try: first_line =3D f.readline() return re.match('^#!.*python', first_line) is not None --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802637886929.2653439261165; Tue, 20 Jul 2021 10:37:17 -0700 (PDT) Received: from localhost ([::1]:59238 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tgC-0006Xn-Sb for importer@patchew.org; Tue, 20 Jul 2021 13:37:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55652) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcu-0005eT-MG for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45171) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcs-0005cp-NF for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:52 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-163-vVm6O2AAOLWweNxB8xZsuw-1; Tue, 20 Jul 2021 13:33:48 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BA8571015DA1; Tue, 20 Jul 2021 17:33:47 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id A23C269CB4; Tue, 20 Jul 2021 17:33:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802430; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TlW+O/Agw4fmvSQABqWMqPQidkykJu4Fz0dvqJnM6oo=; b=G2DosyEyL+O5vQ9WzJSAhPG86C88deDkMi9VLO4WSeqbrxlEm6cZmq8GKiCZCEt7o5dpfZ Li4oJcFv3BXMho228Gw7McX1+9+7em3ildNUhx//uqoui5V1qW9SsFkdLpRNTrkoNZSXBi WuUGYkRczPGPR3Hzgv+Mo8Y3ij26vJQ= X-MC-Unique: vVm6O2AAOLWweNxB8xZsuw-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 07/17] iotests/297: Add get_files() function Date: Tue, 20 Jul 2021 13:33:26 -0400 Message-Id: <20210720173336.1876937-8-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802639462100001 Content-Type: text/plain; charset="utf-8" Split out file discovery into its own method to begin separating out the "environment setup" and "test execution" phases. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 493dda17fb6..0bc11958059 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,6 +21,7 @@ import re import shutil import subprocess import sys +from typing import List =20 import iotests =20 @@ -56,9 +57,15 @@ def is_python_file(filename: str, directory: str =3D '.'= ) -> bool: return False =20 =20 +def get_test_files(directory: str =3D '.') -> List[str]: + return [ + f for f in (set(os.listdir(directory)) - set(SKIP_FILES)) + if is_python_file(f, directory) + ] + + def run_linters(): - files =3D [filename for filename in (set(os.listdir('.')) - set(SKIP_F= ILES)) - if is_python_file(filename)] + files =3D get_test_files() =20 iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626803041916591.9105083643432; Tue, 20 Jul 2021 10:44:01 -0700 (PDT) Received: from localhost ([::1]:47522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tmi-0000p3-Se for importer@patchew.org; Tue, 20 Jul 2021 13:44:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55686) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcw-0005jP-Ct for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:44346) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcu-0005dv-GP for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:54 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-427-iEzTu7hYMkiLmSbuSUvI2A-1; Tue, 20 Jul 2021 13:33:50 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F2E72803F5B; Tue, 20 Jul 2021 17:33:48 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id E13AF69CB4; Tue, 20 Jul 2021 17:33:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802431; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y94/5VMGNyufcv+PB8ODzD4VPZdS8XBLnCFC8whrLW8=; b=Yp178no8YsoTNllQq3ekIH5WJOQSVT9BcLZy5Ej5xeT81vFxAgyt2hhB7EWRPow8SY2eNz J5XpeBRIzHPLoMDW+1Dmrwn3RRt4pbCiySIWqv9Pxz5FypyMoZ07hunlJdSiE2JlO20sRt nUw143DTlq8xAa+4AVeBYDA5eOStACM= X-MC-Unique: iEzTu7hYMkiLmSbuSUvI2A-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 08/17] iotests/297: Include sub-directories when finding tests to lint Date: Tue, 20 Jul 2021 13:33:27 -0400 Message-Id: <20210720173336.1876937-9-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626803043696100001 Content-Type: text/plain; charset="utf-8" Choosing to interpret the SKIP_FILES list as a list of filenames instead of a list of paths, to keep things simple. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 0bc11958059..665ac0aa361 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -40,13 +40,8 @@ SKIP_FILES =3D ( ) =20 =20 -def is_python_file(filename: str, directory: str =3D '.') -> bool: - filepath =3D os.path.join(directory, filename) - - if not os.path.isfile(filepath): - return False - - if filename.endswith('.py'): +def is_python_file(filepath: str) -> bool: + if filepath.endswith('.py'): return True =20 with open(filepath) as f: @@ -58,10 +53,20 @@ def is_python_file(filename: str, directory: str =3D '.= ') -> bool: =20 =20 def get_test_files(directory: str =3D '.') -> List[str]: - return [ - f for f in (set(os.listdir(directory)) - set(SKIP_FILES)) - if is_python_file(f, directory) - ] + files =3D [] + + iotests.logger.debug("get_test_files(%s)", directory) + for dirent in os.scandir(directory): + if dirent.name in SKIP_FILES: + continue + + relpath =3D os.path.join(directory, dirent.name) + if dirent.is_dir(): + files.extend(get_test_files(relpath)) + elif is_python_file(relpath): + files.append(relpath) + + return files =20 =20 def run_linters(): --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626803262681443.15751218459457; Tue, 20 Jul 2021 10:47:42 -0700 (PDT) Received: from localhost ([::1]:54982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tqG-0005sz-Ew for importer@patchew.org; Tue, 20 Jul 2021 13:47:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55692) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcw-0005ki-Nz for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47367) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcv-0005eA-24 for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:54 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-269-lB-1I-dgPWi5XFTIHLs1ig-1; Tue, 20 Jul 2021 13:33:51 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 27E3410C1ADC; Tue, 20 Jul 2021 17:33:50 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 27CC669CB4; Tue, 20 Jul 2021 17:33:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802432; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fSLa34YY4+NqHwZotV63ArZkyZ/DYSJWfeGmgDeChRM=; b=ZDNbA5CsAnf4XCLk7WwCOulobkNS9fgDrEskD8Oiroihjtjpg62oQ1HmfRiPlgvTgvfY4X dVVstli7k45nMD76lcf4NAQXblfKk+ie3WptJZwj0CrJ450x57KoN524897hOtWnggMJZk NEF70Zo6fvsJux/1ChCJGICHi5Ap7MU= X-MC-Unique: lB-1I-dgPWi5XFTIHLs1ig-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 09/17] iotests/297: Don't rely on distro-specific linter binaries Date: Tue, 20 Jul 2021 13:33:28 -0400 Message-Id: <20210720173336.1876937-10-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626803265259100001 Content-Type: text/plain; charset="utf-8" 'pylint-3' is another Fedora-ism. Use "python3 -m pylint" or "python3 -m mypy" to access these scripts instead. This style of invocation will prefer the "correct" tool when run in a virtual environment. Note that we still check for "pylint-3" before the test begins -- this check is now "overly strict", but shouldn't cause anything that was already running correctly to start failing. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Philippe Mathieu-Daud=C3=A9 --- tests/qemu-iotests/297 | 45 ++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 665ac0aa361..dc4460dd84b 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -87,8 +87,11 @@ def run_linters(): env['PYTHONPATH'] +=3D os.pathsep + qemu_module_path except KeyError: env['PYTHONPATH'] =3D qemu_module_path - subprocess.run(('pylint-3', '--score=3Dn', '--notes=3DFIXME,XXX', *fil= es), - env=3Denv, check=3DFalse) + subprocess.run( + ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), + env=3Denv, + check=3DFalse, + ) =20 print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() @@ -99,23 +102,27 @@ def run_linters(): # must not both define the __main__ module). env['MYPYPATH'] =3D env['PYTHONPATH'] for filename in files: - p =3D subprocess.run(('mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - filename), - env=3Denv, - check=3DFalse, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.STDOUT, - universal_newlines=3DTrue) + p =3D subprocess.run( + ( + 'python3', '-m', 'mypy', + '--warn-unused-configs', + '--disallow-subclassing-any', + '--disallow-any-generics', + '--disallow-incomplete-defs', + '--disallow-untyped-decorators', + '--no-implicit-optional', + '--warn-redundant-casts', + '--warn-unused-ignores', + '--no-implicit-reexport', + '--namespace-packages', + filename, + ), + env=3Denv, + check=3DFalse, + stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT, + universal_newlines=3DTrue + ) =20 if p.returncode !=3D 0: print(p.stdout) --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626803403731947.6541072932455; Tue, 20 Jul 2021 10:50:03 -0700 (PDT) Received: from localhost ([::1]:33062 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tsY-0001iY-OQ for importer@patchew.org; Tue, 20 Jul 2021 13:50:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcz-0005xy-WF for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:36841) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcy-0005gW-Ff for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:57 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-167-EXsn7ahWNNy7BiU254rimg-1; Tue, 20 Jul 2021 13:33:52 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8FF17C73A1; Tue, 20 Jul 2021 17:33:51 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4FA5F69CB4; Tue, 20 Jul 2021 17:33:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Mgm+XUnejwbrM08kxmWD8re2D+lycmJfyuoxD+Q5gg=; b=M36mt2K9AGaJM8hWFS+sfSL4XKxPtxhfmPbM4Vey/GGwQXHPCasA3NjDfc3WfhPsd3B8+H xgRYoBnb3VSV88AdTZzTZUqAohQG2Lk7WS4OR5zPrNdw6UnxXqnfQ7ShIF0oaeY5IFohHQ 8z0/rdjQNomGCDGUM6kLHHdmgLrzjTw= X-MC-Unique: EXsn7ahWNNy7BiU254rimg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 10/17] iotests/297: Create main() function Date: Tue, 20 Jul 2021 13:33:29 -0400 Message-Id: <20210720173336.1876937-11-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626803404158100001 Content-Type: text/plain; charset="utf-8" Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index dc4460dd84b..7ef4cdc7615 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -128,8 +128,12 @@ def run_linters(): print(p.stdout) =20 =20 -for linter in ('pylint-3', 'mypy'): - if shutil.which(linter) is None: - iotests.notrun(f'{linter} not found') +def main() -> None: + for linter in ('pylint-3', 'mypy'): + if shutil.which(linter) is None: + iotests.notrun(f'{linter} not found') =20 -iotests.script_main(run_linters) + run_linters() + + +iotests.script_main(main) --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802644232431.18475419536094; Tue, 20 Jul 2021 10:37:24 -0700 (PDT) Received: from localhost ([::1]:59796 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tgJ-0006vV-6j for importer@patchew.org; Tue, 20 Jul 2021 13:37:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55780) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td0-0005zz-HF for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:40835) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcx-0005g2-RP for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-146-czMoq7m-P3Gqv3OzN8p8JQ-1; Tue, 20 Jul 2021 13:33:53 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BD38E8030D7; Tue, 20 Jul 2021 17:33:52 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id C071D369A; Tue, 20 Jul 2021 17:33:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h2XyFkTh2hNaP5ds8DvNjj36ovlrnTS3MNjp/Wqib5U=; b=XOW6qNoazdxdfDHpKoAhLJ3m7Whsp2MZHlwWr779rQZRj+V4KW9ahvphrbZ4nx49MvduDC C/OkgdxTLldn54XmR4Ea0aY3U7g070ZkBKZ9bXWSecuBz9VwaH0Kve/MbRhSQC+/P0rdSG NVK3lwIYCVeQvsYkC2lHFjFM/qMNSY8= X-MC-Unique: czMoq7m-P3Gqv3OzN8p8JQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 11/17] iotests/297: Separate environment setup from test execution Date: Tue, 20 Jul 2021 13:33:30 -0400 Message-Id: <20210720173336.1876937-12-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802646161100001 Content-Type: text/plain; charset="utf-8" Move environment setup into main(), leaving pure test execution behind in run_linters(). Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 7ef4cdc7615..961c9170d21 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,7 +21,7 @@ import re import shutil import subprocess import sys -from typing import List +from typing import List, Mapping, Optional =20 import iotests =20 @@ -69,24 +69,16 @@ def get_test_files(directory: str =3D '.') -> List[str]: return files =20 =20 -def run_linters(): - files =3D get_test_files() - - iotests.logger.debug('Files to be checked:') - iotests.logger.debug(', '.join(sorted(files))) +def run_linters( + files: List[str], + env: Optional[Mapping[str, str]] =3D None, +) -> None: =20 print('=3D=3D=3D pylint =3D=3D=3D') sys.stdout.flush() =20 # Todo notes are fine, but fixme's or xxx's should probably just be # fixed (in tests, at least) - env =3D os.environ.copy() - qemu_module_path =3D os.path.join(os.path.dirname(__file__), - '..', '..', 'python') - try: - env['PYTHONPATH'] +=3D os.pathsep + qemu_module_path - except KeyError: - env['PYTHONPATH'] =3D qemu_module_path subprocess.run( ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), env=3Denv, @@ -100,7 +92,6 @@ def run_linters(): # will interpret all given files as belonging together (i.e., they # may not both define the same classes, etc.; most notably, they # must not both define the __main__ module). - env['MYPYPATH'] =3D env['PYTHONPATH'] for filename in files: p =3D subprocess.run( ( @@ -133,7 +124,22 @@ def main() -> None: if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') =20 - run_linters() + files =3D get_test_files() + + iotests.logger.debug('Files to be checked:') + iotests.logger.debug(', '.join(sorted(files))) + + env =3D os.environ.copy() + qemu_module_path =3D os.path.join(os.path.dirname(__file__), + '..', '..', 'python') + try: + env['PYTHONPATH'] +=3D os.pathsep + qemu_module_path + except KeyError: + env['PYTHONPATH'] =3D qemu_module_path + + env['MYPYPATH'] =3D env['PYTHONPATH'] + + run_linters(files, env=3Denv) =20 =20 iotests.script_main(main) --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802927900964.8751579933077; Tue, 20 Jul 2021 10:42:07 -0700 (PDT) Received: from localhost ([::1]:40856 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tks-0004nD-QT for importer@patchew.org; Tue, 20 Jul 2021 13:42:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55802) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td1-000648-CO for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37290) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5tcy-0005hQ-UF for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-517-4tSpNlljN7-IefY7uFKUNQ-1; Tue, 20 Jul 2021 13:33:55 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E40C21015DA2; Tue, 20 Jul 2021 17:33:53 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id E63C269CB6; Tue, 20 Jul 2021 17:33:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802436; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qxHVkCjQ6a5kL/cftUQN55WMndbzC7tqsVOoMpLDISI=; b=E/t07dlvR6FExqvdpmCbRW+IFLKFC2wr52qBzt1Q6B8/7rqUnXZr562Eqs2SDHB6lptkRk AzDyzRZK2yvdV+OuEYlPS4lfvPQThOiqzyrlwFdexGEpzGz89fcecpRG7gjpo7b1Mu4SNv g8Rk7BhaLSJowiCcQ5nKvJLrx9f+LY0= X-MC-Unique: 4tSpNlljN7-IefY7uFKUNQ-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 12/17] iotests/297: Add 'directory' argument to run_linters Date: Tue, 20 Jul 2021 13:33:31 -0400 Message-Id: <20210720173336.1876937-13-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802928889100001 Content-Type: text/plain; charset="utf-8" Allow run_linters to work well if it's executed from a different directory. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 961c9170d21..b2bf7928c5d 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -71,6 +71,7 @@ def get_test_files(directory: str =3D '.') -> List[str]: =20 def run_linters( files: List[str], + directory: str =3D '.', env: Optional[Mapping[str, str]] =3D None, ) -> None: =20 @@ -81,6 +82,7 @@ def run_linters( # fixed (in tests, at least) subprocess.run( ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), + cwd=3Ddirectory, env=3Denv, check=3DFalse, ) @@ -108,6 +110,7 @@ def run_linters( '--namespace-packages', filename, ), + cwd=3Ddirectory, env=3Denv, check=3DFalse, stdout=3Dsubprocess.PIPE, --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626803488145371.9903074083469; Tue, 20 Jul 2021 10:51:28 -0700 (PDT) Received: from localhost ([::1]:36078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ttv-0003kM-3A for importer@patchew.org; Tue, 20 Jul 2021 13:51:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55808) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td1-00065z-UA for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45077) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td0-0005iN-4C for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:33:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-276-CGBYx7LVP-qyDF867uenqg-1; Tue, 20 Jul 2021 13:33:56 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3305B804140; Tue, 20 Jul 2021 17:33:55 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1698F69CB4; Tue, 20 Jul 2021 17:33:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802437; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aWtb7FdN/HkMYfIN8ZihJb6ffd6EUUmav2t3n4/Ysd4=; b=XfURy1uZiXheeYip6iCI05Oi1YxmviiAmtKAELpVL15ngXkpngeIrTwVXd9bHJbTEWUnUC C9PutNfYut8UEbN2kaTSMJCingDf8LW29ig8EjWWc8itQ4jT93bE7HN5rlZjQSLXOFzDEb EkAP7hUQgYo2EQaRLmUVIlDpQ9UY5jQ= X-MC-Unique: CGBYx7LVP-qyDF867uenqg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 13/17] iotests/297: return error code from run_linters() Date: Tue, 20 Jul 2021 13:33:32 -0400 Message-Id: <20210720173336.1876937-14-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626803489531100001 Content-Type: text/plain; charset="utf-8" This turns run_linters() into a bit of a hybrid test; returning non-zero on failed execution while also printing diffable information. This is done for the benefit of the avocado simple test runner, which will soon be attempting to execute this test from a different environment. (Note: universal_newlines is added to the pylint invocation for type consistency with the mypy run -- it's not strictly necessary, but it avoids some typing errors caused by our re-use of the 'p' variable.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/297 | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b2bf7928c5d..ca9ee72d2fa 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -73,19 +73,22 @@ def run_linters( files: List[str], directory: str =3D '.', env: Optional[Mapping[str, str]] =3D None, -) -> None: +) -> int: + ret =3D 0 =20 print('=3D=3D=3D pylint =3D=3D=3D') sys.stdout.flush() =20 # Todo notes are fine, but fixme's or xxx's should probably just be # fixed (in tests, at least) - subprocess.run( + p =3D subprocess.run( ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), cwd=3Ddirectory, env=3Denv, check=3DFalse, + universal_newlines=3DTrue, ) + ret +=3D p.returncode =20 print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() @@ -118,9 +121,12 @@ def run_linters( universal_newlines=3DTrue ) =20 + ret +=3D p.returncode if p.returncode !=3D 0: print(p.stdout) =20 + return ret + =20 def main() -> None: for linter in ('pylint-3', 'mypy'): --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626803557615902.1584892765181; Tue, 20 Jul 2021 10:52:37 -0700 (PDT) Received: from localhost ([::1]:39282 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tv2-00064P-Gd for importer@patchew.org; Tue, 20 Jul 2021 13:52:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55900) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td6-0006N0-19 for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41689) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td2-0005kh-BI for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:03 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-511-V95ziEG9MVK0e0b1FoO_ag-1; Tue, 20 Jul 2021 13:33:57 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 807A2186840B; Tue, 20 Jul 2021 17:33:56 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 744A4369A; Tue, 20 Jul 2021 17:33:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802439; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DoVTWKVir/Qq/LuiaI9w3+7/8eKuXOHf5Iu/+9gn7uo=; b=VpS64/w1Vu+HOlkGXb5eA9h8dwWS3m9OiQwh8lQhwtyS7MvkRBhatb4qItZiOKTgU5/E7U KEANEu28iTjD1POfh4DmHHC+flHehkKmLB/NNMI9HN8mon4vF0/CoLwZfN5MxTyjvi/G1d HNbyqyHNrvnrcnAYXZl6x2Y6T4VZyk0= X-MC-Unique: V95ziEG9MVK0e0b1FoO_ag-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 14/17] iotests/297: split linters.py off from 297 Date: Tue, 20 Jul 2021 13:33:33 -0400 Message-Id: <20210720173336.1876937-15-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626803558701100001 Content-Type: text/plain; charset="utf-8" Split the linter execution itself out from 297, leaving just environment setup in 297. This is done so that non-iotest code can invoke the linters without needing to worry about imports of unpackaged iotest code. Eventually, it should be possible to replace linters.py with mypy.ini and pylintrc files that instruct these tools how to run properly in this directory, but ... not yet, and not in this series. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 110 ++----------------------------- tests/qemu-iotests/linters.py | 121 ++++++++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+), 106 deletions(-) create mode 100755 tests/qemu-iotests/linters.py diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index ca9ee72d2fa..3d29af5b78a 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -17,115 +17,13 @@ # along with this program. If not, see . =20 import os -import re import shutil -import subprocess -import sys -from typing import List, Mapping, Optional =20 import iotests +import linters =20 =20 -# TODO: Empty this list! -SKIP_FILES =3D ( - '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', - '096', '118', '124', '132', '136', '139', '147', '148', '149', - '151', '152', '155', '163', '165', '169', '194', '196', '199', '202', - '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', - '218', '219', '222', '224', '228', '234', '235', '236', '237', '238', - '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', - '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', - '299', '302', '303', '304', '307', - 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' -) - - -def is_python_file(filepath: str) -> bool: - if filepath.endswith('.py'): - return True - - with open(filepath) as f: - try: - first_line =3D f.readline() - return re.match('^#!.*python', first_line) is not None - except UnicodeDecodeError: # Ignore binary files - return False - - -def get_test_files(directory: str =3D '.') -> List[str]: - files =3D [] - - iotests.logger.debug("get_test_files(%s)", directory) - for dirent in os.scandir(directory): - if dirent.name in SKIP_FILES: - continue - - relpath =3D os.path.join(directory, dirent.name) - if dirent.is_dir(): - files.extend(get_test_files(relpath)) - elif is_python_file(relpath): - files.append(relpath) - - return files - - -def run_linters( - files: List[str], - directory: str =3D '.', - env: Optional[Mapping[str, str]] =3D None, -) -> int: - ret =3D 0 - - print('=3D=3D=3D pylint =3D=3D=3D') - sys.stdout.flush() - - # Todo notes are fine, but fixme's or xxx's should probably just be - # fixed (in tests, at least) - p =3D subprocess.run( - ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), - cwd=3Ddirectory, - env=3Denv, - check=3DFalse, - universal_newlines=3DTrue, - ) - ret +=3D p.returncode - - print('=3D=3D=3D mypy =3D=3D=3D') - sys.stdout.flush() - - # We have to call mypy separately for each file. Otherwise, it - # will interpret all given files as belonging together (i.e., they - # may not both define the same classes, etc.; most notably, they - # must not both define the __main__ module). - for filename in files: - p =3D subprocess.run( - ( - 'python3', '-m', 'mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - filename, - ), - cwd=3Ddirectory, - env=3Denv, - check=3DFalse, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.STDOUT, - universal_newlines=3DTrue - ) - - ret +=3D p.returncode - if p.returncode !=3D 0: - print(p.stdout) - - return ret +# Looking for the list of excluded tests? See linters.py ! =20 =20 def main() -> None: @@ -133,7 +31,7 @@ def main() -> None: if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') =20 - files =3D get_test_files() + files =3D linters.get_test_files() =20 iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) @@ -148,7 +46,7 @@ def main() -> None: =20 env['MYPYPATH'] =3D env['PYTHONPATH'] =20 - run_linters(files, env=3Denv) + linters.run_linters(files, env=3Denv) =20 =20 iotests.script_main(main) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py new file mode 100755 index 00000000000..e7e8baded02 --- /dev/null +++ b/tests/qemu-iotests/linters.py @@ -0,0 +1,121 @@ +# Copyright (C) 2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import re +import subprocess +import sys +from typing import List, Mapping, Optional + + +# TODO: Empty this list! +SKIP_FILES =3D ( + '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', + '096', '118', '124', '132', '136', '139', '147', '148', '149', + '151', '152', '155', '163', '165', '169', '194', '196', '199', '202', + '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', + '218', '219', '222', '224', '228', '234', '235', '236', '237', '238', + '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', + '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', + '299', '302', '303', '304', '307', + 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' +) + + +def is_python_file(filepath: str) -> bool: + if filepath.endswith('.py'): + return True + + with open(filepath) as f: + try: + first_line =3D f.readline() + return re.match('^#!.*python', first_line) is not None + except UnicodeDecodeError: # Ignore binary files + return False + + +def get_test_files(directory: str =3D '.') -> List[str]: + files =3D [] + + for dirent in os.scandir(directory): + if dirent.name in SKIP_FILES: + continue + + relpath =3D os.path.join(directory, dirent.name) + if dirent.is_dir(): + files.extend(get_test_files(relpath)) + elif is_python_file(relpath): + files.append(relpath) + + return files + + +def run_linters( + files: List[str], + directory: str =3D '.', + env: Optional[Mapping[str, str]] =3D None, +) -> int: + ret =3D 0 + + print('=3D=3D=3D pylint =3D=3D=3D') + sys.stdout.flush() + + # Todo notes are fine, but fixme's or xxx's should probably just be + # fixed (in tests, at least) + p =3D subprocess.run( + ('python3', '-m', 'pylint', '--score=3Dn', '--notes=3DFIXME,XXX', = *files), + cwd=3Ddirectory, + env=3Denv, + check=3DFalse, + universal_newlines=3DTrue, + ) + ret +=3D p.returncode + + print('=3D=3D=3D mypy =3D=3D=3D') + sys.stdout.flush() + + # We have to call mypy separately for each file. Otherwise, it + # will interpret all given files as belonging together (i.e., they + # may not both define the same classes, etc.; most notably, they + # must not both define the __main__ module). + for filename in files: + p =3D subprocess.run( + ( + 'python3', '-m', 'mypy', + '--warn-unused-configs', + '--disallow-subclassing-any', + '--disallow-any-generics', + '--disallow-incomplete-defs', + '--disallow-untyped-decorators', + '--no-implicit-optional', + '--warn-redundant-casts', + '--warn-unused-ignores', + '--no-implicit-reexport', + '--namespace-packages', + filename, + ), + cwd=3Ddirectory, + env=3Denv, + check=3DFalse, + stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT, + universal_newlines=3DTrue + ) + + ret +=3D p.returncode + if p.returncode !=3D 0: + print(p.stdout) + + return ret --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626802888333992.8191097406067; Tue, 20 Jul 2021 10:41:28 -0700 (PDT) Received: from localhost ([::1]:39744 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tkF-0003xC-6V for importer@patchew.org; Tue, 20 Jul 2021 13:41:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55922) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td6-0006PZ-L2 for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50086) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td4-0005m0-Vm for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:04 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-366-AnSaSn2qNwKYCnqnE5lv7Q-1; Tue, 20 Jul 2021 13:33:58 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A79051868408; Tue, 20 Jul 2021 17:33:57 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id A80BB69CB7; Tue, 20 Jul 2021 17:33:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802442; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zzr7T/rdOMIjMuAuc7b5aXQL0BH80109T0o1ATwMZFI=; b=QQAuX6j55SZOz1cDWAqpDm7yC5KL+2OOebrSfU3FY0OubOlY4khQdlduSDpx84+hK8lpix 3ulv04/bGN2wN1l8hty1ObtRpImbv5vDZpx3o3YtZNuSYsgchA5ApVCsi0l2CZBsOuBsxq HqZEB89SiRl/Ewgf8pwv76r1PD9FBlY= X-MC-Unique: AnSaSn2qNwKYCnqnE5lv7Q-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 15/17] iotests/linters: Add entry point for Python CI linters Date: Tue, 20 Jul 2021 13:33:34 -0400 Message-Id: <20210720173336.1876937-16-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626802888937100001 Content-Type: text/plain; charset="utf-8" Add a main() function to linters.py so that the Python CI infrastructure has something it can run. Now, linters.py represents an invocation of the linting scripts that more resembles a "normal" execution of pylint/mypy, like you'd expect to use if 'qemu' was a bona-fide package you obtained from PyPI. 297, by contrast, now represents the iotests-specific configuration bits you need to get it to function correctly as a part of iotests, and with 'qemu' as a namespace package that isn't "installed" to the current environment, but just lives elsewhere in our source tree. By doing this, we will able to run the same linting configuration from the Python CI tests without calling iotest logging functions or messing around with PYTHONPATH / MYPYPATH. iotest 297 continues to operate in a standalone fashion for now -- presumably, it's convenient for block maintainers and contributors to run in this manner. See the following commit for how this is used from the Python packaging sid= e. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/linters.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index e7e8baded02..242cb9c3e69 100755 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -119,3 +119,16 @@ def run_linters( print(p.stdout) =20 return ret + + +def main() -> int: + """ + Used by the Python CI system as an entry point to run these linters. + """ + directory =3D os.path.dirname(os.path.realpath(__file__)) + files =3D get_test_files(directory) + return run_linters(files, directory) + + +if __name__ =3D=3D '__main__': + sys.exit(main()) --=20 2.31.1 From nobody Sat May 18 06:50:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626803049949790.6369990794531; Tue, 20 Jul 2021 10:44:09 -0700 (PDT) Received: from localhost ([::1]:48236 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tmq-0001Kl-KL for importer@patchew.org; Tue, 20 Jul 2021 13:44:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55926) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td6-0006QQ-SJ for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42132) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td4-0005lj-36 for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:04 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-83-T1qq0XrkOt2gZtWDcd-3uA-1; Tue, 20 Jul 2021 13:34:00 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D16A6100CC85; Tue, 20 Jul 2021 17:33:58 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id CEF69369A; Tue, 20 Jul 2021 17:33:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yu3wjys6GOJRU0OQcLDejdxI3XQdB2+uOyd/k4IQo5k=; b=gjO92vObYS2huYj02C2LrVP/6qCxO0RI6MFMx48rQ/xM0AQ6vJ9dG1sLs0dUUJpDwAm8W6 NbJiQSSF5J1bHBQMnftYqvjTdkN097IJ6ZRD2gyNqK8uy+ehjf/+7GEnUUTmCJOL7oeYTy JakeBE5pjW5CI2x39xZCtYdL9BJVOI8= X-MC-Unique: T1qq0XrkOt2gZtWDcd-3uA-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 16/17] python: Add iotest linters to test suite Date: Tue, 20 Jul 2021 13:33:35 -0400 Message-Id: <20210720173336.1876937-17-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626803050351100001 Content-Type: text/plain; charset="utf-8" As a convenience, since iotests is an extremely prominent user of the qemu.qmp and qemu.machine packages and already implements a linting regime, run those tests as well so that it's very hard to miss regressions caused by changes to the python library. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- python/tests/iotests.sh | 2 ++ 1 file changed, 2 insertions(+) create mode 100755 python/tests/iotests.sh diff --git a/python/tests/iotests.sh b/python/tests/iotests.sh new file mode 100755 index 00000000000..ec2fc58066d --- /dev/null +++ b/python/tests/iotests.sh @@ -0,0 +1,2 @@ +#!/bin/sh -e +PYTHONPATH=3D../tests/qemu-iotests/ python3 -m linters --=20 2.31.1 From nobody Sat May 18 06:50:36 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1626803276482521.1997909444348; Tue, 20 Jul 2021 10:47:56 -0700 (PDT) Received: from localhost ([::1]:55376 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5tqV-0006AR-D2 for importer@patchew.org; Tue, 20 Jul 2021 13:47:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55928) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td7-0006RT-7L for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:30936) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5td5-0005mN-Ch for qemu-devel@nongnu.org; Tue, 20 Jul 2021 13:34:04 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-373-paKxyZL2PiOQOc3ethAfzg-1; Tue, 20 Jul 2021 13:34:01 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2BC441868408; Tue, 20 Jul 2021 17:34:00 +0000 (UTC) Received: from scv.redhat.com (ovpn-119-193.rdu2.redhat.com [10.10.119.193]) by smtp.corp.redhat.com (Postfix) with ESMTP id 05E181962F; Tue, 20 Jul 2021 17:33:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1626802442; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bT2JRCelTLmpuKyrWpH94ls7oTdNseuKE9FQE08O8G4=; b=EdWmzKLKbJfHfJv5nzozr/pxQt/q/xzF5RSggUwOEkLjxtH59zAiEQRF+94pdNpR25N5Cz eU3bDblMsAzXMBYHq4xLt1F7pTF39bbYngAJ9xJ7QPFRc8nnsAoNR5K5aJK1pVMlw76Gkz eZbSwjHEcGrl6FYFqP+kTKHPH0aLh4Y= X-MC-Unique: paKxyZL2PiOQOc3ethAfzg-1 From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 17/17] iotests/linters: check mypy files all at once Date: Tue, 20 Jul 2021 13:33:36 -0400 Message-Id: <20210720173336.1876937-18-jsnow@redhat.com> In-Reply-To: <20210720173336.1876937-1-jsnow@redhat.com> References: <20210720173336.1876937-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.474, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Markus Armbruster , Max Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1626803277776100001 Content-Type: text/plain; charset="utf-8" We can circumvent the '__main__' redefinition problem by passing --scripts-are-modules. Take mypy out of the loop per-filename and check everything in one go: it's quite a bit faster. Signed-off-by: John Snow --- tests/qemu-iotests/linters.py | 56 ++++++++++++++++------------------- 1 file changed, 26 insertions(+), 30 deletions(-) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index 242cb9c3e69..b95425e81aa 100755 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -86,37 +86,33 @@ def run_linters( print('=3D=3D=3D mypy =3D=3D=3D') sys.stdout.flush() =20 - # We have to call mypy separately for each file. Otherwise, it - # will interpret all given files as belonging together (i.e., they - # may not both define the same classes, etc.; most notably, they - # must not both define the __main__ module). - for filename in files: - p =3D subprocess.run( - ( - 'python3', '-m', 'mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - filename, - ), - cwd=3Ddirectory, - env=3Denv, - check=3DFalse, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.STDOUT, - universal_newlines=3DTrue - ) + p =3D subprocess.run( + ( + 'python3', '-m', 'mypy', + '--warn-unused-configs', + '--disallow-subclassing-any', + '--disallow-any-generics', + '--disallow-incomplete-defs', + '--disallow-untyped-decorators', + '--no-implicit-optional', + '--warn-redundant-casts', + '--warn-unused-ignores', + '--no-implicit-reexport', + '--namespace-packages', + '--scripts-are-modules', + *files, + ), + cwd=3Ddirectory, + env=3Denv, + check=3DFalse, + stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT, + universal_newlines=3DTrue + ) =20 - ret +=3D p.returncode - if p.returncode !=3D 0: - print(p.stdout) + ret +=3D p.returncode + if p.returncode !=3D 0: + print(p.stdout) =20 return ret =20 --=20 2.31.1