From nobody Wed May 1 21:46:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1620039778; cv=none; d=zohomail.com; s=zohoarc; b=VRRpdeLBC+1LtlluA1uaxHVdXI47ppFuVMUZfJnd3vETIrXa+FKKUELbem3EA/imHp25Z+DP9CPoZHQdRHXWDGkEDctsSUQg41KyH7X9XzO2/ZDtjcwSZ6nD27d34iOFylYmzunz3JlzrOV5UW82oJMP+2aG0DQ66tF0710Gpm0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620039778; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Ml1ydu8XKSxP1Q8Lrnm8dec/A8hAffqNR+5PqaZfm2o=; b=MlK56pDlXfOD/sZQWcZ44wmlPQ49csL3d0z2l4J6uuYaE3lMEUg6rbItfHeCFGw8kDW903460NEk9YTlZMejR5Fsx/GKNzAhAVNgS+S/rnY9lL2L8jF38ZJYzoZ3yUdHc1ugWrYMe2unqgvDF/X+wZiiv/gQ5y9070qNIMpP6hs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1620039778938870.0350409040988; Mon, 3 May 2021 04:02:58 -0700 (PDT) Received: from localhost ([::1]:36804 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ldWLp-00039W-Bw for importer@patchew.org; Mon, 03 May 2021 07:02:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33844) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ldWKK-0001rm-Oc for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:24 -0400 Received: from mail-ed1-x534.google.com ([2a00:1450:4864:20::534]:40785) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ldWKI-0004qx-0E for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:24 -0400 Received: by mail-ed1-x534.google.com with SMTP id c22so5762837edn.7 for ; Mon, 03 May 2021 04:01:21 -0700 (PDT) Received: from localhost.localdomain ([2001:b07:6468:f312:63a7:c72e:ea0e:6045]) by smtp.gmail.com with ESMTPSA id v16sm12041464edt.53.2021.05.03.04.01.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 May 2021 04:01:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ml1ydu8XKSxP1Q8Lrnm8dec/A8hAffqNR+5PqaZfm2o=; b=EQcZawyVDL2cBkBqNDk4mt4PMwQKsxJMuTizNqA2C189sOq93rjNSNHtoKl9IDEuib baXVVTLArQG5tqJCbv+qXD1xV5XrkOJIX2nx2YQk1SjN0IuxKZeiNMkc1uni4HJdAuyM 9YkRq+5qubm9KI4wx3Vpc/ZfLrtaTPWv8zgJnF4DpgeKJPQQ1q6s27G8ERuoUayjALUm jNJl6vb55VqhFoOB7NwU2ql8oYX89yFVQH3wQL/79HWgVZpI0OhDJFjgL5ZfJvO5y3kC /dx1R98dETLJaIr4aRcpIYRiFrsRMkucLLlDRA6eC7+iR/etM43m3O1PjMl3JVNpSHJc bQdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=Ml1ydu8XKSxP1Q8Lrnm8dec/A8hAffqNR+5PqaZfm2o=; b=e9QLSBmyDkqIgp2HUj0iCzXF7DK+8km3d5Ghx04R3x1vkI4hoEKroo3OdwL0/Bmk1n syykfWfVTkzypsHgM4iNmzWqNPGJ4H5VXC2KRBTvJ10JlZUOKKVek5E2VF69PJ41aeZ0 YU24xAr0rpVzuhD1K+K1e4KjH2D9WKYNEjqnzpO/Ot+6TWE4gYOzcbFc76oe4fIz7h5S OXK8ko28vLh2PBkHf8/Y3fHZ1woPtg/h04stUfwOICJ4zZUeKTGLeQchAk3zsCPCJKs/ Iob5GAiv5fdjFrmiHKzdbp+/MBNavpHAwmsVxCAWERB43ay18kh6NhzKagNNKv+R9WGk Hfgg== X-Gm-Message-State: AOAM5314z8Cul5WmCpY2UsMLpmIis2C7YW2ZOIT2BJdMuIkmO8cFOiiB BYTo6+M7XgWKWEvN4saqz5mHngbizEQ= X-Google-Smtp-Source: ABdhPJyj06e6rwmVP4YprZVTEJ51E3X8/KVlswX2yo2WVYpjBdm0O2pt9CTz9/GQyagiPjremLLR9A== X-Received: by 2002:aa7:c34b:: with SMTP id j11mr19778380edr.188.1620039675072; Mon, 03 May 2021 04:01:15 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v4 1/5] qemu-iotests: do not buffer the test output Date: Mon, 3 May 2021 13:01:06 +0200 Message-Id: <20210503110110.476887-2-pbonzini@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503110110.476887-1-pbonzini@redhat.com> References: <20210503110110.476887-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::534; envelope-from=paolo.bonzini@gmail.com; helo=mail-ed1-x534.google.com X-Spam_score_int: 5 X-Spam_score: 0.5 X-Spam_bar: / X-Spam_report: (0.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, PDS_OTHER_BAD_TLD=1.997, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eesposit@redhat.com, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Instead of buffering the test output into a StringIO, patch it on the fly by wrapping sys.stdout's write method. This can be done unconditionally, even if using -d, which makes execute_unittest a bit simpler. Signed-off-by: Paolo Bonzini Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Emanuele Giuseppe Esposito Message-Id: <20210323181928.311862-2-pbonzini@redhat.com> --- tests/qemu-iotests/240.out | 8 ++-- tests/qemu-iotests/245.out | 8 ++-- tests/qemu-iotests/295.out | 6 +-- tests/qemu-iotests/296.out | 8 ++-- tests/qemu-iotests/iotests.py | 70 ++++++++++++++++++++--------------- 5 files changed, 56 insertions(+), 44 deletions(-) diff --git a/tests/qemu-iotests/240.out b/tests/qemu-iotests/240.out index e0982831ae..89ed25e506 100644 --- a/tests/qemu-iotests/240.out +++ b/tests/qemu-iotests/240.out @@ -15,7 +15,7 @@ {"return": {}} {"execute": "blockdev-del", "arguments": {"node-name": "hd0"}} {"return": {}} -=3D=3DAttach two SCSI disks using the same block device and the same iothr= ead=3D=3D +.=3D=3DAttach two SCSI disks using the same block device and the same ioth= read=3D=3D {"execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name"= : "hd0", "read-only": true, "read-zeroes": true}} {"return": {}} {"execute": "object-add", "arguments": {"id": "iothread0", "qom-type": "io= thread"}} @@ -32,7 +32,7 @@ {"return": {}} {"execute": "blockdev-del", "arguments": {"node-name": "hd0"}} {"return": {}} -=3D=3DAttach two SCSI disks using the same block device but different ioth= reads=3D=3D +.=3D=3DAttach two SCSI disks using the same block device but different iot= hreads=3D=3D {"execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name"= : "hd0", "read-only": true, "read-zeroes": true}} {"return": {}} {"execute": "object-add", "arguments": {"id": "iothread0", "qom-type": "io= thread"}} @@ -55,7 +55,7 @@ {"return": {}} {"execute": "blockdev-del", "arguments": {"node-name": "hd0"}} {"return": {}} -=3D=3DAttach a SCSI disks using the same block device as a NBD server=3D= =3D +.=3D=3DAttach a SCSI disks using the same block device as a NBD server=3D= =3D {"execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name"= : "hd0", "read-only": true, "read-zeroes": true}} {"return": {}} {"execute": "nbd-server-start", "arguments": {"addr": {"data": {"path": "S= OCK_DIR/PID-nbd.sock"}, "type": "unix"}}} @@ -68,7 +68,7 @@ {"return": {}} {"execute": "device_add", "arguments": {"drive": "hd0", "driver": "scsi-hd= ", "id": "scsi-hd0"}} {"return": {}} -.... +. ---------------------------------------------------------------------- Ran 4 tests =20 diff --git a/tests/qemu-iotests/245.out b/tests/qemu-iotests/245.out index 4b33dcaf5c..99c12f4f98 100644 --- a/tests/qemu-iotests/245.out +++ b/tests/qemu-iotests/245.out @@ -1,16 +1,16 @@ -{"execute": "job-finalize", "arguments": {"id": "commit0"}} +..{"execute": "job-finalize", "arguments": {"id": "commit0"}} {"return": {}} {"data": {"id": "commit0", "type": "commit"}, "event": "BLOCK_JOB_PENDING"= , "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} {"data": {"device": "commit0", "len": 3145728, "offset": 3145728, "speed":= 0, "type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micro= seconds": "USECS", "seconds": "SECS"}} -{"execute": "job-finalize", "arguments": {"id": "stream0"}} +...{"execute": "job-finalize", "arguments": {"id": "stream0"}} {"return": {}} {"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING"= , "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} {"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed":= 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micro= seconds": "USECS", "seconds": "SECS"}} -{"execute": "job-finalize", "arguments": {"id": "stream0"}} +.{"execute": "job-finalize", "arguments": {"id": "stream0"}} {"return": {}} {"data": {"id": "stream0", "type": "stream"}, "event": "BLOCK_JOB_PENDING"= , "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} {"data": {"device": "stream0", "len": 3145728, "offset": 3145728, "speed":= 0, "type": "stream"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"micro= seconds": "USECS", "seconds": "SECS"}} -..................... +............... ---------------------------------------------------------------------- Ran 21 tests =20 diff --git a/tests/qemu-iotests/295.out b/tests/qemu-iotests/295.out index ad34b2ca2c..5ff91f116c 100644 --- a/tests/qemu-iotests/295.out +++ b/tests/qemu-iotests/295.out @@ -4,7 +4,7 @@ {"return": {}} {"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} {"return": {}} -{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +.{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} {"return": {}} {"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} {"return": {}} @@ -13,7 +13,7 @@ Job failed: Invalid password, cannot unlock any keyslot {"return": {}} {"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} {"return": {}} -{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +.{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} {"return": {}} {"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} {"return": {}} @@ -33,7 +33,7 @@ Job failed: All the active keyslots match the (old) passw= ord that was given and {"return": {}} {"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} {"return": {}} -... +. ---------------------------------------------------------------------- Ran 3 tests =20 diff --git a/tests/qemu-iotests/296.out b/tests/qemu-iotests/296.out index cb2859a15c..6c69735604 100644 --- a/tests/qemu-iotests/296.out +++ b/tests/qemu-iotests/296.out @@ -13,7 +13,7 @@ Job failed: Failed to get shared "consistent read" lock qemu-img: Failed to get shared "consistent read" lock Is another process using the image [TEST_DIR/test.img]? =20 -Formatting 'TEST_DIR/test.img', fmt=3Dluks size=3D1048576 key-secret=3Dkey= sec0 iter-time=3D10 +.Formatting 'TEST_DIR/test.img', fmt=3Dluks size=3D1048576 key-secret=3Dke= ysec0 iter-time=3D10 =20 Job failed: Block node is read-only {"execute": "job-dismiss", "arguments": {"id": "job0"}} @@ -26,15 +26,15 @@ Job failed: Failed to get shared "consistent read" lock {"return": {}} {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} -Formatting 'TEST_DIR/test.img', fmt=3Dluks size=3D1048576 key-secret=3Dkey= sec0 iter-time=3D10 +.Formatting 'TEST_DIR/test.img', fmt=3Dluks size=3D1048576 key-secret=3Dke= ysec0 iter-time=3D10 =20 {"return": {}} {"error": {"class": "GenericError", "desc": "Failed to get \"write\" lock"= }} -Formatting 'TEST_DIR/test.img', fmt=3Dluks size=3D1048576 key-secret=3Dkey= sec0 iter-time=3D10 +.Formatting 'TEST_DIR/test.img', fmt=3Dluks size=3D1048576 key-secret=3Dke= ysec0 iter-time=3D10 =20 {"return": {}} {"return": {}} -.... +. ---------------------------------------------------------------------- Ran 4 tests =20 diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 5af0182895..55a017577f 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -20,7 +20,6 @@ import bz2 from collections import OrderedDict import faulthandler -import io import json import logging import os @@ -32,7 +31,7 @@ import sys import time from typing import (Any, Callable, Dict, Iterable, - List, Optional, Sequence, Tuple, TypeVar) + List, Optional, Sequence, TextIO, Tuple, Type, TypeVar) import unittest =20 from contextlib import contextmanager @@ -1271,37 +1270,50 @@ def func_wrapper(*args, **kwargs): return func(*args, **kwargs) return func_wrapper =20 +# We need to filter out the time taken from the output so that +# qemu-iotest can reliably diff the results against master output, +# and hide skipped tests from the reference output. + +class ReproducibleTestResult(unittest.TextTestResult): + def addSkip(self, test, reason): + # Same as TextTestResult, but print dot instead of "s" + unittest.TestResult.addSkip(self, test, reason) + if self.showAll: + self.stream.writeln("skipped {0!r}".format(reason)) + elif self.dots: + self.stream.write(".") + self.stream.flush() + +class ReproducibleStreamWrapper: + def __init__(self, stream: TextIO): + self.stream =3D stream + + def __getattr__(self, attr): + if attr in ('stream', '__getstate__'): + raise AttributeError(attr) + return getattr(self.stream, attr) + + def write(self, arg=3DNone): + arg =3D re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests', ar= g) + arg =3D re.sub(r' \(skipped=3D\d+\)', r'', arg) + self.stream.write(arg) + +class ReproducibleTestRunner(unittest.TextTestRunner): + def __init__(self, stream: Optional[TextIO] =3D None, + resultclass: Type[unittest.TestResult] =3D ReproducibleTestRe= sult, + **kwargs: Any) -> None: + rstream =3D ReproducibleStreamWrapper(stream or sys.stdout) + super().__init__(stream=3Drstream, # type: ignore + descriptions=3DTrue, + resultclass=3Dresultclass, + **kwargs) + def execute_unittest(debug=3DFalse): """Executes unittests within the calling module.""" =20 verbosity =3D 2 if debug else 1 - - if debug: - output =3D sys.stdout - else: - # We need to filter out the time taken from the output so that - # qemu-iotest can reliably diff the results against master output. - output =3D io.StringIO() - - runner =3D unittest.TextTestRunner(stream=3Doutput, descriptions=3DTru= e, - verbosity=3Dverbosity) - try: - # unittest.main() will use sys.exit(); so expect a SystemExit - # exception - unittest.main(testRunner=3Drunner) - finally: - # We need to filter out the time taken from the output so that - # qemu-iotest can reliably diff the results against master output. - if not debug: - out =3D output.getvalue() - out =3D re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests'= , out) - - # Hide skipped tests from the reference output - out =3D re.sub(r'OK \(skipped=3D\d+\)', 'OK', out) - out_first_line, out_rest =3D out.split('\n', 1) - out =3D out_first_line.replace('s', '.') + '\n' + out_rest - - sys.stderr.write(out) + runner =3D ReproducibleTestRunner(verbosity=3Dverbosity) + unittest.main(testRunner=3Drunner) =20 def execute_setup_common(supported_fmts: Sequence[str] =3D (), supported_platforms: Sequence[str] =3D (), --=20 2.31.1 From nobody Wed May 1 21:46:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1620039777; cv=none; d=zohomail.com; s=zohoarc; b=KdGxw0iVfUjzSngUzXPzwWtwgA9magn828KQLjxBwaFCcD5pzzM0FmNNzpiEOZXgNvgA7S+n/sk5eHYTyqdZCXybPYh3mUDQEG7G3slSy0sSNraVDfNhsf7qG6fqKngNu773g1epsAN8h4mlLha0PdZJaUr4lkFHRZF9UbUheE0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620039777; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=f2TUvLfi/bunpIY74wAKSOpvmTKQ7RTzmZv40TStSOI=; b=UddWP0v/2V5BZtKxI3VCF5f5tzEBv0eLDbfdBSvPdaVysdSUqITmF3oexcjK/UnW606UZKBKqoz7ChUKzM/wPuJl9b2cS4IjbIhhCJgWb8DHApN6BBzrw21wntZLb43ev74UwsVvdu9NsV4e0+KSbKMg7KCh6GdL1qjxO3RVzNc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1620039777971518.8525784486295; Mon, 3 May 2021 04:02:57 -0700 (PDT) Received: from localhost ([::1]:36736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ldWLo-00037e-88 for importer@patchew.org; Mon, 03 May 2021 07:02:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33834) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ldWKK-0001rg-D2 for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:24 -0400 Received: from mail-ed1-x52e.google.com ([2a00:1450:4864:20::52e]:45055) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ldWKI-0004rG-8p for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:24 -0400 Received: by mail-ed1-x52e.google.com with SMTP id bf4so5749607edb.11 for ; Mon, 03 May 2021 04:01:21 -0700 (PDT) Received: from localhost.localdomain ([2001:b07:6468:f312:63a7:c72e:ea0e:6045]) by smtp.gmail.com with ESMTPSA id v16sm12041464edt.53.2021.05.03.04.01.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 May 2021 04:01:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f2TUvLfi/bunpIY74wAKSOpvmTKQ7RTzmZv40TStSOI=; b=P+6SPlcgcgaNAA58JqJJAcXSXv94sssT/ETaL4VCwHx9/WiLUNvpHWEiCL69xSWrFZ Fr8UC89Hh/tkg884MPm9siyDBybq2DIUkgfgFcLFxPCuw0l53CWT0k2fjqcbZc7O4+aY 2pmcoBTLx4mwvnIFvoJFHADP7SpMqhYF1NfrnZiUso85i0ECeU92QsykEZOQG2TNzcAs juJoPTU8Omo8S26w0tLAhaxzfGiExtdjGtQ+nH2LS2boF+quE5BKH4XLasfUHUDiwCnX GheLMHih0//rZOJHveHGFbs6Bn2fYX72bNOEYGPdae/WhN4oU65vjrNXB9A2hN37mfq3 Q8dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=f2TUvLfi/bunpIY74wAKSOpvmTKQ7RTzmZv40TStSOI=; b=gwhkgt2/O/lknCyLrwTlLlaxyV5Z7qZuuFlizDyLPh86kC2IPaeVpjP3/vQ9s9/xoF plD0QMUxASt5E1wO3uNECO8lILv9jlp4ABOhHfZ2N0Yx+ikIAwsp4e3zUDl33qQ10sCp HPfFUaNUJ+9PU6LGLg+qnpiKcH2KtTVIQZdnEG7cJ7E98nYPPJ/1h4iZx/2l2dbzD8Ji 4DQNfrqAQbd0TO+Cd9oVQMHnexSCLH20MAGYzMzYx7F1XtHJSG1RHrofNPgWD/slMXN7 FaKkQNdv/psGFx4CtZHBPU7OCwoWPzyrsjRUBwPzokphD/pZZmixI1QauyihYn+EYNJv f4Xg== X-Gm-Message-State: AOAM533HGv6yycTNI2eP9B/q2kyvSrQdwDm/ksMHsISchDIuCbLkfWrJ MvVTQ9oDG39lPxoh2FzCv70w9SeB5Q0= X-Google-Smtp-Source: ABdhPJzAvPzbHw0WszKbMwemlxZyxhgZfmWIYMdLzS+oTUesOna/NvS2+hZyZWKAjHYbdRAqn7jkPw== X-Received: by 2002:a05:6402:30bc:: with SMTP id df28mr18021666edb.67.1620039675964; Mon, 03 May 2021 04:01:15 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v4 2/5] qemu-iotests: allow passing unittest.main arguments to the test scripts Date: Mon, 3 May 2021 13:01:07 +0200 Message-Id: <20210503110110.476887-3-pbonzini@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503110110.476887-1-pbonzini@redhat.com> References: <20210503110110.476887-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::52e; envelope-from=paolo.bonzini@gmail.com; helo=mail-ed1-x52e.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eesposit@redhat.com, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Python test scripts that use unittest consist of multiple tests. unittest.main allows selecting which tests to run, but currently this is not possible because the iotests wrapper ignores sys.argv. unittest.main command line options also allow the user to pick the desired options for verbosity, failfast mode, etc. While "-d" is currently translated to "-v", it also enables extra debug output, and other options are not available at all. These command line options only work if the unittest.main testRunner argument is a type, rather than a TestRunner instance. Therefore, pass the class name and "verbosity" argument to unittest.main, and adjust for the different default warnings between TextTestRunner and unittest.main. Signed-off-by: Paolo Bonzini Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Emanuele Giuseppe Esposito Message-Id: <20210323181928.311862-3-pbonzini@redhat.com> --- tests/qemu-iotests/iotests.py | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 55a017577f..5ead94229f 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -1308,12 +1308,16 @@ def __init__(self, stream: Optional[TextIO] =3D Non= e, resultclass=3Dresultclass, **kwargs) =20 -def execute_unittest(debug=3DFalse): +def execute_unittest(argv: List[str], debug: bool =3D False) -> None: """Executes unittests within the calling module.""" =20 - verbosity =3D 2 if debug else 1 - runner =3D ReproducibleTestRunner(verbosity=3Dverbosity) - unittest.main(testRunner=3Drunner) + # Some tests have warnings, especially ResourceWarnings for unclosed + # files and sockets. Ignore them for now to ensure reproducibility of + # the test output. + unittest.main(argv=3Dargv, + testRunner=3DReproducibleTestRunner, + verbosity=3D2 if debug else 1, + warnings=3DNone if sys.warnoptions else 'ignore') =20 def execute_setup_common(supported_fmts: Sequence[str] =3D (), supported_platforms: Sequence[str] =3D (), @@ -1350,7 +1354,7 @@ def execute_test(*args, test_function=3DNone, **kwarg= s): =20 debug =3D execute_setup_common(*args, **kwargs) if not test_function: - execute_unittest(debug) + execute_unittest(sys.argv, debug) else: test_function() =20 --=20 2.31.1 From nobody Wed May 1 21:46:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1620039908; cv=none; d=zohomail.com; s=zohoarc; b=nGvvU18osg9yR2VpmAKHERrz+3f9DzpO6joswAW0Oa2SHKzNvGyVTqWrudFS5yY5W09cqJqu8RWbT58Ixjyb7PguayiuIb3Du2T2pQXAph2FXTyakEzUTjMipRmZLg0sEAvYwt3yLcFtjxQHswB6Mg3gkqbd+ODUqFgsXD9lpH8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620039908; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3ICg7BIgvQas5k23bDctJ3vrOhEuRu5Sz5mmBFT/OUU=; b=mAUdnlp25ZId0Qj+Ee6JZtLQxtkI+bkx8Y6sscCSpxo8dgc3KmRi04s95LNpF18ovufvFDKhT9vxBjqFrP546+vLiqvkKheg8BxnCxXQbHshMKmo+Te8qMNJxomg/vClmncwTjXnKk4noAA4FwxYvCCV0HN7ykp2EDS1aicPfUk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1620039908805186.98719855276534; Mon, 3 May 2021 04:05:08 -0700 (PDT) Received: from localhost ([::1]:42244 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ldWNv-0005To-MQ for importer@patchew.org; Mon, 03 May 2021 07:05:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33848) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ldWKL-0001rs-2j for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:25 -0400 Received: from mail-ed1-x529.google.com ([2a00:1450:4864:20::529]:40775) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ldWKJ-0004s3-23 for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:24 -0400 Received: by mail-ed1-x529.google.com with SMTP id c22so5762900edn.7 for ; Mon, 03 May 2021 04:01:22 -0700 (PDT) Received: from localhost.localdomain ([2001:b07:6468:f312:63a7:c72e:ea0e:6045]) by smtp.gmail.com with ESMTPSA id v16sm12041464edt.53.2021.05.03.04.01.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 May 2021 04:01:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3ICg7BIgvQas5k23bDctJ3vrOhEuRu5Sz5mmBFT/OUU=; b=Qw4KdZ0FS5TrBj+X8gQFEoWiQnpD1E2pVopqiYjMKujQ8ParIRLZ/NV+mNFXDmDACT RukM0hWNscv1F0NqVaLHw+ygjUg1ACEqJWKCOXrw6ZsfKy7mZvHNDl6MCnGQKHNLZiTq 3uClU81oCzhj3siAyj1MyGrUFxjKueVGWZg6eyH9Dvk0N1yN2FO96XcuU0W9uO8s+cCL /OCR+0DX37LTJecxQhCHmxV99GAjryRhX4tf63cIjpAcqa03S/SHdkk+2h7FjOsiXw0X fH/3ZloU5ZkF4ZGychy7g7Gt8GYLW86/ExGIEAIeVMO36RZAwWFPHzm09xz5kFhqpMI7 ksIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=3ICg7BIgvQas5k23bDctJ3vrOhEuRu5Sz5mmBFT/OUU=; b=YlAXKvckjvyOM00L7VqPzQOSE41MCLPLZHPHr6AgFeHYxL3819N8/8YhAioAjNa4Ws tNFFQs9MtWFHtQVDhSasOF0pPwr14IHt1Tyw8tSjYlpJ4ZSUcJzhZGdMKrdkbtCwJQif jlOehEUkZ+jAATONiCoSEFTvAwzexGcWZcr9vbeLHfmMzxrYxo4Wa1BR2xoH+Qc2g5Zt OBGlMlZY+XalJRq1Y97xeabhhjsgCRmNBJh98irkIjmxPPkMwkxTpX1oCEwOUKNF0oEc V1QoVtw/EkIFGg5mR5GnPDRYkoE6VKFOJxRJZvt0aGjStanEDuwvAukdaI+B9OOsNOpE 6itg== X-Gm-Message-State: AOAM533ig/RM7dK8leEAcqW4+rBFGX+HE0I7xra35DcXyQVp5WRbpqIW FUlj+4famHa6ekQuUCqmONhyTKMYAlI= X-Google-Smtp-Source: ABdhPJx93RqqyY81XKE5D4OLRM7i7MoGn+shWiX800jv3N3QFETXtl473RmPkFDKHawEJF8qqe0xJg== X-Received: by 2002:a05:6402:51d4:: with SMTP id r20mr19173829edd.112.1620039676772; Mon, 03 May 2021 04:01:16 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v4 3/5] qemu-iotests: move command line and environment handling from TestRunner to TestEnv Date: Mon, 3 May 2021 13:01:08 +0200 Message-Id: <20210503110110.476887-4-pbonzini@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503110110.476887-1-pbonzini@redhat.com> References: <20210503110110.476887-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::529; envelope-from=paolo.bonzini@gmail.com; helo=mail-ed1-x529.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eesposit@redhat.com, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" In the next patch, "check" will learn how to execute a test script without going through TestRunner. To enable this, keep only the text output and subprocess handling in the TestRunner; move into TestEnv the logic to prepare for running a subprocess. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Tested-by: Emanuele Giuseppe Esposito Message-Id: <20210323181928.311862-4-pbonzini@redhat.com> --- tests/qemu-iotests/testenv.py | 17 ++++++++++++++++- tests/qemu-iotests/testrunner.py | 14 +------------- 2 files changed, 17 insertions(+), 14 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 6d27712617..fca3a609e0 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -25,7 +25,7 @@ import random import subprocess import glob -from typing import Dict, Any, Optional, ContextManager +from typing import List, Dict, Any, Optional, ContextManager =20 =20 def isxfile(path: str) -> bool: @@ -74,6 +74,21 @@ class TestEnv(ContextManager['TestEnv']): 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYN= TAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTU= RB_'] =20 + def prepare_subprocess(self, args: List[str]) -> Dict[str, str]: + if self.debug: + args.append('-d') + + with open(args[0], encoding=3D"utf-8") as f: + try: + if f.readline().rstrip() =3D=3D '#!/usr/bin/env python3': + args.insert(0, self.python) + except UnicodeDecodeError: # binary test? for future. + pass + + os_env =3D os.environ.copy() + os_env.update(self.get_env()) + return os_env + def get_env(self) -> Dict[str, str]: env =3D {} for v in self.env_variables: diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunn= er.py index 1fc61fcaa3..519924dc81 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -129,7 +129,6 @@ class TestRunner(ContextManager['TestRunner']): def __init__(self, env: TestEnv, makecheck: bool =3D False, color: str =3D 'auto') -> None: self.env =3D env - self.test_run_env =3D self.env.get_env() self.makecheck =3D makecheck self.last_elapsed =3D LastElapsedTime('.last-elapsed-cache', env) =20 @@ -243,18 +242,7 @@ def do_run_test(self, test: str) -> TestResult: silent_unlink(p) =20 args =3D [str(f_test.resolve())] - if self.env.debug: - args.append('-d') - - with f_test.open(encoding=3D"utf-8") as f: - try: - if f.readline().rstrip() =3D=3D '#!/usr/bin/env python3': - args.insert(0, self.env.python) - except UnicodeDecodeError: # binary test? for future. - pass - - env =3D os.environ.copy() - env.update(self.test_run_env) + env =3D self.env.prepare_subprocess(args) =20 t0 =3D time.time() with f_bad.open('w', encoding=3D"utf-8") as f: --=20 2.31.1 From nobody Wed May 1 21:46:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1620039907; cv=none; d=zohomail.com; s=zohoarc; b=Vb+LYMIpGWrdpxZscWakmo3Nyp8FgTnsdDtMY/FFOzXAPdkWomyn0ApevclLYhpj0a6rBH+S7IRW0pjzeWBs0nqfNFQBqkTXHZSEuFKVYx71BVam47HXPAL2IfzGlCDWY1/Tx8b8wdQnoJa4XrvKC/MG+6RCWH6BUgH5LOfarkE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620039907; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CiZrOooOA5sLacn8XFpn97ElSByrel0Yqy/GkIYhy78=; b=UvUXF4eyuGDYAyuGqRTWGsYLHwq+8MOu/Fmz4B8hoXswph459eK3lXUC7ShtL8SSUKwPwzsWsLiHQSVwf0oypLTRJvBfv8b8gMjUk8G/41VogTx0F9pSm0VQCDlmMUEjkC8Yo55hNDw+kumBP+O8jbl05+damjqNpt9GUZQmaUM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1620039907197958.5037014000267; Mon, 3 May 2021 04:05:07 -0700 (PDT) Received: from localhost ([::1]:42168 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ldWNu-0005Ru-4f for importer@patchew.org; Mon, 03 May 2021 07:05:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33852) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ldWKL-0001ry-FN for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:25 -0400 Received: from mail-ed1-x529.google.com ([2a00:1450:4864:20::529]:33661) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ldWKJ-0004t4-RF for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:25 -0400 Received: by mail-ed1-x529.google.com with SMTP id b17so2744272ede.0 for ; Mon, 03 May 2021 04:01:23 -0700 (PDT) Received: from localhost.localdomain ([2001:b07:6468:f312:63a7:c72e:ea0e:6045]) by smtp.gmail.com with ESMTPSA id v16sm12041464edt.53.2021.05.03.04.01.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 May 2021 04:01:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CiZrOooOA5sLacn8XFpn97ElSByrel0Yqy/GkIYhy78=; b=qe6JWCPTGRVjPNg/ZPqM+ZBd8qV1uPqAEYTFuDj5hzSxCoRrqeNHY6JWk2nuhQk9B/ zhTy8btk3sMU1WFfVz7OAxXau6qgU9PnGo+4/WiTYLW8V2RRe4KqaJgovr6qTUdgSdKU gE71huYr6Vx94DuS+jmolgBYtmtzDxwAv7NePRf97xhRhUZ2sh5U+QwZhlr0rh+xRyuI dWUYqXOZ7NCGayeWn72XlmhHZGQbQHKp4xjge+s/Imq7rGOwYhBqEJ+FiuKUj55XJdmP 5BSxZUIhMKkmHT5w8vASjuKyb53hKA9uyQGU6MQgYShhXuHPw6JlXuYvACdRW1a5UrtI t+hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=CiZrOooOA5sLacn8XFpn97ElSByrel0Yqy/GkIYhy78=; b=jioyCBxGy0GwU0ihjoGqlsCDB9CfkHRNFqT8him1tv+3tBAO7g7Mqace92QsclNuns MV1v2LsaBHb+DvmSBZsr2nkFkj9GT5luQErreQHSpryZW/aN3Tw5L1WVks5PlbIuN9Yl ZDJ71dP7SurCObUklE12oegH83GV99Td1TcXTB0xCcqlxMI/OIlpRjpxXBTEJIzWxo0m s4Oj3oLPF3/0noV166DWAs8AbCPqcZHPCmWsnuKQXcesyO6fMxCKJtarK+pIGwY+q8BE uTVFv+0AQ+0OiB5vuLOW3wT7oHIpkHi54fqCkGNaWSnjFJnMBvx866xXwSskdaAfgwcA 6xOA== X-Gm-Message-State: AOAM5310SszdZGF2Q1I3dIvyf0OY4j/Wn1PRnrYW7KCZVs0TWZoD0QZX 49Lyb3s4p7EpQWunKJ7h1rjjkD2U6hs= X-Google-Smtp-Source: ABdhPJzPqIt0rS9K5kqoXgUqv1j80lYsU77kaI2oCagpTfiDLrsIuu996nkcXPQhulPsn4B7SWApMg== X-Received: by 2002:aa7:ccc4:: with SMTP id y4mr18841994edt.171.1620039677727; Mon, 03 May 2021 04:01:17 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v4 4/5] qemu-iotests: let "check" spawn an arbitrary test command Date: Mon, 3 May 2021 13:01:09 +0200 Message-Id: <20210503110110.476887-5-pbonzini@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503110110.476887-1-pbonzini@redhat.com> References: <20210503110110.476887-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::529; envelope-from=paolo.bonzini@gmail.com; helo=mail-ed1-x529.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eesposit@redhat.com, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Right now there is no easy way for "check" to print a reproducer command. Because such a reproducer command line would be huge, we can instead teach check to start a command of our choice. This can be for example a Python unit test with arguments to only run a specific subtest. Move the trailing empty line to print_env(), since it always looks better and one caller was not adding it. Signed-off-by: Paolo Bonzini Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Emanuele Giuseppe Esposito Message-Id: <20210323181928.311862-5-pbonzini@redhat.com> --- tests/qemu-iotests/check | 19 ++++++++++++++++++- tests/qemu-iotests/testenv.py | 3 ++- tests/qemu-iotests/testrunner.py | 1 - 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index d1c87ceaf1..7c9d3a0852 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -19,6 +19,9 @@ import os import sys import argparse +import shutil +from pathlib import Path + from findtests import TestFinder from testenv import TestEnv from testrunner import TestRunner @@ -101,7 +104,7 @@ def make_argparser() -> argparse.ArgumentParser: 'rerun failed ./check command, starting from the ' 'middle of the process.') g_sel.add_argument('tests', metavar=3D'TEST_FILES', nargs=3D'*', - help=3D'tests to run') + help=3D'tests to run, or "--" followed by a command= ') =20 return p =20 @@ -114,6 +117,20 @@ if __name__ =3D=3D '__main__': imgopts=3Dargs.imgopts, misalign=3Dargs.misalign, debug=3Dargs.debug, valgrind=3Dargs.valgrind) =20 + if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] =3D=3D '--': + if not args.tests: + sys.exit("missing command after '--'") + cmd =3D args.tests + env.print_env() + exec_pathstr =3D shutil.which(cmd[0]) + if exec_pathstr is None: + sys.exit('command not found: ' + cmd[0]) + exec_path =3D Path(exec_pathstr).resolve() + cmd[0] =3D str(exec_path) + full_env =3D env.prepare_subprocess(cmd) + os.chdir(exec_path.parent) + os.execve(cmd[0], cmd, full_env) + testfinder =3D TestFinder(test_dir=3Denv.source_iotests) =20 groups =3D args.groups.split(',') if args.groups else None diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index fca3a609e0..cd0e39b789 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -284,7 +284,8 @@ def print_env(self) -> None: PLATFORM -- {platform} TEST_DIR -- {TEST_DIR} SOCK_DIR -- {SOCK_DIR} -SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER}""" +SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} +""" =20 args =3D collections.defaultdict(str, self.get_env()) =20 diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunn= er.py index 519924dc81..2f56ac545d 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -316,7 +316,6 @@ def run_tests(self, tests: List[str]) -> bool: =20 if not self.makecheck: self.env.print_env() - print() =20 test_field_width =3D max(len(os.path.basename(t)) for t in tests) = + 2 =20 --=20 2.31.1 From nobody Wed May 1 21:46:02 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1620039781; cv=none; d=zohomail.com; s=zohoarc; b=PZGuBw6KjpA+6girdyOExrKfcxgVoMM2L55xk+kALuYsOsx/bamCkVoflg/u4qX2V73fNv1V+kkfSJLqqhw/KuNvLTEmWRs32bkY8xFhdLpfQj+SvsoMReq3Ioq3fjAPiNGGFAILOLzGYhsRGCPQi7M7knpOvmDi4ARVjI2SWDs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620039781; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+OffDydTkfXkZuszDV/SXSbi7BWb1yf+LZMphMSDbec=; b=QN8BV3kY/Vry0l0AunF4OEWE0XURbZMythuh3i+e236qmX/g6i8JQUTUyAxH5z9c5z5bKMkpcXL6jJEOLcZ5KaK2LZrC6G7Q25B7fs/YX+pscbmUL1UL5TFULbIJwNiW36ia9yK+nD0PuBw0SZPsqzPO4LwINLnUpllNXBosRW4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1620039781418526.7101708198808; Mon, 3 May 2021 04:03:01 -0700 (PDT) Received: from localhost ([::1]:37062 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ldWLs-0003Hh-3c for importer@patchew.org; Mon, 03 May 2021 07:03:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33880) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ldWKM-0001su-S8 for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:26 -0400 Received: from mail-ej1-x62c.google.com ([2a00:1450:4864:20::62c]:43663) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ldWKL-0004tI-A7 for qemu-devel@nongnu.org; Mon, 03 May 2021 07:01:26 -0400 Received: by mail-ej1-x62c.google.com with SMTP id l4so7179155ejc.10 for ; Mon, 03 May 2021 04:01:24 -0700 (PDT) Received: from localhost.localdomain ([2001:b07:6468:f312:63a7:c72e:ea0e:6045]) by smtp.gmail.com with ESMTPSA id v16sm12041464edt.53.2021.05.03.04.01.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 May 2021 04:01:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+OffDydTkfXkZuszDV/SXSbi7BWb1yf+LZMphMSDbec=; b=b3okjBnBsXD/O+h6jQNVHRjYYPbhWC9oCFMkIkB9joHlryQtFGlxshjqJVe4WqZfX+ Ow+HeuR828DJAvitcplkuHbEj4hd0hPU58WtwBhv0eeDKNhp9Lr3ILNcXZSDHhSR04cS g5h4SCqVUGJCbdLI8WenDkIUFnI+0eCuHMJ7xjspkc/YXg8AzyiYAqhkNp/wYZHz1HUE I/Fhaw83f/1CFo1W2bewHv1JZnUzILLNJ8lqrFEGEhp666BusmhfB+dLbfNSu2LA+DVV trw/OyXDKISRls3po/NlTXpMxL4piOlPiRNqO2y6BiDHWg36Yd27Khd4HjDPmqzjW8Ml 9HLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=+OffDydTkfXkZuszDV/SXSbi7BWb1yf+LZMphMSDbec=; b=bjQQxGE0yhLRmvbW6qBIsTRZAJUgxlG1MJVEL5jbeqcv/uo2Fh2hpR1AZ8uyK4/dtn qFC9Lt3Ps8I+qPUv6SaDkDvc2ILqaXa9F+zSHCDOEAS0THwA0xLP6ZGunNOTvGus1woA IbRD9wSNtRA2pVmZpCitreuVU92wwrGeMlMEXLCWhcNtKiI0tc8gfbNt8kpt6QCMSZM4 ZagPdA9EXoR1xsvDthe+qYMCjt/ngHGvJVoj8RLWlvYkjVbaQVmnxkTXBgDmDxIyHc4b HqcgYsEJdwMh61AqRogreaDi6sTwinYKALKj+KZa9Au0GIuKMNBiph4xNlF/YkvXRfUw L3lw== X-Gm-Message-State: AOAM530o5dreGhWu7V14zb7fJigiQbKfOfD9j3x9IFcHnjg22HJePHCw eIZwoUMvBp3Eg+0u5Tuye+yS9/M5BKk= X-Google-Smtp-Source: ABdhPJxa3aJf4SwnHJjl2Uc0dq9nc8t70IwSzrgM0MT8+OtplP1q/Su8Zq1mI82GzCNiggq6wl9Wxw== X-Received: by 2002:a17:906:430f:: with SMTP id j15mr16055646ejm.543.1620039678548; Mon, 03 May 2021 04:01:18 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v4 5/5] qemu-iotests: fix case of SOCK_DIR already in the environment Date: Mon, 3 May 2021 13:01:10 +0200 Message-Id: <20210503110110.476887-6-pbonzini@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210503110110.476887-1-pbonzini@redhat.com> References: <20210503110110.476887-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::62c; envelope-from=paolo.bonzini@gmail.com; helo=mail-ej1-x62c.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eesposit@redhat.com, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Due to a typo, in this case the SOCK_DIR was not being created. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Tested-by: Emanuele Giuseppe Esposito Message-Id: <20210323181928.311862-6-pbonzini@redhat.com> --- tests/qemu-iotests/testenv.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index cd0e39b789..0c3fe75636 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -120,7 +120,7 @@ def init_directories(self) -> None: try: self.sock_dir =3D os.environ['SOCK_DIR'] self.tmp_sock_dir =3D False - Path(self.test_dir).mkdir(parents=3DTrue, exist_ok=3DTrue) + Path(self.sock_dir).mkdir(parents=3DTrue, exist_ok=3DTrue) except KeyError: self.sock_dir =3D tempfile.mkdtemp() self.tmp_sock_dir =3D True --=20 2.31.1