From nobody Sun May 5 20:46:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642691672966593.406738041441; Thu, 20 Jan 2022 07:14:32 -0800 (PST) Received: from localhost ([::1]:46558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAZ8x-0004XD-He for importer@patchew.org; Thu, 20 Jan 2022 10:14:31 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55864) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAVjq-0007MY-SH; Thu, 20 Jan 2022 06:36:24 -0500 Received: from [2a00:1450:4864:20::329] (port=38726 helo=mail-wm1-x329.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nAVjo-0001pb-79; Thu, 20 Jan 2022 06:36:22 -0500 Received: by mail-wm1-x329.google.com with SMTP id ay14-20020a05600c1e0e00b0034d7bef1b5dso14515175wmb.3; Thu, 20 Jan 2022 03:36:08 -0800 (PST) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id w8sm2653423wrq.3.2022.01.20.03.36.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jan 2022 03:36:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4n8YX6duwnriKZKmbthkv+dt+z7zVqKii17JMYGRe8k=; b=nZAbIhpv37md8UbGEyjm/1RTpMDe4JJyqz7x0A6Vg9kJzfag87A7HI9HACeVThzKEf UElg5rWTbFXE7KoSxqum1M4OoEYOjSMEivIIYs1NGqKgK/buyFC+MZ0Xn9kPu+ihfDPZ ZXMc/W5zSGDzMj3aeqD+cCq7j6BvFeUplXFK4dOfTPHWsFab3bhoQbnnWpm5ufX8h54n PhULhT9IY1CE7p8IZyiqM3zaxG//fKZuxNfP8Q97u1jq5+fQJvDyWc7tMo49krQrND2O CIQFuDSsfNb2GoqEbEMZ4XwXTl9OxfhvIt2Ouvkhjed4FF5xDkNEWwODQiboDUQKq2Ni nIhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=4n8YX6duwnriKZKmbthkv+dt+z7zVqKii17JMYGRe8k=; b=oufuCRqikXpAqlnCgP8GRmN36gm+GkmQbxvGpHktSuFKMpUXo3l1CBvaToBN9O1c3g qf3pV5W74jUp/xubkbwRvx2EqNMWUffu8FAYPB0mDyOJMLEuSvB/GitLFBQJveg/w3ba YbhT5mCQUBD6ovilUrDLBYF2A8v72hCXG8KvGmOwnZ3inGof1pvJpMquk9FVBoL4WGzG eJs5XUg8HNTbQLQkL4QyFYJaLlSv4cqMSHA127qh/HMddne7WnGtWNQvgHO0XhUkbhWL Je4CsZ0JpghtZu4oX3p/GkuZQTvs/KGFexHllfHTdgsvN4HAIOggpZrxvbLngXd29d0H 9xRA== X-Gm-Message-State: AOAM530N9NqTRWzKGLXfPq2IfarqE2sdTD8huVCqPjDtuFvHD+bjN4Qr NR3QYIaqK/EPYXGeoHLYlqAw+0gVmZk= X-Google-Smtp-Source: ABdhPJycDdD43eKdd2s8w/uk8YeNCdCdiT4xR2MPZaIn3V3VYdeHZlG9VyWigLiwgUfxLTJxjreLrw== X-Received: by 2002:adf:dc45:: with SMTP id m5mr10100131wrj.625.1642678567315; Thu, 20 Jan 2022 03:36:07 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 1/4] scripts/mtest2make: add support for SPEED=thorough Date: Thu, 20 Jan 2022 12:36:00 +0100 Message-Id: <20220120113603.276144-2-pbonzini@redhat.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220120113603.276144-1-pbonzini@redhat.com> References: <20220120113603.276144-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::329 (failed) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x329.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.248, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642691675703100001 Content-Type: text/plain; charset="utf-8" Signed-off-by: Paolo Bonzini --- meson.build | 5 +++-- scripts/mtest2make.py | 10 +++++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/meson.build b/meson.build index 97c6fd6f1b..dcd2a1d5e3 100644 --- a/meson.build +++ b/meson.build @@ -3,8 +3,9 @@ project('qemu', ['c'], meson_version: '>=3D0.58.2', 'b_staticpic=3Dfalse', 'stdsplit=3Dfalse'], version: files('VERSION')) =20 -add_test_setup('quick', exclude_suites: 'slow', is_default: true) -add_test_setup('slow', env: ['G_TEST_SLOW=3D1', 'SPEED=3Dslow']) +add_test_setup('quick', exclude_suites: ['slow', 'thorough'], is_default: = true) +add_test_setup('slow', exclude_suites: ['thorough'], env: ['G_TEST_SLOW=3D= 1', 'SPEED=3Dslow']) +add_test_setup('thorough', env: ['G_TEST_SLOW=3D1', 'SPEED=3Dthorough']) =20 not_found =3D dependency('', required: false) keyval =3D import('keyval') diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py index 7067bdadf5..4b9c561b30 100644 --- a/scripts/mtest2make.py +++ b/scripts/mtest2make.py @@ -23,8 +23,9 @@ def names(self, base): print(''' SPEED =3D quick =20 -.speed.quick =3D $(foreach s,$(sort $(filter-out %-slow, $1)), --suite $s) -.speed.slow =3D $(foreach s,$(sort $1), --suite $s) +.speed.quick =3D $(foreach s,$(sort $(filter-out %-slow %-thorough, $1)), = --suite $s) +.speed.slow =3D $(foreach s,$(sort $(filter-out %-thorough, $1)), --suite = $s) +.speed.thorough =3D $(foreach s,$(sort $1), --suite $s) =20 .mtestargs =3D --no-rebuild -t 0 ifneq ($(SPEED), quick) @@ -52,11 +53,14 @@ def process_tests(test, targets, suites): for s in test_suites: # The suite name in the introspection info is "PROJECT:SUITE" s =3D s.split(':')[1] - if s =3D=3D 'slow': + if s =3D=3D 'slow' or s =3D=3D 'thorough': continue if s.endswith('-slow'): s =3D s[:-5] suites[s].speeds.append('slow') + if s.endswith('-thorough'): + s =3D s[:-9] + suites[s].speeds.append('thorough') suites[s].deps.update(deps) =20 def emit_prolog(suites, prefix): --=20 2.33.1 From nobody Sun May 5 20:46:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642692133696540.9927424747963; Thu, 20 Jan 2022 07:22:13 -0800 (PST) Received: from localhost ([::1]:56504 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAZGO-0003da-L0 for importer@patchew.org; Thu, 20 Jan 2022 10:22:12 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55878) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAVjr-0007Mh-MD; Thu, 20 Jan 2022 06:36:24 -0500 Received: from [2a00:1450:4864:20::332] (port=40945 helo=mail-wm1-x332.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nAVjo-0001pj-5d; Thu, 20 Jan 2022 06:36:22 -0500 Received: by mail-wm1-x332.google.com with SMTP id r9-20020a1c4409000000b0034e043aaac7so79884wma.5; Thu, 20 Jan 2022 03:36:09 -0800 (PST) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id w8sm2653423wrq.3.2022.01.20.03.36.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jan 2022 03:36:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lu1Fde8IAsccMn3APv01Jm6gytpYFvEamfjGg8V6kpQ=; b=V9J/n2VIpk5GyqomMl2jego4ukrdFMdFSUMYy4o8AskDP5L30edvq9bBKObzzv6bik 1us/X2YILqQ0Cl17FPtXtpVx20+WeZ+vy7VZqXB71bHzJoXfpjT58hazkUqee9KcbmZc A08t+QrCtQD1f7fIFC5CE8e8LE8ssKjgYXagzQeqD9GLP4SFTb9iReeVDhxkYJmwsL93 Bbnrv50XnXJPxSC0mDnokZ1g+sj/NMJq2QWd7vOK/F1ilg3rVrV2NcSfUbqv/0VE6xUY 3iKK0Pr95QfHJ5OVD8u20gB7SbrUr065MieRaDXcWsB80JLIOSD8LHaEn++wfQ9T7JXy CyjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=Lu1Fde8IAsccMn3APv01Jm6gytpYFvEamfjGg8V6kpQ=; b=ecRz3CYfgUwVLjfB1KRvBDEJiQVaKzHXrl7SRwh7qF8CgZbqiV9NeKZgWNcl269bZV OPrWMvCgzGwMQwh+QUqoGT0gLGvqgpqTLrPL5CtMMGabS1L2n5e0BBhqmKkn6ekUWnRk CmSLxZJX4bZcMxOzS99zSV/bkog2a50rH0X7eN4XOsaBnSRh1NgZTfFul3J1qLshH4/i K2bLPasldp+dqtO+rIJojs4xZLGpqxPJeBTEBRjNCSq+C/Vpo4i5WdK7loujgUcAC0X+ LihHx9/cITb1aOa6PU2CERt9UpTeR2ZW9sQPwjREoGdDdpJJUCr1aDY/yiadGSUPsCJ9 bUEg== X-Gm-Message-State: AOAM531Zj7fYlhXx+hCldnzsz36AU6Ocx3K4fGsEmELEIkFw/ByqOp/s bdCA2MbZZLd5VgGHLAUMygMl/VCvLMs= X-Google-Smtp-Source: ABdhPJyD/7attTRDizWI4IgcTBHFJFsUikpB7F00sw57sJ5600nJaFzBArFKV4krFwJWW7kvcGRxyg== X-Received: by 2002:a05:6000:1a8a:: with SMTP id f10mr30902473wry.103.1642678568317; Thu, 20 Jan 2022 03:36:08 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 2/4] build: make check-block a meson test Date: Thu, 20 Jan 2022 12:36:01 +0100 Message-Id: <20220120113603.276144-3-pbonzini@redhat.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220120113603.276144-1-pbonzini@redhat.com> References: <20220120113603.276144-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::332 (failed) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x332.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.248, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642692134675100001 Content-Type: text/plain; charset="utf-8" "meson test" can be asked to run tests verbosely; this makes it usable also for qemu-iotests's own harness, and it lets "make check-block" reuse mtest2make.py's infrastructure to find and build test dependencies. Adjust check-block.sh to use the standard exit code that reports a test as skipped. Alternatively, in the future we could make it produce TAP output, which is consistent with all other "make check" tests. A patch has been posted to Meson to force verbose output for specific tests, which will allow to remove the special support in scripts/mtest2make.py and tests/Makefile.include; see https://github.com/mesonbuild/meson/pull/9834. Signed-off-by: Paolo Bonzini --- meson.build | 6 +++--- scripts/mtest2make.py | 10 +++++++++- tests/Makefile.include | 16 +++------------- tests/check-block.sh | 28 +++++++++++++--------------- tests/meson.build | 1 + tests/qemu-iotests/meson.build | 29 +++++++++++++++++++++++++++++ 6 files changed, 58 insertions(+), 32 deletions(-) create mode 100644 tests/qemu-iotests/meson.build diff --git a/meson.build b/meson.build index dcd2a1d5e3..df6930d074 100644 --- a/meson.build +++ b/meson.build @@ -3,9 +3,9 @@ project('qemu', ['c'], meson_version: '>=3D0.58.2', 'b_staticpic=3Dfalse', 'stdsplit=3Dfalse'], version: files('VERSION')) =20 -add_test_setup('quick', exclude_suites: ['slow', 'thorough'], is_default: = true) -add_test_setup('slow', exclude_suites: ['thorough'], env: ['G_TEST_SLOW=3D= 1', 'SPEED=3Dslow']) -add_test_setup('thorough', env: ['G_TEST_SLOW=3D1', 'SPEED=3Dthorough']) +add_test_setup('quick', exclude_suites: ['block', 'slow', 'thorough'], is_= default: true) +add_test_setup('slow', exclude_suites: ['block', 'thorough'], env: ['G_TES= T_SLOW=3D1', 'SPEED=3Dslow']) +add_test_setup('thorough', exclude_suites: ['block'], env: ['G_TEST_SLOW= =3D1', 'SPEED=3Dthorough']) =20 not_found =3D dependency('', required: false) keyval =3D import('keyval') diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py index 4b9c561b30..4d542e8aaa 100644 --- a/scripts/mtest2make.py +++ b/scripts/mtest2make.py @@ -79,7 +79,7 @@ def emit_prolog(suites, prefix): print(f'{prefix}-report.junit.xml $(all-{prefix}-xml): {prefix}-report= %.junit.xml: run-ninja') print(f'\t$(MAKE) {prefix}$* MTESTARGS=3D"$(MTESTARGS) --logbase {pref= ix}-report$*" && ln -f meson-logs/$@ .') =20 -def emit_suite(name, suite, prefix): +def emit_suite_deps(name, suite, prefix): deps =3D ' '.join(suite.deps) targets =3D f'{prefix}-{name} {prefix}-report-{name}.junit.xml {prefix= } {prefix}-report.junit.xml' print() @@ -87,6 +87,10 @@ def emit_suite(name, suite, prefix): print(f'ifneq ($(filter {prefix}-build {targets}, $(MAKECMDGOALS)),)') print(f'.{prefix}.build-suites +=3D {name}') print(f'endif') + +def emit_suite(name, suite, prefix): + emit_suite_deps(name, suite, prefix) + targets =3D f'{prefix}-{name} {prefix}-report-{name}.junit.xml {prefix= } {prefix}-report.junit.xml' print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)') print(f'.{prefix}.mtest-suites +=3D ' + ' '.join(suite.names(name))) print(f'endif') @@ -97,6 +101,10 @@ def emit_suite(name, suite, prefix): testsuites =3D defaultdict(Suite) for test in introspect['tests']: process_tests(test, targets, testsuites) +# HACK: check-block is a separate target so that it runs with --verbose; +# only write the dependencies +emit_suite_deps('block', testsuites['block'], 'check') +del testsuites['block'] emit_prolog(testsuites, 'check') for name, suite in testsuites.items(): emit_suite(name, suite, 'check') diff --git a/tests/Makefile.include b/tests/Makefile.include index 3aba622400..9157a57b1a 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -156,19 +156,9 @@ check: =20 ifeq ($(CONFIG_TOOLS)$(CONFIG_POSIX),yy) check: check-block -export PYTHON - -ifneq ($(filter check check-block check-build, $(MAKECMDGOALS)),) -ninja-cmd-goals +=3D \ - qemu-img$(EXESUF) \ - qemu-io$(EXESUF) \ - qemu-nbd$(EXESUF) \ - storage-daemon/qemu-storage-daemon$(EXESUF) \ - $(filter qemu-system-%, $(ninja-targets)) -endif - -check-block: $(SRC_PATH)/tests/check-block.sh run-ninja - @$< +check-block: run-ninja + $(if $(MAKE.n),,+)$(MESON) test $(MTESTARGS) $(.mtestargs) --verbose \ + --logbase iotestslog $(call .speed.$(SPEED), block block-slow block-thor= ough) endif =20 check-build: run-ninja diff --git a/tests/check-block.sh b/tests/check-block.sh index d98d49ad63..9afeea5275 100755 --- a/tests/check-block.sh +++ b/tests/check-block.sh @@ -1,6 +1,6 @@ #!/bin/sh =20 -# Honor the SPEED environment variable, just like we do it for the qtests. +# Honor the SPEED environment variable, just like we do it for "meson test" if [ "$SPEED" =3D "slow" ]; then format_list=3D"raw qcow2" group=3D @@ -16,9 +16,13 @@ if [ "$#" -ne 0 ]; then format_list=3D"$@" fi =20 +skip() { + echo "$*" + exit 77 +} + if grep -q "CONFIG_GPROF=3Dy" config-host.mak 2>/dev/null ; then - echo "GPROF is enabled =3D=3D> Not running the qemu-iotests." - exit 0 + skip "GPROF is enabled =3D=3D> Not running the qemu-iotests." fi =20 # Disable tests with any sanitizer except for specific ones @@ -36,36 +40,30 @@ for j in ${ALLOWED_SANITIZE_FLAGS}; do done if echo ${SANITIZE_FLAGS} | grep -q "\-fsanitize" 2>/dev/null; then # Have a sanitize flag that is not allowed, stop - echo "Sanitizers are enabled =3D=3D> Not running the qemu-iotests." - exit 0 + skip "Sanitizers are enabled =3D=3D> Not running the qemu-iotests." fi =20 if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then - echo "No qemu-system binary available =3D=3D> Not running the qemu-iot= ests." - exit 0 + skip "No qemu-system binary available =3D=3D> Not running the qemu-iot= ests." fi =20 if ! command -v bash >/dev/null 2>&1 ; then - echo "bash not available =3D=3D> Not running the qemu-iotests." - exit 0 + skip "bash not available =3D=3D> Not running the qemu-iotests." fi =20 if LANG=3DC bash --version | grep -q 'GNU bash, version [123]' ; then - echo "bash version too old =3D=3D> Not running the qemu-iotests." - exit 0 + skip "bash version too old =3D=3D> Not running the qemu-iotests." fi =20 if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then if ! command -v gsed >/dev/null 2>&1; then - echo "GNU sed not available =3D=3D> Not running the qemu-iotests." - exit 0 + skip "GNU sed not available =3D=3D> Not running the qemu-iotests." fi else # Double-check that we're not using BusyBox' sed which says # that "This is not GNU sed version 4.0" ... if sed --version | grep -q 'not GNU sed' ; then - echo "BusyBox sed not supported =3D=3D> Not running the qemu-iotes= ts." - exit 0 + skip "BusyBox sed not supported =3D=3D> Not running the qemu-iotes= ts." fi fi =20 diff --git a/tests/meson.build b/tests/meson.build index 3f3882748a..d5e168d714 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -1,6 +1,7 @@ py3 =3D import('python').find_installation() =20 subdir('bench') +subdir('qemu-iotests') =20 test_qapi_outputs =3D [ 'qapi-builtin-types.c', diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build new file mode 100644 index 0000000000..3a9425d15c --- /dev/null +++ b/tests/qemu-iotests/meson.build @@ -0,0 +1,29 @@ +if have_tools and targetos !=3D 'windows' + qemu_iotests_binaries =3D [qemu_img, qemu_io, qemu_nbd, qsd] + qemu_iotests_env =3D {'PYTHON': python.full_path()} + qemu_iotests_formats =3D { + 'qcow2': 'quick', + 'raw': 'slow', + 'qed': 'thorough', + 'vmdk': 'thorough', + 'vpc': 'thorough' + } + + foreach k, v : emulators + if k.startswith('qemu-system-') + qemu_iotests_binaries +=3D v + endif + endforeach + foreach format, speed: qemu_iotests_formats + if speed =3D=3D 'quick' + suites =3D 'block' + else + suites =3D ['block-' + speed, speed] + endif + test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), = format], + depends: qemu_iotests_binaries, env: qemu_iotests_env, + suite: suites, + timeout: 0, + is_parallel: false) + endforeach +endif --=20 2.33.1 From nobody Sun May 5 20:46:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642692536573828.2210407337086; Thu, 20 Jan 2022 07:28:56 -0800 (PST) Received: from localhost ([::1]:36982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAZMr-00019O-Ch for importer@patchew.org; Thu, 20 Jan 2022 10:28:53 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55862) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAVjq-0007MN-Sy; Thu, 20 Jan 2022 06:36:24 -0500 Received: from [2a00:1450:4864:20::32a] (port=38728 helo=mail-wm1-x32a.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nAVjo-0001qF-6E; Thu, 20 Jan 2022 06:36:22 -0500 Received: by mail-wm1-x32a.google.com with SMTP id ay14-20020a05600c1e0e00b0034d7bef1b5dso14515380wmb.3; Thu, 20 Jan 2022 03:36:10 -0800 (PST) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id w8sm2653423wrq.3.2022.01.20.03.36.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jan 2022 03:36:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NPHfXnowRQD2+TFf0g2fYmXJjkrHfOL9fmenURj2Zdw=; b=oKpz4hMCMyzM0Zqxgb5Z9EpFGQi76RwoBkdwnewJvqsXAiPPl5G5jDs+a755adAbey L9f8p7vA7Qgve2BUaP0QNbplqjApTITwgAn+1Pi0GfXOxGhmLxC3TnWRk6mFojtLhsq/ sEQ4zgD+2afNIMWj2d5ahwidhPGpsDKZm3W88umrODNxnDePDCLIUpAD20DHAeOdT+Dw L79V5M/0ARZr/aMTcLHsGl2xZgxN/Z2HnavU5e0ltU+zxdXEUN4ZV9qXBl4G0BbDyxeP Q36rHdg+BjL0+h34aX/+KOEY2//WJZp7isHTMbbY/P1vcVKzAKrReb2AE36uVOPTnMl4 iRIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=NPHfXnowRQD2+TFf0g2fYmXJjkrHfOL9fmenURj2Zdw=; b=TerjCzE9AV2AikgFIogP6tvJlR9S1bxR4p8bcy2MFtHHaSsP9oZ9CEKNgGVv79P5GZ d0uvXNW6LY7dK0eKumFbYXeRO1BXIUTp12LwAFuSUrPg6jK2wcVzim2vrEdojkssNOsH BXDlLWU1zWJc8GMSw/EubXlB+4eovHDYQOBGZMvuAQPT0LNqdNNExZrfiXsQi2ZcJGjG do/Y5SQcwCVO21M5/9S1ojgLXUfHjIFCYOJvJOwE/PbAANYXQY7Dih9ua51UVyAXAVuj wCnms5Ugb4f+lZsU27a7why2GWWpPH21FhQd2FiaFKZsL2EV1/nmloMMggCcr576DXQP RtkA== X-Gm-Message-State: AOAM530snwhGha5SkpIM019VaCOD5fSSiDoOvPJCLCW3hgudvcZKDI6e x0fojrBq+pVMUPs9MoIDyCx0F+pPRgo= X-Google-Smtp-Source: ABdhPJyi9lEgQiSH0LSUWS5+jaylxcnxDdPhJW/LOteiO2WwWdX5y3zOwxi2Mz4vRMqI1flE2Aw3Ig== X-Received: by 2002:adf:cd92:: with SMTP id q18mr33183136wrj.705.1642678569386; Thu, 20 Jan 2022 03:36:09 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 3/4] qemu-iotests: require at least an argument to check-block.sh Date: Thu, 20 Jan 2022 12:36:02 +0100 Message-Id: <20220120113603.276144-4-pbonzini@redhat.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220120113603.276144-1-pbonzini@redhat.com> References: <20220120113603.276144-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::32a (failed) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x32a.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.248, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642692537550100001 Content-Type: text/plain; charset="utf-8" This is anyway how check-block.sh is used in practice, and by removing the list of formats in the script we avoid duplication between meson.build and check-block.sh. Signed-off-by: Paolo Bonzini --- tests/check-block.sh | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/tests/check-block.sh b/tests/check-block.sh index 9afeea5275..88e02453d2 100755 --- a/tests/check-block.sh +++ b/tests/check-block.sh @@ -1,19 +1,16 @@ #!/bin/sh =20 -# Honor the SPEED environment variable, just like we do it for "meson test" -if [ "$SPEED" =3D "slow" ]; then - format_list=3D"raw qcow2" - group=3D -elif [ "$SPEED" =3D "thorough" ]; then - format_list=3D"raw qcow2 qed vmdk vpc" - group=3D -else - format_list=3Dqcow2 - group=3D"-g auto" +if [ "$#" -eq 0 ]; then + echo "Usage: $0 fmt..." >&2 + exit 99 fi =20 -if [ "$#" -ne 0 ]; then - format_list=3D"$@" +# Honor the SPEED environment variable, just like we do it for "meson test" +format_list=3D"$@" +if [ "$SPEED" =3D "slow" ] || [ "$SPEED" =3D "thorough" ]; then + group=3D +else + group=3D"-g auto" fi =20 skip() { --=20 2.33.1 From nobody Sun May 5 20:46:38 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1642689491493207.72904339335116; Thu, 20 Jan 2022 06:38:11 -0800 (PST) Received: from localhost ([::1]:43498 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAYZl-00070D-IN for importer@patchew.org; Thu, 20 Jan 2022 09:38:09 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55858) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAVjq-0007ME-Rs; Thu, 20 Jan 2022 06:36:24 -0500 Received: from [2a00:1450:4864:20::32c] (port=53978 helo=mail-wm1-x32c.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nAVjo-0001rJ-2i; Thu, 20 Jan 2022 06:36:21 -0500 Received: by mail-wm1-x32c.google.com with SMTP id n8so11419381wmk.3; Thu, 20 Jan 2022 03:36:11 -0800 (PST) Received: from avogadro.lan ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id w8sm2653423wrq.3.2022.01.20.03.36.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jan 2022 03:36:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gE7kVOUWYMuX83tWlO/qFxRTGfJ+gjxaeF9Ye1Re6So=; b=VtFL2+7gedaLSXNStLSn6MJVbkfuSRF1G75KbVpeb6ddk/i16iNyUDBNKbCWrgXZMv KiwS5bDKvByhGd87jl/buADhAtOU9obBLrnx9UcASFfw0h8UuFgYnDU6dNiZdhS//uJh CTVdshMg8qcTRmQ8g1SslbOeS+xNxitUgGU8dywQLK/l48XjQvjVGlcdGmcuTVwwXPfi maMgMZQ8nh7f/B9YJ1Ogjl7sU+cHTPbLssDpVQ7IJUzN21UvqlZtGxPUmJ8R2HW2X6GD Ci777vc9fT/MzMMk0EKorUG5pzeO4FZZjZjgHGNRyNWfIG8Em2/wuxEAcwD6dVQPqzhH AbZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=gE7kVOUWYMuX83tWlO/qFxRTGfJ+gjxaeF9Ye1Re6So=; b=f43dhDR/nk0GZO+WIRQrEq7ptdLHUXVNCs590iQZ5MnaT0Ojcx3FW+ZcM/H7LRuqhE kvaZA5l42yHugIKHc/Grh49kXKWC3Mzh4lJuy876DXOduMDgSOn+irsXpCjtVsQI0l7y 4LRn6mUvgQrogXtcxBxpHYRrRFnkidizlMTkyfl9ipz/F1i2NmmsEmUMDK1q1lzLW4Pw d7lRNtxKLJbPKeVHc7wn+6+mTuykN3HM1Mm3i4ozCGVJwzGKVdndGejdrUbs69fDQX/q v4YiI8AuJzv/hYwDsFGkwTvxBg1AGcj7TO1nBptN8bmJZwh9BWmfCyifq1p0+NuItwf3 KV+w== X-Gm-Message-State: AOAM533mqfhj2rT9zylpQeh6xIT7kv5+MCJr89fXt8pRv0CY4ZJGXLSK skgGbE5L5jT30J/ee0k1/qSIUP+JC2g= X-Google-Smtp-Source: ABdhPJx7Cg23fAC3eywv98yXin9Y8nyX8bJCcF7Vr6PL28c4RvcDB2gW1o07IkbQeBhYCuhCz72KBg== X-Received: by 2002:a05:600c:2690:: with SMTP id 16mr8353057wmt.146.1642678570125; Thu, 20 Jan 2022 03:36:10 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH v2 4/4] check-block: replace -makecheck with TAP output Date: Thu, 20 Jan 2022 12:36:03 +0100 Message-Id: <20220120113603.276144-5-pbonzini@redhat.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220120113603.276144-1-pbonzini@redhat.com> References: <20220120113603.276144-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::32c (failed) Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x32c.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.248, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: thuth@redhat.com, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1642689493364100001 Content-Type: text/plain; charset="utf-8" Let "meson test" take care of showing the results of the individual tests, consistently with other output from "make check V=3D1". Signed-off-by: Paolo Bonzini --- tests/check-block.sh | 6 ++-- tests/qemu-iotests/check | 6 ++-- tests/qemu-iotests/meson.build | 1 + tests/qemu-iotests/testenv.py | 30 +++++++++---------- tests/qemu-iotests/testrunner.py | 49 +++++++++++++++++--------------- 5 files changed, 48 insertions(+), 44 deletions(-) diff --git a/tests/check-block.sh b/tests/check-block.sh index 88e02453d2..720a46bc36 100755 --- a/tests/check-block.sh +++ b/tests/check-block.sh @@ -14,8 +14,8 @@ else fi =20 skip() { - echo "$*" - exit 77 + echo "1..0 #SKIP $*" + exit 0 } =20 if grep -q "CONFIG_GPROF=3Dy" config-host.mak 2>/dev/null ; then @@ -79,7 +79,7 @@ JOBS=3D$(echo "$MAKEFLAGS" | sed -n 's/\(^\|.* \)-j\([0-9= ]\+\)\( .*\|$\)/-j \2/p') =20 ret=3D0 for fmt in $format_list ; do - ${PYTHON} ./check $JOBS -makecheck -$fmt $group || ret=3D1 + ${PYTHON} ./check $JOBS -tap -$fmt $group || ret=3D1 done =20 exit $ret diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 0c27721a41..75de1b4691 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -32,8 +32,6 @@ def make_argparser() -> argparse.ArgumentParser: =20 p.add_argument('-n', '--dry-run', action=3D'store_true', help=3D'show me, do not run tests') - p.add_argument('-makecheck', action=3D'store_true', - help=3D'pretty print output for make check') p.add_argument('-j', dest=3D'jobs', type=3Dint, default=3D1, help=3D'run tests in multiple parallel jobs') =20 @@ -53,6 +51,8 @@ def make_argparser() -> argparse.ArgumentParser: p.add_argument('--color', choices=3D['on', 'off', 'auto'], default=3D'auto', help=3D"use terminal colors. The defa= ult " "'auto' value means use colors if terminal stdout detec= ted") + p.add_argument('-tap', action=3D'store_true', + help=3D'produce TAP output') =20 g_env =3D p.add_argument_group('test environment options') mg =3D g_env.add_mutually_exclusive_group() @@ -164,7 +164,7 @@ if __name__ =3D=3D '__main__': if args.dry_run: print('\n'.join(tests)) else: - with TestRunner(env, makecheck=3Dargs.makecheck, + with TestRunner(env, tap=3Dargs.tap, color=3Dargs.color) as tr: paths =3D [os.path.join(env.source_iotests, t) for t in tests] ok =3D tr.run_tests(paths, args.jobs) diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build index 3a9425d15c..5be3c74127 100644 --- a/tests/qemu-iotests/meson.build +++ b/tests/qemu-iotests/meson.build @@ -22,6 +22,7 @@ if have_tools and targetos !=3D 'windows' endif test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), = format], depends: qemu_iotests_binaries, env: qemu_iotests_env, + protocol: 'tap', suite: suites, timeout: 0, is_parallel: false) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index c33454fa68..0f32897fe8 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -287,21 +287,21 @@ def __enter__(self) -> 'TestEnv': def __exit__(self, exc_type: Any, exc_value: Any, traceback: Any) -> N= one: self.close() =20 - def print_env(self) -> None: + def print_env(self, prefix: str =3D '') -> None: template =3D """\ -QEMU -- "{QEMU_PROG}" {QEMU_OPTIONS} -QEMU_IMG -- "{QEMU_IMG_PROG}" {QEMU_IMG_OPTIONS} -QEMU_IO -- "{QEMU_IO_PROG}" {QEMU_IO_OPTIONS} -QEMU_NBD -- "{QEMU_NBD_PROG}" {QEMU_NBD_OPTIONS} -IMGFMT -- {IMGFMT}{imgopts} -IMGPROTO -- {IMGPROTO} -PLATFORM -- {platform} -TEST_DIR -- {TEST_DIR} -SOCK_DIR -- {SOCK_DIR} -GDB_OPTIONS -- {GDB_OPTIONS} -VALGRIND_QEMU -- {VALGRIND_QEMU} -PRINT_QEMU_OUTPUT -- {PRINT_QEMU} -""" +{prefix}QEMU -- "{QEMU_PROG}" {QEMU_OPTIONS} +{prefix}QEMU_IMG -- "{QEMU_IMG_PROG}" {QEMU_IMG_OPTIONS} +{prefix}QEMU_IO -- "{QEMU_IO_PROG}" {QEMU_IO_OPTIONS} +{prefix}QEMU_NBD -- "{QEMU_NBD_PROG}" {QEMU_NBD_OPTIONS} +{prefix}IMGFMT -- {IMGFMT}{imgopts} +{prefix}IMGPROTO -- {IMGPROTO} +{prefix}PLATFORM -- {platform} +{prefix}TEST_DIR -- {TEST_DIR} +{prefix}SOCK_DIR -- {SOCK_DIR} +{prefix}GDB_OPTIONS -- {GDB_OPTIONS} +{prefix}VALGRIND_QEMU -- {VALGRIND_QEMU} +{prefix}PRINT_QEMU_OUTPUT -- {PRINT_QEMU} +{prefix}""" =20 args =3D collections.defaultdict(str, self.get_env()) =20 @@ -310,5 +310,5 @@ def print_env(self) -> None: =20 u =3D os.uname() args['platform'] =3D f'{u.sysname}/{u.machine} {u.nodename} {u.rel= ease}' - + args['prefix'] =3D prefix print(template.format_map(args)) diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunn= er.py index 15788f919e..0eace147b8 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -152,10 +152,10 @@ def run_tests_pool(self, tests: List[str], =20 return results =20 - def __init__(self, env: TestEnv, makecheck: bool =3D False, + def __init__(self, env: TestEnv, tap: bool =3D False, color: str =3D 'auto') -> None: self.env =3D env - self.makecheck =3D makecheck + self.tap =3D tap self.last_elapsed =3D LastElapsedTime('.last-elapsed-cache', env) =20 assert color in ('auto', 'on', 'off') @@ -185,13 +185,16 @@ def test_print_one_line(self, test: str, """ Print short test info before/after test run """ test =3D os.path.basename(test) =20 - if self.makecheck and status !=3D '...': - if status and status !=3D 'pass': - status =3D f' [{status}]' - else: - status =3D '' + if test_field_width is None: + test_field_width =3D 8 =20 - print(f' TEST iotest-{self.env.imgfmt}: {test}{status}') + if self.tap: + if status =3D=3D 'pass': + print(f'ok {self.env.imgfmt} {test}') + elif status =3D=3D 'fail': + print(f'not ok {self.env.imgfmt} {test}') + elif status =3D=3D 'not run': + print(f'ok {self.env.imgfmt} {test} # SKIP') return =20 if lasttime: @@ -343,7 +346,7 @@ def run_test(self, test: str, last_el =3D self.last_elapsed.get(test) start =3D datetime.datetime.now().strftime('%H:%M:%S') =20 - if not self.makecheck: + if not self.tap: self.test_print_one_line(test=3Dtest, test_field_width=3Dtest_field_width, status =3D 'started' if mp else '...', @@ -372,7 +375,9 @@ def run_tests(self, tests: List[str], jobs: int =3D 1) = -> bool: notrun =3D [] casenotrun =3D [] =20 - if not self.makecheck: + if self.tap: + self.env.print_env('# ') + else: self.env.print_env() =20 test_field_width =3D max(len(os.path.basename(t)) for t in tests) = + 2 @@ -398,8 +403,6 @@ def run_tests(self, tests: List[str], jobs: int =3D 1) = -> bool: =20 if res.status =3D=3D 'fail': failed.append(name) - if self.makecheck: - self.env.print_env() if res.diff: print('\n'.join(res.diff)) elif res.status =3D=3D 'not run': @@ -412,16 +415,16 @@ def run_tests(self, tests: List[str], jobs: int =3D 1= ) -> bool: if res.interrupted: break =20 - if notrun: - print('Not run:', ' '.join(notrun)) + if not self.tap: + if notrun: + print('Not run:', ' '.join(notrun)) =20 - if casenotrun: - print('Some cases not run in:', ' '.join(casenotrun)) + if casenotrun: + print('Some cases not run in:', ' '.join(casenotrun)) =20 - if failed: - print('Failures:', ' '.join(failed)) - print(f'Failed {len(failed)} of {n_run} iotests') - return False - else: - print(f'Passed all {n_run} iotests') - return True + if failed: + print('Failures:', ' '.join(failed)) + print(f'Failed {len(failed)} of {n_run} iotests') + else: + print(f'Passed all {n_run} iotests') + return not failed --=20 2.33.1