From nobody Mon Sep 22 03:33:32 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1757518769; cv=none; d=zohomail.com; s=zohoarc; b=cyqt5P6PSBL4XapQsoD4Osky4tH/MucaBfc5/0TLun+x6yjTtzDiPbJQOnEICSRJSzBUrPjOfaFP18yHfVRqCYB0U/kz00xK+EzN04EqSF10yhMM5IXI9Kc8fU6MlgPgv+EnSuPG2y72zl3GlyWss+w1cbwVyz/fi8Wmicbeyo0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1757518769; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jyAcvr1iUk8f5T3ySMmHEeZHnIB4NK1NnTCdMCnMGrI=; b=iIniVxDVci1N6x7iR9xbmcqn5xCcwVDBsJdDQnUUCQGN08O31mgTXRoLwvWZ0kOQnpN1VyEN4xWQzmHr9h8J0nQXZJ6z2WMGeUbU2N1LaaJPt9EjQMSCjjUsdvjGHAAx4u5DBnr+rtKohbH5EpgtnzSpXmtg6hJ7m//eYYgyDjg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1757518769206857.5478728921831; Wed, 10 Sep 2025 08:39:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uwMtU-0001W2-Bz; Wed, 10 Sep 2025 11:38:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uwMtS-0001V4-Am for qemu-devel@nongnu.org; Wed, 10 Sep 2025 11:37:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uwMtN-0000rT-C4 for qemu-devel@nongnu.org; Wed, 10 Sep 2025 11:37:58 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-75-igLjQuJ1ONW2PG4-5XYE_A-1; Wed, 10 Sep 2025 11:37:50 -0400 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B9A0B19560B2; Wed, 10 Sep 2025 15:37:49 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.44.33.173]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 37DC719560B8; Wed, 10 Sep 2025 15:37:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1757518672; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jyAcvr1iUk8f5T3ySMmHEeZHnIB4NK1NnTCdMCnMGrI=; b=D/rbeH4kYbgTfycw2WGrjnmJD8xLBBOURCh1Nj7OWCMwDcMvWh1D9Oc+3sHCQBFQBNjMWQ MpRI7mnZRyBOTHY1oclOwKF7swff/JqGj1VTx15eERcdwiNl3R7yXzH0Emb92pbUkAC1iX 06cpKM79IjzxwmOZtutTXH8nFF/4lRE= X-MC-Unique: igLjQuJ1ONW2PG4-5XYE_A-1 X-Mimecast-MFC-AGG-ID: igLjQuJ1ONW2PG4-5XYE_A_1757518669 From: Thomas Huth To: qemu-block@nongnu.org, Kevin Wolf Cc: Hanna Czenczek , qemu-devel@nongnu.org, Thomas Huth Subject: [PATCH 2/3] tests/qemu-iotests: Improve the dry run list to speed up thorough testing Date: Wed, 10 Sep 2025 17:37:25 +0200 Message-ID: <20250910153727.226217-3-thuth@redhat.com> In-Reply-To: <20250910153727.226217-1-thuth@redhat.com> References: <20250910153727.226217-1-thuth@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1757518771523116600 Content-Type: text/plain; charset="utf-8" From: Thomas Huth When running the tests in thorough mode, e.g. with: make -j$(nproc) check SPEED=3Dthorough we currently always get a huge amount of total tests that the test runner tries to execute (2457 in my case), but a big bunch of them are only skipped (1099 in my case, meaning that only 1358 got executed). This happens because we try to run the whole set of iotests for multiple image formats while a lot of the tests can only run with one certain format only and thus are marked as SKIP during execution. This is quite a waste of time during each test run, and also unnecessarily blows up the displayed list of executed tests in the console output. Thus let's try to be a little bit smarter: If the "check" script is run with "-n" and an image format switch (like "-qed") at the same time (which is what we do already for discovering the tests for the meson test runner), only report the tests that likely support the given format instead of providing the whole list of all tests. We can determine whether a test supports a format or not by looking at the lines in the file that contain a "supported_fmt" or "unsupported_fmt" statement. This is only heuristics, of course, but it is good enough for running the iotests via "make check-block" - I double-checked that the list of executed tests does not get changed by this patch, it's only the tests that are skipped anyway that are now not run anymore. This way the amount of total tests drops from 2457 to 1432 for me, and the amount of skipped tests drops from 1099 to just 74 (meaning that we still properly run 1432 - 74 =3D 1358 tests as we did before). Signed-off-by: Thomas Huth --- tests/qemu-iotests/check | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 545f9ec7bdd..fb3d8f95b94 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -81,7 +81,7 @@ def make_argparser() -> argparse.ArgumentParser: g_env.add_argument('-i', dest=3D'aiomode', default=3D'threads', help=3D'sets AIOMODE environment variable') =20 - p.set_defaults(imgfmt=3D'raw', imgproto=3D'file') + p.set_defaults(imgproto=3D'file') =20 format_list =3D ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2', 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg', 'vv= fat'] @@ -136,12 +136,41 @@ def make_argparser() -> argparse.ArgumentParser: return p =20 =20 +def dry_run_list(test_dir, imgfmt, testlist): + for t in testlist: + if not imgfmt: + print('\n'.join([os.path.basename(t)])) + continue + # If a format has been given, we look for the "supported_fmt" + # and the "unsupported_fmt" lines in the test and try to find out + # whether the format is supported or not. This is only heuristics, + # but it should be good enough for "make check-block" + with open(os.path.join(test_dir, t), 'r', encoding=3D'utf-8') as f= h: + supported =3D True + check_next_line =3D False + for line in fh: + if 'unsupported_fmt' in line: + if imgfmt in line: + supported =3D False + break + elif 'supported_fmt' in line or check_next_line: + supported =3D imgfmt in line or 'generic' in line + check_next_line =3D not ']' in line and \ + ('supported_fmts=3D[' in line or check_next_li= ne) + if supported or not check_next_line: + break + if supported: + print('\n'.join([os.path.basename(t)])) + + if __name__ =3D=3D '__main__': args =3D make_argparser().parse_args() =20 + image_format =3D args.imgfmt or 'raw' + env =3D TestEnv(source_dir=3Dargs.source_dir, build_dir=3Dargs.build_dir, - imgfmt=3Dargs.imgfmt, imgproto=3Dargs.imgproto, + imgfmt=3Dimage_format, imgproto=3Dargs.imgproto, aiomode=3Dargs.aiomode, cachemode=3Dargs.cachemode, imgopts=3Dargs.imgopts, misalign=3Dargs.misalign, debug=3Dargs.debug, valgrind=3Dargs.valgrind, @@ -185,7 +214,7 @@ if __name__ =3D=3D '__main__': =20 if args.dry_run: with env: - print('\n'.join([os.path.basename(t) for t in tests])) + dry_run_list(env.source_iotests, args.imgfmt, tests) else: with TestRunner(env, tap=3Dargs.tap, color=3Dargs.color) as tr: --=20 2.51.0