From nobody Sat Nov 15 09:31:56 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1753436549; cv=none; d=zohomail.com; s=zohoarc; b=kkkUUO0F+e9HRloNaDQagWcVaHFrhj7Lirpbhtw4u5+0mbHYlaCR36m67RIhvJrIYkj0FL+u3nio26NaC7EaGshIPsyQpQehOsLLqHc2a3e5IlT5vhPdmmQSWzz3eTm3n3nb03yxDdb4NTjK2ff9TnTP8AxTdpOY6VTs/rFR7/4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753436549; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=R12CwYsYaoDPuMu9oLrbneP+jbQddhEatX0ucdtMTlI=; b=iY31+Y61GqinOCXIavwHN6qj6KWEfn9VdfFIXwtDK3v9yhNv4Cmlb3BWUkU3qnXVEzpMNI8cj1DQyyQXTm0zCXz3Ne0utQnJIlCLzq/iAoIRMHr28ouffRN8Mxda+OQRP3ADPaEIX/b53WZk51AnnfOvGuShjyIu730vzINBW6Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753436549686108.08221925308897; Fri, 25 Jul 2025 02:42:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ufEwB-0008JN-S9; Fri, 25 Jul 2025 05:41:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ufEvu-0007vz-RF for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:46 -0400 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ufEvs-0006NG-LF for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:42 -0400 Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-ae3be3eabd8so426152466b.1 for ; Fri, 25 Jul 2025 02:41:40 -0700 (PDT) Received: from [127.0.1.1] (ppp-2-86-210-241.home.otenet.gr. [2.86.210.241]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af47ff42fc5sm243280266b.143.2025.07.25.02.41.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 02:41:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1753436498; x=1754041298; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=R12CwYsYaoDPuMu9oLrbneP+jbQddhEatX0ucdtMTlI=; b=Hm9JClivvimN+oQxEV5q8lQZwWAtraNFuoBvdFFxPdNw14/80kK0s53hFNaIrooTBV 2GIccWvW4wmvPGlviyTmyAoK3Wxgr3FkDxZ/WoS3PzPuShoHIRA/N7rfb3+2uvMos6yI wVjN9KIlV1vPLnBLvBYkEyeKx6fiIBXH6G8Q5Vw3bzR9mMu1ledUODUswGqit69NSkQo jOIXUL0viAdz7Yi61ViEyjNpY2fRAbJg6dd7J26vbHIV0IO6rJY5NtUHx+0g0kUADCdf wrHXcSb4GXGGBsznwjr4Ah2cbkgnc+t/YURmvMR8fJYENDBAg6Qm/1oxVOAEIuE9vpvO RBJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753436499; x=1754041299; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=R12CwYsYaoDPuMu9oLrbneP+jbQddhEatX0ucdtMTlI=; b=jXBWrsjId+O5ZwT07qPq1VWsVcCgxxrX6/OAfcAzo6leIIhg1/WdqHIAXEXognON2k WUy9h7mZ8ZLZwyvDNZ7Syn/F69ejrUMwHXhlWbHhJ6S9xruVqMNZCrRxhUF8pd1+0zNn LwkmxHAQ1zDBATfWlY0bVDYieI7dQYER9vbLCUZ0yHmEGna/iuY13/i1jFhoOBRcN8w1 0eIU/bz+Lr7xqo4zx3G28qi2Y7Th+nmnaE8rRhfM2pY2Br07SaenyrLfqjjHtwZn78my kyRrY+NQ880dDcG0iSDo8UgSB4UAtchbrNq6/WHitZj7KtoI+i0ihYzq6pGE9G3aJDSr 9TIw== X-Gm-Message-State: AOJu0Yx0oEURw3UJMvwvSg2ly3ZeVOcRdyqtWMcz3n5YSbzI+TuTQ5Hk +4Nqpbx3Nr5mhLgX52Fpr/hyn8ovZBl08WPMLyUnDki/y31TwluExugcBlpOCoPan5nSu2kTlCJ LL6GN X-Gm-Gg: ASbGncul51JPgxzTUTI3AKIT4VE4tJGrw18Cfh6/8ssFQVlX5/fIWEn4hT0MTBiAZAP FntmH30wn1DDcOupfLL2LpYF8KJZLtNy8NnUTlJAxG1sVcM/Lc6FSq1yWO67ShEohaYpWVHobre zkS6Ybfo5jDtmle7nmCjywCZzKb0UjU8Xog0uBarMTkPBnrXNhwQWQNNLbaGL2kHqWVS+ZAPASl RHOzLlrwTqKUQMdtj0Ib8SOvGGxzgEgIYRlBSYLvpBz0NcK/XDuuKaa/x1QBlMIlSkVJHRx8bZu k1v52s7nPoN+kbHaCGqNk719i9hsgm0qLaDQtmVfuoXCfMQ8tKf2XMFaXZYYRjZG8SR97KpDB+q 8iAVFPkDjL+3oQ7q80s3s5ysU8ZouKxHjDd63TJh07DfVcSrmdNnlQFY1kVOZNXSB3hk= X-Google-Smtp-Source: AGHT+IE6Ifwlhv4VwQxV7JJf5/osdrZRDRfQIWyFI8zpUrGoJXlMSUR4/qoOaSDtBEzmZhclQvDm+Q== X-Received: by 2002:a17:907:1c17:b0:ae3:d0fe:a35e with SMTP id a640c23a62f3a-af61c8ae363mr163237366b.11.1753436498435; Fri, 25 Jul 2025 02:41:38 -0700 (PDT) From: Manos Pitsidianakis Date: Fri, 25 Jul 2025 12:41:22 +0300 Subject: [PATCH v3 1/4] tests/functional: add --debug CLI arg MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250725-functional_tests_debug_arg-v3-1-b89921baf810@linaro.org> References: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> In-Reply-To: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> To: qemu-devel@nongnu.org Cc: Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Gustavo Romero , Pierrick Bouvier , =?utf-8?q?Alex_Benn=C3=A9e?= , Paolo Bonzini , Manos Pitsidianakis X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6552; i=manos.pitsidianakis@linaro.org; h=from:subject:message-id; bh=wzGRiYiFLh7FI47HaxpmpAOmaNwmxjp+OV9Rwk9bGuA=; b=LS0tLS1CRUdJTiBQR1AgTUVTU0FHRS0tLS0tCgpvd0VCYlFLUy9aQU5Bd0FLQVhjcHgzQi9mZ 25RQWNzbVlnQm9nMUZQL3FDUUxCTXRIZnlPZWlaUitCbllwQnVOCk9XVTlicHpLS01DMnNWbndW citKQWpNRUFBRUtBQjBXSVFUTVhCdE9SS0JXODRkd0hSQjNLY2R3ZjM0SjBBVUMKYUlOUlR3QUt DUkIzS2Nkd2YzNEowTDJZRUFDYVdZMXEyc1FKSlh4bzFvdTAva2d3VHRWUVhhMnAxTnQxbEJ2bw pndnFnRWM4UnpoSG01UDBrUHJKRG1wdk40eHhHS2pwOFIyamVEbVg1MWhmNHRlWjRYb2FWb3Rpa nlOMkRQaUoyClpkamwvSjN6R0dZQStxVTVBbzRlek1meVFVbDZGa2V6S2JuUE5JNm5zMmVsV2ZB d0VnenBDY09DTStnSEJOZXkKanQzbzVMcnl6UkZJWG14VWl0TXZMNmljQ1JxMVJ5elFtL08wLzl VZXlVS2c1NVVOOXlMOUVXT0hUVVNRQ09OKwpRaEVxbEV5L2xmQWhpbXRhVmI3SUdzWkcyRXZhVF dhcmZkcm5YbVpUdXJlcFJ5Zy85QjJvcDVkT2VGdENXalAyCjRseHBrUTdLRVZjNHZkOEhEb1hre Gw3M05KQTd2VWgrMzZyU3pnQ09tQ0cwck5Wdk9MdVhON0lJdWRKS0QrWGYKNHZ2bkd2MXF0dVlT dXBYUy9KdVI5QUhueW9jVGYvSWdzSCtjV2FhZjVyNTNwS1NoaWgxcGdpNVRqKzI1Ty9pcgptWDZ pV3hwVGM1U3RNMnlDamQzdnpBWEJRRHk0dDJYOVpQcVVxUnlpSnppZ0JyYlJDWXlVdjVlYUJsVG dqSnBLCjVITkpIMUNDUEYycWMxTkFCZVFyRnpDTmVmQytvTFpEUjZ4bXQ5bFhnaEhEcWY5ek5BM HF0c2ZzOUNLZU1INHAKVlRpODBzM2RMbmo1dkR4UW5PWFVBTzhqemMyeEUxZmx1Z0dsQVV1VU5O ZlZyR0hsaWdDbFJaRDBRQ3d6MnZhZQovemgveEVMU2ttRURqTDFGOHljQ2NNeklPM1BOMjY2Nzl FcTZBMDE1ZGh0K3g2QzdHb0RLRFd0ODdwRFN0eU9uCmNHQ0ZEUT09Cj1QejE0Ci0tLS0tRU5EIF BHUCBNRVNTQUdFLS0tLS0K X-Developer-Key: i=manos.pitsidianakis@linaro.org; a=openpgp; fpr=7C721DF9DB3CC7182311C0BF68BC211D47B421E1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::62e; envelope-from=manos.pitsidianakis@linaro.org; helo=mail-ej1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1753436550660116600 Add argument parsing to functional tests to improve developer experience when running individual tests. All logs are printed to stdout interspersed with TAP output. Example usage, assuming current build directory with qemu source code in the parent directory (see docs/devel/testing/functional.rst for details): $ export PYTHONPATH=3D../python:../tests/functional $ export QEMU_TEST_QEMU_BINARY=3D"$(pwd)/qemu-system-aarch64" $ ./pyvenv/bin/python3 ../tests/functional/test_aarch64_virt.py --help usage: test_aarch64_virt [-h] [-d] QEMU Functional test options: -h, --help show this help message and exit -d, --debug Also print test and console logs on stdout. This will make the TAP output invalid and is meant for debugging only. Signed-off-by: Manos Pitsidianakis Tested-by: Alex Benn=C3=A9e --- docs/devel/testing/functional.rst | 2 ++ tests/functional/qemu_test/testcase.py | 54 ++++++++++++++++++++++++++++++= +--- 2 files changed, 52 insertions(+), 4 deletions(-) diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functio= nal.rst index 3728bab6c0c4b0cbacd00744bdb5c1462c71d7a8..2725633e09104db3912ec6167bb= da652f40aa969 100644 --- a/docs/devel/testing/functional.rst +++ b/docs/devel/testing/functional.rst @@ -63,6 +63,8 @@ directory should be your build folder. For example:: $ export QEMU_TEST_QEMU_BINARY=3D$PWD/qemu-system-x86_64 $ pyvenv/bin/python3 ../tests/functional/test_file.py =20 +By default, functional tests redirect informational logs and console outpu= t to +log files. Specify the ``--debug`` flag to also print those to standard ou= tput. The test framework will automatically purge any scratch files created duri= ng the tests. If needing to debug a failed test, it is possible to keep these files around on disk by setting ``QEMU_TEST_KEEP_SCRATCH=3D1`` as an env diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index 2a78e735f1604f21efd18e38ee0d586496d6b38c..eedca7f1ad29c9e654cf56535ac= f9639d679f5c4 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -11,6 +11,7 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 +import argparse import logging import os from pathlib import Path @@ -33,6 +34,28 @@ =20 =20 class QemuBaseTest(unittest.TestCase): + debug: bool =3D False + + """ + Class method that initializes class attributes from given command-line + arguments. + """ + @staticmethod + def parse_args(): + test_name =3D os.path.basename(sys.argv[0])[:-3] + parser =3D argparse.ArgumentParser( + prog=3Dtest_name, description=3D"QEMU Functional test" + ) + parser.add_argument( + "-d", + "--debug", + action=3D"store_true", + help=3D"Also print test and console logs on stdout. This will = make " + "the TAP output invalid and is meant for debugging only.", + ) + args =3D parser.parse_args() + QemuBaseTest.debug =3D args.debug + return =20 ''' @params compressed: filename, Asset, or file-like object to uncompress @@ -197,6 +220,14 @@ def assets_available(self): return True =20 def setUp(self): + self.stdout_handler =3D None + if QemuBaseTest.debug: + self.stdout_handler =3D logging.StreamHandler(sys.stdout) + self.stdout_handler.setLevel(logging.DEBUG) + formatter =3D logging.Formatter( + "%(asctime)s - %(name)s - %(levelname)s - %(message)s" + ) + self.stdout_handler.setFormatter(formatter) self.qemu_bin =3D os.getenv('QEMU_TEST_QEMU_BINARY') self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be= set') self.arch =3D self.qemu_bin.split('-')[-1] @@ -216,12 +247,17 @@ def setUp(self): '%(asctime)s - %(levelname)s: %(message)s') self._log_fh.setFormatter(fileFormatter) self.log.addHandler(self._log_fh) + if self.stdout_handler: + self.log.addHandler(self.stdout_handler) =20 # Capture QEMUMachine logging self.machinelog =3D logging.getLogger('qemu.machine') self.machinelog.setLevel(logging.DEBUG) self.machinelog.addHandler(self._log_fh) =20 + if self.stdout_handler: + self.machinelog.addHandler(self.stdout_handler) + if not self.assets_available(): self.skipTest('One or more assets is not available') =20 @@ -231,15 +267,19 @@ def tearDown(self): if self.socketdir is not None: shutil.rmtree(self.socketdir.name) self.socketdir =3D None - self.machinelog.removeHandler(self._log_fh) - self.log.removeHandler(self._log_fh) - self._log_fh.close() + for handler in [self._log_fh, self.stdout_handler]: + if handler is None: + continue + self.machinelog.removeHandler(handler) + self.log.removeHandler(handler) + handler.close() =20 def main(): warnings.simplefilter("default") os.environ["PYTHONWARNINGS"] =3D "default" =20 path =3D os.path.basename(sys.argv[0])[:-3] + QemuBaseTest.parse_args() =20 cache =3D os.environ.get("QEMU_TEST_PRECACHE", None) if cache is not None: @@ -297,6 +337,8 @@ def setUp(self): fileFormatter =3D logging.Formatter('%(asctime)s: %(message)s') self._console_log_fh.setFormatter(fileFormatter) console_log.addHandler(self._console_log_fh) + if self.stdout_handler: + console_log.addHandler(self.stdout_handler) =20 def set_machine(self, machinename): # TODO: We should use QMP to get the list of available machines @@ -403,6 +445,10 @@ def set_vm_arg(self, arg, value): def tearDown(self): for vm in self._vms.values(): vm.shutdown() - logging.getLogger('console').removeHandler(self._console_log_fh) + console_log =3D logging.getLogger("console") + console_log.removeHandler(self._console_log_fh) self._console_log_fh.close() + if self.stdout_handler: + console_log.removeHandler(self.stdout_handler) + self.stdout_handler.close() super().tearDown() --=20 2.47.2 From nobody Sat Nov 15 09:31:56 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1753436609; cv=none; d=zohomail.com; s=zohoarc; b=AQcNImewOEU7TyqdqDCDyyVJo2s01h24m4Wp4IfbA43veWoX+V8XiHpsg1lxuWWDFq8JTtgFvxbzI1xM5G632ncXbCX+lGsmyXvgjJAfu6SW7UUpMBAUFpcVMAsvveWoMxJfJKYrG2piSnOtN9pWxJo41/Nz0kw9YQQQNrthIWA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753436609; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=XM5OB303OsLObnsO70x9zZIJY/dDj9tedjFVb26nI4Q=; b=cb3yiR+luPijqYdoSCBhiEAl4hoLLiOg1tZrGaIW2+rV5vP/ckYYcNkyZdukliw63V23DCGqWZ7aEPT6A4lQsMvHox/aIhozGYR8MNXoKILMpzrQoUu6mRd32+/GthYSEndcpW+Mu/K7Y0+VtLG1fHjiYk1zPiPC9boyMHOZm/U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753436609183320.2470738875711; Fri, 25 Jul 2025 02:43:29 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ufEw8-0008Ab-3N; Fri, 25 Jul 2025 05:41:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ufEvv-0007wM-R8 for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:47 -0400 Received: from mail-ej1-x635.google.com ([2a00:1450:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ufEvt-0006Nk-W2 for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:43 -0400 Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-ae3be3eabd8so426157666b.1 for ; Fri, 25 Jul 2025 02:41:41 -0700 (PDT) Received: from [127.0.1.1] (ppp-2-86-210-241.home.otenet.gr. [2.86.210.241]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af47ff42fc5sm243280266b.143.2025.07.25.02.41.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 02:41:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1753436500; x=1754041300; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XM5OB303OsLObnsO70x9zZIJY/dDj9tedjFVb26nI4Q=; b=XfXpOuo6Kbv+p8yrp3ixg2OpSNa+6yrrDvhCilxRYspHcouKbG+g6hP1vjHp3b8hUY SDg97aeVclQRzG5ajPabpK+b/I4ruPN7sL1RvlucRYgvi1M35uUg/pINfLLnyBz9of1H IADH/vHvkY+vIkq1FO1kKto7/Z4rqxNKd2jgu69YZzZRcbu9E/ATrr0pg5bfkdwhtsky XDqqNQLspALoH7fbmHATydeP6OQ6KrXKRQ1cH2eEHPPlhqFxND+aSbOeRsImg+/ZMOF/ AbfSw8tqMM20oVuf2WYT7AGpRNdhIz59uP9Z1LwjSB8QSh9JfPcP311Eu+G44WPJI5Fr KSiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753436500; x=1754041300; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XM5OB303OsLObnsO70x9zZIJY/dDj9tedjFVb26nI4Q=; b=ItACm20KvcfCQ16FVvpdxy2BBTA8MNISJpl63NGVrritSx9Xqn7i1e05d04j3c1nUy o21oh3P8LYlYMRd70xMcVG/R8qhFM8wI8YY/KOAKyQj8KqKWrZAA2PTQ4My+FxHk8j9F ootsyTy9r8Q3oZut9c07aOxaKYMMDGaqMq7IVPmyLBEmA24SPUCioUSL1CboG4cgsf7B rj3f6IMGxfGp34DQF1lQnTADA1oyoIk7SfUShGREbAs5uJFcR3NOyEYFFTh5VLGpu83G 7rRtbbWaEen3+g5d5V5/rycb96S1qbiJqig0Az3F2ibF0zUow4z/z/iEO3SxCCWY9DEs h6wA== X-Gm-Message-State: AOJu0Yx/HwSw/epwf07dff+Llt0z7qZkIZC63M9LI7yr4uL9N/fVsyV+ PwvBcD+BJrClh4mZzl2BC3VH9zBen1j5Ah63gXf9mivadcj1xfrHkh3+92VkmJOdb8ivfSw4GrO qn+4R X-Gm-Gg: ASbGnctDfjcOeT9MOKZ+xdF3FcKIGzEq8mYTg6U7KEpCawpGucDr+63eT1dE1a3681t Mk2EzFmI2dPIfzBCqmXoogfuwQmXT6Wcq7JZmIQLLrxTyzUO7ZkG90dPHbS4sjGO2Okt/Sg6urX KJ98L0PlI4NRUqXWibLK60ZdWEopKspaIdcl1P/eqgD9eNF5KeddN4zGT6azDfE/47sWBW2c6Y0 N960yhqflCJTKYo6vcSFTBd6JXYehi6b5hUGt0bFYwsQmyUGJaeTex7Ouva89B7Ydx3NKrDAHVa JKR+Y/+UQ6AU7EahGklVq36Pi2bUBnSamRZ6vY/R0emkQ7VjKvHEuNb/Qh7Pncgy0VGPsXtVSSD tI4l8svfGPktsxaPPvH+ci3L5LBCBp99uH00l7uHB6/UFcs+U5ov/yjU10VaEKvcXdpU= X-Google-Smtp-Source: AGHT+IHgQyhFlUJs2PEJu+gYnhmMJabWpzZ4B5zplCflJHyBTajVvjsRm7PEO03elzmKtjJEtVVjBQ== X-Received: by 2002:a17:906:4fd6:b0:adb:2f9b:e16f with SMTP id a640c23a62f3a-af61df590eemr145282266b.16.1753436500060; Fri, 25 Jul 2025 02:41:40 -0700 (PDT) From: Manos Pitsidianakis Date: Fri, 25 Jul 2025 12:41:23 +0300 Subject: [PATCH v3 2/4] tests/functional: add --keep-scratch CLI arg MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250725-functional_tests_debug_arg-v3-2-b89921baf810@linaro.org> References: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> In-Reply-To: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> To: qemu-devel@nongnu.org Cc: Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Gustavo Romero , Pierrick Bouvier , =?utf-8?q?Alex_Benn=C3=A9e?= , Paolo Bonzini , Manos Pitsidianakis X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2095; i=manos.pitsidianakis@linaro.org; h=from:subject:message-id; bh=lTEBmaVIcOg7XE2IMXRsNH+9p1M+nUMOfAZ/dVUkjyk=; b=LS0tLS1CRUdJTiBQR1AgTUVTU0FHRS0tLS0tCgpvd0VCYlFLUy9aQU5Bd0FLQVhjcHgzQi9mZ 25RQWNzbVlnQm9nMUZQd3lXRy9YTk9ITHVCUllJeWh6eENBYmM3CmpkNzZoN3R3VEJqSnoyVnpC YnVKQWpNRUFBRUtBQjBXSVFUTVhCdE9SS0JXODRkd0hSQjNLY2R3ZjM0SjBBVUMKYUlOUlR3QUt DUkIzS2Nkd2YzNEowSHloRC8wZHl1RU9mQkQ3a2lOVEJlSGRzV1lNUE5HNlQ3dmFYQjdiY3VKbw pxVDV4cWpDRVFFUm9xZ1Vwc2U2aFVFVERRM3NmcUtjb0w1c2pYNmhHS0FiUEE1NThVZTFJWGFjO XFXbnk3c0xHCnBIZ2ZVWFdGVHI0WHJveVRZZGlFRjVNR01kdG00Q05MaXl1VVdDUkEwQ0JkWWxX aVU4WUlrYlhMTkRRdVRaUkkKRVNIQkk4TWk4UTgwc013eXdmWWEvK0g1MVNoQUdPVUVmc3ZRdm1 3eEkxQzFMaE95ZmdhOGtZZFBLSmpDdDNmQQpuZUFxZjVER2QxcHcveDErTHZLWVZJV2pubXZoSk F5ZGg4T3N0Zk5QUGVxSXhnU0Z5VkhtV2RnOXZrYXhlWW16Cjg4eG1uaTU0NEJ5QnRoSC9SM2I5S kYrRnhieHZQbm1WaHJYek5UK2tRcS9yRmRRUEQzN3NjOC83QlEyVFBheW8KUGNYa29yTDFnMXUy MkV3TUJTOWgveGlodHp5M25XLzA4NFFEZXAvVEZGdWM5S0F0SDJTVW1uUnhZK0xnM1pFcApROEh SRGkyY2JHTHhkMURQM3ZzOUR6aFpkR04rUy94QUs2Rll5eFZZQlVPTFZ1TXFzWkJUQUtCU3hNQn FIMjFoCitXMVRncWNUcDArbEpSU1pndko4amtqd3hpV3d4Ym1VK0FCbjVDb2FtYUx2bm9UR0lQR DZhVjVOYTByUFRQQ0QKV3hueWM4dndnTnpvVFF3a3NPNEVRdG1ZaTNjTzYyaGpaaUhyY0EzUEdN MUoxU2U2bk5kTXBiQWxmWGc4S2IxTwpOcXFXTWhtVDhBUjBGTEEzU0ljVzlGK1hhZFFvWEVmVFl CRzA1Tjh1WHlxaXJlbzRvZS9kN2JSYVB2Z1dTRnZWCkMzQ3N3QT09Cj1nbmNnCi0tLS0tRU5EIF BHUCBNRVNTQUdFLS0tLS0K X-Developer-Key: i=manos.pitsidianakis@linaro.org; a=openpgp; fpr=7C721DF9DB3CC7182311C0BF68BC211D47B421E1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::635; envelope-from=manos.pitsidianakis@linaro.org; helo=mail-ej1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1753436611171116600 Add CLI arg to keep scratch files after test execution, equivalent to setting QEMU_TEST_KEEP_SCRATCH env var. Suggested-by: Alex Benn=C3=A9e Acked-by: Thomas Huth Signed-off-by: Manos Pitsidianakis --- tests/functional/qemu_test/testcase.py | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index eedca7f1ad29c9e654cf56535acf9639d679f5c4..f7e306cf749e8b24a712b09dfe8= 673660cbb5085 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -35,6 +35,7 @@ =20 class QemuBaseTest(unittest.TestCase): debug: bool =3D False + keep_scratch: bool =3D "QEMU_TEST_KEEP_SCRATCH" in os.environ =20 """ Class method that initializes class attributes from given command-line @@ -53,8 +54,16 @@ def parse_args(): help=3D"Also print test and console logs on stdout. This will = make " "the TAP output invalid and is meant for debugging only.", ) + parser.add_argument( + "--keep-scratch", + action=3D"store_true", + help=3D"Do not purge any scratch files created during the test= s. " + "This is equivalent to setting QEMU_TEST_KEEP_SCRATCH=3D1 in t= he " + "environment.", + ) args =3D parser.parse_args() QemuBaseTest.debug =3D args.debug + QemuBaseTest.keep_scratch |=3D args.keep_scratch return =20 ''' @@ -262,8 +271,10 @@ def setUp(self): self.skipTest('One or more assets is not available') =20 def tearDown(self): - if "QEMU_TEST_KEEP_SCRATCH" not in os.environ: + if not QemuBaseTest.keep_scratch: shutil.rmtree(self.workdir) + else: + self.log.info(f"Kept scratch files in {self.workdir}") if self.socketdir is not None: shutil.rmtree(self.socketdir.name) self.socketdir =3D None --=20 2.47.2 From nobody Sat Nov 15 09:31:56 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1753436527; cv=none; d=zohomail.com; s=zohoarc; b=aONt37LvLAhIntDtCsHHSIAIwV+pHRfAu7dypeK8V3FYrbglynBBap+3xXTJaVQ2gGIIwdL5dNvc0haG/joWyhRKvu7NQdO788+DplhpmPCHMcm4XW/IalTG2okAwx5NlOwI+sYEkw314NTsNH7GBiX+NRgx+F3UucmjSr6/a0Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753436527; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=81X9JWZLMcoyz9677dNtF74dQOQRKg/cSylyBcaNqrc=; b=Tnj2GLdjzLJlgcDgoBjvqpFA0RQ81KlF+jXc5v0ubo8YejgJ4A3ikvKsfJ1oT14DpY7jG8H1puj9yDXIMhFU0PlJC0io673ZqUDRj5FQlbh15RB2mgyiAgODYKmKil3BiW3nmpuOp/godLqJjdBuV6pB+NUbK7sI1Zp5FzCTBWI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753436527018977.6399919257001; Fri, 25 Jul 2025 02:42:07 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ufEw6-00086v-PE; Fri, 25 Jul 2025 05:41:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ufEvx-0007ww-An for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:49 -0400 Received: from mail-ej1-x630.google.com ([2a00:1450:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ufEvv-0006O5-FX for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:45 -0400 Received: by mail-ej1-x630.google.com with SMTP id a640c23a62f3a-ae0c571f137so405236666b.0 for ; Fri, 25 Jul 2025 02:41:43 -0700 (PDT) Received: from [127.0.1.1] (ppp-2-86-210-241.home.otenet.gr. [2.86.210.241]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af47ff42fc5sm243280266b.143.2025.07.25.02.41.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 02:41:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1753436501; x=1754041301; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=81X9JWZLMcoyz9677dNtF74dQOQRKg/cSylyBcaNqrc=; b=WNZDi7e6smm/Abvm4/lug7QMrI12gnpTQTvEf8hy9QxlbVeU8yY60Cvh13+1H4Jksi Sq40rMZd9vGEDdwZV9dF3/zo6bE+B4jVxoIUAZra2WVywYQ12AeYKixypLE3AEmAnFNN LPNepqT20J6XUkNzoFB468DZPS6hwvnZbgECuyno9R7XaPt/KU7AusDbOo045ELPAxJi h2zd3Mkd8rJNlg56UWZMBQnQFMIRbBjV5gxOfgIDt/zCvUl+lvvOPbKsFEGoduuuw3OH g7LD5aBxBCIlAXv905xXsZQcEEwWaqtkqqJZS5BmesfAY9iNAvT1WtGE/geumyinbGJY 9WuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753436501; x=1754041301; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=81X9JWZLMcoyz9677dNtF74dQOQRKg/cSylyBcaNqrc=; b=gqDZP5biYlq198WiUR9QKEB4mRtxjThY6VXM/sdI78QJn/Bnt+vbQhgdPJ2K858AlD 9N/OWKc1f03Vxc4tdNeeqwEhUjJaZbZFTuFTmz5JI9j/ravW4qczpuZcfztXMjuGgnJx 0suYVkbkHqBKKOrhKkFOnUMP6uraDQhqTgsF4wZqnlktV86uzXX26hf+te/CLXBiTtv/ bXu0t4H8yPdkXmTFepFQm1opxUJlaE1SyuHXUNFQwZEp+k03lx9zMIlwRweieQt45Hwh wto4n2iD16v3BHtfEEUgYQvLtjwKNNo/J9RkEszpAl/PcpvJHfBqvHa1nC/E865KJovo bG8Q== X-Gm-Message-State: AOJu0YxQALw9wszSf1QWskFmQuNlhhRibGEs95gcEJlNkSf/bO/e70ZN RR62nKXKzpWcgGPdXJrPiZnkuH29LYosNwMjOZYggPRsKZmWvwVERRj235w+SwQS2v62DIZM9lw jJ5S/ X-Gm-Gg: ASbGncsot5s+wHZL+bAHJOaM1ZNI1SM2Z1svVFU2MFP64GoyT9QTFTmOU1JyCt0e62C 5l9V46JcoikblmCwgBw+KKNKE9IhTjpyD63XHIKznpq10af3OLbopmcvZo0OZp+UbI8k2Yi1PIc IUBF0uYkI3aUSmXNDcpygJTEHGFk7A3lCqSxSUzaC8Hx4F6sd1kqwSa7keHIXyLaQEuhvK5GBeT bakQFH82HP/Y9rwdTLwEfVQiSJ5J50teW9391SlQhMfOvsg9IRcuSwbIJ9FzNBmtHQaYt6UbYNZ GDahoQ0yGyKPmMc8R1keWg2sWltqU7hvdGYfDdDj2opyaEsizALqSgq4dtVVOeCUnioCVDZCTKe yO1MZyQdTmxxPW4u7p3YDIZyQm23HbiHKievhnnsKttHGBAEpfGhEamxuYLN6usb1vmY= X-Google-Smtp-Source: AGHT+IF22fBq65tZ3cIRKezujYQzES6STcdPHk8+DGMGsC3h+rQ3mt36Hp5EuP+NyubEQhNzTCK+8w== X-Received: by 2002:a17:907:db16:b0:ad5:5210:749c with SMTP id a640c23a62f3a-af6175096b5mr143516766b.22.1753436501304; Fri, 25 Jul 2025 02:41:41 -0700 (PDT) From: Manos Pitsidianakis Date: Fri, 25 Jul 2025 12:41:24 +0300 Subject: [PATCH v3 3/4] tests/functional: add --list-tests CLI arg MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250725-functional_tests_debug_arg-v3-3-b89921baf810@linaro.org> References: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> In-Reply-To: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> To: qemu-devel@nongnu.org Cc: Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Gustavo Romero , Pierrick Bouvier , =?utf-8?q?Alex_Benn=C3=A9e?= , Paolo Bonzini , Manos Pitsidianakis X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2401; i=manos.pitsidianakis@linaro.org; h=from:subject:message-id; bh=xoXEOsuC3Si79RsCr5E0Hl1RUWvEca1gYm5ywI/ZdJ8=; b=LS0tLS1CRUdJTiBQR1AgTUVTU0FHRS0tLS0tCgpvd0VCYlFLUy9aQU5Bd0FLQVhjcHgzQi9mZ 25RQWNzbVlnQm9nMUZQdmNFNmxoU29OV0lNamorMmFDaEZqSVBaCkhrYUZ1dUZzRzA2cmlLVHJB MmVKQWpNRUFBRUtBQjBXSVFUTVhCdE9SS0JXODRkd0hSQjNLY2R3ZjM0SjBBVUMKYUlOUlR3QUt DUkIzS2Nkd2YzNEowTmFxRUFDTnRqU3ErQTZFQXk3c3hXN25EVDVLcDNTRzJEOGVjMUVNTnRyZA pMVUFUNERoRDdEYkU4aDVMRkZPOC84ZXJlQ2FZTFFqcmViakdVMTJvb0paQTBqWTVXeEVaOHRzV jFxQ012ZVdXCnZJWTFVakI0ZUtKTlh2Yk1LZ00reFZ4VStMR0EvT25lN2RGdk1YRWsyZ3ZHZ3ky YklpcHpuZFhBaTkxeVhFcmEKdDZvNWd5b2hnM1pqemdvNDR0b21nRzhEQkUvM2lNUHhleVk4M1Z PenRiWld0UmZtM3puZHZHb002ejhIL2F0cwpPWk12Y2Y3ZEcxZTBKSDBIZE4rZE1HejUzeVdPck 9uZng0YlhxSkRDWVlWcUQ2U0JQdTZXakhuS3MyYVlrby9HClp3a3U5VjRsL205anloSGt6QWlPM GI4K2pSc2RFUEQ5SmRhVjcvTzM0ZkdTN2k0b3pnT3A2bXZMYStLQUxZUGgKSVcvZW9Hb0dSNXV1 eEFUNzFRNUpVbjc5QmFLY3pFOGpmWEIrSjJzVmZ5R0JOZVgyeTVubTI5aTJnRFQ1R2Q4RworNjd qaytUUUgvNk1NTHdKczc0bjRSaXA4RThXR3NiZ2w4NXN5K2VjdVIrMi9ZQ1EyVXJkRjNuSXJ4Un Q3UU5sClhReUhFUTR6WWhoVkpEdU9wNHpKVld0SXZGY1dTMmNPVHU5WjU3SXArNTFGakRKRmVvW GQ2OXV1MVVFYXdqUnkKQkNERFAxbHBSYzRuMm96dU05VjNPQXdWQUtTdzRFMnhIc29HUHNZemhk NVVYaGN0N0JRamwxTGpjWDZTTUprdwphUVdyNTEwYTV6cEsydzVjSVNVaDcwVktYRFIzeGJkY1B sTC9vdWJUNXQyc1ZxY2MxcWJoMWJmVzdYSG8zY0tFClFEWUJUZz09Cj0rQSt1Ci0tLS0tRU5EIF BHUCBNRVNTQUdFLS0tLS0K X-Developer-Key: i=manos.pitsidianakis@linaro.org; a=openpgp; fpr=7C721DF9DB3CC7182311C0BF68BC211D47B421E1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::630; envelope-from=manos.pitsidianakis@linaro.org; helo=mail-ej1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1753436528481116600 Add CLI argument to list tests and exit. Example output (current dir is build directory under root dir): $ export PYTHONPATH=3D../python:../tests/functional $ export QEMU_TEST_QEMU_BINARY=3D"$(pwd)/qemu-system-aarch64" $ ./pyvenv/bin/python3 ../tests/functional/test_aarch64_virt.py --list-te= sts test_aarch64_virt_gicv2 (test_aarch64_virt.Aarch64VirtMachine.test_aarch6= 4_virt_gicv2) test_aarch64_virt_gicv3 (test_aarch64_virt.Aarch64VirtMachine.test_aarch6= 4_virt_gicv3) test_alpine_virt_tcg_gic_max (test_aarch64_virt.Aarch64VirtMachine.test_a= lpine_virt_tcg_gic_max) Signed-off-by: Manos Pitsidianakis --- tests/functional/qemu_test/testcase.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index f7e306cf749e8b24a712b09dfe8673660cbb5085..ab564f873c303bcc28c3bf7bec8= c8c4569fae91c 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -36,6 +36,7 @@ class QemuBaseTest(unittest.TestCase): debug: bool =3D False keep_scratch: bool =3D "QEMU_TEST_KEEP_SCRATCH" in os.environ + list_tests: bool =3D False =20 """ Class method that initializes class attributes from given command-line @@ -61,9 +62,15 @@ def parse_args(): "This is equivalent to setting QEMU_TEST_KEEP_SCRATCH=3D1 in t= he " "environment.", ) + parser.add_argument( + "--list-tests", + action=3D"store_true", + help=3D"List all tests that would be executed and exit.", + ) args =3D parser.parse_args() QemuBaseTest.debug =3D args.debug QemuBaseTest.keep_scratch |=3D args.keep_scratch + QemuBaseTest.list_tests =3D args.list_tests return =20 ''' @@ -292,6 +299,13 @@ def main(): path =3D os.path.basename(sys.argv[0])[:-3] QemuBaseTest.parse_args() =20 + if QemuBaseTest.list_tests: + loader =3D unittest.TestLoader() + for test_suite in loader.loadTestsFromName(path): + for test in test_suite: + print(test) + return + cache =3D os.environ.get("QEMU_TEST_PRECACHE", None) if cache is not None: Asset.precache_suites(path, cache) --=20 2.47.2 From nobody Sat Nov 15 09:31:56 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=linaro.org ARC-Seal: i=1; a=rsa-sha256; t=1753436528; cv=none; d=zohomail.com; s=zohoarc; b=gY8uBRYAhPZC2hmM1NkXPv7AIpV2XHqvl7xl/XhYxjZznqQC+9WkM+HO6VSs0A/N2D+80H+MIrHIG6HEEJBjNPMZeTj6QefmP2d2YJaN0b/CVRv58HXOGRne8xcNrzBTIUfBO6upPNxC0J1Z6plX2xDOAdxcB45cevlJ7QZ7GN0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753436528; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=MypPh5tWMN+S2SCsIb0uToDST9X7A/yBEEXT6W5D7OM=; b=gRZEsffTroQ8CdXAWqjRS/z2A+J+cwYq6mWYnY3Y/7GJHUKteWsZpy4DpI7S7odO2eXVVHrtGZ6pL56lFyz0wbVlNIOFh7jv+2mW92GWDnrgTG6J+cCf+xz7m9/HGMwNpXd/AxkWpjFnIDMtc/Q1hK5aYMODJpDgvVz+Vak9fXI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753436528628787.003047963165; Fri, 25 Jul 2025 02:42:08 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ufEw8-0008Dw-To; Fri, 25 Jul 2025 05:41:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ufEw0-0007zO-1o for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:49 -0400 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ufEvx-0006Od-2a for qemu-devel@nongnu.org; Fri, 25 Jul 2025 05:41:47 -0400 Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-ae0bc7aa21bso381424266b.2 for ; Fri, 25 Jul 2025 02:41:44 -0700 (PDT) Received: from [127.0.1.1] (ppp-2-86-210-241.home.otenet.gr. [2.86.210.241]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-af47ff42fc5sm243280266b.143.2025.07.25.02.41.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jul 2025 02:41:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1753436504; x=1754041304; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=MypPh5tWMN+S2SCsIb0uToDST9X7A/yBEEXT6W5D7OM=; b=eNT+wphlkNkxwqCTzh0Fg5/VU5Zm6JB6wEYaw110uL2UQKidnEF5IrAlMaTWFSHpBf jwCcfy6b/jSoNHJl5umwqmVyeqMvreO1Gl6vaVxWP7ggUhWlhg4tXH6m+dHysTxmUbbC w6ieDu7DZUeweNNHd/dGdwAR3uxyDffmIGz/4QvK1HrzkHHsg+ssfvihK1+yu6F9pWak Y6m9PKqx4nk9ayLe+prM6zN8xrCP+hmYVFnwNAxOr6H7+n28VVb9j2/iTvLavAgws08b 6Whj17+MtZXKi5/EJ2jZ/JCpg3hIecj3DvLftmfqINS32Kwg2zA2+yanLQbeg6TlD2CX 1xYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753436504; x=1754041304; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MypPh5tWMN+S2SCsIb0uToDST9X7A/yBEEXT6W5D7OM=; b=Jlvwvo/HAbBsu0BWYUw8lp/QeWd1ahDEEvH0ynnMUZTRNjhGU4ibv7pOkGdcU6wyQm AigYQPJqKqYZrzQhbZCxXxOHx0/uRMAQT8Pkqk8aaw+3jR40ZcoW68BenOZ2dTv8X91l U+PKaEuPscpzpsbuBRREjO13AxgrSyG/9AZ6V7UGBxcUwF9qogsPNtYxMc6jhgb26smG va4/eIv/0v1ASw/7OjGDgiv4kFc1O4iodXNoIqvgcIUZutWY0KCrhhmoEwVCsPmVM/od NOE5xM2i1t2jql8MLbQaHH9E5hUzqaltI1w9ASX5Z2Of8/Xiq+Mle4YbvsYvw870wX5q opQA== X-Gm-Message-State: AOJu0YyEPqrWiAKnc6DbBkfNVeWDo5YF2J6vyW82iUXxcKtWbWmR0jMd tYDYwEifvs5wbDp01EXyYag7z7u6i5dMwpioYsVYN+IP742sRv3WFuSGhdiK3HQ25vbSVL31uX4 eCx/H X-Gm-Gg: ASbGnctmzYeBF85vLdkuz8ZzG1qCEOwc1uJYqOtvix/WtrhU65NEQDLSb8i9UnW1r2h 1xp5gWNgjdUNnqhQgzGzcZk52ZcQZ0xnve812hd7CVTBLuKv0e1S1rYqIJvq6CDLOc0c6nhZlkY mZWNRF4uPvHZwajtVx2DOiSVguaJSJBer8uab0icLNHVRI565X+u4WSCtNLL0XVH5A5QoBJYayV yiLmggE34FtdOATTZeyfrmICqYJ3PnQSbHz3EXT0nhc+rATClU+MJLPCkHCSqdS83BGdyIA+qwD hIKA3SK6v3qcYSEOFOCWB/nBA8UyeSkjkYH8y+6UsjvQDtfb1Px7PvPnnF79SU61Y/2VHnnhrd6 X5r5x5PjrTmXELIRxMf0iYmBkvTlBZMxSEPy1th0gsZATBQPnOo5jGxymMgRzlDHNv8js/dzSQM Gg8g== X-Google-Smtp-Source: AGHT+IGTHbyJr3iFkumqsc5ePl9evw47e7IOlw+Uhfz3ws9CrsVMYqNLQpwwUDI6xDxV8XEUTdWZ6g== X-Received: by 2002:a17:906:c149:b0:ae6:c0a3:1606 with SMTP id a640c23a62f3a-af61c2b4156mr142833666b.7.1753436503510; Fri, 25 Jul 2025 02:41:43 -0700 (PDT) From: Manos Pitsidianakis Date: Fri, 25 Jul 2025 12:41:25 +0300 Subject: [PATCH v3 4/4] tests/functional: add -k TEST_NAME_PATTERN CLI arg MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250725-functional_tests_debug_arg-v3-4-b89921baf810@linaro.org> References: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> In-Reply-To: <20250725-functional_tests_debug_arg-v3-0-b89921baf810@linaro.org> To: qemu-devel@nongnu.org Cc: Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Gustavo Romero , Pierrick Bouvier , =?utf-8?q?Alex_Benn=C3=A9e?= , Paolo Bonzini , Manos Pitsidianakis X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2807; i=manos.pitsidianakis@linaro.org; h=from:subject:message-id; bh=IdEewpLmVYPdyTy6Htl/RYvxDMvxHLws16hFCO1dk0A=; b=LS0tLS1CRUdJTiBQR1AgTUVTU0FHRS0tLS0tCgpvd0VCYlFLUy9aQU5Bd0FLQVhjcHgzQi9mZ 25RQWNzbVlnQm9nMUZQQWN3dWQ5WDJIbHVxZCtPZ2JWZ0JiamkyClFiM1E5MTdrSVArTkxnWDVy YmFKQWpNRUFBRUtBQjBXSVFUTVhCdE9SS0JXODRkd0hSQjNLY2R3ZjM0SjBBVUMKYUlOUlR3QUt DUkIzS2Nkd2YzNEowSmh3RC85aGYvdEtUT05ya2VzcjJUS1RicnJBRWp5QzBvSURPaStVUlN1RQ pvTEp5WXRHSVhQaXJHMTc1WkdCbEVPeUZHNnhWcHlPQkFxMUhmMi9KQWUvYTd0dWhMbHNxa2RpR jJSZ3UwQURJClN0V084OG9lYXluVHZQNmIxWTZUT3FJcHVQZzNMSXluVjcvRktpZlZjYzNVK1J5 VnVUb2xWMUtSamRoQzl6L3gKaXZjeFJzNjNtK2RwZzNtdjBIZFVneUZURkYyc2wxNHR5eTVDWFJ DUmNuSTBMNjdQYU96Ty91TzhZTTkxRDhVOAo0cWUyUnlXWGpGRHc3L0xVQVhyZ2JIOHhEczNDan J3KzFWT3pvWjZ4eHJ0cmQ3Ym5vVTBJdElNdm9vc1NwY3hPClBCTmRac3NnaVVBNXRMbi9LT1VZe Hd3eWhwZThwSE40USt0NHJuMjl2em0xR1ZidGVYRHJOU01KN2IxYVUwWi8KTVhhOHRHaXhuQkc5 WkZuNWdxQi9IZ2pTRDh5QmNDZlBBVGJTTlpWS1o1aXpFa0t2dEljelZnZ1dKU3JwRlBWUgp3YkJ oZk10WkMvSlA2YXozcXRNNG85amZUTFVwQXZVWFllZXFjZGhiemlYd2RQUXdiai96Rzg5a3VIZm tsbmZWCmlORUlEMks3bWdhQVMwL1pOV3lHdDZYRXNHTnhxNjRHMTNFRVZRclptaWtHV0dlQzlTK 2psRWpzR0xQbTBGbVgKUnRBSmFLUTRBcEFFcjVjOXdlam9uWFE1dnM0YmdmWkNmekxYazlCQmZo Z0NmL01DaFB1aGVqUWRLS2ZIdmlIWgplSngwb04xSm5RRjBiMGFoWHZBUUF2b09PNndNSVM1YmF Gd3JlYWJROWFYS2NsZUN5SFlWRDJRcEprNjV5WFltCnVxd0xZdz09Cj1wdGlVCi0tLS0tRU5EIF BHUCBNRVNTQUdFLS0tLS0K X-Developer-Key: i=manos.pitsidianakis@linaro.org; a=openpgp; fpr=7C721DF9DB3CC7182311C0BF68BC211D47B421E1 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::62e; envelope-from=manos.pitsidianakis@linaro.org; helo=mail-ej1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @linaro.org) X-ZM-MESSAGEID: 1753436530664116600 Add a CLI argument that takes fnmatch(3)-style patterns as value and can be specified many times. Only tests that match the pattern will be executed. This argument is passed to unittest.main which takes the same argument. Acked-by: Thomas Huth Signed-off-by: Manos Pitsidianakis Tested-by: Alex Benn=C3=A9e --- tests/functional/qemu_test/testcase.py | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index ab564f873c303bcc28c3bf7bec8c8c4569fae91c..b045d82caa79d9d161fb868b0b0= 748ad7de453d9 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -16,6 +16,7 @@ import os from pathlib import Path import pycotap +import itertools import shutil from subprocess import run import sys @@ -37,6 +38,7 @@ class QemuBaseTest(unittest.TestCase): debug: bool =3D False keep_scratch: bool =3D "QEMU_TEST_KEEP_SCRATCH" in os.environ list_tests: bool =3D False + test_name_patterns: list[str] =3D [] =20 """ Class method that initializes class attributes from given command-line @@ -67,10 +69,19 @@ def parse_args(): action=3D"store_true", help=3D"List all tests that would be executed and exit.", ) + parser.add_argument( + "-k", + dest=3D"test_name_patterns", + action=3D"append", + type=3Dstr, + help=3D"Only run tests which match the given substring. " + "This argument is passed to unittest.main verbatim.", + ) args =3D parser.parse_args() QemuBaseTest.debug =3D args.debug QemuBaseTest.keep_scratch |=3D args.keep_scratch QemuBaseTest.list_tests =3D args.list_tests + QemuBaseTest.test_name_patterns =3D args.test_name_patterns return =20 ''' @@ -313,8 +324,16 @@ def main(): =20 tr =3D pycotap.TAPTestRunner(message_log =3D pycotap.LogMode.LogTo= Error, test_output_log =3D pycotap.LogMode.Log= ToError) - res =3D unittest.main(module =3D None, testRunner =3D tr, exit =3D= False, - argv=3D["__dummy__", path]) + argv =3D ["__dummy__", path] + ( + list( + itertools.chain.from_iterable( + ["-k", x] for x in QemuBaseTest.test_name_patterns + ) + ) + if QemuBaseTest.test_name_patterns + else [] + ) + res =3D unittest.main(module=3DNone, testRunner=3Dtr, exit=3DFalse= , argv=3Dargv) for (test, message) in res.result.errors + res.result.failures: =20 if hasattr(test, "log_filename"): --=20 2.47.2