From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451778; cv=none; d=zohomail.com; s=zohoarc; b=gMMkTkbxzIKGnx4rqDDkU/ieHPTQqb+walnvu7mUB7MxSzov2PXajQl4Iom3fwFi04wtQ08+mtsgqOskMwJTpweb3wRhy0pMDmnhjFtNHkAdEE3IIqVqORz0kziF772pnHRq6/+VYXGR9KQcEQyJjcEWux6iBgmFXd4hmrWAuXI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451778; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=X1FqZHlUAELokAeZhleWmAqzlFP/GclIZNyedA1V2Eg=; b=SCJaBllNTq8/SRqLNTTQdEnknZNlhxuf4yFg2V/Zphe5XpEcW3jv7Pb3h73QI60A4eJYyHcFaJ9RIor4I4Ii+FLpQetl8ZnRDxYJwwkcs/28NPKRfLza64EY5MSaDG2GjCo4+Dmm+CFy2pypGEfQ4Q8iRj5qdt9ISlLLOm89MwU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451778213442.7963146532477; Tue, 17 Dec 2024 08:09:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNZze-0004Bv-0N; Tue, 17 Dec 2024 11:00:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzX-000474-PQ for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:13 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzT-0001tA-Qu for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:10 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-83-4RzrZGKYMf2PdT-b8QGJ8w-1; Tue, 17 Dec 2024 11:00:03 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6621519560A6; Tue, 17 Dec 2024 16:00:00 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5F88130044C1; Tue, 17 Dec 2024 15:59:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451205; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X1FqZHlUAELokAeZhleWmAqzlFP/GclIZNyedA1V2Eg=; b=K3lVzPdh2LPwnVd+ATN6UvHvUBWWVIVujYM+xVF9/cHKEvs61YvVkUysVKeFDorykfN4un y9+HcIDDAMVnu9p7w5Ps9U1vy20Ce76CzF6ZKcp1fdL3L+bLISXUuonmbBlWk/jj046nec 8895575+JbTyRx8Z1hQ2eHgWCYvcJ5A= X-MC-Unique: 4RzrZGKYMf2PdT-b8QGJ8w-1 X-Mimecast-MFC-AGG-ID: 4RzrZGKYMf2PdT-b8QGJ8w From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v3 01/32] tests/functional: add execute permission to aspeed tests Date: Tue, 17 Dec 2024 15:59:22 +0000 Message-ID: <20241217155953.3950506-2-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451780500116600 Tests are expected to be directly invoked when debugging so must have execute permission. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Richard Henderson --- tests/functional/test_aarch64_aspeed.py | 0 tests/functional/test_arm_aspeed_ast1030.py | 0 tests/functional/test_arm_aspeed_ast2500.py | 0 tests/functional/test_arm_aspeed_ast2600.py | 0 tests/functional/test_arm_aspeed_palmetto.py | 0 tests/functional/test_arm_aspeed_rainier.py | 0 tests/functional/test_arm_aspeed_romulus.py | 0 7 files changed, 0 insertions(+), 0 deletions(-) mode change 100644 =3D> 100755 tests/functional/test_aarch64_aspeed.py mode change 100644 =3D> 100755 tests/functional/test_arm_aspeed_ast1030.py mode change 100644 =3D> 100755 tests/functional/test_arm_aspeed_ast2500.py mode change 100644 =3D> 100755 tests/functional/test_arm_aspeed_ast2600.py mode change 100644 =3D> 100755 tests/functional/test_arm_aspeed_palmetto.py mode change 100644 =3D> 100755 tests/functional/test_arm_aspeed_rainier.py mode change 100644 =3D> 100755 tests/functional/test_arm_aspeed_romulus.py diff --git a/tests/functional/test_aarch64_aspeed.py b/tests/functional/tes= t_aarch64_aspeed.py old mode 100644 new mode 100755 diff --git a/tests/functional/test_arm_aspeed_ast1030.py b/tests/functional= /test_arm_aspeed_ast1030.py old mode 100644 new mode 100755 diff --git a/tests/functional/test_arm_aspeed_ast2500.py b/tests/functional= /test_arm_aspeed_ast2500.py old mode 100644 new mode 100755 diff --git a/tests/functional/test_arm_aspeed_ast2600.py b/tests/functional= /test_arm_aspeed_ast2600.py old mode 100644 new mode 100755 diff --git a/tests/functional/test_arm_aspeed_palmetto.py b/tests/functiona= l/test_arm_aspeed_palmetto.py old mode 100644 new mode 100755 diff --git a/tests/functional/test_arm_aspeed_rainier.py b/tests/functional= /test_arm_aspeed_rainier.py old mode 100644 new mode 100755 diff --git a/tests/functional/test_arm_aspeed_romulus.py b/tests/functional= /test_arm_aspeed_romulus.py old mode 100644 new mode 100755 --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451291; cv=none; d=zohomail.com; s=zohoarc; b=ce6fKnjTJx4xSzOwyte06hpQRDsRgOz778+LbYPCjowa+L99HauE22yXGDncNtu21J1EMWLhmIFhOI0PicZBJ2wxXBWxr45Y7dQo8fy/I6aiLdsoxSVY8q45960QKgQur+MPeGi/cASEi+xPIlPg3U9/u7svP7lxPPOTHkn4eVs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451291; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=t555vTEvyTWhoxnveFMNxWQgwKPFbd3cf+eGUqybrMo=; b=Og2MyCFogPsgfLI8FKElt83I6mFvj6btNF/m6peD8HQpyq8ppdOtJFnRGzUN/r70lYskg8X5lLFuXHtWHZVS98vha6yz3kcLIZY1Bn+Ww8vhGaLrRAc01xWp1vy+EbjSZh9jkw0tT4sX1mMR5jfvhSfVHAceEp5K+Nd4B6dmAXw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451291498707.8906040268804; Tue, 17 Dec 2024 08:01:31 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNZzh-0004DI-Gy; Tue, 17 Dec 2024 11:00:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzb-00049f-8C for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:15 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzV-0001tR-Qp for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:11 -0500 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-629-9HAYMvlJOYaB4tgcL9KiWA-1; Tue, 17 Dec 2024 11:00:04 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EFE8E19560A2; Tue, 17 Dec 2024 16:00:02 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0165D300F9B5; Tue, 17 Dec 2024 16:00:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451207; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t555vTEvyTWhoxnveFMNxWQgwKPFbd3cf+eGUqybrMo=; b=N3Q3vNsG6bS0zX21b70yZVXOMwP/mKs30Mu7wJHEtLq+/2Hy4VdAb8WpFs2YlpXE4md7PU kT7Q6B3JGXmP9s7QlMFIhxw1unEqC0gjhm8nXX229VVCL8r0iB6fTJSzL1IS2LDWcV7PUK HREzYMR7HAZZLFkUZN22CzToV/98dT0= X-MC-Unique: 9HAYMvlJOYaB4tgcL9KiWA-1 X-Mimecast-MFC-AGG-ID: 9HAYMvlJOYaB4tgcL9KiWA From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 02/32] tests/functional: remove many unused imports Date: Tue, 17 Dec 2024 15:59:23 +0000 Message-ID: <20241217155953.3950506-3-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451292619116600 Identified using 'pylint --disable=3Dall --enable=3DW0611' Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/asset.py | 1 - tests/functional/qemu_test/tesseract.py | 1 - tests/functional/qemu_test/tuxruntest.py | 3 +-- tests/functional/test_aarch64_aspeed.py | 1 - tests/functional/test_aarch64_sbsaref_alpine.py | 1 - tests/functional/test_aarch64_sbsaref_freebsd.py | 1 - tests/functional/test_acpi_bits.py | 2 -- tests/functional/test_arm_bpim2u.py | 2 +- tests/functional/test_arm_collie.py | 2 +- tests/functional/test_arm_cubieboard.py | 1 - tests/functional/test_arm_orangepi.py | 2 +- tests/functional/test_arm_smdkc210.py | 3 +-- tests/functional/test_arm_sx1.py | 2 +- tests/functional/test_microblaze_s3adsp1800.py | 1 - tests/functional/test_ppc_amiga.py | 2 +- tests/functional/test_virtio_gpu.py | 1 - 16 files changed, 7 insertions(+), 19 deletions(-) diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_te= st/asset.py index f126cd5863..559af0351f 100644 --- a/tests/functional/qemu_test/asset.py +++ b/tests/functional/qemu_test/asset.py @@ -9,7 +9,6 @@ import logging import os import stat -import subprocess import sys import unittest import urllib.request diff --git a/tests/functional/qemu_test/tesseract.py b/tests/functional/qem= u_test/tesseract.py index db441027b9..ef1833139d 100644 --- a/tests/functional/qemu_test/tesseract.py +++ b/tests/functional/qemu_test/tesseract.py @@ -5,7 +5,6 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import re import logging =20 from . import has_cmd, run_cmd diff --git a/tests/functional/qemu_test/tuxruntest.py b/tests/functional/qe= mu_test/tuxruntest.py index ab3b27da43..d375f2713b 100644 --- a/tests/functional/qemu_test/tuxruntest.py +++ b/tests/functional/qemu_test/tuxruntest.py @@ -11,10 +11,9 @@ =20 import os import stat -import time =20 from qemu_test import QemuSystemTest -from qemu_test import exec_command, exec_command_and_wait_for_pattern +from qemu_test import exec_command_and_wait_for_pattern from qemu_test import wait_for_console_pattern from qemu_test import has_cmd, run_cmd, get_qemu_img =20 diff --git a/tests/functional/test_aarch64_aspeed.py b/tests/functional/tes= t_aarch64_aspeed.py index 59916efd71..e196f88537 100755 --- a/tests/functional/test_aarch64_aspeed.py +++ b/tests/functional/test_aarch64_aspeed.py @@ -6,7 +6,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import sys import os =20 from qemu_test import QemuSystemTest, Asset diff --git a/tests/functional/test_aarch64_sbsaref_alpine.py b/tests/functi= onal/test_aarch64_sbsaref_alpine.py index ebc29b2fb5..6dbc90f30e 100755 --- a/tests/functional/test_aarch64_sbsaref_alpine.py +++ b/tests/functional/test_aarch64_sbsaref_alpine.py @@ -12,7 +12,6 @@ =20 from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from qemu_test import interrupt_interactive_console_until_pattern from unittest import skipUnless from test_aarch64_sbsaref import fetch_firmware =20 diff --git a/tests/functional/test_aarch64_sbsaref_freebsd.py b/tests/funct= ional/test_aarch64_sbsaref_freebsd.py index 80298dd190..77ba2ba1da 100755 --- a/tests/functional/test_aarch64_sbsaref_freebsd.py +++ b/tests/functional/test_aarch64_sbsaref_freebsd.py @@ -12,7 +12,6 @@ =20 from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from qemu_test import interrupt_interactive_console_until_pattern from unittest import skipUnless from test_aarch64_sbsaref import fetch_firmware =20 diff --git a/tests/functional/test_acpi_bits.py b/tests/functional/test_acp= i_bits.py index 63e2c5309d..3df9562394 100755 --- a/tests/functional/test_acpi_bits.py +++ b/tests/functional/test_acpi_bits.py @@ -31,14 +31,12 @@ https://gitlab.com/qemu-project/biosbits-bits . """ =20 -import logging import os import platform import re import shutil import subprocess import tarfile -import tempfile import zipfile =20 from pathlib import Path diff --git a/tests/functional/test_arm_bpim2u.py b/tests/functional/test_ar= m_bpim2u.py index 35ea58d46c..c9cf43c147 100755 --- a/tests/functional/test_arm_bpim2u.py +++ b/tests/functional/test_arm_bpim2u.py @@ -9,7 +9,7 @@ =20 from qemu_test import LinuxKernelTest, exec_command_and_wait_for_pattern from qemu_test import Asset, interrupt_interactive_console_until_pattern -from qemu_test.utils import archive_extract, gzip_uncompress, lzma_uncompr= ess +from qemu_test.utils import gzip_uncompress, lzma_uncompress from qemu_test.utils import image_pow2ceil_expand from unittest import skipUnless =20 diff --git a/tests/functional/test_arm_collie.py b/tests/functional/test_ar= m_collie.py index 7e144a0a8f..fe1be3d079 100755 --- a/tests/functional/test_arm_collie.py +++ b/tests/functional/test_arm_collie.py @@ -6,7 +6,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class CollieTest(LinuxKernelTest): =20 diff --git a/tests/functional/test_arm_cubieboard.py b/tests/functional/tes= t_arm_cubieboard.py index 2b33a1b50b..9e42b72060 100755 --- a/tests/functional/test_arm_cubieboard.py +++ b/tests/functional/test_arm_cubieboard.py @@ -5,7 +5,6 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 import os -import shutil =20 from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pa= ttern from qemu_test import interrupt_interactive_console_until_pattern diff --git a/tests/functional/test_arm_orangepi.py b/tests/functional/test_= arm_orangepi.py index 6d57223a03..a872305e93 100755 --- a/tests/functional/test_arm_orangepi.py +++ b/tests/functional/test_arm_orangepi.py @@ -11,7 +11,7 @@ from qemu_test import LinuxKernelTest, exec_command_and_wait_for_pattern from qemu_test import Asset, interrupt_interactive_console_until_pattern from qemu_test import wait_for_console_pattern -from qemu_test.utils import archive_extract, gzip_uncompress, lzma_uncompr= ess +from qemu_test.utils import gzip_uncompress, lzma_uncompress from qemu_test.utils import image_pow2ceil_expand from unittest import skipUnless =20 diff --git a/tests/functional/test_arm_smdkc210.py b/tests/functional/test_= arm_smdkc210.py index 967752feeb..b3b39b069d 100755 --- a/tests/functional/test_arm_smdkc210.py +++ b/tests/functional/test_arm_smdkc210.py @@ -5,9 +5,8 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 import os -import shutil =20 -from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pa= ttern +from qemu_test import LinuxKernelTest, Asset from qemu_test.utils import gzip_uncompress =20 class Smdkc210Machine(LinuxKernelTest): diff --git a/tests/functional/test_arm_sx1.py b/tests/functional/test_arm_s= x1.py index 2292317946..b85bfaa178 100755 --- a/tests/functional/test_arm_sx1.py +++ b/tests/functional/test_arm_sx1.py @@ -14,7 +14,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class SX1Test(LinuxKernelTest): =20 diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/functio= nal/test_microblaze_s3adsp1800.py index d2be3105a2..d452a0271c 100755 --- a/tests/functional/test_microblaze_s3adsp1800.py +++ b/tests/functional/test_microblaze_s3adsp1800.py @@ -7,7 +7,6 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -from qemu_test import exec_command, exec_command_and_wait_for_pattern from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern from qemu_test.utils import archive_extract diff --git a/tests/functional/test_ppc_amiga.py b/tests/functional/test_ppc= _amiga.py index b793b5c432..f5faa0f0b3 100755 --- a/tests/functional/test_ppc_amiga.py +++ b/tests/functional/test_ppc_amiga.py @@ -10,7 +10,7 @@ import subprocess =20 from qemu_test import QemuSystemTest, Asset -from qemu_test import wait_for_console_pattern, run_cmd +from qemu_test import wait_for_console_pattern from zipfile import ZipFile =20 class AmigaOneMachine(QemuSystemTest): diff --git a/tests/functional/test_virtio_gpu.py b/tests/functional/test_vi= rtio_gpu.py index d5027487ac..2d298b1f02 100755 --- a/tests/functional/test_virtio_gpu.py +++ b/tests/functional/test_virtio_gpu.py @@ -12,7 +12,6 @@ from qemu_test import exec_command_and_wait_for_pattern from qemu_test import is_readable_executable_file =20 -from qemu.utils import kvm_available =20 import os import socket --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451315; cv=none; d=zohomail.com; s=zohoarc; b=GY60DUf/uhyKtK8MSb3MDDGidYFX5W9H2sgZPnBaRC57Sxj98ayxmrk+K8WYedppn/R67FFsou+ao6oScZaQbp9r8KxqkJ3quesGTvGDHd6ihW4lWT2smjA2tehO/C/hLj68M8H+cY+KWx+PfDb9/B2/Cst7H+E7lmcFWWsV/lQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451315; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=7k+tEYKrr5XUuCj4cGmAGM2/Boaa8pNKMmysiZtXWvs=; b=nuN32fpKGOpErSF4rQF6fTCJOQwchm3jPn7UFigvWXguTdxC2wGeaxqqAQFoQTC5MQk3Esd/OBW+AOe2XZCz0lUwshT343Ax24lHaWCMgLOUwOWh6lbWGUrqojwlC4M+d1KIa3Kp74OODPwXrzYYGOAP4/qF5pyOtD9COIImS4Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451315576342.2271643257088; Tue, 17 Dec 2024 08:01:55 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNZzm-0004En-9r; Tue, 17 Dec 2024 11:00:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzh-0004E7-8n for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZza-000250-2A for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:20 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-63-I_MvxSsWNnq2nbpymWb8JQ-1; Tue, 17 Dec 2024 11:00:09 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CDBAF1955F3D; Tue, 17 Dec 2024 16:00:05 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8A51630044C1; Tue, 17 Dec 2024 16:00:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7k+tEYKrr5XUuCj4cGmAGM2/Boaa8pNKMmysiZtXWvs=; b=aRpRk5htQO5VyDGAJR2ckm2Wcs0qMPw0HPlCy7fX0YqoeeIkptYtxUVW9LB4jSfNuWqi86 RjkzXgmMLZG0CN+Xhv7FaOCed1EOyCfwPBvRLbFFG+KS+koBeCxDnbGd1lOQCY1ja07UK1 Av5N+qwtxKBRA0O9X1AZBF0MNIgX37I= X-MC-Unique: I_MvxSsWNnq2nbpymWb8JQ-1 X-Mimecast-MFC-AGG-ID: I_MvxSsWNnq2nbpymWb8JQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v3 03/32] tests/functional: resolve str(Asset) to cache file path Date: Tue, 17 Dec 2024 15:59:24 +0000 Message-ID: <20241217155953.3950506-4-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451316563116600 Allow an Asset object to be used in place of a filename but making its string representation resolve to the cache file path. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Richard Henderson --- tests/functional/qemu_test/asset.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_te= st/asset.py index 559af0351f..c5d3e73c4b 100644 --- a/tests/functional/qemu_test/asset.py +++ b/tests/functional/qemu_test/asset.py @@ -39,6 +39,9 @@ def __repr__(self): return "Asset: url=3D%s hash=3D%s cache=3D%s" % ( self.url, self.hash, self.cache_file) =20 + def __str__(self): + return str(self.cache_file) + def _check(self, cache_file): if self.hash is None: return True --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451271; cv=none; d=zohomail.com; s=zohoarc; b=XFG8+ffz09VA/l4FAe3z1oO3fIYVZYtZnFjYLmN4/SGuzYXw5ab1W6vfPwScRI0dWAQQmYRIh91MyRV0ibxEi7VDVnBXzMWHTf6HPbl5D7okJcyfxUx1IRu9rGu8blYXt4QiC1jXsIfUNAWTKVGdGYFDOHYCxsGDrQkg9Axjc+Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451271; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fhKdokIULRGPGWSO9LdGxnmdKKDtG0WAE0bknPnTpHU=; b=HQoD5CojQdYyqQ5C62kKRDLKbhK9TmAMR3OanjqhjGBxJIvXpKTtOZpB6SWqiW4IoB9CNFS2eobKODguCn5wxOt3KXRT8WTkRoH1I27EpvndvGcxqJM374aa8NM5ZvfpTv4ITjysK9oXs3jjCfLaTi3PE5SUny1u/L1bhfOXraw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451271746664.1770714269909; Tue, 17 Dec 2024 08:01:11 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNZzh-0004Dk-Hj; Tue, 17 Dec 2024 11:00:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzd-0004CG-Aj for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzY-00024n-Kx for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:17 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-627-UeNu2OibNdupKLdV7cSmOQ-1; Tue, 17 Dec 2024 11:00:10 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 67C671956052; Tue, 17 Dec 2024 16:00:08 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6A5F730044C1; Tue, 17 Dec 2024 16:00:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fhKdokIULRGPGWSO9LdGxnmdKKDtG0WAE0bknPnTpHU=; b=ZSx7Qn2LoKjwMiSXzLjE1/jGS6H5xOPSDmA5aT6LiC9bru94cQzO7Paaf1Z7Qn+ae3X5cm Ek4b+OlontE9wQM0ZevJSBIy5TuBW328J+CvqXyELqZNA4CtCEAxdrVOa9zjujTmGMZi+A pxoDq2t5qA1DrvNPGSsDmhyG7GRIOS0= X-MC-Unique: UeNu2OibNdupKLdV7cSmOQ-1 X-Mimecast-MFC-AGG-ID: UeNu2OibNdupKLdV7cSmOQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Richard Henderson Subject: [PATCH v3 04/32] tests/functional: remove duplicated 'which' function impl Date: Tue, 17 Dec 2024 15:59:25 +0000 Message-ID: <20241217155953.3950506-5-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451274243116600 Put the 'which' function into shared code. Reviewed-by: Richard Henderson Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/cmd.py | 10 ++++++++++ tests/functional/test_acpi_bits.py | 13 +------------ tests/functional/test_ppc64_hv.py | 13 +------------ 4 files changed, 13 insertions(+), 25 deletions(-) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu= _test/__init__.py index 67f87be9c4..8fddddbe67 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -10,6 +10,6 @@ from .config import BUILD_DIR from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \ interrupt_interactive_console_until_pattern, wait_for_console_pattern,= \ - exec_command, exec_command_and_wait_for_pattern, get_qemu_img + exec_command, exec_command_and_wait_for_pattern, get_qemu_img, which from .testcase import QemuBaseTest, QemuUserTest, QemuSystemTest from .linuxkernel import LinuxKernelTest diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test= /cmd.py index 11c8334a7c..4106f1ee7c 100644 --- a/tests/functional/qemu_test/cmd.py +++ b/tests/functional/qemu_test/cmd.py @@ -18,6 +18,16 @@ =20 from .config import BUILD_DIR =20 +def which(tool): + """ looks up the full path for @tool, returns None if not found + or if @tool does not have executable permissions. + """ + paths=3Dos.getenv('PATH') + for p in paths.split(os.path.pathsep): + p =3D os.path.join(p, tool) + if os.path.exists(p) and os.access(p, os.X_OK): + return p + return None =20 def has_cmd(name, args=3DNone): """ diff --git a/tests/functional/test_acpi_bits.py b/tests/functional/test_acp= i_bits.py index 3df9562394..1e6d082ecb 100755 --- a/tests/functional/test_acpi_bits.py +++ b/tests/functional/test_acpi_bits.py @@ -47,7 +47,7 @@ ) from qemu.machine import QEMUMachine from unittest import skipIf -from qemu_test import QemuSystemTest, Asset +from qemu_test import QemuSystemTest, Asset, which =20 deps =3D ["xorriso", "mformat"] # dependent tools needed in the test setup= /box. supported_platforms =3D ['x86_64'] # supported test platforms. @@ -55,17 +55,6 @@ # default timeout of 120 secs is sometimes not enough for bits test. BITS_TIMEOUT =3D 200 =20 -def which(tool): - """ looks up the full path for @tool, returns None if not found - or if @tool does not have executable permissions. - """ - paths=3Dos.getenv('PATH') - for p in paths.split(os.path.pathsep): - p =3D os.path.join(p, tool) - if os.path.exists(p) and os.access(p, os.X_OK): - return p - return None - def missing_deps(): """ returns True if any of the test dependent tools are absent. """ diff --git a/tests/functional/test_ppc64_hv.py b/tests/functional/test_ppc6= 4_hv.py index d97b62e364..7c6f8234f5 100755 --- a/tests/functional/test_ppc64_hv.py +++ b/tests/functional/test_ppc64_hv.py @@ -11,7 +11,7 @@ =20 from unittest import skipIf, skipUnless from qemu_test import QemuSystemTest, Asset -from qemu_test import wait_for_console_pattern, exec_command +from qemu_test import wait_for_console_pattern, exec_command, which import os import time import subprocess @@ -19,17 +19,6 @@ =20 deps =3D ["xorriso"] # dependent tools needed in the test setup/box. =20 -def which(tool): - """ looks up the full path for @tool, returns None if not found - or if @tool does not have executable permissions. - """ - paths=3Dos.getenv('PATH') - for p in paths.split(os.path.pathsep): - p =3D os.path.join(p, tool) - if os.path.exists(p) and os.access(p, os.X_OK): - return p - return None - def missing_deps(): """ returns True if any of the test dependent tools are absent. """ --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451251; cv=none; d=zohomail.com; s=zohoarc; b=mRWmTEW1wOLelK04xuMDdBbTT8wNwawTJKbZ2h3IKZage5GL0IOxVDb0YMX+bKqaAr7DXQUyBK41yaiaai1xY9keYwKjLwxHxfCZQJC3gSmrLQ4Ao21Kpl0yeJR3Rami1pMCxHxGbsJGZo6hzV+KgN/onrp5wIG9MB39jZbd4Lc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451251; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6HpAXMLdP0SFP2AlJnA3XK425MQMPoGudZ7pA0RiiIs=; b=Z0rcoy9bXaVUT8wqAm6+28f01FLGgvtZ5RcHbm6n/mQfB01Smf9rqiWqVhQkM1H7YN6ekoc2n4UH1MyGx89/kXlKu+QFN5gBLyMLtJfX/9yksoJrYdvFieraKsKxW2e5r1vETPHjbtPj0c3PHuIlT7vYRCnnluzCvy5L9zFQwmY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451251868273.4807536896709; Tue, 17 Dec 2024 08:00:51 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNZzk-0004Ek-Lm; Tue, 17 Dec 2024 11:00:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzh-0004E4-62 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzd-00025U-Hw for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:19 -0500 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-428-SwSJbPauO0mwc0tLy6Ws-A-1; Tue, 17 Dec 2024 11:00:12 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8448F19560A3; Tue, 17 Dec 2024 16:00:11 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F2F4B300F9B7; Tue, 17 Dec 2024 16:00:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6HpAXMLdP0SFP2AlJnA3XK425MQMPoGudZ7pA0RiiIs=; b=ZWGyJdzys4r9VCMdrJoKMLH5g3Jy7u0FGHTILHK9kIY48A2byLOmL3m6PJMPEs+R76iT/B 7vZNPKoOuE/DJ6IEXayJb8Ul4iPQTBGKRt6PVn0k9lgJXxwyc19CptbmOjYtoFQ+f+qDEX jsFQuR9NH6aPJl4XQC0daZ4GPKYUjlU= X-MC-Unique: SwSJbPauO0mwc0tLy6Ws-A-1 X-Mimecast-MFC-AGG-ID: SwSJbPauO0mwc0tLy6Ws-A From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth , Richard Henderson Subject: [PATCH v3 05/32] tests/functional: simplify 'which' implementation Date: Tue, 17 Dec 2024 15:59:26 +0000 Message-ID: <20241217155953.3950506-6-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451254061116600 The 'access' check implies the file exists. Reviewed-by: Thomas Huth Reviewed-by: Richard Henderson Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/cmd.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test= /cmd.py index 4106f1ee7c..600e0509db 100644 --- a/tests/functional/qemu_test/cmd.py +++ b/tests/functional/qemu_test/cmd.py @@ -25,7 +25,7 @@ def which(tool): paths=3Dos.getenv('PATH') for p in paths.split(os.path.pathsep): p =3D os.path.join(p, tool) - if os.path.exists(p) and os.access(p, os.X_OK): + if os.access(p, os.X_OK): return p return None =20 --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451289; cv=none; d=zohomail.com; s=zohoarc; b=YZBLxqYqchvb7Ind6f2m37qV490un0CW0n+rgUmsOGIVuf2aIIpawVaP1bm05tNgffSzVWb8Ea2khOhpx3Bx7mAcLbrC8Y3AbFMSMTKAUfZOZlFy9/KTNmC9x/p9DyFMnA9MAfcY6tYot5t1k8W52hxq192jorqYU63AbZl7q1E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451289; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=68U+RfBlTpidD3nEgFbqAFrzUZ4u7M4ylio/Pj7h03Q=; b=WURfLIdiyEu0LrUp0SGwGvOumJMvrz/xPXFZZYfQgwDQ3Sr3Al8d7v5UkNvIShwVz014rnCgD2bL8L6TjfJfZOKaAVaBY9AFKhgS12cHeo/y+mjGAwqRFhSJJGjuOSPG9qAGGxZIm7XAlx8vGrv6uR1qjlcLHvBvaDrX5+XPcmc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 173445128931181.56579633466379; Tue, 17 Dec 2024 08:01:29 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNZzp-0004Fr-0Z; Tue, 17 Dec 2024 11:00:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzl-0004Eo-Cb for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzh-00025Z-2E for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:24 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-648-s9E-c5-fOSu7PbxNG89yfQ-1; Tue, 17 Dec 2024 11:00:15 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 18FC6195607A; Tue, 17 Dec 2024 16:00:14 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EBACA30044C1; Tue, 17 Dec 2024 16:00:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451216; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=68U+RfBlTpidD3nEgFbqAFrzUZ4u7M4ylio/Pj7h03Q=; b=PiNym1f6y8zLVbp5sAaRKuJ267Iyv1BWId5LivrthBBwIK0dVwUmCOVYcpINh7skyT5txG JRTHL9Aa0mx7BXp5K4OLcEdGpHzyKULu+RnzpCGM7uJQvQh7jWZJzoZl4ob5c0oK4dyjL5 RpOtd0HkMbV96se1uy4TzvG3iNAqGfQ= X-MC-Unique: s9E-c5-fOSu7PbxNG89yfQ-1 X-Mimecast-MFC-AGG-ID: s9E-c5-fOSu7PbxNG89yfQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v3 06/32] tests/functional: drop 'tesseract_available' helper Date: Tue, 17 Dec 2024 15:59:27 +0000 Message-ID: <20241217155953.3950506-7-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451290352116600 Platforms we target have new enough tesseract that it suffices to merely check if the binary exists. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Richard Henderson --- tests/functional/qemu_test/tesseract.py | 12 +----------- tests/functional/test_m68k_nextcube.py | 8 +++----- 2 files changed, 4 insertions(+), 16 deletions(-) diff --git a/tests/functional/qemu_test/tesseract.py b/tests/functional/qem= u_test/tesseract.py index ef1833139d..1b7818090a 100644 --- a/tests/functional/qemu_test/tesseract.py +++ b/tests/functional/qemu_test/tesseract.py @@ -7,17 +7,7 @@ =20 import logging =20 -from . import has_cmd, run_cmd - -def tesseract_available(expected_version): - (has_tesseract, _) =3D has_cmd('tesseract') - if not has_tesseract: - return False - (stdout, stderr, ret) =3D run_cmd([ 'tesseract', '--version']) - if ret: - return False - version =3D stdout.split()[1] - return int(version.split('.')[0]) >=3D expected_version +from . import run_cmd =20 def tesseract_ocr(image_path, tesseract_args=3D''): console_logger =3D logging.getLogger('console') diff --git a/tests/functional/test_m68k_nextcube.py b/tests/functional/test= _m68k_nextcube.py index 0124622c40..e6e8d4fd3f 100755 --- a/tests/functional/test_m68k_nextcube.py +++ b/tests/functional/test_m68k_nextcube.py @@ -13,7 +13,8 @@ from qemu_test import QemuSystemTest, Asset from unittest import skipUnless =20 -from qemu_test.tesseract import tesseract_available, tesseract_ocr +from qemu_test import has_cmd +from qemu_test.tesseract import tesseract_ocr =20 PIL_AVAILABLE =3D True try: @@ -53,10 +54,7 @@ def test_bootrom_framebuffer_size(self): self.assertEqual(width, 1120) self.assertEqual(height, 832) =20 - # Tesseract 4 adds a new OCR engine based on LSTM neural networks. The - # new version is faster and more accurate than version 3. The drawback= is - # that it is still alpha-level software. - @skipUnless(tesseract_available(4), 'tesseract OCR tool not available') + @skipUnless(*has_cmd('tesseract')) def test_bootrom_framebuffer_ocr_with_tesseract(self): self.set_machine('next-cube') screenshot_path =3D os.path.join(self.workdir, "dump.ppm") --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451288; cv=none; d=zohomail.com; s=zohoarc; b=Mv4RP7W+Z99/Hkggu4krLv8AuYdsmS0ZTx2AbBc1gBmlHek6rafiX7EHTWXj6oh59bLaQ/Xgkaxg2KlmKMlpIVRncbEaXG0OxXar6Z6BIMTbPwInYG7oknSXUF+YRy7+wYa82Yvs9tBDxy/0+HBcTfZapemrwMhYut/y2VFwSXM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451288; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=E2guSRHE88ado12bvC1ekUNYwe9ysPpIG0XqHd+TSjw=; b=b69MS7Q1AQJCokZ/aL+p3mILXiohy6F5XzWW9BWbL/b2r3jR2oACZTK2bS9WU21OHECzU9oCQPTLIA3PxD2Mkr7Hjbp35rxzL53AA70x5A6PLrxsVMU1qdA/xWoNNm2gX1T3wt865wcQ4DHXNzpw/Z1FQF8qK8evWL+bRT043Hw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451288878583.2711386711193; Tue, 17 Dec 2024 08:01:28 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNZzw-0004G9-Mu; Tue, 17 Dec 2024 11:00:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzn-0004FY-7A for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzl-00026q-Fu for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:26 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-92-1ocKsE63PoKgnjVkNlqMcA-1; Tue, 17 Dec 2024 11:00:18 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1A62B1956089; Tue, 17 Dec 2024 16:00:17 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 97B63300F9B7; Tue, 17 Dec 2024 16:00:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451223; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E2guSRHE88ado12bvC1ekUNYwe9ysPpIG0XqHd+TSjw=; b=gYbc7F+Me+DpTgKpQF2lhUpXD51tcE9jW5W6v9VTeW8k+6Ndz9Gxq0Tz1McXlndkKg5Mpy pIAdXq1YYxkarSBjvgdHLARn9WGbYKYHzQQJfJF/mkmGDeB0dseWrKOsyzsHutFxVcAv04 nT9Q0OORaaM16A8jVG26r94+hbe185o= X-MC-Unique: 1ocKsE63PoKgnjVkNlqMcA-1 X-Mimecast-MFC-AGG-ID: 1ocKsE63PoKgnjVkNlqMcA From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 07/32] tests/functional: introduce some helpful decorators Date: Tue, 17 Dec 2024 15:59:28 +0000 Message-ID: <20241217155953.3950506-8-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451290417116600 Reduce repeated boilerplate with some helper decorators: @skipIfNotPlatform("x86_64", "aarch64") =3D> Skip unless the build host platform matches @skipIfMissingCommands("mkisofs", "losetup") =3D> Skips unless all listed commands are found in $PATH @skipIfMissingImports("numpy", "cv2") =3D> Skips unless all listed modules can be imported @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/NNN") =3D> Skips unless env var requests flaky tests with the reason documented in the referenced gitlab bug @skipBigData =3D> Skips unless env var permits tests creating big data files @skipUntrustedTest =3D> Skips unless env var permits tests which are potentially dangerous to the host Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/__init__.py | 3 + tests/functional/qemu_test/decorators.py | 107 +++++++++++++++++++++++ 2 files changed, 110 insertions(+) create mode 100644 tests/functional/qemu_test/decorators.py diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu= _test/__init__.py index 8fddddbe67..7dee3522f2 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -13,3 +13,6 @@ exec_command, exec_command_and_wait_for_pattern, get_qemu_img, which from .testcase import QemuBaseTest, QemuUserTest, QemuSystemTest from .linuxkernel import LinuxKernelTest +from .decorators import skipIfMissingCommands, skipIfNotMachine, \ + skipFlakyTest, skipUntrustedTest, skipBigDataTest, \ + skipIfMissingImports diff --git a/tests/functional/qemu_test/decorators.py b/tests/functional/qe= mu_test/decorators.py new file mode 100644 index 0000000000..df088bc090 --- /dev/null +++ b/tests/functional/qemu_test/decorators.py @@ -0,0 +1,107 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Decorators useful in functional tests + +import os +import platform +from unittest import skipUnless + +from .cmd import which + +''' +Decorator to skip execution of a test if the list +of command binaries is not available in $PATH. +Example: + + @skipIfMissingCommands("mkisofs", "losetup") +''' +def skipIfMissingCommands(*args): + def has_cmds(cmdlist): + for cmd in cmdlist: + if not which(cmd): + return False + return True + + return skipUnless(lambda: has_cmds(args), + 'required command(s) "%s" not installed' % + ", ".join(args)) + +''' +Decorator to skip execution of a test if the current +host machine does not match one of the permitted +machines. +Example + + @skipIfNotMachine("x86_64", "aarch64") +''' +def skipIfNotMachine(*args): + return skipUnless(lambda: platform.machine() in args, + 'not running on one of the required machine(s) "%s= "' % + ", ".join(args)) + +''' +Decorator to skip execution of flaky tests, unless +the $QEMU_TEST_FLAKY_TESTS environment variable is set. +A bug URL must be provided that documents the observed +failure behaviour, so it can be tracked & re-evaluated +in future. + +Historical tests may be providing "None" as the bug_url +but this should not be done for new test. + +Example: + + @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/NNN") +''' +def skipFlakyTest(bug_url): + if bug_url is None: + bug_url =3D "FIXME: reproduce flaky test and file bug report or re= move" + return skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), + f'Test is unstable: {bug_url}') + +''' +Decorator to skip execution of tests which are likely +to execute untrusted commands on the host, or commands +which process untrusted code, unless the +$QEMU_TEST_ALLOW_UNTRUSTED_CODE env var is set. +Example: + + @skipUntrustedTest() +''' +def skipUntrustedTest(): + return skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), + 'Test runs untrusted code / processes untrusted data= ') + +''' +Decorator to skip execution of tests which need large +data storage (over around 500MB-1GB mark) on the host, +unless the $QEMU_TEST_ALLOW_LARGE_STORAGE environment +variable is set + +Example: + + @skipBigDataTest() +''' +def skipBigDataTest(): + return skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), + 'Test requires large host storage space') + +''' +Decorator to skip execution of a test if the list +of python imports is not available. +Example: + + @skipIfMissingImports("numpy", "cv2") +''' +def skipIfMissingImports(*args): + def has_imports(importlist): + for impname in importlist: + try: + import impname + except ImportError: + return False + return True + + return skipUnless(lambda: has_imports(args), + 'required import(s) "%s" not installed' % + ", ".join(args)) --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451810; cv=none; d=zohomail.com; s=zohoarc; b=FJuhs8GlFpPL7sO/lpxbwEKjWhJa8s/9/La98X2qcjzklN3b4O/FUzsNdERMV7DIPCZZguY0izgV6UAiOgI1HBmOU8rSXPyVXAliRXjsZVcOd8qmEBD4AOEUeFnoBpxjJoqQOMEpJjV4iOJeBXIpfWj3SiC0yEShJlD7A4a8bVM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451810; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=cHGZ45//i/AB/+rLnMXUK685IrJCFCJA/gUWhS7JhRE=; b=GjqOGFMsy8YJxgkf/xCiNVINzjR1Ib2jN/xGMve+ZZw6T+VikaiBbGQo0PeBGHfSAKi0v/vmlU7W6okwd+LM/3ueJjE6qp1VFucgX60tV1T/0CZwFkLyHU5CMuw+1KK8/4pNmI9Sh2/i9+Cj4XZq4IWea/KS//m9oFb9qXiIReE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451810374902.4577159163243; Tue, 17 Dec 2024 08:10:10 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa03-0004KZ-BB; Tue, 17 Dec 2024 11:00:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzp-0004G8-9V for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzl-00026y-FY for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:28 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-460-opf9Q9i8NPunC_VtOwiLig-1; Tue, 17 Dec 2024 11:00:21 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 44E101956050; Tue, 17 Dec 2024 16:00:20 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9067A30044C1; Tue, 17 Dec 2024 16:00:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cHGZ45//i/AB/+rLnMXUK685IrJCFCJA/gUWhS7JhRE=; b=DRHHolD0fAc1BjN2JSW9LXoF8ZPTedCQhvWNaRoIhat5a10wEOzhAi9diOcNo4mYWAucU4 9mEKI/cyGudWQXu8HeyWUAAoHTOlxRK0D/U/kH39wRBg8sYQ5QlUGbiphVjvV6FU6Sr9P/ Pf1cUc595qun+Zeo0Ew/k5B9t2ikCPo= X-MC-Unique: opf9Q9i8NPunC_VtOwiLig-1 X-Mimecast-MFC-AGG-ID: opf9Q9i8NPunC_VtOwiLig From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 08/32] tests/functional: switch to new test skip decorators Date: Tue, 17 Dec 2024 15:59:29 +0000 Message-ID: <20241217155953.3950506-9-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451812398116600 This ensures consistency of behaviour across all the tests, and requires that we provide gitlab bug links when marking a test to be skipped due to unreliability. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/test_acpi_bits.py | 25 +++-------------- tests/functional/test_arm_aspeed_ast2600.py | 7 +++-- tests/functional/test_arm_bflt.py | 7 +++-- tests/functional/test_arm_bpim2u.py | 5 ++-- tests/functional/test_arm_cubieboard.py | 5 ++-- tests/functional/test_arm_integratorcp.py | 25 +++++------------ tests/functional/test_arm_orangepi.py | 8 +++--- tests/functional/test_linux_initrd.py | 7 +++-- tests/functional/test_m68k_nextcube.py | 15 +++-------- tests/functional/test_mips64el_fuloong2e.py | 4 +-- tests/functional/test_mips64el_loongson3v.py | 8 +++--- tests/functional/test_mips64el_malta.py | 28 +++++++------------- tests/functional/test_ppc64_hv.py | 18 +++---------- tests/functional/test_ppc_40p.py | 7 ++--- tests/functional/test_rx_gdbsim.py | 6 ++--- tests/functional/test_sh4_r2d.py | 8 +++--- 16 files changed, 61 insertions(+), 122 deletions(-) diff --git a/tests/functional/test_acpi_bits.py b/tests/functional/test_acp= i_bits.py index 1e6d082ecb..8763ea0822 100755 --- a/tests/functional/test_acpi_bits.py +++ b/tests/functional/test_acpi_bits.py @@ -32,7 +32,6 @@ """ =20 import os -import platform import re import shutil import subprocess @@ -46,28 +45,13 @@ Sequence, ) from qemu.machine import QEMUMachine -from unittest import skipIf -from qemu_test import QemuSystemTest, Asset, which +from qemu_test import (QemuSystemTest, Asset, skipIfMissingCommands, + skipIfNotMachine) =20 -deps =3D ["xorriso", "mformat"] # dependent tools needed in the test setup= /box. -supported_platforms =3D ['x86_64'] # supported test platforms. =20 # default timeout of 120 secs is sometimes not enough for bits test. BITS_TIMEOUT =3D 200 =20 -def missing_deps(): - """ returns True if any of the test dependent tools are absent. - """ - for dep in deps: - if which(dep) is None: - return True - return False - -def supported_platform(): - """ checks if the test is running on a supported platform. - """ - return platform.machine() in supported_platforms - class QEMUBitsMachine(QEMUMachine): # pylint: disable=3Dtoo-few-public-met= hods """ A QEMU VM, with isa-debugcon enabled and bits iso passed @@ -110,9 +94,8 @@ def base_args(self): """return the base argument to QEMU binary""" return self._base_args =20 -@skipIf(not supported_platform() or missing_deps(), - 'unsupported platform or dependencies (%s) not installed' \ - % ','.join(deps)) +@skipIfMissingCommands("xorriso", "mformat") +@skipIfNotMachine("x86_64") class AcpiBitsTest(QemuSystemTest): #pylint: disable=3Dtoo-many-instance-a= ttributes """ ACPI and SMBIOS tests using biosbits. diff --git a/tests/functional/test_arm_aspeed_ast2600.py b/tests/functional= /test_arm_aspeed_ast2600.py index 74d025e0fc..9c749f96b4 100755 --- a/tests/functional/test_arm_aspeed_ast2600.py +++ b/tests/functional/test_arm_aspeed_ast2600.py @@ -11,10 +11,9 @@ =20 from qemu_test import Asset from aspeed import AspeedTest -from qemu_test import exec_command_and_wait_for_pattern -from qemu_test import has_cmd +from qemu_test import exec_command_and_wait_for_pattern, skipIfMissingComm= ands from qemu_test.utils import archive_extract -from unittest import skipUnless + =20 class AST2600Machine(AspeedTest): =20 @@ -68,7 +67,7 @@ def test_arm_ast2600_evb_buildroot(self): 'images/ast2600-evb/buildroot-2023.02-tpm/flash.img'), 'a46009ae8a5403a0826d607215e731a8c68d27c14c41e55331706b8f9c7bd997') =20 - @skipUnless(*has_cmd('swtpm')) + @skipIfMissingCommands('swtpm') def test_arm_ast2600_evb_buildroot_tpm(self): self.set_machine('ast2600-evb') =20 diff --git a/tests/functional/test_arm_bflt.py b/tests/functional/test_arm_= bflt.py index 281925d11a..9095b08539 100755 --- a/tests/functional/test_arm_bflt.py +++ b/tests/functional/test_arm_bflt.py @@ -10,9 +10,8 @@ import bz2 =20 from qemu_test import QemuUserTest, Asset -from qemu_test import has_cmd +from qemu_test import skipIfMissingCommands, skipUntrustedTest from qemu_test.utils import cpio_extract -from unittest import skipUnless =20 =20 class LoadBFLT(QemuUserTest): @@ -21,8 +20,8 @@ class LoadBFLT(QemuUserTest): ('https://elinux.org/images/5/51/Stm32_mini_rootfs.cpio.bz2'), 'eefb788e4980c9e8d6c9d60ce7d15d4da6bf4fbc6a80f487673824600d5ba9cc= ') =20 - @skipUnless(*has_cmd('cpio')) - @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted co= de') + @skipIfMissingCommands('cpio') + @skipUntrustedTest() def test_stm32(self): # See https://elinux.org/STM32#User_Space rootfs_path_bz2 =3D self.ASSET_ROOTFS.fetch() diff --git a/tests/functional/test_arm_bpim2u.py b/tests/functional/test_ar= m_bpim2u.py index c9cf43c147..fcd111f59d 100755 --- a/tests/functional/test_arm_bpim2u.py +++ b/tests/functional/test_arm_bpim2u.py @@ -9,9 +9,10 @@ =20 from qemu_test import LinuxKernelTest, exec_command_and_wait_for_pattern from qemu_test import Asset, interrupt_interactive_console_until_pattern +from qemu_test import skipBigDataTest from qemu_test.utils import gzip_uncompress, lzma_uncompress from qemu_test.utils import image_pow2ceil_expand -from unittest import skipUnless + =20 class BananaPiMachine(LinuxKernelTest): =20 @@ -143,7 +144,7 @@ def test_arm_bpim2u_gmac(self): os.remove(dtb_path) os.remove(rootfs_path) =20 - @skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), 'storage limit= ed') + @skipBigDataTest() def test_arm_bpim2u_openwrt_22_03_3(self): self.set_machine('bpim2u') # This test download a 8.9 MiB compressed image and expand it diff --git a/tests/functional/test_arm_cubieboard.py b/tests/functional/tes= t_arm_cubieboard.py index 9e42b72060..fdbd52a33c 100755 --- a/tests/functional/test_arm_cubieboard.py +++ b/tests/functional/test_arm_cubieboard.py @@ -8,8 +8,9 @@ =20 from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pa= ttern from qemu_test import interrupt_interactive_console_until_pattern +from qemu_test import skipBigDataTest from qemu_test.utils import gzip_uncompress, image_pow2ceil_expand -from unittest import skipUnless + =20 class CubieboardMachine(LinuxKernelTest): =20 @@ -105,7 +106,7 @@ def test_arm_cubieboard_sata(self): # Wait for VM to shut down gracefully self.vm.wait() =20 - @skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited= ') + @skipBigDataTest() def test_arm_cubieboard_openwrt_22_03_2(self): # This test download a 7.5 MiB compressed image and expand it # to 126 MiB. diff --git a/tests/functional/test_arm_integratorcp.py b/tests/functional/t= est_arm_integratorcp.py index 0fe083f661..54fa366ad4 100755 --- a/tests/functional/test_arm_integratorcp.py +++ b/tests/functional/test_arm_integratorcp.py @@ -17,20 +17,7 @@ =20 from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from unittest import skipUnless - - -NUMPY_AVAILABLE =3D True -try: - import numpy as np -except ImportError: - NUMPY_AVAILABLE =3D False - -CV2_AVAILABLE =3D True -try: - import cv2 -except ImportError: - CV2_AVAILABLE =3D False +from qemu_test import skipIfMissingImports, skipUntrustedTest =20 =20 class IntegratorMachine(QemuSystemTest): @@ -63,7 +50,7 @@ def boot_integratorcp(self): '-append', 'printk.time=3D0 console=3DttyAMA0') self.vm.launch() =20 - @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted co= de') + @skipUntrustedTest() def test_integratorcp_console(self): """ Boots the Linux kernel and checks that the console is operational @@ -71,13 +58,15 @@ def test_integratorcp_console(self): self.boot_integratorcp() wait_for_console_pattern(self, 'Log in as root') =20 - @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed') - @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed') - @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted co= de') + @skipIfMissingImports("numpy", "cv2") + @skipUntrustedTest() def test_framebuffer_tux_logo(self): """ Boot Linux and verify the Tux logo is displayed on the framebuffer. """ + import numpy as np + import cv2 + screendump_path =3D os.path.join(self.workdir, "screendump.pbm") tuxlogo_path =3D self.ASSET_TUXLOGO.fetch() =20 diff --git a/tests/functional/test_arm_orangepi.py b/tests/functional/test_= arm_orangepi.py index a872305e93..b113adfe61 100755 --- a/tests/functional/test_arm_orangepi.py +++ b/tests/functional/test_arm_orangepi.py @@ -10,10 +10,10 @@ =20 from qemu_test import LinuxKernelTest, exec_command_and_wait_for_pattern from qemu_test import Asset, interrupt_interactive_console_until_pattern -from qemu_test import wait_for_console_pattern +from qemu_test import wait_for_console_pattern, skipBigDataTest from qemu_test.utils import gzip_uncompress, lzma_uncompress from qemu_test.utils import image_pow2ceil_expand -from unittest import skipUnless + =20 class BananaPiMachine(LinuxKernelTest): =20 @@ -149,7 +149,7 @@ def test_arm_orangepi_sd(self): os.remove(dtb_path) os.remove(rootfs_path) =20 - @skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), 'storage limit= ed') + @skipBigDataTest() def test_arm_orangepi_armbian(self): self.set_machine('orangepi-pc') # This test download a 275 MiB compressed image and expand it @@ -185,7 +185,7 @@ def test_arm_orangepi_armbian(self): 'to ') self.wait_for_console_pattern('Starting Load Kernel Modules...') =20 - @skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), 'storage limit= ed') + @skipBigDataTest() def test_arm_orangepi_uboot_netbsd9(self): self.set_machine('orangepi-pc') # This test download a 304MB compressed image and expand it to 2GB diff --git a/tests/functional/test_linux_initrd.py b/tests/functional/test_= linux_initrd.py index c71a59d4c9..2207f83fbf 100755 --- a/tests/functional/test_linux_initrd.py +++ b/tests/functional/test_linux_initrd.py @@ -10,12 +10,10 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import os import logging import tempfile =20 -from qemu_test import QemuSystemTest, Asset -from unittest import skipUnless +from qemu_test import QemuSystemTest, Asset, skipFlakyTest =20 =20 class LinuxInitrd(QemuSystemTest): @@ -60,7 +58,8 @@ def test_with_2gib_file_should_exit_error_msg_with_linux_= v3_6(self): max_size + 1) self.assertRegex(self.vm.get_log(), expected_msg) =20 - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + # XXX file tracking bug + @skipFlakyTest(bug_url=3DNone) def test_with_2gib_file_should_work_with_linux_v4_16(self): """ QEMU has supported up to 4 GiB initrd for recent kernel diff --git a/tests/functional/test_m68k_nextcube.py b/tests/functional/test= _m68k_nextcube.py index e6e8d4fd3f..25a17d4794 100755 --- a/tests/functional/test_m68k_nextcube.py +++ b/tests/functional/test_m68k_nextcube.py @@ -11,17 +11,9 @@ import time =20 from qemu_test import QemuSystemTest, Asset -from unittest import skipUnless - -from qemu_test import has_cmd +from qemu_test import skipIfMissingImports, skipIfMissingCommands from qemu_test.tesseract import tesseract_ocr =20 -PIL_AVAILABLE =3D True -try: - from PIL import Image -except ImportError: - PIL_AVAILABLE =3D False - =20 class NextCubeMachine(QemuSystemTest): =20 @@ -44,17 +36,18 @@ def check_bootrom_framebuffer(self, screenshot_path): self.vm.cmd('human-monitor-command', command_line=3D'screendump %s' % screenshot_path) =20 - @skipUnless(PIL_AVAILABLE, 'Python PIL not installed') + @skipIfMissingImports("PIL") def test_bootrom_framebuffer_size(self): self.set_machine('next-cube') screenshot_path =3D os.path.join(self.workdir, "dump.ppm") self.check_bootrom_framebuffer(screenshot_path) =20 + from PIL import Image width, height =3D Image.open(screenshot_path).size self.assertEqual(width, 1120) self.assertEqual(height, 832) =20 - @skipUnless(*has_cmd('tesseract')) + @skipIfMissingCommands('tesseract') def test_bootrom_framebuffer_ocr_with_tesseract(self): self.set_machine('next-cube') screenshot_path =3D os.path.join(self.workdir, "dump.ppm") diff --git a/tests/functional/test_mips64el_fuloong2e.py b/tests/functional= /test_mips64el_fuloong2e.py index a32d5f9d08..531d16d8f6 100755 --- a/tests/functional/test_mips64el_fuloong2e.py +++ b/tests/functional/test_mips64el_fuloong2e.py @@ -13,7 +13,7 @@ import subprocess =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test import wait_for_console_pattern +from qemu_test import wait_for_console_pattern, skipUntrustedTest from unittest import skipUnless =20 class MipsFuloong2e(LinuxKernelTest): @@ -39,7 +39,7 @@ def test_linux_kernel_3_16(self): console_pattern =3D 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) =20 - @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted co= de') + @skipUntrustedTest() @skipUnless(os.getenv('RESCUE_YL_PATH'), 'RESCUE_YL_PATH not available= ') def test_linux_kernel_2_6_27_isa_serial(self): # Recovery system for the Yeeloong laptop diff --git a/tests/functional/test_mips64el_loongson3v.py b/tests/functiona= l/test_mips64el_loongson3v.py index e57ec5499e..f85371e50c 100755 --- a/tests/functional/test_mips64el_loongson3v.py +++ b/tests/functional/test_mips64el_loongson3v.py @@ -9,11 +9,9 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - -from unittest import skipUnless from qemu_test import QemuSystemTest, Asset -from qemu_test import wait_for_console_pattern +from qemu_test import wait_for_console_pattern, skipUntrustedTest + =20 class MipsLoongson3v(QemuSystemTest): timeout =3D 60 @@ -23,7 +21,7 @@ class MipsLoongson3v(QemuSystemTest): 'releases/download/20210112/pmon-3avirt.bin'), 'fcdf6bb2cb7885a4a62f31fcb0d5e368bac7b6cea28f40c6dfa678af22fea20a') =20 - @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted co= de') + @skipUntrustedTest() def test_pmon_serial_console(self): self.set_machine('loongson3-virt') =20 diff --git a/tests/functional/test_mips64el_malta.py b/tests/functional/tes= t_mips64el_malta.py index 6d1195d362..52283e2dbd 100755 --- a/tests/functional/test_mips64el_malta.py +++ b/tests/functional/test_mips64el_malta.py @@ -14,20 +14,8 @@ =20 from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern +from qemu_test import skipIfMissingImports, skipFlakyTest, skipUntrustedTe= st from qemu_test.utils import gzip_uncompress -from unittest import skipUnless - -NUMPY_AVAILABLE =3D True -try: - import numpy as np -except ImportError: - NUMPY_AVAILABLE =3D False - -CV2_AVAILABLE =3D True -try: - import cv2 -except ImportError: - CV2_AVAILABLE =3D False =20 =20 class MaltaMachineConsole(LinuxKernelTest): @@ -76,7 +64,7 @@ def test_mips64el_malta(self): 'rootfs.mipsel64r1.cpio.gz'), '75ba10cd35fb44e32948eeb26974f061b703c81c4ba2fab1ebcacf1d1bec3b61') =20 - @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted co= de') + @skipUntrustedTest() def test_mips64el_malta_5KEc_cpio(self): kernel_path =3D self.ASSET_KERNEL_3_19_3.fetch() initrd_path_gz =3D self.ASSET_CPIO_R1.fetch() @@ -106,8 +94,7 @@ def test_mips64el_malta_5KEc_cpio(self): self.vm.wait() =20 =20 -@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed') -@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed') +@skipIfMissingImports('numpy', 'cv2') class MaltaMachineFramebuffer(LinuxKernelTest): =20 timeout =3D 30 @@ -126,6 +113,10 @@ def do_test_i6400_framebuffer_logo(self, cpu_cores_cou= nt): """ Boot Linux kernel and check Tux logo is displayed on the framebuff= er. """ + + import numpy as np + import cv2 + screendump_path =3D os.path.join(self.workdir, 'screendump.pbm') =20 kernel_path_gz =3D self.ASSET_KERNEL_4_7_0.fetch() @@ -171,11 +162,12 @@ def do_test_i6400_framebuffer_logo(self, cpu_cores_co= unt): def test_mips_malta_i6400_framebuffer_logo_1core(self): self.do_test_i6400_framebuffer_logo(1) =20 - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + # XXX file tracking bug + @skipFlakyTest(bug_url=3DNone) def test_mips_malta_i6400_framebuffer_logo_7cores(self): self.do_test_i6400_framebuffer_logo(7) =20 - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + @skipFlakyTest(bug_url=3DNone) def test_mips_malta_i6400_framebuffer_logo_8cores(self): self.do_test_i6400_framebuffer_logo(8) =20 diff --git a/tests/functional/test_ppc64_hv.py b/tests/functional/test_ppc6= 4_hv.py index 7c6f8234f5..88f0f99f24 100755 --- a/tests/functional/test_ppc64_hv.py +++ b/tests/functional/test_ppc64_hv.py @@ -9,24 +9,14 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -from unittest import skipIf, skipUnless from qemu_test import QemuSystemTest, Asset -from qemu_test import wait_for_console_pattern, exec_command, which +from qemu_test import wait_for_console_pattern, exec_command +from qemu_test import skipIfMissingCommands, skipBigDataTest import os import time import subprocess from datetime import datetime =20 -deps =3D ["xorriso"] # dependent tools needed in the test setup/box. - -def missing_deps(): - """ returns True if any of the test dependent tools are absent. - """ - for dep in deps: - if which(dep) is None: - return True - return False - # Alpine is a light weight distro that supports QEMU. These tests boot # that on the machine then run a QEMU guest inside it in KVM mode, # that runs the same Alpine distro image. @@ -34,8 +24,8 @@ def missing_deps(): # large download, but it may be more polite to create qcow2 image with # QEMU already installed and use that. # XXX: The order of these tests seems to matter, see git blame. -@skipIf(missing_deps(), 'dependencies (%s) not installed' % ','.join(deps)) -@skipUnless(os.getenv('QEMU_TEST_ALLOW_LARGE_STORAGE'), 'storage limited') +@skipIfMissingCommands("xorriso") +@skipBigDataTest() class HypervisorTest(QemuSystemTest): =20 timeout =3D 1000 diff --git a/tests/functional/test_ppc_40p.py b/tests/functional/test_ppc_4= 0p.py index 67bcdae53a..7a74e0cca7 100755 --- a/tests/functional/test_ppc_40p.py +++ b/tests/functional/test_ppc_40p.py @@ -7,11 +7,8 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import os - -from unittest import skipUnless from qemu_test import QemuSystemTest, Asset -from qemu_test import wait_for_console_pattern +from qemu_test import wait_for_console_pattern, skipUntrustedTest =20 =20 class IbmPrep40pMachine(QemuSystemTest): @@ -37,7 +34,7 @@ class IbmPrep40pMachine(QemuSystemTest): # All rights reserved. # U.S. Government Users Restricted Rights - Use, duplication or disclo= sure # restricted by GSA ADP Schedule Contract with IBM Corp. - @skipUnless(os.getenv('QEMU_TEST_ALLOW_UNTRUSTED_CODE'), 'untrusted co= de') + @skipUntrustedTest() def test_factory_firmware_and_netbsd(self): self.set_machine('40p') self.require_accelerator("tcg") diff --git a/tests/functional/test_rx_gdbsim.py b/tests/functional/test_rx_= gdbsim.py index 5687f756bb..f7621d999d 100755 --- a/tests/functional/test_rx_gdbsim.py +++ b/tests/functional/test_rx_gdbsim.py @@ -12,10 +12,9 @@ =20 import os =20 -from unittest import skipUnless from qemu_test import QemuSystemTest, Asset from qemu_test import exec_command_and_wait_for_pattern -from qemu_test import wait_for_console_pattern +from qemu_test import wait_for_console_pattern, skipFlakyTest from qemu_test.utils import gzip_uncompress =20 =20 @@ -52,9 +51,10 @@ def test_uboot(self): wait_for_console_pattern(self, uboot_version) gcc_version =3D 'rx-unknown-linux-gcc (GCC) 9.0.0 20181105 (experi= mental)' # FIXME limit baudrate on chardev, else we type too fast + # https://gitlab.com/qemu-project/qemu/-/issues/2691 #exec_command_and_wait_for_pattern(self, 'version', gcc_version) =20 - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on G= itLab') + @skipFlakyTest(bug_url=3D"https://gitlab.com/qemu-project/qemu/-/issue= s/2691") def test_linux_sash(self): """ Boots a Linux kernel and checks that the console is operational. diff --git a/tests/functional/test_sh4_r2d.py b/tests/functional/test_sh4_r= 2d.py index c3cfff79ad..e2fcde2d6b 100755 --- a/tests/functional/test_sh4_r2d.py +++ b/tests/functional/test_sh4_r2d.py @@ -4,11 +4,8 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - -from qemu_test import LinuxKernelTest, Asset +from qemu_test import LinuxKernelTest, Asset, skipFlakyTest from qemu_test.utils import archive_extract -from unittest import skipUnless =20 class R2dTest(LinuxKernelTest): =20 @@ -18,7 +15,8 @@ class R2dTest(LinuxKernelTest): =20 # This test has a 6-10% failure rate on various hosts that look # like issues with a buggy kernel. - @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable') + # XXX file tracking bug + @skipFlakyTest(bug_url=3DNone) def test_r2d(self): self.set_machine('r2d') file_path =3D self.ASSET_DAY09.fetch() --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451651; cv=none; d=zohomail.com; s=zohoarc; b=HToXOmQqiZlJ+DYhHN/efMYSgMLlq5fEAr1sgZljDMglgdnMGUfSA3xkbLHkwNbgqggV8gcdj9Rd+t2b/VlQ7YVpAHuJmasrU8xxMjIEYDRKTgCQbw0G+A5h057mgZFMeoHS0aeQRP17cVuukUMfr8Aa/Rp4b3g3r2dxAZSeAf0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451651; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ELr/rM7FSdeA0GsZz078xGh7Vop7TiEuEBNFFMFo/R8=; b=SH0W41knkzdLs+ijyDIkz0MWgoh0qiqkfVw5NFnULr1wD68TZ4EAEUe7n2LVkIzEPslASVlRzxkOGH1hkWzDixvpi+wrw0vR6TV8W0a/2Ospl0CijkxgatXCBiHZsok5yRU60o2icGtKmRgI3S7MA4OwSWGUBf5kdwh2QuFC98k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451651913366.47694911991493; Tue, 17 Dec 2024 08:07:31 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa03-0004Ko-Eo; Tue, 17 Dec 2024 11:00:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzq-0004GD-En for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzo-00027c-PI for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:30 -0500 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-416-ccDHzOj-OW21ZdNZzgs5rg-1; Tue, 17 Dec 2024 11:00:24 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3E6E11955F3F; Tue, 17 Dec 2024 16:00:23 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EE2B730044C1; Tue, 17 Dec 2024 16:00:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ELr/rM7FSdeA0GsZz078xGh7Vop7TiEuEBNFFMFo/R8=; b=ZEvew5ZIDqNb20wCLfDVl2Qc/w10oTckRmCiW9HD9vIrls9URxW2HIj3XqdGRlFtqLJjYT CikA2PYniJ+0mx6zJjvTMDZLp25VbN4l3B4gW1IuKGNhW7Z22wp2jWFmHBJfEjSKcguFHh hr6JwHTnOpcT4lTfuNcMsCme3aBNH/U= X-MC-Unique: ccDHzOj-OW21ZdNZzgs5rg-1 X-Mimecast-MFC-AGG-ID: ccDHzOj-OW21ZdNZzgs5rg From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 09/32] tests/functional: drop 'has_cmd' and 'has_cmds' helpers Date: Tue, 17 Dec 2024 15:59:30 +0000 Message-ID: <20241217155953.3950506-10-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451652963116600 The 'which' helper is simpler, not depending on the external 'which' binary, and is sufficient for test needs. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/cmd.py | 54 ++---------------------- tests/functional/qemu_test/tuxruntest.py | 10 ++--- 3 files changed, 9 insertions(+), 57 deletions(-) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu= _test/__init__.py index 7dee3522f2..fe6cbe3a8a 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -8,7 +8,7 @@ =20 from .asset import Asset from .config import BUILD_DIR -from .cmd import has_cmd, has_cmds, run_cmd, is_readable_executable_file, \ +from .cmd import run_cmd, is_readable_executable_file, \ interrupt_interactive_console_until_pattern, wait_for_console_pattern,= \ exec_command, exec_command_and_wait_for_pattern, get_qemu_img, which from .testcase import QemuBaseTest, QemuUserTest, QemuSystemTest diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test= /cmd.py index 600e0509db..bebcd46dcf 100644 --- a/tests/functional/qemu_test/cmd.py +++ b/tests/functional/qemu_test/cmd.py @@ -29,52 +29,6 @@ def which(tool): return p return None =20 -def has_cmd(name, args=3DNone): - """ - This function is for use in a @skipUnless decorator, e.g.: - - @skipUnless(*has_cmd('sudo -n', ('sudo', '-n', 'true'))) - def test_something_that_needs_sudo(self): - ... - """ - - if args is None: - args =3D ('which', name) - - try: - _, stderr, exitcode =3D run_cmd(args) - except Exception as e: - exitcode =3D -1 - stderr =3D str(e) - - if exitcode !=3D 0: - cmd_line =3D ' '.join(args) - err =3D f'{name} required, but "{cmd_line}" failed: {stderr.strip(= )}' - return (False, err) - else: - return (True, '') - -def has_cmds(*cmds): - """ - This function is for use in a @skipUnless decorator and - allows checking for the availability of multiple commands, e.g.: - - @skipUnless(*has_cmds(('cmd1', ('cmd1', '--some-parameter')), - 'cmd2', 'cmd3')) - def test_something_that_needs_cmd1_and_cmd2(self): - ... - """ - - for cmd in cmds: - if isinstance(cmd, str): - cmd =3D (cmd,) - - ok, errstr =3D has_cmd(*cmd) - if not ok: - return (False, errstr) - - return (True, '') - def run_cmd(args): subp =3D subprocess.Popen(args, stdout=3Dsubprocess.PIPE, @@ -254,7 +208,7 @@ def get_qemu_img(test): qemu_img =3D os.path.join(BUILD_DIR, 'qemu-img') if os.path.exists(qemu_img): return qemu_img - (has_system_qemu_img, errmsg) =3D has_cmd('qemu-img') - if has_system_qemu_img: - return 'qemu-img' - test.skipTest(errmsg) + qemu_img =3D which('qemu-img') + if qemu_img is not None: + return qemu_img + test.skipTest(f"qemu-img not found in {BUILD_DIR} or '$PATH'") diff --git a/tests/functional/qemu_test/tuxruntest.py b/tests/functional/qe= mu_test/tuxruntest.py index d375f2713b..2e5c6d110c 100644 --- a/tests/functional/qemu_test/tuxruntest.py +++ b/tests/functional/qemu_test/tuxruntest.py @@ -15,7 +15,7 @@ from qemu_test import QemuSystemTest from qemu_test import exec_command_and_wait_for_pattern from qemu_test import wait_for_console_pattern -from qemu_test import has_cmd, run_cmd, get_qemu_img +from qemu_test import which, run_cmd, get_qemu_img =20 class TuxRunBaselineTest(QemuSystemTest): =20 @@ -38,10 +38,8 @@ def setUp(self): super().setUp() =20 # We need zstd for all the tuxrun tests - (has_zstd, msg) =3D has_cmd('zstd') - if has_zstd is False: - self.skipTest(msg) - self.zstd =3D 'zstd' + if which('zstd') is None: + self.skipTest("zstd not found in $PATH") =20 # Pre-init TuxRun specific settings: Most machines work with # reasonable defaults but we sometimes need to tweak the @@ -78,7 +76,7 @@ def fetch_tuxrun_assets(self, kernel_asset, rootfs_asset,= dtb_asset=3DNone): =20 disk_image =3D self.workdir + "/rootfs.ext4" =20 - run_cmd([self.zstd, "-f", "-d", disk_image_zst, + run_cmd(['zstd', "-f", "-d", disk_image_zst, "-o", disk_image]) # zstd copies source archive permissions for the output # file, so must make this writable for QEMU --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451902; cv=none; d=zohomail.com; s=zohoarc; b=Rpt4hagsDQ8qgzuhvlxlLuM0PINuSDDn3rb1McuHLN3TjMrB3Ht8v+p2Bz/c7KwN9nEvMhF85gB8NIYvrV2Nx0Mnwi7tmomwF7IVgKhCFQFcblEVrEe5PgufKTccEfu1hRjDQF1ay1MQiVVrOqLWLkKndiNiFhPtEqMrZ0+2s9I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451902; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=H2/PjAowauHLhm8ppho0o8dsbbcBzfqDvQw+ztoguZA=; b=SLxMcwtLWfkA3DiW6wXcXOOZ5rk2m64UYGnXcY5EPOJLpkrFCI6kG+0JUarv9B1hWuV68TX7LpWJCcVqeuqp9ZWlBW/PbWUOECfYDnfmOZI33o2W0LzXsEtprMA9u3XHeA2rwmIlgqbGGxTEIV1kjkvrj/MZqm+UgIDViKMNfW0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451902161122.27025015567028; Tue, 17 Dec 2024 08:11:42 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa04-0004Mr-En; Tue, 17 Dec 2024 11:00:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa01-0004JV-Mr for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:42 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzv-000288-C5 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:41 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-g-cDAI2lMcq08Dsej5_r3A-1; Tue, 17 Dec 2024 11:00:29 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 331AF19560B5; Tue, 17 Dec 2024 16:00:28 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CC8D730044C1; Tue, 17 Dec 2024 16:00:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451233; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H2/PjAowauHLhm8ppho0o8dsbbcBzfqDvQw+ztoguZA=; b=Ox9/bpUxNYaCpwcdYwI/Gijmno92qDVF13CuLPet656EIBbV40aTPIgPqXTcBv3XP8zWzn 712cNPwxJdZKJlUNKxT/Cot3MFe16kjkw9lbZyTgQHEyvv2ne9cb1MuABhqjRYyATCtS7i dMn1dDL40GigQ5E7o8zVG7ffLD0h6nQ= X-MC-Unique: g-cDAI2lMcq08Dsej5_r3A-1 X-Mimecast-MFC-AGG-ID: g-cDAI2lMcq08Dsej5_r3A From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 10/32] tests/functional: add helpers for building file paths Date: Tue, 17 Dec 2024 15:59:31 +0000 Message-ID: <20241217155953.3950506-11-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451902932116600 Add helper methods that construct paths for * log files - to be preserved at the end of a test * scratch files - to be purged at the end of a test * build files - anything relative to the build root * data files - anything relative to the functional test source root * socket files - a short temporary dir to avoid UNIX socket limits These are to be used instead of direct access to the self.workdir, or self.logdir variables, or any other place where paths are built manually. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/testcase.py | 95 ++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index 90ae59eb54..89425b737c 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -13,10 +13,12 @@ =20 import logging import os +from pathlib import Path import pycotap import shutil import subprocess import sys +import tempfile import unittest import uuid =20 @@ -37,9 +39,99 @@ class QemuBaseTest(unittest.TestCase): log =3D None logdir =3D None =20 + ''' + Create a temporary directory suitable for storing UNIX + socket paths. + + Returns: a tempfile.TemporaryDirectory instance + ''' + def socket_dir(self): + if self.socketdir is None: + self.socketdir =3D tempfile.TemporaryDirectory( + prefix=3D"qemu_func_test_sock_") + return self.socketdir + + ''' + @params args list of zero or more subdirectories or file + + Construct a path for accessing a data file located + relative to the source directory that is the root for + functional tests. + + @args may be an empty list to reference the root dir + itself, may be a single element to reference a file in + the root directory, or may be multiple elements to + reference a file nested below. The path components + will be joined using the platform appropriate path + separator. + + Returns: string representing a file path + ''' + def data_file(self, *args): + return str(Path(Path(__file__).parent.parent, *args)) + + ''' + @params args list of zero or more subdirectories or file + + Construct a path for accessing a data file located + relative to the build directory root. + + @args may be an empty list to reference the build dir + itself, may be a single element to reference a file in + the build directory, or may be multiple elements to + reference a file nested below. The path components + will be joined using the platform appropriate path + separator. + + Returns: string representing a file path + ''' + def build_file(self, *args): + return str(Path(BUILD_DIR, *args)) + + ''' + @params args list of zero or more subdirectories or file + + Construct a path for accessing/creating a scratch file + located relative to a temporary directory dedicated to + this test case. The directory and its contents will be + purged upon completion of the test. + + @args may be an empty list to reference the scratch dir + itself, may be a single element to reference a file in + the scratch directory, or may be multiple elements to + reference a file nested below. The path components + will be joined using the platform appropriate path + separator. + + Returns: string representing a file path + ''' + def scratch_file(self, *args): + return str(Path(self.workdir, *args)) + + ''' + @params args list of zero or more subdirectories or file + + Construct a path for accessing/creating a log file + located relative to a temporary directory dedicated to + this test case. The directory and its log files will be + preserved upon completion of the test. + + @args may be an empty list to reference the log dir + itself, may be a single element to reference a file in + the log directory, or may be multiple elements to + reference a file nested below. The path components + will be joined using the platform appropriate path + separator. + + Returns: string representing a file path + ''' + def log_file(self, *args): + return str(Path(self.logdir, *args)) + def setUp(self, bin_prefix): self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be= set') self.arch =3D self.qemu_bin.split('-')[-1] + self.socketdir =3D None =20 self.outputdir =3D os.path.join(BUILD_DIR, 'tests', 'functional', self.arch, self.id()) @@ -65,6 +157,9 @@ def setUp(self, bin_prefix): def tearDown(self): if "QEMU_TEST_KEEP_SCRATCH" not in os.environ: shutil.rmtree(self.workdir) + if self.socketdir is not None: + shutil.rmtree(self.socketdir.name) + self.socketdir =3D None self.machinelog.removeHandler(self._log_fh) self.log.removeHandler(self._log_fh) =20 --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451898; cv=none; d=zohomail.com; s=zohoarc; b=LjRL86/G8MDi4F2vWN1s6T+vVHjsmk4amVejxEPetoN8psxyRc7OfdzuZhSN+6LPivEhOqWSC/nqPwkkx6RUYlr2r0fuqXtBGNrTBZ3W/yDnK4BjG2zf0RPq7o6yikOF7r3/NZLc3LQkq6U+svfzujW7qjOhKYlFmmYECE/AnRI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451898; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=U9esgIthrudvDaRJ+rEBiuyKD4+iIMzGIxm2i96hJI8=; b=OApVqc+QbfbbHqt0QqGxkYtnL42jMuo5LJG+wVhFxZQT8Mrg4Ab1S12bv+eVT4SA2d/26kL8fDdooZPANy3vFhBwKtNJwjruu0gF/0olMEWCP6AkcMqoJP1sXimEZPR7WAqqh+Dq+Kev83ZGEkeBuGaI3q3M2mdkantw6L7A898= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451898961682.9806238941839; Tue, 17 Dec 2024 08:11:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0A-0004Nr-Pn; Tue, 17 Dec 2024 11:00:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa00-0004JO-KO for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:42 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNZzy-00028W-UA for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:40 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-302-nd6-OT58PFyGYdfA8cxuAg-1; Tue, 17 Dec 2024 11:00:34 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 90FBD195608F; Tue, 17 Dec 2024 16:00:33 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9F84430044C1; Tue, 17 Dec 2024 16:00:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U9esgIthrudvDaRJ+rEBiuyKD4+iIMzGIxm2i96hJI8=; b=LHl3qwsH8HZ58+Wcpt1Qtsksw2twVCurQwPQAej4RtkiztabAQosD5AznMncrJbDnOPFDK TJlfcwmA8PVfpDu1lcc+uOhe54ARWTfj3VMynNI1jJNOXet5WarL4SaI4pZ7TxO5LmDM7C 4xrlJfgfjTgHKYXdiqNPsbgxh730mkQ= X-MC-Unique: nd6-OT58PFyGYdfA8cxuAg-1 X-Mimecast-MFC-AGG-ID: nd6-OT58PFyGYdfA8cxuAg From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 11/32] tests/functional: switch over to using self.log_file(...) Date: Tue, 17 Dec 2024 15:59:32 +0000 Message-ID: <20241217155953.3950506-12-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451900829116600 This removes direct access of the 'self.logdir' variable. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/testcase.py | 9 ++++----- tests/functional/test_virtio_gpu.py | 4 +--- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index 89425b737c..2174fbb155 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -126,7 +126,7 @@ def scratch_file(self, *args): Returns: string representing a file path ''' def log_file(self, *args): - return str(Path(self.logdir, *args)) + return str(Path(self.outputdir, *args)) =20 def setUp(self, bin_prefix): self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be= set') @@ -138,8 +138,7 @@ def setUp(self, bin_prefix): self.workdir =3D os.path.join(self.outputdir, 'scratch') os.makedirs(self.workdir, exist_ok=3DTrue) =20 - self.logdir =3D self.outputdir - self.log_filename =3D os.path.join(self.logdir, 'base.log') + self.log_filename =3D self.log_file('base.log') self.log =3D logging.getLogger('qemu-test') self.log.setLevel(logging.DEBUG) self._log_fh =3D logging.FileHandler(self.log_filename, mode=3D'w') @@ -215,7 +214,7 @@ def setUp(self): =20 console_log =3D logging.getLogger('console') console_log.setLevel(logging.DEBUG) - self.console_log_name =3D os.path.join(self.logdir, 'console.log') + self.console_log_name =3D self.log_file('console.log') self._console_log_fh =3D logging.FileHandler(self.console_log_name, mode=3D'w') self._console_log_fh.setLevel(logging.DEBUG) @@ -269,7 +268,7 @@ def _new_vm(self, name, *args): vm =3D QEMUMachine(self.qemu_bin, name=3Dname, base_temp_dir=3Dself.workdir, - log_dir=3Dself.logdir) + log_dir=3Dself.log_file()) self.log.debug('QEMUMachine "%s" created', name) self.log.debug('QEMUMachine "%s" temp_dir: %s', name, vm.temp_dir) =20 diff --git a/tests/functional/test_virtio_gpu.py b/tests/functional/test_vi= rtio_gpu.py index 2d298b1f02..7654421e6b 100755 --- a/tests/functional/test_virtio_gpu.py +++ b/tests/functional/test_virtio_gpu.py @@ -100,9 +100,7 @@ def test_vhost_user_vga_virgl(self): os.set_inheritable(qemu_sock.fileno(), True) os.set_inheritable(vug_sock.fileno(), True) =20 - self._vug_log_path =3D os.path.join( - self.logdir, "vhost-user-gpu.log" - ) + self._vug_log_path =3D self.log_file("vhost-user-gpu.log") self._vug_log_file =3D open(self._vug_log_path, "wb") self.log.info('Complete vhost-user-gpu.log file can be ' 'found at %s', self._vug_log_path) --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451277; cv=none; d=zohomail.com; s=zohoarc; b=YsfNIVuIOACy5tQ5dYPXsaIG9l5QnN5+4v9QvmOBR4MsygqoAh3dOsxyvD/D8WrYA2mMpLdhC0xS0O8Oxd13f4TsbNpLznFXNcwTKxa+zMBR9hAkWRokVHdVzZiws5qL4HLZYjp0pBSOYGMiRK7gScNurxtYtkKtML9YlJJYKnc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451277; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=QAjQiwP0haDcUnPsVDuIIqdIJF1iPPhwSLsXKgeS0eY=; b=bYOwKB8PlSJGhn9lb+/pvCeuJCsWAYysQAP+0ODUSpy0tGDQ5Mwpx6lF346RpG+D4SNDcHQ42cFKeHdDfgCccujv7IwGiGTBLVpM7YRhhfuQgIgRWu+9mNNm5B4ixHxKPtsfmdpRKhckmShkOqLs31+jTN8WIdOiGkGD5uj0EmA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451277454618.0344081129756; Tue, 17 Dec 2024 08:01:17 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0F-0004Nt-FV; Tue, 17 Dec 2024 11:01:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa03-0004Kc-8V for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa01-00028u-FL for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:42 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-Du8dVYYMOIuXR1VRruUjVg-1; Tue, 17 Dec 2024 11:00:37 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8AF601955F4A; Tue, 17 Dec 2024 16:00:36 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2F0FB300F9B5; Tue, 17 Dec 2024 16:00:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451240; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QAjQiwP0haDcUnPsVDuIIqdIJF1iPPhwSLsXKgeS0eY=; b=EX1I57OOXTIfXfMSXNcQ49v/q8g4eF0OXPO2eKBu0f0zG3bc+UQOplESVjdnR9R+wMXxcS T7ZsCluBpy+d97oxWx/OuWLaMdY3CPvyxXBMuzhM+UJnA6s/tUCYPgn9HmzWpPS7B1N84H AOEi6GUChBTf5hkSIG8W7WDSUTD6CG4= X-MC-Unique: Du8dVYYMOIuXR1VRruUjVg-1 X-Mimecast-MFC-AGG-ID: Du8dVYYMOIuXR1VRruUjVg From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 12/32] tests/functional: switch over to using self.build_file(...) Date: Tue, 17 Dec 2024 15:59:33 +0000 Message-ID: <20241217155953.3950506-13-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451278072116600 This removes direct access of the 'BUILD_DIR' variable. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/cmd.py | 5 ++--- tests/functional/qemu_test/testcase.py | 4 ++-- tests/functional/test_aarch64_virt.py | 5 ++--- tests/functional/test_virtio_gpu.py | 11 +++-------- 4 files changed, 9 insertions(+), 16 deletions(-) diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test= /cmd.py index bebcd46dcf..c8971de00a 100644 --- a/tests/functional/qemu_test/cmd.py +++ b/tests/functional/qemu_test/cmd.py @@ -16,7 +16,6 @@ import os.path import subprocess =20 -from .config import BUILD_DIR =20 def which(tool): """ looks up the full path for @tool, returns None if not found @@ -205,10 +204,10 @@ def get_qemu_img(test): =20 # If qemu-img has been built, use it, otherwise the system wide one # will be used. - qemu_img =3D os.path.join(BUILD_DIR, 'qemu-img') + qemu_img =3D test.build_file('qemu-img') if os.path.exists(qemu_img): return qemu_img qemu_img =3D which('qemu-img') if qemu_img is not None: return qemu_img - test.skipTest(f"qemu-img not found in {BUILD_DIR} or '$PATH'") + test.skipTest(f"qemu-img not found in build dir or '$PATH'") diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index 2174fbb155..493938240c 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -133,8 +133,8 @@ def setUp(self, bin_prefix): self.arch =3D self.qemu_bin.split('-')[-1] self.socketdir =3D None =20 - self.outputdir =3D os.path.join(BUILD_DIR, 'tests', 'functional', - self.arch, self.id()) + self.outputdir =3D self.build_file('tests', 'functional', + self.arch, self.id()) self.workdir =3D os.path.join(self.outputdir, 'scratch') os.makedirs(self.workdir, exist_ok=3DTrue) =20 diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_= aarch64_virt.py index c967da41b4..5bc461b482 100755 --- a/tests/functional/test_aarch64_virt.py +++ b/tests/functional/test_aarch64_virt.py @@ -14,7 +14,6 @@ import os import logging =20 -from qemu_test import BUILD_DIR from qemu_test import QemuSystemTest, Asset from qemu_test import exec_command, wait_for_console_pattern from qemu_test import get_qemu_img, run_cmd @@ -54,8 +53,8 @@ def test_alpine_virt_tcg_gic_max(self): "mte=3Don," "gic-version=3Dmax,iommu=3Dsmmuv3") self.vm.add_args("-smp", "2", "-m", "1024") - self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios', - 'edk2-aarch64-code.fd')) + self.vm.add_args('-bios', self.build_file('pc-bios', + 'edk2-aarch64-code.fd')) self.vm.add_args("-drive", f"file=3D{iso_path},media=3Dcdrom,forma= t=3Draw") self.vm.add_args('-device', 'virtio-rng-pci,rng=3Drng0') self.vm.add_args('-object', 'rng-random,id=3Drng0,filename=3D/dev/= urandom') diff --git a/tests/functional/test_virtio_gpu.py b/tests/functional/test_vi= rtio_gpu.py index 7654421e6b..81c9156d63 100755 --- a/tests/functional/test_virtio_gpu.py +++ b/tests/functional/test_virtio_gpu.py @@ -6,7 +6,6 @@ # later. See the COPYING file in the top-level directory. =20 =20 -from qemu_test import BUILD_DIR from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern from qemu_test import exec_command_and_wait_for_pattern @@ -18,12 +17,8 @@ import subprocess =20 =20 -def pick_default_vug_bin(): - relative_path =3D "./contrib/vhost-user-gpu/vhost-user-gpu" - if is_readable_executable_file(relative_path): - return relative_path - - bld_dir_path =3D os.path.join(BUILD_DIR, relative_path) +def pick_default_vug_bin(test): + bld_dir_path =3D test.build_file("contrib", "vhost-user-gpu", "vhost-u= ser-gpu") if is_readable_executable_file(bld_dir_path): return bld_dir_path =20 @@ -86,7 +81,7 @@ def test_vhost_user_vga_virgl(self): # FIXME: should check presence of vhost-user-gpu, virgl, memfd etc self.require_accelerator('kvm') =20 - vug =3D pick_default_vug_bin() + vug =3D pick_default_vug_bin(self) if not vug: self.skipTest("Could not find vhost-user-gpu") =20 --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451833; cv=none; d=zohomail.com; s=zohoarc; b=B8K2VMuWSS+F9Xm1LMYES/W5csbPgBeGVOKARUJb91VSJzRoNkNAPLQRAto3XQFjK+WQAqSuII6S/rnWdvN+Ec8191UuGztDO/GIsOUR1624c7NjdjH2PI1JCZ/LjpWM+xUTUsjGlh8tHK9lv6f0tTxHiKtek2/Xjbm3B+oJyTA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451833; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=joRaw0FxMPb99fJofIlwceyLs8ZRq3mk1I4L2cqj/UU=; b=fCKB8GJemgSYBMDZyajKdAylE/qEIHSrH+/iE+t1LU6MjIv5vWxkh3sytB3fAIgneMwWjjuaJ/vCcYY9PFIKVMHUl11+DwQNRWTNf6RI1Z8AAbWEWJMi35LxbuN/nQ9Tw98LswiONdRbuSVZTjB63jXmTCyr9PU4TwozBfMd+zA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451833478860.8492566973168; Tue, 17 Dec 2024 08:10:33 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0Z-0004l2-KS; Tue, 17 Dec 2024 11:01:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa06-0004NV-4s for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:46 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa04-00029R-LY for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:45 -0500 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-136-A7wSvDHNMIuTy_QwlGEZdg-1; Tue, 17 Dec 2024 11:00:40 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6ED0A19560AF; Tue, 17 Dec 2024 16:00:39 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 24FDB300F9B5; Tue, 17 Dec 2024 16:00:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451244; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=joRaw0FxMPb99fJofIlwceyLs8ZRq3mk1I4L2cqj/UU=; b=g8aOpkMod/xpSFZaiANdsxw+mafwRCO1ED4CEY014TSLiowIWRF4OvPZbMFb2vPO/kDSxm OE4Rr/Q0TTzl6g6vzG2rKDLlXbpjynMtA+FXV+itwtHe4562lmi3aDwhn6hi9M/xEGBMOK 1ajbz9N8atxGv9RhQs3OrHAqAwKkjoQ= X-MC-Unique: A7wSvDHNMIuTy_QwlGEZdg-1 X-Mimecast-MFC-AGG-ID: A7wSvDHNMIuTy_QwlGEZdg From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 13/32] tests/functional: switch over to using self.data_file(...) Date: Tue, 17 Dec 2024 15:59:34 +0000 Message-ID: <20241217155953.3950506-14-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451834470116600 This removes direct path manipulation to figure out the source dir Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/test_acpi_bits.py | 22 +++++++--------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/tests/functional/test_acpi_bits.py b/tests/functional/test_acp= i_bits.py index 8763ea0822..59d0383563 100755 --- a/tests/functional/test_acpi_bits.py +++ b/tests/functional/test_acpi_bits.py @@ -38,7 +38,6 @@ import tarfile import zipfile =20 -from pathlib import Path from typing import ( List, Optional, @@ -119,7 +118,6 @@ class AcpiBitsTest(QemuSystemTest): #pylint: disable=3D= too-many-instance-attribute def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._vm =3D None - self._baseDir =3D None =20 self._debugcon_addr =3D '0x403' self._debugcon_log =3D 'debugcon-log.txt' @@ -134,26 +132,22 @@ def _print_log(self, log): def copy_bits_config(self): """ copies the bios bits config file into bits. """ - config_file =3D 'bits-cfg.txt' - bits_config_dir =3D os.path.join(self._baseDir, 'acpi-bits', - 'bits-config') + bits_config_file =3D self.data_file('acpi-bits', + 'bits-config', + 'bits-cfg.txt') target_config_dir =3D os.path.join(self.workdir, 'bits-%d' %self.BITS_INTERNAL_VER, 'boot') - self.assertTrue(os.path.exists(bits_config_dir)) + self.assertTrue(os.path.exists(bits_config_file)) self.assertTrue(os.path.exists(target_config_dir)) - self.assertTrue(os.access(os.path.join(bits_config_dir, - config_file), os.R_OK)) - shutil.copy2(os.path.join(bits_config_dir, config_file), - target_config_dir) + shutil.copy2(bits_config_file, target_config_dir) self.logger.info('copied config file %s to %s', - config_file, target_config_dir) + bits_config_file, target_config_dir) =20 def copy_test_scripts(self): """copies the python test scripts into bits. """ =20 - bits_test_dir =3D os.path.join(self._baseDir, 'acpi-bits', - 'bits-tests') + bits_test_dir =3D self.data_file('acpi-bits', 'bits-tests') target_test_dir =3D os.path.join(self.workdir, 'bits-%d' %self.BITS_INTERNAL_VER, 'boot', 'python') @@ -256,8 +250,6 @@ def setUp(self): # pylint: disable=3Darguments-differ super().setUp() self.logger =3D self.log =20 - self._baseDir =3D Path(__file__).parent - prebuiltDir =3D os.path.join(self.workdir, 'prebuilt') if not os.path.isdir(prebuiltDir): os.mkdir(prebuiltDir, mode=3D0o775) --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451880; cv=none; d=zohomail.com; s=zohoarc; b=bjDmCVUTudwUeSEPLypfs2Bq5RmMPmlhCA41pLhlsM6JnaDwqwZN2OMHrM5IacVx9b98gHCbvy7OYAuyq6hxkYD28XhG0qxfN6RoPWiEfTpHw2BKJIiXzAChzpi5qGKR6eM3RaiaKsxyRk3k98zMwmciQU4M/f7obBCZ2EDYrME= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451880; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=jNXkRfUWjg+qYfav/IBXJhPeWBKRBIJ+3d7zsorNyTk=; b=dZ6RbzsbIuA0Mv3eQNPjLcYVetGG53PPjJfgzVxL6IOkeTcnjoz+RTPaiQFYz8Y0UYHjoeY2XYIX3cVQvJSz8Dm+1CLWk15JzQZNRnh789HtE6WcPrteRdslW3yXTaP8uVSiO8TywxDlWJHMaDuTUoqSiqipP521cFsgWmVzTfo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 173445188050631.60189246530149; Tue, 17 Dec 2024 08:11:20 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0Z-0004kB-IB; Tue, 17 Dec 2024 11:01:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0A-0004OD-5m for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa06-00029c-3G for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:49 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-153-a8_hPxHkP6iLxQwte3K_kQ-1; Tue, 17 Dec 2024 11:00:43 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7B0371955F42; Tue, 17 Dec 2024 16:00:42 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E05D5300F9B5; Tue, 17 Dec 2024 16:00:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451245; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jNXkRfUWjg+qYfav/IBXJhPeWBKRBIJ+3d7zsorNyTk=; b=NWz24HbJ9CvaFQy6MFJVoSW9Bs03uVgB3ojF22xotlrMWRB0LMfUsXSV94Qlf2oWiizy1C FffRkUYhDVWWvqSNkoVjo3+t/oIe1DlvR5hqOEjSKHKEt4T8rzpV0QaJyM+TFDBk93ni+m O23Uf4nEsvioYdKOvB5DVxn/84iK7Hs= X-MC-Unique: a8_hPxHkP6iLxQwte3K_kQ-1 X-Mimecast-MFC-AGG-ID: a8_hPxHkP6iLxQwte3K_kQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 14/32] tests/functional: switch over to using self.scratch_file() Date: Tue, 17 Dec 2024 15:59:35 +0000 Message-ID: <20241217155953.3950506-15-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451883240116600 Replace any instances of os.path.join(self.workdir, ".../...") self.workdir + "/.../..." with self.scratch_file("...", "...") which is more compact and portable Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/linuxkernel.py | 7 ++- tests/functional/qemu_test/tuxruntest.py | 2 +- tests/functional/test_aarch64_aspeed.py | 17 ++++--- tests/functional/test_aarch64_raspi3.py | 3 +- tests/functional/test_aarch64_raspi4.py | 4 +- tests/functional/test_aarch64_sbsaref.py | 6 +-- tests/functional/test_aarch64_virt.py | 3 +- tests/functional/test_acpi_bits.py | 49 +++++++++---------- tests/functional/test_alpha_clipper.py | 4 +- tests/functional/test_arm_aspeed_ast1030.py | 6 +-- tests/functional/test_arm_aspeed_ast2500.py | 2 +- tests/functional/test_arm_aspeed_ast2600.py | 2 +- tests/functional/test_arm_bflt.py | 3 +- tests/functional/test_arm_bpim2u.py | 6 +-- tests/functional/test_arm_canona1100.py | 3 +- tests/functional/test_arm_cubieboard.py | 6 +-- tests/functional/test_arm_emcraft_sf2.py | 2 +- tests/functional/test_arm_integratorcp.py | 3 +- tests/functional/test_arm_orangepi.py | 8 +-- tests/functional/test_arm_raspi2.py | 4 +- tests/functional/test_arm_smdkc210.py | 2 +- tests/functional/test_arm_vexpress.py | 5 +- tests/functional/test_m68k_mcf5208evb.py | 3 +- tests/functional/test_m68k_nextcube.py | 5 +- .../functional/test_microblaze_s3adsp1800.py | 3 +- .../test_microblazeel_s3adsp1800.py | 5 +- tests/functional/test_mips64el_malta.py | 6 +-- tests/functional/test_mips_malta.py | 4 +- tests/functional/test_mipsel_malta.py | 6 +-- tests/functional/test_or1k_sim.py | 2 +- tests/functional/test_ppc64_e500.py | 2 +- tests/functional/test_ppc64_hv.py | 12 +---- tests/functional/test_ppc_amiga.py | 12 +++-- tests/functional/test_ppc_bamboo.py | 10 ++-- tests/functional/test_ppc_mac.py | 2 +- tests/functional/test_ppc_mpc8544ds.py | 2 +- tests/functional/test_ppc_virtex_ml507.py | 5 +- tests/functional/test_rx_gdbsim.py | 4 +- tests/functional/test_s390x_ccw_virtio.py | 2 +- tests/functional/test_s390x_topology.py | 4 +- tests/functional/test_sh4_r2d.py | 3 +- tests/functional/test_sh4eb_r2d.py | 8 +-- tests/functional/test_sparc64_sun4u.py | 5 +- tests/functional/test_sparc_sun4m.py | 2 +- tests/functional/test_xtensa_lx60.py | 3 +- 45 files changed, 118 insertions(+), 139 deletions(-) diff --git a/tests/functional/qemu_test/linuxkernel.py b/tests/functional/q= emu_test/linuxkernel.py index 2b5b9a5fda..2e4f4e35fd 100644 --- a/tests/functional/qemu_test/linuxkernel.py +++ b/tests/functional/qemu_test/linuxkernel.py @@ -46,8 +46,7 @@ def extract_from_deb(self, deb_path, path): os.chdir(cwd) # Return complete path to extracted file. Because callers to # extract_from_deb() specify 'path' with a leading slash, it is - # necessary to use os.path.relpath() as otherwise os.path.join() - # interprets it as an absolute path and drops the self.workdir par= t. - return os.path.normpath(os.path.join(self.workdir, - os.path.relpath(path, '/'))) + # necessary to use os.path.relpath() as otherwise scratch_file() + # interprets it as an absolute path and drops the required prefix + return os.path.normpath(self.scratch_file(os.path.relpath(path, '/= '))) =20 diff --git a/tests/functional/qemu_test/tuxruntest.py b/tests/functional/qe= mu_test/tuxruntest.py index 2e5c6d110c..0b1bb8f0ed 100644 --- a/tests/functional/qemu_test/tuxruntest.py +++ b/tests/functional/qemu_test/tuxruntest.py @@ -74,7 +74,7 @@ def fetch_tuxrun_assets(self, kernel_asset, rootfs_asset,= dtb_asset=3DNone): kernel_image =3D kernel_asset.fetch() disk_image_zst =3D rootfs_asset.fetch() =20 - disk_image =3D self.workdir + "/rootfs.ext4" + disk_image =3D self.scratch_file("rootfs.ext4") =20 run_cmd(['zstd', "-f", "-d", disk_image_zst, "-o", disk_image]) diff --git a/tests/functional/test_aarch64_aspeed.py b/tests/functional/tes= t_aarch64_aspeed.py index e196f88537..8ba2c67248 100755 --- a/tests/functional/test_aarch64_aspeed.py +++ b/tests/functional/test_aarch64_aspeed.py @@ -38,26 +38,28 @@ def test_aarch64_ast2700_evb_sdk_v09_02(self): archive_extract(image_path, self.workdir) =20 num_cpu =3D 4 - image_dir =3D self.workdir + '/ast2700-default/' - uboot_size =3D os.path.getsize(image_dir + 'u-boot-nodtb.bin') + uboot_size =3D os.path.getsize(self.scratch_file('ast2700-default', + 'u-boot-nodtb.bin')) uboot_dtb_load_addr =3D hex(0x400000000 + uboot_size) =20 load_images_list =3D [ { 'addr': '0x400000000', - 'file': image_dir + 'u-boot-nodtb.bin' + 'file': self.scratch_file('ast2700-default', + 'u-boot-nodtb.bin') }, { 'addr': str(uboot_dtb_load_addr), - 'file': image_dir + 'u-boot.dtb' + 'file': self.scratch_file('ast2700-default', 'u-boot.dtb') }, { 'addr': '0x430000000', - 'file': image_dir + 'bl31.bin' + 'file': self.scratch_file('ast2700-default', 'bl31.bin') }, { 'addr': '0x430080000', - 'file': image_dir + 'optee/tee-raw.bin' + 'file': self.scratch_file('ast2700-default', 'optee', + 'tee-raw.bin') } ] =20 @@ -74,7 +76,8 @@ def test_aarch64_ast2700_evb_sdk_v09_02(self): self.vm.add_args('-smp', str(num_cpu)) self.vm.add_args('-device', 'tmp105,bus=3Daspeed.i2c.bus.1,address=3D0x4d,id= =3Dtmp-test') - self.do_test_aarch64_aspeed_sdk_start(image_dir + 'image-bmc') + self.do_test_aarch64_aspeed_sdk_start( + self.scratch_file('ast2700-default', 'image-bmc')) =20 wait_for_console_pattern(self, 'ast2700-default login:') =20 diff --git a/tests/functional/test_aarch64_raspi3.py b/tests/functional/tes= t_aarch64_raspi3.py index 369f95a3d9..98ed6f9d56 100755 --- a/tests/functional/test_aarch64_raspi3.py +++ b/tests/functional/test_aarch64_raspi3.py @@ -7,7 +7,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os from zipfile import ZipFile =20 from qemu_test import LinuxKernelTest, Asset @@ -26,7 +25,7 @@ def test_aarch64_raspi3_atf(self): =20 with ZipFile(zip_path, 'r') as zf: zf.extract(efi_name, path=3Dself.workdir) - efi_fd =3D os.path.join(self.workdir, efi_name) + efi_fd =3D self.scratch_file(efi_name) =20 self.set_machine('raspi3b') self.vm.set_console(console_index=3D1) diff --git a/tests/functional/test_aarch64_raspi4.py b/tests/functional/tes= t_aarch64_raspi4.py index e5c9f77479..2cda03f86f 100755 --- a/tests/functional/test_aarch64_raspi4.py +++ b/tests/functional/test_aarch64_raspi4.py @@ -5,8 +5,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test.utils import gzip_uncompress @@ -64,7 +62,7 @@ def test_arm_raspi4_initrd(self): kernel_path =3D self.extract_from_deb(deb_path, '/boot/kernel8.img= ') dtb_path =3D self.extract_from_deb(deb_path, '/boot/bcm2711-rpi-4-= b.dtb') initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.set_machine('raspi4b') diff --git a/tests/functional/test_aarch64_sbsaref.py b/tests/functional/te= st_aarch64_sbsaref.py index 6db08da522..533ca64407 100755 --- a/tests/functional/test_aarch64_sbsaref.py +++ b/tests/functional/test_aarch64_sbsaref.py @@ -8,8 +8,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern from qemu_test import interrupt_interactive_console_until_pattern @@ -32,12 +30,12 @@ def fetch_firmware(test): =20 # Secure BootRom (TF-A code) fs0_xz_path =3D Aarch64SbsarefMachine.ASSET_FLASH0.fetch() - fs0_path =3D os.path.join(test.workdir, "SBSA_FLASH0.fd") + fs0_path =3D test.scratch_file("SBSA_FLASH0.fd") lzma_uncompress(fs0_xz_path, fs0_path) =20 # Non-secure rom (UEFI and EFI variables) fs1_xz_path =3D Aarch64SbsarefMachine.ASSET_FLASH1.fetch() - fs1_path =3D os.path.join(test.workdir, "SBSA_FLASH1.fd") + fs1_path =3D test.scratch_file("SBSA_FLASH1.fd") lzma_uncompress(fs1_xz_path, fs1_path) =20 for path in [fs0_path, fs1_path]: diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_= aarch64_virt.py index 5bc461b482..cc49f8963d 100755 --- a/tests/functional/test_aarch64_virt.py +++ b/tests/functional/test_aarch64_virt.py @@ -11,7 +11,6 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 import time -import os import logging =20 from qemu_test import QemuSystemTest, Asset @@ -95,7 +94,7 @@ def common_aarch64_virt(self, machine): =20 # Also add a scratch block device logger.info('creating scratch qcow2 image') - image_path =3D os.path.join(self.workdir, 'scratch.qcow2') + image_path =3D self.scratch_file('scratch.qcow2') qemu_img =3D get_qemu_img(self) run_cmd([qemu_img, 'create', '-f', 'qcow2', image_path, '8M']) =20 diff --git a/tests/functional/test_acpi_bits.py b/tests/functional/test_acp= i_bits.py index 59d0383563..3b99ecf3a4 100755 --- a/tests/functional/test_acpi_bits.py +++ b/tests/functional/test_acpi_bits.py @@ -135,9 +135,9 @@ def copy_bits_config(self): bits_config_file =3D self.data_file('acpi-bits', 'bits-config', 'bits-cfg.txt') - target_config_dir =3D os.path.join(self.workdir, - 'bits-%d' %self.BITS_INTERNAL_VER, - 'boot') + target_config_dir =3D self.scratch_file('bits-%d' % + self.BITS_INTERNAL_VER, + 'boot') self.assertTrue(os.path.exists(bits_config_file)) self.assertTrue(os.path.exists(target_config_dir)) shutil.copy2(bits_config_file, target_config_dir) @@ -148,9 +148,8 @@ def copy_test_scripts(self): """copies the python test scripts into bits. """ =20 bits_test_dir =3D self.data_file('acpi-bits', 'bits-tests') - target_test_dir =3D os.path.join(self.workdir, - 'bits-%d' %self.BITS_INTERNAL_VER, - 'boot', 'python') + target_test_dir =3D self.scratch_file('bits-%d' % self.BITS_INTERN= AL_VER, + 'boot', 'python') =20 self.assertTrue(os.path.exists(bits_test_dir)) self.assertTrue(os.path.exists(target_test_dir)) @@ -187,8 +186,8 @@ def fix_mkrescue(self, mkrescue): the directory where we have extracted our pre-built bits grub tarball. """ - grub_x86_64_mods =3D os.path.join(self.workdir, 'grub-inst-x86_64-= efi') - grub_i386_mods =3D os.path.join(self.workdir, 'grub-inst') + grub_x86_64_mods =3D self.scratch_file('grub-inst-x86_64-efi') + grub_i386_mods =3D self.scratch_file('grub-inst') =20 self.assertTrue(os.path.exists(grub_x86_64_mods)) self.assertTrue(os.path.exists(grub_i386_mods)) @@ -209,13 +208,11 @@ def generate_bits_iso(self): """ Uses grub-mkrescue to generate a fresh bits iso with the python test scripts """ - bits_dir =3D os.path.join(self.workdir, - 'bits-%d' %self.BITS_INTERNAL_VER) - iso_file =3D os.path.join(self.workdir, - 'bits-%d.iso' %self.BITS_INTERNAL_VER) - mkrescue_script =3D os.path.join(self.workdir, - 'grub-inst-x86_64-efi', 'bin', - 'grub-mkrescue') + bits_dir =3D self.scratch_file('bits-%d' % self.BITS_INTERNAL_VER) + iso_file =3D self.scratch_file('bits-%d.iso' % self.BITS_INTERNAL_= VER) + mkrescue_script =3D self.scratch_file('grub-inst-x86_64-efi', + 'bin', + 'grub-mkrescue') =20 self.assertTrue(os.access(mkrescue_script, os.R_OK | os.W_OK | os.X_OK)) @@ -250,17 +247,18 @@ def setUp(self): # pylint: disable=3Darguments-differ super().setUp() self.logger =3D self.log =20 - prebuiltDir =3D os.path.join(self.workdir, 'prebuilt') + prebuiltDir =3D self.scratch_file('prebuilt') if not os.path.isdir(prebuiltDir): os.mkdir(prebuiltDir, mode=3D0o775) =20 - bits_zip_file =3D os.path.join(prebuiltDir, 'bits-%d-%s.zip' - %(self.BITS_INTERNAL_VER, - self.BITS_COMMIT_HASH)) - grub_tar_file =3D os.path.join(prebuiltDir, - 'bits-%d-%s-grub.tar.gz' - %(self.BITS_INTERNAL_VER, - self.BITS_COMMIT_HASH)) + bits_zip_file =3D self.scratch_file('prebuilt', + 'bits-%d-%s.zip' + %(self.BITS_INTERNAL_VER, + self.BITS_COMMIT_HASH)) + grub_tar_file =3D self.scratch_file('prebuilt', + 'bits-%d-%s-grub.tar.gz' + %(self.BITS_INTERNAL_VER, + self.BITS_COMMIT_HASH)) =20 bitsLocalArtLoc =3D self.ASSET_BITS.fetch() self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLo= c) @@ -284,7 +282,7 @@ def parse_log(self): """parse the log generated by running bits tests and check for failures. """ - debugconf =3D os.path.join(self.workdir, self._debugcon_log) + debugconf =3D self.scratch_file(self._debugcon_log) log =3D "" with open(debugconf, 'r', encoding=3D'utf-8') as filehandle: log =3D filehandle.read() @@ -316,8 +314,7 @@ def test_acpi_smbios_bits(self): """The main test case implementation.""" =20 self.set_machine('pc') - iso_file =3D os.path.join(self.workdir, - 'bits-%d.iso' %self.BITS_INTERNAL_VER) + iso_file =3D self.scratch_file('bits-%d.iso' % self.BITS_INTERNAL_= VER) =20 self.assertTrue(os.access(iso_file, os.R_OK)) =20 diff --git a/tests/functional/test_alpha_clipper.py b/tests/functional/test= _alpha_clipper.py index c1fbf0e395..72cd7b57e6 100755 --- a/tests/functional/test_alpha_clipper.py +++ b/tests/functional/test_alpha_clipper.py @@ -5,8 +5,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - from qemu_test import LinuxKernelTest, Asset from qemu_test.utils import gzip_uncompress =20 @@ -22,7 +20,7 @@ def test_alpha_clipper(self): self.set_machine('clipper') kernel_path =3D self.ASSET_KERNEL.fetch() =20 - uncompressed_kernel =3D os.path.join(self.workdir, 'vmlinux') + uncompressed_kernel =3D self.scratch_file('vmlinux') gzip_uncompress(kernel_path, uncompressed_kernel) =20 self.vm.set_console() diff --git a/tests/functional/test_arm_aspeed_ast1030.py b/tests/functional= /test_arm_aspeed_ast1030.py index 380a76ec01..01b13c5591 100755 --- a/tests/functional/test_arm_aspeed_ast1030.py +++ b/tests/functional/test_arm_aspeed_ast1030.py @@ -6,8 +6,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern from zipfile import ZipFile @@ -27,7 +25,7 @@ def test_ast1030_zephyros_1_04(self): kernel_name =3D "ast1030-evb-demo/zephyr.elf" with ZipFile(zip_file, 'r') as zf: zf.extract(kernel_name, path=3Dself.workdir) - kernel_file =3D os.path.join(self.workdir, kernel_name) + kernel_file =3D self.scratch_file(kernel_name) =20 self.vm.set_console() self.vm.add_args('-kernel', kernel_file, '-nographic') @@ -49,7 +47,7 @@ def test_ast1030_zephyros_1_07(self): kernel_name =3D "ast1030-evb-demo/zephyr.bin" with ZipFile(zip_file, 'r') as zf: zf.extract(kernel_name, path=3Dself.workdir) - kernel_file =3D os.path.join(self.workdir, kernel_name) + kernel_file =3D self.scratch_file(kernel_name) =20 self.vm.set_console() self.vm.add_args('-kernel', kernel_file, '-nographic') diff --git a/tests/functional/test_arm_aspeed_ast2500.py b/tests/functional= /test_arm_aspeed_ast2500.py index 79baf37537..8c5593cdc8 100755 --- a/tests/functional/test_arm_aspeed_ast2500.py +++ b/tests/functional/test_arm_aspeed_ast2500.py @@ -50,7 +50,7 @@ def test_arm_ast2500_evb_sdk(self): archive_extract(image_path, self.workdir) =20 self.do_test_arm_aspeed_sdk_start( - self.workdir + '/ast2500-default/image-bmc') + self.scratch_file("ast2500-default", "image-bmc")) =20 self.wait_for_console_pattern('ast2500-default login:') =20 diff --git a/tests/functional/test_arm_aspeed_ast2600.py b/tests/functional= /test_arm_aspeed_ast2600.py index 9c749f96b4..25948c1c34 100755 --- a/tests/functional/test_arm_aspeed_ast2600.py +++ b/tests/functional/test_arm_aspeed_ast2600.py @@ -114,7 +114,7 @@ def test_arm_ast2600_evb_sdk(self): self.vm.add_args('-device', 'ds1338,bus=3Daspeed.i2c.bus.5,address=3D0x32'); self.do_test_arm_aspeed_sdk_start( - self.workdir + '/ast2600-a2/image-bmc') + self.scratch_file("ast2600-a2", "image-bmc")) =20 self.wait_for_console_pattern('ast2600-a2 login:') =20 diff --git a/tests/functional/test_arm_bflt.py b/tests/functional/test_arm_= bflt.py index 9095b08539..88ef7b1edc 100755 --- a/tests/functional/test_arm_bflt.py +++ b/tests/functional/test_arm_bflt.py @@ -6,7 +6,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os import bz2 =20 from qemu_test import QemuUserTest, Asset @@ -25,7 +24,7 @@ class LoadBFLT(QemuUserTest): def test_stm32(self): # See https://elinux.org/STM32#User_Space rootfs_path_bz2 =3D self.ASSET_ROOTFS.fetch() - busybox_path =3D os.path.join(self.workdir, "bin/busybox") + busybox_path =3D self.scratch_file("bin", "busybox") =20 with bz2.open(rootfs_path_bz2, 'rb') as cpio_handle: cpio_extract(cpio_handle, self.workdir) diff --git a/tests/functional/test_arm_bpim2u.py b/tests/functional/test_ar= m_bpim2u.py index fcd111f59d..2af6d9a18d 100755 --- a/tests/functional/test_arm_bpim2u.py +++ b/tests/functional/test_arm_bpim2u.py @@ -68,7 +68,7 @@ def test_arm_bpim2u_initrd(self): 'sun8i-r40-bananapi-m2-ultra.dtb') dtb_path =3D self.extract_from_deb(deb_path, dtb_path) initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.vm.set_console() @@ -106,7 +106,7 @@ def test_arm_bpim2u_gmac(self): 'sun8i-r40-bananapi-m2-ultra.dtb') dtb_path =3D self.extract_from_deb(deb_path, dtb_path) rootfs_path_xz =3D self.ASSET_ROOTFS.fetch() - rootfs_path =3D os.path.join(self.workdir, 'rootfs.cpio') + rootfs_path =3D self.scratch_file('rootfs.cpio') lzma_uncompress(rootfs_path_xz, rootfs_path) image_pow2ceil_expand(rootfs_path) =20 @@ -150,7 +150,7 @@ def test_arm_bpim2u_openwrt_22_03_3(self): # This test download a 8.9 MiB compressed image and expand it # to 127 MiB. image_path_gz =3D self.ASSET_SD_IMAGE.fetch() - image_path =3D os.path.join(self.workdir, 'sdcard.img') + image_path =3D self.scratch_file('sdcard.img') gzip_uncompress(image_path_gz, image_path) image_pow2ceil_expand(image_path) =20 diff --git a/tests/functional/test_arm_canona1100.py b/tests/functional/tes= t_arm_canona1100.py index 65f1228296..b4e3633422 100755 --- a/tests/functional/test_arm_canona1100.py +++ b/tests/functional/test_arm_canona1100.py @@ -31,7 +31,8 @@ def test_arm_canona1100(self): member=3D"day18/barebox.canon-a1100.bin") self.vm.set_console() self.vm.add_args('-bios', - self.workdir + '/day18/barebox.canon-a1100.bin') + self.scratch_file('day18', + 'barebox.canon-a1100.bin')) self.vm.launch() wait_for_console_pattern(self, 'running /env/bin/init') =20 diff --git a/tests/functional/test_arm_cubieboard.py b/tests/functional/tes= t_arm_cubieboard.py index fdbd52a33c..d81c333d0c 100755 --- a/tests/functional/test_arm_cubieboard.py +++ b/tests/functional/test_arm_cubieboard.py @@ -44,7 +44,7 @@ def test_arm_cubieboard_initrd(self): dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun4i-a10-= cubieboard.dtb' dtb_path =3D self.extract_from_deb(deb_path, dtb_path) initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.vm.set_console() @@ -78,7 +78,7 @@ def test_arm_cubieboard_sata(self): dtb_path =3D self.extract_from_deb(deb_path, dtb_path) =20 rootfs_path_gz =3D self.ASSET_SATA_ROOTFS.fetch() - rootfs_path =3D os.path.join(self.workdir, 'rootfs.cpio') + rootfs_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(rootfs_path_gz, rootfs_path) =20 self.vm.set_console() @@ -112,7 +112,7 @@ def test_arm_cubieboard_openwrt_22_03_2(self): # to 126 MiB. self.set_machine('cubieboard') image_path_gz =3D self.ASSET_OPENWRT.fetch() - image_path =3D os.path.join(self.workdir, 'sdcard.img') + image_path =3D self.scratch_file('sdcard.img') gzip_uncompress(image_path_gz, image_path) image_pow2ceil_expand(image_path) =20 diff --git a/tests/functional/test_arm_emcraft_sf2.py b/tests/functional/te= st_arm_emcraft_sf2.py index ada4dfd82e..f9f3f069e2 100755 --- a/tests/functional/test_arm_emcraft_sf2.py +++ b/tests/functional/test_arm_emcraft_sf2.py @@ -28,7 +28,7 @@ def test_arm_emcraft_sf2(self): =20 uboot_path =3D self.ASSET_UBOOT.fetch() spi_path =3D self.ASSET_SPI.fetch() - spi_path_rw =3D os.path.join(self.workdir, 'spi.bin') + spi_path_rw =3D self.scratch_file('spi.bin') shutil.copy(spi_path, spi_path_rw) os.chmod(spi_path_rw, 0o600) =20 diff --git a/tests/functional/test_arm_integratorcp.py b/tests/functional/t= est_arm_integratorcp.py index 54fa366ad4..a85b339d77 100755 --- a/tests/functional/test_arm_integratorcp.py +++ b/tests/functional/test_arm_integratorcp.py @@ -12,7 +12,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os import logging =20 from qemu_test import QemuSystemTest, Asset @@ -67,7 +66,7 @@ def test_framebuffer_tux_logo(self): import numpy as np import cv2 =20 - screendump_path =3D os.path.join(self.workdir, "screendump.pbm") + screendump_path =3D self.scratch_file("screendump.pbm") tuxlogo_path =3D self.ASSET_TUXLOGO.fetch() =20 self.boot_integratorcp() diff --git a/tests/functional/test_arm_orangepi.py b/tests/functional/test_= arm_orangepi.py index b113adfe61..bea67cfb6a 100755 --- a/tests/functional/test_arm_orangepi.py +++ b/tests/functional/test_arm_orangepi.py @@ -77,7 +77,7 @@ def test_arm_orangepi_initrd(self): dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun8i-h3-o= rangepi-pc.dtb' dtb_path =3D self.extract_from_deb(deb_path, dtb_path) initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.vm.set_console() @@ -113,7 +113,7 @@ def test_arm_orangepi_sd(self): dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun8i-h3-o= rangepi-pc.dtb' dtb_path =3D self.extract_from_deb(deb_path, dtb_path) rootfs_path_xz =3D self.ASSET_ROOTFS.fetch() - rootfs_path =3D os.path.join(self.workdir, 'rootfs.cpio') + rootfs_path =3D self.scratch_file('rootfs.cpio') lzma_uncompress(rootfs_path_xz, rootfs_path) image_pow2ceil_expand(rootfs_path) =20 @@ -156,7 +156,7 @@ def test_arm_orangepi_armbian(self): # to 1036 MiB, but the underlying filesystem is 1552 MiB... # As we expand it to 2 GiB we are safe. image_path_xz =3D self.ASSET_ARMBIAN.fetch() - image_path =3D os.path.join(self.workdir, 'armbian.img') + image_path =3D self.scratch_file('armbian.img') lzma_uncompress(image_path_xz, image_path) image_pow2ceil_expand(image_path) =20 @@ -197,7 +197,7 @@ def test_arm_orangepi_uboot_netbsd9(self): uboot_path =3D '/usr/lib/u-boot/orangepi_plus/u-boot-sunxi-with-sp= l.bin' uboot_path =3D self.extract_from_deb(deb_path, uboot_path) image_path_gz =3D self.ASSET_NETBSD.fetch() - image_path =3D os.path.join(self.workdir, 'armv7.img') + image_path =3D self.scratch_file('armv7.img') gzip_uncompress(image_path_gz, image_path) image_pow2ceil_expand(image_path) image_drive_args =3D 'if=3Dsd,format=3Draw,snapshot=3Don,file=3D' = + image_path diff --git a/tests/functional/test_arm_raspi2.py b/tests/functional/test_ar= m_raspi2.py index 3bf079dc4d..075f6b3301 100755 --- a/tests/functional/test_arm_raspi2.py +++ b/tests/functional/test_arm_raspi2.py @@ -7,8 +7,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test.utils import gzip_uncompress @@ -65,7 +63,7 @@ def test_arm_raspi2_initrd(self): kernel_path =3D self.extract_from_deb(deb_path, '/boot/kernel7.img= ') dtb_path =3D self.extract_from_deb(deb_path, '/boot/bcm2709-rpi-2-= b.dtb') initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.set_machine('raspi2b') diff --git a/tests/functional/test_arm_smdkc210.py b/tests/functional/test_= arm_smdkc210.py index b3b39b069d..c6c8f9a5f4 100755 --- a/tests/functional/test_arm_smdkc210.py +++ b/tests/functional/test_arm_smdkc210.py @@ -32,7 +32,7 @@ def test_arm_exynos4210_initrd(self): dtb_path =3D self.extract_from_deb(deb_path, dtb_path) =20 initrd_path_gz =3D self.ASSET_ROOTFS.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.vm.set_console() diff --git a/tests/functional/test_arm_vexpress.py b/tests/functional/test_= arm_vexpress.py index 6bd6290030..b1ac63ac36 100755 --- a/tests/functional/test_arm_vexpress.py +++ b/tests/functional/test_arm_vexpress.py @@ -18,8 +18,9 @@ def test_arm_vexpressa9(self): self.set_machine('vexpress-a9') file_path =3D self.ASSET_DAY16.fetch() archive_extract(file_path, self.workdir) - self.launch_kernel(self.workdir + '/day16/winter.zImage', - dtb=3Dself.workdir + '/day16/vexpress-v2p-ca9.d= tb', + self.launch_kernel(self.scratch_file('day16', 'winter.zImage'), + dtb=3Dself.scratch_file('day16', + 'vexpress-v2p-ca9.dtb'), wait_for=3D'QEMU advent calendar') =20 if __name__ =3D=3D '__main__': diff --git a/tests/functional/test_m68k_mcf5208evb.py b/tests/functional/te= st_m68k_mcf5208evb.py index fb178fde1c..449248c3e8 100755 --- a/tests/functional/test_m68k_mcf5208evb.py +++ b/tests/functional/test_m68k_mcf5208evb.py @@ -19,7 +19,8 @@ def test_m68k_mcf5208evb(self): file_path =3D self.ASSET_DAY07.fetch() archive_extract(file_path, self.workdir) self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/day07/sanity-clause.e= lf') + self.vm.add_args('-kernel', + self.scratch_file('day07', 'sanity-clause.elf')) self.vm.launch() self.wait_for_console_pattern('QEMU advent calendar') =20 diff --git a/tests/functional/test_m68k_nextcube.py b/tests/functional/test= _m68k_nextcube.py index 25a17d4794..ff773a7994 100755 --- a/tests/functional/test_m68k_nextcube.py +++ b/tests/functional/test_m68k_nextcube.py @@ -7,7 +7,6 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import os import time =20 from qemu_test import QemuSystemTest, Asset @@ -39,7 +38,7 @@ def check_bootrom_framebuffer(self, screenshot_path): @skipIfMissingImports("PIL") def test_bootrom_framebuffer_size(self): self.set_machine('next-cube') - screenshot_path =3D os.path.join(self.workdir, "dump.ppm") + screenshot_path =3D self.scratch_file("dump.ppm") self.check_bootrom_framebuffer(screenshot_path) =20 from PIL import Image @@ -50,7 +49,7 @@ def test_bootrom_framebuffer_size(self): @skipIfMissingCommands('tesseract') def test_bootrom_framebuffer_ocr_with_tesseract(self): self.set_machine('next-cube') - screenshot_path =3D os.path.join(self.workdir, "dump.ppm") + screenshot_path =3D self.scratch_file("dump.ppm") self.check_bootrom_framebuffer(screenshot_path) lines =3D tesseract_ocr(screenshot_path) text =3D '\n'.join(lines) diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/functio= nal/test_microblaze_s3adsp1800.py index d452a0271c..61c4d6bbf8 100755 --- a/tests/functional/test_microblaze_s3adsp1800.py +++ b/tests/functional/test_microblaze_s3adsp1800.py @@ -25,7 +25,8 @@ def test_microblaze_s3adsp1800(self): file_path =3D self.ASSET_IMAGE.fetch() archive_extract(file_path, self.workdir) self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/day17/ballerina.bin') + self.vm.add_args('-kernel', + self.scratch_file('day17', 'ballerina.bin')) self.vm.launch() wait_for_console_pattern(self, 'This architecture does not have ' 'kernel memory protection') diff --git a/tests/functional/test_microblazeel_s3adsp1800.py b/tests/funct= ional/test_microblazeel_s3adsp1800.py index faa3927f2e..926c885f63 100755 --- a/tests/functional/test_microblazeel_s3adsp1800.py +++ b/tests/functional/test_microblazeel_s3adsp1800.py @@ -27,8 +27,9 @@ def test_microblazeel_s3adsp1800(self): file_path =3D self.ASSET_IMAGE.fetch() archive_extract(file_path, self.workdir) self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/day13/xmaton.bin') - self.vm.add_args('-nic', 'user,tftp=3D' + self.workdir + '/day13/') + self.vm.add_args('-kernel', self.scratch_file('day13', 'xmaton.bin= ')) + tftproot =3D self.scratch_file('day13') + self.vm.add_args('-nic', f'user,tftp=3D{tftproot}') self.vm.launch() wait_for_console_pattern(self, 'QEMU Advent Calendar 2023') time.sleep(0.1) diff --git a/tests/functional/test_mips64el_malta.py b/tests/functional/tes= t_mips64el_malta.py index 52283e2dbd..ea362cf335 100755 --- a/tests/functional/test_mips64el_malta.py +++ b/tests/functional/test_mips64el_malta.py @@ -68,7 +68,7 @@ def test_mips64el_malta(self): def test_mips64el_malta_5KEc_cpio(self): kernel_path =3D self.ASSET_KERNEL_3_19_3.fetch() initrd_path_gz =3D self.ASSET_CPIO_R1.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.set_machine('malta') @@ -117,10 +117,10 @@ def do_test_i6400_framebuffer_logo(self, cpu_cores_co= unt): import numpy as np import cv2 =20 - screendump_path =3D os.path.join(self.workdir, 'screendump.pbm') + screendump_path =3D self.scratch_file('screendump.pbm') =20 kernel_path_gz =3D self.ASSET_KERNEL_4_7_0.fetch() - kernel_path =3D self.workdir + "/vmlinux" + kernel_path =3D self.scratch_file("vmlinux") gzip_uncompress(kernel_path_gz, kernel_path) =20 tuxlogo_path =3D self.ASSET_TUXLOGO.fetch() diff --git a/tests/functional/test_mips_malta.py b/tests/functional/test_mi= ps_malta.py index a012081382..a6d80d0012 100755 --- a/tests/functional/test_mips_malta.py +++ b/tests/functional/test_mips_malta.py @@ -6,8 +6,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test.utils import gzip_uncompress @@ -52,7 +50,7 @@ def test_mips_malta_cpio(self): kernel_path =3D self.extract_from_deb(deb_path, '/boot/vmlinux-4.5.0-2-4kc-mal= ta') initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'rootfs.cpio') + initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) =20 self.set_machine('malta') diff --git a/tests/functional/test_mipsel_malta.py b/tests/functional/test_= mipsel_malta.py index b8dfddd856..77671e0081 100755 --- a/tests/functional/test_mipsel_malta.py +++ b/tests/functional/test_mipsel_malta.py @@ -9,8 +9,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os - from qemu_test import QemuSystemTest, LinuxKernelTest, Asset from qemu_test import interrupt_interactive_console_until_pattern from qemu_test import wait_for_console_pattern @@ -37,7 +35,7 @@ class MaltaMachineConsole(LinuxKernelTest): 'ce21ff4b07a981ecb8a39db2876616f5a2473eb2ab459c6f67465b9914b0c6b6') =20 def do_test_mips_malta32el_nanomips(self, kernel_path_xz): - kernel_path =3D os.path.join(self.workdir, 'kernel') + kernel_path =3D self.scratch_file('kernel') lzma_uncompress(kernel_path_xz, kernel_path) =20 self.set_machine('malta') @@ -78,7 +76,7 @@ def test_mipsel_malta_yamon(self): zip_path =3D self.ASSET_YAMON_ROM.fetch() with ZipFile(zip_path, 'r') as zf: zf.extract(yamon_bin, path=3Dself.workdir) - yamon_path =3D os.path.join(self.workdir, yamon_bin) + yamon_path =3D self.scratch_file(yamon_bin) =20 self.set_machine('malta') self.vm.set_console() diff --git a/tests/functional/test_or1k_sim.py b/tests/functional/test_or1k= _sim.py index 5b68b6b628..a5b2b5b1e5 100755 --- a/tests/functional/test_or1k_sim.py +++ b/tests/functional/test_or1k_sim.py @@ -19,7 +19,7 @@ def test_or1k_sim(self): file_path =3D self.ASSET_DAY20.fetch() archive_extract(file_path, self.workdir) self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/day20/vmlinux') + self.vm.add_args('-kernel', self.scratch_file('day20', 'vmlinux')) self.vm.launch() self.wait_for_console_pattern('QEMU advent calendar') =20 diff --git a/tests/functional/test_ppc64_e500.py b/tests/functional/test_pp= c64_e500.py index f1af92373e..bf4a6af9d4 100755 --- a/tests/functional/test_ppc64_e500.py +++ b/tests/functional/test_ppc64_e500.py @@ -18,7 +18,7 @@ def test_ppc64_e500(self): self.cpu =3D 'e5500' file_path =3D self.ASSET_DAY19.fetch() archive_extract(file_path, self.workdir) - self.launch_kernel(self.workdir + '/day19/uImage', + self.launch_kernel(self.scratch_file('day19', 'uImage'), wait_for=3D'QEMU advent calendar') =20 if __name__ =3D=3D '__main__': diff --git a/tests/functional/test_ppc64_hv.py b/tests/functional/test_ppc6= 4_hv.py index 88f0f99f24..037dfdf87e 100755 --- a/tests/functional/test_ppc64_hv.py +++ b/tests/functional/test_ppc64_hv.py @@ -46,23 +46,15 @@ def extract_from_iso(self, iso, path): :param path: path within the iso file of the file to be extracted :returns: path of the extracted file """ - filename =3D os.path.basename(path) - - cwd =3D os.getcwd() - os.chdir(self.workdir) + filename =3D self.scratch_file(os.path.basename(path)) =20 cmd =3D "xorriso -osirrox on -indev %s -cpx %s %s" % (iso, path, f= ilename) subprocess.run(cmd.split(), stdout=3Dsubprocess.DEVNULL, stderr=3Dsubprocess.DE= VNULL) =20 os.chmod(filename, 0o600) - os.chdir(cwd) =20 - # Return complete path to extracted file. Because callers to - # extract_from_iso() specify 'path' with a leading slash, it is - # necessary to use os.path.relpath() as otherwise os.path.join() - # interprets it as an absolute path and drops the self.workdir par= t. - return os.path.normpath(os.path.join(self.workdir, filename)) + return filename =20 def setUp(self): super().setUp() diff --git a/tests/functional/test_ppc_amiga.py b/tests/functional/test_ppc= _amiga.py index f5faa0f0b3..9ed23a1f0f 100755 --- a/tests/functional/test_ppc_amiga.py +++ b/tests/functional/test_ppc_amiga.py @@ -29,13 +29,15 @@ def test_ppc_amigaone(self): zip_file =3D self.ASSET_IMAGE.fetch() with ZipFile(zip_file, 'r') as zf: zf.extractall(path=3Dself.workdir) - bios_fh =3D open(self.workdir + "/u-boot-amigaone.bin", "wb") - subprocess.run(['tail', '-c', '524288', - self.workdir + "/floppy_edition/updater.image"], - stdout=3Dbios_fh) + bios =3D self.scratch_file("u-boot-amigaone.bin") + with open(bios, "wb") as bios_fh: + subprocess.run(['tail', '-c', '524288', + self.scratch_file("floppy_edition", + "updater.image")], + stdout=3Dbios_fh) =20 self.vm.set_console() - self.vm.add_args('-bios', self.workdir + '/u-boot-amigaone.bin') + self.vm.add_args('-bios', bios) self.vm.launch() wait_for_console_pattern(self, 'FLASH:') =20 diff --git a/tests/functional/test_ppc_bamboo.py b/tests/functional/test_pp= c_bamboo.py index e72cbdee12..1ae2f47bcc 100755 --- a/tests/functional/test_ppc_bamboo.py +++ b/tests/functional/test_ppc_bamboo.py @@ -28,10 +28,12 @@ def test_ppc_bamboo(self): file_path =3D self.ASSET_IMAGE.fetch() archive_extract(file_path, self.workdir) self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + - '/system-image-powerpc-440fp/linux', - '-initrd', self.workdir + - '/system-image-powerpc-440fp/rootfs.cpi= o.gz', + self.vm.add_args('-kernel', + self.scratch_file('system-image-powerpc-440fp', + 'linux'), + '-initrd', + self.scratch_file('system-image-powerpc-440fp', + 'rootfs.cpio.gz'), '-nic', 'user,model=3Drtl8139,restrict=3Don') self.vm.launch() wait_for_console_pattern(self, 'Type exit when done') diff --git a/tests/functional/test_ppc_mac.py b/tests/functional/test_ppc_m= ac.py index 3f45e37a45..10812824bd 100755 --- a/tests/functional/test_ppc_mac.py +++ b/tests/functional/test_ppc_mac.py @@ -23,7 +23,7 @@ def do_day15_test(self): file_path =3D self.ASSET_DAY15.fetch() archive_extract(file_path, self.workdir) self.vm.add_args('-M', 'graphics=3Doff') - self.launch_kernel(self.workdir + '/day15/invaders.elf', + self.launch_kernel(self.scratch_file('day15', 'invaders.elf'), wait_for=3D'QEMU advent calendar') =20 def test_ppc_g3beige(self): diff --git a/tests/functional/test_ppc_mpc8544ds.py b/tests/functional/test= _ppc_mpc8544ds.py index 2b3f0894ae..87b5d4d12b 100755 --- a/tests/functional/test_ppc_mpc8544ds.py +++ b/tests/functional/test_ppc_mpc8544ds.py @@ -28,7 +28,7 @@ def test_ppc_mpc8544ds(self): file_path =3D self.ASSET_IMAGE.fetch() archive_extract(file_path, self.workdir, member=3D'creek/creek.bin= ') self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/creek/creek.bin') + self.vm.add_args('-kernel', self.scratch_file('creek', 'creek.bin'= )) self.vm.launch() wait_for_console_pattern(self, 'QEMU advent calendar 2020', self.panic_message) diff --git a/tests/functional/test_ppc_virtex_ml507.py b/tests/functional/t= est_ppc_virtex_ml507.py index ffa9a0633e..f297651e64 100755 --- a/tests/functional/test_ppc_virtex_ml507.py +++ b/tests/functional/test_ppc_virtex_ml507.py @@ -28,8 +28,9 @@ def test_ppc_virtex_ml507(self): file_path =3D self.ASSET_IMAGE.fetch() archive_extract(file_path, self.workdir) self.vm.set_console() - self.vm.add_args('-kernel', self.workdir + '/hippo/hippo.linux', - '-dtb', self.workdir + '/hippo/virtex440-ml507.dt= b', + self.vm.add_args('-kernel', self.scratch_file('hippo', 'hippo.linu= x'), + '-dtb', self.scratch_file('hippo', + 'virtex440-ml507.dtb'), '-m', '512') self.vm.launch() wait_for_console_pattern(self, 'QEMU advent calendar 2020', diff --git a/tests/functional/test_rx_gdbsim.py b/tests/functional/test_rx_= gdbsim.py index f7621d999d..b0adb38a45 100755 --- a/tests/functional/test_rx_gdbsim.py +++ b/tests/functional/test_rx_gdbsim.py @@ -10,8 +10,6 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import os - from qemu_test import QemuSystemTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test import wait_for_console_pattern, skipFlakyTest @@ -40,7 +38,7 @@ def test_uboot(self): self.set_machine('gdbsim-r5f562n8') =20 uboot_path_gz =3D self.ASSET_UBOOT.fetch() - uboot_path =3D os.path.join(self.workdir, 'u-boot.bin') + uboot_path =3D self.scratch_file('u-boot.bin') gzip_uncompress(uboot_path_gz, uboot_path) =20 self.vm.set_console() diff --git a/tests/functional/test_s390x_ccw_virtio.py b/tests/functional/t= est_s390x_ccw_virtio.py index f7acd90a89..e5884a4dd0 100755 --- a/tests/functional/test_s390x_ccw_virtio.py +++ b/tests/functional/test_s390x_ccw_virtio.py @@ -175,7 +175,7 @@ def test_s390x_fedora(self): kernel_path =3D self.ASSET_F31_KERNEL.fetch() =20 initrd_path_xz =3D self.ASSET_F31_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'initrd-raw.img') + initrd_path =3D self.scratch_file('initrd-raw.img') lzma_uncompress(initrd_path_xz, initrd_path) =20 self.vm.set_console() diff --git a/tests/functional/test_s390x_topology.py b/tests/functional/tes= t_s390x_topology.py index c54c7a8177..82acff4e55 100755 --- a/tests/functional/test_s390x_topology.py +++ b/tests/functional/test_s390x_topology.py @@ -10,8 +10,6 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import os - from qemu_test import QemuSystemTest, Asset from qemu_test import exec_command from qemu_test import exec_command_and_wait_for_pattern @@ -89,7 +87,7 @@ def kernel_init(self): self.require_accelerator("kvm") kernel_path =3D self.ASSET_F35_KERNEL.fetch() initrd_path_xz =3D self.ASSET_F35_INITRD.fetch() - initrd_path =3D os.path.join(self.workdir, 'initrd-raw.img') + initrd_path =3D self.scratch_file('initrd-raw.img') lzma_uncompress(initrd_path_xz, initrd_path) =20 self.vm.set_console() diff --git a/tests/functional/test_sh4_r2d.py b/tests/functional/test_sh4_r= 2d.py index e2fcde2d6b..dca4601392 100755 --- a/tests/functional/test_sh4_r2d.py +++ b/tests/functional/test_sh4_r2d.py @@ -22,7 +22,8 @@ def test_r2d(self): file_path =3D self.ASSET_DAY09.fetch() archive_extract(file_path, self.workdir) self.vm.add_args('-append', 'console=3DttySC1') - self.launch_kernel(self.workdir + '/day09/zImage', console_index= =3D1, + self.launch_kernel(self.scratch_file('day09', 'zImage'), + console_index=3D1, wait_for=3D'QEMU advent calendar') =20 if __name__ =3D=3D '__main__': diff --git a/tests/functional/test_sh4eb_r2d.py b/tests/functional/test_sh4= eb_r2d.py index cd46007942..b8dadabf3c 100755 --- a/tests/functional/test_sh4eb_r2d.py +++ b/tests/functional/test_sh4eb_r2d.py @@ -4,7 +4,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import os import shutil =20 from qemu_test import LinuxKernelTest, Asset @@ -22,11 +21,12 @@ def test_sh4eb_r2d(self): file_path =3D self.ASSET_TGZ.fetch() archive_extract(file_path, self.workdir) self.vm.add_args('-append', 'console=3DttySC1 noiotrap') - self.launch_kernel(os.path.join(self.workdir, 'sh4eb/linux-kernel'= ), - initrd=3Dos.path.join(self.workdir, 'sh4eb/init= ramfs.cpio.gz'), + self.launch_kernel(self.scratch_file('sh4eb', 'linux-kernel'), + initrd=3Dself.scratch_file('sh4eb', + 'initramfs.cpio.gz'), console_index=3D1, wait_for=3D'Type exit when d= one') exec_command_and_wait_for_pattern(self, 'exit', 'Restarting system= ') - shutil.rmtree(os.path.join(self.workdir, 'sh4eb')) + shutil.rmtree(self.scratch_file('sh4eb')) =20 if __name__ =3D=3D '__main__': LinuxKernelTest.main() diff --git a/tests/functional/test_sparc64_sun4u.py b/tests/functional/test= _sparc64_sun4u.py index 32e245f4ad..e7f6db0f24 100755 --- a/tests/functional/test_sparc64_sun4u.py +++ b/tests/functional/test_sparc64_sun4u.py @@ -10,12 +10,11 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import os - from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern from qemu_test.utils import archive_extract =20 + class Sun4uMachine(QemuSystemTest): """Boots the Linux kernel and checks that the console is operational""" =20 @@ -32,7 +31,7 @@ def test_sparc64_sun4u(self): kernel_name =3D 'day23/vmlinux' archive_extract(file_path, self.workdir, kernel_name) self.vm.set_console() - self.vm.add_args('-kernel', os.path.join(self.workdir, kernel_name= ), + self.vm.add_args('-kernel', self.scratch_file(kernel_name), '-append', 'printk.time=3D0') self.vm.launch() wait_for_console_pattern(self, 'Starting logging: OK') diff --git a/tests/functional/test_sparc_sun4m.py b/tests/functional/test_s= parc_sun4m.py index 573f85222a..619c03d36a 100755 --- a/tests/functional/test_sparc_sun4m.py +++ b/tests/functional/test_sparc_sun4m.py @@ -18,7 +18,7 @@ def test_sparc_ss20(self): self.set_machine('SS-20') file_path =3D self.ASSET_DAY11.fetch() archive_extract(file_path, self.workdir) - self.launch_kernel(self.workdir + '/day11/zImage.elf', + self.launch_kernel(self.scratch_file('day11', 'zImage.elf'), wait_for=3D'QEMU advent calendar') =20 if __name__ =3D=3D '__main__': diff --git a/tests/functional/test_xtensa_lx60.py b/tests/functional/test_x= tensa_lx60.py index d4ad92dc6c..5048e4c69e 100755 --- a/tests/functional/test_xtensa_lx60.py +++ b/tests/functional/test_xtensa_lx60.py @@ -19,7 +19,8 @@ def test_xtensa_lx60(self): self.cpu =3D 'dc233c' file_path =3D self.ASSET_DAY02.fetch() archive_extract(file_path, self.workdir) - self.launch_kernel(self.workdir + '/day02/santas-sleigh-ride.elf', + self.launch_kernel(self.scratch_file('day02', + 'santas-sleigh-ride.elf'), wait_for=3D'QEMU advent calendar') =20 if __name__ =3D=3D '__main__': --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451881; cv=none; d=zohomail.com; s=zohoarc; b=PO5Msuwj2tItDzs4naesBMpQOydQHHpJEPMEELrfSAvn9WaYIvh5cy63QeEXQJHo6OD5XwDraBpwHXeiuKptvqKRzCZQBZAiX3u4b3tkh5dBruuOXlpPrPRp2geeVTI9sQZc4Wz4jaELhOhw1fb+Xj/hRBAAYrJYBXeXt87dNhM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451881; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=2mGODSMuPQz9KoSxfoSlSzJp+nFYF+zH6/PXo6OcUQA=; b=Zk/dpToh6uw/Oj/hQ2U2HuUdIBxaY9E5lR3sU/35yuZRvcFokY5sbN5/mBkyk1FQEScnF367Ew3i/kN2YmokOfbVON+SQXdIefWegLXVw1x4rGTDBu99BIBT3RWirJfAP+ww3V4y3q8d/ZivuOGv1HJtzLzItpwYvPOavg2UsW4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451881810405.78989527495423; Tue, 17 Dec 2024 08:11:21 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0f-00057t-C0; Tue, 17 Dec 2024 11:01:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0G-0004Qx-LJ for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:00 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0B-0002AC-Fz for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:53 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-595-vLSkMs52Ob6KEwrXK4FOmw-1; Tue, 17 Dec 2024 11:00:46 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 72267195608B; Tue, 17 Dec 2024 16:00:45 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F162A30044C1; Tue, 17 Dec 2024 16:00:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451250; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2mGODSMuPQz9KoSxfoSlSzJp+nFYF+zH6/PXo6OcUQA=; b=Rik4VOFfjMQCqVuC7VgX6eGT8HeKY5lt7k3MYeyXeupKK17S7YnfF1mhRDJVGsP055gqv3 ovjZPIo5Zazum2+u1Lh8Tlqkau3OWvHbykE7bDNX2BmdCa0yJu3V7V4FvKnFFB6iJJu3pM CgH1TBN3U269iqE9KFB1b80O0HzgHNg= X-MC-Unique: vLSkMs52Ob6KEwrXK4FOmw-1 X-Mimecast-MFC-AGG-ID: vLSkMs52Ob6KEwrXK4FOmw From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 15/32] tests/functional: remove redundant 'rmtree' call Date: Tue, 17 Dec 2024 15:59:36 +0000 Message-ID: <20241217155953.3950506-16-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451882736116600 Everything in the scratch directory is automatically purged. Calling 'rmtree' again breaks the ability to optionally preserve the scratch directory contents. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/test_sh4eb_r2d.py | 3 --- 1 file changed, 3 deletions(-) diff --git a/tests/functional/test_sh4eb_r2d.py b/tests/functional/test_sh4= eb_r2d.py index b8dadabf3c..c8954c93eb 100755 --- a/tests/functional/test_sh4eb_r2d.py +++ b/tests/functional/test_sh4eb_r2d.py @@ -4,8 +4,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -import shutil - from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test.utils import archive_extract @@ -26,7 +24,6 @@ def test_sh4eb_r2d(self): 'initramfs.cpio.gz'), console_index=3D1, wait_for=3D'Type exit when d= one') exec_command_and_wait_for_pattern(self, 'exit', 'Restarting system= ') - shutil.rmtree(self.scratch_file('sh4eb')) =20 if __name__ =3D=3D '__main__': LinuxKernelTest.main() --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451428; cv=none; d=zohomail.com; s=zohoarc; b=gyGG3Hi2qvzGkOTA9qtHnj8jri9R/MdH/AiWrP3gHKS5xKv3gv8OX4juKzebL4fXIr8HR+3AvztV8oTFIctxFVRBL0D8lbK+XhlJhpFJRDwXMtztDZr/Yi8s0LTsSGl0qK6KSb1QNs5oWHjpdpT7j3zrKDzwd9YaW3gpRP+DtXI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451428; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZqMTiiD0E5HRxNd/+TqSUc1lmXPbcBRNJkAvf7IW0KA=; b=E7ta1DEmnO6RzLN/2Xoq/FYCNI5XelzQOg9kkrX872SnldYS5BGK420spLs159kwaJWu+TOfZT5VxigRSvSvaKmKJ2drolWXmXsjlrelNPm+r1lFM0K3ZxK81boKNzxyxD4MXPP2k2MvXw8AD4CzSJEuPU2z8OcdWY1hc+BMeyk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451428801123.56968207513512; Tue, 17 Dec 2024 08:03:48 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0b-0004qj-3N; Tue, 17 Dec 2024 11:01:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0H-0004R3-Pq for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0D-0002AR-Ec for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:56 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-22-HGdXTIe9OMqHmTScjQfeRg-1; Tue, 17 Dec 2024 11:00:49 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 399FE19560A3; Tue, 17 Dec 2024 16:00:48 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DB45C300F9B5; Tue, 17 Dec 2024 16:00:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZqMTiiD0E5HRxNd/+TqSUc1lmXPbcBRNJkAvf7IW0KA=; b=DbuK7A56tjP/2OTFMw/fdgzf5zdmbv09xCjC3egCPS1A28EyBskSTXUR1pAs90U/qYgXp1 0fwHPDZ5HhScHPs8XUXXxBXpjanDiFAQ6x2fuV/9Gjf0Fa/BwdIZux5bPu4gGPphoLtquH EBXwPHI3kkEMaXVd2z/mwNDsj689eVc= X-MC-Unique: HGdXTIe9OMqHmTScjQfeRg-1 X-Mimecast-MFC-AGG-ID: HGdXTIe9OMqHmTScjQfeRg From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 16/32] tests/functional: move archive handling into new archive.py file Date: Tue, 17 Dec 2024 15:59:37 +0000 Message-ID: <20241217155953.3950506-17-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451429457116600 More archive related code will be added shortly, so having a separate file makes more sense. The utils.py imports the functions from archive.py, so that existing callers don't need to be modified. This avoids redundant code churn until later in the series when all calls will be adapted for other reasons. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/archive.py | 31 +++++++++++++++++++++++++++ tests/functional/qemu_test/utils.py | 23 +++----------------- 2 files changed, 34 insertions(+), 20 deletions(-) create mode 100644 tests/functional/qemu_test/archive.py diff --git a/tests/functional/qemu_test/archive.py b/tests/functional/qemu_= test/archive.py new file mode 100644 index 0000000000..9872f08d23 --- /dev/null +++ b/tests/functional/qemu_test/archive.py @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Utilities for python-based QEMU tests +# +# Copyright 2024 Red Hat, Inc. +# +# Authors: +# Thomas Huth + +import os +import subprocess +import tarfile + + +def tar_extract(archive, dest_dir, member=3DNone): + with tarfile.open(archive) as tf: + if hasattr(tarfile, 'data_filter'): + tf.extraction_filter =3D getattr(tarfile, 'data_filter', + (lambda member, path: member)) + if member: + tf.extract(member=3Dmember, path=3Ddest_dir) + else: + tf.extractall(path=3Ddest_dir) + +def cpio_extract(cpio_handle, output_path): + cwd =3D os.getcwd() + os.chdir(output_path) + subprocess.run(['cpio', '-i'], + input=3Dcpio_handle.read(), + stderr=3Dsubprocess.DEVNULL) + os.chdir(cwd) diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_te= st/utils.py index 1bf1c410d5..5ce1c4388e 100644 --- a/tests/functional/qemu_test/utils.py +++ b/tests/functional/qemu_test/utils.py @@ -12,8 +12,9 @@ import lzma import os import shutil -import subprocess -import tarfile + +from .archive import tar_extract as archive_extract +from .archive import cpio_extract =20 """ Round up to next power of 2 @@ -36,16 +37,6 @@ def image_pow2ceil_expand(path): with open(path, 'ab+') as fd: fd.truncate(size_aligned) =20 -def archive_extract(archive, dest_dir, member=3DNone): - with tarfile.open(archive) as tf: - if hasattr(tarfile, 'data_filter'): - tf.extraction_filter =3D getattr(tarfile, 'data_filter', - (lambda member, path: member)) - if member: - tf.extract(member=3Dmember, path=3Ddest_dir) - else: - tf.extractall(path=3Ddest_dir) - def gzip_uncompress(gz_path, output_path): if os.path.exists(output_path): return @@ -67,11 +58,3 @@ def lzma_uncompress(xz_path, output_path): except: os.remove(output_path) raise - -def cpio_extract(cpio_handle, output_path): - cwd =3D os.getcwd() - os.chdir(output_path) - subprocess.run(['cpio', '-i'], - input=3Dcpio_handle.read(), - stderr=3Dsubprocess.DEVNULL) - os.chdir(cwd) --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451780; cv=none; d=zohomail.com; s=zohoarc; b=WQzK9+zdQeY9RW2dnk90fldICgeMBz5O8IGZzxF2jLjLsl8ZoiUkrubz59EoWOFvLqteinX31wkuNqGCaD/TTK/3RrGVFMLkLZCPWZq7J4cCb3xoqn82qD/hkKth8SzeY6IUFTuO4BEuHiEBqjJv1ugB4DG2FgnL2xaW1H+9Of4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451780; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6LK8etIFWDJ/LjbLZxdN9FkLtAHDK52IAP39LAri79E=; b=fGqbK/YJAxViYaD4a+dd2GoVVkQP5TYva+GY4aZo3JZWCwjFv0Rv3QTyLSJl8eTNp9cgtClMa/Exe3yUY6ExTVKD8D11r5fayf1s3j0yWlIkytQqul/9jurJvibNdAx//4hzxnqeycEalbsyputfBbPwbNHoIg4+WCictwkC6a4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451780198135.98323271890501; Tue, 17 Dec 2024 08:09:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0c-0004zI-Jg; Tue, 17 Dec 2024 11:01:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0J-0004RA-76 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0H-0002Am-J8 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:00:58 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-475-I9Yd9n8cOGCra9ej9LdFZA-1; Tue, 17 Dec 2024 11:00:51 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C7AB51956087; Tue, 17 Dec 2024 16:00:50 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B51DF300F9B5; Tue, 17 Dec 2024 16:00:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451256; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6LK8etIFWDJ/LjbLZxdN9FkLtAHDK52IAP39LAri79E=; b=Yd4quX0mdUkjZpPejQGsCiR63FqNMcBoUHm0gtMDmm6CsC9/x6y2oYiugoe7BpHOm/mrzS qUlsxxrQhh9W+goUNBf3e5QiXcto0hN0zh3xgqZX5md8S7HCx6zvF8yvc+HMvO6MP8KzRb GzNrGU/B9DOusVXl9Po79RAXe0+s69s= X-MC-Unique: I9Yd9n8cOGCra9ej9LdFZA-1 X-Mimecast-MFC-AGG-ID: I9Yd9n8cOGCra9ej9LdFZA From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 17/32] tests/functional: move uncompress handling into new uncompress.py file Date: Tue, 17 Dec 2024 15:59:38 +0000 Message-ID: <20241217155953.3950506-18-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451782384116600 More uncompress related code will be added shortly, so having a separate file makes more sense. The utils.py imports the functions from archive.py, so that existing callers don't need to be modified. This avoids redundant code churn until later in the series when all calls will be adapted for other reasons. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/uncompress.py | 36 ++++++++++++++++++++++++ tests/functional/qemu_test/utils.py | 27 ++---------------- 2 files changed, 38 insertions(+), 25 deletions(-) create mode 100644 tests/functional/qemu_test/uncompress.py diff --git a/tests/functional/qemu_test/uncompress.py b/tests/functional/qe= mu_test/uncompress.py new file mode 100644 index 0000000000..955170df65 --- /dev/null +++ b/tests/functional/qemu_test/uncompress.py @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Utilities for python-based QEMU tests +# +# Copyright 2024 Red Hat, Inc. +# +# Authors: +# Thomas Huth + +import gzip +import lzma +import os +import shutil + + +def gzip_uncompress(gz_path, output_path): + if os.path.exists(output_path): + return + with gzip.open(gz_path, 'rb') as gz_in: + try: + with open(output_path, 'wb') as raw_out: + shutil.copyfileobj(gz_in, raw_out) + except: + os.remove(output_path) + raise + +def lzma_uncompress(xz_path, output_path): + if os.path.exists(output_path): + return + with lzma.open(xz_path, 'rb') as lzma_in: + try: + with open(output_path, 'wb') as raw_out: + shutil.copyfileobj(lzma_in, raw_out) + except: + os.remove(output_path) + raise diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_te= st/utils.py index 5ce1c4388e..6b87af4414 100644 --- a/tests/functional/qemu_test/utils.py +++ b/tests/functional/qemu_test/utils.py @@ -8,13 +8,12 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import gzip -import lzma import os -import shutil =20 from .archive import tar_extract as archive_extract from .archive import cpio_extract +from .uncompress import gzip_uncompress +from .uncompress import lzma_uncompress =20 """ Round up to next power of 2 @@ -36,25 +35,3 @@ def image_pow2ceil_expand(path): if size !=3D size_aligned: with open(path, 'ab+') as fd: fd.truncate(size_aligned) - -def gzip_uncompress(gz_path, output_path): - if os.path.exists(output_path): - return - with gzip.open(gz_path, 'rb') as gz_in: - try: - with open(output_path, 'wb') as raw_out: - shutil.copyfileobj(gz_in, raw_out) - except: - os.remove(output_path) - raise - -def lzma_uncompress(xz_path, output_path): - if os.path.exists(output_path): - return - with lzma.open(xz_path, 'rb') as lzma_in: - try: - with open(output_path, 'wb') as raw_out: - shutil.copyfileobj(lzma_in, raw_out) - except: - os.remove(output_path) - raise --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451429; cv=none; d=zohomail.com; s=zohoarc; b=VR39byHCBxuBjybW4jJtliQX9p0IbbFd1Vz1nwKbhNyU13JzXu0FCzxy9wLmGBMIoBTLvAiNoi0hWU6HNd0VXbS6wb2EbUBMZN7qa/dn7vu+1/kfonQItI2ZO4blgY61rk4s/rnXDObXYVdzgBPVrtqxSFuVbWuu2FFy/gw5/TQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451429; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CoU+Lj/d176FqG6ibolioaYJacTb1kl+Tss5Fm5ZrwE=; b=PBTPIXgKXypQM4gYo/zX6NS9uON9uzvf9HKHqHtpUrVCjnDl8ln7DhWZoON3C2gSlcXcQ1/z3CHzxaiCohji2tk/vI60B6DpuTNClDXu4a50yf9t+bA17SiY6SHwgqTPbHsF4JqyUx/JHag19KtmFHbFaCP+4yTC0gM+rjZQ/50= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451429473202.395741660716; Tue, 17 Dec 2024 08:03:49 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0d-0004uG-JX; Tue, 17 Dec 2024 11:01:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0K-0004RK-Gb for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0J-0002Aw-1p for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:00 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-533-wpvVz0_iONO2isdaSpP4aA-1; Tue, 17 Dec 2024 11:00:55 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B00811956064; Tue, 17 Dec 2024 16:00:53 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5D5B230044C1; Tue, 17 Dec 2024 16:00:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CoU+Lj/d176FqG6ibolioaYJacTb1kl+Tss5Fm5ZrwE=; b=T/yZWKWLU7K53IzT52bsHvRbcgSdyEyapWVim1OwQJPfqkBfvlZfBYJglArTvdtgoDRNzq 5VadBIH0Tjt1T9GCju160BD5rGWvaLFTPyiS2AexfwYSHXPxNT4T6ko1Ckj7SAuVDcMJBS ad0dTUP7vjSA4iyztOqUDlKbYDYBAxo= X-MC-Unique: wpvVz0_iONO2isdaSpP4aA-1 X-Mimecast-MFC-AGG-ID: wpvVz0_iONO2isdaSpP4aA From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 18/32] tests/functional: add common zip_extract helper Date: Tue, 17 Dec 2024 15:59:39 +0000 Message-ID: <20241217155953.3950506-19-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451431278116600 This mirrors the existing archive_extract and cpio_extract helpers Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/archive.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/functional/qemu_test/archive.py b/tests/functional/qemu_= test/archive.py index 9872f08d23..06b66701c0 100644 --- a/tests/functional/qemu_test/archive.py +++ b/tests/functional/qemu_test/archive.py @@ -10,6 +10,7 @@ import os import subprocess import tarfile +import zipfile =20 =20 def tar_extract(archive, dest_dir, member=3DNone): @@ -29,3 +30,10 @@ def cpio_extract(cpio_handle, output_path): input=3Dcpio_handle.read(), stderr=3Dsubprocess.DEVNULL) os.chdir(cwd) + +def zip_extract(archive, dest_dir, member=3DNone): + with zipfile.ZipFile(archive, 'r') as zf: + if member: + zf.extract(member=3Dmember, path=3Ddest_dir) + else: + zf.extractall(path=3Ddest_dir) --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451298; cv=none; d=zohomail.com; s=zohoarc; b=TDK0BzgXp1ExTufXs73TItLrNuXuzSy6/l1TQGoT4WvBRFDuzR3L1FRuz1n8tSdFuE6OfzoM2xYrjneKxdQ/Bqjrql9aI9CMPl+0rrar/pJW7clzzLDoMhOWOSLGNMBrgw8SX4EG7/Kg6WinmyEAWeAipDy0bWVClWg0WfhVO00= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451298; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=DfumUBoRViDE7+x3kZfqjdKrOAwIC+ZXpod5c/cGKOw=; b=YH6uSDTn1OHjw8uq/KVwmdIO6GE8gvgXu+3u5ZeLHWLCSvsfa0fWhYtjdOdNH31fpyxngi4NHVvjMrKmjLTjmoXdCNJ0gPJtz/I+F35506rEM+VS6DyNSlBfM9ktcfUHA65y0UyNAd8ttLNHZvetGaFZSWJbE6KrBOLFtrng0c8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451298416542.7122786916407; Tue, 17 Dec 2024 08:01:38 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0h-0005Jb-S4; Tue, 17 Dec 2024 11:01:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0T-0004bd-5t for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:10 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0O-0002Bc-OH for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:08 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-648-Agah3EHdNcqs2Lvle3tYIQ-1; Tue, 17 Dec 2024 11:00:57 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4FE15195608B; Tue, 17 Dec 2024 16:00:56 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0C3DE300F9B8; Tue, 17 Dec 2024 16:00:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DfumUBoRViDE7+x3kZfqjdKrOAwIC+ZXpod5c/cGKOw=; b=fc0rcthigYL2DNr16jeIvDlhCoLiJTuSRP5C1fKsT8xog9fB6hkRjTko6sYEfhNWzCeuDo qN9DI3cb8K9fmtFudjybs9NRki4JNJ0tYQClWcl4crgKBjdiNTeA0frab/hneajwJHI3Co 4D0qst8Cirnx0gZxCmWrp24kcCIsCC0= X-MC-Unique: Agah3EHdNcqs2Lvle3tYIQ-1 X-Mimecast-MFC-AGG-ID: Agah3EHdNcqs2Lvle3tYIQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 19/32] tests/functional: add common deb_extract helper Date: Tue, 17 Dec 2024 15:59:40 +0000 Message-ID: <20241217155953.3950506-20-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451300404116600 This mirrors the existing archive_extract, cpio_extract and zip_extract helpers Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/archive.py | 13 +++++++++++++ tests/functional/qemu_test/linuxkernel.py | 13 ++++--------- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/tests/functional/qemu_test/archive.py b/tests/functional/qemu_= test/archive.py index 06b66701c0..a6fc97a557 100644 --- a/tests/functional/qemu_test/archive.py +++ b/tests/functional/qemu_test/archive.py @@ -12,6 +12,8 @@ import tarfile import zipfile =20 +from .cmd import run_cmd + =20 def tar_extract(archive, dest_dir, member=3DNone): with tarfile.open(archive) as tf: @@ -37,3 +39,14 @@ def zip_extract(archive, dest_dir, member=3DNone): zf.extract(member=3Dmember, path=3Ddest_dir) else: zf.extractall(path=3Ddest_dir) + +def deb_extract(archive, dest_dir, member=3DNone): + cwd =3D os.getcwd() + os.chdir(dest_dir) + try: + (stdout, stderr, ret) =3D run_cmd(['ar', 't', archive]) + file_path =3D stdout.split()[2] + run_cmd(['ar', 'x', archive, file_path]) + tar_extract(file_path, dest_dir, member) + finally: + os.chdir(cwd) diff --git a/tests/functional/qemu_test/linuxkernel.py b/tests/functional/q= emu_test/linuxkernel.py index 2e4f4e35fd..8f2810f3af 100644 --- a/tests/functional/qemu_test/linuxkernel.py +++ b/tests/functional/qemu_test/linuxkernel.py @@ -6,8 +6,9 @@ import os =20 from .testcase import QemuSystemTest -from .cmd import run_cmd, wait_for_console_pattern -from .utils import archive_extract +from .cmd import wait_for_console_pattern +from .archive import deb_extract + =20 class LinuxKernelTest(QemuSystemTest): KERNEL_COMMON_COMMAND_LINE =3D 'printk.time=3D0 ' @@ -37,13 +38,7 @@ def extract_from_deb(self, deb_path, path): :param path: path within the deb archive of the file to be extract= ed :returns: path of the extracted file """ - cwd =3D os.getcwd() - os.chdir(self.workdir) - (stdout, stderr, ret) =3D run_cmd(['ar', 't', deb_path]) - file_path =3D stdout.split()[2] - run_cmd(['ar', 'x', deb_path, file_path]) - archive_extract(file_path, self.workdir) - os.chdir(cwd) + deb_extract(deb_path, self.workdir, member=3D"." + path) # Return complete path to extracted file. Because callers to # extract_from_deb() specify 'path' with a leading slash, it is # necessary to use os.path.relpath() as otherwise scratch_file() --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451816; cv=none; d=zohomail.com; s=zohoarc; b=UHe+FCPsAMEryh4Rex8BaUp475fR3r8DAZnjDJJTMjguhQ+JGC9FqYwGSpDneS9jsZa3yAF2quWXEKfwib215hz+T69Pg1eo4CK3cWV9B8hIu5MVPJbx4ooITJUoYxm9eX6QbZljjaQtRouDKWfMEvuZ0XeNqy3VZ470zOm/jsg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451816; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Br+ai5bGjWpU1X9MgHS3GJ2Q1rZhPbkDQDewh0iUynU=; b=LwYZ2gF+ZYpvfWEYqBDtfdLimY2JuHf6f/ZImnA2jAKuAzx4yvEtrcv7le3SEaXW5MUf3B5GD7HTUNGfvyRGVDDgjPf8U6O8zTMOkj4ZEYoakiqI6D4jCRVTJnXxKdrbcR+AwkXv/FY4dXGsuZ54AJSElk//NGCFE/Y8+xt1VuI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 17344518163151001.7865943389119; Tue, 17 Dec 2024 08:10:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0g-0005FC-GR; Tue, 17 Dec 2024 11:01:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0Q-0004VI-RB for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:08 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0O-0002BZ-OH for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:05 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-75-hR7MuDhFPdyxuzKYSFBuTg-1; Tue, 17 Dec 2024 11:01:00 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1C5D91956051; Tue, 17 Dec 2024 16:00:59 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DF00A30044C1; Tue, 17 Dec 2024 16:00:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Br+ai5bGjWpU1X9MgHS3GJ2Q1rZhPbkDQDewh0iUynU=; b=E0W4DJqCrzhpJIjOtEfacOuqCmWrpgOGngiIUSH7+kawSTVftwk9CcOMUrOnDtvhU7WTFS ohF52PUEYm7TOOXyw05hEzzD6Q8n9nKIhf81xRqdnRMz9t9eZOWCi51EuyuVnWN4ixMuNw EzdynO0/fSffC4uNalpz4wvXH/en5/0= X-MC-Unique: hR7MuDhFPdyxuzKYSFBuTg-1 X-Mimecast-MFC-AGG-ID: hR7MuDhFPdyxuzKYSFBuTg From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 20/32] tests/functional: let cpio_extract accept filenames Date: Tue, 17 Dec 2024 15:59:41 +0000 Message-ID: <20241217155953.3950506-21-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451818421116600 Currently cpio_extract differs from tar_extract/zip_extract in that it only allows a file-like object as input. Adapt it to also support filenames. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/archive.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/tests/functional/qemu_test/archive.py b/tests/functional/qemu_= test/archive.py index a6fc97a557..bc448dee4a 100644 --- a/tests/functional/qemu_test/archive.py +++ b/tests/functional/qemu_test/archive.py @@ -8,7 +8,7 @@ # Thomas Huth =20 import os -import subprocess +from subprocess import check_call, run, DEVNULL import tarfile import zipfile =20 @@ -25,12 +25,18 @@ def tar_extract(archive, dest_dir, member=3DNone): else: tf.extractall(path=3Ddest_dir) =20 -def cpio_extract(cpio_handle, output_path): +def cpio_extract(archive, output_path): cwd =3D os.getcwd() os.chdir(output_path) - subprocess.run(['cpio', '-i'], - input=3Dcpio_handle.read(), - stderr=3Dsubprocess.DEVNULL) + # Not passing 'check=3DTrue' as cpio exits with non-zero + # status if the archive contains any device nodes :-( + if type(archive) =3D=3D str: + run(['cpio', '-i', '-F', archive], + stdout=3DDEVNULL, stderr=3DDEVNULL) + else: + run(['cpio', '-i'], + input=3Darchive.read(), + stdout=3DDEVNULL, stderr=3DDEVNULL) os.chdir(cwd) =20 def zip_extract(archive, dest_dir, member=3DNone): --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451353; cv=none; d=zohomail.com; s=zohoarc; b=fyKlaYZ5b/tb+3elgYLLX9EaNhD8Rv9yaOLtfZ1kHyWZS2SLk99itnbKmJ1uxFau0Vqld3/GbjcXIlVrmDaPly9Ztko0dEqvCwsfqM9k+tMAFswnDdkEouJGxQEUFQ05+hvfCk17tIr+9dD5RWGskbDxbiNTtqtE6GSHyWj/qG0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451353; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=/tNbaN/pfEEqvoAc7QRrmNMsFOLnlzNoHWXb98e86zo=; b=lILld+s8kLHkPFkahrEYtNAzlEpj3VmdPQMBVKl7L6llkR/43OD3pM8v74jzzlA+86kYQP+HKuHT16vsBgKCX5nd1RFoj6pj8MxAxZdbV/YBxyQyt6tPtZqs3eefq/fCJlYNUSDKZ8X9Dy1OaQc1VQr58kd94By3s9+mHKO2dNE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451353701814.9560317074007; Tue, 17 Dec 2024 08:02:33 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0k-0005Lt-16; Tue, 17 Dec 2024 11:01:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0d-00053j-IQ for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:19 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0W-0002Cv-Sx for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:18 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-167-hUHAsRkWPgiEfMrfCAqb7w-1; Tue, 17 Dec 2024 11:01:08 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D20C71955F09; Tue, 17 Dec 2024 16:01:01 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 91922300F9B5; Tue, 17 Dec 2024 16:00:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/tNbaN/pfEEqvoAc7QRrmNMsFOLnlzNoHWXb98e86zo=; b=G4qtvDaHxq1cQriHOd5GDQb9GYJ4oLPQcy85ZV46sLMZx5xGKTSqNqcLRBVgvXechx1EKd QzVDYxRVpM4Mq+PvLvsMZlzcnmZelm8FybF+ifa7i14WaIW5NPUwAVzt3jgOQuGhwgs3us bsUKNuOIp+te9S8b6Ffui5hjvasuV0E= X-MC-Unique: hUHAsRkWPgiEfMrfCAqb7w-1 X-Mimecast-MFC-AGG-ID: hUHAsRkWPgiEfMrfCAqb7w From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 21/32] tests/functional: add a generalized archive_extract Date: Tue, 17 Dec 2024 15:59:42 +0000 Message-ID: <20241217155953.3950506-22-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451354908116600 There are many types of archives that the tests deal with. Provide a generalized 'archive_extract' that can detect the format and delegate to the appropriate helper for extraction. This ensures that all archive extraction code follows the same design pattern. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/__init__.py | 1 + tests/functional/qemu_test/archive.py | 58 ++++++++++++++++++++++++++ 2 files changed, 59 insertions(+) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu= _test/__init__.py index fe6cbe3a8a..665c482d13 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -16,3 +16,4 @@ from .decorators import skipIfMissingCommands, skipIfNotMachine, \ skipFlakyTest, skipUntrustedTest, skipBigDataTest, \ skipIfMissingImports +from .archive import archive_extract diff --git a/tests/functional/qemu_test/archive.py b/tests/functional/qemu_= test/archive.py index bc448dee4a..c439d9413a 100644 --- a/tests/functional/qemu_test/archive.py +++ b/tests/functional/qemu_test/archive.py @@ -10,8 +10,10 @@ import os from subprocess import check_call, run, DEVNULL import tarfile +from urllib.parse import urlparse import zipfile =20 +from .asset import Asset from .cmd import run_cmd =20 =20 @@ -56,3 +58,59 @@ def deb_extract(archive, dest_dir, member=3DNone): tar_extract(file_path, dest_dir, member) finally: os.chdir(cwd) + +''' +@params archive: filename, Asset, or file-like object to extract +@params dest_dir: target directory to extract into +@params member: optional member file to limit extraction to + +Extracts @archive into @dest_dir. All files are extracted +unless @member specifies a limit. + +If @format is None, heuristics will be applied to guess the format +from the filename or Asset URL. @format must be non-None if @archive +is a file-like object. +''' +def archive_extract(archive, dest_dir, format=3DNone, member=3DNone): + if format is None: + format =3D guess_archive_format(archive) + if type(archive) =3D=3D Asset: + archive =3D str(archive) + + if format =3D=3D "tar": + tar_extract(archive, dest_dir, member) + elif format =3D=3D "zip": + zip_extract(archive, dest_dir, member) + elif format =3D=3D "cpio": + if member is not None: + raise Exception("Unable to filter cpio extraction") + cpio_extract(archive, dest_dir) + elif format =3D=3D "deb": + if type(archive) !=3D str: + raise Exception("Unable to use file-like object with deb archi= ves") + deb_extract(archive, dest_dir, "./" + member) + else: + raise Exception(f"Unknown archive format {format}") + +''' +@params archive: filename, or Asset to guess + +Guess the format of @compressed, raising an exception if +no format can be determined +''' +def guess_archive_format(archive): + if type(archive) =3D=3D Asset: + archive =3D urlparse(archive.url).path + elif type(archive) !=3D str: + raise Exception(f"Unable to guess archive format for {archive}") + + if ".tar." in archive or archive.endswith("tgz"): + return "tar" + elif archive.endswith(".zip"): + return "zip" + elif archive.endswith(".cpio"): + return "cpio" + elif archive.endswith(".deb") or archive.endswith(".udeb"): + return "deb" + else: + raise Exception(f"Unknown archive format for {archive}") --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451481; cv=none; d=zohomail.com; s=zohoarc; b=lbwhsQzG+KZbOjbeZafsQV9efwPZyfAWRPYiBnzkxNyEGcEbATLUwQOBlwSGe2vtekcQe+gpe4K9w2usMMa4RVRqDZuDrVLfcZjE5bX0ada75y4HXXNLPuzkfikJebolnpRnqocrertef7NWU8JEKS0mWOBJ68Xc3wWhHsykSw0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451481; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=w6BpfwBKsOvfGGc+m4xw1mdADVVXjWk9/VXy9Bp9SNw=; b=C45Q0UWBnFdJtnkh86C1Ss5IHT9Xo7BlWRXjHhyiek+Rs/EZ9qDbwL82ZKlo/86CV6KaiPiEGtKTIJD3RX0YPsl6BzXrNJBDtPKYfF9iBNL0rVl2vi2mk1Jjp9bx+2UIUlEd1a5yiP9MIt+wjwla9lFOspU05trKLj6G6hpp6hk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451481162834.1316310671332; Tue, 17 Dec 2024 08:04:41 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0h-0005IP-Br; Tue, 17 Dec 2024 11:01:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0a-0004nx-0w for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:16 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0W-0002Ch-E6 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:15 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-26-xnuHMhAQMGmz0i8WbZWndQ-1; Tue, 17 Dec 2024 11:01:06 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CC58819560A1; Tue, 17 Dec 2024 16:01:04 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6C2CF300F9B7; Tue, 17 Dec 2024 16:01:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w6BpfwBKsOvfGGc+m4xw1mdADVVXjWk9/VXy9Bp9SNw=; b=EQGDUcDoKjuyvIFMagzvbTuPNbwwN3cee0Ii87W/G9AweVf/DmBZD6nToFvOJSAhx2WoNu x6E5sbXsmijn6HYXcOc63G1jw5HFfhmdRGEbYbnPACIvCnEbb/xquioBuhZ2r0ZxKuFVGj WXZzq7LIxO4N9f+jWHDOLQEeNYRrzlg= X-MC-Unique: xnuHMhAQMGmz0i8WbZWndQ-1 X-Mimecast-MFC-AGG-ID: xnuHMhAQMGmz0i8WbZWndQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 22/32] tests/functional: add 'archive_extract' to QemuBaseTest Date: Tue, 17 Dec 2024 15:59:43 +0000 Message-ID: <20241217155953.3950506-23-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451481637116600 This helper wrappers archive.archive_extract, forcing the use of the scratch directory, to ensure any extracted files are cleaned at test termination. If a specific member is requested, then the path to the extracted file is also returned. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/testcase.py | 32 ++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index 493938240c..19fb1d0c07 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -25,6 +25,7 @@ from qemu.machine import QEMUMachine from qemu.utils import kvm_available, tcg_available =20 +from .archive import archive_extract from .asset import Asset from .cmd import run_cmd from .config import BUILD_DIR @@ -39,6 +40,37 @@ class QemuBaseTest(unittest.TestCase): log =3D None logdir =3D None =20 + ''' + @params archive: filename, Asset, or file-like object to extract + @params format: optional archive format (tar, zip, deb, cpio) + @params sub_dir: optional sub-directory to extract into + @params member: optional member file to limit extraction to + + Extracts @archive into the scratch directory, or a directory beneath + named by @sub_dir. All files are extracted unless @member specifies + a limit. + + If @format is None, heuristics will be applied to guess the format + from the filename or Asset URL. @format must be non-None if @archive + is a file-like object. + + If @member is non-None, returns the fully qualified path to @member + ''' + def archive_extract(self, archive, format=3DNone, sub_dir=3DNone, memb= er=3DNone): + self.log.debug(f"Extract {archive} format=3D{format}" + + f"sub_dir=3D{sub_dir} member=3D{member}") + if type(archive) =3D=3D Asset: + archive.fetch() + if sub_dir is None: + archive_extract(archive, self.scratch_file(), format, member) + else: + archive_extract(archive, self.scratch_file(sub_dir), + format, member) + + if member is not None: + return self.scratch_file(member) + return None + ''' Create a temporary directory suitable for storing UNIX socket paths. --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451886; cv=none; d=zohomail.com; s=zohoarc; b=OEroFpaB/CQybufvWDJDFE425IvaXjVZrMrPEvQ5xJpjdWkxdp3gbNTIHYEpJRWKA6cg++VU6XExhs4SY9G5mh692Jkqoe5KBly2r1NSlIac19wcE9qhh0Udp7OpnTgqviHVJEf0msZYhzwMLbFtjfo22trUsPOICpkYImjLuRY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451886; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=oDjuLanGID/HLMbXE+PjjB6rDvixIxSrfc7QlURc5Xw=; b=ZrNfMKNzRPyYlVh7pGJMYvSG2OqtvSYKRvd2XauaC1St2IemWlcbCTHZ/nn+NYs5IHKN5lXoCTg2Zaq1cNz3/nBx5v9BM8vh7gY6V/u48r5B2QyZGhDiT4yl2XxaQ9fbHspDP5+0e2R+3chXpnltzOcfEOL7y5y5THz9BWijENE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451886739304.41064778683915; Tue, 17 Dec 2024 08:11:26 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0h-0005JA-KK; Tue, 17 Dec 2024 11:01:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0a-0004sq-Mc for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0V-0002Cf-K5 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:16 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-212-GyYOaeDjMRygv-l1A1p9lQ-1; Tue, 17 Dec 2024 11:01:08 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AA1511956048; Tue, 17 Dec 2024 16:01:07 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 69BB9300F9B5; Tue, 17 Dec 2024 16:01:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oDjuLanGID/HLMbXE+PjjB6rDvixIxSrfc7QlURc5Xw=; b=bCAn6TnpTnP61yYbcA1OMsipBsalfGxWxyQDf9zBaEquHMpHeEJSIvwn09sGybxf7lrBs3 J69K/oWuCZ0svWuHNSBP2rEQul7wHNCdanbW9SA/11u09GxufAvgYL2lwC1O82U7258/uY TvHid8HP+CfuX+LFYjfMsaoL6dMjgXI= X-MC-Unique: GyYOaeDjMRygv-l1A1p9lQ-1 X-Mimecast-MFC-AGG-ID: GyYOaeDjMRygv-l1A1p9lQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 23/32] tests/functional: convert tests to new archive_extract helper Date: Tue, 17 Dec 2024 15:59:44 +0000 Message-ID: <20241217155953.3950506-24-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451889120116600 Replace use of utils.archive_extract and extract_from_deb with the new archive_extract helper. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/linuxkernel.py | 19 ---------- tests/functional/test_aarch64_aspeed.py | 5 ++- tests/functional/test_aarch64_raspi3.py | 8 +---- tests/functional/test_aarch64_raspi4.py | 14 ++++---- tests/functional/test_acpi_bits.py | 15 ++------ tests/functional/test_arm_aspeed_ast1030.py | 16 +++------ tests/functional/test_arm_aspeed_ast2500.py | 6 ++-- tests/functional/test_arm_aspeed_ast2600.py | 5 +-- tests/functional/test_arm_aspeed_rainier.py | 11 +++--- tests/functional/test_arm_bflt.py | 3 +- tests/functional/test_arm_bpim2u.py | 26 +++++++------- tests/functional/test_arm_canona1100.py | 11 +++--- tests/functional/test_arm_cubieboard.py | 20 +++++------ tests/functional/test_arm_orangepi.py | 35 +++++++++---------- tests/functional/test_arm_raspi2.py | 14 ++++---- tests/functional/test_arm_smdkc210.py | 9 +++-- tests/functional/test_arm_vexpress.py | 5 ++- tests/functional/test_m68k_mcf5208evb.py | 5 ++- tests/functional/test_m68k_q800.py | 5 ++- .../functional/test_microblaze_s3adsp1800.py | 5 ++- .../test_microblazeel_s3adsp1800.py | 5 ++- tests/functional/test_mips64el_fuloong2e.py | 6 ++-- tests/functional/test_mips64el_malta.py | 6 ++-- tests/functional/test_mips_malta.py | 12 +++---- tests/functional/test_mipsel_malta.py | 5 +-- tests/functional/test_or1k_sim.py | 5 ++- tests/functional/test_ppc64_e500.py | 5 ++- tests/functional/test_ppc_amiga.py | 6 ++-- tests/functional/test_ppc_bamboo.py | 5 ++- tests/functional/test_ppc_mac.py | 6 ++-- tests/functional/test_ppc_mpc8544ds.py | 8 ++--- tests/functional/test_ppc_virtex_ml507.py | 5 ++- tests/functional/test_sh4_r2d.py | 5 ++- tests/functional/test_sh4eb_r2d.py | 5 ++- tests/functional/test_sparc64_sun4u.py | 8 ++--- tests/functional/test_sparc_sun4m.py | 5 ++- tests/functional/test_xtensa_lx60.py | 5 ++- 37 files changed, 134 insertions(+), 205 deletions(-) diff --git a/tests/functional/qemu_test/linuxkernel.py b/tests/functional/q= emu_test/linuxkernel.py index 8f2810f3af..2c9598102d 100644 --- a/tests/functional/qemu_test/linuxkernel.py +++ b/tests/functional/qemu_test/linuxkernel.py @@ -3,11 +3,8 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -import os - from .testcase import QemuSystemTest from .cmd import wait_for_console_pattern -from .archive import deb_extract =20 =20 class LinuxKernelTest(QemuSystemTest): @@ -29,19 +26,3 @@ def launch_kernel(self, kernel, initrd=3DNone, dtb=3DNon= e, console_index=3D0, self.vm.launch() if wait_for: self.wait_for_console_pattern(wait_for) - - def extract_from_deb(self, deb_path, path): - """ - Extracts a file from a deb package into the test workdir - - :param deb_path: path to the deb archive - :param path: path within the deb archive of the file to be extract= ed - :returns: path of the extracted file - """ - deb_extract(deb_path, self.workdir, member=3D"." + path) - # Return complete path to extracted file. Because callers to - # extract_from_deb() specify 'path' with a leading slash, it is - # necessary to use os.path.relpath() as otherwise scratch_file() - # interprets it as an absolute path and drops the required prefix - return os.path.normpath(self.scratch_file(os.path.relpath(path, '/= '))) - diff --git a/tests/functional/test_aarch64_aspeed.py b/tests/functional/tes= t_aarch64_aspeed.py index 8ba2c67248..141d863859 100755 --- a/tests/functional/test_aarch64_aspeed.py +++ b/tests/functional/test_aarch64_aspeed.py @@ -11,7 +11,7 @@ from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern from qemu_test import exec_command_and_wait_for_pattern -from qemu_test.utils import archive_extract + =20 class AST2x00MachineSDK(QemuSystemTest): =20 @@ -34,8 +34,7 @@ def do_test_aarch64_aspeed_sdk_start(self, image): def test_aarch64_ast2700_evb_sdk_v09_02(self): self.set_machine('ast2700-evb') =20 - image_path =3D self.ASSET_SDK_V902_AST2700.fetch() - archive_extract(image_path, self.workdir) + self.archive_extract(self.ASSET_SDK_V902_AST2700) =20 num_cpu =3D 4 uboot_size =3D os.path.getsize(self.scratch_file('ast2700-default', diff --git a/tests/functional/test_aarch64_raspi3.py b/tests/functional/tes= t_aarch64_raspi3.py index 98ed6f9d56..74f6630ed2 100755 --- a/tests/functional/test_aarch64_raspi3.py +++ b/tests/functional/test_aarch64_raspi3.py @@ -7,8 +7,6 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 -from zipfile import ZipFile - from qemu_test import LinuxKernelTest, Asset =20 =20 @@ -21,11 +19,7 @@ class Aarch64Raspi3Machine(LinuxKernelTest): =20 def test_aarch64_raspi3_atf(self): efi_name =3D 'RPI_EFI.fd' - zip_path =3D self.ASSET_RPI3_UEFI.fetch() - - with ZipFile(zip_path, 'r') as zf: - zf.extract(efi_name, path=3Dself.workdir) - efi_fd =3D self.scratch_file(efi_name) + efi_fd =3D self.archive_extract(self.ASSET_RPI3_UEFI, member=3Defi= _name) =20 self.set_machine('raspi3b') self.vm.set_console(console_index=3D1) diff --git a/tests/functional/test_aarch64_raspi4.py b/tests/functional/tes= t_aarch64_raspi4.py index 2cda03f86f..3918e35e82 100755 --- a/tests/functional/test_aarch64_raspi4.py +++ b/tests/functional/test_aarch64_raspi4.py @@ -30,9 +30,10 @@ class Aarch64Raspi4Machine(LinuxKernelTest): '7c0b16d1853772f6f4c3ca63e789b3b9ff4936efac9c8a01fb0c98c05c7a7648') =20 def test_arm_raspi4(self): - deb_path =3D self.ASSET_KERNEL_20190215.fetch() - kernel_path =3D self.extract_from_deb(deb_path, '/boot/kernel8.img= ') - dtb_path =3D self.extract_from_deb(deb_path, '/boot/bcm2711-rpi-4-= b.dtb') + kernel_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/kernel8.img') + dtb_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/bcm2711-rpi-4-b.dtb= ') =20 self.set_machine('raspi4b') self.vm.set_console() @@ -58,9 +59,10 @@ def test_arm_raspi4(self): =20 =20 def test_arm_raspi4_initrd(self): - deb_path =3D self.ASSET_KERNEL_20190215.fetch() - kernel_path =3D self.extract_from_deb(deb_path, '/boot/kernel8.img= ') - dtb_path =3D self.extract_from_deb(deb_path, '/boot/bcm2711-rpi-4-= b.dtb') + kernel_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/kernel8.img') + dtb_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/bcm2711-rpi-4-b.dtb= ') initrd_path_gz =3D self.ASSET_INITRD.fetch() initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) diff --git a/tests/functional/test_acpi_bits.py b/tests/functional/test_acp= i_bits.py index 3b99ecf3a4..20da435687 100755 --- a/tests/functional/test_acpi_bits.py +++ b/tests/functional/test_acpi_bits.py @@ -35,8 +35,6 @@ import re import shutil import subprocess -import tarfile -import zipfile =20 from typing import ( List, @@ -260,19 +258,12 @@ def setUp(self): # pylint: disable=3Darguments-differ %(self.BITS_INTERNAL_VER, self.BITS_COMMIT_HASH)) =20 - bitsLocalArtLoc =3D self.ASSET_BITS.fetch() - self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLo= c) - # extract the bits artifact in the temp working directory - with zipfile.ZipFile(bitsLocalArtLoc, 'r') as zref: - zref.extractall(prebuiltDir) + self.archive_extract(self.ASSET_BITS, sub_dir=3D'prebuilt', format= =3D'zip') =20 # extract the bits software in the temp working directory - with zipfile.ZipFile(bits_zip_file, 'r') as zref: - zref.extractall(self.workdir) - - with tarfile.open(grub_tar_file, 'r', encoding=3D'utf-8') as tarba= ll: - tarball.extractall(self.workdir) + self.archive_extract(bits_zip_file) + self.archive_extract(grub_tar_file) =20 self.copy_test_scripts() self.copy_bits_config() diff --git a/tests/functional/test_arm_aspeed_ast1030.py b/tests/functional= /test_arm_aspeed_ast1030.py index 01b13c5591..d45d9f7c1c 100755 --- a/tests/functional/test_arm_aspeed_ast1030.py +++ b/tests/functional/test_arm_aspeed_ast1030.py @@ -8,7 +8,7 @@ =20 from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern -from zipfile import ZipFile + =20 class AST1030Machine(LinuxKernelTest): =20 @@ -20,12 +20,9 @@ class AST1030Machine(LinuxKernelTest): def test_ast1030_zephyros_1_04(self): self.set_machine('ast1030-evb') =20 - zip_file =3D self.ASSET_ZEPHYR_1_04.fetch() - kernel_name =3D "ast1030-evb-demo/zephyr.elf" - with ZipFile(zip_file, 'r') as zf: - zf.extract(kernel_name, path=3Dself.workdir) - kernel_file =3D self.scratch_file(kernel_name) + kernel_file =3D self.archive_extract( + self.ASSET_ZEPHYR_1_04, member=3Dkernel_name) =20 self.vm.set_console() self.vm.add_args('-kernel', kernel_file, '-nographic') @@ -42,12 +39,9 @@ def test_ast1030_zephyros_1_04(self): def test_ast1030_zephyros_1_07(self): self.set_machine('ast1030-evb') =20 - zip_file =3D self.ASSET_ZEPHYR_1_07.fetch() - kernel_name =3D "ast1030-evb-demo/zephyr.bin" - with ZipFile(zip_file, 'r') as zf: - zf.extract(kernel_name, path=3Dself.workdir) - kernel_file =3D self.scratch_file(kernel_name) + kernel_file =3D self.archive_extract( + self.ASSET_ZEPHYR_1_07, member=3Dkernel_name) =20 self.vm.set_console() self.vm.add_args('-kernel', kernel_file, '-nographic') diff --git a/tests/functional/test_arm_aspeed_ast2500.py b/tests/functional= /test_arm_aspeed_ast2500.py index 8c5593cdc8..743fc46eb2 100755 --- a/tests/functional/test_arm_aspeed_ast2500.py +++ b/tests/functional/test_arm_aspeed_ast2500.py @@ -7,7 +7,7 @@ from qemu_test import Asset from aspeed import AspeedTest from qemu_test import exec_command_and_wait_for_pattern -from qemu_test.utils import archive_extract + =20 class AST2500Machine(AspeedTest): =20 @@ -45,9 +45,7 @@ def test_arm_ast2500_evb_buildroot(self): def test_arm_ast2500_evb_sdk(self): self.set_machine('ast2500-evb') =20 - image_path =3D self.ASSET_SDK_V806_AST2500.fetch() - - archive_extract(image_path, self.workdir) + self.archive_extract(self.ASSET_SDK_V806_AST2500) =20 self.do_test_arm_aspeed_sdk_start( self.scratch_file("ast2500-default", "image-bmc")) diff --git a/tests/functional/test_arm_aspeed_ast2600.py b/tests/functional= /test_arm_aspeed_ast2600.py index 25948c1c34..21640123ee 100755 --- a/tests/functional/test_arm_aspeed_ast2600.py +++ b/tests/functional/test_arm_aspeed_ast2600.py @@ -12,7 +12,6 @@ from qemu_test import Asset from aspeed import AspeedTest from qemu_test import exec_command_and_wait_for_pattern, skipIfMissingComm= ands -from qemu_test.utils import archive_extract =20 =20 class AST2600Machine(AspeedTest): @@ -105,9 +104,7 @@ def test_arm_ast2600_evb_buildroot_tpm(self): def test_arm_ast2600_evb_sdk(self): self.set_machine('ast2600-evb') =20 - image_path =3D self.ASSET_SDK_V806_AST2600_A2.fetch() - - archive_extract(image_path, self.workdir) + self.archive_extract(self.ASSET_SDK_V806_AST2600_A2) =20 self.vm.add_args('-device', 'tmp105,bus=3Daspeed.i2c.bus.5,address=3D0x4d,id=3Dtmp-test'); diff --git a/tests/functional/test_arm_aspeed_rainier.py b/tests/functional= /test_arm_aspeed_rainier.py index b856aea6db..602d6194ac 100755 --- a/tests/functional/test_arm_aspeed_rainier.py +++ b/tests/functional/test_arm_aspeed_rainier.py @@ -43,11 +43,12 @@ def test_arm_aspeed_emmc_boot(self): def test_arm_debian_kernel_boot(self): self.set_machine('rainier-bmc') =20 - deb_path =3D self.ASSET_DEBIAN_LINUX_ARMHF_DEB.fetch() - - kernel_path =3D self.extract_from_deb(deb_path, '/boot/vmlinuz-5.1= 7.0-2-armmp') - dtb_path =3D self.extract_from_deb(deb_path, - '/usr/lib/linux-image-5.17.0-2-armmp/aspeed-bmc-ibm-rainie= r.dtb') + kernel_path =3D self.archive_extract( + self.ASSET_DEBIAN_LINUX_ARMHF_DEB, + member=3D'boot/vmlinuz-5.17.0-2-armmp') + dtb_path =3D self.archive_extract( + self.ASSET_DEBIAN_LINUX_ARMHF_DEB, + member=3D'usr/lib/linux-image-5.17.0-2-armmp/aspeed-bmc-ibm-ra= inier.dtb') =20 self.vm.set_console() self.vm.add_args('-kernel', kernel_path, diff --git a/tests/functional/test_arm_bflt.py b/tests/functional/test_arm_= bflt.py index 88ef7b1edc..f273fc8354 100755 --- a/tests/functional/test_arm_bflt.py +++ b/tests/functional/test_arm_bflt.py @@ -10,7 +10,6 @@ =20 from qemu_test import QemuUserTest, Asset from qemu_test import skipIfMissingCommands, skipUntrustedTest -from qemu_test.utils import cpio_extract =20 =20 class LoadBFLT(QemuUserTest): @@ -27,7 +26,7 @@ def test_stm32(self): busybox_path =3D self.scratch_file("bin", "busybox") =20 with bz2.open(rootfs_path_bz2, 'rb') as cpio_handle: - cpio_extract(cpio_handle, self.workdir) + self.archive_extract(cpio_handle, format=3D"cpio") =20 res =3D self.run_cmd(busybox_path) ver =3D 'BusyBox v1.24.0.git (2015-02-03 22:17:13 CET) multi-call = binary.' diff --git a/tests/functional/test_arm_bpim2u.py b/tests/functional/test_ar= m_bpim2u.py index 2af6d9a18d..91c56b0930 100755 --- a/tests/functional/test_arm_bpim2u.py +++ b/tests/functional/test_arm_bpim2u.py @@ -39,12 +39,11 @@ class BananaPiMachine(LinuxKernelTest): =20 def test_arm_bpim2u(self): self.set_machine('bpim2u') - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D ('/usr/lib/linux-image-6.6.16-current-sunxi/' + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' 'sun8i-r40-bananapi-m2-ultra.dtb') - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + @@ -61,12 +60,11 @@ def test_arm_bpim2u(self): =20 def test_arm_bpim2u_initrd(self): self.set_machine('bpim2u') - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D ('/usr/lib/linux-image-6.6.16-current-sunxi/' + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' 'sun8i-r40-bananapi-m2-ultra.dtb') - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) initrd_path_gz =3D self.ASSET_INITRD.fetch() initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) @@ -100,11 +98,11 @@ def test_arm_bpim2u_gmac(self): self.require_netdev('user') =20 deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D ('/usr/lib/linux-image-6.6.16-current-sunxi/' + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' 'sun8i-r40-bananapi-m2-ultra.dtb') - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) rootfs_path_xz =3D self.ASSET_ROOTFS.fetch() rootfs_path =3D self.scratch_file('rootfs.cpio') lzma_uncompress(rootfs_path_xz, rootfs_path) diff --git a/tests/functional/test_arm_canona1100.py b/tests/functional/tes= t_arm_canona1100.py index b4e3633422..21a1a596a0 100755 --- a/tests/functional/test_arm_canona1100.py +++ b/tests/functional/test_arm_canona1100.py @@ -12,7 +12,7 @@ =20 from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from qemu_test.utils import archive_extract + =20 class CanonA1100Machine(QemuSystemTest): """Boots the barebox firmware and checks that the console is operation= al""" @@ -26,13 +26,10 @@ class CanonA1100Machine(QemuSystemTest): def test_arm_canona1100(self): self.set_machine('canon-a1100') =20 - file_path =3D self.ASSET_BIOS.fetch() - archive_extract(file_path, dest_dir=3Dself.workdir, - member=3D"day18/barebox.canon-a1100.bin") + bios =3D self.archive_extract(self.ASSET_BIOS, + member=3D"day18/barebox.canon-a1100.bi= n") self.vm.set_console() - self.vm.add_args('-bios', - self.scratch_file('day18', - 'barebox.canon-a1100.bin')) + self.vm.add_args('-bios', bios) self.vm.launch() wait_for_console_pattern(self, 'running /env/bin/init') =20 diff --git a/tests/functional/test_arm_cubieboard.py b/tests/functional/tes= t_arm_cubieboard.py index d81c333d0c..32db303253 100755 --- a/tests/functional/test_arm_cubieboard.py +++ b/tests/functional/test_arm_cubieboard.py @@ -38,11 +38,11 @@ class CubieboardMachine(LinuxKernelTest): =20 def test_arm_cubieboard_initrd(self): self.set_machine('cubieboard') - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun4i-a10-= cubieboard.dtb' - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + + 'sun4i-a10-cubieboard.dtb') + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) initrd_path_gz =3D self.ASSET_INITRD.fetch() initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) @@ -71,11 +71,11 @@ def test_arm_cubieboard_initrd(self): =20 def test_arm_cubieboard_sata(self): self.set_machine('cubieboard') - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun4i-a10-= cubieboard.dtb' - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + + 'sun4i-a10-cubieboard.dtb') + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) =20 rootfs_path_gz =3D self.ASSET_SATA_ROOTFS.fetch() rootfs_path =3D self.scratch_file('rootfs.cpio') diff --git a/tests/functional/test_arm_orangepi.py b/tests/functional/test_= arm_orangepi.py index bea67cfb6a..aa2d9d19a4 100755 --- a/tests/functional/test_arm_orangepi.py +++ b/tests/functional/test_arm_orangepi.py @@ -50,11 +50,11 @@ class BananaPiMachine(LinuxKernelTest): =20 def test_arm_orangepi(self): self.set_machine('orangepi-pc') - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun8i-h3-o= rangepi-pc.dtb' - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + + 'sun8i-h3-orangepi-pc.dtb') + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + @@ -71,11 +71,11 @@ def test_arm_orangepi(self): =20 def test_arm_orangepi_initrd(self): self.set_machine('orangepi-pc') - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun8i-h3-o= rangepi-pc.dtb' - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + + 'sun8i-h3-orangepi-pc.dtb') + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) initrd_path_gz =3D self.ASSET_INITRD.fetch() initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) @@ -107,11 +107,11 @@ def test_arm_orangepi_initrd(self): def test_arm_orangepi_sd(self): self.set_machine('orangepi-pc') self.require_netdev('user') - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-6.6.16-current-= sunxi') - dtb_path =3D '/usr/lib/linux-image-6.6.16-current-sunxi/sun8i-h3-o= rangepi-pc.dtb' - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + kernel_path =3D self.archive_extract( + self.ASSET_DEB, member=3D'boot/vmlinuz-6.6.16-current-sunxi') + dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + + 'sun8i-h3-orangepi-pc.dtb') + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) rootfs_path_xz =3D self.ASSET_ROOTFS.fetch() rootfs_path =3D self.scratch_file('rootfs.cpio') lzma_uncompress(rootfs_path_xz, rootfs_path) @@ -189,13 +189,12 @@ def test_arm_orangepi_armbian(self): def test_arm_orangepi_uboot_netbsd9(self): self.set_machine('orangepi-pc') # This test download a 304MB compressed image and expand it to 2GB - deb_path =3D self.ASSET_UBOOT.fetch() # We use the common OrangePi PC 'plus' build of U-Boot for our sec= ondary # program loader (SPL). We will then set the path to the more spec= ific # OrangePi "PC" device tree blob with 'setenv fdtfile' in U-Boot p= rompt, # before to boot NetBSD. - uboot_path =3D '/usr/lib/u-boot/orangepi_plus/u-boot-sunxi-with-sp= l.bin' - uboot_path =3D self.extract_from_deb(deb_path, uboot_path) + uboot_path =3D 'usr/lib/u-boot/orangepi_plus/u-boot-sunxi-with-spl= .bin' + uboot_path =3D self.archive_extract(self.ASSET_UBOOT, member=3Dubo= ot_path) image_path_gz =3D self.ASSET_NETBSD.fetch() image_path =3D self.scratch_file('armv7.img') gzip_uncompress(image_path_gz, image_path) diff --git a/tests/functional/test_arm_raspi2.py b/tests/functional/test_ar= m_raspi2.py index 075f6b3301..5e38d1a937 100755 --- a/tests/functional/test_arm_raspi2.py +++ b/tests/functional/test_arm_raspi2.py @@ -35,9 +35,10 @@ def do_test_arm_raspi2(self, uart_id): serial_kernel_cmdline =3D { 0: 'earlycon=3Dpl011,0x3f201000 console=3DttyAMA0', } - deb_path =3D self.ASSET_KERNEL_20190215.fetch() - kernel_path =3D self.extract_from_deb(deb_path, '/boot/kernel7.img= ') - dtb_path =3D self.extract_from_deb(deb_path, '/boot/bcm2709-rpi-2-= b.dtb') + kernel_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/kernel7.img') + dtb_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/bcm2709-rpi-2-b.dtb= ') =20 self.set_machine('raspi2b') self.vm.set_console() @@ -59,9 +60,10 @@ def test_arm_raspi2_uart0(self): self.do_test_arm_raspi2(0) =20 def test_arm_raspi2_initrd(self): - deb_path =3D self.ASSET_KERNEL_20190215.fetch() - kernel_path =3D self.extract_from_deb(deb_path, '/boot/kernel7.img= ') - dtb_path =3D self.extract_from_deb(deb_path, '/boot/bcm2709-rpi-2-= b.dtb') + kernel_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/kernel7.img') + dtb_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, + member=3D'boot/bcm2709-rpi-2-b.dtb= ') initrd_path_gz =3D self.ASSET_INITRD.fetch() initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) diff --git a/tests/functional/test_arm_smdkc210.py b/tests/functional/test_= arm_smdkc210.py index c6c8f9a5f4..f4e86f7e2b 100755 --- a/tests/functional/test_arm_smdkc210.py +++ b/tests/functional/test_arm_smdkc210.py @@ -25,11 +25,10 @@ class Smdkc210Machine(LinuxKernelTest): def test_arm_exynos4210_initrd(self): self.set_machine('smdkc210') =20 - deb_path =3D self.ASSET_DEB.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinuz-4.19.0-6-armmp') - dtb_path =3D '/usr/lib/linux-image-4.19.0-6-armmp/exynos4210-smdkv= 310.dtb' - dtb_path =3D self.extract_from_deb(deb_path, dtb_path) + kernel_path =3D self.archive_extract(self.ASSET_DEB, + member=3D'boot/vmlinuz-4.19.0-6= -armmp') + dtb_path =3D 'usr/lib/linux-image-4.19.0-6-armmp/exynos4210-smdkv3= 10.dtb' + dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) =20 initrd_path_gz =3D self.ASSET_ROOTFS.fetch() initrd_path =3D self.scratch_file('rootfs.cpio') diff --git a/tests/functional/test_arm_vexpress.py b/tests/functional/test_= arm_vexpress.py index b1ac63ac36..6b11552894 100755 --- a/tests/functional/test_arm_vexpress.py +++ b/tests/functional/test_arm_vexpress.py @@ -6,7 +6,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class VExpressTest(LinuxKernelTest): =20 @@ -16,8 +16,7 @@ class VExpressTest(LinuxKernelTest): =20 def test_arm_vexpressa9(self): self.set_machine('vexpress-a9') - file_path =3D self.ASSET_DAY16.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY16) self.launch_kernel(self.scratch_file('day16', 'winter.zImage'), dtb=3Dself.scratch_file('day16', 'vexpress-v2p-ca9.dtb'), diff --git a/tests/functional/test_m68k_mcf5208evb.py b/tests/functional/te= st_m68k_mcf5208evb.py index 449248c3e8..c7d1998933 100755 --- a/tests/functional/test_m68k_mcf5208evb.py +++ b/tests/functional/test_m68k_mcf5208evb.py @@ -6,7 +6,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class Mcf5208EvbTest(LinuxKernelTest): =20 @@ -16,8 +16,7 @@ class Mcf5208EvbTest(LinuxKernelTest): =20 def test_m68k_mcf5208evb(self): self.set_machine('mcf5208evb') - file_path =3D self.ASSET_DAY07.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY07) self.vm.set_console() self.vm.add_args('-kernel', self.scratch_file('day07', 'sanity-clause.elf')) diff --git a/tests/functional/test_m68k_q800.py b/tests/functional/test_m68= k_q800.py index 3b17244b98..400b7aeb5d 100755 --- a/tests/functional/test_m68k_q800.py +++ b/tests/functional/test_m68k_q800.py @@ -18,9 +18,8 @@ class Q800MachineTest(LinuxKernelTest): def test_m68k_q800(self): self.set_machine('q800') =20 - deb_path =3D self.ASSET_KERNEL.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinux-5.3.0-1-m68k') + kernel_path =3D self.archive_extract(self.ASSET_KERNEL, + member=3D'boot/vmlinux-5.3.0-1-= m68k') =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + diff --git a/tests/functional/test_microblaze_s3adsp1800.py b/tests/functio= nal/test_microblaze_s3adsp1800.py index 61c4d6bbf8..2c4464bd05 100755 --- a/tests/functional/test_microblaze_s3adsp1800.py +++ b/tests/functional/test_microblaze_s3adsp1800.py @@ -9,7 +9,7 @@ =20 from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from qemu_test.utils import archive_extract + =20 class MicroblazeMachine(QemuSystemTest): =20 @@ -22,8 +22,7 @@ class MicroblazeMachine(QemuSystemTest): =20 def test_microblaze_s3adsp1800(self): self.set_machine('petalogix-s3adsp1800') - file_path =3D self.ASSET_IMAGE.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_IMAGE) self.vm.set_console() self.vm.add_args('-kernel', self.scratch_file('day17', 'ballerina.bin')) diff --git a/tests/functional/test_microblazeel_s3adsp1800.py b/tests/funct= ional/test_microblazeel_s3adsp1800.py index 926c885f63..c382afe6bf 100755 --- a/tests/functional/test_microblazeel_s3adsp1800.py +++ b/tests/functional/test_microblazeel_s3adsp1800.py @@ -11,7 +11,7 @@ from qemu_test import exec_command, exec_command_and_wait_for_pattern from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from qemu_test.utils import archive_extract + =20 class MicroblazeelMachine(QemuSystemTest): =20 @@ -24,8 +24,7 @@ class MicroblazeelMachine(QemuSystemTest): def test_microblazeel_s3adsp1800(self): self.require_netdev('user') self.set_machine('petalogix-s3adsp1800') - file_path =3D self.ASSET_IMAGE.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_IMAGE) self.vm.set_console() self.vm.add_args('-kernel', self.scratch_file('day13', 'xmaton.bin= ')) tftproot =3D self.scratch_file('day13') diff --git a/tests/functional/test_mips64el_fuloong2e.py b/tests/functional= /test_mips64el_fuloong2e.py index 531d16d8f6..35e500b022 100755 --- a/tests/functional/test_mips64el_fuloong2e.py +++ b/tests/functional/test_mips64el_fuloong2e.py @@ -26,9 +26,9 @@ class MipsFuloong2e(LinuxKernelTest): '2a70f15b397f4ced632b0c15cb22660394190644146d804d60a4796eefbe1f50') =20 def test_linux_kernel_3_16(self): - deb_path =3D self.ASSET_KERNEL.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinux-3.16.0-6-loongs= on-2e') + kernel_path =3D self.archive_extract( + self.ASSET_KERNEL, + member=3D'boot/vmlinux-3.16.0-6-loongson-2e') =20 self.set_machine('fuloong2e') self.vm.set_console() diff --git a/tests/functional/test_mips64el_malta.py b/tests/functional/tes= t_mips64el_malta.py index ea362cf335..39fafb8bf1 100755 --- a/tests/functional/test_mips64el_malta.py +++ b/tests/functional/test_mips64el_malta.py @@ -39,9 +39,9 @@ def test_mips64el_malta(self): [2] https://kernel-team.pages.debian.net/kernel-handbook/ ch-common-tasks.html#s-common-official """ - deb_path =3D self.ASSET_KERNEL_2_63_2.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinux-2.6.32-5-5kc-ma= lta') + kernel_path =3D self.archive_extract( + self.ASSET_KERNEL_2_63_2, + member=3D'boot/vmlinux-2.6.32-5-5kc-malta') =20 self.set_machine('malta') self.vm.set_console() diff --git a/tests/functional/test_mips_malta.py b/tests/functional/test_mi= ps_malta.py index a6d80d0012..6ab6c0832a 100755 --- a/tests/functional/test_mips_malta.py +++ b/tests/functional/test_mips_malta.py @@ -20,9 +20,9 @@ class MaltaMachineConsole(LinuxKernelTest): '16ca524148afb0626f483163e5edf352bc1ab0e4fc7b9f9d473252762f2c7a43') =20 def test_mips_malta(self): - deb_path =3D self.ASSET_KERNEL_2_63_2.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinux-2.6.32-5-4kc-ma= lta') + kernel_path =3D self.archive_extract( + self.ASSET_KERNEL_2_63_2, + member=3D'boot/vmlinux-2.6.32-5-4kc-malta') =20 self.set_machine('malta') self.vm.set_console() @@ -46,9 +46,9 @@ def test_mips_malta(self): 'dcfe3a7fe3200da3a00d176b95caaa086495eb158f2bff64afc67d7e1eb2cddc') =20 def test_mips_malta_cpio(self): - deb_path =3D self.ASSET_KERNEL_4_5_0.fetch() - kernel_path =3D self.extract_from_deb(deb_path, - '/boot/vmlinux-4.5.0-2-4kc-mal= ta') + kernel_path =3D self.archive_extract( + self.ASSET_KERNEL_4_5_0, + member=3D'boot/vmlinux-4.5.0-2-4kc-malta') initrd_path_gz =3D self.ASSET_INITRD.fetch() initrd_path =3D self.scratch_file('rootfs.cpio') gzip_uncompress(initrd_path_gz, initrd_path) diff --git a/tests/functional/test_mipsel_malta.py b/tests/functional/test_= mipsel_malta.py index 77671e0081..1f44881c78 100755 --- a/tests/functional/test_mipsel_malta.py +++ b/tests/functional/test_mipsel_malta.py @@ -13,7 +13,6 @@ from qemu_test import interrupt_interactive_console_until_pattern from qemu_test import wait_for_console_pattern from qemu_test.utils import lzma_uncompress -from zipfile import ZipFile =20 =20 class MaltaMachineConsole(LinuxKernelTest): @@ -73,9 +72,7 @@ class MaltaMachineYAMON(QemuSystemTest): =20 def test_mipsel_malta_yamon(self): yamon_bin =3D 'yamon-02.22.bin' - zip_path =3D self.ASSET_YAMON_ROM.fetch() - with ZipFile(zip_path, 'r') as zf: - zf.extract(yamon_bin, path=3Dself.workdir) + self.archive_extract(self.ASSET_YAMON_ROM) yamon_path =3D self.scratch_file(yamon_bin) =20 self.set_machine('malta') diff --git a/tests/functional/test_or1k_sim.py b/tests/functional/test_or1k= _sim.py index a5b2b5b1e5..f9f0b690a0 100755 --- a/tests/functional/test_or1k_sim.py +++ b/tests/functional/test_or1k_sim.py @@ -6,7 +6,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class OpenRISC1kSimTest(LinuxKernelTest): =20 @@ -16,8 +16,7 @@ class OpenRISC1kSimTest(LinuxKernelTest): =20 def test_or1k_sim(self): self.set_machine('or1k-sim') - file_path =3D self.ASSET_DAY20.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY20) self.vm.set_console() self.vm.add_args('-kernel', self.scratch_file('day20', 'vmlinux')) self.vm.launch() diff --git a/tests/functional/test_ppc64_e500.py b/tests/functional/test_pp= c64_e500.py index bf4a6af9d4..b92fe0b0e7 100755 --- a/tests/functional/test_ppc64_e500.py +++ b/tests/functional/test_ppc64_e500.py @@ -5,7 +5,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class E500Test(LinuxKernelTest): =20 @@ -16,8 +16,7 @@ class E500Test(LinuxKernelTest): def test_ppc64_e500(self): self.set_machine('ppce500') self.cpu =3D 'e5500' - file_path =3D self.ASSET_DAY19.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY19) self.launch_kernel(self.scratch_file('day19', 'uImage'), wait_for=3D'QEMU advent calendar') =20 diff --git a/tests/functional/test_ppc_amiga.py b/tests/functional/test_ppc= _amiga.py index 9ed23a1f0f..8600e2e963 100755 --- a/tests/functional/test_ppc_amiga.py +++ b/tests/functional/test_ppc_amiga.py @@ -11,7 +11,7 @@ =20 from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from zipfile import ZipFile + =20 class AmigaOneMachine(QemuSystemTest): =20 @@ -26,9 +26,7 @@ def test_ppc_amigaone(self): self.require_accelerator("tcg") self.set_machine('amigaone') tar_name =3D 'A1Firmware_Floppy_05-Mar-2005.zip' - zip_file =3D self.ASSET_IMAGE.fetch() - with ZipFile(zip_file, 'r') as zf: - zf.extractall(path=3Dself.workdir) + self.archive_extract(self.ASSET_IMAGE, format=3D"zip") bios =3D self.scratch_file("u-boot-amigaone.bin") with open(bios, "wb") as bios_fh: subprocess.run(['tail', '-c', '524288', diff --git a/tests/functional/test_ppc_bamboo.py b/tests/functional/test_pp= c_bamboo.py index 1ae2f47bcc..fddcc24d0d 100755 --- a/tests/functional/test_ppc_bamboo.py +++ b/tests/functional/test_ppc_bamboo.py @@ -7,11 +7,11 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -from qemu_test.utils import archive_extract from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern from qemu_test import exec_command_and_wait_for_pattern =20 + class BambooMachine(QemuSystemTest): =20 timeout =3D 90 @@ -25,8 +25,7 @@ def test_ppc_bamboo(self): self.set_machine('bamboo') self.require_accelerator("tcg") self.require_netdev('user') - file_path =3D self.ASSET_IMAGE.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_IMAGE) self.vm.set_console() self.vm.add_args('-kernel', self.scratch_file('system-image-powerpc-440fp', diff --git a/tests/functional/test_ppc_mac.py b/tests/functional/test_ppc_m= ac.py index 10812824bd..9e4bc1a52c 100755 --- a/tests/functional/test_ppc_mac.py +++ b/tests/functional/test_ppc_mac.py @@ -5,7 +5,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class MacTest(LinuxKernelTest): =20 @@ -19,9 +19,7 @@ def do_day15_test(self): # we're running kvm_hv or kvm_pr. For now let's disable this test # if we don't have TCG support. self.require_accelerator("tcg") - - file_path =3D self.ASSET_DAY15.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY15) self.vm.add_args('-M', 'graphics=3Doff') self.launch_kernel(self.scratch_file('day15', 'invaders.elf'), wait_for=3D'QEMU advent calendar') diff --git a/tests/functional/test_ppc_mpc8544ds.py b/tests/functional/test= _ppc_mpc8544ds.py index 87b5d4d12b..0715410d7a 100755 --- a/tests/functional/test_ppc_mpc8544ds.py +++ b/tests/functional/test_ppc_mpc8544ds.py @@ -7,10 +7,10 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -from qemu_test.utils import archive_extract from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern =20 + class Mpc8544dsMachine(QemuSystemTest): =20 timeout =3D 90 @@ -25,10 +25,10 @@ class Mpc8544dsMachine(QemuSystemTest): def test_ppc_mpc8544ds(self): self.require_accelerator("tcg") self.set_machine('mpc8544ds') - file_path =3D self.ASSET_IMAGE.fetch() - archive_extract(file_path, self.workdir, member=3D'creek/creek.bin= ') + kernel_file =3D self.archive_extract(self.ASSET_IMAGE, + member=3D'creek/creek.bin') self.vm.set_console() - self.vm.add_args('-kernel', self.scratch_file('creek', 'creek.bin'= )) + self.vm.add_args('-kernel', kernel_file) self.vm.launch() wait_for_console_pattern(self, 'QEMU advent calendar 2020', self.panic_message) diff --git a/tests/functional/test_ppc_virtex_ml507.py b/tests/functional/t= est_ppc_virtex_ml507.py index f297651e64..8fe43549b7 100755 --- a/tests/functional/test_ppc_virtex_ml507.py +++ b/tests/functional/test_ppc_virtex_ml507.py @@ -7,10 +7,10 @@ # This work is licensed under the terms of the GNU GPL, version 2 or # later. See the COPYING file in the top-level directory. =20 -from qemu_test.utils import archive_extract from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern =20 + class VirtexMl507Machine(QemuSystemTest): =20 timeout =3D 90 @@ -25,8 +25,7 @@ class VirtexMl507Machine(QemuSystemTest): def test_ppc_virtex_ml507(self): self.require_accelerator("tcg") self.set_machine('virtex-ml507') - file_path =3D self.ASSET_IMAGE.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_IMAGE) self.vm.set_console() self.vm.add_args('-kernel', self.scratch_file('hippo', 'hippo.linu= x'), '-dtb', self.scratch_file('hippo', diff --git a/tests/functional/test_sh4_r2d.py b/tests/functional/test_sh4_r= 2d.py index dca4601392..03a648374c 100755 --- a/tests/functional/test_sh4_r2d.py +++ b/tests/functional/test_sh4_r2d.py @@ -5,7 +5,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset, skipFlakyTest -from qemu_test.utils import archive_extract + =20 class R2dTest(LinuxKernelTest): =20 @@ -19,8 +19,7 @@ class R2dTest(LinuxKernelTest): @skipFlakyTest(bug_url=3DNone) def test_r2d(self): self.set_machine('r2d') - file_path =3D self.ASSET_DAY09.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY09) self.vm.add_args('-append', 'console=3DttySC1') self.launch_kernel(self.scratch_file('day09', 'zImage'), console_index=3D1, diff --git a/tests/functional/test_sh4eb_r2d.py b/tests/functional/test_sh4= eb_r2d.py index c8954c93eb..473093bbe1 100755 --- a/tests/functional/test_sh4eb_r2d.py +++ b/tests/functional/test_sh4eb_r2d.py @@ -6,7 +6,7 @@ =20 from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern -from qemu_test.utils import archive_extract + =20 class R2dEBTest(LinuxKernelTest): =20 @@ -16,8 +16,7 @@ class R2dEBTest(LinuxKernelTest): =20 def test_sh4eb_r2d(self): self.set_machine('r2d') - file_path =3D self.ASSET_TGZ.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_TGZ) self.vm.add_args('-append', 'console=3DttySC1 noiotrap') self.launch_kernel(self.scratch_file('sh4eb', 'linux-kernel'), initrd=3Dself.scratch_file('sh4eb', diff --git a/tests/functional/test_sparc64_sun4u.py b/tests/functional/test= _sparc64_sun4u.py index e7f6db0f24..27ac289659 100755 --- a/tests/functional/test_sparc64_sun4u.py +++ b/tests/functional/test_sparc64_sun4u.py @@ -12,7 +12,6 @@ =20 from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern -from qemu_test.utils import archive_extract =20 =20 class Sun4uMachine(QemuSystemTest): @@ -27,11 +26,10 @@ class Sun4uMachine(QemuSystemTest): =20 def test_sparc64_sun4u(self): self.set_machine('sun4u') - file_path =3D self.ASSET_IMAGE.fetch() - kernel_name =3D 'day23/vmlinux' - archive_extract(file_path, self.workdir, kernel_name) + kernel_file =3D self.archive_extract(self.ASSET_IMAGE, + member=3D'day23/vmlinux') self.vm.set_console() - self.vm.add_args('-kernel', self.scratch_file(kernel_name), + self.vm.add_args('-kernel', kernel_file, '-append', 'printk.time=3D0') self.vm.launch() wait_for_console_pattern(self, 'Starting logging: OK') diff --git a/tests/functional/test_sparc_sun4m.py b/tests/functional/test_s= parc_sun4m.py index 619c03d36a..7cd28ebdd1 100755 --- a/tests/functional/test_sparc_sun4m.py +++ b/tests/functional/test_sparc_sun4m.py @@ -6,7 +6,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class Sun4mTest(LinuxKernelTest): =20 @@ -16,8 +16,7 @@ class Sun4mTest(LinuxKernelTest): =20 def test_sparc_ss20(self): self.set_machine('SS-20') - file_path =3D self.ASSET_DAY11.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY11) self.launch_kernel(self.scratch_file('day11', 'zImage.elf'), wait_for=3D'QEMU advent calendar') =20 diff --git a/tests/functional/test_xtensa_lx60.py b/tests/functional/test_x= tensa_lx60.py index 5048e4c69e..147c920899 100755 --- a/tests/functional/test_xtensa_lx60.py +++ b/tests/functional/test_xtensa_lx60.py @@ -6,7 +6,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import archive_extract + =20 class XTensaLX60Test(LinuxKernelTest): =20 @@ -17,8 +17,7 @@ class XTensaLX60Test(LinuxKernelTest): def test_xtensa_lx60(self): self.set_machine('lx60') self.cpu =3D 'dc233c' - file_path =3D self.ASSET_DAY02.fetch() - archive_extract(file_path, self.workdir) + self.archive_extract(self.ASSET_DAY02) self.launch_kernel(self.scratch_file('day02', 'santas-sleigh-ride.elf'), wait_for=3D'QEMU advent calendar') --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451903; cv=none; d=zohomail.com; s=zohoarc; b=koBMbJJBxS/lYMOYaMlyegz027wHrxSOG6CGkOkSXUMS7x2ItzU/DUmp/F0R+jip9cUxNKXA5keUHeBECZIdJWnNDjHpGUNpSrHmhbWqRiCEl/LtpBmP3D7ImxPRB4wurxtGybkMnQCShqvUia6XtbNLfH+bxf52rFVEYAMSi40= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451903; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=shEyQbtzxEHCSLnIh2H3SKPNkLOycBJ1B5KXtp/FWSI=; b=IlJoXMf8WyVIPkli+zU4MCD5DCBmODM+DzumCsTs6FqBWGiFZR3FRIkXeqGp5Abk4xlrgFlR/4p8soeGSjRxdOJ0/pJdAPx9Z2Asag1ujcToJf7GXwodCrYbnVNXo+4LglNOrXaEuzEtkXoxggnBk/UvC1HIPaebdsesyjzHUcw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451903983839.7248628473774; Tue, 17 Dec 2024 08:11:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0i-0005LG-Tn; Tue, 17 Dec 2024 11:01:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0b-0004wm-VV for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:18 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0a-0002D8-CT for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:17 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-187-OoHMDI_DNbqltfzrtdnWsg-1; Tue, 17 Dec 2024 11:01:11 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A0BB819560A3; Tue, 17 Dec 2024 16:01:10 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 44618300F9B5; Tue, 17 Dec 2024 16:01:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=shEyQbtzxEHCSLnIh2H3SKPNkLOycBJ1B5KXtp/FWSI=; b=d91F+VQHO/0R6H9ilPZjoRvz1cfAwdgf6nU9dUbPCw8WIDCUal8yqPV9wakHjS+JZ+wbDc IjAn3CfsU8jaAZvtJhCcSGHvjSjZi5zHIBIlQiIchLO/7NoZtMg2dCZLCwIzYSOTsza7dC DJdlMr90385pW0Kcl7LSKldCb9SkCPM= X-MC-Unique: OoHMDI_DNbqltfzrtdnWsg-1 X-Mimecast-MFC-AGG-ID: OoHMDI_DNbqltfzrtdnWsg From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 24/32] tests/functional: add a generalized uncompress helper Date: Tue, 17 Dec 2024 15:59:45 +0000 Message-ID: <20241217155953.3950506-25-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451904891116600 There are many types of compression that the tests deal with, and it makes sense to have a single helper 'uncompress' that can deal with all. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/__init__.py | 1 + tests/functional/qemu_test/uncompress.py | 47 ++++++++++++++++++++++++ 2 files changed, 48 insertions(+) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu= _test/__init__.py index 665c482d13..3bd043e608 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -17,3 +17,4 @@ skipFlakyTest, skipUntrustedTest, skipBigDataTest, \ skipIfMissingImports from .archive import archive_extract +from .uncompress import uncompress diff --git a/tests/functional/qemu_test/uncompress.py b/tests/functional/qe= mu_test/uncompress.py index 955170df65..6d02ded066 100644 --- a/tests/functional/qemu_test/uncompress.py +++ b/tests/functional/qemu_test/uncompress.py @@ -11,6 +11,9 @@ import lzma import os import shutil +from urllib.parse import urlparse + +from .asset import Asset =20 =20 def gzip_uncompress(gz_path, output_path): @@ -34,3 +37,47 @@ def lzma_uncompress(xz_path, output_path): except: os.remove(output_path) raise + +''' +@params compressed: filename, Asset, or file-like object to uncompress +@params uncompressed: filename to uncompress into +@params format: optional compression format (gzip, lzma) + +Uncompresses @compressed into @uncompressed + +If @format is None, heuristics will be applied to guess the format +from the filename or Asset URL. @format must be non-None if @uncompressed +is a file-like object. + +Returns the fully qualified path to the uncompessed file +''' +def uncompress(compressed, uncompressed, format=3DNone): + if format is None: + format =3D guess_uncompress_format(compressed) + + if format =3D=3D "xz": + lzma_uncompress(str(compressed), uncompressed) + elif format =3D=3D "gz": + gzip_uncompress(str(compressed), uncompressed) + else: + raise Exception(f"Unknown compression format {format}") + +''' +@params compressed: filename, Asset, or file-like object to guess + +Guess the format of @compressed, raising an exception if +no format can be determined +''' +def guess_uncompress_format(compressed): + if type(compressed) =3D=3D Asset: + compressed =3D urlparse(compressed.url).path + elif type(compressed) !=3D str: + raise Exception(f"Unable to guess compression cformat for {compres= sed}") + + (name, ext) =3D os.path.splitext(compressed) + if ext =3D=3D ".xz": + return "xz" + elif ext =3D=3D ".gz": + return "gz" + else: + raise Exception(f"Unknown compression format for {compressed}") --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451840; cv=none; d=zohomail.com; s=zohoarc; b=RNotAiiJn3QY713LHbsV2L6uFOhmUnbwIdgB028uN2i6l5S+Lde41ldXpnDWlxjzHuP534d7FUkhpPa3QmYU3K5WZgLpGo1LMBzhMknbxM9yqCrY4yAIKfUhQ8IpQJ8mTqGWMDfbW9CX9EiMN85dMyV+DtKvc+jYNoT/lB9IHHo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451840; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=GOP9T65Y1N1eKANKaDuXNmoW4XhSoOjwKVDIlKN5TO4=; b=ifv2pL2mVXaGUGb28HhogEpugJCmqyfCrkGSX3Qr5kspNtImiDHhyFvNk8nr3e+ViU21TIqAPN/yYe4n7jNVwDq4F83gNOmglLyQRxNkRna8A8OsN9Dc6APxWlr8gTGQue1oIzpKkAQX1Y6IiGXeHMNck53FW12gHe1Kk1cWGok= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451840486305.7509979215895; Tue, 17 Dec 2024 08:10:40 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0m-0005S4-6g; Tue, 17 Dec 2024 11:01:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0i-0005KC-0M for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0d-0002Dj-De for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:23 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-611-EpPgYyWyMRCQ-tWd2abLyQ-1; Tue, 17 Dec 2024 11:01:14 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 80267195609D; Tue, 17 Dec 2024 16:01:13 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3F15D300F9B7; Tue, 17 Dec 2024 16:01:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GOP9T65Y1N1eKANKaDuXNmoW4XhSoOjwKVDIlKN5TO4=; b=EElsX4/NfTH0JhD8BhwLDcz1TmYZ7PJ5fBy/X3EazRo+A6/CHftMZQt/5wZ5mPbLg4dXBD FHZBSvq2ICo/rfUdxKu202QENE0Wd4HWBQzIDFIGnHfSwx2tIfB9MdMyfNUjK90F/MSN2m nMR+A1+XNHVLD6UWNeAeW5x/hEyZ6Hk= X-MC-Unique: EpPgYyWyMRCQ-tWd2abLyQ-1 X-Mimecast-MFC-AGG-ID: EpPgYyWyMRCQ-tWd2abLyQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 25/32] tests/functional: add 'uncompress' to QemuBaseTest Date: Tue, 17 Dec 2024 15:59:46 +0000 Message-ID: <20241217155953.3950506-26-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451842561116600 This helper wrappers utils.uncompress, forcing the use of the scratch directory, to ensure any uncompressed files are cleaned at test termination. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/testcase.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index 19fb1d0c07..d0bb3141d5 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -29,6 +29,7 @@ from .asset import Asset from .cmd import run_cmd from .config import BUILD_DIR +from .uncompress import uncompress =20 =20 class QemuBaseTest(unittest.TestCase): @@ -40,6 +41,30 @@ class QemuBaseTest(unittest.TestCase): log =3D None logdir =3D None =20 + ''' + @params compressed: filename, Asset, or file-like object to uncompress + @params format: optional compression format (gzip, lzma) + + Uncompresses @compressed into the scratch directory. + + If @format is None, heuristics will be applied to guess the format + from the filename or Asset URL. @format must be non-None if @uncompres= sed + is a file-like object. + + Returns the fully qualified path to the uncompressed file + ''' + def uncompress(self, compressed, format=3DNone): + self.log.debug(f"Uncompress {compressed} format=3D{format}") + if type(compressed) =3D=3D Asset: + compressed.fetch() + + (name, ext) =3D os.path.splitext(str(compressed)) + uncompressed =3D self.scratch_file(os.path.basename(name)) + + uncompress(compressed, uncompressed, format) + + return uncompressed + ''' @params archive: filename, Asset, or file-like object to extract @params format: optional archive format (tar, zip, deb, cpio) --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451807; cv=none; d=zohomail.com; s=zohoarc; b=T9g1CTgzrlZwAN1bsd5FjxDk2q6xkC/l2sCBhTfW4Joaqx/1o3jnAqJ0kmZb4TpIt/3brYt9IxPudXKq+9uHmifasZfbSgQGlC0HSsbnSybP4+oQrWIIGmU6CFE37vGu+YUm+bNTnj/MNM7zSB0ywBfpfxcdOXWIXkEIqzNfyEc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451807; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Ch76bbC64yQqfIteF7QKTIH/aMk7uJFkuVjRD/eCd+0=; b=idYPiRo3Z0cAeJ6Ir5WTJLqVBUFH5fC2I+1uLGUOsvGa2Z+W11ULoFPPj/bZfuZrRs0xFbLyBlXKY3ObW97/0A6Lbu0Xb94hvuMnMnyr4UGev1fueeIw12FfMhBwKObzB5JZ405u14KosV0xdXWGCsiCZ5u+YUYNTUjQdVGjqRQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451807879920.9041683431254; Tue, 17 Dec 2024 08:10:07 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0r-0005Yk-Es; Tue, 17 Dec 2024 11:01:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0k-0005P2-9Z for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0g-0002E6-Cw for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:26 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-640-jMwnZR-ePZueKxF_0UxW3Q-1; Tue, 17 Dec 2024 11:01:17 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7BB5119560B3; Tue, 17 Dec 2024 16:01:16 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1CABF30044C1; Tue, 17 Dec 2024 16:01:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451281; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ch76bbC64yQqfIteF7QKTIH/aMk7uJFkuVjRD/eCd+0=; b=f9f+mlH1Xe96qOJeOelCX/kWlHMViqLhVHFNJs3j9Kdy5saO8oCtot1i4oRJn+p/vWbnvc i5W3fSUoz8crIKWAD2TmXVG/j7fFbnjaLvrFBwoVG7UCLe1EvKB62YnpZJusVsbKFTkuCj WpozNvJv4ms01UIpuVZGxwq9vbVBFfE= X-MC-Unique: jMwnZR-ePZueKxF_0UxW3Q-1 X-Mimecast-MFC-AGG-ID: jMwnZR-ePZueKxF_0UxW3Q From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 26/32] tests/functional: convert tests to new uncompress helper Date: Tue, 17 Dec 2024 15:59:47 +0000 Message-ID: <20241217155953.3950506-27-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451808962116600 Replace use of lzma_uncompress and gzip_uncompress with the new uncompress helper. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/test_aarch64_raspi4.py | 5 +---- tests/functional/test_aarch64_sbsaref.py | 10 +++------- tests/functional/test_alpha_clipper.py | 4 +--- tests/functional/test_arm_bpim2u.py | 13 +++---------- tests/functional/test_arm_cubieboard.py | 14 ++++---------- tests/functional/test_arm_orangepi.py | 17 ++++------------- tests/functional/test_arm_raspi2.py | 5 +---- tests/functional/test_arm_smdkc210.py | 6 ++---- tests/functional/test_mips64el_malta.py | 9 ++------- tests/functional/test_mips_malta.py | 5 +---- tests/functional/test_mipsel_malta.py | 15 +++++---------- tests/functional/test_rx_gdbsim.py | 5 +---- tests/functional/test_s390x_ccw_virtio.py | 6 ++---- tests/functional/test_s390x_topology.py | 5 +---- 14 files changed, 31 insertions(+), 88 deletions(-) diff --git a/tests/functional/test_aarch64_raspi4.py b/tests/functional/tes= t_aarch64_raspi4.py index 3918e35e82..7a4302b0c5 100755 --- a/tests/functional/test_aarch64_raspi4.py +++ b/tests/functional/test_aarch64_raspi4.py @@ -7,7 +7,6 @@ =20 from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern -from qemu_test.utils import gzip_uncompress =20 =20 class Aarch64Raspi4Machine(LinuxKernelTest): @@ -63,9 +62,7 @@ def test_arm_raspi4_initrd(self): member=3D'boot/kernel8.img') dtb_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, member=3D'boot/bcm2711-rpi-4-b.dtb= ') - initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_INITRD) =20 self.set_machine('raspi4b') self.vm.set_console() diff --git a/tests/functional/test_aarch64_sbsaref.py b/tests/functional/te= st_aarch64_sbsaref.py index 533ca64407..2d756efdab 100755 --- a/tests/functional/test_aarch64_sbsaref.py +++ b/tests/functional/test_aarch64_sbsaref.py @@ -11,7 +11,7 @@ from qemu_test import QemuSystemTest, Asset from qemu_test import wait_for_console_pattern from qemu_test import interrupt_interactive_console_until_pattern -from qemu_test.utils import lzma_uncompress + =20 def fetch_firmware(test): """ @@ -29,14 +29,10 @@ def fetch_firmware(test): """ =20 # Secure BootRom (TF-A code) - fs0_xz_path =3D Aarch64SbsarefMachine.ASSET_FLASH0.fetch() - fs0_path =3D test.scratch_file("SBSA_FLASH0.fd") - lzma_uncompress(fs0_xz_path, fs0_path) + fs0_path =3D test.uncompress(Aarch64SbsarefMachine.ASSET_FLASH0) =20 # Non-secure rom (UEFI and EFI variables) - fs1_xz_path =3D Aarch64SbsarefMachine.ASSET_FLASH1.fetch() - fs1_path =3D test.scratch_file("SBSA_FLASH1.fd") - lzma_uncompress(fs1_xz_path, fs1_path) + fs1_path =3D test.uncompress(Aarch64SbsarefMachine.ASSET_FLASH1) =20 for path in [fs0_path, fs1_path]: with open(path, "ab+") as fd: diff --git a/tests/functional/test_alpha_clipper.py b/tests/functional/test= _alpha_clipper.py index 72cd7b57e6..c5d7181953 100755 --- a/tests/functional/test_alpha_clipper.py +++ b/tests/functional/test_alpha_clipper.py @@ -6,7 +6,6 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import gzip_uncompress =20 =20 class AlphaClipperTest(LinuxKernelTest): @@ -20,8 +19,7 @@ def test_alpha_clipper(self): self.set_machine('clipper') kernel_path =3D self.ASSET_KERNEL.fetch() =20 - uncompressed_kernel =3D self.scratch_file('vmlinux') - gzip_uncompress(kernel_path, uncompressed_kernel) + uncompressed_kernel =3D self.uncompress(self.ASSET_KERNEL, format= =3D"gz") =20 self.vm.set_console() kernel_command_line =3D self.KERNEL_COMMON_COMMAND_LINE + 'console= =3DttyS0' diff --git a/tests/functional/test_arm_bpim2u.py b/tests/functional/test_ar= m_bpim2u.py index 91c56b0930..12cd359746 100755 --- a/tests/functional/test_arm_bpim2u.py +++ b/tests/functional/test_arm_bpim2u.py @@ -10,7 +10,6 @@ from qemu_test import LinuxKernelTest, exec_command_and_wait_for_pattern from qemu_test import Asset, interrupt_interactive_console_until_pattern from qemu_test import skipBigDataTest -from qemu_test.utils import gzip_uncompress, lzma_uncompress from qemu_test.utils import image_pow2ceil_expand =20 =20 @@ -65,9 +64,7 @@ def test_arm_bpim2u_initrd(self): dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' 'sun8i-r40-bananapi-m2-ultra.dtb') dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) - initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_INITRD) =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + @@ -103,9 +100,7 @@ def test_arm_bpim2u_gmac(self): dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' 'sun8i-r40-bananapi-m2-ultra.dtb') dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) - rootfs_path_xz =3D self.ASSET_ROOTFS.fetch() - rootfs_path =3D self.scratch_file('rootfs.cpio') - lzma_uncompress(rootfs_path_xz, rootfs_path) + rootfs_path =3D self.uncompress(self.ASSET_ROOTFS) image_pow2ceil_expand(rootfs_path) =20 self.vm.set_console() @@ -147,9 +142,7 @@ def test_arm_bpim2u_openwrt_22_03_3(self): self.set_machine('bpim2u') # This test download a 8.9 MiB compressed image and expand it # to 127 MiB. - image_path_gz =3D self.ASSET_SD_IMAGE.fetch() - image_path =3D self.scratch_file('sdcard.img') - gzip_uncompress(image_path_gz, image_path) + image_path =3D self.uncompress(self.ASSET_SD_IMAGE) image_pow2ceil_expand(image_path) =20 self.vm.set_console() diff --git a/tests/functional/test_arm_cubieboard.py b/tests/functional/tes= t_arm_cubieboard.py index 32db303253..423db710e8 100755 --- a/tests/functional/test_arm_cubieboard.py +++ b/tests/functional/test_arm_cubieboard.py @@ -9,7 +9,7 @@ from qemu_test import LinuxKernelTest, Asset, exec_command_and_wait_for_pa= ttern from qemu_test import interrupt_interactive_console_until_pattern from qemu_test import skipBigDataTest -from qemu_test.utils import gzip_uncompress, image_pow2ceil_expand +from qemu_test.utils import image_pow2ceil_expand =20 =20 class CubieboardMachine(LinuxKernelTest): @@ -43,9 +43,7 @@ def test_arm_cubieboard_initrd(self): dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + 'sun4i-a10-cubieboard.dtb') dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) - initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_INITRD) =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + @@ -77,9 +75,7 @@ def test_arm_cubieboard_sata(self): 'sun4i-a10-cubieboard.dtb') dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) =20 - rootfs_path_gz =3D self.ASSET_SATA_ROOTFS.fetch() - rootfs_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(rootfs_path_gz, rootfs_path) + rootfs_path =3D self.uncompress(self.ASSET_SATA_ROOTFS) =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + @@ -111,9 +107,7 @@ def test_arm_cubieboard_openwrt_22_03_2(self): # This test download a 7.5 MiB compressed image and expand it # to 126 MiB. self.set_machine('cubieboard') - image_path_gz =3D self.ASSET_OPENWRT.fetch() - image_path =3D self.scratch_file('sdcard.img') - gzip_uncompress(image_path_gz, image_path) + image_path =3D self.uncompress(self.ASSET_OPENWRT) image_pow2ceil_expand(image_path) =20 self.vm.set_console() diff --git a/tests/functional/test_arm_orangepi.py b/tests/functional/test_= arm_orangepi.py index aa2d9d19a4..18ee50216b 100755 --- a/tests/functional/test_arm_orangepi.py +++ b/tests/functional/test_arm_orangepi.py @@ -11,7 +11,6 @@ from qemu_test import LinuxKernelTest, exec_command_and_wait_for_pattern from qemu_test import Asset, interrupt_interactive_console_until_pattern from qemu_test import wait_for_console_pattern, skipBigDataTest -from qemu_test.utils import gzip_uncompress, lzma_uncompress from qemu_test.utils import image_pow2ceil_expand =20 =20 @@ -76,9 +75,7 @@ def test_arm_orangepi_initrd(self): dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + 'sun8i-h3-orangepi-pc.dtb') dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) - initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_INITRD) =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + @@ -112,9 +109,7 @@ def test_arm_orangepi_sd(self): dtb_path =3D ('usr/lib/linux-image-6.6.16-current-sunxi/' + 'sun8i-h3-orangepi-pc.dtb') dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) - rootfs_path_xz =3D self.ASSET_ROOTFS.fetch() - rootfs_path =3D self.scratch_file('rootfs.cpio') - lzma_uncompress(rootfs_path_xz, rootfs_path) + rootfs_path =3D self.uncompress(self.ASSET_ROOTFS) image_pow2ceil_expand(rootfs_path) =20 self.vm.set_console() @@ -155,9 +150,7 @@ def test_arm_orangepi_armbian(self): # This test download a 275 MiB compressed image and expand it # to 1036 MiB, but the underlying filesystem is 1552 MiB... # As we expand it to 2 GiB we are safe. - image_path_xz =3D self.ASSET_ARMBIAN.fetch() - image_path =3D self.scratch_file('armbian.img') - lzma_uncompress(image_path_xz, image_path) + image_path =3D self.uncompress(self.ASSET_ARMBIAN) image_pow2ceil_expand(image_path) =20 self.vm.set_console() @@ -195,9 +188,7 @@ def test_arm_orangepi_uboot_netbsd9(self): # before to boot NetBSD. uboot_path =3D 'usr/lib/u-boot/orangepi_plus/u-boot-sunxi-with-spl= .bin' uboot_path =3D self.archive_extract(self.ASSET_UBOOT, member=3Dubo= ot_path) - image_path_gz =3D self.ASSET_NETBSD.fetch() - image_path =3D self.scratch_file('armv7.img') - gzip_uncompress(image_path_gz, image_path) + image_path =3D self.uncompress(self.ASSET_NETBSD) image_pow2ceil_expand(image_path) image_drive_args =3D 'if=3Dsd,format=3Draw,snapshot=3Don,file=3D' = + image_path =20 diff --git a/tests/functional/test_arm_raspi2.py b/tests/functional/test_ar= m_raspi2.py index 5e38d1a937..d3c7aaa39b 100755 --- a/tests/functional/test_arm_raspi2.py +++ b/tests/functional/test_arm_raspi2.py @@ -9,7 +9,6 @@ =20 from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern -from qemu_test.utils import gzip_uncompress =20 =20 class ArmRaspi2Machine(LinuxKernelTest): @@ -64,9 +63,7 @@ def test_arm_raspi2_initrd(self): member=3D'boot/kernel7.img') dtb_path =3D self.archive_extract(self.ASSET_KERNEL_20190215, member=3D'boot/bcm2709-rpi-2-b.dtb= ') - initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_INITRD) =20 self.set_machine('raspi2b') self.vm.set_console() diff --git a/tests/functional/test_arm_smdkc210.py b/tests/functional/test_= arm_smdkc210.py index f4e86f7e2b..0fda45c63a 100755 --- a/tests/functional/test_arm_smdkc210.py +++ b/tests/functional/test_arm_smdkc210.py @@ -7,7 +7,7 @@ import os =20 from qemu_test import LinuxKernelTest, Asset -from qemu_test.utils import gzip_uncompress + =20 class Smdkc210Machine(LinuxKernelTest): =20 @@ -30,9 +30,7 @@ def test_arm_exynos4210_initrd(self): dtb_path =3D 'usr/lib/linux-image-4.19.0-6-armmp/exynos4210-smdkv3= 10.dtb' dtb_path =3D self.archive_extract(self.ASSET_DEB, member=3Ddtb_pat= h) =20 - initrd_path_gz =3D self.ASSET_ROOTFS.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_ROOTFS) =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + diff --git a/tests/functional/test_mips64el_malta.py b/tests/functional/tes= t_mips64el_malta.py index 39fafb8bf1..a8da15a26b 100755 --- a/tests/functional/test_mips64el_malta.py +++ b/tests/functional/test_mips64el_malta.py @@ -15,7 +15,6 @@ from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test import skipIfMissingImports, skipFlakyTest, skipUntrustedTe= st -from qemu_test.utils import gzip_uncompress =20 =20 class MaltaMachineConsole(LinuxKernelTest): @@ -67,9 +66,7 @@ def test_mips64el_malta(self): @skipUntrustedTest() def test_mips64el_malta_5KEc_cpio(self): kernel_path =3D self.ASSET_KERNEL_3_19_3.fetch() - initrd_path_gz =3D self.ASSET_CPIO_R1.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_CPIO_R1) =20 self.set_machine('malta') self.vm.set_console() @@ -119,9 +116,7 @@ def do_test_i6400_framebuffer_logo(self, cpu_cores_coun= t): =20 screendump_path =3D self.scratch_file('screendump.pbm') =20 - kernel_path_gz =3D self.ASSET_KERNEL_4_7_0.fetch() - kernel_path =3D self.scratch_file("vmlinux") - gzip_uncompress(kernel_path_gz, kernel_path) + kernel_path =3D self.uncompress(self.ASSET_KERNEL_4_7_0) =20 tuxlogo_path =3D self.ASSET_TUXLOGO.fetch() =20 diff --git a/tests/functional/test_mips_malta.py b/tests/functional/test_mi= ps_malta.py index 6ab6c0832a..3b15038d89 100755 --- a/tests/functional/test_mips_malta.py +++ b/tests/functional/test_mips_malta.py @@ -8,7 +8,6 @@ =20 from qemu_test import LinuxKernelTest, Asset from qemu_test import exec_command_and_wait_for_pattern -from qemu_test.utils import gzip_uncompress =20 =20 class MaltaMachineConsole(LinuxKernelTest): @@ -49,9 +48,7 @@ def test_mips_malta_cpio(self): kernel_path =3D self.archive_extract( self.ASSET_KERNEL_4_5_0, member=3D'boot/vmlinux-4.5.0-2-4kc-malta') - initrd_path_gz =3D self.ASSET_INITRD.fetch() - initrd_path =3D self.scratch_file('rootfs.cpio') - gzip_uncompress(initrd_path_gz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_INITRD) =20 self.set_machine('malta') self.vm.set_console() diff --git a/tests/functional/test_mipsel_malta.py b/tests/functional/test_= mipsel_malta.py index 1f44881c78..fe9c3a172e 100755 --- a/tests/functional/test_mipsel_malta.py +++ b/tests/functional/test_mipsel_malta.py @@ -12,7 +12,6 @@ from qemu_test import QemuSystemTest, LinuxKernelTest, Asset from qemu_test import interrupt_interactive_console_until_pattern from qemu_test import wait_for_console_pattern -from qemu_test.utils import lzma_uncompress =20 =20 class MaltaMachineConsole(LinuxKernelTest): @@ -33,9 +32,8 @@ class MaltaMachineConsole(LinuxKernelTest): 'generic_nano32r6el_page64k_dbg.xz'), 'ce21ff4b07a981ecb8a39db2876616f5a2473eb2ab459c6f67465b9914b0c6b6') =20 - def do_test_mips_malta32el_nanomips(self, kernel_path_xz): - kernel_path =3D self.scratch_file('kernel') - lzma_uncompress(kernel_path_xz, kernel_path) + def do_test_mips_malta32el_nanomips(self, kernel): + kernel_path =3D self.uncompress(kernel) =20 self.set_machine('malta') self.vm.set_console() @@ -51,16 +49,13 @@ def do_test_mips_malta32el_nanomips(self, kernel_path_x= z): self.wait_for_console_pattern(console_pattern) =20 def test_mips_malta32el_nanomips_4k(self): - kernel_path_xz =3D self.ASSET_KERNEL_4K.fetch() - self.do_test_mips_malta32el_nanomips(kernel_path_xz) + self.do_test_mips_malta32el_nanomips(self.ASSET_KERNEL_4K) =20 def test_mips_malta32el_nanomips_16k_up(self): - kernel_path_xz =3D self.ASSET_KERNEL_16K.fetch() - self.do_test_mips_malta32el_nanomips(kernel_path_xz) + self.do_test_mips_malta32el_nanomips(self.ASSET_KERNEL_16K) =20 def test_mips_malta32el_nanomips_64k_dbg(self): - kernel_path_xz =3D self.ASSET_KERNEL_16K.fetch() - self.do_test_mips_malta32el_nanomips(kernel_path_xz) + self.do_test_mips_malta32el_nanomips(self.ASSET_KERNEL_64K) =20 =20 class MaltaMachineYAMON(QemuSystemTest): diff --git a/tests/functional/test_rx_gdbsim.py b/tests/functional/test_rx_= gdbsim.py index b0adb38a45..20623aa51c 100755 --- a/tests/functional/test_rx_gdbsim.py +++ b/tests/functional/test_rx_gdbsim.py @@ -13,7 +13,6 @@ from qemu_test import QemuSystemTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test import wait_for_console_pattern, skipFlakyTest -from qemu_test.utils import gzip_uncompress =20 =20 class RxGdbSimMachine(QemuSystemTest): @@ -37,9 +36,7 @@ def test_uboot(self): """ self.set_machine('gdbsim-r5f562n8') =20 - uboot_path_gz =3D self.ASSET_UBOOT.fetch() - uboot_path =3D self.scratch_file('u-boot.bin') - gzip_uncompress(uboot_path_gz, uboot_path) + uboot_path =3D self.uncompress(self.ASSET_UBOOT) =20 self.vm.set_console() self.vm.add_args('-bios', uboot_path, diff --git a/tests/functional/test_s390x_ccw_virtio.py b/tests/functional/t= est_s390x_ccw_virtio.py index e5884a4dd0..453711aa0f 100755 --- a/tests/functional/test_s390x_ccw_virtio.py +++ b/tests/functional/test_s390x_ccw_virtio.py @@ -17,7 +17,7 @@ from qemu_test import QemuSystemTest, Asset from qemu_test import exec_command_and_wait_for_pattern from qemu_test import wait_for_console_pattern -from qemu_test.utils import lzma_uncompress + =20 class S390CCWVirtioMachine(QemuSystemTest): KERNEL_COMMON_COMMAND_LINE =3D 'printk.time=3D0 ' @@ -174,9 +174,7 @@ def test_s390x_fedora(self): =20 kernel_path =3D self.ASSET_F31_KERNEL.fetch() =20 - initrd_path_xz =3D self.ASSET_F31_INITRD.fetch() - initrd_path =3D self.scratch_file('initrd-raw.img') - lzma_uncompress(initrd_path_xz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_F31_INITRD, format=3D"x= z") =20 self.vm.set_console() kernel_command_line =3D (self.KERNEL_COMMON_COMMAND_LINE + ' audit= =3D0 ' diff --git a/tests/functional/test_s390x_topology.py b/tests/functional/tes= t_s390x_topology.py index 82acff4e55..eefd9729cb 100755 --- a/tests/functional/test_s390x_topology.py +++ b/tests/functional/test_s390x_topology.py @@ -14,7 +14,6 @@ from qemu_test import exec_command from qemu_test import exec_command_and_wait_for_pattern from qemu_test import wait_for_console_pattern -from qemu_test.utils import lzma_uncompress =20 =20 class S390CPUTopology(QemuSystemTest): @@ -86,9 +85,7 @@ def kernel_init(self): """ self.require_accelerator("kvm") kernel_path =3D self.ASSET_F35_KERNEL.fetch() - initrd_path_xz =3D self.ASSET_F35_INITRD.fetch() - initrd_path =3D self.scratch_file('initrd-raw.img') - lzma_uncompress(initrd_path_xz, initrd_path) + initrd_path =3D self.uncompress(self.ASSET_F35_INITRD, format=3D"x= z") =20 self.vm.set_console() kernel_command_line =3D self.KERNEL_COMMON_COMMAND_LINE --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451334; cv=none; d=zohomail.com; s=zohoarc; b=GgiDxEOQdSZ3C+LIq+OC8XVQIZdehgI9R5dBXmGnGaZwea1eoZUjHRWmF+yvadhL7ieFvS/0t9owL2IJ9hvWS4eMsgLg1Ecir1HGa3WweNhowdOZGGTaD7hN7xMeiuQcSPTRj1jUdFjMGgeN9mLt9GhVwktnFhhqGb4c1CJM+q0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451334; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=AuZoHCRQ3ZhbwIAzUFXhFI0CRt5RmVkqYgSpFaOEhrY=; b=RimNyOQvehorOBsK9TerviZC5OBy1Q3EUu2omfXYjdWgegH8VV5/UHF66BzuP5xwEYIysdsMA5l7+IkhiNa42R3kGGpI0Sy3+xfoJ4geBXPqOYutUdB8YO1NAcjch6stLkgy/SpkcFz607J8FmMUK5ya40YOQVmFti+/RVcttMQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451333983246.70364364855436; Tue, 17 Dec 2024 08:02:13 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa0p-0005XF-L9; Tue, 17 Dec 2024 11:01:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0k-0005PA-K1 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0j-0002EZ-4F for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:26 -0500 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-237-_8fNEfsgPyi_qFU7GUiKgw-1; Tue, 17 Dec 2024 11:01:20 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5929519560A2; Tue, 17 Dec 2024 16:01:19 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 14FB630044C1; Tue, 17 Dec 2024 16:01:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AuZoHCRQ3ZhbwIAzUFXhFI0CRt5RmVkqYgSpFaOEhrY=; b=YhwRBerNRRMLAu4kZP8qNdg+92wivIw1inA410Atb92JBv8GyTPzUWqsF2eaJWAB2oJ+/e z2tA8cm1C+cUOQtoPipMsr9xyh9JNHim5nXoNK3xPsOUrSINVsxxQkYLO8uJkIUemdaoL+ 0aiJb1ULNXqg71J8Sj4BbEHb6BHBbSM= X-MC-Unique: _8fNEfsgPyi_qFU7GUiKgw-1 X-Mimecast-MFC-AGG-ID: _8fNEfsgPyi_qFU7GUiKgw From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 27/32] tests/functional: drop back compat imports from utils.py Date: Tue, 17 Dec 2024 15:59:48 +0000 Message-ID: <20241217155953.3950506-28-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451334626116600 Now that all tests are converted over to the higher level wrapper functions, the back compat imports from utils.py are redundant. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/utils.py | 5 ----- 1 file changed, 5 deletions(-) diff --git a/tests/functional/qemu_test/utils.py b/tests/functional/qemu_te= st/utils.py index 6b87af4414..43853b4366 100644 --- a/tests/functional/qemu_test/utils.py +++ b/tests/functional/qemu_test/utils.py @@ -10,11 +10,6 @@ =20 import os =20 -from .archive import tar_extract as archive_extract -from .archive import cpio_extract -from .uncompress import gzip_uncompress -from .uncompress import lzma_uncompress - """ Round up to next power of 2 """ --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451608; cv=none; d=zohomail.com; s=zohoarc; b=oBVEia+tabBfrcVKjuej3znE64QFzQSId2vHXKYP2Y6nCTyT+Y4SxA0fkLabTwY8nv+bFK5jdSK60jZvZPBWxTtwhvuoRu31SzCC47TA7A+3h05u9BIqAPrbrMsgpZn2S02EPfJlGS+b4JBzOLB/gP9kR4OWWuY/GHS4yeLdYTc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451608; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=sAmrbxoAoZ9PR/4gbrF0EEd1IpdvHSc5XD3afoe70/Q=; b=idUKiBFvVqHlI+iJARpZvuvS/W7GMwsT1U8nIgUswqynddQl/2u4K1w7j35vIViS8wKYb/BXB3oIcJ5W0Ck7w6AFVOYPLUWkvNoMn9NeFLatAnPXzn4ySctrQmFHYKh9paaT/TZh4pk6rBgLNzljM4fKK/IpWm1i841+Map6+4A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451608131115.3108526835972; Tue, 17 Dec 2024 08:06:48 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa12-00060e-Vy; Tue, 17 Dec 2024 11:01:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0r-0005cD-B9 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0m-0002F7-8t for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:31 -0500 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-543-ivtFr7s1MwyQ1lxfSn-rUQ-1; Tue, 17 Dec 2024 11:01:23 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 83C59195605E; Tue, 17 Dec 2024 16:01:22 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E950A30044C1; Tue, 17 Dec 2024 16:01:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451287; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sAmrbxoAoZ9PR/4gbrF0EEd1IpdvHSc5XD3afoe70/Q=; b=XaW1uTxEZSWISHbagSxsLwSVnHlOHElPOIM8T0v23cVu5wDh6OLRWy9snscCULkG+mzXvF QQ6lUM5Ny6IuNNtkJHKoW6+SwOcR1C6euEdVDOT7HQ2Rci5Dl2pduhkzvCudp9fZuBV2im SG43qAMs6XNOvSzpRQ49A842PN0mMHw= X-MC-Unique: ivtFr7s1MwyQ1lxfSn-rUQ-1 X-Mimecast-MFC-AGG-ID: ivtFr7s1MwyQ1lxfSn-rUQ From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 28/32] tests/functional: replace 'run_cmd' with subprocess helpers Date: Tue, 17 Dec 2024 15:59:49 +0000 Message-ID: <20241217155953.3950506-29-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451608445116600 The 'run_cmd' helper is re-implementing a convenient helper that already exists in the form of the 'run' and 'check_call' methods provided by 'subprocess'. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/archive.py | 9 ++++--- tests/functional/qemu_test/tesseract.py | 10 ++++---- tests/functional/qemu_test/testcase.py | 31 +++++++++++++----------- tests/functional/qemu_test/tuxruntest.py | 8 +++--- tests/functional/test_aarch64_virt.py | 6 +++-- tests/functional/test_ppc64_tuxrun.py | 7 ++++-- 6 files changed, 41 insertions(+), 30 deletions(-) diff --git a/tests/functional/qemu_test/archive.py b/tests/functional/qemu_= test/archive.py index c439d9413a..c803fdaf6d 100644 --- a/tests/functional/qemu_test/archive.py +++ b/tests/functional/qemu_test/archive.py @@ -14,7 +14,6 @@ import zipfile =20 from .asset import Asset -from .cmd import run_cmd =20 =20 def tar_extract(archive, dest_dir, member=3DNone): @@ -52,9 +51,11 @@ def deb_extract(archive, dest_dir, member=3DNone): cwd =3D os.getcwd() os.chdir(dest_dir) try: - (stdout, stderr, ret) =3D run_cmd(['ar', 't', archive]) - file_path =3D stdout.split()[2] - run_cmd(['ar', 'x', archive, file_path]) + proc =3D run(['ar', 't', archive], + check=3DTrue, capture_output=3DTrue, encoding=3D'utf8') + file_path =3D proc.stdout.split()[2] + check_call(['ar', 'x', archive, file_path], + stdout=3DDEVNULL, stderr=3DDEVNULL) tar_extract(file_path, dest_dir, member) finally: os.chdir(cwd) diff --git a/tests/functional/qemu_test/tesseract.py b/tests/functional/qem= u_test/tesseract.py index 1b7818090a..ede6c6501e 100644 --- a/tests/functional/qemu_test/tesseract.py +++ b/tests/functional/qemu_test/tesseract.py @@ -6,18 +6,18 @@ # later. See the COPYING file in the top-level directory. =20 import logging +from subprocess import run =20 -from . import run_cmd =20 def tesseract_ocr(image_path, tesseract_args=3D''): console_logger =3D logging.getLogger('console') console_logger.debug(image_path) - (stdout, stderr, ret) =3D run_cmd(['tesseract', image_path, - 'stdout']) - if ret: + proc =3D run(['tesseract', image_path, 'stdout'], + capture_output=3DTrue, encoding=3D'utf8') + if proc.returncode: return None lines =3D [] - for line in stdout.split('\n'): + for line in proc.stdout.split('\n'): sline =3D line.strip() if len(sline): console_logger.debug(sline) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index d0bb3141d5..aa6c9c0d64 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -16,7 +16,7 @@ from pathlib import Path import pycotap import shutil -import subprocess +from subprocess import run import sys import tempfile import unittest @@ -27,7 +27,6 @@ =20 from .archive import archive_extract from .asset import Asset -from .cmd import run_cmd from .config import BUILD_DIR from .uncompress import uncompress =20 @@ -251,11 +250,11 @@ def add_ldpath(self, ldpath): self._ldpath.append(os.path.abspath(ldpath)) =20 def run_cmd(self, bin_path, args=3D[]): - return subprocess.run([self.qemu_bin] - + ["-L %s" % ldpath for ldpath in self._ldpa= th] - + [bin_path] - + args, - text=3DTrue, capture_output=3DTrue) + return run([self.qemu_bin] + + ["-L %s" % ldpath for ldpath in self._ldpath] + + [bin_path] + + args, + text=3DTrue, capture_output=3DTrue) =20 class QemuSystemTest(QemuBaseTest): """Facilitates system emulation tests.""" @@ -282,7 +281,9 @@ def setUp(self): def set_machine(self, machinename): # TODO: We should use QMP to get the list of available machines if not self._machinehelp: - self._machinehelp =3D run_cmd([self.qemu_bin, '-M', 'help'])[0= ]; + self._machinehelp =3D run( + [self.qemu_bin, '-M', 'help'], + capture_output=3DTrue, check=3DTrue, encoding=3D'utf8').st= dout if self._machinehelp.find(machinename) < 0: self.skipTest('no support for machine ' + machinename) self.machine =3D machinename @@ -310,15 +311,17 @@ def require_accelerator(self, accelerator): "available" % accelerator) =20 def require_netdev(self, netdevname): - netdevhelp =3D run_cmd([self.qemu_bin, - '-M', 'none', '-netdev', 'help'])[0]; - if netdevhelp.find('\n' + netdevname + '\n') < 0: + help =3D run([self.qemu_bin, + '-M', 'none', '-netdev', 'help'], + capture_output=3DTrue, check=3DTrue, encoding=3D'utf8')= .stdout; + if help.find('\n' + netdevname + '\n') < 0: self.skipTest('no support for " + netdevname + " networking') =20 def require_device(self, devicename): - devhelp =3D run_cmd([self.qemu_bin, - '-M', 'none', '-device', 'help'])[0]; - if devhelp.find(devicename) < 0: + help =3D run([self.qemu_bin, + '-M', 'none', '-device', 'help'], + capture_output=3DTrue, check=3DTrue, encoding=3D'utf8')= .stdout; + if help.find(devicename) < 0: self.skipTest('no support for device ' + devicename) =20 def _new_vm(self, name, *args): diff --git a/tests/functional/qemu_test/tuxruntest.py b/tests/functional/qe= mu_test/tuxruntest.py index 0b1bb8f0ed..7227a83757 100644 --- a/tests/functional/qemu_test/tuxruntest.py +++ b/tests/functional/qemu_test/tuxruntest.py @@ -11,11 +11,12 @@ =20 import os import stat +from subprocess import check_call, DEVNULL =20 from qemu_test import QemuSystemTest from qemu_test import exec_command_and_wait_for_pattern from qemu_test import wait_for_console_pattern -from qemu_test import which, run_cmd, get_qemu_img +from qemu_test import which, get_qemu_img =20 class TuxRunBaselineTest(QemuSystemTest): =20 @@ -76,8 +77,9 @@ def fetch_tuxrun_assets(self, kernel_asset, rootfs_asset,= dtb_asset=3DNone): =20 disk_image =3D self.scratch_file("rootfs.ext4") =20 - run_cmd(['zstd', "-f", "-d", disk_image_zst, - "-o", disk_image]) + check_call(['zstd', "-f", "-d", disk_image_zst, + "-o", disk_image], + stdout=3DDEVNULL, stderr=3DDEVNULL) # zstd copies source archive permissions for the output # file, so must make this writable for QEMU os.chmod(disk_image, stat.S_IRUSR | stat.S_IWUSR) diff --git a/tests/functional/test_aarch64_virt.py b/tests/functional/test_= aarch64_virt.py index cc49f8963d..08576b0694 100755 --- a/tests/functional/test_aarch64_virt.py +++ b/tests/functional/test_aarch64_virt.py @@ -12,10 +12,11 @@ =20 import time import logging +from subprocess import check_call, DEVNULL =20 from qemu_test import QemuSystemTest, Asset from qemu_test import exec_command, wait_for_console_pattern -from qemu_test import get_qemu_img, run_cmd +from qemu_test import get_qemu_img =20 =20 class Aarch64VirtMachine(QemuSystemTest): @@ -96,7 +97,8 @@ def common_aarch64_virt(self, machine): logger.info('creating scratch qcow2 image') image_path =3D self.scratch_file('scratch.qcow2') qemu_img =3D get_qemu_img(self) - run_cmd([qemu_img, 'create', '-f', 'qcow2', image_path, '8M']) + check_call([qemu_img, 'create', '-f', 'qcow2', image_path, '8M'], + stdout=3DDEVNULL, stderr=3DDEVNULL) =20 # Add the device self.vm.add_args('-blockdev', diff --git a/tests/functional/test_ppc64_tuxrun.py b/tests/functional/test_= ppc64_tuxrun.py index 03b47e07f2..8a98d18ab3 100755 --- a/tests/functional/test_ppc64_tuxrun.py +++ b/tests/functional/test_ppc64_tuxrun.py @@ -11,9 +11,10 @@ # # SPDX-License-Identifier: GPL-2.0-or-later =20 +from subprocess import check_call, DEVNULL import tempfile =20 -from qemu_test import run_cmd, Asset +from qemu_test import Asset from qemu_test.tuxruntest import TuxRunBaselineTest =20 class TuxRunPPC64Test(TuxRunBaselineTest): @@ -70,7 +71,9 @@ def ppc64_common_tuxrun(self, kernel_asset, rootfs_asset,= prefix): # Create a temporary qcow2 and launch the test-case with tempfile.NamedTemporaryFile(prefix=3Dprefix, suffix=3D'.qcow2') as qcow2: - run_cmd([self.qemu_img, 'create', '-f', 'qcow2', qcow2.name, '= 1G']) + check_call([self.qemu_img, 'create', '-f', 'qcow2', + qcow2.name, ' 1G'], + stdout=3DDEVNULL, stderr=3DDEVNULL) =20 self.vm.add_args('-drive', 'file=3D' + qcow2.name + ',format=3Dqcow2,if=3Dnone,id=3D' --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451819; cv=none; d=zohomail.com; s=zohoarc; b=Nq9UekJcwXuAGIdrFfG7SUevvVRiEVmmY+50ABjO0ox7xAt+ItIUD4AXQ7WxSeHscMnfy/G4kc/qgbpzu0vblFWFhm0vMQTJfS/WjDn420ZdQjRpDMoUNLlNt+FS+EAJ5NGcrfqZuLCuMwwccxWVpUx/ZQPL+6+JYXSbGQ7sv7o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451819; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=9JgAzoDpeTQFBC9n+2CaNMSXOqGQ9KwDLWynNApg7Qs=; b=UuYvc9qM04KG4gHmPxXxdkY0h82ZyYA/5SV3YNZYQz9O9YJYNX1N1MBPWUmCq/DAUqx7wXawjSkOPqZGsN2rIeSBUr+onT5xFoFj/1ECYE07snAONlBQ39AlMS/1ZMVJyktlLpJQrb7li3+izhCCnNal+7y78mPZt/3bsCPaMW0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451819068415.0913976705946; Tue, 17 Dec 2024 08:10:19 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa16-0006E1-Cm; Tue, 17 Dec 2024 11:01:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0t-0005q6-FF for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0r-0002FY-2h for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:34 -0500 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-642-TEDREFR1MKytSIQ8twz82Q-1; Tue, 17 Dec 2024 11:01:27 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7D9961956046; Tue, 17 Dec 2024 16:01:25 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0772430044C1; Tue, 17 Dec 2024 16:01:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451290; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9JgAzoDpeTQFBC9n+2CaNMSXOqGQ9KwDLWynNApg7Qs=; b=EJTycXAP13a1mc8xtG1w79krOqXT02OzB3VcCSQWOx46KlxGe4ZRPd7T3iKLG4+4XFeGni ILwBii/axv/zWNzPIKMx0Z4upTJEGtGShL+3z3po79tRcW7GsTMSsrzitqkJ1V5XxmDsUa 64dA9FtY9Pj69VTHKPLl7hJN97iGsag= X-MC-Unique: TEDREFR1MKytSIQ8twz82Q-1 X-Mimecast-MFC-AGG-ID: TEDREFR1MKytSIQ8twz82Q From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 29/32] tests/functional: remove now unused 'run_cmd' helper Date: Tue, 17 Dec 2024 15:59:50 +0000 Message-ID: <20241217155953.3950506-30-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451820328116600 All usage has been replaced by direct 'subprocess' helpers. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/cmd.py | 11 ----------- 2 files changed, 1 insertion(+), 12 deletions(-) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu= _test/__init__.py index 3bd043e608..da1830286d 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -8,7 +8,7 @@ =20 from .asset import Asset from .config import BUILD_DIR -from .cmd import run_cmd, is_readable_executable_file, \ +from .cmd import is_readable_executable_file, \ interrupt_interactive_console_until_pattern, wait_for_console_pattern,= \ exec_command, exec_command_and_wait_for_pattern, get_qemu_img, which from .testcase import QemuBaseTest, QemuUserTest, QemuSystemTest diff --git a/tests/functional/qemu_test/cmd.py b/tests/functional/qemu_test= /cmd.py index c8971de00a..dc5f422b77 100644 --- a/tests/functional/qemu_test/cmd.py +++ b/tests/functional/qemu_test/cmd.py @@ -14,7 +14,6 @@ import logging import os import os.path -import subprocess =20 =20 def which(tool): @@ -28,16 +27,6 @@ def which(tool): return p return None =20 -def run_cmd(args): - subp =3D subprocess.Popen(args, - stdout=3Dsubprocess.PIPE, - stderr=3Dsubprocess.PIPE, - universal_newlines=3DTrue) - stdout, stderr =3D subp.communicate() - ret =3D subp.returncode - - return (stdout, stderr, ret) - def is_readable_executable_file(path): return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK) =20 --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451959; cv=none; d=zohomail.com; s=zohoarc; b=mCYMr2rrE51t5byjMVIujYeTFHGtfwri8a/4PSR4UzCRKmVBC8XuO3IX0QYKe+zZwh0mT/TYQhY1LxNMKv2qnt6AF2E8iLjHWrwexoMoB65D+TpBrsBbXnCddeoDdJDcSHHHnpTwYtg1FsMnJvt7GGZswG5xT3E0noZ401IhBzU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451959; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=tlB1PN4HKsqWwJiz+Iw+8YRN7XwwJoyAT4q/5izh0W4=; b=EpCWukjgMoGyR8xkwq+V9s69toMYp5dyMvFsct7K3hr8RPF975jLriGYmI6jRCiRH9zku1gHOgRp0KSQoT9x9RE9ZgGQ3Hf92aMydoUH8yD2odhftlYIeKmMQFu25cd6cteRzrLXfYUSC/u9MDC7fbSmmcVr2fTP2d3XVI/ZhNg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451959488535.7747854017987; Tue, 17 Dec 2024 08:12:39 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa17-0006PU-Dv; Tue, 17 Dec 2024 11:01:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0v-0005tC-CP for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0r-0002Fr-Bh for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:36 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-534-8qf2bvrNNlCEsOpIIJRhjw-1; Tue, 17 Dec 2024 11:01:29 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5C2AD1955F41; Tue, 17 Dec 2024 16:01:28 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id F33EA300F9B5; Tue, 17 Dec 2024 16:01:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451292; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tlB1PN4HKsqWwJiz+Iw+8YRN7XwwJoyAT4q/5izh0W4=; b=CoiBHV6CpZC5KQdwcTyPapJllrbDy96p31syVfgfZCkso0pxaU6bkfT0Cj6Vdy3Zw0kxkK k+9bb033JSuC3IUMVQ58MoghVfVMJDNsuvg/tluQ5ppXN9HWgNKpDRVUL7S1adGacRK+/g YBrbgDM45/U5fbKhSQkzT5+2Zk6rR+U= X-MC-Unique: 8qf2bvrNNlCEsOpIIJRhjw-1 X-Mimecast-MFC-AGG-ID: 8qf2bvrNNlCEsOpIIJRhjw From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 30/32] tests/functional: skip tests if assets are not available Date: Tue, 17 Dec 2024 15:59:51 +0000 Message-ID: <20241217155953.3950506-31-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451961072116600 If downloading of assets has been disabled, then skip running a test if the assets it has registered are not already downloaded. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/asset.py | 8 +++++++- tests/functional/qemu_test/testcase.py | 11 +++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_te= st/asset.py index c5d3e73c4b..39832b2587 100644 --- a/tests/functional/qemu_test/asset.py +++ b/tests/functional/qemu_test/asset.py @@ -65,6 +65,12 @@ def _check(self, cache_file): def valid(self): return self.cache_file.exists() and self._check(self.cache_file) =20 + def fetchable(self): + return not os.environ.get("QEMU_TEST_NO_DOWNLOAD", False) + + def available(self): + return self.valid() or self.fetchable() + def _wait_for_other_download(self, tmp_cache_file): # Another thread already seems to download the asset, so wait until # it is done, while also checking the size to see whether it is st= uck @@ -103,7 +109,7 @@ def fetch(self): self.cache_file, self.url) return str(self.cache_file) =20 - if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False): + if not self.fetchable(): raise Exception("Asset cache is invalid and downloads disabled= ") =20 self.log.info("Downloading %s to %s...", self.url, self.cache_file) diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu= _test/testcase.py index aa6c9c0d64..869f3949fe 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -184,6 +184,14 @@ def scratch_file(self, *args): def log_file(self, *args): return str(Path(self.outputdir, *args)) =20 + def assets_available(self): + for name, asset in vars(self.__class__).items(): + if name.startswith("ASSET_") and type(asset) =3D=3D Asset: + if not asset.available(): + self.log.debug(f"Asset {asset.url} not available") + return False + return True + def setUp(self, bin_prefix): self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be= set') self.arch =3D self.qemu_bin.split('-')[-1] @@ -209,6 +217,9 @@ def setUp(self, bin_prefix): self.machinelog.setLevel(logging.DEBUG) self.machinelog.addHandler(self._log_fh) =20 + if not self.assets_available(): + self.skipTest('One or more assets is not available') + def tearDown(self): if "QEMU_TEST_KEEP_SCRATCH" not in os.environ: shutil.rmtree(self.workdir) --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451835; cv=none; d=zohomail.com; s=zohoarc; b=cD0OeKJfejtGyuYSCoX7roGCB8S5aJPf/rmeTc26DNZd8ptT/5jPcAdgCnTWsPiydtJ2zEk8CRIyhNBs9ZvOag64wjmTaIZSlyOXqldArwHSpdjLVYzKvcWCvAKfkObpyO2rVQecrIPv64I5mkh8CBTTK4QpTu45ckjOr4SWCAE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451835; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ujZ23YBoEtRTbeD5bsCjK9gju82WtT8tPgmcvcVFHr4=; b=EP4JpNskKKy+R5XHGHkgZ/U+f+HZIbgcgqzs7frTkLzCWGzRAtaQobQw/r7HRIeyPRN8p65MWTUX2Hzc0Mzj94ZnUrjBH3K6tjfnrictP1ESWYiJE1aoAw3M8n34qXNQXw83egLCykkVeVan6V3qqHvW0QDIlUaPL2brcKccf+0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451835317729.0313800617337; Tue, 17 Dec 2024 08:10:35 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa1A-0006jt-4n; Tue, 17 Dec 2024 11:01:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0x-0005up-8h for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0v-0002Gn-GC for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:38 -0500 Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-437-fRLLplPAPVWc0E1F0YbL2Q-1; Tue, 17 Dec 2024 11:01:32 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6CA3F19560B2; Tue, 17 Dec 2024 16:01:31 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EC50230044C1; Tue, 17 Dec 2024 16:01:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451296; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ujZ23YBoEtRTbeD5bsCjK9gju82WtT8tPgmcvcVFHr4=; b=d/eU3LyenXSF+SgzaRmg2lOXjUvFO9rOgW4razbcGPYDEx+4QIDSBeNSFWafmoOaMKoWDB DEW2ACpVXCBPBbLuNyD986jxX3pWPmfEf5gO8ZQtkL3YC40a2934s/rLGFbttdGQ7y1pM2 Jk5+DoDLkcOYSYh6vw0FHy+8Lrp6uH8= X-MC-Unique: fRLLplPAPVWc0E1F0YbL2Q-1 X-Mimecast-MFC-AGG-ID: fRLLplPAPVWc0E1F0YbL2Q From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Thomas Huth Subject: [PATCH v3 31/32] tests/functional: ignore errors when caching assets, except for 404 Date: Tue, 17 Dec 2024 15:59:52 +0000 Message-ID: <20241217155953.3950506-32-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451836450116600 We see periodic errors caching assets due to a combination of transient networking and server problems. With the previous patch to skip running a test when it has missing assets, we can now treat most cache download errors as non-fatal. Only HTTP 404 is retained as fatal, since it is a strong indicator of a fully broken test rather than a transient error. Reviewed-by: Thomas Huth Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/functional/qemu_test/asset.py | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_te= st/asset.py index 39832b2587..f0730695f0 100644 --- a/tests/functional/qemu_test/asset.py +++ b/tests/functional/qemu_test/asset.py @@ -15,6 +15,7 @@ from time import sleep from pathlib import Path from shutil import copyfileobj +from urllib.error import HTTPError =20 =20 # Instances of this class must be declared as class level variables @@ -170,7 +171,18 @@ def precache_test(test): for name, asset in vars(test.__class__).items(): if name.startswith("ASSET_") and type(asset) =3D=3D Asset: log.info("Attempting to cache '%s'" % asset) - asset.fetch() + try: + asset.fetch() + except HTTPError as e: + # Treat 404 as fatal, since it is highly likely to + # indicate a broken test rather than a transient + # server or networking problem + if e.code =3D=3D 404: + raise + + log.debug(f"HTTP error {e.code} from {asset.url} " + + "skipping asset precache") + log.removeHandler(handler) =20 def precache_suite(suite): --=20 2.46.0 From nobody Fri Dec 27 18:20:19 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1734451428; cv=none; d=zohomail.com; s=zohoarc; b=Z48cqVQRoUC28ogCd2jQKyfUy8OuwzOQ/pidcIdf7QDwUPZxzRnSlR5UX2d9apU955nIkQt4txHLVXXzOlhPcTxrlnkzD+EvpWwg+TuHzsY1LDkYL7J6E/2dh7VmE+lhTvyhreLyHPKSn8HLmu++fIulX88/YzNQkMhKoAGP8E4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1734451428; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=uBq4gWrXk9b7m3TiygA89cqA2sa3jmGreZJ1+JibDZA=; b=jtQivfLySbWyBruHeglVrz9cExlnBd7KFFwmZ9mrmmHBCgiLkoS2B4p6BUX0HZERHbKFMx4n9bntROl6d7QK27ETSCNuggB/VCEGLj+uyrmJDBrqbLfDT+cM3WBDq+HLStOKxo7I9HzRcpwb+LBp5azD//AxG3i94JEsXCl8ORI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1734451428557641.7998132104784; Tue, 17 Dec 2024 08:03:48 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNa19-0006f2-JN; Tue, 17 Dec 2024 11:01:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa12-00066s-04 for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:45 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNa0x-0002H4-5N for qemu-devel@nongnu.org; Tue, 17 Dec 2024 11:01:43 -0500 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-332-ekt2H3Y5OGW82uE70t8k_Q-1; Tue, 17 Dec 2024 11:01:35 -0500 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 157BD19560A2; Tue, 17 Dec 2024 16:01:34 +0000 (UTC) Received: from toolbox.redhat.com (unknown [10.42.28.136]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0907D30044C1; Tue, 17 Dec 2024 16:01:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734451298; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uBq4gWrXk9b7m3TiygA89cqA2sa3jmGreZJ1+JibDZA=; b=QE9o0vrpQCi7wkod6hMmJFW7uoNDnDUKvlTZ6U8c/FJSjerfYaSZyx7IwyPCt4eQJHkq10 qn2OrMtidnk9RlknKbXHJd6BsWqWJbv5NWBMMSMEkXAAWlsVp5mY80CVk4VguV/KTshk2F fJhvsmwIDiUhQEizVht6NzM6b915l/c= X-MC-Unique: ekt2H3Y5OGW82uE70t8k_Q-1 X-Mimecast-MFC-AGG-ID: ekt2H3Y5OGW82uE70t8k_Q From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v3 32/32] MAINTAINERS: add myself as reviewer for functional test suite Date: Tue, 17 Dec 2024 15:59:53 +0000 Message-ID: <20241217155953.3950506-33-berrange@redhat.com> In-Reply-To: <20241217155953.3950506-1-berrange@redhat.com> References: <20241217155953.3950506-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1.116, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1734451429347116600 Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Thomas Huth --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index 822f34344b..54201da578 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4157,6 +4157,7 @@ W: https://cirrus-ci.com/github/qemu/qemu Functional testing framework M: Thomas Huth R: Philippe Mathieu-Daud=C3=A9 +R: Daniel P. Berrange F: tests/functional/qemu_test/ =20 Windows Hosted Continuous Integration --=20 2.46.0