From nobody Sun Nov 24 11:46:37 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1724229091; cv=none; d=zohomail.com; s=zohoarc; b=IiySeRs838qwbycmZ0NHpglL7sZ5aP38Kt9vM+VnQqdEAXWt5XHfBTD86iW5+mZiEH/iaqK4R07S8CIH5YJpd0RTh2ryeBWr9W6W/U3BBjyUd1d5FyUuQJJVRrOb7+fRlgB0SD+4VKuqZppVvWvrM/pM0MYdoNgUdV1BXnu6hqQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1724229091; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=aGYO6RHNseD9Ebbu8ITyVRjMOKl64xcvvlkbGijEVaE=; b=I5DuAaNt2wRtKu+naNVrH+/aYoCrWPQhnkBt43EIr0eYT6XsHoOJ/OJV0odqS2FaLK/9iosq6DK0YuKtERiqQwpVLwAygTqb+Ks8UiceZ3Gr6srNVDjQRBHL+iIaYS4RsrASfWrb4LD13yTCaFQrBVRonUiJcwZiwJSfAXqwawM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1724229091251554.7668343335247; Wed, 21 Aug 2024 01:31:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sggje-0008Bo-I7; Wed, 21 Aug 2024 04:30:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sggjB-0005lB-8m for qemu-devel@nongnu.org; Wed, 21 Aug 2024 04:30:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sggj0-0004Y2-Ru for qemu-devel@nongnu.org; Wed, 21 Aug 2024 04:30:00 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-261-5mPEfN3-M026zd1lQDEmPw-1; Wed, 21 Aug 2024 04:29:45 -0400 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 178B21955BFC; Wed, 21 Aug 2024 08:29:44 +0000 (UTC) Received: from thuth-p1g4.str.redhat.com (dhcp-192-191.str.redhat.com [10.33.192.191]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 124EE1955F1B; Wed, 21 Aug 2024 08:29:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1724228989; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aGYO6RHNseD9Ebbu8ITyVRjMOKl64xcvvlkbGijEVaE=; b=b9AIg4HOhZsBpsQLSILxVbhVesLtAMUbIGMr8kLFt0tVqfQZY4qI+Rrz8YhdsN8eOchg+k EXpRt3FqYOYcnnrD1N7PLDY5p/UAFYsGUhVzOvyun5LbRnmZXMthlWKwrqzqqsju5SGMuL eMU8h7vDUzOW0o2FliNHZKpYTYwRtFI= X-MC-Unique: 5mPEfN3-M026zd1lQDEmPw-1 From: Thomas Huth To: =?UTF-8?q?Alex=20Benn=C3=A9e?= , qemu-devel@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , "Daniel P . Berrange" Cc: Ani Sinha , Richard Henderson , John Snow , qemu-ppc@nongnu.org, Fabiano Rosas Subject: [PATCH v4 28/35] tests/functional: Convert the acpi-bits test into a standalone test Date: Wed, 21 Aug 2024 10:27:29 +0200 Message-ID: <20240821082748.65853-29-thuth@redhat.com> In-Reply-To: <20240821082748.65853-1-thuth@redhat.com> References: <20240821082748.65853-1-thuth@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 11 X-Spam_score: 1.1 X-Spam_bar: + X-Spam_report: (1.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.144, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1724229092817116600 Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore. Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Thomas Huth --- MAINTAINERS | 4 +- docs/devel/acpi-bits.rst | 62 ++++++-------- .../acpi-bits/bits-config/bits-cfg.txt | 0 .../acpi-bits/bits-tests/smbios.py2 | 0 .../acpi-bits/bits-tests/smilatency.py2 | 0 .../acpi-bits/bits-tests/testacpi.py2 | 0 .../acpi-bits/bits-tests/testcpuid.py2 | 0 tests/functional/meson.build | 2 + .../test_acpi_bits.py} | 83 ++++++++++--------- 9 files changed, 71 insertions(+), 80 deletions(-) rename tests/{avocado =3D> functional}/acpi-bits/bits-config/bits-cfg.txt = (100%) rename tests/{avocado =3D> functional}/acpi-bits/bits-tests/smbios.py2 (10= 0%) rename tests/{avocado =3D> functional}/acpi-bits/bits-tests/smilatency.py2= (100%) rename tests/{avocado =3D> functional}/acpi-bits/bits-tests/testacpi.py2 (= 100%) rename tests/{avocado =3D> functional}/acpi-bits/bits-tests/testcpuid.py2 = (100%) rename tests/{avocado/acpi-bits.py =3D> functional/test_acpi_bits.py} (86%) mode change 100644 =3D> 100755 diff --git a/MAINTAINERS b/MAINTAINERS index 394d704515..c2dfb3dad3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2075,8 +2075,8 @@ ACPI/AVOCADO/BIOSBITS M: Ani Sinha M: Michael S. Tsirkin S: Supported -F: tests/avocado/acpi-bits/* -F: tests/avocado/acpi-bits.py +F: tests/functional/acpi-bits/* +F: tests/functional/test_acpi_bits.py F: docs/devel/acpi-bits.rst =20 ACPI/HEST/GHES diff --git a/docs/devel/acpi-bits.rst b/docs/devel/acpi-bits.rst index 1ec394f5fb..78aeb6aa3c 100644 --- a/docs/devel/acpi-bits.rst +++ b/docs/devel/acpi-bits.rst @@ -1,6 +1,6 @@ -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D -ACPI/SMBIOS avocado tests using biosbits -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +ACPI/SMBIOS testing using biosbits +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ************ Introduction ************ @@ -35,7 +35,7 @@ for developing biosbits and its real life uses can be fou= nd in [#a]_ and [#b]_. For QEMU, we maintain a fork of bios bits in gitlab along with all the dependent submodules `here = `__. This fork contains numerous fixes, a newer acpica and changes specific to -running this avocado QEMU tests using bits. The author of this document +running these functional QEMU tests using bits. The author of this document is the sole maintainer of the QEMU fork of bios bits repository. For more information, please see author's `FOSDEM talk on this bios-bits based test framework `__. @@ -44,12 +44,12 @@ framework diff --git a/tests/avocado/acpi-bits/bits-config/bits-cfg.txt b/tests/funct= ional/acpi-bits/bits-config/bits-cfg.txt similarity index 100% rename from tests/avocado/acpi-bits/bits-config/bits-cfg.txt rename to tests/functional/acpi-bits/bits-config/bits-cfg.txt diff --git a/tests/avocado/acpi-bits/bits-tests/smbios.py2 b/tests/function= al/acpi-bits/bits-tests/smbios.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/smbios.py2 rename to tests/functional/acpi-bits/bits-tests/smbios.py2 diff --git a/tests/avocado/acpi-bits/bits-tests/smilatency.py2 b/tests/func= tional/acpi-bits/bits-tests/smilatency.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/smilatency.py2 rename to tests/functional/acpi-bits/bits-tests/smilatency.py2 diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 b/tests/functi= onal/acpi-bits/bits-tests/testacpi.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/testacpi.py2 rename to tests/functional/acpi-bits/bits-tests/testacpi.py2 diff --git a/tests/avocado/acpi-bits/bits-tests/testcpuid.py2 b/tests/funct= ional/acpi-bits/bits-tests/testcpuid.py2 similarity index 100% rename from tests/avocado/acpi-bits/bits-tests/testcpuid.py2 rename to tests/functional/acpi-bits/bits-tests/testcpuid.py2 diff --git a/tests/functional/meson.build b/tests/functional/meson.build index e95b2c5592..aa999cb880 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -11,6 +11,7 @@ endif =20 # Timeouts for individual tests that can be slow e.g. with debugging enabl= ed test_timeouts =3D { + 'acpi_bits' : 240, 'netdev_ethtool' : 180, 'ppc_40p' : 240, 'ppc64_hv' : 1000, @@ -99,6 +100,7 @@ tests_x86_64_quick =3D [ ] =20 tests_x86_64_thorough =3D [ + 'acpi_bits', 'netdev_ethtool', 'virtio_gpu', ] diff --git a/tests/avocado/acpi-bits.py b/tests/functional/test_acpi_bits.py old mode 100644 new mode 100755 similarity index 86% rename from tests/avocado/acpi-bits.py rename to tests/functional/test_acpi_bits.py index efe4f52ee0..ee40647d5b --- a/tests/avocado/acpi-bits.py +++ b/tests/functional/test_acpi_bits.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -# group: rw quick +# # Exercise QEMU generated ACPI/SMBIOS tables using biosbits, # https://biosbits.org/ # @@ -24,7 +24,7 @@ # pylint: disable=3Dconsider-using-f-string =20 """ -This is QEMU ACPI/SMBIOS avocado tests using biosbits. +This is QEMU ACPI/SMBIOS functional tests using biosbits. Biosbits is available originally at https://biosbits.org/. This test uses a fork of the upstream bits and has numerous fixes including an upgraded acpica. The fork is located here: @@ -41,15 +41,16 @@ import tempfile import time import zipfile + +from pathlib import Path from typing import ( List, Optional, Sequence, ) from qemu.machine import QEMUMachine -from avocado import skipIf -from avocado.utils import datadrainer as drainer -from avocado_qemu import QemuBaseTest +from unittest import skipIf +from qemu_test import QemuBaseTest, Asset =20 deps =3D ["xorriso", "mformat"] # dependent tools needed in the test setup= /box. supported_platforms =3D ['x86_64'] # supported test platforms. @@ -129,34 +130,32 @@ def base_args(self): class AcpiBitsTest(QemuBaseTest): #pylint: disable=3Dtoo-many-instance-att= ributes """ ACPI and SMBIOS tests using biosbits. - - :avocado: tags=3Darch:x86_64 - :avocado: tags=3Dacpi - """ # in slower systems the test can take as long as 3 minutes to complete. timeout =3D BITS_TIMEOUT =20 + # following are some standard configuration constants + # gitlab CI does shallow clones of depth 20 + BITS_INTERNAL_VER =3D 2020 + # commit hash must match the artifact tag below + BITS_COMMIT_HASH =3D 'c7920d2b' + # this is the latest bits release as of today. + BITS_TAG =3D "qemu-bits-10262023" + + ASSET_BITS =3D Asset(("https://gitlab.com/qemu-project/" + "biosbits-bits/-/jobs/artifacts/%s/" + "download?job=3Dqemu-bits-build" % BITS_TAG), + '1b8dd612c6831a6b491716a77acc486666aaa867051cdc34f7= ce169c2e25f487') + def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self._vm =3D None self._workDir =3D None self._baseDir =3D None =20 - # following are some standard configuration constants - self._bitsInternalVer =3D 2020 # gitlab CI does shallow clones of = depth 20 - self._bitsCommitHash =3D 'c7920d2b' # commit hash must match - # the artifact tag below - self._bitsTag =3D "qemu-bits-10262023" # this is the latest bits - # release as of today. - self._bitsArtSHA1Hash =3D 'b22cdfcfc7453875297d06d626f5474ee36a343= f' - self._bitsArtURL =3D ("https://gitlab.com/qemu-project/" - "biosbits-bits/-/jobs/artifacts/%s/" - "download?job=3Dqemu-bits-build" %self._bitsTa= g) self._debugcon_addr =3D '0x403' self._debugcon_log =3D 'debugcon-log.txt' - logging.basicConfig(level=3Dlogging.INFO) - self.logger =3D logging.getLogger('acpi-bits') + self.logger =3D self.log =20 def _print_log(self, log): self.logger.info('\nlogs from biosbits follows:') @@ -171,7 +170,7 @@ def copy_bits_config(self): bits_config_dir =3D os.path.join(self._baseDir, 'acpi-bits', 'bits-config') target_config_dir =3D os.path.join(self._workDir, - 'bits-%d' %self._bitsInternalVer, + 'bits-%d' %self.BITS_INTERNAL_VER, 'boot') self.assertTrue(os.path.exists(bits_config_dir)) self.assertTrue(os.path.exists(target_config_dir)) @@ -188,7 +187,7 @@ def copy_test_scripts(self): bits_test_dir =3D os.path.join(self._baseDir, 'acpi-bits', 'bits-tests') target_test_dir =3D os.path.join(self._workDir, - 'bits-%d' %self._bitsInternalVer, + 'bits-%d' %self.BITS_INTERNAL_VER, 'boot', 'python') =20 self.assertTrue(os.path.exists(bits_test_dir)) @@ -248,9 +247,9 @@ def generate_bits_iso(self): test scripts """ bits_dir =3D os.path.join(self._workDir, - 'bits-%d' %self._bitsInternalVer) + 'bits-%d' %self.BITS_INTERNAL_VER) iso_file =3D os.path.join(self._workDir, - 'bits-%d.iso' %self._bitsInternalVer) + 'bits-%d.iso' %self.BITS_INTERNAL_VER) mkrescue_script =3D os.path.join(self._workDir, 'grub-inst-x86_64-efi', 'bin', 'grub-mkrescue') @@ -264,8 +263,12 @@ def generate_bits_iso(self): =20 try: if os.getenv('V') or os.getenv('BITS_DEBUG'): - subprocess.check_call([mkrescue_script, '-o', iso_file, - bits_dir], stderr=3Dsubprocess.STDO= UT) + proc =3D subprocess.run([mkrescue_script, '-o', iso_file, + bits_dir], + stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT, + check=3DTrue) + self.logger.info("grub-mkrescue output %s" % proc.stdout) else: subprocess.check_call([mkrescue_script, '-o', iso_file, bits_dir], @@ -282,8 +285,9 @@ def generate_bits_iso(self): =20 def setUp(self): # pylint: disable=3Darguments-differ super().setUp('qemu-system-') + self.logger =3D self.log =20 - self._baseDir =3D os.getenv('AVOCADO_TEST_BASEDIR') + self._baseDir =3D Path(__file__).parent =20 # workdir could also be avocado's own workdir in self.workdir. # At present, I prefer to maintain my own temporary working @@ -300,15 +304,14 @@ def setUp(self): # pylint: disable=3Darguments-differ os.mkdir(prebuiltDir, mode=3D0o775) =20 bits_zip_file =3D os.path.join(prebuiltDir, 'bits-%d-%s.zip' - %(self._bitsInternalVer, - self._bitsCommitHash)) + %(self.BITS_INTERNAL_VER, + self.BITS_COMMIT_HASH)) grub_tar_file =3D os.path.join(prebuiltDir, 'bits-%d-%s-grub.tar.gz' - %(self._bitsInternalVer, - self._bitsCommitHash)) + %(self.BITS_INTERNAL_VER, + self.BITS_COMMIT_HASH)) =20 - bitsLocalArtLoc =3D self.fetch_asset(self._bitsArtURL, - asset_hash=3Dself._bitsArtSHA1H= ash) + bitsLocalArtLoc =3D self.ASSET_BITS.fetch() self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLo= c) =20 # extract the bits artifact in the temp working directory @@ -369,7 +372,7 @@ def test_acpi_smbios_bits(self): """The main test case implementation.""" =20 iso_file =3D os.path.join(self._workDir, - 'bits-%d.iso' %self._bitsInternalVer) + 'bits-%d.iso' %self.BITS_INTERNAL_VER) =20 self.assertTrue(os.access(iso_file, os.R_OK)) =20 @@ -393,12 +396,6 @@ def test_acpi_smbios_bits(self): self._vm.set_console() self._vm.launch() =20 - self.logger.debug("Console output from bits VM follows ...") - c_drainer =3D drainer.LineLogger(self._vm.console_socket.fileno(), - logger=3Dself.logger.getChild("cons= ole"), - stop_check=3D(lambda : - not self._vm.is_running= ())) - c_drainer.start() =20 # biosbits has been configured to run all the specified test suites # in batch mode and then automatically initiate a vm shutdown. @@ -406,4 +403,8 @@ def test_acpi_smbios_bits(self): # with the avocado test timeout. self._vm.event_wait('SHUTDOWN', timeout=3DBITS_TIMEOUT) self._vm.wait(timeout=3DNone) + self.logger.debug("Checking console output ...") self.parse_log() + +if __name__ =3D=3D '__main__': + QemuBaseTest.main() --=20 2.46.0