From nobody Sat May 4 13:18:25 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1636127703; cv=none; d=zohomail.com; s=zohoarc; b=MFmH8gH0vUuKYHQ3/gVEBZrZayLXH+gvS4yzzLnaHcSt+JaWnPRSPXoR8O7Gf+SZdVNKeC14E+0OY+Ze2LrVmQNaROaw20UROsyKLGFjFxjP3ci6FI+u8Y/phZhcBzWuE7GXMXRgqA2Ue6OpYTFvN+78GLDQhVVgTbFXjLYKTHQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1636127703; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=YuRAummLDMr6+sUmTgsiz1zomCt7IF/ZJCgj5uSec9g=; b=ftxsxfKUrteoGYB8Sd7E8OC6WG7tO/8nbtqlyPox/h82mCyYuOccUvII//Ej21FHXkO6bTVuNDvzmKvpuQ+tngVWrLHiamalvm8GE1ploPXFHaqnZS0sDC32rMWMqsY3o5UMJhRtI29wOSXWsklegzTr3++zuTWSTFH4t2RJ3Ls= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 16361277033681003.4871296673845; Fri, 5 Nov 2021 08:55:03 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-14-s3QpahSYPkOzJ--baCD1Cw-1; Fri, 05 Nov 2021 11:54:58 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 347018144EE; Fri, 5 Nov 2021 15:54:54 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1548860C17; Fri, 5 Nov 2021 15:54:54 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id D25E64E9E2; Fri, 5 Nov 2021 15:54:53 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1A5FsAfb031469 for ; Fri, 5 Nov 2021 11:54:10 -0400 Received: by smtp.corp.redhat.com (Postfix) id 263A81252D4; Fri, 5 Nov 2021 15:54:08 +0000 (UTC) Received: from wrampazz.redhat.com (unknown [10.22.8.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC2D012546A; Fri, 5 Nov 2021 15:54:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1636127702; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=YuRAummLDMr6+sUmTgsiz1zomCt7IF/ZJCgj5uSec9g=; b=X0zNDH9D39FSSEuTwplUDz9hYVfJgeuEGGcUDUKVUZl1//69CTQxEToqlzV5OngwOoOSp7 UNPVF2r6eYOtFOGN8/sHS8sKCLQNyYzihQU4cFW5WVgdUq///wmAgMNSCUAuD35c8fEzvO EQejDSsbIRwd8PQuPGQUsKxxAcJ3RWw= X-MC-Unique: s3QpahSYPkOzJ--baCD1Cw-1 From: Willian Rampazzo To: qemu-devel@nongnu.org Subject: [PATCH v2 1/2] tests/acceptance: introduce new check-avocado tartget Date: Fri, 5 Nov 2021 12:53:53 -0300 Message-Id: <20211105155354.154864-2-willianr@redhat.com> In-Reply-To: <20211105155354.154864-1-willianr@redhat.com> References: <20211105155354.154864-1-willianr@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com X-Mailman-Approved-At: Fri, 05 Nov 2021 11:54:49 -0400 Cc: qemu-ppc@nongnu.org, Thomas Huth , Eduardo Habkost , libvir-list@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Niek Linnenbank , qemu-arm@nongnu.org, =?UTF-8?q?Herv=C3=A9=20Poussineau?= , Cleber Rosa , Michael Rolnik , John Snow X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1636127703755100001 This introduces a new `make` target, `check-avocado`, and adds a deprecation message about the `check-acceptance` target. This is a preparation for renaming the `tests/acceptance` folder to `tests/avocado`. The plan is to remove the call to the `check-avocado` target one or two months after the release and leave the warning to force people to move to the new `check-avocado` target. Later, the `check-acceptance` target can be removed. The intent is to avoid a direct impact during the current soft freeze. Suggested-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Willian Rampazzo Acked-by: Thomas Huth Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 --- docs/about/deprecated.rst | 13 +++++++++++++ tests/Makefile.include | 17 ++++++++++++----- 2 files changed, 25 insertions(+), 5 deletions(-) diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 56f9ad15ab..7bf8da8325 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -410,3 +410,16 @@ nanoMIPS ISA =20 The ``nanoMIPS`` ISA has never been upstreamed to any compiler toolchain. As it is hard to generate binaries for it, declare it deprecated. + +Testing +------- + +Renaming of the acceptance folder to avocado +'''''''''''''''''''''''''''''''''''''''''''' + +The ``tests/acceptance`` folder was never used to store acceptance tests +in terms of software engineering. This naming can confuse developers +adding tests using the Avocado Framework to this folder. The folder +name change to ``tests/avocado`` also changed the ``make`` target from +``check-acceptance`` to ``check-avocado``. In this case, the use of the +``check-acceptance`` target is deprecated. diff --git a/tests/Makefile.include b/tests/Makefile.include index 8434a33fe6..8e8ee58493 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -16,7 +16,7 @@ ifneq ($(filter $(all-check-targets), check-softfloat),) @echo " $(MAKE) check-tcg Run TCG tests" @echo " $(MAKE) check-softfloat Run FPU emulation tests" endif - @echo " $(MAKE) check-acceptance Run acceptance (functional) tests fo= r currently configured targets" + @echo " $(MAKE) check-avocado Run avocado (integration) tests for = currently configured targets" @echo @echo " $(MAKE) check-report.tap Generates an aggregated TAP test rep= ort" @echo " $(MAKE) check-venv Creates a Python venv for tests" @@ -24,7 +24,7 @@ endif @echo @echo "The following are useful for CI builds" @echo " $(MAKE) check-build Build most test binaris" - @echo " $(MAKE) get-vm-images Downloads all images used by accepta= nce tests, according to configured targets (~350 MB each, 1.5 GB max)" + @echo " $(MAKE) get-vm-images Downloads all images used by avocado= tests, according to configured targets (~350 MB each, 1.5 GB max)" @echo @echo @echo "The variable SPEED can be set to control the gtester speed setting= ." @@ -83,7 +83,7 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES) =20 # Python venv for running tests =20 -.PHONY: check-venv check-acceptance +.PHONY: check-venv check-avocado check-acceptance check-acceptance-depreca= ted-warning =20 TESTS_VENV_DIR=3D$(BUILD_DIR)/tests/venv TESTS_VENV_REQ=3D$(SRC_PATH)/tests/requirements.txt @@ -127,12 +127,12 @@ get-vm-image-fedora-31-%: check-venv $(call quiet-command, \ $(TESTS_VENV_DIR)/bin/python -m avocado vmimage get \ --distro=3Dfedora --distro-version=3D31 --arch=3D$*, \ - "AVOCADO", "Downloading acceptance tests VM image for $*") + "AVOCADO", "Downloading avocado tests VM image for $*") =20 # download all vm images, according to defined targets get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_= 31_DOWNLOAD)) =20 -check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images +check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images $(call quiet-command, \ $(TESTS_VENV_DIR)/bin/python -m avocado \ --show=3D$(AVOCADO_SHOW) run --job-results-dir=3D$(TESTS_RESUL= TS_DIR) \ @@ -142,6 +142,13 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-= vm-images $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \ "AVOCADO", "tests/acceptance") =20 +check-acceptance-deprecated-warning: + @echo + @echo "Note '$(MAKE) check-acceptance' is deprecated, use '$(MAKE) check-= avocado' instead." + @echo + +check-acceptance: check-acceptance-deprecated-warning | check-avocado + # Consolidated targets =20 .PHONY: check-block check check-clean get-vm-images --=20 2.33.1 From nobody Sat May 4 13:18:25 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1636127710; cv=none; d=zohomail.com; s=zohoarc; b=E5djIh+zDk6PY106MY0uBw48F+DMJ0PdwgmQw7jEih8ECwlz73rqqZqBAqxt38iOvNi3Gvw1oebKoz/oT6CUJEqv28v7ksDpCaTSKfrwWNAOS+2r6LNUrQN0p84v1iB2T3TpBrziqZ3a8W/iaehbyHXOBeLhGuOtmsp1esFOZUg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1636127710; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jVJALXED82dSEqPJSpMi03MrAvnqlFA9OAp6s6sRtiU=; b=aZo3lPvtH0jRgq8WHzek5uoA3rCjiEsPbu7eXuqrg5QeOvuRLiQHTIM+l0wfHW8ShrkhB0d9jPNpQzri7FsludGu+5pfJ60istoGFs7afA46XvEYxVxslAnZN/MNoHEyX3/WcV5XrvgdfgB5azXc6W8JpDkAz04TQYMM/AQFToY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1636127710178984.6283332342986; Fri, 5 Nov 2021 08:55:10 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-297-ZDxAZvhjNyG5nU6ZhQsw-w-1; Fri, 05 Nov 2021 11:55:05 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2B78A19253E6; Fri, 5 Nov 2021 15:54:57 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0397260FFD; Fri, 5 Nov 2021 15:54:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id BB55E1806D03; Fri, 5 Nov 2021 15:54:56 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 1A5FsEkn031501 for ; Fri, 5 Nov 2021 11:54:14 -0400 Received: by smtp.corp.redhat.com (Postfix) id A25ED67610; Fri, 5 Nov 2021 15:54:13 +0000 (UTC) Received: from wrampazz.redhat.com (unknown [10.22.8.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9C3BA99AE3; Fri, 5 Nov 2021 15:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1636127709; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=jVJALXED82dSEqPJSpMi03MrAvnqlFA9OAp6s6sRtiU=; b=iLiG6TWOcyYEEqJeO6zPDYoMhc3iBUkO5rAWPsDphxqYrAg3mBbkUadOQzLGi8HaYEfy+2 bSvX/BM3nO4GwoqTiFapCrgIbc5405dcRa2/Acz6I/QcFnk1fyEUSU+z3w0xEVjEe4Qoz1 dlFAK9PfFBZTLvZibSOwbZPXvSyd6GA= X-MC-Unique: ZDxAZvhjNyG5nU6ZhQsw-w-1 From: Willian Rampazzo To: qemu-devel@nongnu.org Subject: [PATCH v2 2/2] tests/acceptance: rename tests acceptance to tests avocado Date: Fri, 5 Nov 2021 12:53:54 -0300 Message-Id: <20211105155354.154864-3-willianr@redhat.com> In-Reply-To: <20211105155354.154864-1-willianr@redhat.com> References: <20211105155354.154864-1-willianr@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com X-Mailman-Approved-At: Fri, 05 Nov 2021 11:54:49 -0400 Cc: qemu-ppc@nongnu.org, Thomas Huth , Eduardo Habkost , libvir-list@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Willian Rampazzo , Niek Linnenbank , qemu-arm@nongnu.org, =?UTF-8?q?Herv=C3=A9=20Poussineau?= , Cleber Rosa , Michael Rolnik , John Snow X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1636127710709100001 In the discussion about renaming the `tests/acceptance` [1], the conclusion was that the folders inside `tests` are related to the framework running the tests and not directly related to the type of the tests. This changes the folder to `tests/avocado` and adjusts the MAKEFILE, the CI related files and the documentation. [1] https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg06553.html For the Orange Pi PC documentation: Reviewed-by: Niek Linnenbank Reviewed-by: Philippe Mathieu-Daud=C3=A9 Tested-by: Philippe Mathieu-Daud=C3=A9 Signed-off-by: Willian Rampazzo --- .gitlab-ci.d/buildtest-template.yml | 2 +- .gitlab-ci.d/buildtest.yml | 56 +++++++++---------- MAINTAINERS | 52 ++++++++--------- configure | 2 +- docs/devel/build-system.rst | 2 +- docs/devel/ci-definitions.rst.inc | 2 +- docs/devel/testing.rst | 49 ++++++++-------- docs/system/arm/orangepi.rst | 8 +-- python/qemu/machine/README.rst | 2 +- python/qemu/qmp/README.rst | 2 +- python/qemu/utils/README.rst | 2 +- tests/Makefile.include | 4 +- tests/acceptance/README.rst | 10 ---- tests/avocado/README.rst | 10 ++++ .../avocado_qemu/__init__.py | 2 +- tests/{acceptance =3D> avocado}/boot_linux.py | 0 .../boot_linux_console.py | 0 tests/{acceptance =3D> avocado}/boot_xen.py | 0 tests/{acceptance =3D> avocado}/cpu_queries.py | 0 .../empty_cpu_model.py | 0 tests/{acceptance =3D> avocado}/hotplug_cpu.py | 0 tests/{acceptance =3D> avocado}/info_usernet.py | 0 tests/{acceptance =3D> avocado}/intel_iommu.py | 0 tests/{acceptance =3D> avocado}/linux_initrd.py | 2 +- .../linux_ssh_mips_malta.py | 0 .../machine_arm_canona1100.py | 0 .../machine_arm_integratorcp.py | 0 .../machine_arm_n8x0.py | 0 tests/{acceptance =3D> avocado}/machine_avr6.py | 2 +- .../machine_m68k_nextcube.py | 0 .../machine_microblaze.py | 0 .../machine_mips_fuloong2e.py | 0 .../machine_mips_loongson3v.py | 0 .../machine_mips_malta.py | 0 .../machine_rx_gdbsim.py | 0 .../machine_s390_ccw_virtio.py | 0 .../machine_sparc64_sun4u.py | 0 .../machine_sparc_leon3.py | 0 tests/{acceptance =3D> avocado}/migration.py | 0 tests/{acceptance =3D> avocado}/multiprocess.py | 0 .../pc_cpu_hotplug_props.py | 0 tests/{acceptance =3D> avocado}/ppc_405.py | 0 tests/{acceptance =3D> avocado}/ppc_bamboo.py | 0 .../{acceptance =3D> avocado}/ppc_mpc8544ds.py | 0 tests/{acceptance =3D> avocado}/ppc_prep_40p.py | 0 tests/{acceptance =3D> avocado}/ppc_pseries.py | 0 .../ppc_virtex_ml507.py | 0 .../{acceptance =3D> avocado}/replay_kernel.py | 0 tests/{acceptance =3D> avocado}/replay_linux.py | 0 .../reverse_debugging.py | 0 tests/{acceptance =3D> avocado}/smmu.py | 0 tests/{acceptance =3D> avocado}/tcg_plugins.py | 0 .../tesseract_utils.py | 0 tests/{acceptance =3D> avocado}/version.py | 0 tests/{acceptance =3D> avocado}/virtio-gpu.py | 0 .../virtio_check_params.py | 0 .../{acceptance =3D> avocado}/virtio_version.py | 0 .../virtiofs_submounts.py | 0 .../virtiofs_submounts.py.data/cleanup.sh | 0 .../guest-cleanup.sh | 0 .../virtiofs_submounts.py.data/guest.sh | 0 .../virtiofs_submounts.py.data/host.sh | 0 tests/{acceptance =3D> avocado}/vnc.py | 0 .../x86_cpu_model_versions.py | 0 64 files changed, 104 insertions(+), 105 deletions(-) delete mode 100644 tests/acceptance/README.rst create mode 100644 tests/avocado/README.rst rename tests/{acceptance =3D> avocado}/avocado_qemu/__init__.py (99%) rename tests/{acceptance =3D> avocado}/boot_linux.py (100%) rename tests/{acceptance =3D> avocado}/boot_linux_console.py (100%) rename tests/{acceptance =3D> avocado}/boot_xen.py (100%) rename tests/{acceptance =3D> avocado}/cpu_queries.py (100%) rename tests/{acceptance =3D> avocado}/empty_cpu_model.py (100%) rename tests/{acceptance =3D> avocado}/hotplug_cpu.py (100%) rename tests/{acceptance =3D> avocado}/info_usernet.py (100%) rename tests/{acceptance =3D> avocado}/intel_iommu.py (100%) rename tests/{acceptance =3D> avocado}/linux_initrd.py (99%) rename tests/{acceptance =3D> avocado}/linux_ssh_mips_malta.py (100%) rename tests/{acceptance =3D> avocado}/machine_arm_canona1100.py (100%) rename tests/{acceptance =3D> avocado}/machine_arm_integratorcp.py (100%) rename tests/{acceptance =3D> avocado}/machine_arm_n8x0.py (100%) rename tests/{acceptance =3D> avocado}/machine_avr6.py (98%) rename tests/{acceptance =3D> avocado}/machine_m68k_nextcube.py (100%) rename tests/{acceptance =3D> avocado}/machine_microblaze.py (100%) rename tests/{acceptance =3D> avocado}/machine_mips_fuloong2e.py (100%) rename tests/{acceptance =3D> avocado}/machine_mips_loongson3v.py (100%) rename tests/{acceptance =3D> avocado}/machine_mips_malta.py (100%) rename tests/{acceptance =3D> avocado}/machine_rx_gdbsim.py (100%) rename tests/{acceptance =3D> avocado}/machine_s390_ccw_virtio.py (100%) rename tests/{acceptance =3D> avocado}/machine_sparc64_sun4u.py (100%) rename tests/{acceptance =3D> avocado}/machine_sparc_leon3.py (100%) rename tests/{acceptance =3D> avocado}/migration.py (100%) rename tests/{acceptance =3D> avocado}/multiprocess.py (100%) rename tests/{acceptance =3D> avocado}/pc_cpu_hotplug_props.py (100%) rename tests/{acceptance =3D> avocado}/ppc_405.py (100%) rename tests/{acceptance =3D> avocado}/ppc_bamboo.py (100%) rename tests/{acceptance =3D> avocado}/ppc_mpc8544ds.py (100%) rename tests/{acceptance =3D> avocado}/ppc_prep_40p.py (100%) rename tests/{acceptance =3D> avocado}/ppc_pseries.py (100%) rename tests/{acceptance =3D> avocado}/ppc_virtex_ml507.py (100%) rename tests/{acceptance =3D> avocado}/replay_kernel.py (100%) rename tests/{acceptance =3D> avocado}/replay_linux.py (100%) rename tests/{acceptance =3D> avocado}/reverse_debugging.py (100%) rename tests/{acceptance =3D> avocado}/smmu.py (100%) rename tests/{acceptance =3D> avocado}/tcg_plugins.py (100%) rename tests/{acceptance =3D> avocado}/tesseract_utils.py (100%) rename tests/{acceptance =3D> avocado}/version.py (100%) rename tests/{acceptance =3D> avocado}/virtio-gpu.py (100%) rename tests/{acceptance =3D> avocado}/virtio_check_params.py (100%) rename tests/{acceptance =3D> avocado}/virtio_version.py (100%) rename tests/{acceptance =3D> avocado}/virtiofs_submounts.py (100%) rename tests/{acceptance =3D> avocado}/virtiofs_submounts.py.data/cleanup.= sh (100%) rename tests/{acceptance =3D> avocado}/virtiofs_submounts.py.data/guest-cl= eanup.sh (100%) rename tests/{acceptance =3D> avocado}/virtiofs_submounts.py.data/guest.sh= (100%) rename tests/{acceptance =3D> avocado}/virtiofs_submounts.py.data/host.sh = (100%) rename tests/{acceptance =3D> avocado}/vnc.py (100%) rename tests/{acceptance =3D> avocado}/x86_cpu_model_versions.py (100%) diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-t= emplate.yml index fcbcc4e627..2c7980a4f6 100644 --- a/.gitlab-ci.d/buildtest-template.yml +++ b/.gitlab-ci.d/buildtest-template.yml @@ -37,7 +37,7 @@ # Avoid recompiling by hiding ninja with NINJA=3D":" - make NINJA=3D":" $MAKE_CHECK_ARGS =20 -.acceptance_test_job_template: +.avocado_test_job_template: extends: .native_test_job_template cache: key: "${CI_JOB_NAME}-cache" diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 6c1301e912..71d0f407ad 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -26,14 +26,14 @@ check-system-alpine: IMAGE: alpine MAKE_CHECK_ARGS: check =20 -acceptance-system-alpine: - extends: .acceptance_test_job_template +avocado-system-alpine: + extends: .avocado_test_job_template needs: - job: build-system-alpine artifacts: true variables: IMAGE: alpine - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 build-system-ubuntu: extends: .native_build_job_template @@ -59,14 +59,14 @@ check-system-ubuntu: IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check =20 -acceptance-system-ubuntu: - extends: .acceptance_test_job_template +avocado-system-ubuntu: + extends: .avocado_test_job_template needs: - job: build-system-ubuntu artifacts: true variables: IMAGE: ubuntu2004 - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 build-system-debian: extends: .native_build_job_template @@ -91,14 +91,14 @@ check-system-debian: IMAGE: debian-amd64 MAKE_CHECK_ARGS: check =20 -acceptance-system-debian: - extends: .acceptance_test_job_template +avocado-system-debian: + extends: .avocado_test_job_template needs: - job: build-system-debian artifacts: true variables: IMAGE: debian-amd64 - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 build-system-fedora: extends: .native_build_job_template @@ -125,14 +125,14 @@ check-system-fedora: IMAGE: fedora MAKE_CHECK_ARGS: check =20 -acceptance-system-fedora: - extends: .acceptance_test_job_template +avocado-system-fedora: + extends: .avocado_test_job_template needs: - job: build-system-fedora artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 build-system-centos: extends: .native_build_job_template @@ -159,14 +159,14 @@ check-system-centos: IMAGE: centos8 MAKE_CHECK_ARGS: check =20 -acceptance-system-centos: - extends: .acceptance_test_job_template +avocado-system-centos: + extends: .avocado_test_job_template needs: - job: build-system-centos artifacts: true variables: IMAGE: centos8 - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 build-system-opensuse: extends: .native_build_job_template @@ -191,14 +191,14 @@ check-system-opensuse: IMAGE: opensuse-leap MAKE_CHECK_ARGS: check =20 -acceptance-system-opensuse: - extends: .acceptance_test_job_template +avocado-system-opensuse: + extends: .avocado_test_job_template needs: - job: build-system-opensuse artifacts: true variables: IMAGE: opensuse-leap - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 =20 # This jobs explicitly disable TCG (--disable-tcg), KVM is detected by @@ -317,7 +317,7 @@ clang-user: # This can be accomplished by using -enable-slirp=3Dgit, which avoids the = use of # a system-wide version of the library # -# Split in three sets of build/check/acceptance to limit the execution tim= e of each +# Split in three sets of build/check/avocado to limit the execution time o= f each # job build-cfi-aarch64: extends: .native_build_job_template @@ -352,14 +352,14 @@ check-cfi-aarch64: IMAGE: fedora MAKE_CHECK_ARGS: check =20 -acceptance-cfi-aarch64: - extends: .acceptance_test_job_template +avocado-cfi-aarch64: + extends: .avocado_test_job_template needs: - job: build-cfi-aarch64 artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 build-cfi-ppc64-s390x: extends: .native_build_job_template @@ -394,14 +394,14 @@ check-cfi-ppc64-s390x: IMAGE: fedora MAKE_CHECK_ARGS: check =20 -acceptance-cfi-ppc64-s390x: - extends: .acceptance_test_job_template +avocado-cfi-ppc64-s390x: + extends: .avocado_test_job_template needs: - job: build-cfi-ppc64-s390x artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 build-cfi-x86_64: extends: .native_build_job_template @@ -430,14 +430,14 @@ check-cfi-x86_64: IMAGE: fedora MAKE_CHECK_ARGS: check =20 -acceptance-cfi-x86_64: - extends: .acceptance_test_job_template +avocado-cfi-x86_64: + extends: .avocado_test_job_template needs: - job: build-cfi-x86_64 artifacts: true variables: IMAGE: fedora - MAKE_CHECK_ARGS: check-acceptance + MAKE_CHECK_ARGS: check-avocado =20 tsan-build: extends: .native_build_job_template diff --git a/MAINTAINERS b/MAINTAINERS index 797be5b366..53b63df407 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -177,7 +177,7 @@ L: qemu-arm@nongnu.org S: Maintained F: hw/arm/smmu* F: include/hw/arm/smmu* -F: tests/acceptance/smmu.py +F: tests/avocado/smmu.py =20 AVR TCG CPUs M: Michael Rolnik @@ -185,7 +185,7 @@ S: Maintained F: docs/system/target-avr.rst F: gdb-xml/avr-cpu.xml F: target/avr/ -F: tests/acceptance/machine_avr6.py +F: tests/avocado/machine_avr6.py =20 CRIS TCG CPUs M: Edgar E. Iglesias @@ -657,7 +657,7 @@ S: Odd Fixes F: include/hw/arm/digic.h F: hw/*/digic* F: include/hw/*/digic* -F: tests/acceptance/machine_arm_canona1100.py +F: tests/avocado/machine_arm_canona1100.py F: docs/system/arm/digic.rst =20 Goldfish RTC @@ -708,7 +708,7 @@ S: Maintained F: hw/arm/integratorcp.c F: hw/misc/arm_integrator_debug.c F: include/hw/misc/arm_integrator_debug.h -F: tests/acceptance/machine_arm_integratorcp.py +F: tests/avocado/machine_arm_integratorcp.py F: docs/system/arm/integratorcp.rst =20 MCIMX6UL EVK / i.MX6ul @@ -805,7 +805,7 @@ F: include/hw/display/blizzard.h F: include/hw/input/lm832x.h F: include/hw/input/tsc2xxx.h F: include/hw/misc/cbus.h -F: tests/acceptance/machine_arm_n8x0.py +F: tests/avocado/machine_arm_n8x0.py F: docs/system/arm/nseries.rst =20 Palm @@ -1159,7 +1159,7 @@ M: Edgar E. Iglesias S: Maintained F: hw/microblaze/petalogix_s3adsp1800_mmu.c F: include/hw/char/xilinx_uartlite.h -F: tests/acceptance/machine_microblaze.py +F: tests/avocado/machine_microblaze.py =20 petalogix_ml605 M: Edgar E. Iglesias @@ -1192,8 +1192,8 @@ F: hw/acpi/piix4.c F: hw/mips/malta.c F: hw/mips/gt64xxx_pci.c F: include/hw/southbridge/piix.h -F: tests/acceptance/linux_ssh_mips_malta.py -F: tests/acceptance/machine_mips_malta.py +F: tests/avocado/linux_ssh_mips_malta.py +F: tests/avocado/machine_mips_malta.py =20 Mipssim R: Aleksandar Rikalo @@ -1211,7 +1211,7 @@ F: hw/isa/vt82c686.c F: hw/pci-host/bonito.c F: hw/usb/vt82c686-uhci-pci.c F: include/hw/isa/vt82c686.h -F: tests/acceptance/machine_mips_fuloong2e.py +F: tests/avocado/machine_mips_fuloong2e.py =20 Loongson-3 virtual platforms M: Huacai Chen @@ -1221,7 +1221,7 @@ F: hw/intc/loongson_liointc.c F: hw/mips/loongson3_bootp.c F: hw/mips/loongson3_bootp.h F: hw/mips/loongson3_virt.c -F: tests/acceptance/machine_mips_loongson3v.py +F: tests/avocado/machine_mips_loongson3v.py =20 Boston M: Paul Burton @@ -1250,7 +1250,7 @@ Bamboo L: qemu-ppc@nongnu.org S: Orphan F: hw/ppc/ppc440_bamboo.c -F: tests/acceptance/ppc_bamboo.py +F: tests/avocado/ppc_bamboo.py =20 e500 L: qemu-ppc@nongnu.org @@ -1271,7 +1271,7 @@ L: qemu-ppc@nongnu.org S: Orphan F: hw/ppc/mpc8544ds.c F: hw/ppc/mpc8544_guts.c -F: tests/acceptance/ppc_mpc8544ds.py +F: tests/avocado/ppc_mpc8544ds.py =20 New World (mac99) M: Mark Cave-Ayland @@ -1318,7 +1318,7 @@ F: hw/dma/i82374.c F: hw/rtc/m48t59-isa.c F: include/hw/isa/pc87312.h F: include/hw/rtc/m48t59.h -F: tests/acceptance/ppc_prep_40p.py +F: tests/avocado/ppc_prep_40p.py =20 sPAPR M: David Gibson @@ -1336,7 +1336,7 @@ F: tests/qtest/spapr* F: tests/qtest/libqos/*spapr* F: tests/qtest/rtas* F: tests/qtest/libqos/rtas* -F: tests/acceptance/ppc_pseries.py +F: tests/avocado/ppc_pseries.py =20 PowerNV (Non-Virtualized) M: C=C3=A9dric Le Goater @@ -1356,7 +1356,7 @@ M: Edgar E. Iglesias L: qemu-ppc@nongnu.org S: Odd Fixes F: hw/ppc/virtex_ml507.c -F: tests/acceptance/ppc_virtex_ml507.py +F: tests/avocado/ppc_virtex_ml507.py =20 sam460ex M: BALATON Zoltan @@ -1443,7 +1443,7 @@ R: Yoshinori Sato S: Orphan F: docs/system/target-rx.rst F: hw/rx/rx-gdbsim.c -F: tests/acceptance/machine_rx_gdbsim.py +F: tests/avocado/machine_rx_gdbsim.py =20 SH4 Machines ------------ @@ -1497,7 +1497,7 @@ F: include/hw/pci-host/sabre.h F: hw/pci-bridge/simba.c F: include/hw/pci-bridge/simba.h F: pc-bios/openbios-sparc64 -F: tests/acceptance/machine_sparc64_sun4u.py +F: tests/avocado/machine_sparc64_sun4u.py =20 Sun4v M: Artyom Tarasenko @@ -1513,7 +1513,7 @@ S: Maintained F: hw/sparc/leon3.c F: hw/*/grlib* F: include/hw/*/grlib* -F: tests/acceptance/machine_sparc_leon3.py +F: tests/avocado/machine_sparc_leon3.py =20 S390 Machines ------------- @@ -1528,7 +1528,7 @@ F: include/hw/s390x/ F: hw/watchdog/wdt_diag288.c F: include/hw/watchdog/wdt_diag288.h F: configs/devices/s390x-softmmu/default.mak -F: tests/acceptance/machine_s390_ccw_virtio.py +F: tests/avocado/machine_s390_ccw_virtio.py T: git https://github.com/borntraeger/qemu.git s390-next L: qemu-s390x@nongnu.org =20 @@ -2112,7 +2112,7 @@ M: Alex Benn=C3=A9e S: Maintained F: hw/core/guest-loader.c F: docs/system/guest-loader.rst -F: tests/acceptance/boot_xen.py +F: tests/avocado/boot_xen.py =20 Intel Hexadecimal Object File Loader M: Su Hang @@ -2986,9 +2986,9 @@ F: net/filter-replay.c F: include/sysemu/replay.h F: docs/replay.txt F: stubs/replay.c -F: tests/acceptance/replay_kernel.py -F: tests/acceptance/replay_linux.py -F: tests/acceptance/reverse_debugging.py +F: tests/avocado/replay_kernel.py +F: tests/avocado/replay_linux.py +F: tests/avocado/reverse_debugging.py F: qapi/replay.json =20 IOVA Tree @@ -3105,7 +3105,7 @@ S: Maintained F: docs/devel/tcg-plugins.rst F: plugins/ F: tests/plugin/ -F: tests/acceptance/tcg_plugins.py +F: tests/avocado/tcg_plugins.py F: contrib/plugins/ =20 AArch64 TCG target @@ -3494,14 +3494,14 @@ S: Maintained F: tests/tcg/Makefile F: tests/tcg/Makefile.include =20 -Acceptance (Integration) Testing with the Avocado framework +Integration Testing with the Avocado framework W: https://trello.com/b/6Qi1pxVn/avocado-qemu R: Cleber Rosa R: Philippe Mathieu-Daud=C3=A9 R: Wainer dos Santos Moschetta R: Willian Rampazzo S: Odd Fixes -F: tests/acceptance/ +F: tests/avocado/ =20 Documentation ------------- diff --git a/configure b/configure index 33682cb971..19d5e97b27 100755 --- a/configure +++ b/configure @@ -3832,7 +3832,7 @@ LINKS=3D"$LINKS pc-bios/s390-ccw/Makefile" LINKS=3D"$LINKS roms/seabios/Makefile" LINKS=3D"$LINKS pc-bios/qemu-icon.bmp" LINKS=3D"$LINKS .gdbinit scripts" # scripts needed by relative path in .gd= binit -LINKS=3D"$LINKS tests/acceptance tests/data" +LINKS=3D"$LINKS tests/avocado tests/data" LINKS=3D"$LINKS tests/qemu-iotests/check" LINKS=3D"$LINKS python" LINKS=3D"$LINKS contrib/plugins/Makefile " diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index 7f106d2f1c..911ee5a8df 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -416,7 +416,7 @@ number of dynamically created files listed later. =20 ``tests/Makefile.include`` Rules for external test harnesses. These include the TCG tests, - ``qemu-iotests`` and the Avocado-based acceptance tests. + ``qemu-iotests`` and the Avocado-based integration tests. =20 ``tests/docker/Makefile.include`` Rules for Docker tests. Like tests/Makefile, this file is included diff --git a/docs/devel/ci-definitions.rst.inc b/docs/devel/ci-definitions.= rst.inc index 32e22ff468..6d5c6fd9f2 100644 --- a/docs/devel/ci-definitions.rst.inc +++ b/docs/devel/ci-definitions.rst.inc @@ -59,7 +59,7 @@ to system testing [5]_. Note that, in some cases, system = testing may require interaction with third-party software, like operating system images, datab= ases, networks, and so on. =20 -On QEMU, system testing is represented by the 'check-acceptance' target fr= om +On QEMU, system testing is represented by the 'check-avocado' target from 'make'. =20 Flaky tests diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 7500f076c2..dc5dbd057d 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -653,17 +653,16 @@ supported. To start the fuzzer, run Alternatively, some command different from "qemu-img info" can be tested, = by changing the ``-c`` option. =20 -Acceptance tests using the Avocado Framework --------------------------------------------- +Integration tests using the Avocado Framework +--------------------------------------------- =20 -The ``tests/acceptance`` directory hosts functional tests, also known -as acceptance level tests. They're usually higher level tests, and -may interact with external resources and with various guest operating -systems. +The ``tests/avocado`` directory hosts integration tests. They're usually +higher level tests, and may interact with external resources and with +various guest operating systems. =20 These tests are written using the Avocado Testing Framework (which must be installed separately) in conjunction with a the ``avocado_qemu.Test`` -class, implemented at ``tests/acceptance/avocado_qemu``. +class, implemented at ``tests/avocado/avocado_qemu``. =20 Tests based on ``avocado_qemu.Test`` can easily: =20 @@ -695,11 +694,11 @@ Tests based on ``avocado_qemu.Test`` can easily: Running tests ~~~~~~~~~~~~~ =20 -You can run the acceptance tests simply by executing: +You can run the avocado tests simply by executing: =20 .. code:: =20 - make check-acceptance + make check-avocado =20 This involves the automatic creation of Python virtual environment within the build tree (at ``tests/venv``) which will have all the @@ -714,12 +713,12 @@ specific version, they may be on packages named ``pyt= hon3-venv`` and ``python3-pip``. =20 It is also possible to run tests based on tags using the -``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment +``make check-avocado`` command and the ``AVOCADO_TAGS`` environment variable: =20 .. code:: =20 - make check-acceptance AVOCADO_TAGS=3Dquick + make check-avocado AVOCADO_TAGS=3Dquick =20 Note that tags separated with commas have an AND behavior, while tags separated by spaces have an OR behavior. For more information on Avocado @@ -728,31 +727,31 @@ tags, see: https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/t= ags.html =20 To run a single test file, a couple of them, or a test within a file -using the ``make check-acceptance`` command, set the ``AVOCADO_TESTS`` +using the ``make check-avocado`` command, set the ``AVOCADO_TESTS`` environment variable with the test files or test names. To run all tests from a single file, use: =20 .. code:: =20 - make check-acceptance AVOCADO_TESTS=3D$FILEPATH + make check-avocado AVOCADO_TESTS=3D$FILEPATH =20 The same is valid to run tests from multiple test files: =20 .. code:: =20 - make check-acceptance AVOCADO_TESTS=3D'$FILEPATH1 $FILEPATH2' + make check-avocado AVOCADO_TESTS=3D'$FILEPATH1 $FILEPATH2' =20 To run a single test within a file, use: =20 .. code:: =20 - make check-acceptance AVOCADO_TESTS=3D$FILEPATH:$TESTCLASS.$TESTNAME + make check-avocado AVOCADO_TESTS=3D$FILEPATH:$TESTCLASS.$TESTNAME =20 The same is valid to run single tests from multiple test files: =20 .. code:: =20 - make check-acceptance AVOCADO_TESTS=3D'$FILEPATH1:$TESTCLASS1.$TESTNAME1= $FILEPATH2:$TESTCLASS2.$TESTNAME2' + make check-avocado AVOCADO_TESTS=3D'$FILEPATH1:$TESTCLASS1.$TESTNAME1 $F= ILEPATH2:$TESTCLASS2.$TESTNAME2' =20 The scripts installed inside the virtual environment may be used without an "activation". For instance, the Avocado test runner @@ -760,9 +759,9 @@ may be invoked by running: =20 .. code:: =20 - tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/ + tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/ =20 -Note that if ``make check-acceptance`` was not executed before, it is +Note that if ``make check-avocado`` was not executed before, it is possible to create the Python virtual environment with the dependencies needed running: =20 @@ -775,20 +774,20 @@ a test file. To run tests from a single file within t= he build tree, use: =20 .. code:: =20 - tests/venv/bin/avocado run tests/acceptance/$TESTFILE + tests/venv/bin/avocado run tests/avocado/$TESTFILE =20 To run a single test within a test file, use: =20 .. code:: =20 - tests/venv/bin/avocado run tests/acceptance/$TESTFILE:$TESTCLASS.$TESTNA= ME + tests/venv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME =20 Valid test names are visible in the output from any previous execution -of Avocado or ``make check-acceptance``, and can also be queried using: +of Avocado or ``make check-avocado``, and can also be queried using: =20 .. code:: =20 - tests/venv/bin/avocado list tests/acceptance + tests/venv/bin/avocado list tests/avocado =20 Manual Installation ~~~~~~~~~~~~~~~~~~~ @@ -806,7 +805,7 @@ Alternatively, follow the instructions on this link: Overview ~~~~~~~~ =20 -The ``tests/acceptance/avocado_qemu`` directory provides the +The ``tests/avocado/avocado_qemu`` directory provides the ``avocado_qemu`` Python module, containing the ``avocado_qemu.Test`` class. Here's a simple usage example: =20 @@ -913,7 +912,7 @@ like this: self.ssh_command('some_command_to_be_run_in_the_guest') =20 Please refer to tests that use ``avocado_qemu.LinuxTest`` under -``tests/acceptance`` for more examples. +``tests/avocado`` for more examples. =20 QEMUMachine ~~~~~~~~~~~ @@ -1204,7 +1203,7 @@ And remove any package you want with:: =20 pip uninstall =20 -If you've used ``make check-acceptance``, the Python virtual environment w= here +If you've used ``make check-avocado``, the Python virtual environment where Avocado is installed will be cleaned up as part of ``make check-clean``. =20 .. _checktcg-ref: diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst index 6f23907fb6..c55694dd91 100644 --- a/docs/system/arm/orangepi.rst +++ b/docs/system/arm/orangepi.rst @@ -250,14 +250,14 @@ and set the following environment variables before bo= oting: Optionally you may save the environment variables to SD card with 'saveenv= '. To continue booting simply give the 'boot' command and NetBSD boots. =20 -Orange Pi PC acceptance tests -""""""""""""""""""""""""""""" +Orange Pi PC integration tests +"""""""""""""""""""""""""""""" =20 -The Orange Pi PC machine has several acceptance tests included. +The Orange Pi PC machine has several integration tests included. To run the whole set of tests, build QEMU from source and simply provide the following command: =20 .. code-block:: bash =20 $ AVOCADO_ALLOW_LARGE_STORAGE=3Dyes avocado --show=3Dapp,console run \ - -t machine:orangepi-pc tests/acceptance/boot_linux_console.py + -t machine:orangepi-pc tests/avocado/boot_linux_console.py diff --git a/python/qemu/machine/README.rst b/python/qemu/machine/README.rst index ac2b4fffb4..8de2c3d772 100644 --- a/python/qemu/machine/README.rst +++ b/python/qemu/machine/README.rst @@ -2,7 +2,7 @@ qemu.machine package =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 This package provides core utilities used for testing and debugging -QEMU. It is used by the iotests, vm tests, acceptance tests, and several +QEMU. It is used by the iotests, vm tests, avocado tests, and several other utilities in the ./scripts directory. It is not a fully-fledged SDK and it is subject to change at any time. =20 diff --git a/python/qemu/qmp/README.rst b/python/qemu/qmp/README.rst index c21951491c..5bfb82535f 100644 --- a/python/qemu/qmp/README.rst +++ b/python/qemu/qmp/README.rst @@ -3,7 +3,7 @@ qemu.qmp package =20 This package provides a library used for connecting to and communicating with QMP servers. It is used extensively by iotests, vm tests, -acceptance tests, and other utilities in the ./scripts directory. It is +avocado tests, and other utilities in the ./scripts directory. It is not a fully-fledged SDK and is subject to change at any time. =20 See the documentation in ``__init__.py`` for more information. diff --git a/python/qemu/utils/README.rst b/python/qemu/utils/README.rst index 975fbf4d7d..d5f2da1454 100644 --- a/python/qemu/utils/README.rst +++ b/python/qemu/utils/README.rst @@ -2,6 +2,6 @@ qemu.utils package =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 This package provides miscellaneous utilities used for testing and -debugging QEMU. It is used primarily by the vm and acceptance tests. +debugging QEMU. It is used primarily by the vm and avocado tests. =20 See the documentation in ``__init__.py`` for more information. diff --git a/tests/Makefile.include b/tests/Makefile.include index 8e8ee58493..4c564cf789 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -89,7 +89,7 @@ TESTS_VENV_DIR=3D$(BUILD_DIR)/tests/venv TESTS_VENV_REQ=3D$(SRC_PATH)/tests/requirements.txt TESTS_RESULTS_DIR=3D$(BUILD_DIR)/tests/results ifndef AVOCADO_TESTS - AVOCADO_TESTS=3Dtests/acceptance + AVOCADO_TESTS=3Dtests/avocado endif # Controls the output generated by Avocado when running tests. # Any number of command separated loggers are accepted. For more @@ -140,7 +140,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-i= mages --filter-by-tags-include-empty-key) \ $(AVOCADO_CMDLINE_TAGS) \ $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \ - "AVOCADO", "tests/acceptance") + "AVOCADO", "tests/avocado") =20 check-acceptance-deprecated-warning: @echo diff --git a/tests/acceptance/README.rst b/tests/acceptance/README.rst deleted file mode 100644 index 89260faed6..0000000000 --- a/tests/acceptance/README.rst +++ /dev/null @@ -1,10 +0,0 @@ -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Acceptance tests using the Avocado Framework -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - -This directory contains functional tests, also known as acceptance -level tests. They're usually higher level, and may interact with -external resources and with various guest operating systems. - -For more information, please refer to ``docs/devel/testing.rst``, -section "Acceptance tests using the Avocado Framework". diff --git a/tests/avocado/README.rst b/tests/avocado/README.rst new file mode 100644 index 0000000000..94488371bb --- /dev/null +++ b/tests/avocado/README.rst @@ -0,0 +1,10 @@ +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Integration tests using the Avocado Framework +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This directory contains integration tests. They're usually higher +level, and may interact with external resources and with various +guest operating systems. + +For more information, please refer to ``docs/devel/testing.rst``, +section "Integration tests using the Avocado Framework". diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/avocado/avoc= ado_qemu/__init__.py similarity index 99% rename from tests/acceptance/avocado_qemu/__init__.py rename to tests/avocado/avocado_qemu/__init__.py index 1841053e2c..cd21b59e04 100644 --- a/tests/acceptance/avocado_qemu/__init__.py +++ b/tests/avocado/avocado_qemu/__init__.py @@ -27,7 +27,7 @@ BUILD_DIR =3D os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirn= ame(__file__)))) =20 if os.path.islink(os.path.dirname(os.path.dirname(__file__))): - # The link to the acceptance tests dir in the source code directory + # The link to the avocado tests dir in the source code directory lnk =3D os.path.dirname(os.path.dirname(__file__)) #: The QEMU root source directory SOURCE_DIR =3D os.path.dirname(os.path.dirname(os.readlink(lnk))) diff --git a/tests/acceptance/boot_linux.py b/tests/avocado/boot_linux.py similarity index 100% rename from tests/acceptance/boot_linux.py rename to tests/avocado/boot_linux.py diff --git a/tests/acceptance/boot_linux_console.py b/tests/avocado/boot_li= nux_console.py similarity index 100% rename from tests/acceptance/boot_linux_console.py rename to tests/avocado/boot_linux_console.py diff --git a/tests/acceptance/boot_xen.py b/tests/avocado/boot_xen.py similarity index 100% rename from tests/acceptance/boot_xen.py rename to tests/avocado/boot_xen.py diff --git a/tests/acceptance/cpu_queries.py b/tests/avocado/cpu_queries.py similarity index 100% rename from tests/acceptance/cpu_queries.py rename to tests/avocado/cpu_queries.py diff --git a/tests/acceptance/empty_cpu_model.py b/tests/avocado/empty_cpu_= model.py similarity index 100% rename from tests/acceptance/empty_cpu_model.py rename to tests/avocado/empty_cpu_model.py diff --git a/tests/acceptance/hotplug_cpu.py b/tests/avocado/hotplug_cpu.py similarity index 100% rename from tests/acceptance/hotplug_cpu.py rename to tests/avocado/hotplug_cpu.py diff --git a/tests/acceptance/info_usernet.py b/tests/avocado/info_usernet.= py similarity index 100% rename from tests/acceptance/info_usernet.py rename to tests/avocado/info_usernet.py diff --git a/tests/acceptance/intel_iommu.py b/tests/avocado/intel_iommu.py similarity index 100% rename from tests/acceptance/intel_iommu.py rename to tests/avocado/intel_iommu.py diff --git a/tests/acceptance/linux_initrd.py b/tests/avocado/linux_initrd.= py similarity index 99% rename from tests/acceptance/linux_initrd.py rename to tests/avocado/linux_initrd.py index a249e2f14a..9b4880cd8c 100644 --- a/tests/acceptance/linux_initrd.py +++ b/tests/avocado/linux_initrd.py @@ -1,4 +1,4 @@ -# Linux initrd acceptance test. +# Linux initrd integration test. # # Copyright (c) 2018 Red Hat, Inc. # diff --git a/tests/acceptance/linux_ssh_mips_malta.py b/tests/avocado/linux= _ssh_mips_malta.py similarity index 100% rename from tests/acceptance/linux_ssh_mips_malta.py rename to tests/avocado/linux_ssh_mips_malta.py diff --git a/tests/acceptance/machine_arm_canona1100.py b/tests/avocado/mac= hine_arm_canona1100.py similarity index 100% rename from tests/acceptance/machine_arm_canona1100.py rename to tests/avocado/machine_arm_canona1100.py diff --git a/tests/acceptance/machine_arm_integratorcp.py b/tests/avocado/m= achine_arm_integratorcp.py similarity index 100% rename from tests/acceptance/machine_arm_integratorcp.py rename to tests/avocado/machine_arm_integratorcp.py diff --git a/tests/acceptance/machine_arm_n8x0.py b/tests/avocado/machine_a= rm_n8x0.py similarity index 100% rename from tests/acceptance/machine_arm_n8x0.py rename to tests/avocado/machine_arm_n8x0.py diff --git a/tests/acceptance/machine_avr6.py b/tests/avocado/machine_avr6.= py similarity index 98% rename from tests/acceptance/machine_avr6.py rename to tests/avocado/machine_avr6.py index 6baf4e9c7f..6bab31342a 100644 --- a/tests/acceptance/machine_avr6.py +++ b/tests/avocado/machine_avr6.py @@ -1,5 +1,5 @@ # -# QEMU AVR acceptance tests +# QEMU AVR integration tests # # Copyright (c) 2019-2020 Michael Rolnik # diff --git a/tests/acceptance/machine_m68k_nextcube.py b/tests/avocado/mach= ine_m68k_nextcube.py similarity index 100% rename from tests/acceptance/machine_m68k_nextcube.py rename to tests/avocado/machine_m68k_nextcube.py diff --git a/tests/acceptance/machine_microblaze.py b/tests/avocado/machine= _microblaze.py similarity index 100% rename from tests/acceptance/machine_microblaze.py rename to tests/avocado/machine_microblaze.py diff --git a/tests/acceptance/machine_mips_fuloong2e.py b/tests/avocado/mac= hine_mips_fuloong2e.py similarity index 100% rename from tests/acceptance/machine_mips_fuloong2e.py rename to tests/avocado/machine_mips_fuloong2e.py diff --git a/tests/acceptance/machine_mips_loongson3v.py b/tests/avocado/ma= chine_mips_loongson3v.py similarity index 100% rename from tests/acceptance/machine_mips_loongson3v.py rename to tests/avocado/machine_mips_loongson3v.py diff --git a/tests/acceptance/machine_mips_malta.py b/tests/avocado/machine= _mips_malta.py similarity index 100% rename from tests/acceptance/machine_mips_malta.py rename to tests/avocado/machine_mips_malta.py diff --git a/tests/acceptance/machine_rx_gdbsim.py b/tests/avocado/machine_= rx_gdbsim.py similarity index 100% rename from tests/acceptance/machine_rx_gdbsim.py rename to tests/avocado/machine_rx_gdbsim.py diff --git a/tests/acceptance/machine_s390_ccw_virtio.py b/tests/avocado/ma= chine_s390_ccw_virtio.py similarity index 100% rename from tests/acceptance/machine_s390_ccw_virtio.py rename to tests/avocado/machine_s390_ccw_virtio.py diff --git a/tests/acceptance/machine_sparc64_sun4u.py b/tests/avocado/mach= ine_sparc64_sun4u.py similarity index 100% rename from tests/acceptance/machine_sparc64_sun4u.py rename to tests/avocado/machine_sparc64_sun4u.py diff --git a/tests/acceptance/machine_sparc_leon3.py b/tests/avocado/machin= e_sparc_leon3.py similarity index 100% rename from tests/acceptance/machine_sparc_leon3.py rename to tests/avocado/machine_sparc_leon3.py diff --git a/tests/acceptance/migration.py b/tests/avocado/migration.py similarity index 100% rename from tests/acceptance/migration.py rename to tests/avocado/migration.py diff --git a/tests/acceptance/multiprocess.py b/tests/avocado/multiprocess.= py similarity index 100% rename from tests/acceptance/multiprocess.py rename to tests/avocado/multiprocess.py diff --git a/tests/acceptance/pc_cpu_hotplug_props.py b/tests/avocado/pc_cp= u_hotplug_props.py similarity index 100% rename from tests/acceptance/pc_cpu_hotplug_props.py rename to tests/avocado/pc_cpu_hotplug_props.py diff --git a/tests/acceptance/ppc_405.py b/tests/avocado/ppc_405.py similarity index 100% rename from tests/acceptance/ppc_405.py rename to tests/avocado/ppc_405.py diff --git a/tests/acceptance/ppc_bamboo.py b/tests/avocado/ppc_bamboo.py similarity index 100% rename from tests/acceptance/ppc_bamboo.py rename to tests/avocado/ppc_bamboo.py diff --git a/tests/acceptance/ppc_mpc8544ds.py b/tests/avocado/ppc_mpc8544d= s.py similarity index 100% rename from tests/acceptance/ppc_mpc8544ds.py rename to tests/avocado/ppc_mpc8544ds.py diff --git a/tests/acceptance/ppc_prep_40p.py b/tests/avocado/ppc_prep_40p.= py similarity index 100% rename from tests/acceptance/ppc_prep_40p.py rename to tests/avocado/ppc_prep_40p.py diff --git a/tests/acceptance/ppc_pseries.py b/tests/avocado/ppc_pseries.py similarity index 100% rename from tests/acceptance/ppc_pseries.py rename to tests/avocado/ppc_pseries.py diff --git a/tests/acceptance/ppc_virtex_ml507.py b/tests/avocado/ppc_virte= x_ml507.py similarity index 100% rename from tests/acceptance/ppc_virtex_ml507.py rename to tests/avocado/ppc_virtex_ml507.py diff --git a/tests/acceptance/replay_kernel.py b/tests/avocado/replay_kerne= l.py similarity index 100% rename from tests/acceptance/replay_kernel.py rename to tests/avocado/replay_kernel.py diff --git a/tests/acceptance/replay_linux.py b/tests/avocado/replay_linux.= py similarity index 100% rename from tests/acceptance/replay_linux.py rename to tests/avocado/replay_linux.py diff --git a/tests/acceptance/reverse_debugging.py b/tests/avocado/reverse_= debugging.py similarity index 100% rename from tests/acceptance/reverse_debugging.py rename to tests/avocado/reverse_debugging.py diff --git a/tests/acceptance/smmu.py b/tests/avocado/smmu.py similarity index 100% rename from tests/acceptance/smmu.py rename to tests/avocado/smmu.py diff --git a/tests/acceptance/tcg_plugins.py b/tests/avocado/tcg_plugins.py similarity index 100% rename from tests/acceptance/tcg_plugins.py rename to tests/avocado/tcg_plugins.py diff --git a/tests/acceptance/tesseract_utils.py b/tests/avocado/tesseract_= utils.py similarity index 100% rename from tests/acceptance/tesseract_utils.py rename to tests/avocado/tesseract_utils.py diff --git a/tests/acceptance/version.py b/tests/avocado/version.py similarity index 100% rename from tests/acceptance/version.py rename to tests/avocado/version.py diff --git a/tests/acceptance/virtio-gpu.py b/tests/avocado/virtio-gpu.py similarity index 100% rename from tests/acceptance/virtio-gpu.py rename to tests/avocado/virtio-gpu.py diff --git a/tests/acceptance/virtio_check_params.py b/tests/avocado/virtio= _check_params.py similarity index 100% rename from tests/acceptance/virtio_check_params.py rename to tests/avocado/virtio_check_params.py diff --git a/tests/acceptance/virtio_version.py b/tests/avocado/virtio_vers= ion.py similarity index 100% rename from tests/acceptance/virtio_version.py rename to tests/avocado/virtio_version.py diff --git a/tests/acceptance/virtiofs_submounts.py b/tests/avocado/virtiof= s_submounts.py similarity index 100% rename from tests/acceptance/virtiofs_submounts.py rename to tests/avocado/virtiofs_submounts.py diff --git a/tests/acceptance/virtiofs_submounts.py.data/cleanup.sh b/tests= /avocado/virtiofs_submounts.py.data/cleanup.sh similarity index 100% rename from tests/acceptance/virtiofs_submounts.py.data/cleanup.sh rename to tests/avocado/virtiofs_submounts.py.data/cleanup.sh diff --git a/tests/acceptance/virtiofs_submounts.py.data/guest-cleanup.sh b= /tests/avocado/virtiofs_submounts.py.data/guest-cleanup.sh similarity index 100% rename from tests/acceptance/virtiofs_submounts.py.data/guest-cleanup.sh rename to tests/avocado/virtiofs_submounts.py.data/guest-cleanup.sh diff --git a/tests/acceptance/virtiofs_submounts.py.data/guest.sh b/tests/a= vocado/virtiofs_submounts.py.data/guest.sh similarity index 100% rename from tests/acceptance/virtiofs_submounts.py.data/guest.sh rename to tests/avocado/virtiofs_submounts.py.data/guest.sh diff --git a/tests/acceptance/virtiofs_submounts.py.data/host.sh b/tests/av= ocado/virtiofs_submounts.py.data/host.sh similarity index 100% rename from tests/acceptance/virtiofs_submounts.py.data/host.sh rename to tests/avocado/virtiofs_submounts.py.data/host.sh diff --git a/tests/acceptance/vnc.py b/tests/avocado/vnc.py similarity index 100% rename from tests/acceptance/vnc.py rename to tests/avocado/vnc.py diff --git a/tests/acceptance/x86_cpu_model_versions.py b/tests/avocado/x86= _cpu_model_versions.py similarity index 100% rename from tests/acceptance/x86_cpu_model_versions.py rename to tests/avocado/x86_cpu_model_versions.py --=20 2.33.1