From nobody Tue Feb 10 09:57:53 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1633009458117197.17931011998598; Thu, 30 Sep 2021 06:44:18 -0700 (PDT) Received: from localhost ([::1]:58462 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mVwMD-0003TZ-3B for importer@patchew.org; Thu, 30 Sep 2021 09:44:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36562) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mVwBX-0008AW-NE for qemu-devel@nongnu.org; Thu, 30 Sep 2021 09:33:15 -0400 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]:41570) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mVwBT-0005JW-Fs for qemu-devel@nongnu.org; Thu, 30 Sep 2021 09:33:14 -0400 Received: by mail-wm1-x32d.google.com with SMTP id g19-20020a1c9d13000000b003075062d4daso4384209wme.0 for ; Thu, 30 Sep 2021 06:33:08 -0700 (PDT) Received: from localhost.localdomain ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id c4sm3037168wrt.23.2021.09.30.06.33.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 06:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WIMLHBiyDPK1hazF+HIFOa18J+vh2aSGumNwFu7JSos=; b=cb2Of3q2Ymlr6RggMNA273N5e7609BIDJDE1RTfcpWIBRVExwESymk8We28yYIZevI +3JHkIzTr9YKmclPkVNZwqiuKcoRpT2AphBHpRAXtYlT4NMZuQVaWYcACCp7vCliXyKJ 6VxYGu9WUK+88Hhqew0hobnH9Y1RJWjlPS3wkcWqh4RsK0e32AjGFqxjVFSxm45VNqUs zOp8jlmdaNLP52YY9U8DmeLyBx21BI3wree1nYPzIQ0JuHM8cEpQL57BmqVZ238ycwWR mf6yCzQ+we6FNvhb0OKLqGbKQufBE4mMvX7E9cCaRokgvA2tdl6GrzrzbGg0cjKfI9og meqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=WIMLHBiyDPK1hazF+HIFOa18J+vh2aSGumNwFu7JSos=; b=vgc9JnDEco+4ybS4E8HOFTA6J0SpcH69FbduduaUYFElEMwzbTddSM/IMtlGr3+U93 FAOb/KTLKn0pxorXRRTrylYlD88scJEOkV6+DIwSC1yCBf/7T0P2QrXQ2pRko6K++0Ke ofJaWjJ3kMlEVQCHS/CcsTimW9u9DzsVidyE3fIj2sziONDfT1NQU8q4zyUC3Y0ylov0 P4YsdtwjFhfDDI2ZdvWF7hJ4xNT24TXXJRlEl/+duPFiMTaZFa4Y+vOSc2s3AOyE/1dh oGuN6TkS2AzUD9n/VSVtIAsl3zPdqAx7xbyPrhwQGlOyB7dhMleEK4Sj7P5VrSeYj74Q u7WA== X-Gm-Message-State: AOAM532COCfPvY8bGydUilgpi4nnCW4bAeuv8CjxWdDD8oYcNlWX7RAH oH1wVEPHTx/sckMg/zWyRXQY2//tbx8= X-Google-Smtp-Source: ABdhPJyk6yI7Nd8ekk9/k5rQiBMd6UQiT+oAw4RtMkzxBh2TD9b346PbnuOsE1r4XHtOjILt7OlMPw== X-Received: by 2002:a7b:c842:: with SMTP id c2mr15904189wml.93.1633008785506; Thu, 30 Sep 2021 06:33:05 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PATCH 7/7] docs: reorganize testing.rst Date: Thu, 30 Sep 2021 15:32:50 +0200 Message-Id: <20210930133250.181156-8-pbonzini@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210930133250.181156-1-pbonzini@redhat.com> References: <20210930133250.181156-1-pbonzini@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=2a00:1450:4864:20::32d; envelope-from=paolo.bonzini@gmail.com; helo=mail-wm1-x32d.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1633009459652100001 Content-Type: text/plain; charset="utf-8" Clean up the heading levels to use =3D=3D=3D --- ~~~ ^^^ '''. Reorganize t= he outline for the Avocado part, and always include headings for the class names. Signed-off-by: Paolo Bonzini --- docs/devel/testing.rst | 146 +++++++++++++++++++++-------------------- 1 file changed, 76 insertions(+), 70 deletions(-) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 200ce46c63..8df38f751d 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -1,11 +1,10 @@ -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Testing in QEMU =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 This document describes the testing infrastructure in QEMU. =20 Testing with "make check" -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D +------------------------- =20 The "make check" testing family includes most of the C based tests in QEMU= . For a quick help, run ``make check-help`` from the source tree. @@ -24,7 +23,7 @@ expect the executables to exist and will fail with obscur= e messages if they cannot find them. =20 Unit tests ----------- +~~~~~~~~~~ =20 Unit tests, which can be invoked with ``make check-unit``, are simple C te= sts that typically link to individual QEMU object files and exercise them by @@ -67,7 +66,7 @@ and copy the actual command line which executes the unit = test, then run it from the command line. =20 QTest ------ +~~~~~ =20 QTest is a device emulation testing framework. It can be very useful to t= est device models; it could also control certain aspects of QEMU (such as virt= ual @@ -81,7 +80,7 @@ QTest cases can be executed with make check-qtest =20 QAPI schema tests ------------------ +~~~~~~~~~~~~~~~~~ =20 The QAPI schema tests validate the QAPI parser used by QMP, by feeding predefined input to the parser and comparing the result with the reference @@ -108,14 +107,14 @@ parser (either fixing a bug or extending/modifying th= e syntax). To do this: ``qapi-schema +=3D foo.json`` =20 check-block ------------ +~~~~~~~~~~~ =20 ``make check-block`` runs a subset of the block layer iotests (the tests t= hat are in the "auto" group). See the "QEMU iotests" section below for more information. =20 QEMU iotests -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +------------ =20 QEMU iotests, under the directory ``tests/qemu-iotests``, is the testing framework widely used to test block layer related features. It is higher l= evel @@ -152,7 +151,7 @@ More options are supported by the ``./check`` script, r= un ``./check -h`` for help. =20 Writing a new test case ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ =20 Consider writing a tests case when you are making any changes to the block layer. An iotest case is usually the choice for that. There are already ma= ny @@ -206,7 +205,8 @@ test failure. If using such devices are explicitly des= ired, consider adding ``locking=3Doff`` option to disable image locking. =20 Debugging a test case ------------------------ +~~~~~~~~~~~~~~~~~~~~~ + The following options to the ``check`` script can be useful when debugging a failing test: =20 @@ -235,7 +235,7 @@ a failing test: ``$TEST_DIR/qemu-machine-``. =20 Test case groups ----------------- +~~~~~~~~~~~~~~~~ =20 "Tests may belong to one or more test groups, which are defined in the form of a comment in the test source file. By convention, test groups are listed @@ -285,10 +285,10 @@ Note that the following group names have a special me= aning: .. _container-ref: =20 Container based tests -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +--------------------- =20 Introduction ------------- +~~~~~~~~~~~~ =20 The container testing framework in QEMU utilizes public images to build and test QEMU in predefined and widely accessible Linux @@ -303,7 +303,7 @@ The container images are also used to augment the gener= ation of tests for testing TCG. See :ref:`checktcg-ref` for more details. =20 Docker Prerequisites --------------------- +~~~~~~~~~~~~~~~~~~~~ =20 Install "docker" with the system package manager and start the Docker serv= ice on your development machine, then make sure you have the privilege to run @@ -334,7 +334,7 @@ exploit the whole host with Docker bind mounting or oth= er privileged operations. So only do it on development machines. =20 Podman Prerequisites --------------------- +~~~~~~~~~~~~~~~~~~~~ =20 Install "podman" with the system package manager. =20 @@ -346,7 +346,7 @@ Install "podman" with the system package manager. The last command should print an empty table, to verify the system is read= y. =20 Quickstart ----------- +~~~~~~~~~~ =20 From source tree, type ``make docker-help`` to see the help. Testing can be started without configuring or building QEMU (``configure`` and @@ -362,7 +362,7 @@ is downloaded and initialized automatically), in which = the ``test-build`` job is executed. =20 Registry --------- +~~~~~~~~ =20 The QEMU project has a container registry hosted by GitLab at ``registry.gitlab.com/qemu-project/qemu`` which will automatically be @@ -376,7 +376,7 @@ locally by using the ``NOCACHE`` build option: make docker-image-debian10 NOCACHE=3D1 =20 Images ------- +~~~~~~ =20 Along with many other images, the ``centos8`` image is defined in a Docker= file in ``tests/docker/dockerfiles/``, called ``centos8.docker``. ``make docker= -help`` @@ -391,7 +391,7 @@ mainly used to do necessary host side setup. One such s= etup is ``binfmt_misc``, for example, to make qemu-user powered cross build containers work. =20 Tests ------ +~~~~~ =20 Different tests are added to cover various configurations to build and test QEMU. Docker tests are the executables under ``tests/docker`` named @@ -402,7 +402,7 @@ source and build it. The full list of tests is printed in the ``make docker-help`` help. =20 Debugging a Docker test failure -------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 When CI tasks, maintainers or yourself report a Docker test failure, follo= w the below steps to debug it: @@ -419,7 +419,7 @@ below steps to debug it: the prompt for debug. =20 Options -------- +~~~~~~~ =20 Various options can be used to affect how Docker tests are done. The full list is in the ``make docker`` help text. The frequently used ones are: @@ -433,7 +433,7 @@ list is in the ``make docker`` help text. The frequentl= y used ones are: failure" section. =20 Thread Sanitizer -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +---------------- =20 Thread Sanitizer (TSan) is a tool which can detect data races. QEMU suppo= rts building and testing with this tool. @@ -443,7 +443,7 @@ For more information on TSan: https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual =20 Thread Sanitizer in Docker ---------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~ TSan is currently supported in the ubuntu2004 docker. =20 The test-tsan test will build using TSan and then run make check. @@ -458,7 +458,7 @@ We recommend using DEBUG=3D1 to allow launching the tes= t from inside the docker, and to allow review of the warnings generated by TSan. =20 Building and Testing with TSan ------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ =20 It is possible to build and test with TSan, with a few additional steps. These steps are normally done automatically in the docker. @@ -497,7 +497,7 @@ This allows for running the test and then checking the = warnings afterwards. If you want TSan to stop and exit with error on warnings, use exitcode=3D6= 6. =20 TSan Suppressions ------------------ +~~~~~~~~~~~~~~~~~ Keep in mind that for any data race warning, although there might be a dat= a race detected by TSan, there might be no actual bug here. TSan provides several different mechanisms for suppressing warnings. In general it is recommend= ed @@ -523,7 +523,7 @@ More information on the file format can be found here u= nder "Blacklist Format": https://github.com/google/sanitizers/wiki/ThreadSanitizerFlags =20 TSan Annotations ----------------- +~~~~~~~~~~~~~~~~ include/qemu/tsan.h defines annotations. See this file for more descripti= ons of the annotations themselves. Annotations can be used to suppress TSan warnings or give TSan more information so that it can detect proper @@ -540,14 +540,14 @@ The full set of annotations can be found here: https://github.com/llvm/llvm-project/blob/master/compiler-rt/lib/tsan/rtl/= tsan_interface_ann.cpp =20 VM testing -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +---------- =20 This test suite contains scripts that bootstrap various guest images that = have necessary packages to build QEMU. The basic usage is documented in ``Makef= ile`` help which is displayed with ``make vm-help``. =20 Quickstart ----------- +~~~~~~~~~~ =20 Run ``make vm-help`` to list available make targets. Invoke a specific make command to run build test in an image. For example, ``make vm-build-freebs= d`` @@ -562,7 +562,7 @@ concerned about attackers taking control of the guest a= nd potentially exploiting a QEMU security bug to compromise the host. =20 QEMU binaries -------------- +~~~~~~~~~~~~~ =20 By default, qemu-system-x86_64 is searched in $PATH to run the guest. If t= here isn't one, or if it is older than 2.10, the test won't work. In this case, @@ -571,20 +571,20 @@ provide the QEMU binary in env var: ``QEMU=3D/path/to= /qemu-2.10+``. Likewise the path to qemu-img can be set in QEMU_IMG environment variable. =20 Make jobs ---------- +~~~~~~~~~ =20 The ``-j$X`` option in the make command line is not propagated into the VM, specify ``J=3D$X`` to control the make jobs in the guest. =20 Debugging ---------- +~~~~~~~~~ =20 Add ``DEBUG=3D1`` and/or ``V=3D1`` to the make command to allow interactive debugging and verbose output. If this is not enough, see the next section. ``V=3D1`` will be propagated down into the make jobs in the guest. =20 Manual invocation ------------------ +~~~~~~~~~~~~~~~~~ =20 Each guest script is an executable script with the same command line optio= ns. For example to work with the netbsd guest, use ``$QEMU_SRC/tests/vm/netbsd= ``: @@ -608,7 +608,7 @@ For example to work with the netbsd guest, use ``$QEMU_= SRC/tests/vm/netbsd``: $ ./netbsd --interactive --image /var/tmp/netbsd.img sh =20 Adding new guests ------------------ +~~~~~~~~~~~~~~~~~ =20 Please look at existing guest scripts for how to add new guests. =20 @@ -641,7 +641,7 @@ the script's ``main()``. recommended. =20 Image fuzzer testing -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +-------------------- =20 An image fuzzer was added to exercise format drivers. Currently only qcow2= is supported. To start the fuzzer, run @@ -654,7 +654,7 @@ Alternatively, some command different from "qemu-img in= fo" can be tested, by changing the ``-c`` option. =20 Acceptance tests using the Avocado Framework -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +-------------------------------------------- =20 The ``tests/acceptance`` directory hosts functional tests, also known as acceptance level tests. They're usually higher level tests, and @@ -693,7 +693,7 @@ Tests based on ``avocado_qemu.Test`` can easily: - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.u= tils.html =20 Running tests -------------- +~~~~~~~~~~~~~ =20 You can run the acceptance tests simply by executing: =20 @@ -722,7 +722,7 @@ may be invoked by running: tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/ =20 Manual Installation -------------------- +~~~~~~~~~~~~~~~~~~~ =20 To manually install Avocado and its dependencies, run: =20 @@ -735,7 +735,7 @@ Alternatively, follow the instructions on this link: https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/= installing.html =20 Overview --------- +~~~~~~~~ =20 The ``tests/acceptance/avocado_qemu`` directory provides the ``avocado_qemu`` Python module, containing the ``avocado_qemu.Test`` @@ -771,7 +771,7 @@ in the current directory, tagged as "quick", run: avocado run -t quick . =20 The ``avocado_qemu.Test`` base test class ------------------------------------------ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ =20 The ``avocado_qemu.Test`` class has a number of characteristics that are worth being mentioned right away. @@ -821,7 +821,7 @@ At test "tear down", ``avocado_qemu.Test`` handles all = the QEMUMachines shutdown. =20 The ``avocado_qemu.LinuxTest`` base test class -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ =20 The ``avocado_qemu.LinuxTest`` is further specialization of the ``avocado_qemu.Test`` class, so it contains all the characteristics of @@ -864,7 +864,7 @@ execution of a QEMU binary, giving its users: a more succinct and intuitive way =20 QEMU binary selection -~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^ =20 The QEMU binary used for the ``self.vm`` QEMUMachine instance will primarily depend on the value of the ``qemu_bin`` parameter. If it's @@ -885,20 +885,23 @@ The resulting ``qemu_bin`` value will be preserved in= the ``avocado_qemu.Test`` as an attribute with the same name. =20 Attribute reference -------------------- +~~~~~~~~~~~~~~~~~~~ + +Test +^^^^ =20 Besides the attributes and methods that are part of the base ``avocado.Test`` class, the following attributes are available on any ``avocado_qemu.Test`` instance. =20 vm -~~ +'' =20 A QEMUMachine instance, initially configured according to the given ``qemu_bin`` parameter. =20 arch -~~~~ +'''' =20 The architecture can be used on different levels of the stack, e.g. by the framework or by the test itself. At the framework level, it will @@ -915,7 +918,7 @@ name. If one is not given explicitly, it will either b= e set to ``:avocado: tags=3Darch:VALUE`` tag, it will be set to ``VALUE``. =20 cpu -~~~ +''' =20 The cpu model that will be set to all QEMUMachine instances created by the test. @@ -926,7 +929,7 @@ name. If one is not given explicitly, it will either be= set to ``:avocado: tags=3Dcpu:VALUE`` tag, it will be set to ``VALUE``. =20 machine -~~~~~~~ +''''''' =20 The machine type that will be set to all QEMUMachine instances created by the test. @@ -937,20 +940,20 @@ name. If one is not given explicitly, it will either= be set to ``:avocado: tags=3Dmachine:VALUE`` tag, it will be set to ``VALUE``. =20 qemu_bin -~~~~~~~~ +'''''''' =20 The preserved value of the ``qemu_bin`` parameter or the result of the dynamic probe for a QEMU binary in the current working directory or source tree. =20 LinuxTest -~~~~~~~~~ +^^^^^^^^^ =20 Besides the attributes present on the ``avocado_qemu.Test`` base class, the ``avocado_qemu.LinuxTest`` adds the following attributes: =20 distro -...... +'''''' =20 The name of the Linux distribution used as the guest image for the test. The name should match the **Provider** column on the list @@ -959,7 +962,7 @@ of images supported by the avocado.utils.vmimage librar= y: https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmim= age.html#supported-images =20 distro_version -.............. +'''''''''''''' =20 The version of the Linux distribution as the guest image for the test. The name should match the **Version** column on the list @@ -968,7 +971,7 @@ of images supported by the avocado.utils.vmimage librar= y: https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmim= age.html#supported-images =20 distro_checksum -............... +''''''''''''''' =20 The sha256 hash of the guest image file used for the test. =20 @@ -977,7 +980,7 @@ same name), no validation on the integrity of the image= will be performed. =20 Parameter reference -------------------- +~~~~~~~~~~~~~~~~~~~ =20 To understand how Avocado parameters are accessed by tests, and how they can be passed to tests, please refer to:: @@ -991,8 +994,11 @@ like the following: =20 PARAMS (key=3Dqemu_bin, path=3D*, default=3D./qemu-system-x86_64) =3D> '= ./qemu-system-x86_64 =20 +Test +^^^^ + arch -~~~~ +'''' =20 The architecture that will influence the selection of a QEMU binary (when one is not explicitly given). @@ -1005,31 +1011,30 @@ This parameter has a direct relation with the ``arc= h`` attribute. If not given, it will default to None. =20 cpu -~~~ +''' =20 The cpu model that will be set to all QEMUMachine instances created by the test. =20 machine -~~~~~~~ +''''''' =20 The machine type that will be set to all QEMUMachine instances created by the test. =20 - qemu_bin -~~~~~~~~ +'''''''' =20 The exact QEMU binary to be used on QEMUMachine. =20 LinuxTest -~~~~~~~~~ +^^^^^^^^^ =20 Besides the parameters present on the ``avocado_qemu.Test`` base class, the ``avocado_qemu.LinuxTest`` adds the following parameters: =20 distro -...... +'''''' =20 The name of the Linux distribution used as the guest image for the test. The name should match the **Provider** column on the list @@ -1038,7 +1043,7 @@ of images supported by the avocado.utils.vmimage libr= ary: https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmim= age.html#supported-images =20 distro_version -.............. +'''''''''''''' =20 The version of the Linux distribution as the guest image for the test. The name should match the **Version** column on the list @@ -1047,7 +1052,7 @@ of images supported by the avocado.utils.vmimage libr= ary: https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmim= age.html#supported-images =20 distro_checksum -............... +''''''''''''''' =20 The sha256 hash of the guest image file used for the test. =20 @@ -1055,7 +1060,8 @@ If this value is not set in the code or by this param= eter no validation on the integrity of the image will be performed. =20 Skipping tests --------------- +~~~~~~~~~~~~~~ + The Avocado framework provides Python decorators which allow for easily sk= ip tests running under certain conditions. For example, on the lack of a bina= ry on the test system or when the running environment is a CI system. For fur= ther @@ -1070,7 +1076,7 @@ environment variables became a kind of standard way t= o enable/disable tests. Here is a list of the most used variables: =20 AVOCADO_ALLOW_LARGE_STORAGE -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^ Tests which are going to fetch or produce assets considered *large* are not going to run unless that ``AVOCADO_ALLOW_LARGE_STORAGE=3D1`` is exported on the environment. @@ -1079,7 +1085,7 @@ The definition of *large* is a bit arbitrary here, bu= t it usually means an asset which occupies at least 1GB of size on disk when uncompressed. =20 AVOCADO_ALLOW_UNTRUSTED_CODE -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ There are tests which will boot a kernel image or firmware that can be considered not safe to run on the developer's workstation, thus they are skipped by default. The definition of *not safe* is also arbitrary but @@ -1090,7 +1096,7 @@ You should export ``AVOCADO_ALLOW_UNTRUSTED_CODE=3D1`= ` on the environment in order to allow tests which make use of those kind of assets. =20 AVOCADO_TIMEOUT_EXPECTED -~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^ The Avocado framework has a timeout mechanism which interrupts tests to av= oid the test suite of getting stuck. The timeout value can be set via test paramet= er or property defined in the test class, for further details:: @@ -1104,7 +1110,7 @@ compiled with debug flags. Therefore, the ``AVOCADO_T= IMEOUT_EXPECTED`` variable has been used to determine whether those tests should run or not. =20 GITLAB_CI -~~~~~~~~~ +^^^^^^^^^ A number of tests are flagged to not run on the GitLab CI. Usually because they proved to the flaky or there are constraints on the CI environment wh= ich would make them fail. If you encounter a similar situation then use that @@ -1117,7 +1123,7 @@ variable as shown on the code snippet below to skip t= he test: do_something() =20 Uninstalling Avocado --------------------- +~~~~~~~~~~~~~~~~~~~~ =20 If you've followed the manual installation instructions above, you can easily uninstall Avocado. Start by listing the packages you have @@ -1135,7 +1141,7 @@ Avocado is installed will be cleaned up as part of ``= make check-clean``. .. _checktcg-ref: =20 Testing with "make check-tcg" -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D +----------------------------- =20 The check-tcg tests are intended for simple smoke tests of both linux-user and softmmu TCG functionality. However to build test @@ -1168,7 +1174,7 @@ itself. See :ref:`container-ref` for more details. =20 Running subset of tests ------------------------ +~~~~~~~~~~~~~~~~~~~~~~~ =20 You can build the tests for one architecture:: =20 @@ -1182,7 +1188,7 @@ Adding ``V=3D1`` to the invocation will show the deta= ils of how to invoke QEMU for the test which is useful for debugging tests. =20 TCG test dependencies ---------------------- +~~~~~~~~~~~~~~~~~~~~~ =20 The TCG tests are deliberately very light on dependencies and are either totally bare with minimal gcc lib support (for softmmu tests) @@ -1216,7 +1222,7 @@ to run to exercise QEMU's linux-user code:: https://linux-test-project.github.io/ =20 GCC gcov support -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +---------------- =20 ``gcov`` is a GCC tool to analyze the testing coverage by instrumenting the tested code. To use it, configure QEMU with --=20 2.31.1