From nobody Mon Feb 9 02:55:56 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) client-ip=170.10.129.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1657629895; cv=none; d=zohomail.com; s=zohoarc; b=H9C7eNZbTel/to32QujYqen9ThFnegvyCYmuGms7EDIdccn1XJmNPKlSOUn9nVqwPn0ZvU96bF8HjQ6MiSIyiLSZBjIUSHpjo4rs8l5t8uUKDtvTL8UhaQU8tVQ68AErL+KwNCypqPvey3IY5hBloPauMw5/Gpv/BMTde2u9LdI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1657629895; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CJaTuWa2iTwH8IVnYM08D3iI8/aC6G//ZZTGtW/kJqo=; b=HxfkA9vOAH6CT2E76lDHxEM6JcZjNP3fsEqqHuQn2tnITjBC19ZVAKl2LfcBq5xYAJqvcGA4ZXdl67hzpPMq48qmvOR7U7hni4QlBnj3uzPGOdFZrGEh0F+V8+B0+aS4oYVG6bJC2ZJv+YDHzvqqosSkuwSv6loLm9AfOLFRqNM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.zohomail.com with SMTPS id 1657629895922398.0503275267031; Tue, 12 Jul 2022 05:44:55 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-375-XCo2n-b-Pdq9EVtbn5jjaA-1; Tue, 12 Jul 2022 08:44:53 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7B8D5181E06C; Tue, 12 Jul 2022 12:44:48 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 65BA540D296E; Tue, 12 Jul 2022 12:44:48 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id E737B194705E; Tue, 12 Jul 2022 12:44:47 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 2D0341947058 for ; Tue, 12 Jul 2022 12:44:47 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 0FF16492CA2; Tue, 12 Jul 2022 12:44:47 +0000 (UTC) Received: from nautilus.redhat.com (unknown [10.40.193.6]) by smtp.corp.redhat.com (Postfix) with ESMTP id 82F59492C3B; Tue, 12 Jul 2022 12:44:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657629894; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=CJaTuWa2iTwH8IVnYM08D3iI8/aC6G//ZZTGtW/kJqo=; b=ZUhcCvAfIY6AHz0M08opA9cNFIHvaLEG6tul/KWiat3LO4HFUG75RVv/KAtjul1Yyh71vG J4/hibZlvOOISRsWeEjfAYizmTdTOwKoyt6692WixSkEMil++KDreOyF+068JX/QBFKrQf 261qa0AfDeBKC1Qs8S7Q5PJTThYxRPs= X-MC-Unique: XCo2n-b-Pdq9EVtbn5jjaA-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Erik Skultety To: libvir-list@redhat.com Subject: [libvirt PATCH 04/10] docs: Provide an article on testing Date: Tue, 12 Jul 2022 14:44:36 +0200 Message-Id: <57592e2914da8c618b11d0c6888e2c07664edcb7.1657629865.git.eskultet@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.9 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eskultet@redhat.com Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1657629896395100002 Content-Type: text/plain; charset="utf-8"; x-default="true" Currently we don't have much information on how testing is done in libvirt and the little we have is scattered among multiple files. This patch creates a common landing page containing all important bits about testing in libvirt, providing links to respective sections which deserve their own articles. Signed-off-by: Erik Skultety --- docs/meson.build | 1 + docs/testing.rst | 176 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 177 insertions(+) create mode 100644 docs/testing.rst diff --git a/docs/meson.build b/docs/meson.build index 0c7c54a75f..6d61e18385 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -105,6 +105,7 @@ docs_rst_files =3D [ 'submitting-patches', 'support', 'testapi', + 'testing', 'testsuites', 'testtck', 'uri', diff --git a/docs/testing.rst b/docs/testing.rst new file mode 100644 index 0000000000..af0f334633 --- /dev/null +++ b/docs/testing.rst @@ -0,0 +1,176 @@ +=3D=3D=3D=3D=3D=3D=3D +Testing +=3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Different types of tests are available to libvirt developers for testing a +given libvirt release. + +Unit tests +---------- + +The unit test suite present in the source code is mainly used to test our +XML parser/formatter, QEMU command line generator, QEMU capabilities probi= ng, +etc. It is run by developers before submitting patches upstream and is +mandatory to pass for any contribution to be accepted upstream. One can run +the test suite in the source tree with the following + +:: + + make check (libvirt 6.6.0 and older) + +:: + + ninja test (libvirt 6.7.0 and newer) + + +Container builds +---------------- + +Technically speaking these are not tests in the common sense. However, usa= ge of +public container images to build libvirt in predefined and widely accessib= le +environments makes it possible to expand our coverage across distros, +architectures, toolchain flavors and library versions and as such is a very +valuable marker when accepting upstream contributions. Therefore, it is +recommended to run libvirt builds against your changes in various containe= rs to +either locally or by using GitLab's shared CI runners to make sure everyth= ing +runs cleanly before submitting your patches. The images themselves come fr= om +libvirt's GitLab container registry, but this can be overriden if needed, = see +below. + +Registry +~~~~~~~~ + +Libvirt project has its container registry hosted by GitLab at +``registry.gitlab.com/libvirt/libvirt`` which will automatically be +used to pull in pre-built layers. This avoids the need to build all the +containers locally using the Dockerfile recipes found in ``ci/containers/`= `. + + +Running container builds with GitLab CI +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As long as your GitLab account has CI minutes available, pipelines will run +automatically on every branch push to your fork. + +Running container builds locally +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to run container builds locally, we have a ``helper`` script insi= de +the ``ci`` directory that can pull, build, and test (if applicable) change= s on +your current local branch. It supports both the Docker and Podman runtimes +with an automatic selection of whichever runtime is configured on your sys= tem. +In case neither has been enabled/configured, please go through the followi= ng +prerequisites. + +Docker Prerequisites +~~~~~~~~~~~~~~~~~~~~ + +Install "docker" with the system package manager and start the Docker serv= ice +on your development machine, then make sure you have the privilege to run +Docker commands. Typically it means setting up passwordless ``sudo docker`` +command or login as root. For example: + +.. code:: + + $ sudo yum install docker + $ # or `apt-get install docker` for Ubuntu, etc. + $ sudo systemctl start docker + $ sudo docker ps + +The last command should print an empty table, to verify the system is read= y. + +An alternative method to set up permissions is by adding the current user = to +"docker" group and making the docker daemon socket file (by default +``/var/run/docker.sock``) accessible to the group: + +.. code:: + + $ sudo groupadd docker + $ sudo usermod $USER -a -G docker + $ sudo chown :docker /var/run/docker.sock + +Note that any one of above configurations makes it possible for the user to +exploit the whole host with Docker bind mounting or other privileged +operations. So only do it on development machines. + +Podman Prerequisites +~~~~~~~~~~~~~~~~~~~~ + +Install "podman" with the system package manager. + +.. code:: + + $ sudo dnf install podman + $ podman ps + +The last command should print an empty table, to verify the system is read= y. + +Examples of executing local container builds +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +All of the following examples will utilize ``helper`` script mentioned ear= lier +sections. Let's start with the basics - listing available container images= in +the default libvirt registry: + +:: + + $ cd /ci + $ ./helper --help + $ ./helper list-images + Available x86 container images: + + ... + alpine-edge + fedora-rawhide + ... + + Available cross-compiler container images: + + ... + debian-sid-cross-s390x + fedora-rawhide-cross-mingw32 + fedora-rawhide-cross-mingw64 + ... + +Now let's say one would want to build their local libvirt changes on Alpine +Edge using their own GitLab's registry container. They'd then proceed with + +:: + + $ ci/helper build --image-prefix registry.gitlab.com//libvirt/ci= - alpine-edge + +Finally, it would be nice if one could get an interactive shell inside the +test environment to debug potential build issues. This can be achieved wit= h the +following: + +:: + + $ ci/helper shell alpine-edge + + +Integration tests +----------------- + +There are a few frameworks for writing and running functional tests in lib= virt +with TCK being the one that runs in our upstream CI. + +- the `TCK test suite `__ is a functional test suite implem= ented + using the `Perl bindings `__ of + libvirt. This is the recommended framework to use for writing upstream + functional tests at the moment. You can start by cloning the + `TCK git repo `__. + +- the `Avocado VT `__ te= st + suite with the libvirt plugin is another framework implementing functio= nal + testing utilizing the Avocado test framework underneath. Although writt= en in + Python, the vast majority of the tests are exercising libvirt through t= he + command line client ``virsh``. + +- the `libvirt-test-API `__ is also a functional test suite= , but + implemented using the `Python bindings `__ of libvirt. + Unfortunately this framework is the least recommended one as it's large= ly + unmaintained and may be completely deprecated in the future in favour o= f TCK. + You can get it by cloning the + `git repo `__. --=20 2.36.1