From nobody Tue Feb 10 06:52:25 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) client-ip=216.205.24.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1625073308; cv=none; d=zohomail.com; s=zohoarc; b=atIOS4z3ds+lG1ktiSKFU8ypzuHRZdsSS3XXr8aWFpCb1AF91n3UWplcx7PuseRlmEik4c4DVEhJ+/xrkXEUwxnEGqKu90PAL+hAbaO3UXh6q8n3hKT50QfuvQWYjlTb3qhxogYPUgr+T8QiwTats9K9Y+rPRT3hiDLiaONjKQM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1625073308; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=V6ZNFkxaNf/CGssdZjqufS/KznGcZtC6ju33T6qINH8=; b=ZrOSaIfvkrwxaUA29lc3pW+nqpOIZKIbyT+9vy7QjphLMBORPiTWz6VpdS2v3ZaV06G8INUlMUgjWgRfGHue1crh69OnqvzzUc1kD5n3zRpURqOW8tKVCuxtlqk2ayBE3GfovW3y8Yx1TSGZPTbYyiIWH+llD389wVlidpmPcGo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 216.205.24.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.zohomail.com with SMTPS id 1625073308092117.44087069159707; Wed, 30 Jun 2021 10:15:08 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-479-d1sil5FdOimGvVFcIkC4-g-1; Wed, 30 Jun 2021 13:14:04 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6CD2881C863; Wed, 30 Jun 2021 17:13:57 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 40876131F7; Wed, 30 Jun 2021 17:13:57 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 089A11809C9A; Wed, 30 Jun 2021 17:13:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 15UGapsC025803 for ; Wed, 30 Jun 2021 12:36:52 -0400 Received: by smtp.corp.redhat.com (Postfix) id D7685202877C; Wed, 30 Jun 2021 16:36:51 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D29F2202877B for ; Wed, 30 Jun 2021 16:36:51 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7B6D380D2B7 for ; Wed, 30 Jun 2021 16:36:51 +0000 (UTC) Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-27-PB-JTA7CNyekojFmI6Dy2w-1; Wed, 30 Jun 2021 12:36:50 -0400 Received: by mail-pf1-f199.google.com with SMTP id h6-20020a056a001a46b029030cacfca7bdso2089533pfv.13 for ; Wed, 30 Jun 2021 09:36:49 -0700 (PDT) Received: from localhost ([181.191.236.153]) by smtp.gmail.com with ESMTPSA id l22sm6957056pjy.1.2021.06.30.09.36.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Jun 2021 09:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625073306; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=V6ZNFkxaNf/CGssdZjqufS/KznGcZtC6ju33T6qINH8=; b=D85kHk54aRBK2YgoIGail92jLJtnyvS6Gz1hngPuvqT5Td5kvESsCeMPBhxC8sjXeBkOsE maGIUJ9JKZjjYd9YPL6aglNDXE7Uvn8/1P/F2WtKGEzBym2t7jUVwlmowZC9PuK79I91/0 sw7O59tnd+G5qUr6eWLDdhKWum/62ts= X-MC-Unique: d1sil5FdOimGvVFcIkC4-g-1 X-MC-Unique: PB-JTA7CNyekojFmI6Dy2w-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V6ZNFkxaNf/CGssdZjqufS/KznGcZtC6ju33T6qINH8=; b=ukPBDBAHLVL8nzDn3ujDBIbSDEU45fx/FRlq6Q8nm1mCt5Y2ffVqbhPqFoRLfKYt9i QcggzlBFeRQK0tqWrqU2pWwN4Ry5KUjuJsMqxHD9v3VlBjZDFUNuiQQlgcCA0/KbVvAl kvkuKWNHn0LMZpWkJzYsVYIkV2UzwvA+3K2BuB1D3STUZ/nDpye7PknmTqSRPKktTEUC 4oAd3wWs2m4z+m9zuFo8QIIeyguNE1GX7sOnPoPZa7h0Bt+syp9zVZweGuI8xbC3z2yd VdSaxQMH1dJ58Lgxfln1MqxR1d/e8fde5FhISAPM5RM0N6JLkZQa39wl5OpqMzBmOvTC yqag== X-Gm-Message-State: AOAM530vx7iac8ioKzg8YYzETWlA+IxQm25a0zG84/H2kVLms92nL+Ga 5IFHVhD/d3FDLFPEQWgYl+TRIAutl+T+ZNyKxdPy1Pbiy/U1fRJ01kpaCYW2nJm8K6RfpuYxDJV J0iwAlcydOdKKqPRNjPyDpmw2M0+SLqMB1CrW+3O2ZQlWuqLfoKiQoNkunDWRfrTRMPI= X-Received: by 2002:a63:338d:: with SMTP id z135mr15753346pgz.314.1625071008715; Wed, 30 Jun 2021 09:36:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxULi5rPRaEVJq0hSj8PAREkj6YG1dnYL9RwaM/AnaA3saiUKfDlw7iP/Iaw8RAXpxRijIr6w== X-Received: by 2002:a63:338d:: with SMTP id z135mr15753328pgz.314.1625071008435; Wed, 30 Jun 2021 09:36:48 -0700 (PDT) From: Beraldo Leal To: libvir-list@redhat.com Subject: [libvirt PATCH 4/4] tests.lavocado: adding a README and Makefile for convenience Date: Wed, 30 Jun 2021 13:36:34 -0300 Message-Id: <20210630163634.2801636-5-bleal@redhat.com> In-Reply-To: <20210630163634.2801636-1-bleal@redhat.com> References: <20210630163634.2801636-1-bleal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-loop: libvir-list@redhat.com Cc: Beraldo Leal X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Signed-off-by: Beraldo Leal --- tests/lavocado/Makefile | 2 + tests/lavocado/README.md | 124 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 126 insertions(+) create mode 100644 tests/lavocado/Makefile create mode 100644 tests/lavocado/README.md diff --git a/tests/lavocado/Makefile b/tests/lavocado/Makefile new file mode 100644 index 0000000000..33df604105 --- /dev/null +++ b/tests/lavocado/Makefile @@ -0,0 +1,2 @@ +libvirt-tests: + avocado run --test-runner=3D'nrunner' tests/ diff --git a/tests/lavocado/README.md b/tests/lavocado/README.md new file mode 100644 index 0000000000..b22015fe46 --- /dev/null +++ b/tests/lavocado/README.md @@ -0,0 +1,124 @@ +# lavocado - libvirt test framework based on Avocado + +The lavocado aims to be an alternative test framework for the libvirt proj= ect +using as muscles only Avocado and python-libvirt. This can be used to write +unit, functional and integration tests and it is inspired on the libvirt-t= ck +framework, but instead of Perl it written in Python. + +The idea is to provide to the test writers helper classes to avoid boilerp= late +code, improve code readability and maintenance. + +## Disclaimer + +**For now, this framework assumes that you are going to run the tests in a= fresh +clean environment, i.e. a VM. If you decide to use your local system, bewa= re +that execution of the tests may affect your system.** + +One of the future goals of this framework is to utilize nested virtualizat= ion +technologies and hence make sure an L1 guest is provisioned automatically = for +the tests to be executed in this environment and not tamper with your main +system. + +## Requirements + +The libvirt interactions will all be done via the libvirt Python bindings +(libvirt-python), we also rely on Avocado, not only to execute the tests b= ut +also to generate multiple output format artifacts (that can be uploaded to= CI +environments) and also the image fetch, so we don't need to handle with lo= cal +cache, checksum and any other test's requirement task. + +In order to install the requirements, please execute the following command: + +```bash + $ pip3 install -r requirements.txt +``` + +If you wish you can run this command under a virtual environment. + +## Features + +Bellow, we list some features that can be explored from this proposal: + + * Parallel execution: If your tests are independent of each other, you ca= n run + those in parallel, when getting images, Avocado will create a snapshot = to + avoid conflicts. Also domains created by tests have unique names to avo= id + collisions. + + * Creating domains easily: You can use the `.create_domain()` method to c= reate + a generic domain which is based on a Jinja2 template. If you want to us= e an + already existing XML domain description, no problem, use + `Domain.from_xml_path()` instead. + + * Multiple output formats: TAP, HTML, JSON, xUnit (Gitlab CI ready). + + * Test tags: Ability to mark tests with tags, for filtering during execut= ion + and/or results. + + * Different isolation models: By default the avocado runner will spawn ea= ch + test on local processes (subprocess), but we can configure for specific= use + cases, running the tests on different isolation models (containers for + instance). + + * Requirements resolver: Avocado has implemented the 'requirements resolv= er' + feature that makes it easy to define test requirements. Currently, The + RequirementsResolver can handle `assets` (any local or remote files) and + `packages` but new requirements are on the roadmap, including `ansible + playbooks`. For now, it is still only an experimental feature though. + + * Test result artifacts for future analysis: you can take a look at `avoc= ado + jobs` command or inspect your `~/avocado/test-results/` folder for more + detailed results. + + * Multiple test execution with variants: the variants subsystem is what a= llows + the creation of multiple variations of parameters, and the execution of + tests with those parameter variations. + + * Clear separation between tests and bootstrap stages: If something fails + during the setUp() metod execution, your test will be not marked as FAI= L, + instead it will be flagged as ERROR. You can also use some decorators + (@cancel_on, @skipUnless, ...) around your test to avoid false positive= s. + + * Ready-to-use utility libraries: `avocado.utils` are rich, and cover + common operating system level needs, such as service managment, + hardware introspection, storage management, networking, etc. + + * Avocado Job API: a programmable interface for defining test jobs, which + allows one to create custom test jobs that are suitable to developer's + systems and CI enviroments. + + * Output artifacts for debug: Avocado stores all job results in + `~/avocado/job-results/` (you can also see details with `avocado jobs` + command). If any test fails we can debug the files there. Besides that,= using + `sysinfo` plugin, we could collect additional system information (such = as + libvirt/qemu logs). + +## Running + +After installing the requirements, you can run the tests with the following +commands: + +```bash + $ export PYTHONPATH=3D.:$PYTHONPATH + $ avocado run --test-runner=3D'nrunner' ./tests/domain/*.py +``` + +Please note that the Next Runner (nrunner) will be the default runner soon= in +Avocado, and so the `--test-runner=3D'nrunner'` option will no longer be n= eeded. + +Or, if you prefer, you can also execute the tests with `make`: + +```bash + $ make libvirt-tests +``` + +## Writing Tests + +You can write your tests here the same way you write for the [Avocado +Framework](https://avocado-framework.readthedocs.io/en/latest/). +Avocado supports "simple tests" (just executables) and "instrumented tests" +(Python tests). + +See the `tests/` folder for some references and ideas. In addition, feel f= ree +to read the [Avocado Test Writer=E2=80=99s +Guide](https://avocado-framework.readthedocs.io/en/latest/guides/writer/) = to +play with some advanced features of the framework. --=20 2.26.3