From nobody Fri May 17 05:26:31 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1625598498; cv=none; d=zohomail.com; s=zohoarc; b=mPrKeURp2dBaiAq5wYlV6TAk7/E5YldfiJEVC8i3HtuYNhtmC/82WjYBlb7DyS0SUDrBLkEjFhgv6fenOUGMgelj4fQDxRpPdHWsye+2qgpbckrIKEf91zpTyeVFHRqdOETkBZUWOitKfBtYOsYV+aeKYhKLBwQOI6afIXQ7SGw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1625598498; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=YyhG+M8nVcpsAVybwnL3VozwfpwR/4r5MJhaBcsug+8=; b=jMTqaXKN5FbUWR+pO/jhiLeTDt/vnU0tZOAZ6kmW6fBumLAogEJN3smlhK8j5gZnLwVSxL8zUzx9cxDy0axyXP08FCJ8AhEg9nUSHVBUg6yAYQ7WqDtA7afFC2rAP/6UGXVH14x1vVxFIRAv4fYERJ66IlK68uHGJznM6kgc/h8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1625598498801850.044536594985; Tue, 6 Jul 2021 12:08:18 -0700 (PDT) Received: from localhost ([::1]:51000 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m0qQb-0001CU-Em for importer@patchew.org; Tue, 06 Jul 2021 15:08:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m0qPM-0000Sa-ND for qemu-devel@nongnu.org; Tue, 06 Jul 2021 15:07:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43461) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m0qPK-0003WB-Ab for qemu-devel@nongnu.org; Tue, 06 Jul 2021 15:06:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-249-EL3LjK8MNwGpKXjPDt21PA-1; Tue, 06 Jul 2021 15:06:56 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0F85980414B; Tue, 6 Jul 2021 19:06:55 +0000 (UTC) Received: from wrampazz.redhat.com (ovpn-114-66.rdu2.redhat.com [10.10.114.66]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC9285D703; Tue, 6 Jul 2021 19:06:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625598417; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=YyhG+M8nVcpsAVybwnL3VozwfpwR/4r5MJhaBcsug+8=; b=REZ6qh6M7wWW9aTxBxj2knmZ2Dea1hgCzYCRllQxJGN0TAUyrvNWF3nRq9J5fzYQRPCgkT oalV4fKK/uZkRSZWmhoNUJmDzTAcNe+YxXWdYy2Qsnc3BqpEbxWmJXC9gCB8l2sXPMZ4uQ 5fu8gmcccREoO3ALxnYn5DqyaXTgePw= X-MC-Unique: EL3LjK8MNwGpKXjPDt21PA-1 From: Willian Rampazzo To: alex.bennee@linaro.org, f4bug@amsat.org, crosa@redhat.com, thuth@redhat.com, wainersm@redhat.com, qemu-devel@nongnu.org, philmd@redhat.com Subject: [PATCH] tests/acceptance: allow control over tags during check-acceptance Date: Tue, 6 Jul 2021 16:06:46 -0300 Message-Id: <20210706190646.209440-1-willianr@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=willianr@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=willianr@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.442, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1625598499888100001 Content-Type: text/plain; charset="utf-8" Although it is possible to run a specific test using the avocado command-line, a user may want to use a specific tag while running the ``make check-acceptance`` during the development or debugging. This allows using the TAGS environment variable where the user takes total control of which tests should run based on the tags defined. This also makes the check-acceptance command flexible to restrict tests based on tags while running on CI. e.g.: TAGS=3D"foo bar baz" make check-acceptance Signed-off-by: Willian Rampazzo --- docs/devel/testing.rst | 7 +++++++ tests/Makefile.include | 10 ++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 4e42392810..6e03c3449b 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -760,6 +760,13 @@ in the current directory, tagged as "quick", run: =20 avocado run -t quick . =20 +It is also possible to run tests based on tags using the +``make check-acceptance`` command and the ``TAGS`` environment variable: + +.. code:: + + TAGS=3Dquick make check-acceptance + The ``avocado_qemu.Test`` base test class ----------------------------------------- =20 diff --git a/tests/Makefile.include b/tests/Makefile.include index 8f220e15d1..5869ab8a04 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -92,7 +92,11 @@ TESTS_RESULTS_DIR=3D$(BUILD_DIR)/tests/results # Any number of command separated loggers are accepted. For more # information please refer to "avocado --help". AVOCADO_SHOW=3Dapp -AVOCADO_TAGS=3D$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGET= S))) +ifndef TAGS + AVOCADO_TAGS=3D$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGE= TS))) +else + AVOCADO_TAGS=3D$(addprefix -t , $(TAGS)) +endif =20 $(TESTS_VENV_DIR): $(TESTS_VENV_REQ) $(call quiet-command, \ @@ -125,10 +129,12 @@ get-vm-image-fedora-31-%: check-venv get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_= 31_DOWNLOAD)) =20 check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images + @echo "AVOCADO_TAGS $(AVOCADO_TAGS)" $(call quiet-command, \ $(TESTS_VENV_DIR)/bin/python -m avocado \ --show=3D$(AVOCADO_SHOW) run --job-results-dir=3D$(TESTS_RESUL= TS_DIR) \ - --filter-by-tags-include-empty --filter-by-tags-include-empty-= key \ + $(if $(TAGS),, --filter-by-tags-include-empty \ + --filter-by-tags-include-empty-key) \ $(AVOCADO_TAGS) \ $(if $(GITLAB_CI),,--failfast) tests/acceptance, \ "AVOCADO", "tests/acceptance") --=20 2.31.1