docs/devel/testing.rst | 14 ++++++++++++++ tests/Makefile.include | 12 +++++++++--- 2 files changed, 23 insertions(+), 3 deletions(-)
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.
This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.
This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.
e.g.:
AVOCADO_TAGS="foo bar baz" make check-acceptance
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
---
docs/devel/testing.rst | 14 ++++++++++++++
tests/Makefile.include | 12 +++++++++---
2 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 4e42392810..a81b44cf18 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -760,6 +760,20 @@ in the current directory, tagged as "quick", run:
avocado run -t quick .
+It is also possible to run tests based on tags using the
+``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment
+variable:
+
+.. code::
+
+ AVOCADO_TAGS=quick make check-acceptance
+
+Note that tags separated with commas have an AND behavior, while tags
+separated by spaces have an OR behavior. For more information on Avocado
+tags, see:
+
+ https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html
+
The ``avocado_qemu.Test`` base test class
-----------------------------------------
diff --git a/tests/Makefile.include b/tests/Makefile.include
index e4dcb17329..0ce331f9a9 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -92,7 +92,12 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
# Any number of command separated loggers are accepted. For more
# information please refer to "avocado --help".
AVOCADO_SHOW=app
-AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
+ifndef AVOCADO_TAGS
+ AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \
+ $(filter %-softmmu,$(TARGETS)))
+else
+ AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS))
+endif
$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
$(call quiet-command, \
@@ -128,8 +133,9 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
$(call quiet-command, \
$(TESTS_VENV_DIR)/bin/python -m avocado \
--show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
- --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
- $(AVOCADO_TAGS) \
+ $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
+ --filter-by-tags-include-empty-key) \
+ $(AVOCADO_CMDLINE_TAGS) \
$(if $(GITLAB_CI),,--failfast) tests/acceptance, \
"AVOCADO", "tests/acceptance")
--
2.31.1
On 13/07/2021 23.01, Willian Rampazzo wrote: > Although it is possible to run a specific test using the avocado > command-line, a user may want to use a specific tag while running the > ``make check-acceptance`` during the development or debugging. > > This allows using the AVOCADO_TAGS environment variable where the user > takes total control of which tests should run based on the tags defined. > > This also makes the check-acceptance command flexible to restrict tests > based on tags while running on CI. > > e.g.: > > AVOCADO_TAGS="foo bar baz" make check-acceptance > > Signed-off-by: Willian Rampazzo <willianr@redhat.com> > Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > --- > docs/devel/testing.rst | 14 ++++++++++++++ > tests/Makefile.include | 12 +++++++++--- > 2 files changed, 23 insertions(+), 3 deletions(-) > > diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst > index 4e42392810..a81b44cf18 100644 > --- a/docs/devel/testing.rst > +++ b/docs/devel/testing.rst > @@ -760,6 +760,20 @@ in the current directory, tagged as "quick", run: > > avocado run -t quick . > > +It is also possible to run tests based on tags using the > +``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment > +variable: > + > +.. code:: > + > + AVOCADO_TAGS=quick make check-acceptance > + > +Note that tags separated with commas have an AND behavior, while tags > +separated by spaces have an OR behavior. For more information on Avocado > +tags, see: > + > + https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html > + > The ``avocado_qemu.Test`` base test class > ----------------------------------------- > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index e4dcb17329..0ce331f9a9 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -92,7 +92,12 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results > # Any number of command separated loggers are accepted. For more > # information please refer to "avocado --help". > AVOCADO_SHOW=app > -AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS))) > +ifndef AVOCADO_TAGS > + AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \ > + $(filter %-softmmu,$(TARGETS))) > +else > + AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS)) > +endif > > $(TESTS_VENV_DIR): $(TESTS_VENV_REQ) > $(call quiet-command, \ > @@ -128,8 +133,9 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images > $(call quiet-command, \ > $(TESTS_VENV_DIR)/bin/python -m avocado \ > --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ > - --filter-by-tags-include-empty --filter-by-tags-include-empty-key \ > - $(AVOCADO_TAGS) \ > + $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \ > + --filter-by-tags-include-empty-key) \ > + $(AVOCADO_CMDLINE_TAGS) \ > $(if $(GITLAB_CI),,--failfast) tests/acceptance, \ > "AVOCADO", "tests/acceptance") > > Reviewed-by: Thomas Huth <thuth@redhat.com>
Ping? Any good soul to send a pull request? On Tue, Jul 13, 2021 at 6:03 PM Willian Rampazzo <willianr@redhat.com> wrote: > > Although it is possible to run a specific test using the avocado > command-line, a user may want to use a specific tag while running the > ``make check-acceptance`` during the development or debugging. > > This allows using the AVOCADO_TAGS environment variable where the user > takes total control of which tests should run based on the tags defined. > > This also makes the check-acceptance command flexible to restrict tests > based on tags while running on CI. > > e.g.: > > AVOCADO_TAGS="foo bar baz" make check-acceptance > > Signed-off-by: Willian Rampazzo <willianr@redhat.com> > Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> > --- > docs/devel/testing.rst | 14 ++++++++++++++ > tests/Makefile.include | 12 +++++++++--- > 2 files changed, 23 insertions(+), 3 deletions(-) > > diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst > index 4e42392810..a81b44cf18 100644 > --- a/docs/devel/testing.rst > +++ b/docs/devel/testing.rst > @@ -760,6 +760,20 @@ in the current directory, tagged as "quick", run: > > avocado run -t quick . > > +It is also possible to run tests based on tags using the > +``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment > +variable: > + > +.. code:: > + > + AVOCADO_TAGS=quick make check-acceptance > + > +Note that tags separated with commas have an AND behavior, while tags > +separated by spaces have an OR behavior. For more information on Avocado > +tags, see: > + > + https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html > + > The ``avocado_qemu.Test`` base test class > ----------------------------------------- > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index e4dcb17329..0ce331f9a9 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -92,7 +92,12 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results > # Any number of command separated loggers are accepted. For more > # information please refer to "avocado --help". > AVOCADO_SHOW=app > -AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS))) > +ifndef AVOCADO_TAGS > + AVOCADO_CMDLINE_TAGS=$(patsubst %-softmmu,-t arch:%, \ > + $(filter %-softmmu,$(TARGETS))) > +else > + AVOCADO_CMDLINE_TAGS=$(addprefix -t , $(AVOCADO_TAGS)) > +endif > > $(TESTS_VENV_DIR): $(TESTS_VENV_REQ) > $(call quiet-command, \ > @@ -128,8 +133,9 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images > $(call quiet-command, \ > $(TESTS_VENV_DIR)/bin/python -m avocado \ > --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ > - --filter-by-tags-include-empty --filter-by-tags-include-empty-key \ > - $(AVOCADO_TAGS) \ > + $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \ > + --filter-by-tags-include-empty-key) \ > + $(AVOCADO_CMDLINE_TAGS) \ > $(if $(GITLAB_CI),,--failfast) tests/acceptance, \ > "AVOCADO", "tests/acceptance") > > -- > 2.31.1 > >
© 2016 - 2024 Red Hat, Inc.