Avocado tests: filter out tests using TCG booting full blown distros

[PATCH 4/9] Avocado: bump to version 95.0

Cleber Rosa posted 9 patches 3 years, 11 months ago

Download mbox

Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, Thomas Huth <thuth@redhat.com>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Beraldo Leal <bleal@redhat.com>, Cleber Rosa <crosa@redhat.com>, Aurelien Jarno <aurelien@aurel32.net>, Antony Pavlov <antonynpavlov@gmail.com>, Peter Maydell <peter.maydell@linaro.org>, Michael Rolnik <mrolnik@gmail.com>, "Edgar E. Iglesias" <edgar.iglesias@gmail.com>, Huacai Chen <chenhuacai@kernel.org>, Jiaxun Yang <jiaxun.yang@flygoat.com>, Yoshinori Sato <ysato@users.sourceforge.jp>, Halil Pasic <pasic@linux.ibm.com>, Christian Borntraeger <borntraeger@linux.ibm.com>, Eric Farman <farman@linux.ibm.com>, Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>, Fabien Chouteau <chouteau@adacore.com>, KONRAD Frederic <frederic.konrad@adacore.com>, "Hervé Poussineau" <hpoussin@reactos.org>, "Cédric Le Goater" <clg@kaod.org>, Daniel Henrique Barboza <danielhb413@gmail.com>, David Gibson <david@gibson.dropbear.id.au>, Greg Kurz <groug@kaod.org>, Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>, Paolo Bonzini <pbonzini@redhat.com>, Eric Auger <eric.auger@redhat.com>, Alexandre Iooss <erdnaxe@crans.org>, Mahmoud Mandour <ma.mandourr@gmail.com>

Expand all Fold all

[PATCH 4/9] Avocado: bump to version 95.0

Posted by Cleber Rosa 3 years, 11 months ago

Even though there have been a number of improvements (and some pretty
deep internal changes) since Avocado 88.1, only one change should
affect "make check-avocado".

With the nrunner architecture, test execution happens in parallel by
default.  But, tests may fail due to insufficient timeouts or similar
reasons when run under systems with limited or shared resources.  To
avoid breakages, especially on CI, let's keep the serial execution
until proven that it won't impact the CI jobs.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/Makefile.include | 1 +
 tests/requirements.txt | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index e7153c8e91..676aa0d944 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -135,6 +135,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
             $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
 			--filter-by-tags-include-empty-key) \
             $(AVOCADO_CMDLINE_TAGS) \
+            --nrunner-max-parallel-tasks=1 \
             $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \
             "AVOCADO", "tests/avocado")
 
diff --git a/tests/requirements.txt b/tests/requirements.txt
index a21b59b443..49aa0fd6f6 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,5 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==88.1
+avocado-framework==95.0
 pycdlib==1.11.0
-- 
2.35.1

Re: [PATCH 4/9] Avocado: bump to version 95.0

Posted by Thomas Huth 3 years, 11 months ago

On 25/02/2022 22.01, Cleber Rosa wrote:
> Even though there have been a number of improvements (and some pretty
> deep internal changes) since Avocado 88.1, only one change should
> affect "make check-avocado".
> 
> With the nrunner architecture, test execution happens in parallel by
> default.  But, tests may fail due to insufficient timeouts or similar
> reasons when run under systems with limited or shared resources.  To
> avoid breakages, especially on CI, let's keep the serial execution
> until proven that it won't impact the CI jobs.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>   tests/Makefile.include | 1 +
>   tests/requirements.txt | 2 +-
>   2 files changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index e7153c8e91..676aa0d944 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -135,6 +135,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
>               $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
>   			--filter-by-tags-include-empty-key) \
>               $(AVOCADO_CMDLINE_TAGS) \
> +            --nrunner-max-parallel-tasks=1 \

Could you maybe change it so that it uses the "-jX" setting from "make" 
instead? ... that way the users could decide whether they want to run the 
tests in parallel or not.

I think you could get that value from the MAKEFLAGS env variable, see 
tests/check-block.sh :

JOBS=$(echo "$MAKEFLAGS" | sed -n 's/\(^\|.* \)-j\([0-9]\+\)\( .*\|$\)/-j \2/p')


  Thomas