From: Kautuk Consul <kconsul@linux.vnet.ibm.com>
Avocado version 101.0 has a fix to re-compute the checksum
of an asset file if the algorithm used in the *-CHECKSUM
file isn't the same as the one being passed to it by the
avocado user (i.e. the avocado_qemu python module).
In the earlier avocado versions this fix wasn't there due
to which if the checksum wouldn't match the earlier
checksum (calculated by a different algorithm), the avocado
code would start downloading a fresh image from the internet
URL thus making the test-cases take longer to execute.
Bump up the avocado-framework version to 101.0.
Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
Tested-by: Hariharan T S <hariharan.ts@linux.vnet.ibm.com>
Message-Id: <20230327115030.3418323-2-kconsul@linux.vnet.ibm.com>
---
tests/requirements.txt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/requirements.txt b/tests/requirements.txt
index 0ba561b6bd..a6f73da681 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -2,5 +2,5 @@
# in the tests/venv Python virtual environment. For more info,
# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
# Note that qemu.git/python/ is always implicitly installed.
-avocado-framework==88.1
+avocado-framework==101.0
pycdlib==1.11.0
--
2.39.2On 30/03/2023 12.11, Alex Bennée wrote: > From: Kautuk Consul <kconsul@linux.vnet.ibm.com> > > Avocado version 101.0 has a fix to re-compute the checksum > of an asset file if the algorithm used in the *-CHECKSUM > file isn't the same as the one being passed to it by the > avocado user (i.e. the avocado_qemu python module). > In the earlier avocado versions this fix wasn't there due > to which if the checksum wouldn't match the earlier > checksum (calculated by a different algorithm), the avocado > code would start downloading a fresh image from the internet > URL thus making the test-cases take longer to execute. > > Bump up the avocado-framework version to 101.0. > > Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com> > Tested-by: Hariharan T S <hariharan.ts@linux.vnet.ibm.com> > Message-Id: <20230327115030.3418323-2-kconsul@linux.vnet.ibm.com> > --- > tests/requirements.txt | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/requirements.txt b/tests/requirements.txt > index 0ba561b6bd..a6f73da681 100644 > --- a/tests/requirements.txt > +++ b/tests/requirements.txt > @@ -2,5 +2,5 @@ > # in the tests/venv Python virtual environment. For more info, > # refer to: https://pip.pypa.io/en/stable/user_guide/#id1 > # Note that qemu.git/python/ is always implicitly installed. > -avocado-framework==88.1 > +avocado-framework==101.0 > pycdlib==1.11.0 Did you check whether the same amount of avocado tests still works as before? ... last time I tried to bump the version, a lot of things were failing, and I think Cleber was recently working on fixing things, but I haven't heart anything back from him yet that it would be OK to bump to a newer version now ... So upgrading to a new version of Avocado during the softfreeze sounds somewhat risky to me right now - I'd appreciate if we could do that after the release instead. Thomas
Thomas Huth <thuth@redhat.com> writes: > On 30/03/2023 12.11, Alex Bennée wrote: >> From: Kautuk Consul <kconsul@linux.vnet.ibm.com> >> Avocado version 101.0 has a fix to re-compute the checksum >> of an asset file if the algorithm used in the *-CHECKSUM >> file isn't the same as the one being passed to it by the >> avocado user (i.e. the avocado_qemu python module). >> In the earlier avocado versions this fix wasn't there due >> to which if the checksum wouldn't match the earlier >> checksum (calculated by a different algorithm), the avocado >> code would start downloading a fresh image from the internet >> URL thus making the test-cases take longer to execute. >> Bump up the avocado-framework version to 101.0. >> Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com> >> Tested-by: Hariharan T S <hariharan.ts@linux.vnet.ibm.com> >> Message-Id: <20230327115030.3418323-2-kconsul@linux.vnet.ibm.com> >> --- >> tests/requirements.txt | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> diff --git a/tests/requirements.txt b/tests/requirements.txt >> index 0ba561b6bd..a6f73da681 100644 >> --- a/tests/requirements.txt >> +++ b/tests/requirements.txt >> @@ -2,5 +2,5 @@ >> # in the tests/venv Python virtual environment. For more info, >> # refer to: https://pip.pypa.io/en/stable/user_guide/#id1 >> # Note that qemu.git/python/ is always implicitly installed. >> -avocado-framework==88.1 >> +avocado-framework==101.0 >> pycdlib==1.11.0 > > Did you check whether the same amount of avocado tests still works as > before? ... last time I tried to bump the version, a lot of things > were failing, and I think Cleber was recently working on fixing > things, but I haven't heart anything back from him yet that it would > be OK to bump to a newer version now ... I ran it on my default build and the only failure was: (008/222) tests/avocado/boot_linux.py:BootLinuxS390X.test_s390_ccw_virtio_tcg: INTERRUPTED: timeout (240.01 s) which passed on a retry. But now I realise with failfast it skipped a bunch: RESULTS : PASS 46 | ERROR 0 | FAIL 0 | SKIP 174 | WARN 0 | INTERRUPT 1 | CANCEL 1 JOB TIME : 290.26 s > So upgrading to a new version of Avocado during the softfreeze sounds > somewhat risky to me right now - I'd appreciate if we could do that > after the release instead. Sure. I was hoping we would speed up avocado a little by avoiding double downloads. > > Thomas -- Alex Bennée Virtualisation Tech Lead @ Linaro
On 30/03/2023 14.12, Alex Bennée wrote: > > Thomas Huth <thuth@redhat.com> writes: > >> On 30/03/2023 12.11, Alex Bennée wrote: >>> From: Kautuk Consul <kconsul@linux.vnet.ibm.com> >>> Avocado version 101.0 has a fix to re-compute the checksum >>> of an asset file if the algorithm used in the *-CHECKSUM >>> file isn't the same as the one being passed to it by the >>> avocado user (i.e. the avocado_qemu python module). >>> In the earlier avocado versions this fix wasn't there due >>> to which if the checksum wouldn't match the earlier >>> checksum (calculated by a different algorithm), the avocado >>> code would start downloading a fresh image from the internet >>> URL thus making the test-cases take longer to execute. >>> Bump up the avocado-framework version to 101.0. >>> Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com> >>> Tested-by: Hariharan T S <hariharan.ts@linux.vnet.ibm.com> >>> Message-Id: <20230327115030.3418323-2-kconsul@linux.vnet.ibm.com> >>> --- >>> tests/requirements.txt | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> diff --git a/tests/requirements.txt b/tests/requirements.txt >>> index 0ba561b6bd..a6f73da681 100644 >>> --- a/tests/requirements.txt >>> +++ b/tests/requirements.txt >>> @@ -2,5 +2,5 @@ >>> # in the tests/venv Python virtual environment. For more info, >>> # refer to: https://pip.pypa.io/en/stable/user_guide/#id1 >>> # Note that qemu.git/python/ is always implicitly installed. >>> -avocado-framework==88.1 >>> +avocado-framework==101.0 >>> pycdlib==1.11.0 >> >> Did you check whether the same amount of avocado tests still works as >> before? ... last time I tried to bump the version, a lot of things >> were failing, and I think Cleber was recently working on fixing >> things, but I haven't heart anything back from him yet that it would >> be OK to bump to a newer version now ... > > I ran it on my default build and the only failure was: > > (008/222) tests/avocado/boot_linux.py:BootLinuxS390X.test_s390_ccw_virtio_tcg: INTERRUPTED: timeout (240.01 s) > > which passed on a retry. But now I realise with failfast it skipped a bunch: That one is also failing for me here when I apply the patch. Without the patch, the test is working fine. I think this needs more careful testing first - e.g. the tests are run in parallel now by default, which breaks a lot of our timeout settings. Thomas
On 30/03/2023 14.21, Thomas Huth wrote:
> On 30/03/2023 14.12, Alex Bennée wrote:
>>
>> Thomas Huth <thuth@redhat.com> writes:
>>
>>> On 30/03/2023 12.11, Alex Bennée wrote:
>>>> From: Kautuk Consul <kconsul@linux.vnet.ibm.com>
>>>> Avocado version 101.0 has a fix to re-compute the checksum
>>>> of an asset file if the algorithm used in the *-CHECKSUM
>>>> file isn't the same as the one being passed to it by the
>>>> avocado user (i.e. the avocado_qemu python module).
>>>> In the earlier avocado versions this fix wasn't there due
>>>> to which if the checksum wouldn't match the earlier
>>>> checksum (calculated by a different algorithm), the avocado
>>>> code would start downloading a fresh image from the internet
>>>> URL thus making the test-cases take longer to execute.
>>>> Bump up the avocado-framework version to 101.0.
>>>> Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
>>>> Tested-by: Hariharan T S <hariharan.ts@linux.vnet.ibm.com>
>>>> Message-Id: <20230327115030.3418323-2-kconsul@linux.vnet.ibm.com>
>>>> ---
>>>> tests/requirements.txt | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>> diff --git a/tests/requirements.txt b/tests/requirements.txt
>>>> index 0ba561b6bd..a6f73da681 100644
>>>> --- a/tests/requirements.txt
>>>> +++ b/tests/requirements.txt
>>>> @@ -2,5 +2,5 @@
>>>> # in the tests/venv Python virtual environment. For more info,
>>>> # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
>>>> # Note that qemu.git/python/ is always implicitly installed.
>>>> -avocado-framework==88.1
>>>> +avocado-framework==101.0
>>>> pycdlib==1.11.0
>>>
>>> Did you check whether the same amount of avocado tests still works as
>>> before? ... last time I tried to bump the version, a lot of things
>>> were failing, and I think Cleber was recently working on fixing
>>> things, but I haven't heart anything back from him yet that it would
>>> be OK to bump to a newer version now ...
>>
>> I ran it on my default build and the only failure was:
>>
>> (008/222)
>> tests/avocado/boot_linux.py:BootLinuxS390X.test_s390_ccw_virtio_tcg:
>> INTERRUPTED: timeout (240.01 s)
>>
>> which passed on a retry. But now I realise with failfast it skipped a bunch:
>
> That one is also failing for me here when I apply the patch. Without the
> patch, the test is working fine. I think this needs more careful testing
> first - e.g. the tests are run in parallel now by default, which breaks a
> lot of our timeout settings.
FWIW, I think we likely want something like this added to this patch,
so we avoid to run those tests in parallel (unless requested with -jX):
diff a/tests/Makefile.include b/tests/Makefile.include
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -138,12 +138,15 @@ get-vm-image-fedora-31-%: check-venv
# download all vm images, according to defined targets
get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
+JOBS_OPTION=$(lastword -j1 $(filter-out -j, $(filter -j%, $(MAKEFLAGS))))
+
check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
$(call quiet-command, \
$(TESTS_PYTHON) -m avocado \
--show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
$(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
--filter-by-tags-include-empty-key) \
+ --max-parallel-tasks $(JOBS_OPTION:-j%=%) \
$(AVOCADO_CMDLINE_TAGS) \
$(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \
"AVOCADO", "tests/avocado")
That way we can avoid the timeout problems unless we found a
proper solution for those.
Thomas
© 2016 - 2025 Red Hat, Inc.