testing updates and mips deprecation

[PATCH v1 02/25] tests/avocado: reduce the default timeout to 120s

Posted by Alex Bennée 3 years, 5 months ago

We should be aiming to keep our tests under 2 minutes so lets reduce
the default timeout to that. Tests that we know take longer should
explicitly set a longer timeout.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/avocado/avocado_qemu/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
index 9d17a287cf..0efd2bd212 100644
--- a/tests/avocado/avocado_qemu/__init__.py
+++ b/tests/avocado/avocado_qemu/__init__.py
@@ -229,7 +229,7 @@ def exec_command_and_wait_for_pattern(test, command,
 class QemuBaseTest(avocado.Test):
 
     # default timeout for all tests, can be overridden
-    timeout = 900
+    timeout = 120
 
     def _get_unique_tag_val(self, tag_name):
         """
-- 
2.30.2

Re: [PATCH v1 02/25] tests/avocado: reduce the default timeout to 120s

Posted by Thomas Huth 3 years, 5 months ago

On 26/08/2022 19.21, Alex Bennée wrote:
> We should be aiming to keep our tests under 2 minutes so lets reduce
> the default timeout to that. Tests that we know take longer should
> explicitly set a longer timeout.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   tests/avocado/avocado_qemu/__init__.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py
> index 9d17a287cf..0efd2bd212 100644
> --- a/tests/avocado/avocado_qemu/__init__.py
> +++ b/tests/avocado/avocado_qemu/__init__.py
> @@ -229,7 +229,7 @@ def exec_command_and_wait_for_pattern(test, command,
>   class QemuBaseTest(avocado.Test):
>   
>       # default timeout for all tests, can be overridden
> -    timeout = 900
> +    timeout = 120

Did you try this on gitlab already? I guess it will fail in
some cases, e.g.:

  https://gitlab.com/qemu-project/qemu/-/jobs/2928561388

there are some tests that run definitely longer, e.g.:

  (005/192) tests/avocado/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv2:  PASS (342.26 s)
  (006/192) tests/avocado/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv3:  PASS (330.37 s)

and I can't spot a "timeout = ..." in that file.

  Thomas

Re: [PATCH v1 02/25] tests/avocado: reduce the default timeout to 120s

Posted by Alex Bennée 3 years, 5 months ago

Thomas Huth <thuth@redhat.com> writes:

> On 26/08/2022 19.21, Alex Bennée wrote:
>> We should be aiming to keep our tests under 2 minutes so lets reduce
>> the default timeout to that. Tests that we know take longer should
>> explicitly set a longer timeout.
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>   tests/avocado/avocado_qemu/__init__.py | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>> diff --git a/tests/avocado/avocado_qemu/__init__.py
>> b/tests/avocado/avocado_qemu/__init__.py
>> index 9d17a287cf..0efd2bd212 100644
>> --- a/tests/avocado/avocado_qemu/__init__.py
>> +++ b/tests/avocado/avocado_qemu/__init__.py
>> @@ -229,7 +229,7 @@ def exec_command_and_wait_for_pattern(test, command,
>>   class QemuBaseTest(avocado.Test):
>>         # default timeout for all tests, can be overridden
>> -    timeout = 900
>> +    timeout = 120
>
> Did you try this on gitlab already? I guess it will fail in
> some cases, e.g.:
>
>  https://gitlab.com/qemu-project/qemu/-/jobs/2928561388
>
> there are some tests that run definitely longer, e.g.:
>
>  (005/192) tests/avocado/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv2:  PASS (342.26 s)
>  (006/192) tests/avocado/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv3:  PASS (330.37 s)
>
> and I can't spot a "timeout = ..." in that file.

I've added some patches for that, ppc64le and s390 into v2

>
>  Thomas


-- 
Alex Bennée