tests/functional: Bump some arm test timeouts

[PATCH] tests/functional: Bump some arm test timeouts

Posted by Peter Maydell 1 day, 7 hours ago

On my local machine, for a debug build, sbsaref_alpine takes
nearly 900s:

$ (cd build/x86 && ./pyvenv/bin/meson test --setup thorough --suite func-thorough func-aarch64-aarch64_sbsaref_alpine
)

1/1 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_sbsaref_alpine
                      OK 896.90s

arm_aspeed_rainier can also run close to its current timeout:
 6/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_rainier
                      OK 215.75s

and arm_aspeed_ast2500 and arm_aspeed_ast2600 can go over:
13/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_ast2600
                      OK 792.94s

27/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_ast2500
                 TIMEOUT 480.01s

The sx1 test fails not on the overall meson timeout but on the
60 second timeout in some of the subtests.

Bump all these timeouts up a bit.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
This at least gets 'make -j4 check-functional' to passing for me
for an arm/aarch64 debug build, apart from the gpu test hang which
we're discussing in a different thread. Whole thing takes 25 mins...
---
 tests/functional/meson.build     | 8 ++++----
 tests/functional/test_arm_sx1.py | 6 +++---
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index b516d21cba1..effa31701cf 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -15,16 +15,16 @@ test_timeouts = {
   'aarch64_raspi4' : 480,
   'aarch64_rme_virt' : 1200,
   'aarch64_rme_sbsaref' : 1200,
-  'aarch64_sbsaref_alpine' : 720,
+  'aarch64_sbsaref_alpine' : 1200,
   'aarch64_sbsaref_freebsd' : 720,
   'aarch64_tuxrun' : 240,
   'aarch64_virt' : 720,
   'acpi_bits' : 420,
   'arm_aspeed_palmetto' : 120,
   'arm_aspeed_romulus' : 120,
-  'arm_aspeed_ast2500' : 480,
-  'arm_aspeed_ast2600' : 720,
-  'arm_aspeed_rainier' : 240,
+  'arm_aspeed_ast2500' : 720,
+  'arm_aspeed_ast2600' : 1200,
+  'arm_aspeed_rainier' : 480,
   'arm_bpim2u' : 500,
   'arm_collie' : 180,
   'arm_cubieboard' : 360,
diff --git a/tests/functional/test_arm_sx1.py b/tests/functional/test_arm_sx1.py
index b85bfaa178f..4dd1e1859fa 100755
--- a/tests/functional/test_arm_sx1.py
+++ b/tests/functional/test_arm_sx1.py
@@ -44,7 +44,7 @@ def test_arm_sx1_initrd(self):
         self.vm.add_args('-no-reboot')
         self.launch_kernel(zimage_path,
                            initrd=initrd_path)
-        self.vm.wait(timeout=60)
+        self.vm.wait(timeout=120)
 
     def test_arm_sx1_sd(self):
         self.set_machine('sx1')
@@ -55,7 +55,7 @@ def test_arm_sx1_sd(self):
         self.vm.add_args('-snapshot')
         self.vm.add_args('-drive', f'format=raw,if=sd,file={sd_fs_path}')
         self.launch_kernel(zimage_path)
-        self.vm.wait(timeout=60)
+        self.vm.wait(timeout=120)
 
     def test_arm_sx1_flash(self):
         self.set_machine('sx1')
@@ -66,7 +66,7 @@ def test_arm_sx1_flash(self):
         self.vm.add_args('-snapshot')
         self.vm.add_args('-drive', f'format=raw,if=pflash,file={flash_path}')
         self.launch_kernel(zimage_path)
-        self.vm.wait(timeout=60)
+        self.vm.wait(timeout=120)
 
 if __name__ == '__main__':
     LinuxKernelTest.main()
-- 
2.43.0

Re: [PATCH] tests/functional: Bump some arm test timeouts

Posted by Thomas Huth 1 day, 7 hours ago

On 21/02/2025 15.06, Peter Maydell wrote:
> On my local machine, for a debug build, sbsaref_alpine takes
> nearly 900s:
> 
> $ (cd build/x86 && ./pyvenv/bin/meson test --setup thorough --suite func-thorough func-aarch64-aarch64_sbsaref_alpine
> )
> 
> 1/1 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_sbsaref_alpine
>                        OK 896.90s
> 
> arm_aspeed_rainier can also run close to its current timeout:
>   6/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_rainier
>                        OK 215.75s
> 
> and arm_aspeed_ast2500 and arm_aspeed_ast2600 can go over:
> 13/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_ast2600
>                        OK 792.94s
> 
> 27/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_ast2500
>                   TIMEOUT 480.01s
> 
> The sx1 test fails not on the overall meson timeout but on the
> 60 second timeout in some of the subtests.
> 
> Bump all these timeouts up a bit.
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> This at least gets 'make -j4 check-functional' to passing for me
> for an arm/aarch64 debug build, apart from the gpu test hang which
> we're discussing in a different thread. Whole thing takes 25 mins...
> ---
>   tests/functional/meson.build     | 8 ++++----
>   tests/functional/test_arm_sx1.py | 6 +++---
>   2 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> index b516d21cba1..effa31701cf 100644
> --- a/tests/functional/meson.build
> +++ b/tests/functional/meson.build
> @@ -15,16 +15,16 @@ test_timeouts = {
>     'aarch64_raspi4' : 480,
>     'aarch64_rme_virt' : 1200,
>     'aarch64_rme_sbsaref' : 1200,
> -  'aarch64_sbsaref_alpine' : 720,
> +  'aarch64_sbsaref_alpine' : 1200,

I wonder whether we should disable that test with @skipSlowTest() by 
default, since it's really very slow...?

Anyway, for this patch here:
Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH] tests/functional: Bump some arm test timeouts

Posted by Alex Bennée 1 day, 6 hours ago

Thomas Huth <thuth@redhat.com> writes:

> On 21/02/2025 15.06, Peter Maydell wrote:
>> On my local machine, for a debug build, sbsaref_alpine takes
>> nearly 900s:
>> $ (cd build/x86 && ./pyvenv/bin/meson test --setup thorough --suite
>> func-thorough func-aarch64-aarch64_sbsaref_alpine
>> )
>> 1/1 qemu:func-thorough+func-aarch64-thorough+thorough /
>> func-aarch64-aarch64_sbsaref_alpine
>>                        OK 896.90s
>> arm_aspeed_rainier can also run close to its current timeout:
>>   6/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_rainier
>>                        OK 215.75s
>> and arm_aspeed_ast2500 and arm_aspeed_ast2600 can go over:
>> 13/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_ast2600
>>                        OK 792.94s
>> 27/44 qemu:func-thorough+func-arm-thorough+thorough /
>> func-arm-arm_aspeed_ast2500
>>                   TIMEOUT 480.01s
>> The sx1 test fails not on the overall meson timeout but on the
>> 60 second timeout in some of the subtests.
>> Bump all these timeouts up a bit.
>> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>> ---
>> This at least gets 'make -j4 check-functional' to passing for me
>> for an arm/aarch64 debug build, apart from the gpu test hang which
>> we're discussing in a different thread. Whole thing takes 25 mins...
>> ---
>>   tests/functional/meson.build     | 8 ++++----
>>   tests/functional/test_arm_sx1.py | 6 +++---
>>   2 files changed, 7 insertions(+), 7 deletions(-)
>> diff --git a/tests/functional/meson.build
>> b/tests/functional/meson.build
>> index b516d21cba1..effa31701cf 100644
>> --- a/tests/functional/meson.build
>> +++ b/tests/functional/meson.build
>> @@ -15,16 +15,16 @@ test_timeouts = {
>>     'aarch64_raspi4' : 480,
>>     'aarch64_rme_virt' : 1200,
>>     'aarch64_rme_sbsaref' : 1200,
>> -  'aarch64_sbsaref_alpine' : 720,
>> +  'aarch64_sbsaref_alpine' : 1200,
>
> I wonder whether we should disable that test with @skipSlowTest() by
> default, since it's really very slow...?

The pauth subtest is marked as @skipSlowTest() - but I think we can drop
test_sbsaref_alpine_linux_cortex_a57 and
test_sbsaref_alpine_linux_max_pauth_off because we don't actually verify
anything so its wasted cycles booting something that we know already
boots.

>
> Anyway, for this patch here:
> Reviewed-by: Thomas Huth <thuth@redhat.com>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro