[v3] tests/acceptance: Disable Spartan-3A DSP 1800A test

[PATCH-for-5.2 v3] tests/acceptance: Disable Spartan-3A DSP 1800A test

Posted by Philippe Mathieu-Daudé 3 years, 5 months ago

This test is regularly failing on CI:

   (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
  Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
  ...
  Freeing unused kernel memory: 1444K
  This architecture does not have kernel memory protection.
  [nothing happens here]
  Runner error occurred: Timeout reached (90.91 s)

This is a regression. Until someone figure out the problem,
disable the test to keep CI pipeline useful.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
v3: Simply disable with @skip
---
 tests/acceptance/boot_linux_console.py | 2 ++
 tests/acceptance/replay_kernel.py      | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 8f433a67f84..cc6ec0f8c15 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -13,6 +13,7 @@
 import gzip
 import shutil
 
+from avocado import skip
 from avocado import skipUnless
 from avocado_qemu import Test
 from avocado_qemu import exec_command_and_wait_for_pattern
@@ -1025,6 +1026,7 @@ def test_m68k_mcf5208evb(self):
         tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
         self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
 
+    @skip("Test currently broken") # Console stuck as of 5.2-rc1
     def test_microblaze_s3adsp1800(self):
         """
         :avocado: tags=arch:microblaze
diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index 00c228382bd..772633b01da 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -14,6 +14,7 @@
 import logging
 import time
 
+from avocado import skip
 from avocado import skipIf
 from avocado import skipUnless
 from avocado_qemu import wait_for_console_pattern
@@ -280,6 +281,7 @@ def test_m68k_mcf5208evb(self):
         file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
         self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
 
+    @skip("Test currently broken") # Console stuck as of 5.2-rc1
     def test_microblaze_s3adsp1800(self):
         """
         :avocado: tags=arch:microblaze
-- 
2.26.2

Re: [PATCH-for-5.2 v3] tests/acceptance: Disable Spartan-3A DSP 1800A test

Posted by Thomas Huth 3 years, 5 months ago

On 09/11/2020 10.17, Philippe Mathieu-Daudé wrote:
> This test is regularly failing on CI:
> 
>    (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
>   Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
>   ...
>   Freeing unused kernel memory: 1444K
>   This architecture does not have kernel memory protection.
>   [nothing happens here]
>   Runner error occurred: Timeout reached (90.91 s)
> 
> This is a regression. Until someone figure out the problem,
> disable the test to keep CI pipeline useful.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> v3: Simply disable with @skip
> ---
>  tests/acceptance/boot_linux_console.py | 2 ++
>  tests/acceptance/replay_kernel.py      | 2 ++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 8f433a67f84..cc6ec0f8c15 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -13,6 +13,7 @@
>  import gzip
>  import shutil
>  
> +from avocado import skip
>  from avocado import skipUnless
>  from avocado_qemu import Test
>  from avocado_qemu import exec_command_and_wait_for_pattern
> @@ -1025,6 +1026,7 @@ def test_m68k_mcf5208evb(self):
>          tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>          self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index 00c228382bd..772633b01da 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -14,6 +14,7 @@
>  import logging
>  import time
>  
> +from avocado import skip
>  from avocado import skipIf
>  from avocado import skipUnless
>  from avocado_qemu import wait_for_console_pattern
> @@ -280,6 +281,7 @@ def test_m68k_mcf5208evb(self):
>          file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
>          self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> 

Acked-by: Thomas Huth <thuth@redhat.com>

FWIW, I tried to bisect this issue today to see where it broke, but if I run
the test often enough, I even got failures with QEMU v5.0 already, so it's
likely not a new issue after all:

make check-venv
for ((i=0;i<500;i++)); do echo $i ; \
 AVOCADO_ALLOW_UNTRUSTED_CODE=1 tests/venv/bin/avocado run \
 tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800 \
 || break ; sleep 0.1 ; \
done

Re: [PATCH-for-5.2 v3] tests/acceptance: Disable Spartan-3A DSP 1800A test

Posted by Alex Bennée 3 years, 5 months ago

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> This test is regularly failing on CI:
>
>    (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
>   Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
>   ...
>   Freeing unused kernel memory: 1444K
>   This architecture does not have kernel memory protection.
>   [nothing happens here]
>   Runner error occurred: Timeout reached (90.91 s)
>
> This is a regression. Until someone figure out the problem,
> disable the test to keep CI pipeline useful.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Queued to for-5.2/various-fixes, thanks.

> ---
> v3: Simply disable with @skip
> ---
>  tests/acceptance/boot_linux_console.py | 2 ++
>  tests/acceptance/replay_kernel.py      | 2 ++
>  2 files changed, 4 insertions(+)
>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 8f433a67f84..cc6ec0f8c15 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -13,6 +13,7 @@
>  import gzip
>  import shutil
>  
> +from avocado import skip
>  from avocado import skipUnless
>  from avocado_qemu import Test
>  from avocado_qemu import exec_command_and_wait_for_pattern
> @@ -1025,6 +1026,7 @@ def test_m68k_mcf5208evb(self):
>          tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>          self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index 00c228382bd..772633b01da 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -14,6 +14,7 @@
>  import logging
>  import time
>  
> +from avocado import skip
>  from avocado import skipIf
>  from avocado import skipUnless
>  from avocado_qemu import wait_for_console_pattern
> @@ -280,6 +281,7 @@ def test_m68k_mcf5208evb(self):
>          file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
>          self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze


-- 
Alex Bennée