[v1] Test more in less time in the Gitlab-CI

[PATCH 3/4] tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab

Posted by Thomas Huth 5 years, 6 months ago

These tests always time out on Gitlab, not sure what's happening here.
Let's disable them until somebody has enough spare time to debug the
issues.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/acceptance/machine_rx_gdbsim.py | 4 ++++
 tests/acceptance/replay_kernel.py     | 1 +
 2 files changed, 5 insertions(+)

diff --git a/tests/acceptance/machine_rx_gdbsim.py b/tests/acceptance/machine_rx_gdbsim.py
index bff63e421d..0c72506028 100644
--- a/tests/acceptance/machine_rx_gdbsim.py
+++ b/tests/acceptance/machine_rx_gdbsim.py
@@ -8,6 +8,9 @@
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
 
+import os
+
+from avocado import skipIf
 from avocado_qemu import Test
 from avocado_qemu import exec_command_and_wait_for_pattern
 from avocado_qemu import wait_for_console_pattern
@@ -42,6 +45,7 @@ class RxGdbSimMachine(Test):
         # FIXME limit baudrate on chardev, else we type too fast
         #exec_command_and_wait_for_pattern(self, 'version', gcc_version)
 
+    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
     def test_linux_sash(self):
         """
         Boots a Linux kernel and checks that the console is operational.
diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index 62d2db8c64..b79fc8daf8 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -126,6 +126,7 @@ class ReplayKernel(LinuxKernelTest):
 
         self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
 
+    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
-- 
2.18.1

Re: [PATCH 3/4] tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab

Posted by Wainer dos Santos Moschetta 5 years, 6 months ago

On 7/23/20 9:27 AM, Thomas Huth wrote:
> These tests always time out on Gitlab, not sure what's happening here.
> Let's disable them until somebody has enough spare time to debug the
> issues.

It's fair to me.

>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/acceptance/machine_rx_gdbsim.py | 4 ++++
>   tests/acceptance/replay_kernel.py     | 1 +
>   2 files changed, 5 insertions(+)


Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
> diff --git a/tests/acceptance/machine_rx_gdbsim.py b/tests/acceptance/machine_rx_gdbsim.py
> index bff63e421d..0c72506028 100644
> --- a/tests/acceptance/machine_rx_gdbsim.py
> +++ b/tests/acceptance/machine_rx_gdbsim.py
> @@ -8,6 +8,9 @@
>   # This work is licensed under the terms of the GNU GPL, version 2 or
>   # later.  See the COPYING file in the top-level directory.
>   
> +import os
> +
> +from avocado import skipIf
>   from avocado_qemu import Test
>   from avocado_qemu import exec_command_and_wait_for_pattern
>   from avocado_qemu import wait_for_console_pattern
> @@ -42,6 +45,7 @@ class RxGdbSimMachine(Test):
>           # FIXME limit baudrate on chardev, else we type too fast
>           #exec_command_and_wait_for_pattern(self, 'version', gcc_version)
>   
> +    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
>       def test_linux_sash(self):
>           """
>           Boots a Linux kernel and checks that the console is operational.
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index 62d2db8c64..b79fc8daf8 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -126,6 +126,7 @@ class ReplayKernel(LinuxKernelTest):
>   
>           self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
>   
> +    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
>       def test_arm_cubieboard_initrd(self):
>           """
>           :avocado: tags=arch:arm

Re: [PATCH 3/4] tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab

Posted by Philippe Mathieu-Daudé 5 years, 6 months ago

On 7/23/20 2:27 PM, Thomas Huth wrote:
> These tests always time out on Gitlab, not sure what's happening here.
> Let's disable them until somebody has enough spare time to debug the
> issues.

Is the Avocado cache working? Is it failing with an empty cache?
If so, maybe we need to run avocado cache fetching in a previous
step, before the testing job.

Can you share a url of failed job so I can have a look?

> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/acceptance/machine_rx_gdbsim.py | 4 ++++
>  tests/acceptance/replay_kernel.py     | 1 +
>  2 files changed, 5 insertions(+)
> 
> diff --git a/tests/acceptance/machine_rx_gdbsim.py b/tests/acceptance/machine_rx_gdbsim.py
> index bff63e421d..0c72506028 100644
> --- a/tests/acceptance/machine_rx_gdbsim.py
> +++ b/tests/acceptance/machine_rx_gdbsim.py
> @@ -8,6 +8,9 @@
>  # This work is licensed under the terms of the GNU GPL, version 2 or
>  # later.  See the COPYING file in the top-level directory.
>  
> +import os
> +
> +from avocado import skipIf
>  from avocado_qemu import Test
>  from avocado_qemu import exec_command_and_wait_for_pattern
>  from avocado_qemu import wait_for_console_pattern
> @@ -42,6 +45,7 @@ class RxGdbSimMachine(Test):
>          # FIXME limit baudrate on chardev, else we type too fast
>          #exec_command_and_wait_for_pattern(self, 'version', gcc_version)
>  
> +    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
>      def test_linux_sash(self):
>          """
>          Boots a Linux kernel and checks that the console is operational.
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index 62d2db8c64..b79fc8daf8 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -126,6 +126,7 @@ class ReplayKernel(LinuxKernelTest):
>  
>          self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
>  
> +    @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
>      def test_arm_cubieboard_initrd(self):
>          """
>          :avocado: tags=arch:arm
>

Re: [PATCH 3/4] tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab

Posted by Thomas Huth 5 years, 6 months ago

On 23/07/2020 14.53, Philippe Mathieu-Daudé wrote:
> On 7/23/20 2:27 PM, Thomas Huth wrote:
>> These tests always time out on Gitlab, not sure what's happening here.
>> Let's disable them until somebody has enough spare time to debug the
>> issues.
> 
> Is the Avocado cache working? Is it failing with an empty cache?
> If so, maybe we need to run avocado cache fetching in a previous
> step, before the testing job.
> 
> Can you share a url of failed job so I can have a look?

CentOS:
 https://gitlab.com/huth/qemu/-/jobs/651179303

Debian:
 https://gitlab.com/huth/qemu/-/jobs/651179296

... might be a cache issue, indeed.

 Thomas

Re: [PATCH 3/4] tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab

Posted by Cleber Rosa 5 years, 6 months ago

----- Thomas Huth <thuth@redhat.com> wrote:
> On 23/07/2020 14.53, Philippe Mathieu-Daudé wrote:
> > On 7/23/20 2:27 PM, Thomas Huth wrote:
> >> These tests always time out on Gitlab, not sure what's happening here.
> >> Let's disable them until somebody has enough spare time to debug the
> >> issues.
> > 
> > Is the Avocado cache working? Is it failing with an empty cache?
> > If so, maybe we need to run avocado cache fetching in a previous
> > step, before the testing job.
> > 
> > Can you share a url of failed job so I can have a look?
> 
> CentOS:
>  https://gitlab.com/huth/qemu/-/jobs/651179303
> 
I don't see the sash files being downloaded/checked ahead of test execution, so that may be an issue indeed. But for the cubieboard, that is being done, so does not look like a cache issue.

Either way, it's better to disable these and re-enable them after a proper investigation (I'll follow up with that).

Reviewed-by: Cleber Rosa <crosa@redhat.com>


> Debian:
>  https://gitlab.com/huth/qemu/-/jobs/651179296
> 
> ... might be a cache issue, indeed.
> 
>  Thomas