1. Patchew
  2. QEMU
  3. View series

[PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com

Philippe Mathieu-Daudé posted 1 patch 3 years, 7 months ago
Test docker-quick@centos7 failed
Test docker-mingw@fedora failed
Test checkpatch failed
Test FreeBSD failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200917163954.50514-1-philmd@redhat.com
tests/acceptance/boot_linux_console.py | 10 ++++++++++
tests/acceptance/replay_kernel.py      |  2 ++
2 files changed, 12 insertions(+)
[PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago 
Armbian servers are not very reliable and confused the GitLab CI
users a few times this month (path updated, archives moved, and
now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add
them to the cache).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/boot_linux_console.py | 10 ++++++++++
 tests/acceptance/replay_kernel.py      |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 4a366ce93e4..68534693109 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -485,6 +485,8 @@ def test_arm_exynos4210_initrd(self):
         self.wait_for_console_pattern('Boot successful.')
         # TODO user command, for now the uart is stuck
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
@@ -525,6 +527,8 @@ def test_arm_cubieboard_initrd(self):
                                                 'system-control@1c00000')
         # cubieboard's reboot is not functioning; omit reboot test.
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_sata(self):
         """
         :avocado: tags=arch:arm
@@ -651,6 +655,8 @@ def test_arm_quanta_gsj_initrd(self):
         self.wait_for_console_pattern(
                 'Give root password for system maintenance')
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi(self):
         """
         :avocado: tags=arch:arm
@@ -676,6 +682,8 @@ def test_arm_orangepi(self):
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi_initrd(self):
         """
         :avocado: tags=arch:arm
@@ -718,6 +726,8 @@ def test_arm_orangepi_initrd(self):
         # Wait for VM to shut down gracefully
         self.vm.wait()
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi_sd(self):
         """
         :avocado: tags=arch:arm
diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index b79fc8daf8e..7b4310fccf3 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -127,6 +127,8 @@ def test_arm_virt(self):
         self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
 
     @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
-- 
2.26.2
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by no-reply@patchew.org 3 years, 7 months ago 
Patchew URL: https://patchew.org/QEMU/20200917163954.50514-1-philmd@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

N/A. Internal error while reading log file



The full log is available at
http://patchew.org/logs/20200917163954.50514-1-philmd@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by Thomas Huth 3 years, 7 months ago 
On 17/09/2020 18.39, Philippe Mathieu-Daudé wrote:
> Armbian servers are not very reliable and confused the GitLab CI
> users a few times this month (path updated, archives moved, and
> now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
> error). Time to disable these tests.
> Users can still use the artifacts from the cache (or manually add
> them to the cache).
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 10 ++++++++++
>  tests/acceptance/replay_kernel.py      |  2 ++
>  2 files changed, 12 insertions(+)

These failures in the Gitlab-CI are quite annoying, indeed.

Acked-by: Thomas Huth <thuth@redhat.com>
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago 
On 9/17/20 6:53 PM, Thomas Huth wrote:
> On 17/09/2020 18.39, Philippe Mathieu-Daudé wrote:
>> Armbian servers are not very reliable and confused the GitLab CI
>> users a few times this month (path updated, archives moved, and
>> now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
>> error). Time to disable these tests.
>> Users can still use the artifacts from the cache (or manually add
>> them to the cache).
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>>  tests/acceptance/boot_linux_console.py | 10 ++++++++++
>>  tests/acceptance/replay_kernel.py      |  2 ++
>>  2 files changed, 12 insertions(+)
> 
> These failures in the Gitlab-CI are quite annoying, indeed.

I'd rather not go via this path, but since we don't save the
artifacts cache, I can't find a better way to not block all
the other developers using GitLab CI.

But I prefer we disable the tests until something better
emerge, rather that having the developers stopping running
these tests. So this is a quick band-aid patch.

> 
> Acked-by: Thomas Huth <thuth@redhat.com>
> 

Thanks!

Phil.
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by Gerd Hoffmann 3 years, 7 months ago 
On Thu, Sep 17, 2020 at 07:01:22PM +0200, Philippe Mathieu-Daudé wrote:
> On 9/17/20 6:53 PM, Thomas Huth wrote:
> > On 17/09/2020 18.39, Philippe Mathieu-Daudé wrote:
> >> Armbian servers are not very reliable and confused the GitLab CI
> >> users a few times this month (path updated, archives moved, and
> >> now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
> >> error). Time to disable these tests.
> >> Users can still use the artifacts from the cache (or manually add
> >> them to the cache).
> >>
> >> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> >> ---
> >>  tests/acceptance/boot_linux_console.py | 10 ++++++++++
> >>  tests/acceptance/replay_kernel.py      |  2 ++
> >>  2 files changed, 12 insertions(+)
> > 
> > These failures in the Gitlab-CI are quite annoying, indeed.
> 
> I'd rather not go via this path, but since we don't save the
> artifacts cache, I can't find a better way to not block all
> the other developers using GitLab CI.

Well, we would need to cache them somewhere, either on qemu.org,
or maybe have a download job in ci which publishes them on
qemu.gitlab.io (gitlab pages).

> But I prefer we disable the tests until something better
> emerge, rather that having the developers stopping running
> these tests. So this is a quick band-aid patch.

Yep, lets disable them until we have a solution.

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

take care,
  Gerd
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by no-reply@patchew.org 3 years, 7 months ago 
Patchew URL: https://patchew.org/QEMU/20200917163954.50514-1-philmd@redhat.com/



Hi,

This series failed build test on FreeBSD host. Please find the details below.






The full log is available at
http://patchew.org/logs/20200917163954.50514-1-philmd@redhat.com/testing.FreeBSD/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago 
On 9/17/20 6:39 PM, Philippe Mathieu-Daudé wrote:
> Armbian servers are not very reliable and confused the GitLab CI
> users a few times this month (path updated, archives moved, and
> now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
> error). Time to disable these tests.
> Users can still use the artifacts from the cache (or manually add
> them to the cache).
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
[...]
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index b79fc8daf8e..7b4310fccf3 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -127,6 +127,8 @@ def test_arm_virt(self):
>          self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
>  
>      @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
> +                'Test artifacts fetched from unreliable apt.armbian.com')
>      def test_arm_cubieboard_initrd(self):
>          """
>          :avocado: tags=arch:arm
> 

Oops for this one I forgot the "from avocado import skipUnless" line.
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by Philippe Mathieu-Daudé 3 years, 7 months ago 
On 9/17/20 6:39 PM, Philippe Mathieu-Daudé wrote:
> Armbian servers are not very reliable and confused the GitLab CI
> users a few times this month (path updated, archives moved, and
> now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
> error). Time to disable these tests.
> Users can still use the artifacts from the cache (or manually add
> them to the cache).
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 10 ++++++++++
>  tests/acceptance/replay_kernel.py      |  2 ++
>  2 files changed, 12 insertions(+)

Thanks, applied to my acceptance-testing tree.
Re: [PATCH] tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Posted by Willian Rampazzo 3 years, 7 months ago 
On Thu, Sep 17, 2020 at 1:41 PM Philippe Mathieu-Daudé
<philmd@redhat.com> wrote:
>
> Armbian servers are not very reliable and confused the GitLab CI
> users a few times this month (path updated, archives moved, and
> now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
> error). Time to disable these tests.
> Users can still use the artifacts from the cache (or manually add
> them to the cache).
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 10 ++++++++++
>  tests/acceptance/replay_kernel.py      |  2 ++
>  2 files changed, 12 insertions(+)
>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 4a366ce93e4..68534693109 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -485,6 +485,8 @@ def test_arm_exynos4210_initrd(self):
>          self.wait_for_console_pattern('Boot successful.')
>          # TODO user command, for now the uart is stuck
>
> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
> +                'Test artifacts fetched from unreliable apt.armbian.com')
>      def test_arm_cubieboard_initrd(self):
>          """
>          :avocado: tags=arch:arm
> @@ -525,6 +527,8 @@ def test_arm_cubieboard_initrd(self):
>                                                  'system-control@1c00000')
>          # cubieboard's reboot is not functioning; omit reboot test.
>
> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
> +                'Test artifacts fetched from unreliable apt.armbian.com')
>      def test_arm_cubieboard_sata(self):
>          """
>          :avocado: tags=arch:arm
> @@ -651,6 +655,8 @@ def test_arm_quanta_gsj_initrd(self):
>          self.wait_for_console_pattern(
>                  'Give root password for system maintenance')
>
> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
> +                'Test artifacts fetched from unreliable apt.armbian.com')
>      def test_arm_orangepi(self):
>          """
>          :avocado: tags=arch:arm
> @@ -676,6 +682,8 @@ def test_arm_orangepi(self):
>          console_pattern = 'Kernel command line: %s' % kernel_command_line
>          self.wait_for_console_pattern(console_pattern)
>
> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
> +                'Test artifacts fetched from unreliable apt.armbian.com')
>      def test_arm_orangepi_initrd(self):
>          """
>          :avocado: tags=arch:arm
> @@ -718,6 +726,8 @@ def test_arm_orangepi_initrd(self):
>          # Wait for VM to shut down gracefully
>          self.vm.wait()
>
> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
> +                'Test artifacts fetched from unreliable apt.armbian.com')
>      def test_arm_orangepi_sd(self):
>          """
>          :avocado: tags=arch:arm
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index b79fc8daf8e..7b4310fccf3 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -127,6 +127,8 @@ def test_arm_virt(self):
>          self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
>
>      @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
> +    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
> +                'Test artifacts fetched from unreliable apt.armbian.com')
>      def test_arm_cubieboard_initrd(self):
>          """
>          :avocado: tags=arch:arm
> --
> 2.26.2
>
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.