[v3] Acceptance Tests: target architecture support

[Qemu-devel] [PATCH v3 11/20] Boot Linux Console Test: increase timeout

Posted by Cleber Rosa 6 years, 8 months ago

When running on very low powered environments, some tests may time out
causing false negatives.  As a conservative change, and for
considering that human time (investigating false negatives) is worth
more than some extra machine cycles (and time), let's increase the
overall timeout.

CC: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/acceptance/boot_linux_console.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index cc5dcd7373..fa721a7355 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -21,7 +21,7 @@ class BootLinuxConsole(Test):
     :avocado: enable
     """
 
-    timeout = 60
+    timeout = 90
 
     KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
 
-- 
2.20.1

Re: [Qemu-devel] [PATCH v3 11/20] Boot Linux Console Test: increase timeout

Posted by Cornelia Huck 6 years, 8 months ago

On Wed, 20 Feb 2019 19:57:44 -0500
Cleber Rosa <crosa@redhat.com> wrote:

> When running on very low powered environments, some tests may time out
> causing false negatives.  As a conservative change, and for
> considering that human time (investigating false negatives) is worth
> more than some extra machine cycles (and time), let's increase the
> overall timeout.
> 
> CC: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index cc5dcd7373..fa721a7355 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -21,7 +21,7 @@ class BootLinuxConsole(Test):
>      :avocado: enable
>      """
>  
> -    timeout = 60
> +    timeout = 90
>  
>      KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>  

Not sure what a good value is, but it is easy to tweak later if needed.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>

Re: [Qemu-devel] [PATCH v3 11/20] Boot Linux Console Test: increase timeout

Posted by Philippe Mathieu-Daudé 6 years, 8 months ago

On 2/21/19 1:57 AM, Cleber Rosa wrote:
> When running on very low powered environments, some tests may time out
> causing false negatives.  As a conservative change, and for
> considering that human time (investigating false negatives) is worth
> more than some extra machine cycles (and time), let's increase the
> overall timeout.

However this is annoying when a test is stuck and you work on battery...

> 
> CC: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index cc5dcd7373..fa721a7355 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -21,7 +21,7 @@ class BootLinuxConsole(Test):
>      :avocado: enable
>      """
>  
> -    timeout = 60
> +    timeout = 90
>  
>      KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>  
>

Re: [Qemu-devel] [PATCH v3 11/20] Boot Linux Console Test: increase timeout

Posted by Cleber Rosa 6 years, 8 months ago

On Sat, Mar 02, 2019 at 10:43:52PM +0100, Philippe Mathieu-Daudé wrote:
> On 2/21/19 1:57 AM, Cleber Rosa wrote:
> > When running on very low powered environments, some tests may time out
> > causing false negatives.  As a conservative change, and for
> > considering that human time (investigating false negatives) is worth
> > more than some extra machine cycles (and time), let's increase the
> > overall timeout.
> 
> However this is annoying when a test is stuck and you work on battery...
>

I agree, but at this point, the more we try to fix all use cases, the
furthest we get to something reasonable IMO.

Any ideas on how to do better here while still keeping things simple?

- Cleber.

> > 
> > CC: Alex Bennée <alex.bennee@linaro.org>
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> >  tests/acceptance/boot_linux_console.py | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> > index cc5dcd7373..fa721a7355 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -21,7 +21,7 @@ class BootLinuxConsole(Test):
> >      :avocado: enable
> >      """
> >  
> > -    timeout = 60
> > +    timeout = 90
> >  
> >      KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
> >  
> >