diff mbox series

[v3,11/20] Boot Linux Console Test: increase timeout

Message ID 20190221005753.27955-12-crosa@redhat.com
State New
Headers show
Series Acceptance Tests: target architecture support | expand

Commit Message

Cleber Rosa Feb. 21, 2019, 12:57 a.m. UTC
When running on very low powered environments, some tests may time out
causing false negatives.  As a conservative change, and for
considering that human time (investigating false negatives) is worth
more than some extra machine cycles (and time), let's increase the
overall timeout.

CC: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/acceptance/boot_linux_console.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Cornelia Huck March 1, 2019, 10:45 a.m. UTC | #1
On Wed, 20 Feb 2019 19:57:44 -0500
Cleber Rosa <crosa@redhat.com> wrote:

> When running on very low powered environments, some tests may time out
> causing false negatives.  As a conservative change, and for
> considering that human time (investigating false negatives) is worth
> more than some extra machine cycles (and time), let's increase the
> overall timeout.
> 
> CC: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index cc5dcd7373..fa721a7355 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -21,7 +21,7 @@ class BootLinuxConsole(Test):
>      :avocado: enable
>      """
>  
> -    timeout = 60
> +    timeout = 90
>  
>      KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>  

Not sure what a good value is, but it is easy to tweak later if needed.

Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Philippe Mathieu-Daudé March 2, 2019, 9:43 p.m. UTC | #2
On 2/21/19 1:57 AM, Cleber Rosa wrote:
> When running on very low powered environments, some tests may time out
> causing false negatives.  As a conservative change, and for
> considering that human time (investigating false negatives) is worth
> more than some extra machine cycles (and time), let's increase the
> overall timeout.

However this is annoying when a test is stuck and you work on battery...

> 
> CC: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  tests/acceptance/boot_linux_console.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index cc5dcd7373..fa721a7355 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -21,7 +21,7 @@ class BootLinuxConsole(Test):
>      :avocado: enable
>      """
>  
> -    timeout = 60
> +    timeout = 90
>  
>      KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
>  
>
Cleber Rosa March 12, 2019, 12:04 a.m. UTC | #3
On Sat, Mar 02, 2019 at 10:43:52PM +0100, Philippe Mathieu-Daudé wrote:
> On 2/21/19 1:57 AM, Cleber Rosa wrote:
> > When running on very low powered environments, some tests may time out
> > causing false negatives.  As a conservative change, and for
> > considering that human time (investigating false negatives) is worth
> > more than some extra machine cycles (and time), let's increase the
> > overall timeout.
> 
> However this is annoying when a test is stuck and you work on battery...
>

I agree, but at this point, the more we try to fix all use cases, the
furthest we get to something reasonable IMO.

Any ideas on how to do better here while still keeping things simple?

- Cleber.

> > 
> > CC: Alex Bennée <alex.bennee@linaro.org>
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> >  tests/acceptance/boot_linux_console.py | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> > index cc5dcd7373..fa721a7355 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -21,7 +21,7 @@ class BootLinuxConsole(Test):
> >      :avocado: enable
> >      """
> >  
> > -    timeout = 60
> > +    timeout = 90
> >  
> >      KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
> >  
> >
diff mbox series

Patch

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index cc5dcd7373..fa721a7355 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -21,7 +21,7 @@  class BootLinuxConsole(Test):
     :avocado: enable
     """
 
-    timeout = 60
+    timeout = 90
 
     KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '