so we keep making timeouts longer for CI but one has to stop
somewhere. netdev socket test recently failed for me again
even though it's at 2 minutes already.
here's an experiment for netdev-socket.
if this works well for a while we can generalize to other
tests.

Michael S. Tsirkin (2):
  osdep: add getloadavg
  netdev: set timeout depending on loadavg

 include/qemu/osdep.h        | 10 ++++++++++
 tests/qtest/netdev-socket.c | 28 +++++++++++++++++++++++++++-
 meson.build                 |  1 +
 3 files changed, 38 insertions(+), 1 deletion(-)

-- 
MST

Hi Michael,

On 7/11/23 10:49, Michael S. Tsirkin wrote:
> so we keep making timeouts longer for CI but one has to stop
> somewhere. netdev socket test recently failed for me again
> even though it's at 2 minutes already.
> here's an experiment for netdev-socket.
> if this works well for a while we can generalize to other
> tests.
> 
> Michael S. Tsirkin (2):
>    osdep: add getloadavg
>    netdev: set timeout depending on loadavg

The 2 problems I see with this approach are:

- Machine load can vary *after* the test is started,
- Test environment isn't really reproducible.

Are those netdev tests only failing due to high load?

Maybe we need to disable them on CI and run them manually...

Regards,

Phil.

On Tue, Nov 07, 2023 at 10:57:23AM +0100, Philippe Mathieu-Daudé wrote:
> Hi Michael,
> 
> On 7/11/23 10:49, Michael S. Tsirkin wrote:
> > so we keep making timeouts longer for CI but one has to stop
> > somewhere. netdev socket test recently failed for me again
> > even though it's at 2 minutes already.
> > here's an experiment for netdev-socket.
> > if this works well for a while we can generalize to other
> > tests.
> > 
> > Michael S. Tsirkin (2):
> >    osdep: add getloadavg
> >    netdev: set timeout depending on loadavg
> 
> The 2 problems I see with this approach are:
> 
> - Machine load can vary *after* the test is started,

Then it won't help. But better than increasing timeout
all the time, no?

> - Test environment isn't really reproducible.

But then load intrinsically is.

> Are those netdev tests only failing due to high load?

yep

> Maybe we need to disable them on CI and run them manually...
> 
> Regards,
> 
> Phil.

Most of our tests are like this.

-- 
MST