[v1] travis-ci: Add a KVM-only s390x job

[PATCH 3/4] tests/migration: Disable autoconverge test on Travis-CI s390x

Posted by Philippe Mathieu-Daudé 5 years, 8 months ago

This test fails on Travis-CI s390x when configured with --disable-tcg:

  $ make check-qtest
    TEST    check-qtest-s390x: tests/qtest/boot-serial-test
  qemu-system-s390x: -accel tcg: invalid accelerator tcg
  qemu-system-s390x: falling back to KVM
    TEST    check-qtest-s390x: tests/qtest/pxe-test
    TEST    check-qtest-s390x: tests/qtest/test-netfilter
    TEST    check-qtest-s390x: tests/qtest/test-filter-mirror
    TEST    check-qtest-s390x: tests/qtest/test-filter-redirector
    TEST    check-qtest-s390x: tests/qtest/drive_del-test
    TEST    check-qtest-s390x: tests/qtest/device-plug-test
    TEST    check-qtest-s390x: tests/qtest/virtio-ccw-test
    TEST    check-qtest-s390x: tests/qtest/cpu-plug-test
    TEST    check-qtest-s390x: tests/qtest/migration-test
  **
  ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
  ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
  make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1

Per David Gilbert, it might be a threshold requiring tuning:
"it could just be the writing is slow on s390 and the migration
thread fast; in which case the autocomplete wouldn't be needed.
Perhaps we just need to reduce the bandwidth limit."

Disable the autoconverge test for now.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/qtest/migration-test.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 3d6cc83b88..878399666e 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -1181,6 +1181,16 @@ static void test_migrate_auto_converge(void)
     MigrateStart *args = migrate_start_new();
     QTestState *from, *to;
     int64_t remaining, percentage;
+    char *travis_arch;
+
+    /*
+     * This test does not work reliably on Travis s390x.
+     */
+    travis_arch = getenv("TRAVIS_CPU_ARCH");
+    if (travis_arch && g_str_equal(travis_arch, "s390x")) {
+        g_test_skip("Test does not work reliably on s390x Travis containers.");
+        return;
+    }
 
     /*
      * We want the test to be stable and as fast as possible.
-- 
2.21.1

Re: [PATCH 3/4] tests/migration: Disable autoconverge test on Travis-CI s390x

Posted by Dr. David Alan Gilbert 5 years, 8 months ago

* Philippe Mathieu-Daudé (philmd@redhat.com) wrote:
> This test fails on Travis-CI s390x when configured with --disable-tcg:
> 
>   $ make check-qtest
>     TEST    check-qtest-s390x: tests/qtest/boot-serial-test
>   qemu-system-s390x: -accel tcg: invalid accelerator tcg
>   qemu-system-s390x: falling back to KVM
>     TEST    check-qtest-s390x: tests/qtest/pxe-test
>     TEST    check-qtest-s390x: tests/qtest/test-netfilter
>     TEST    check-qtest-s390x: tests/qtest/test-filter-mirror
>     TEST    check-qtest-s390x: tests/qtest/test-filter-redirector
>     TEST    check-qtest-s390x: tests/qtest/drive_del-test
>     TEST    check-qtest-s390x: tests/qtest/device-plug-test
>     TEST    check-qtest-s390x: tests/qtest/virtio-ccw-test
>     TEST    check-qtest-s390x: tests/qtest/cpu-plug-test
>     TEST    check-qtest-s390x: tests/qtest/migration-test
>   **
>   ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
>   ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
>   make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1

Before we take the hammer to it, could you try reducing it's initial
bandwidth, the code that says:

    /*
     * Set the initial parameters so that the migration could not converge
     * without throttling.
     */
    migrate_set_parameter_int(from, "downtime-limit", 1);
    migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */

knock a couple of 0's off that and see if it's happy.

Dave

> Per David Gilbert, it might be a threshold requiring tuning:
> "it could just be the writing is slow on s390 and the migration
> thread fast; in which case the autocomplete wouldn't be needed.
> Perhaps we just need to reduce the bandwidth limit."
> 
> Disable the autoconverge test for now.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/qtest/migration-test.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> index 3d6cc83b88..878399666e 100644
> --- a/tests/qtest/migration-test.c
> +++ b/tests/qtest/migration-test.c
> @@ -1181,6 +1181,16 @@ static void test_migrate_auto_converge(void)
>      MigrateStart *args = migrate_start_new();
>      QTestState *from, *to;
>      int64_t remaining, percentage;
> +    char *travis_arch;
> +
> +    /*
> +     * This test does not work reliably on Travis s390x.
> +     */
> +    travis_arch = getenv("TRAVIS_CPU_ARCH");
> +    if (travis_arch && g_str_equal(travis_arch, "s390x")) {
> +        g_test_skip("Test does not work reliably on s390x Travis containers.");
> +        return;
> +    }
>  
>      /*
>       * We want the test to be stable and as fast as possible.
> -- 
> 2.21.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH 3/4] tests/migration: Disable autoconverge test on Travis-CI s390x

Posted by Philippe Mathieu-Daudé 5 years, 8 months ago

On 3/17/20 9:26 PM, Dr. David Alan Gilbert wrote:
> * Philippe Mathieu-Daudé (philmd@redhat.com) wrote:
>> This test fails on Travis-CI s390x when configured with --disable-tcg:
>>
>>    $ make check-qtest
>>      TEST    check-qtest-s390x: tests/qtest/boot-serial-test
>>    qemu-system-s390x: -accel tcg: invalid accelerator tcg
>>    qemu-system-s390x: falling back to KVM
>>      TEST    check-qtest-s390x: tests/qtest/pxe-test
>>      TEST    check-qtest-s390x: tests/qtest/test-netfilter
>>      TEST    check-qtest-s390x: tests/qtest/test-filter-mirror
>>      TEST    check-qtest-s390x: tests/qtest/test-filter-redirector
>>      TEST    check-qtest-s390x: tests/qtest/drive_del-test
>>      TEST    check-qtest-s390x: tests/qtest/device-plug-test
>>      TEST    check-qtest-s390x: tests/qtest/virtio-ccw-test
>>      TEST    check-qtest-s390x: tests/qtest/cpu-plug-test
>>      TEST    check-qtest-s390x: tests/qtest/migration-test
>>    **
>>    ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
>>    ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
>>    make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1
> 
> Before we take the hammer to it, could you try reducing it's initial
> bandwidth, the code that says:
> 
>      /*
>       * Set the initial parameters so that the migration could not converge
>       * without throttling.
>       */
>      migrate_set_parameter_int(from, "downtime-limit", 1);
>      migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */
> 
> knock a couple of 0's off that and see if it's happy.

You are right! One 0 is not enough, but a couple makes the test succeed.

Can that change have other (bad) effects on other archs?
IOW can I send a generic patch updating this value, or should I add a 
check on the architecture?

Thanks!

> 
> Dave
> 
>> Per David Gilbert, it might be a threshold requiring tuning:
>> "it could just be the writing is slow on s390 and the migration
>> thread fast; in which case the autocomplete wouldn't be needed.
>> Perhaps we just need to reduce the bandwidth limit."
>>
>> Disable the autoconverge test for now.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>>   tests/qtest/migration-test.c | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>>
>> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
>> index 3d6cc83b88..878399666e 100644
>> --- a/tests/qtest/migration-test.c
>> +++ b/tests/qtest/migration-test.c
>> @@ -1181,6 +1181,16 @@ static void test_migrate_auto_converge(void)
>>       MigrateStart *args = migrate_start_new();
>>       QTestState *from, *to;
>>       int64_t remaining, percentage;
>> +    char *travis_arch;
>> +
>> +    /*
>> +     * This test does not work reliably on Travis s390x.
>> +     */
>> +    travis_arch = getenv("TRAVIS_CPU_ARCH");
>> +    if (travis_arch && g_str_equal(travis_arch, "s390x")) {
>> +        g_test_skip("Test does not work reliably on s390x Travis containers.");
>> +        return;
>> +    }
>>   
>>       /*
>>        * We want the test to be stable and as fast as possible.
>> -- 
>> 2.21.1
>>
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
>

Re: [PATCH 3/4] tests/migration: Disable autoconverge test on Travis-CI s390x

Posted by Dr. David Alan Gilbert 5 years, 8 months ago

* Philippe Mathieu-Daudé (philmd@redhat.com) wrote:
> On 3/17/20 9:26 PM, Dr. David Alan Gilbert wrote:
> > * Philippe Mathieu-Daudé (philmd@redhat.com) wrote:
> > > This test fails on Travis-CI s390x when configured with --disable-tcg:
> > > 
> > >    $ make check-qtest
> > >      TEST    check-qtest-s390x: tests/qtest/boot-serial-test
> > >    qemu-system-s390x: -accel tcg: invalid accelerator tcg
> > >    qemu-system-s390x: falling back to KVM
> > >      TEST    check-qtest-s390x: tests/qtest/pxe-test
> > >      TEST    check-qtest-s390x: tests/qtest/test-netfilter
> > >      TEST    check-qtest-s390x: tests/qtest/test-filter-mirror
> > >      TEST    check-qtest-s390x: tests/qtest/test-filter-redirector
> > >      TEST    check-qtest-s390x: tests/qtest/drive_del-test
> > >      TEST    check-qtest-s390x: tests/qtest/device-plug-test
> > >      TEST    check-qtest-s390x: tests/qtest/virtio-ccw-test
> > >      TEST    check-qtest-s390x: tests/qtest/cpu-plug-test
> > >      TEST    check-qtest-s390x: tests/qtest/migration-test
> > >    **
> > >    ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
> > >    ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
> > >    make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1
> > 
> > Before we take the hammer to it, could you try reducing it's initial
> > bandwidth, the code that says:
> > 
> >      /*
> >       * Set the initial parameters so that the migration could not converge
> >       * without throttling.
> >       */
> >      migrate_set_parameter_int(from, "downtime-limit", 1);
> >      migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */
> > 
> > knock a couple of 0's off that and see if it's happy.
> 
> You are right! One 0 is not enough, but a couple makes the test succeed.
> 
> Can that change have other (bad) effects on other archs?
> IOW can I send a generic patch updating this value, or should I add a check
> on the architecture?

I think generic is fine;  as soon as the code notices that it's
throttling it increases the bandwidth to let it complete, so it should
be OK to have this low.

Dave

> Thanks!
> 
> > 
> > Dave
> > 
> > > Per David Gilbert, it might be a threshold requiring tuning:
> > > "it could just be the writing is slow on s390 and the migration
> > > thread fast; in which case the autocomplete wouldn't be needed.
> > > Perhaps we just need to reduce the bandwidth limit."
> > > 
> > > Disable the autoconverge test for now.
> > > 
> > > Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> > > ---
> > >   tests/qtest/migration-test.c | 10 ++++++++++
> > >   1 file changed, 10 insertions(+)
> > > 
> > > diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> > > index 3d6cc83b88..878399666e 100644
> > > --- a/tests/qtest/migration-test.c
> > > +++ b/tests/qtest/migration-test.c
> > > @@ -1181,6 +1181,16 @@ static void test_migrate_auto_converge(void)
> > >       MigrateStart *args = migrate_start_new();
> > >       QTestState *from, *to;
> > >       int64_t remaining, percentage;
> > > +    char *travis_arch;
> > > +
> > > +    /*
> > > +     * This test does not work reliably on Travis s390x.
> > > +     */
> > > +    travis_arch = getenv("TRAVIS_CPU_ARCH");
> > > +    if (travis_arch && g_str_equal(travis_arch, "s390x")) {
> > > +        g_test_skip("Test does not work reliably on s390x Travis containers.");
> > > +        return;
> > > +    }
> > >       /*
> > >        * We want the test to be stable and as fast as possible.
> > > -- 
> > > 2.21.1
> > > 
> > --
> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> > 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

[PATCH 1/4] tests/test-util-filemonitor: Fix Travis-CI $ARCH env variable name
[PATCH 2/4] tests/test-util-sockets: Skip test on non-x86 Travis containers
[PATCH 3/4] tests/migration: Disable autoconverge test on Travis-CI s390x
[PATCH 4/4] .travis.yml: Add a KVM-only s390x job