[v2] ci: Fixes on the recent cross-binary test case

[PATCH v2 1/3] tests/migration-test: Stick with gicv3 in aarch64 test

Posted by peterx@redhat.com 9 months, 3 weeks ago

From: Peter Xu <peterx@redhat.com>

Recently we introduced cross-binary migration test.  It's always wanted
that migration-test uses stable guest ABI for both QEMU binaries in this
case, so that both QEMU binaries will be compatible on the migration
stream with the cmdline specified.

Switch to a static gic version "3" rather than using version "max", so that
GIC should be stable now across any future QEMU binaries for migration-test.

Here the version can actually be anything as long as the ABI is stable.  We
choose "3" because it's the majority of what we already use in QEMU while
still new enough: "git grep gic-version=3" shows 6 hit, while version 4 has
no direct user yet besides "max".

Note that even with this change, aarch64 won't be able to work yet with
migration cross binary test, but then the only missing piece will be the
stable CPU model.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
 tests/qtest/migration-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
index 7675519cfa..8a5bb1752e 100644
--- a/tests/qtest/migration-test.c
+++ b/tests/qtest/migration-test.c
@@ -819,7 +819,7 @@ static int test_migrate_start(QTestState **from, QTestState **to,
     } else if (strcmp(arch, "aarch64") == 0) {
         memory_size = "150M";
         machine_alias = "virt";
-        machine_opts = "gic-version=max";
+        machine_opts = "gic-version=3";
         arch_opts = g_strdup_printf("-cpu max -kernel %s", bootpath);
         start_address = ARM_TEST_MEM_START;
         end_address = ARM_TEST_MEM_END;
-- 
2.43.0

Re: [PATCH v2 1/3] tests/migration-test: Stick with gicv3 in aarch64 test

Posted by Thomas Huth 9 months, 1 week ago

On 07/02/2024 01.54, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
> 
> Recently we introduced cross-binary migration test.  It's always wanted
> that migration-test uses stable guest ABI for both QEMU binaries in this
> case, so that both QEMU binaries will be compatible on the migration
> stream with the cmdline specified.
> 
> Switch to a static gic version "3" rather than using version "max", so that
> GIC should be stable now across any future QEMU binaries for migration-test.
> 
> Here the version can actually be anything as long as the ABI is stable.  We
> choose "3" because it's the majority of what we already use in QEMU while
> still new enough: "git grep gic-version=3" shows 6 hit, while version 4 has
> no direct user yet besides "max".
> 
> Note that even with this change, aarch64 won't be able to work yet with
> migration cross binary test, but then the only missing piece will be the
> stable CPU model.
> 
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
>   tests/qtest/migration-test.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> index 7675519cfa..8a5bb1752e 100644
> --- a/tests/qtest/migration-test.c
> +++ b/tests/qtest/migration-test.c
> @@ -819,7 +819,7 @@ static int test_migrate_start(QTestState **from, QTestState **to,
>       } else if (strcmp(arch, "aarch64") == 0) {
>           memory_size = "150M";
>           machine_alias = "virt";
> -        machine_opts = "gic-version=max";
> +        machine_opts = "gic-version=3";
>           arch_opts = g_strdup_printf("-cpu max -kernel %s", bootpath);
>           start_address = ARM_TEST_MEM_START;
>           end_address = ARM_TEST_MEM_END;

Looks like the migration test now fails on aarch64 when "configure" has been 
run with "--without-default-devices", since that disables the gicv3 in the 
binary ... is there a way to check whether the gicv3 is available, and use 
"=max" instead if it is not?

  Thomas

Re: [PATCH v2 1/3] tests/migration-test: Stick with gicv3 in aarch64 test

Posted by Peter Maydell 9 months, 1 week ago

On Mon, 19 Feb 2024 at 11:54, Thomas Huth <thuth@redhat.com> wrote:
>
> On 07/02/2024 01.54, peterx@redhat.com wrote:
> > From: Peter Xu <peterx@redhat.com>
> >
> > Recently we introduced cross-binary migration test.  It's always wanted
> > that migration-test uses stable guest ABI for both QEMU binaries in this
> > case, so that both QEMU binaries will be compatible on the migration
> > stream with the cmdline specified.
> >
> > Switch to a static gic version "3" rather than using version "max", so that
> > GIC should be stable now across any future QEMU binaries for migration-test.
> >
> > Here the version can actually be anything as long as the ABI is stable.  We
> > choose "3" because it's the majority of what we already use in QEMU while
> > still new enough: "git grep gic-version=3" shows 6 hit, while version 4 has
> > no direct user yet besides "max".
> >
> > Note that even with this change, aarch64 won't be able to work yet with
> > migration cross binary test, but then the only missing piece will be the
> > stable CPU model.
> >
> > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> > Signed-off-by: Peter Xu <peterx@redhat.com>
> > ---
> >   tests/qtest/migration-test.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
> > index 7675519cfa..8a5bb1752e 100644
> > --- a/tests/qtest/migration-test.c
> > +++ b/tests/qtest/migration-test.c
> > @@ -819,7 +819,7 @@ static int test_migrate_start(QTestState **from, QTestState **to,
> >       } else if (strcmp(arch, "aarch64") == 0) {
> >           memory_size = "150M";
> >           machine_alias = "virt";
> > -        machine_opts = "gic-version=max";
> > +        machine_opts = "gic-version=3";
> >           arch_opts = g_strdup_printf("-cpu max -kernel %s", bootpath);
> >           start_address = ARM_TEST_MEM_START;
> >           end_address = ARM_TEST_MEM_END;
>
> Looks like the migration test now fails on aarch64 when "configure" has been
> run with "--without-default-devices", since that disables the gicv3 in the
> binary ... is there a way to check whether the gicv3 is available, and use
> "=max" instead if it is not?

A QEMU for AArch64 with no GICv3 is of very little practical use,
so I'm not sure it makes sense to allow users to build one like that.
(I'm also a little surprised that it worked with 'max', because
without a GICv3 it would try to fall back to GICv2, and if we're
going to allow users to compile-time disable one of the GICs then
we should definitely allow them to choose to not build GICv2.)

I think I would go for disabling the migration test entirely if
the build doesn't include the GICv3.

thanks
-- PMM

Re: [PATCH v2 1/3] tests/migration-test: Stick with gicv3 in aarch64 test

Posted by Thomas Huth 9 months, 1 week ago

On 19/02/2024 13.50, Peter Maydell wrote:
> On Mon, 19 Feb 2024 at 11:54, Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 07/02/2024 01.54, peterx@redhat.com wrote:
>>> From: Peter Xu <peterx@redhat.com>
>>>
>>> Recently we introduced cross-binary migration test.  It's always wanted
>>> that migration-test uses stable guest ABI for both QEMU binaries in this
>>> case, so that both QEMU binaries will be compatible on the migration
>>> stream with the cmdline specified.
>>>
>>> Switch to a static gic version "3" rather than using version "max", so that
>>> GIC should be stable now across any future QEMU binaries for migration-test.
>>>
>>> Here the version can actually be anything as long as the ABI is stable.  We
>>> choose "3" because it's the majority of what we already use in QEMU while
>>> still new enough: "git grep gic-version=3" shows 6 hit, while version 4 has
>>> no direct user yet besides "max".
>>>
>>> Note that even with this change, aarch64 won't be able to work yet with
>>> migration cross binary test, but then the only missing piece will be the
>>> stable CPU model.
>>>
>>> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>>> Signed-off-by: Peter Xu <peterx@redhat.com>
>>> ---
>>>    tests/qtest/migration-test.c | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
>>> index 7675519cfa..8a5bb1752e 100644
>>> --- a/tests/qtest/migration-test.c
>>> +++ b/tests/qtest/migration-test.c
>>> @@ -819,7 +819,7 @@ static int test_migrate_start(QTestState **from, QTestState **to,
>>>        } else if (strcmp(arch, "aarch64") == 0) {
>>>            memory_size = "150M";
>>>            machine_alias = "virt";
>>> -        machine_opts = "gic-version=max";
>>> +        machine_opts = "gic-version=3";
>>>            arch_opts = g_strdup_printf("-cpu max -kernel %s", bootpath);
>>>            start_address = ARM_TEST_MEM_START;
>>>            end_address = ARM_TEST_MEM_END;
>>
>> Looks like the migration test now fails on aarch64 when "configure" has been
>> run with "--without-default-devices", since that disables the gicv3 in the
>> binary ... is there a way to check whether the gicv3 is available, and use
>> "=max" instead if it is not?
> 
> A QEMU for AArch64 with no GICv3 is of very little practical use,
> so I'm not sure it makes sense to allow users to build one like that.

Ok, if it doesn't make too much sense to build without GICv3, maybe a patch 
like this could be the right solution:

diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
--- a/hw/intc/Kconfig
+++ b/hw/intc/Kconfig
@@ -12,10 +12,6 @@ config IOAPIC
      bool
      select I8259

-config ARM_GIC
-    bool
-    select MSI_NONBROKEN
-
  config OPENPIC
      bool
      select MSI_NONBROKEN
@@ -25,14 +21,18 @@ config APIC
      select MSI_NONBROKEN
      select I8259

+config ARM_GIC
+    bool
+    select ARM_GICV3_TCG if TCG
+    select ARM_GIC_KVM if KVM
+    select MSI_NONBROKEN
+
  config ARM_GICV3_TCG
      bool
-    default y
      depends on ARM_GIC && TCG

  config ARM_GIC_KVM
      bool
-    default y
      depends on ARM_GIC && KVM

  config XICS

?

At least this seems to fix the the migration-test when compiling with 
--without-default-devices.

  Thomas

Re: [PATCH v2 1/3] tests/migration-test: Stick with gicv3 in aarch64 test

Posted by Fabiano Rosas 9 months, 1 week ago

Peter Maydell <peter.maydell@linaro.org> writes:

> On Mon, 19 Feb 2024 at 11:54, Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 07/02/2024 01.54, peterx@redhat.com wrote:
>> > From: Peter Xu <peterx@redhat.com>
>> >
>> > Recently we introduced cross-binary migration test.  It's always wanted
>> > that migration-test uses stable guest ABI for both QEMU binaries in this
>> > case, so that both QEMU binaries will be compatible on the migration
>> > stream with the cmdline specified.
>> >
>> > Switch to a static gic version "3" rather than using version "max", so that
>> > GIC should be stable now across any future QEMU binaries for migration-test.
>> >
>> > Here the version can actually be anything as long as the ABI is stable.  We
>> > choose "3" because it's the majority of what we already use in QEMU while
>> > still new enough: "git grep gic-version=3" shows 6 hit, while version 4 has
>> > no direct user yet besides "max".
>> >
>> > Note that even with this change, aarch64 won't be able to work yet with
>> > migration cross binary test, but then the only missing piece will be the
>> > stable CPU model.
>> >
>> > Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>> > Signed-off-by: Peter Xu <peterx@redhat.com>
>> > ---
>> >   tests/qtest/migration-test.c | 2 +-
>> >   1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c
>> > index 7675519cfa..8a5bb1752e 100644
>> > --- a/tests/qtest/migration-test.c
>> > +++ b/tests/qtest/migration-test.c
>> > @@ -819,7 +819,7 @@ static int test_migrate_start(QTestState **from, QTestState **to,
>> >       } else if (strcmp(arch, "aarch64") == 0) {
>> >           memory_size = "150M";
>> >           machine_alias = "virt";
>> > -        machine_opts = "gic-version=max";
>> > +        machine_opts = "gic-version=3";
>> >           arch_opts = g_strdup_printf("-cpu max -kernel %s", bootpath);
>> >           start_address = ARM_TEST_MEM_START;
>> >           end_address = ARM_TEST_MEM_END;
>>
>> Looks like the migration test now fails on aarch64 when "configure" has been
>> run with "--without-default-devices", since that disables the gicv3 in the
>> binary ... is there a way to check whether the gicv3 is available, and use
>> "=max" instead if it is not?

How about adding aarch64 to the set of targets on the without-defaults
CI job?

>
> A QEMU for AArch64 with no GICv3 is of very little practical use,
> so I'm not sure it makes sense to allow users to build one like that.
> (I'm also a little surprised that it worked with 'max', because
> without a GICv3 it would try to fall back to GICv2, and if we're
> going to allow users to compile-time disable one of the GICs then
> we should definitely allow them to choose to not build GICv2.)
>
> I think I would go for disabling the migration test entirely if
> the build doesn't include the GICv3.

Makes sense. I've got my hands full right now, but I'll work on it later
this week along with your other suggestion to only test TCG.

Thanks