1. Patchew
  2. QEMU
  3. [PATCH 00/22] tests/qtest: migration-test refactoring
  4. View patch

[PATCH 21/22] tests/qtest: Add support for check-qtest-<subsystem>

Fabiano Rosas posted 22 patches 2 weeks, 4 days ago
There is a newer version of this series
[PATCH 21/22] tests/qtest: Add support for check-qtest-<subsystem>
Posted by Fabiano Rosas 2 weeks, 4 days ago 
Allow qtests to be ran by subsystem. Some subsystems, such as
migration, have a large number of tests and we could benefit from
being able to access them from make check without having to run the
full set of qtests.

This adds the following make check targets:

make check-qtest-migration

Signed-off-by: Fabiano Rosas <farosas@suse.de>
---
I did not include check-qtest-<arch>-<subsys> because meson generates a
long line that affects readability.
---
 tests/qtest/meson.build | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
index 806af512d0..e0ba858f5b 100644
--- a/tests/qtest/meson.build
+++ b/tests/qtest/meson.build
@@ -36,6 +36,13 @@ if enable_modules
   qtests_generic += [ 'modules-test' ]
 endif
 
+qtest_subsystems = {
+  'migration-test-smoke': ['migration'],
+  'migration-test': ['migration'],
+  'cdrom-test': ['block'],
+  'ahci-test': ['block'],
+}
+
 qtests_pci = \
   (config_all_devices.has_key('CONFIG_VGA') ? ['display-vga-test'] : []) +                  \
   (config_all_devices.has_key('CONFIG_IVSHMEM_DEVICE') ? ['ivshmem-test'] : [])
@@ -431,6 +438,12 @@ foreach dir : target_dirs
         test: executable(test, src, dependencies: deps)
       }
     endif
+
+    suites = ['qtest', 'qtest-' + target_base]
+    foreach subsys: qtest_subsystems.get(test, [])
+      suites += ['qtest-' + subsys]
+    endforeach
+
     test('qtest-@0@/@1@'.format(target_base, test),
          qtest_executables[test],
          depends: [test_deps, qtest_emulator, emulator_modules],
@@ -439,6 +452,6 @@ foreach dir : target_dirs
          protocol: 'tap',
          timeout: slow_qtests.get(test, 60),
          priority: slow_qtests.get(test, 60),
-         suite: ['qtest', 'qtest-' + target_base])
+         suite: suites)
   endforeach
 endforeach
-- 
2.35.3
Re: [PATCH 21/22] tests/qtest: Add support for check-qtest-<subsystem>
Posted by Daniel P. Berrangé 2 weeks, 3 days ago 
On Tue, Nov 05, 2024 at 03:08:36PM -0300, Fabiano Rosas wrote:
> Allow qtests to be ran by subsystem. Some subsystems, such as
> migration, have a large number of tests and we could benefit from
> being able to access them from make check without having to run the
> full set of qtests.
> 
> This adds the following make check targets:
> 
> make check-qtest-migration
> 
> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> ---
> I did not include check-qtest-<arch>-<subsys> because meson generates a
> long line that affects readability.

Can you give an example of that ?  I'm wondering how much of a
problem it actually is ? Personally when I'm running a subset
of tests, I almost always want to limit to just one arch target.

> ---
>  tests/qtest/meson.build | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)

Despite the above question, this does what it claims to do
at this time, so

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

> 
> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
> index 806af512d0..e0ba858f5b 100644
> --- a/tests/qtest/meson.build
> +++ b/tests/qtest/meson.build
> @@ -36,6 +36,13 @@ if enable_modules
>    qtests_generic += [ 'modules-test' ]
>  endif
>  
> +qtest_subsystems = {
> +  'migration-test-smoke': ['migration'],
> +  'migration-test': ['migration'],
> +  'cdrom-test': ['block'],
> +  'ahci-test': ['block'],
> +}
> +
>  qtests_pci = \
>    (config_all_devices.has_key('CONFIG_VGA') ? ['display-vga-test'] : []) +                  \
>    (config_all_devices.has_key('CONFIG_IVSHMEM_DEVICE') ? ['ivshmem-test'] : [])
> @@ -431,6 +438,12 @@ foreach dir : target_dirs
>          test: executable(test, src, dependencies: deps)
>        }
>      endif
> +
> +    suites = ['qtest', 'qtest-' + target_base]
> +    foreach subsys: qtest_subsystems.get(test, [])
> +      suites += ['qtest-' + subsys]
> +    endforeach
> +
>      test('qtest-@0@/@1@'.format(target_base, test),
>           qtest_executables[test],
>           depends: [test_deps, qtest_emulator, emulator_modules],
> @@ -439,6 +452,6 @@ foreach dir : target_dirs
>           protocol: 'tap',
>           timeout: slow_qtests.get(test, 60),
>           priority: slow_qtests.get(test, 60),
> -         suite: ['qtest', 'qtest-' + target_base])
> +         suite: suites)
>    endforeach
>  endforeach
> -- 
> 2.35.3
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH 21/22] tests/qtest: Add support for check-qtest-<subsystem>
Posted by Fabiano Rosas 2 weeks, 3 days ago 
Daniel P. Berrangé <berrange@redhat.com> writes:

> On Tue, Nov 05, 2024 at 03:08:36PM -0300, Fabiano Rosas wrote:
>> Allow qtests to be ran by subsystem. Some subsystems, such as
>> migration, have a large number of tests and we could benefit from
>> being able to access them from make check without having to run the
>> full set of qtests.
>> 
>> This adds the following make check targets:
>> 
>> make check-qtest-migration
>> 
>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
>> ---
>> I did not include check-qtest-<arch>-<subsys> because meson generates a
>> long line that affects readability.
>
> Can you give an example of that ?  I'm wondering how much of a
> problem it actually is ? Personally when I'm running a subset
> of tests, I almost always want to limit to just one arch target.
>

Ok, you have a different use case, mine is just "run migration tests for
all archs, without bringing all the rest of qtest along". When I want to
limit to one arch, I use the 'QTEST_QEMU_BINARY=qemu-system-<arch>
migration-test' syntax.

The issue is that meson always shows all suites in which a test is
present (quite uselessly IMO):

[1-8/8] 🌑 qemu:qtest+qtest-s390x+qtest-migration+qtest-s390x-migration / qtest-s390x/migration-test                              0/480
1/8 qemu:qtest+qtest-x86_64+qtest-migration+qtest-x86_64-migration / qtest-x86_64/migration-test-smoke           SKIP             0.02s

vs:

1/8 qemu:qtest+qtest-x86_64+qtest-migration / qtest-x86_64/migration-test-smoke          SKIP             0.02s
2/8 qemu:qtest+qtest-ppc64+qtest-migration / qtest-ppc64/migration-test                  SKIP             0.02s
Re: [PATCH 21/22] tests/qtest: Add support for check-qtest-<subsystem>
Posted by Daniel P. Berrangé 2 weeks, 3 days ago 
On Wed, Nov 06, 2024 at 09:40:21AM -0300, Fabiano Rosas wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > On Tue, Nov 05, 2024 at 03:08:36PM -0300, Fabiano Rosas wrote:
> >> Allow qtests to be ran by subsystem. Some subsystems, such as
> >> migration, have a large number of tests and we could benefit from
> >> being able to access them from make check without having to run the
> >> full set of qtests.
> >> 
> >> This adds the following make check targets:
> >> 
> >> make check-qtest-migration
> >> 
> >> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> >> ---
> >> I did not include check-qtest-<arch>-<subsys> because meson generates a
> >> long line that affects readability.
> >
> > Can you give an example of that ?  I'm wondering how much of a
> > problem it actually is ? Personally when I'm running a subset
> > of tests, I almost always want to limit to just one arch target.
> >
> 
> Ok, you have a different use case, mine is just "run migration tests for
> all archs, without bringing all the rest of qtest along". When I want to
> limit to one arch, I use the 'QTEST_QEMU_BINARY=qemu-system-<arch>
> migration-test' syntax.
> 
> The issue is that meson always shows all suites in which a test is
> present (quite uselessly IMO):

Ewwww, yes, that's pretty horrific :-( Very annoying behaviour, as it
actively discourages people from adding more suites :-( I agree with
your decision.

> 
> [1-8/8] 🌑 qemu:qtest+qtest-s390x+qtest-migration+qtest-s390x-migration / qtest-s390x/migration-test                              0/480
> 1/8 qemu:qtest+qtest-x86_64+qtest-migration+qtest-x86_64-migration / qtest-x86_64/migration-test-smoke           SKIP             0.02s
> 
> vs:
> 
> 1/8 qemu:qtest+qtest-x86_64+qtest-migration / qtest-x86_64/migration-test-smoke          SKIP             0.02s
> 2/8 qemu:qtest+qtest-ppc64+qtest-migration / qtest-ppc64/migration-test                  SKIP             0.02s
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Patchew 2.1-devel-b67e4c2

© 2016 - 2024 Red Hat, Inc.