[PATCH v2 04/13] tests: ensure all qcow2 I/O tests are able to be run via make

Daniel P. Berrangé posted 13 patches 3 weeks, 6 days ago
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Thomas Huth <thuth@redhat.com>, John Snow <jsnow@redhat.com>, Cleber Rosa <crosa@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>
There is a newer version of this series
[PATCH v2 04/13] tests: ensure all qcow2 I/O tests are able to be run via make
Posted by Daniel P. Berrangé 3 weeks, 6 days ago
For block formats marked as 'quick', only tests in the 'auto' group are
added to the meson test suite.

The result of this is that qcow2 tests not in the 'auto' group cannot be
run at all, even if passing SPEED=slow or SPEED=thorough.

To fix this we need todo two passes over the I/O test list. First add
all tests from 'auto' group into the 'block' suite, so they are run by
default. Then on the second pass add any tests which were not in 'auto'
into the 'block-slow' suite, so they get run when SPEED=slow or
SPEED=thorough.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index bf588cc2c9..1a24d801a3 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -30,16 +30,48 @@ endforeach
 qemu_iotests_check_cmd = files('check')
 
 foreach format, speed: qemu_iotests_formats
+  # Formats tagged 'quick' get the subset of tests in the 'auto'
+  # group, run by default with 'make check' / 'make check-block'
+  seen = []
   if speed == 'quick'
-    suites = 'block'
+    args = ['-tap', '-' + format, '-g', 'auto']
+    suites = ['block']
+
+    rc = run_command(
+      [python, qemu_iotests_check_cmd] + args + ['-n'],
+      check: true,
+    )
+
+    foreach item: rc.stdout().strip().split()
+      seen += item
+      args = [qemu_iotests_check_cmd,
+              '-tap', '-' + format, item,
+              '--source-dir', meson.current_source_dir(),
+              '--build-dir', meson.current_build_dir()]
+      # Some individual tests take as long as 45 seconds
+      # Bump the timeout to 3 minutes for some headroom
+      # on slow machines to minimize spurious failures
+      test('io-' + format + '-' + item,
+           python,
+           args: args,
+           depends: qemu_iotests_binaries,
+           env: qemu_iotests_env,
+           protocol: 'tap',
+           timeout: 180,
+           suite: suites)
+    endforeach
+  endif
+
+  suites = []
+  # Any format tagged quick or slow also gets added to slow
+  # otherwise its tagged thorough
+  if speed != 'thorough'
+    suites += ['block-slow']
   else
-    suites = ['block-' + speed]
+    suites += ['block-thorough']
   endif
 
   args = ['-tap', '-' + format]
-  if speed == 'quick'
-      args += ['-g', 'auto']
-  endif
 
   rc = run_command(
       [python, qemu_iotests_check_cmd] + args + ['-n'],
@@ -47,6 +79,12 @@ foreach format, speed: qemu_iotests_formats
   )
 
   foreach item: rc.stdout().strip().split()
+      # Skip any tests already added from the 'auto' group
+      # as they're run in the 'quick' suite already
+      if item in seen
+          continue
+      endif
+
       args = [qemu_iotests_check_cmd,
               '-tap', '-' + format, item,
               '--source-dir', meson.current_source_dir(),
-- 
2.52.0


Re: [PATCH v2 04/13] tests: ensure all qcow2 I/O tests are able to be run via make
Posted by Thomas Huth 3 weeks, 5 days ago
On 12/01/2026 21.40, Daniel P. Berrangé wrote:
> For block formats marked as 'quick', only tests in the 'auto' group are
> added to the meson test suite.
> 
> The result of this is that qcow2 tests not in the 'auto' group cannot be
> run at all, even if passing SPEED=slow or SPEED=thorough.
> 
> To fix this we need todo two passes over the I/O test list. First add
> all tests from 'auto' group into the 'block' suite, so they are run by
> default. Then on the second pass add any tests which were not in 'auto'
> into the 'block-slow' suite, so they get run when SPEED=slow or
> SPEED=thorough.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>   tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++++++----
>   1 file changed, 43 insertions(+), 5 deletions(-)
> 
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index bf588cc2c9..1a24d801a3 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -30,16 +30,48 @@ endforeach
>   qemu_iotests_check_cmd = files('check')
>   
>   foreach format, speed: qemu_iotests_formats
> +  # Formats tagged 'quick' get the subset of tests in the 'auto'
> +  # group, run by default with 'make check' / 'make check-block'
> +  seen = []
>     if speed == 'quick'
> -    suites = 'block'
> +    args = ['-tap', '-' + format, '-g', 'auto']
> +    suites = ['block']
> +
> +    rc = run_command(
> +      [python, qemu_iotests_check_cmd] + args + ['-n'],
> +      check: true,
> +    )
> +
> +    foreach item: rc.stdout().strip().split()
> +      seen += item
> +      args = [qemu_iotests_check_cmd,
> +              '-tap', '-' + format, item,
> +              '--source-dir', meson.current_source_dir(),
> +              '--build-dir', meson.current_build_dir()]
> +      # Some individual tests take as long as 45 seconds
> +      # Bump the timeout to 3 minutes for some headroom
> +      # on slow machines to minimize spurious failures
> +      test('io-' + format + '-' + item,
> +           python,
> +           args: args,
> +           depends: qemu_iotests_binaries,
> +           env: qemu_iotests_env,
> +           protocol: 'tap',
> +           timeout: 180,
> +           suite: suites)
> +    endforeach

It's a little bit unfortunate that we have this construct now twice in the 
meson.build file (with just the difference of the "-g auto") ... but I 
currently lack an idea how it could be done in a better way, so:

Reviewed-by: Thomas Huth <thuth@redhat.com>


> +  endif
> +
> +  suites = []
> +  # Any format tagged quick or slow also gets added to slow
> +  # otherwise its tagged thorough
> +  if speed != 'thorough'
> +    suites += ['block-slow']
>     else
> -    suites = ['block-' + speed]
> +    suites += ['block-thorough']
>     endif
>   
>     args = ['-tap', '-' + format]
> -  if speed == 'quick'
> -      args += ['-g', 'auto']
> -  endif
>   
>     rc = run_command(
>         [python, qemu_iotests_check_cmd] + args + ['-n'],
> @@ -47,6 +79,12 @@ foreach format, speed: qemu_iotests_formats
>     )
>   
>     foreach item: rc.stdout().strip().split()
> +      # Skip any tests already added from the 'auto' group
> +      # as they're run in the 'quick' suite already
> +      if item in seen
> +          continue
> +      endif
> +
>         args = [qemu_iotests_check_cmd,
>                 '-tap', '-' + format, item,
>                 '--source-dir', meson.current_source_dir(),