Currently there are 5 block formats, each classified as either quick,
slow or thorough. This classification then determines what tests we
add to meson suites. The suites are as follows
* block
=> tests listed by 'check -g auto -n' for 'quick' formats
* slow, block-slow
=> tests listed by 'check -n' for 'slow' formats or 'check -g auto -n'
for 'quick' formats
* thorough, block-thorough
=> tests listed by 'check -n' for 'thorough' formats
The pairs of suites 'slow' / 'block-slow' and 'thorough' / 'block-thorough'
match in terms of what tests are enabled. The only difference is whether
non-block related tests are also in the suite.
There are two problems with this
* If a format is classified as 'quick', we don't expose any
meson suite for running *all* tests, only the 'auto' tests.
eg there is no suite to run all qcow2 tests, only 'quick'
tests can be run via meson, even if using 'SPEED=slow' we
still filter to only 'auto' tests.
* There is no suite that allows running all tests for a given
format.
eg there is no suite to run only 'raw' tests - you can only
use 'block-slow' which runs both raw and "auto" qcow2 tests.
eg there is no suite to run only 'vpc' tests - you can only
use 'block-thorough' which runs qed, vmdk & vpc tests.
This patch suggests that 'block-slow' and 'block-thorough' are
not actually compelling use cases, and should be dropped. ie it
is not expected that people need to run all VPC, VMDK and QED
tests at the same time. Instead a more useful feature is the
ability to run all tests for a given format. Further the 'auto'
filtering should only apply in the default 'block' target/suite.
IOW, with this patch we get the follows meson suites:
* 'block' - 'auto' tests for any format listed as 'quick'
Currently just qcow2 'auto' tests
* 'block-$FORMAT' - ALL tests for the given $FORMAT, for each
of qcow2, raw, qed, vmdk & vpc
* 'slow' - ALL tests for formats tagged with 'quick' or 'slow'
* 'thorough' - ALL tests formats tagged with 'thorough'
This corresponds to the following make targets.
* 'make check-block'
=> runs only 'auto' qcow2 tests (unchanged)
* 'make check-block SPEED=thorough'
=> runs all 'qed', 'vmdk', 'vpc' tests (unchanged)
* 'make check-block SPEED=slow'
=> runs all 'raw' tests (unchanged)
=> runs all 'qcow2' tests (previously was only 'auto' tests)
* 'make check-block-qcow2'
=> runs all qcow2 tests (new feature)
* 'make check-block-raw'
=> runs all raw tests (new feature)
* 'make check-block-vpc'
=> runs all vpc tests (new feature)
* 'make check-block-qed'
=> runs all qed tests (new feature)
* 'make check-block-vmdk'
=> runs all vmdk tests (new feature)
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++--------
1 file changed, 37 insertions(+), 11 deletions(-)
diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
index fad340ad59..939a14ffae 100644
--- a/tests/qemu-iotests/meson.build
+++ b/tests/qemu-iotests/meson.build
@@ -35,23 +35,21 @@ endforeach
qemu_iotests_check_cmd = files('check')
foreach format, speed: qemu_iotests_formats
- if speed == 'quick'
- suites = 'block'
- else
- suites = ['block-' + speed, speed]
- endif
-
- args = ['-tap', '-' + format]
- if speed == 'quick'
- args += ['-g', 'auto']
- endif
+ listargs = ['-tap', '-' + format]
rc = run_command(
- [python, qemu_iotests_check_cmd] + args + ['-n'],
+ [python, qemu_iotests_check_cmd] + listargs + ['-n'],
check: true,
)
foreach item: rc.stdout().strip().split()
+ suites = ['block-' + format]
+ if speed == 'quick'
+ suites += ['slow']
+ else
+ suites += [speed]
+ endif
+
args = [qemu_iotests_check_cmd,
'-tap', '-' + format, item,
'--source-dir', meson.current_source_dir(),
@@ -68,4 +66,32 @@ foreach format, speed: qemu_iotests_formats
timeout: 180,
suite: suites)
endforeach
+
+ if speed == 'quick'
+ listargs += ['-g', 'auto']
+ suites = ['block']
+
+ rc = run_command(
+ [python, qemu_iotests_check_cmd] + listargs + ['-n'],
+ check: true,
+ )
+
+ foreach item: rc.stdout().strip().split()
+ args = [qemu_iotests_check_cmd,
+ '-tap', '-' + format, item,
+ '--source-dir', meson.current_source_dir(),
+ '--build-dir', meson.current_build_dir()]
+ # Some individual tests take as long as 45 seconds
+ # Bump the timeout to 3 minutes for some headroom
+ # on slow machines to minimize spurious failures
+ test('io-' + format + '-' + item,
+ python,
+ args: args,
+ depends: qemu_iotests_binaries,
+ env: qemu_iotests_env,
+ protocol: 'tap',
+ timeout: 180,
+ suite: suites)
+ endforeach
+ endif
endforeach
--
2.50.1
Am 08.10.2025 um 13:35 hat Daniel P. Berrangé geschrieben:
> Currently there are 5 block formats, each classified as either quick,
> slow or thorough. This classification then determines what tests we
> add to meson suites. The suites are as follows
>
> * block
>
> => tests listed by 'check -g auto -n' for 'quick' formats
>
> * slow, block-slow
>
> => tests listed by 'check -n' for 'slow' formats or 'check -g auto -n'
> for 'quick' formats
>
> * thorough, block-thorough
>
> => tests listed by 'check -n' for 'thorough' formats
>
> The pairs of suites 'slow' / 'block-slow' and 'thorough' / 'block-thorough'
> match in terms of what tests are enabled. The only difference is whether
> non-block related tests are also in the suite.
>
> There are two problems with this
>
> * If a format is classified as 'quick', we don't expose any
> meson suite for running *all* tests, only the 'auto' tests.
>
> eg there is no suite to run all qcow2 tests, only 'quick'
> tests can be run via meson, even if using 'SPEED=slow' we
> still filter to only 'auto' tests.
>
> * There is no suite that allows running all tests for a given
> format.
>
> eg there is no suite to run only 'raw' tests - you can only
> use 'block-slow' which runs both raw and "auto" qcow2 tests.
>
> eg there is no suite to run only 'vpc' tests - you can only
> use 'block-thorough' which runs qed, vmdk & vpc tests.
>
> This patch suggests that 'block-slow' and 'block-thorough' are
> not actually compelling use cases, and should be dropped. ie it
> is not expected that people need to run all VPC, VMDK and QED
> tests at the same time. Instead a more useful feature is the
> ability to run all tests for a given format. Further the 'auto'
> filtering should only apply in the default 'block' target/suite.
>
> IOW, with this patch we get the follows meson suites:
>
> * 'block' - 'auto' tests for any format listed as 'quick'
> Currently just qcow2 'auto' tests
> * 'block-$FORMAT' - ALL tests for the given $FORMAT, for each
> of qcow2, raw, qed, vmdk & vpc
> * 'slow' - ALL tests for formats tagged with 'quick' or 'slow'
> * 'thorough' - ALL tests formats tagged with 'thorough'
>
> This corresponds to the following make targets.
>
> * 'make check-block'
>
> => runs only 'auto' qcow2 tests (unchanged)
>
> * 'make check-block SPEED=thorough'
>
> => runs all 'qed', 'vmdk', 'vpc' tests (unchanged)
Also all qcow2 and raw tests, right?
Quotes are a bit inconsistent here in the commit message. Sometimes
you use quotes for format names like here, but in other places you use
quotes for 'auto' and leave format names without quotes.
> * 'make check-block SPEED=slow'
>
> => runs all 'raw' tests (unchanged)
> => runs all 'qcow2' tests (previously was only 'auto' tests)
>
> * 'make check-block-qcow2'
>
> => runs all qcow2 tests (new feature)
>
> * 'make check-block-raw'
>
> => runs all raw tests (new feature)
>
> * 'make check-block-vpc'
>
> => runs all vpc tests (new feature)
>
> * 'make check-block-qed'
>
> => runs all qed tests (new feature)
>
> * 'make check-block-vmdk'
>
> => runs all vmdk tests (new feature)
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
> tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++--------
> 1 file changed, 37 insertions(+), 11 deletions(-)
>
> diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build
> index fad340ad59..939a14ffae 100644
> --- a/tests/qemu-iotests/meson.build
> +++ b/tests/qemu-iotests/meson.build
> @@ -35,23 +35,21 @@ endforeach
> qemu_iotests_check_cmd = files('check')
>
> foreach format, speed: qemu_iotests_formats
> - if speed == 'quick'
> - suites = 'block'
> - else
> - suites = ['block-' + speed, speed]
> - endif
> -
> - args = ['-tap', '-' + format]
> - if speed == 'quick'
> - args += ['-g', 'auto']
> - endif
> + listargs = ['-tap', '-' + format]
>
> rc = run_command(
> - [python, qemu_iotests_check_cmd] + args + ['-n'],
> + [python, qemu_iotests_check_cmd] + listargs + ['-n'],
> check: true,
> )
>
> foreach item: rc.stdout().strip().split()
> + suites = ['block-' + format]
> + if speed == 'quick'
> + suites += ['slow']
> + else
> + suites += [speed]
> + endif
> +
> args = [qemu_iotests_check_cmd,
> '-tap', '-' + format, item,
> '--source-dir', meson.current_source_dir(),
> @@ -68,4 +66,32 @@ foreach format, speed: qemu_iotests_formats
> timeout: 180,
> suite: suites)
> endforeach
> +
> + if speed == 'quick'
> + listargs += ['-g', 'auto']
> + suites = ['block']
> +
> + rc = run_command(
> + [python, qemu_iotests_check_cmd] + listargs + ['-n'],
> + check: true,
> + )
> +
> + foreach item: rc.stdout().strip().split()
> + args = [qemu_iotests_check_cmd,
> + '-tap', '-' + format, item,
> + '--source-dir', meson.current_source_dir(),
> + '--build-dir', meson.current_build_dir()]
> + # Some individual tests take as long as 45 seconds
> + # Bump the timeout to 3 minutes for some headroom
> + # on slow machines to minimize spurious failures
> + test('io-' + format + '-' + item,
> + python,
> + args: args,
> + depends: qemu_iotests_binaries,
> + env: qemu_iotests_env,
> + protocol: 'tap',
> + timeout: 180,
> + suite: suites)
> + endforeach
> + endif
> endforeach
The code itself looks good to me.
Kevin
On Tue, Oct 14, 2025 at 10:26:49AM +0200, Kevin Wolf wrote:
> Am 08.10.2025 um 13:35 hat Daniel P. Berrangé geschrieben:
> > Currently there are 5 block formats, each classified as either quick,
> > slow or thorough. This classification then determines what tests we
> > add to meson suites. The suites are as follows
> >
> > * block
> >
> > => tests listed by 'check -g auto -n' for 'quick' formats
> >
> > * slow, block-slow
> >
> > => tests listed by 'check -n' for 'slow' formats or 'check -g auto -n'
> > for 'quick' formats
> >
> > * thorough, block-thorough
> >
> > => tests listed by 'check -n' for 'thorough' formats
> >
> > The pairs of suites 'slow' / 'block-slow' and 'thorough' / 'block-thorough'
> > match in terms of what tests are enabled. The only difference is whether
> > non-block related tests are also in the suite.
> >
> > There are two problems with this
> >
> > * If a format is classified as 'quick', we don't expose any
> > meson suite for running *all* tests, only the 'auto' tests.
> >
> > eg there is no suite to run all qcow2 tests, only 'quick'
> > tests can be run via meson, even if using 'SPEED=slow' we
> > still filter to only 'auto' tests.
> >
> > * There is no suite that allows running all tests for a given
> > format.
> >
> > eg there is no suite to run only 'raw' tests - you can only
> > use 'block-slow' which runs both raw and "auto" qcow2 tests.
> >
> > eg there is no suite to run only 'vpc' tests - you can only
> > use 'block-thorough' which runs qed, vmdk & vpc tests.
> >
> > This patch suggests that 'block-slow' and 'block-thorough' are
> > not actually compelling use cases, and should be dropped. ie it
> > is not expected that people need to run all VPC, VMDK and QED
> > tests at the same time. Instead a more useful feature is the
> > ability to run all tests for a given format. Further the 'auto'
> > filtering should only apply in the default 'block' target/suite.
> >
> > IOW, with this patch we get the follows meson suites:
> >
> > * 'block' - 'auto' tests for any format listed as 'quick'
> > Currently just qcow2 'auto' tests
> > * 'block-$FORMAT' - ALL tests for the given $FORMAT, for each
> > of qcow2, raw, qed, vmdk & vpc
> > * 'slow' - ALL tests for formats tagged with 'quick' or 'slow'
> > * 'thorough' - ALL tests formats tagged with 'thorough'
> >
> > This corresponds to the following make targets.
> >
> > * 'make check-block'
> >
> > => runs only 'auto' qcow2 tests (unchanged)
> >
> > * 'make check-block SPEED=thorough'
> >
> > => runs all 'qed', 'vmdk', 'vpc' tests (unchanged)
>
> Also all qcow2 and raw tests, right?
Sigh, I made a mistake here, this should read:
* 'make check' / 'make check-block'
=> runs only 'auto' qcow2 tests (unchanged)
The 'make check-block SPEED=thorough/slow' feature was removed
on the basis that running all targets at once is not that
useful, instead the 'check-block-$FORMAT' targets are
replacing it. IOW the meson 'block-slow' and 'block-through'
suites are gone.
>
> Quotes are a bit inconsistent here in the commit message. Sometimes
> you use quotes for format names like here, but in other places you use
> quotes for 'auto' and leave format names without quotes.
>
> > * 'make check-block SPEED=slow'
> >
> > => runs all 'raw' tests (unchanged)
> > => runs all 'qcow2' tests (previously was only 'auto' tests)
> >
> > * 'make check-block-qcow2'
> >
> > => runs all qcow2 tests (new feature)
> >
> > * 'make check-block-raw'
> >
> > => runs all raw tests (new feature)
> >
> > * 'make check-block-vpc'
> >
> > => runs all vpc tests (new feature)
> >
> > * 'make check-block-qed'
> >
> > => runs all qed tests (new feature)
> >
> > * 'make check-block-vmdk'
> >
> > => runs all vmdk tests (new feature)
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> > tests/qemu-iotests/meson.build | 48 ++++++++++++++++++++++++++--------
> > 1 file changed, 37 insertions(+), 11 deletions(-)
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 15/10/2025 15.44, Daniel P. Berrangé wrote: > On Tue, Oct 14, 2025 at 10:26:49AM +0200, Kevin Wolf wrote: >> Am 08.10.2025 um 13:35 hat Daniel P. Berrangé geschrieben: >>> Currently there are 5 block formats, each classified as either quick, >>> slow or thorough. This classification then determines what tests we >>> add to meson suites. The suites are as follows >>> >>> * block >>> >>> => tests listed by 'check -g auto -n' for 'quick' formats >>> >>> * slow, block-slow >>> >>> => tests listed by 'check -n' for 'slow' formats or 'check -g auto -n' >>> for 'quick' formats >>> >>> * thorough, block-thorough >>> >>> => tests listed by 'check -n' for 'thorough' formats >>> >>> The pairs of suites 'slow' / 'block-slow' and 'thorough' / 'block-thorough' >>> match in terms of what tests are enabled. The only difference is whether >>> non-block related tests are also in the suite. >>> >>> There are two problems with this >>> >>> * If a format is classified as 'quick', we don't expose any >>> meson suite for running *all* tests, only the 'auto' tests. >>> >>> eg there is no suite to run all qcow2 tests, only 'quick' >>> tests can be run via meson, even if using 'SPEED=slow' we >>> still filter to only 'auto' tests. >>> >>> * There is no suite that allows running all tests for a given >>> format. >>> >>> eg there is no suite to run only 'raw' tests - you can only >>> use 'block-slow' which runs both raw and "auto" qcow2 tests. >>> >>> eg there is no suite to run only 'vpc' tests - you can only >>> use 'block-thorough' which runs qed, vmdk & vpc tests. >>> >>> This patch suggests that 'block-slow' and 'block-thorough' are >>> not actually compelling use cases, and should be dropped. ie it >>> is not expected that people need to run all VPC, VMDK and QED >>> tests at the same time. Instead a more useful feature is the >>> ability to run all tests for a given format. Further the 'auto' >>> filtering should only apply in the default 'block' target/suite. >>> >>> IOW, with this patch we get the follows meson suites: >>> >>> * 'block' - 'auto' tests for any format listed as 'quick' >>> Currently just qcow2 'auto' tests >>> * 'block-$FORMAT' - ALL tests for the given $FORMAT, for each >>> of qcow2, raw, qed, vmdk & vpc >>> * 'slow' - ALL tests for formats tagged with 'quick' or 'slow' >>> * 'thorough' - ALL tests formats tagged with 'thorough' >>> >>> This corresponds to the following make targets. >>> >>> * 'make check-block' >>> >>> => runs only 'auto' qcow2 tests (unchanged) >>> >>> * 'make check-block SPEED=thorough' >>> >>> => runs all 'qed', 'vmdk', 'vpc' tests (unchanged) >> >> Also all qcow2 and raw tests, right? > > Sigh, I made a mistake here, this should read: > > * 'make check' / 'make check-block' > > => runs only 'auto' qcow2 tests (unchanged) > > The 'make check-block SPEED=thorough/slow' feature was removed > on the basis that running all targets at once is not that > useful, instead the 'check-block-$FORMAT' targets are > replacing it. IOW the meson 'block-slow' and 'block-through' > suites are gone. Can we still run all tests (i.e. iotests, qtests, unit tests and functional tests in parallel) at once with a simple "make check SPEED=thorough" ? If not, that would be a regression for me. Thomas
On Wed, Oct 15, 2025 at 03:47:26PM +0200, Thomas Huth wrote: > On 15/10/2025 15.44, Daniel P. Berrangé wrote: > > On Tue, Oct 14, 2025 at 10:26:49AM +0200, Kevin Wolf wrote: > > > Am 08.10.2025 um 13:35 hat Daniel P. Berrangé geschrieben: > > > > Currently there are 5 block formats, each classified as either quick, > > > > slow or thorough. This classification then determines what tests we > > > > add to meson suites. The suites are as follows > > > > > > > > * block > > > > > > > > => tests listed by 'check -g auto -n' for 'quick' formats > > > > > > > > * slow, block-slow > > > > > > > > => tests listed by 'check -n' for 'slow' formats or 'check -g auto -n' > > > > for 'quick' formats > > > > > > > > * thorough, block-thorough > > > > > > > > => tests listed by 'check -n' for 'thorough' formats > > > > > > > > The pairs of suites 'slow' / 'block-slow' and 'thorough' / 'block-thorough' > > > > match in terms of what tests are enabled. The only difference is whether > > > > non-block related tests are also in the suite. > > > > > > > > There are two problems with this > > > > > > > > * If a format is classified as 'quick', we don't expose any > > > > meson suite for running *all* tests, only the 'auto' tests. > > > > > > > > eg there is no suite to run all qcow2 tests, only 'quick' > > > > tests can be run via meson, even if using 'SPEED=slow' we > > > > still filter to only 'auto' tests. > > > > > > > > * There is no suite that allows running all tests for a given > > > > format. > > > > > > > > eg there is no suite to run only 'raw' tests - you can only > > > > use 'block-slow' which runs both raw and "auto" qcow2 tests. > > > > > > > > eg there is no suite to run only 'vpc' tests - you can only > > > > use 'block-thorough' which runs qed, vmdk & vpc tests. > > > > > > > > This patch suggests that 'block-slow' and 'block-thorough' are > > > > not actually compelling use cases, and should be dropped. ie it > > > > is not expected that people need to run all VPC, VMDK and QED > > > > tests at the same time. Instead a more useful feature is the > > > > ability to run all tests for a given format. Further the 'auto' > > > > filtering should only apply in the default 'block' target/suite. > > > > > > > > IOW, with this patch we get the follows meson suites: > > > > > > > > * 'block' - 'auto' tests for any format listed as 'quick' > > > > Currently just qcow2 'auto' tests > > > > * 'block-$FORMAT' - ALL tests for the given $FORMAT, for each > > > > of qcow2, raw, qed, vmdk & vpc > > > > * 'slow' - ALL tests for formats tagged with 'quick' or 'slow' > > > > * 'thorough' - ALL tests formats tagged with 'thorough' > > > > > > > > This corresponds to the following make targets. > > > > > > > > * 'make check-block' > > > > > > > > => runs only 'auto' qcow2 tests (unchanged) > > > > > > > > * 'make check-block SPEED=thorough' > > > > > > > > => runs all 'qed', 'vmdk', 'vpc' tests (unchanged) > > > > > > Also all qcow2 and raw tests, right? > > > > Sigh, I made a mistake here, this should read: > > * 'make check' / 'make check-block' > > => runs only 'auto' qcow2 tests (unchanged) > > > > The 'make check-block SPEED=thorough/slow' feature was removed > > on the basis that running all targets at once is not that > > useful, instead the 'check-block-$FORMAT' targets are > > replacing it. IOW the meson 'block-slow' and 'block-through' > > suites are gone. > > Can we still run all tests (i.e. iotests, qtests, unit tests and functional > tests in parallel) at once with a simple "make check SPEED=thorough" ? If > not, that would be a regression for me. Yes, the tests are still added to the 'thorough' and 'slow' meson suites, only the 'block-thorough' and 'block-slow' suites were removed. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
© 2016 - 2025 Red Hat, Inc.