Currently tests can be classified into three speed groups depending on
whether the meson suite name ends in '-slow' or '-thorough' or neither.

This gets turned into make targets that match the name of the meson
suite, with the speed suffix stripped. e.g.

 * suite=block ->  'make check-block'
 * suite=block-slow -> 'make check-block SPEED=slow'
 * suite=block-thorough -> 'make check-block SPEED=thorough'

The set of tests under the "thorough" speed, however, can get rather
large and it would be useful to have a way to expose further make
targets for directly running a particular subset of tests.

This needs a way to run a target without requiring the SPEED variable,
while also not having them enabled by default as if they were 'quick'
tests.

This modifies mtest2make.py to support this idea by allowing for a new
suffix '-optional' on a suite. When this is present, a correspondingly
named make target will be created without the '-optional' suffix which
will never be run automatically.

This is intended to be combined with use of other suites. For example,
a single NBD test might be added to two suites, 'block-thorough' and
'block-nbd-optional'.

This would allow running it as part of all the block tests with
'make check-block SPEED=thorough', and as part of a standalone target
'make check-block-nbd'.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 scripts/mtest2make.py | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 915f02d600..383ea68b16 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -22,7 +22,7 @@ def names(self, base):
 print(r'''
 SPEED = quick
 
-.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
+.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
 .speed.slow = $(sort $(filter-out %-thorough, $1))
 .speed.thorough = $(sort $1)
 
@@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
             s = s[:-9]
             suites[s].speeds.add('thorough')
 
+def target_name(suite):
+    if suite.endswith('-optional'):
+        return suite[0:-9]
+    return suite
+
 def emit_prolog(suites, prefix):
-    all_targets = ' '.join((f'{prefix}-{k}'
+    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
                             for k in sorted(suites.keys())))
-    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
+    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
                         for k in sorted(suites.keys())))
     print()
     print(f'all-{prefix}-targets = {all_targets}')
@@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
     print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
 
 def emit_suite(name, suite, prefix):
+    tgtname = target_name(name)
     deps = ' '.join(sorted(suite.deps))
     print()
-    print(f'.{prefix}-{name}.deps = {deps}')
-    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
+    print(f'.{prefix}-{tgtname}.deps = {deps}')
+    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
 
     names = ' '.join(sorted(suite.names(name)))
-    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
-    if not name.endswith('-slow') and not name.endswith('-thorough'):
+    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
+    if not name.endswith('-slow') and \
+       not name.endswith('-thorough') and \
+       not name.endswith('-optional'):
         targets += f' {prefix} {prefix}-report.junit.xml'
     print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
     # for the "base" suite possibly add FOO-slow and FOO-thorough
-- 
2.53.0

On 4/24/2026 8:41 AM, Daniel P. Berrangé wrote:
> Currently tests can be classified into three speed groups depending on
> whether the meson suite name ends in '-slow' or '-thorough' or neither.
> 
> This gets turned into make targets that match the name of the meson
> suite, with the speed suffix stripped. e.g.
> 
>  * suite=block ->  'make check-block'
>  * suite=block-slow -> 'make check-block SPEED=slow'
>  * suite=block-thorough -> 'make check-block SPEED=thorough'
> 
> The set of tests under the "thorough" speed, however, can get rather
> large and it would be useful to have a way to expose further make
> targets for directly running a particular subset of tests.
> 
> This needs a way to run a target without requiring the SPEED variable,
> while also not having them enabled by default as if they were 'quick'
> tests.
> 
> This modifies mtest2make.py to support this idea by allowing for a new
> suffix '-optional' on a suite. When this is present, a correspondingly
> named make target will be created without the '-optional' suffix which
> will never be run automatically.
> 
> This is intended to be combined with use of other suites. For example,
> a single NBD test might be added to two suites, 'block-thorough' and
> 'block-nbd-optional'.
> 
> This would allow running it as part of all the block tests with
> 'make check-block SPEED=thorough', and as part of a standalone target
> 'make check-block-nbd'.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  scripts/mtest2make.py | 22 +++++++++++++++-------
>  1 file changed, 15 insertions(+), 7 deletions(-)
> 
> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
> index 915f02d600..383ea68b16 100644
> --- a/scripts/mtest2make.py
> +++ b/scripts/mtest2make.py
> @@ -22,7 +22,7 @@ def names(self, base):
>  print(r'''
>  SPEED = quick
>  
> -.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
> +.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
>  .speed.slow = $(sort $(filter-out %-thorough, $1))
>  .speed.thorough = $(sort $1)
>  
> @@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
>              s = s[:-9]
>              suites[s].speeds.add('thorough')
>  
> +def target_name(suite):
> +    if suite.endswith('-optional'):
> +        return suite[0:-9]
> +    return suite
> +
>  def emit_prolog(suites, prefix):
> -    all_targets = ' '.join((f'{prefix}-{k}'
> +    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
>                              for k in sorted(suites.keys())))
> -    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
> +    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
>                          for k in sorted(suites.keys())))
>      print()
>      print(f'all-{prefix}-targets = {all_targets}')
> @@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
>      print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
>  
>  def emit_suite(name, suite, prefix):
> +    tgtname = target_name(name)
>      deps = ' '.join(sorted(suite.deps))
>      print()
> -    print(f'.{prefix}-{name}.deps = {deps}')
> -    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
> +    print(f'.{prefix}-{tgtname}.deps = {deps}')
> +    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
>  
>      names = ' '.join(sorted(suite.names(name)))
> -    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
> -    if not name.endswith('-slow') and not name.endswith('-thorough'):
> +    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
> +    if not name.endswith('-slow') and \
> +       not name.endswith('-thorough') and \
> +       not name.endswith('-optional'):
>          targets += f' {prefix} {prefix}-report.junit.xml'
>      print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
>      # for the "base" suite possibly add FOO-slow and FOO-thorough

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>

Not blocking this patch, thus reviewed, but I think it would be a better
abstraction to use meson suites for this (a test can be added in
multiple suites), instead of adding more clunk to makefile layer.

Regards,
Pierrick

On Tue, May 12, 2026 at 08:45:42AM -0700, Pierrick Bouvier wrote:
> On 4/24/2026 8:41 AM, Daniel P. Berrangé wrote:
> > Currently tests can be classified into three speed groups depending on
> > whether the meson suite name ends in '-slow' or '-thorough' or neither.
> > 
> > This gets turned into make targets that match the name of the meson
> > suite, with the speed suffix stripped. e.g.
> > 
> >  * suite=block ->  'make check-block'
> >  * suite=block-slow -> 'make check-block SPEED=slow'
> >  * suite=block-thorough -> 'make check-block SPEED=thorough'
> > 
> > The set of tests under the "thorough" speed, however, can get rather
> > large and it would be useful to have a way to expose further make
> > targets for directly running a particular subset of tests.
> > 
> > This needs a way to run a target without requiring the SPEED variable,
> > while also not having them enabled by default as if they were 'quick'
> > tests.
> > 
> > This modifies mtest2make.py to support this idea by allowing for a new
> > suffix '-optional' on a suite. When this is present, a correspondingly
> > named make target will be created without the '-optional' suffix which
> > will never be run automatically.
> > 
> > This is intended to be combined with use of other suites. For example,
> > a single NBD test might be added to two suites, 'block-thorough' and
> > 'block-nbd-optional'.
> > 
> > This would allow running it as part of all the block tests with
> > 'make check-block SPEED=thorough', and as part of a standalone target
> > 'make check-block-nbd'.
> > 
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >  scripts/mtest2make.py | 22 +++++++++++++++-------
> >  1 file changed, 15 insertions(+), 7 deletions(-)
> > 
> > diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
> > index 915f02d600..383ea68b16 100644
> > --- a/scripts/mtest2make.py
> > +++ b/scripts/mtest2make.py
> > @@ -22,7 +22,7 @@ def names(self, base):
> >  print(r'''
> >  SPEED = quick
> >  
> > -.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
> > +.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
> >  .speed.slow = $(sort $(filter-out %-thorough, $1))
> >  .speed.thorough = $(sort $1)
> >  
> > @@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
> >              s = s[:-9]
> >              suites[s].speeds.add('thorough')
> >  
> > +def target_name(suite):
> > +    if suite.endswith('-optional'):
> > +        return suite[0:-9]
> > +    return suite
> > +
> >  def emit_prolog(suites, prefix):
> > -    all_targets = ' '.join((f'{prefix}-{k}'
> > +    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
> >                              for k in sorted(suites.keys())))
> > -    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
> > +    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
> >                          for k in sorted(suites.keys())))
> >      print()
> >      print(f'all-{prefix}-targets = {all_targets}')
> > @@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
> >      print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
> >  
> >  def emit_suite(name, suite, prefix):
> > +    tgtname = target_name(name)
> >      deps = ' '.join(sorted(suite.deps))
> >      print()
> > -    print(f'.{prefix}-{name}.deps = {deps}')
> > -    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
> > +    print(f'.{prefix}-{tgtname}.deps = {deps}')
> > +    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
> >  
> >      names = ' '.join(sorted(suite.names(name)))
> > -    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
> > -    if not name.endswith('-slow') and not name.endswith('-thorough'):
> > +    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
> > +    if not name.endswith('-slow') and \
> > +       not name.endswith('-thorough') and \
> > +       not name.endswith('-optional'):
> >          targets += f' {prefix} {prefix}-report.junit.xml'
> >      print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
> >      # for the "base" suite possibly add FOO-slow and FOO-thorough
> 
> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> 
> Not blocking this patch, thus reviewed, but I think it would be a better
> abstraction to use meson suites for this (a test can be added in
> multiple suites), instead of adding more clunk to makefile layer.

This *is* using meson suites. The mtest2make script takes the list
of meson suites and generates makefile targets which can be used to
select execution of the meson suites. This change is exposing more
of the suites as makefile targets, since make is our primary entry
point.


With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

On 5/13/2026 3:08 AM, Daniel P. Berrangé wrote:
> On Tue, May 12, 2026 at 08:45:42AM -0700, Pierrick Bouvier wrote:
>> On 4/24/2026 8:41 AM, Daniel P. Berrangé wrote:
>>> Currently tests can be classified into three speed groups depending on
>>> whether the meson suite name ends in '-slow' or '-thorough' or neither.
>>>
>>> This gets turned into make targets that match the name of the meson
>>> suite, with the speed suffix stripped. e.g.
>>>
>>>  * suite=block ->  'make check-block'
>>>  * suite=block-slow -> 'make check-block SPEED=slow'
>>>  * suite=block-thorough -> 'make check-block SPEED=thorough'
>>>
>>> The set of tests under the "thorough" speed, however, can get rather
>>> large and it would be useful to have a way to expose further make
>>> targets for directly running a particular subset of tests.
>>>
>>> This needs a way to run a target without requiring the SPEED variable,
>>> while also not having them enabled by default as if they were 'quick'
>>> tests.
>>>
>>> This modifies mtest2make.py to support this idea by allowing for a new
>>> suffix '-optional' on a suite. When this is present, a correspondingly
>>> named make target will be created without the '-optional' suffix which
>>> will never be run automatically.
>>>
>>> This is intended to be combined with use of other suites. For example,
>>> a single NBD test might be added to two suites, 'block-thorough' and
>>> 'block-nbd-optional'.
>>>
>>> This would allow running it as part of all the block tests with
>>> 'make check-block SPEED=thorough', and as part of a standalone target
>>> 'make check-block-nbd'.
>>>
>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>> ---
>>>  scripts/mtest2make.py | 22 +++++++++++++++-------
>>>  1 file changed, 15 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
>>> index 915f02d600..383ea68b16 100644
>>> --- a/scripts/mtest2make.py
>>> +++ b/scripts/mtest2make.py
>>> @@ -22,7 +22,7 @@ def names(self, base):
>>>  print(r'''
>>>  SPEED = quick
>>>  
>>> -.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
>>> +.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
>>>  .speed.slow = $(sort $(filter-out %-thorough, $1))
>>>  .speed.thorough = $(sort $1)
>>>  
>>> @@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
>>>              s = s[:-9]
>>>              suites[s].speeds.add('thorough')
>>>  
>>> +def target_name(suite):
>>> +    if suite.endswith('-optional'):
>>> +        return suite[0:-9]
>>> +    return suite
>>> +
>>>  def emit_prolog(suites, prefix):
>>> -    all_targets = ' '.join((f'{prefix}-{k}'
>>> +    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
>>>                              for k in sorted(suites.keys())))
>>> -    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
>>> +    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
>>>                          for k in sorted(suites.keys())))
>>>      print()
>>>      print(f'all-{prefix}-targets = {all_targets}')
>>> @@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
>>>      print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
>>>  
>>>  def emit_suite(name, suite, prefix):
>>> +    tgtname = target_name(name)
>>>      deps = ' '.join(sorted(suite.deps))
>>>      print()
>>> -    print(f'.{prefix}-{name}.deps = {deps}')
>>> -    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
>>> +    print(f'.{prefix}-{tgtname}.deps = {deps}')
>>> +    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
>>>  
>>>      names = ' '.join(sorted(suite.names(name)))
>>> -    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
>>> -    if not name.endswith('-slow') and not name.endswith('-thorough'):
>>> +    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
>>> +    if not name.endswith('-slow') and \
>>> +       not name.endswith('-thorough') and \
>>> +       not name.endswith('-optional'):
>>>          targets += f' {prefix} {prefix}-report.junit.xml'
>>>      print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
>>>      # for the "base" suite possibly add FOO-slow and FOO-thorough
>>
>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>
>> Not blocking this patch, thus reviewed, but I think it would be a better
>> abstraction to use meson suites for this (a test can be added in
>> multiple suites), instead of adding more clunk to makefile layer.
> 
> This *is* using meson suites. The mtest2make script takes the list
> of meson suites and generates makefile targets which can be used to
> select execution of the meson suites. This change is exposing more
> of the suites as makefile targets, since make is our primary entry
> point.
> 
> 
> With regards,
> Daniel

Maybe I can't read this patch correctly, but from it, and the message
description, I understand that -optional is a pure Makefile abstraction,
and does not reflect any real meson suite.

```
This modifies mtest2make.py to support this idea by allowing for a new
suffix '-optional' on a suite. When this is present, a correspondingly
named make target will be created without the '-optional' suffix which
will never be run automatically.
```

Regards,
Pierrick

On Wed, May 13, 2026 at 08:49:39AM -0700, Pierrick Bouvier wrote:
> On 5/13/2026 3:08 AM, Daniel P. Berrangé wrote:
> > On Tue, May 12, 2026 at 08:45:42AM -0700, Pierrick Bouvier wrote:
> >> On 4/24/2026 8:41 AM, Daniel P. Berrangé wrote:
> >>> Currently tests can be classified into three speed groups depending on
> >>> whether the meson suite name ends in '-slow' or '-thorough' or neither.
> >>>
> >>> This gets turned into make targets that match the name of the meson
> >>> suite, with the speed suffix stripped. e.g.
> >>>
> >>>  * suite=block ->  'make check-block'
> >>>  * suite=block-slow -> 'make check-block SPEED=slow'
> >>>  * suite=block-thorough -> 'make check-block SPEED=thorough'
> >>>
> >>> The set of tests under the "thorough" speed, however, can get rather
> >>> large and it would be useful to have a way to expose further make
> >>> targets for directly running a particular subset of tests.
> >>>
> >>> This needs a way to run a target without requiring the SPEED variable,
> >>> while also not having them enabled by default as if they were 'quick'
> >>> tests.
> >>>
> >>> This modifies mtest2make.py to support this idea by allowing for a new
> >>> suffix '-optional' on a suite. When this is present, a correspondingly
> >>> named make target will be created without the '-optional' suffix which
> >>> will never be run automatically.
> >>>
> >>> This is intended to be combined with use of other suites. For example,
> >>> a single NBD test might be added to two suites, 'block-thorough' and
> >>> 'block-nbd-optional'.
> >>>
> >>> This would allow running it as part of all the block tests with
> >>> 'make check-block SPEED=thorough', and as part of a standalone target
> >>> 'make check-block-nbd'.
> >>>
> >>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> >>> ---
> >>>  scripts/mtest2make.py | 22 +++++++++++++++-------
> >>>  1 file changed, 15 insertions(+), 7 deletions(-)
> >>>
> >>> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
> >>> index 915f02d600..383ea68b16 100644
> >>> --- a/scripts/mtest2make.py
> >>> +++ b/scripts/mtest2make.py
> >>> @@ -22,7 +22,7 @@ def names(self, base):
> >>>  print(r'''
> >>>  SPEED = quick
> >>>  
> >>> -.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
> >>> +.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
> >>>  .speed.slow = $(sort $(filter-out %-thorough, $1))
> >>>  .speed.thorough = $(sort $1)
> >>>  
> >>> @@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
> >>>              s = s[:-9]
> >>>              suites[s].speeds.add('thorough')
> >>>  
> >>> +def target_name(suite):
> >>> +    if suite.endswith('-optional'):
> >>> +        return suite[0:-9]
> >>> +    return suite
> >>> +
> >>>  def emit_prolog(suites, prefix):
> >>> -    all_targets = ' '.join((f'{prefix}-{k}'
> >>> +    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
> >>>                              for k in sorted(suites.keys())))
> >>> -    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
> >>> +    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
> >>>                          for k in sorted(suites.keys())))
> >>>      print()
> >>>      print(f'all-{prefix}-targets = {all_targets}')
> >>> @@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
> >>>      print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
> >>>  
> >>>  def emit_suite(name, suite, prefix):
> >>> +    tgtname = target_name(name)
> >>>      deps = ' '.join(sorted(suite.deps))
> >>>      print()
> >>> -    print(f'.{prefix}-{name}.deps = {deps}')
> >>> -    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
> >>> +    print(f'.{prefix}-{tgtname}.deps = {deps}')
> >>> +    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
> >>>  
> >>>      names = ' '.join(sorted(suite.names(name)))
> >>> -    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
> >>> -    if not name.endswith('-slow') and not name.endswith('-thorough'):
> >>> +    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
> >>> +    if not name.endswith('-slow') and \
> >>> +       not name.endswith('-thorough') and \
> >>> +       not name.endswith('-optional'):
> >>>          targets += f' {prefix} {prefix}-report.junit.xml'
> >>>      print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
> >>>      # for the "base" suite possibly add FOO-slow and FOO-thorough
> >>
> >> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> >>
> >> Not blocking this patch, thus reviewed, but I think it would be a better
> >> abstraction to use meson suites for this (a test can be added in
> >> multiple suites), instead of adding more clunk to makefile layer.
> > 
> > This *is* using meson suites. The mtest2make script takes the list
> > of meson suites and generates makefile targets which can be used to
> > select execution of the meson suites. This change is exposing more
> > of the suites as makefile targets, since make is our primary entry
> > point.
> > 
> > 
> > With regards,
> > Daniel
> 
> Maybe I can't read this patch correctly, but from it, and the message
> description, I understand that -optional is a pure Makefile abstraction,
> and does not reflect any real meson suite.

Actually its the other way around. In the next patch in this series
we create meson suites like  "block-nbd-optional" and that gets turned
into a makefile target  "check-block-nbd". The "optional" part of the
meson suite name determines that "check-block-nbd" is NOT added as a
depedency of "check".  IOW, it doesn't get run by "make check", but
must be optionally run through "make check-block-nbd" when needed.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

On 5/13/2026 10:15 AM, Daniel P. Berrangé wrote:
> On Wed, May 13, 2026 at 08:49:39AM -0700, Pierrick Bouvier wrote:
>> On 5/13/2026 3:08 AM, Daniel P. Berrangé wrote:
>>> On Tue, May 12, 2026 at 08:45:42AM -0700, Pierrick Bouvier wrote:
>>>> On 4/24/2026 8:41 AM, Daniel P. Berrangé wrote:
>>>>> Currently tests can be classified into three speed groups depending on
>>>>> whether the meson suite name ends in '-slow' or '-thorough' or neither.
>>>>>
>>>>> This gets turned into make targets that match the name of the meson
>>>>> suite, with the speed suffix stripped. e.g.
>>>>>
>>>>>  * suite=block ->  'make check-block'
>>>>>  * suite=block-slow -> 'make check-block SPEED=slow'
>>>>>  * suite=block-thorough -> 'make check-block SPEED=thorough'
>>>>>
>>>>> The set of tests under the "thorough" speed, however, can get rather
>>>>> large and it would be useful to have a way to expose further make
>>>>> targets for directly running a particular subset of tests.
>>>>>
>>>>> This needs a way to run a target without requiring the SPEED variable,
>>>>> while also not having them enabled by default as if they were 'quick'
>>>>> tests.
>>>>>
>>>>> This modifies mtest2make.py to support this idea by allowing for a new
>>>>> suffix '-optional' on a suite. When this is present, a correspondingly
>>>>> named make target will be created without the '-optional' suffix which
>>>>> will never be run automatically.
>>>>>
>>>>> This is intended to be combined with use of other suites. For example,
>>>>> a single NBD test might be added to two suites, 'block-thorough' and
>>>>> 'block-nbd-optional'.
>>>>>
>>>>> This would allow running it as part of all the block tests with
>>>>> 'make check-block SPEED=thorough', and as part of a standalone target
>>>>> 'make check-block-nbd'.
>>>>>
>>>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>>>> ---
>>>>>  scripts/mtest2make.py | 22 +++++++++++++++-------
>>>>>  1 file changed, 15 insertions(+), 7 deletions(-)
>>>>>
>>>>> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
>>>>> index 915f02d600..383ea68b16 100644
>>>>> --- a/scripts/mtest2make.py
>>>>> +++ b/scripts/mtest2make.py
>>>>> @@ -22,7 +22,7 @@ def names(self, base):
>>>>>  print(r'''
>>>>>  SPEED = quick
>>>>>  
>>>>> -.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
>>>>> +.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
>>>>>  .speed.slow = $(sort $(filter-out %-thorough, $1))
>>>>>  .speed.thorough = $(sort $1)
>>>>>  
>>>>> @@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
>>>>>              s = s[:-9]
>>>>>              suites[s].speeds.add('thorough')
>>>>>  
>>>>> +def target_name(suite):
>>>>> +    if suite.endswith('-optional'):
>>>>> +        return suite[0:-9]
>>>>> +    return suite
>>>>> +
>>>>>  def emit_prolog(suites, prefix):
>>>>> -    all_targets = ' '.join((f'{prefix}-{k}'
>>>>> +    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
>>>>>                              for k in sorted(suites.keys())))
>>>>> -    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
>>>>> +    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
>>>>>                          for k in sorted(suites.keys())))
>>>>>      print()
>>>>>      print(f'all-{prefix}-targets = {all_targets}')
>>>>> @@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
>>>>>      print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
>>>>>  
>>>>>  def emit_suite(name, suite, prefix):
>>>>> +    tgtname = target_name(name)
>>>>>      deps = ' '.join(sorted(suite.deps))
>>>>>      print()
>>>>> -    print(f'.{prefix}-{name}.deps = {deps}')
>>>>> -    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
>>>>> +    print(f'.{prefix}-{tgtname}.deps = {deps}')
>>>>> +    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
>>>>>  
>>>>>      names = ' '.join(sorted(suite.names(name)))
>>>>> -    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
>>>>> -    if not name.endswith('-slow') and not name.endswith('-thorough'):
>>>>> +    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
>>>>> +    if not name.endswith('-slow') and \
>>>>> +       not name.endswith('-thorough') and \
>>>>> +       not name.endswith('-optional'):
>>>>>          targets += f' {prefix} {prefix}-report.junit.xml'
>>>>>      print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
>>>>>      # for the "base" suite possibly add FOO-slow and FOO-thorough
>>>>
>>>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>>>
>>>> Not blocking this patch, thus reviewed, but I think it would be a better
>>>> abstraction to use meson suites for this (a test can be added in
>>>> multiple suites), instead of adding more clunk to makefile layer.
>>>
>>> This *is* using meson suites. The mtest2make script takes the list
>>> of meson suites and generates makefile targets which can be used to
>>> select execution of the meson suites. This change is exposing more
>>> of the suites as makefile targets, since make is our primary entry
>>> point.
>>>
>>>
>>> With regards,
>>> Daniel
>>
>> Maybe I can't read this patch correctly, but from it, and the message
>> description, I understand that -optional is a pure Makefile abstraction,
>> and does not reflect any real meson suite.
> 
> Actually its the other way around. In the next patch in this series
> we create meson suites like  "block-nbd-optional" and that gets turned
> into a makefile target  "check-block-nbd". The "optional" part of the
> meson suite name determines that "check-block-nbd" is NOT added as a
> depedency of "check".  IOW, it doesn't get run by "make check", but
> must be optionally run through "make check-block-nbd" when needed.
>

Oh right.
Just to double check, will everything still run when using
make check-block SPEED=thorough, or is there some new behaviour on this
side?

> With regards,
> Daniel

On Wed, May 13, 2026 at 10:23:07AM -0700, Pierrick Bouvier wrote:
> On 5/13/2026 10:15 AM, Daniel P. Berrangé wrote:
> > On Wed, May 13, 2026 at 08:49:39AM -0700, Pierrick Bouvier wrote:
> >> On 5/13/2026 3:08 AM, Daniel P. Berrangé wrote:
> >>> On Tue, May 12, 2026 at 08:45:42AM -0700, Pierrick Bouvier wrote:
> >>>> On 4/24/2026 8:41 AM, Daniel P. Berrangé wrote:
> >>>>> Currently tests can be classified into three speed groups depending on
> >>>>> whether the meson suite name ends in '-slow' or '-thorough' or neither.
> >>>>>
> >>>>> This gets turned into make targets that match the name of the meson
> >>>>> suite, with the speed suffix stripped. e.g.
> >>>>>
> >>>>>  * suite=block ->  'make check-block'
> >>>>>  * suite=block-slow -> 'make check-block SPEED=slow'
> >>>>>  * suite=block-thorough -> 'make check-block SPEED=thorough'
> >>>>>
> >>>>> The set of tests under the "thorough" speed, however, can get rather
> >>>>> large and it would be useful to have a way to expose further make
> >>>>> targets for directly running a particular subset of tests.
> >>>>>
> >>>>> This needs a way to run a target without requiring the SPEED variable,
> >>>>> while also not having them enabled by default as if they were 'quick'
> >>>>> tests.
> >>>>>
> >>>>> This modifies mtest2make.py to support this idea by allowing for a new
> >>>>> suffix '-optional' on a suite. When this is present, a correspondingly
> >>>>> named make target will be created without the '-optional' suffix which
> >>>>> will never be run automatically.
> >>>>>
> >>>>> This is intended to be combined with use of other suites. For example,
> >>>>> a single NBD test might be added to two suites, 'block-thorough' and
> >>>>> 'block-nbd-optional'.
> >>>>>
> >>>>> This would allow running it as part of all the block tests with
> >>>>> 'make check-block SPEED=thorough', and as part of a standalone target
> >>>>> 'make check-block-nbd'.
> >>>>>
> >>>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> >>>>> ---
> >>>>>  scripts/mtest2make.py | 22 +++++++++++++++-------
> >>>>>  1 file changed, 15 insertions(+), 7 deletions(-)
> >>>>>
> >>>>> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
> >>>>> index 915f02d600..383ea68b16 100644
> >>>>> --- a/scripts/mtest2make.py
> >>>>> +++ b/scripts/mtest2make.py
> >>>>> @@ -22,7 +22,7 @@ def names(self, base):
> >>>>>  print(r'''
> >>>>>  SPEED = quick
> >>>>>  
> >>>>> -.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
> >>>>> +.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
> >>>>>  .speed.slow = $(sort $(filter-out %-thorough, $1))
> >>>>>  .speed.thorough = $(sort $1)
> >>>>>  
> >>>>> @@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
> >>>>>              s = s[:-9]
> >>>>>              suites[s].speeds.add('thorough')
> >>>>>  
> >>>>> +def target_name(suite):
> >>>>> +    if suite.endswith('-optional'):
> >>>>> +        return suite[0:-9]
> >>>>> +    return suite
> >>>>> +
> >>>>>  def emit_prolog(suites, prefix):
> >>>>> -    all_targets = ' '.join((f'{prefix}-{k}'
> >>>>> +    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
> >>>>>                              for k in sorted(suites.keys())))
> >>>>> -    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
> >>>>> +    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
> >>>>>                          for k in sorted(suites.keys())))
> >>>>>      print()
> >>>>>      print(f'all-{prefix}-targets = {all_targets}')
> >>>>> @@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
> >>>>>      print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
> >>>>>  
> >>>>>  def emit_suite(name, suite, prefix):
> >>>>> +    tgtname = target_name(name)
> >>>>>      deps = ' '.join(sorted(suite.deps))
> >>>>>      print()
> >>>>> -    print(f'.{prefix}-{name}.deps = {deps}')
> >>>>> -    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
> >>>>> +    print(f'.{prefix}-{tgtname}.deps = {deps}')
> >>>>> +    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
> >>>>>  
> >>>>>      names = ' '.join(sorted(suite.names(name)))
> >>>>> -    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
> >>>>> -    if not name.endswith('-slow') and not name.endswith('-thorough'):
> >>>>> +    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
> >>>>> +    if not name.endswith('-slow') and \
> >>>>> +       not name.endswith('-thorough') and \
> >>>>> +       not name.endswith('-optional'):
> >>>>>          targets += f' {prefix} {prefix}-report.junit.xml'
> >>>>>      print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
> >>>>>      # for the "base" suite possibly add FOO-slow and FOO-thorough
> >>>>
> >>>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
> >>>>
> >>>> Not blocking this patch, thus reviewed, but I think it would be a better
> >>>> abstraction to use meson suites for this (a test can be added in
> >>>> multiple suites), instead of adding more clunk to makefile layer.
> >>>
> >>> This *is* using meson suites. The mtest2make script takes the list
> >>> of meson suites and generates makefile targets which can be used to
> >>> select execution of the meson suites. This change is exposing more
> >>> of the suites as makefile targets, since make is our primary entry
> >>> point.
> >>>
> >>>
> >>> With regards,
> >>> Daniel
> >>
> >> Maybe I can't read this patch correctly, but from it, and the message
> >> description, I understand that -optional is a pure Makefile abstraction,
> >> and does not reflect any real meson suite.
> > 
> > Actually its the other way around. In the next patch in this series
> > we create meson suites like  "block-nbd-optional" and that gets turned
> > into a makefile target  "check-block-nbd". The "optional" part of the
> > meson suite name determines that "check-block-nbd" is NOT added as a
> > depedency of "check".  IOW, it doesn't get run by "make check", but
> > must be optionally run through "make check-block-nbd" when needed.
> >
> 
> Oh right.
> Just to double check, will everything still run when using
> make check-block SPEED=thorough, or is there some new behaviour on this
> side?

Yes, this commit doesn't affect that existing make target, so it
operates as before.

Patch 5 does however fix a pre-existing problem - currently even
with SPEED=thorough most of the qcow2 tests didn't get run, only
the "quick" tests. With this series SPEED=thorough correctly does
what people thought it was already doing.

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

On 5/13/2026 10:26 AM, Daniel P. Berrangé wrote:
> On Wed, May 13, 2026 at 10:23:07AM -0700, Pierrick Bouvier wrote:
>> On 5/13/2026 10:15 AM, Daniel P. Berrangé wrote:
>>> On Wed, May 13, 2026 at 08:49:39AM -0700, Pierrick Bouvier wrote:
>>>> On 5/13/2026 3:08 AM, Daniel P. Berrangé wrote:
>>>>> On Tue, May 12, 2026 at 08:45:42AM -0700, Pierrick Bouvier wrote:
>>>>>> On 4/24/2026 8:41 AM, Daniel P. Berrangé wrote:
>>>>>>> Currently tests can be classified into three speed groups depending on
>>>>>>> whether the meson suite name ends in '-slow' or '-thorough' or neither.
>>>>>>>
>>>>>>> This gets turned into make targets that match the name of the meson
>>>>>>> suite, with the speed suffix stripped. e.g.
>>>>>>>
>>>>>>>  * suite=block ->  'make check-block'
>>>>>>>  * suite=block-slow -> 'make check-block SPEED=slow'
>>>>>>>  * suite=block-thorough -> 'make check-block SPEED=thorough'
>>>>>>>
>>>>>>> The set of tests under the "thorough" speed, however, can get rather
>>>>>>> large and it would be useful to have a way to expose further make
>>>>>>> targets for directly running a particular subset of tests.
>>>>>>>
>>>>>>> This needs a way to run a target without requiring the SPEED variable,
>>>>>>> while also not having them enabled by default as if they were 'quick'
>>>>>>> tests.
>>>>>>>
>>>>>>> This modifies mtest2make.py to support this idea by allowing for a new
>>>>>>> suffix '-optional' on a suite. When this is present, a correspondingly
>>>>>>> named make target will be created without the '-optional' suffix which
>>>>>>> will never be run automatically.
>>>>>>>
>>>>>>> This is intended to be combined with use of other suites. For example,
>>>>>>> a single NBD test might be added to two suites, 'block-thorough' and
>>>>>>> 'block-nbd-optional'.
>>>>>>>
>>>>>>> This would allow running it as part of all the block tests with
>>>>>>> 'make check-block SPEED=thorough', and as part of a standalone target
>>>>>>> 'make check-block-nbd'.
>>>>>>>
>>>>>>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>>>>>>> ---
>>>>>>>  scripts/mtest2make.py | 22 +++++++++++++++-------
>>>>>>>  1 file changed, 15 insertions(+), 7 deletions(-)
>>>>>>>
>>>>>>> diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
>>>>>>> index 915f02d600..383ea68b16 100644
>>>>>>> --- a/scripts/mtest2make.py
>>>>>>> +++ b/scripts/mtest2make.py
>>>>>>> @@ -22,7 +22,7 @@ def names(self, base):
>>>>>>>  print(r'''
>>>>>>>  SPEED = quick
>>>>>>>  
>>>>>>> -.speed.quick = $(sort $(filter-out %-slow %-thorough, $1))
>>>>>>> +.speed.quick = $(sort $(filter-out %-slow %-thorough %-optional, $1))
>>>>>>>  .speed.slow = $(sort $(filter-out %-thorough, $1))
>>>>>>>  .speed.thorough = $(sort $1)
>>>>>>>  
>>>>>>> @@ -66,10 +66,15 @@ def process_tests(test, targets, suites):
>>>>>>>              s = s[:-9]
>>>>>>>              suites[s].speeds.add('thorough')
>>>>>>>  
>>>>>>> +def target_name(suite):
>>>>>>> +    if suite.endswith('-optional'):
>>>>>>> +        return suite[0:-9]
>>>>>>> +    return suite
>>>>>>> +
>>>>>>>  def emit_prolog(suites, prefix):
>>>>>>> -    all_targets = ' '.join((f'{prefix}-{k}'
>>>>>>> +    all_targets = ' '.join((f'{prefix}-{target_name(k)}'
>>>>>>>                              for k in sorted(suites.keys())))
>>>>>>> -    all_xml = ' '.join((f'{prefix}-report-{k}.junit.xml'
>>>>>>> +    all_xml = ' '.join((f'{prefix}-report-{target_name(k)}.junit.xml'
>>>>>>>                          for k in sorted(suites.keys())))
>>>>>>>      print()
>>>>>>>      print(f'all-{prefix}-targets = {all_targets}')
>>>>>>> @@ -83,14 +88,17 @@ def emit_prolog(suites, prefix):
>>>>>>>      print(f'\t$(MAKE) {prefix}$* MTESTARGS="$(MTESTARGS) --logbase {prefix}-report$*" && ln -f meson-logs/$@ .')
>>>>>>>  
>>>>>>>  def emit_suite(name, suite, prefix):
>>>>>>> +    tgtname = target_name(name)
>>>>>>>      deps = ' '.join(sorted(suite.deps))
>>>>>>>      print()
>>>>>>> -    print(f'.{prefix}-{name}.deps = {deps}')
>>>>>>> -    print(f'.ninja-goals.check-build += $(.{prefix}-{name}.deps)')
>>>>>>> +    print(f'.{prefix}-{tgtname}.deps = {deps}')
>>>>>>> +    print(f'.ninja-goals.check-build += $(.{prefix}-{tgtname}.deps)')
>>>>>>>  
>>>>>>>      names = ' '.join(sorted(suite.names(name)))
>>>>>>> -    targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml'
>>>>>>> -    if not name.endswith('-slow') and not name.endswith('-thorough'):
>>>>>>> +    targets = f'{prefix}-{tgtname} {prefix}-report-{tgtname}.junit.xml'
>>>>>>> +    if not name.endswith('-slow') and \
>>>>>>> +       not name.endswith('-thorough') and \
>>>>>>> +       not name.endswith('-optional'):
>>>>>>>          targets += f' {prefix} {prefix}-report.junit.xml'
>>>>>>>      print(f'ifneq ($(filter {targets}, $(MAKECMDGOALS)),)')
>>>>>>>      # for the "base" suite possibly add FOO-slow and FOO-thorough
>>>>>>
>>>>>> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@oss.qualcomm.com>
>>>>>>
>>>>>> Not blocking this patch, thus reviewed, but I think it would be a better
>>>>>> abstraction to use meson suites for this (a test can be added in
>>>>>> multiple suites), instead of adding more clunk to makefile layer.
>>>>>
>>>>> This *is* using meson suites. The mtest2make script takes the list
>>>>> of meson suites and generates makefile targets which can be used to
>>>>> select execution of the meson suites. This change is exposing more
>>>>> of the suites as makefile targets, since make is our primary entry
>>>>> point.
>>>>>
>>>>>
>>>>> With regards,
>>>>> Daniel
>>>>
>>>> Maybe I can't read this patch correctly, but from it, and the message
>>>> description, I understand that -optional is a pure Makefile abstraction,
>>>> and does not reflect any real meson suite.
>>>
>>> Actually its the other way around. In the next patch in this series
>>> we create meson suites like  "block-nbd-optional" and that gets turned
>>> into a makefile target  "check-block-nbd". The "optional" part of the
>>> meson suite name determines that "check-block-nbd" is NOT added as a
>>> depedency of "check".  IOW, it doesn't get run by "make check", but
>>> must be optionally run through "make check-block-nbd" when needed.
>>>
>>
>> Oh right.
>> Just to double check, will everything still run when using
>> make check-block SPEED=thorough, or is there some new behaviour on this
>> side?
> 
> Yes, this commit doesn't affect that existing make target, so it
> operates as before.
> 
> Patch 5 does however fix a pre-existing problem - currently even
> with SPEED=thorough most of the qcow2 tests didn't get run, only
> the "quick" tests. With this series SPEED=thorough correctly does
> what people thought it was already doing.
>

Sounds good, thanks Daniel.

> With regards,
> Daniel