[PATCH v3 0/4] Improve integration of iotests in the meson test harness

Thomas Huth posted 4 patches 2 years, 2 months ago
Failed in applying to current master (apply log)
Test checkpatch passed
Maintainers: Kevin Wolf <kwolf@redhat.com>, Cleber Rosa <crosa@redhat.com>, Eduardo Habkost <eduardo@habkost.net>, Hanna Reitz <hreitz@redhat.com>
meson.build                      |  6 +--
scripts/mtest2make.py            |  4 --
tests/Makefile.include           |  9 +---
tests/check-block.sh             | 38 -----------------
tests/qemu-iotests/271           |  2 +-
tests/qemu-iotests/common.filter | 65 ++++++++++++++--------------
tests/qemu-iotests/common.rc     | 45 ++++++++++----------
tests/qemu-iotests/meson.build   | 73 ++++++++++++++++++++------------
8 files changed, 106 insertions(+), 136 deletions(-)
[PATCH v3 0/4] Improve integration of iotests in the meson test harness
Posted by Thomas Huth 2 years, 2 months ago
Though "make check-block" is currently already run via the meson test
runner, it still looks like an oddball in the output of "make check". It
would be nicer if the iotests would show up like the other tests suites.

My original plan was to add each iotests individually from meson.build,
but I did not get that done reliably yet [*], so here's now a cut-down
version to improve the situation at least a little bit: The first three
patches are preparation for the clean-up (long-term goal is to get rid
of check-block.sh, though we're not quite there yet), and the final
patch adds the iotests not as separate test target in the meson test
harness anymore. This way, we can now finally get the output of failed
tests on the console again (unless you're running meson test in verbose
mode, where meson only puts this to the log file - for incomprehensible
reasons), so this should hopefully help to diagnose problems with the
iotests in most cases more easily.

[*] See v2 here:
    https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg01942.html

Thomas Huth (4):
  tests/qemu-iotests: Rework the checks and spots using GNU sed
  tests/qemu-iotests/meson.build: Improve the indentation
  tests/qemu-iotests: Move the bash and sanitizer checks to meson.build
  tests: Do not treat the iotests as separate meson test target anymore

 meson.build                      |  6 +--
 scripts/mtest2make.py            |  4 --
 tests/Makefile.include           |  9 +---
 tests/check-block.sh             | 38 -----------------
 tests/qemu-iotests/271           |  2 +-
 tests/qemu-iotests/common.filter | 65 ++++++++++++++--------------
 tests/qemu-iotests/common.rc     | 45 ++++++++++----------
 tests/qemu-iotests/meson.build   | 73 ++++++++++++++++++++------------
 8 files changed, 106 insertions(+), 136 deletions(-)

-- 
2.27.0


Re: [PATCH v3 0/4] Improve integration of iotests in the meson test harness
Posted by Hanna Reitz 2 years, 1 month ago
On 23.02.22 10:38, Thomas Huth wrote:
> Though "make check-block" is currently already run via the meson test
> runner, it still looks like an oddball in the output of "make check". It
> would be nicer if the iotests would show up like the other tests suites.
>
> My original plan was to add each iotests individually from meson.build,
> but I did not get that done reliably yet [*], so here's now a cut-down
> version to improve the situation at least a little bit: The first three
> patches are preparation for the clean-up (long-term goal is to get rid
> of check-block.sh, though we're not quite there yet), and the final
> patch adds the iotests not as separate test target in the meson test
> harness anymore. This way, we can now finally get the output of failed
> tests on the console again (unless you're running meson test in verbose
> mode, where meson only puts this to the log file - for incomprehensible
> reasons), so this should hopefully help to diagnose problems with the
> iotests in most cases more easily.
>
> [*] See v2 here:
>      https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg01942.html
>
> Thomas Huth (4):
>    tests/qemu-iotests: Rework the checks and spots using GNU sed
>    tests/qemu-iotests/meson.build: Improve the indentation
>    tests/qemu-iotests: Move the bash and sanitizer checks to meson.build
>    tests: Do not treat the iotests as separate meson test target anymore

What’s the status of this series?  I wonder why you split it apart, mainly.

Patch 1 was already merged, and I took patch 4 today.  So what about 
patches 2 and 3?  They look sensible to me, but is this series still 
relevant and fresh, considering you sent new versions of patches 1 and 4?

(And are there any other iotests patches from you that flew under my radar?)

Hanna


Re: [PATCH v3 0/4] Improve integration of iotests in the meson test harness
Posted by Thomas Huth 2 years, 1 month ago
On 21/03/2022 17.14, Hanna Reitz wrote:
> On 23.02.22 10:38, Thomas Huth wrote:
>> Though "make check-block" is currently already run via the meson test
>> runner, it still looks like an oddball in the output of "make check". It
>> would be nicer if the iotests would show up like the other tests suites.
>>
>> My original plan was to add each iotests individually from meson.build,
>> but I did not get that done reliably yet [*], so here's now a cut-down
>> version to improve the situation at least a little bit: The first three
>> patches are preparation for the clean-up (long-term goal is to get rid
>> of check-block.sh, though we're not quite there yet), and the final
>> patch adds the iotests not as separate test target in the meson test
>> harness anymore. This way, we can now finally get the output of failed
>> tests on the console again (unless you're running meson test in verbose
>> mode, where meson only puts this to the log file - for incomprehensible
>> reasons), so this should hopefully help to diagnose problems with the
>> iotests in most cases more easily.
>>
>> [*] See v2 here:
>>      https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg01942.html
>>
>> Thomas Huth (4):
>>    tests/qemu-iotests: Rework the checks and spots using GNU sed
>>    tests/qemu-iotests/meson.build: Improve the indentation
>>    tests/qemu-iotests: Move the bash and sanitizer checks to meson.build
>>    tests: Do not treat the iotests as separate meson test target anymore
> 
> What’s the status of this series?  I wonder why you split it apart, mainly.

I've mainly split the fourth patch apart since Paolo mentioned that the 
commit message should mention the meson bug (IIRC), and  since QEMU was 
entering soft-freeze, thus I doubt that a patch like "Move the bash and 
sanitizer checks to meson.build" is still acceptable at this point in time. 
The meson.build clean-up is rather something for 7.1 instead.

> Patch 1 was already merged, and I took patch 4 today.  So what about patches 
> 2 and 3?  They look sensible to me, but is this series still relevant and 
> fresh, considering you sent new versions of patches 1 and 4?

If you think they are still ok for 7.0, you can certainly also pick the 2nd 
and 3rd patch ... otherwise I'll respin them later for 7.1.

> (And are there any other iotests patches from you that flew under my radar?)

There's my "Supply a test plan in TAP mode" patch on the list, but 
apparently you already found it :-)

Thanks!

   Thomas


Re: [PATCH v3 0/4] Improve integration of iotests in the meson test harness
Posted by Hanna Reitz 2 years, 1 month ago
On 21.03.22 18:26, Thomas Huth wrote:
> On 21/03/2022 17.14, Hanna Reitz wrote:
>> On 23.02.22 10:38, Thomas Huth wrote:
>>> Though "make check-block" is currently already run via the meson test
>>> runner, it still looks like an oddball in the output of "make 
>>> check". It
>>> would be nicer if the iotests would show up like the other tests 
>>> suites.
>>>
>>> My original plan was to add each iotests individually from meson.build,
>>> but I did not get that done reliably yet [*], so here's now a cut-down
>>> version to improve the situation at least a little bit: The first three
>>> patches are preparation for the clean-up (long-term goal is to get rid
>>> of check-block.sh, though we're not quite there yet), and the final
>>> patch adds the iotests not as separate test target in the meson test
>>> harness anymore. This way, we can now finally get the output of failed
>>> tests on the console again (unless you're running meson test in verbose
>>> mode, where meson only puts this to the log file - for incomprehensible
>>> reasons), so this should hopefully help to diagnose problems with the
>>> iotests in most cases more easily.
>>>
>>> [*] See v2 here:
>>> https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg01942.html
>>>
>>> Thomas Huth (4):
>>>    tests/qemu-iotests: Rework the checks and spots using GNU sed
>>>    tests/qemu-iotests/meson.build: Improve the indentation
>>>    tests/qemu-iotests: Move the bash and sanitizer checks to 
>>> meson.build
>>>    tests: Do not treat the iotests as separate meson test target 
>>> anymore
>>
>> What’s the status of this series?  I wonder why you split it apart, 
>> mainly.
>
> I've mainly split the fourth patch apart since Paolo mentioned that 
> the commit message should mention the meson bug (IIRC), and since QEMU 
> was entering soft-freeze, thus I doubt that a patch like "Move the 
> bash and sanitizer checks to meson.build" is still acceptable at this 
> point in time. The meson.build clean-up is rather something for 7.1 
> instead.
>
>> Patch 1 was already merged, and I took patch 4 today.  So what about 
>> patches 2 and 3?  They look sensible to me, but is this series still 
>> relevant and fresh, considering you sent new versions of patches 1 
>> and 4?
>
> If you think they are still ok for 7.0, you can certainly also pick 
> the 2nd and 3rd patch ... otherwise I'll respin them later for 7.1.

That sounds like you don’t really need to respin, so I’ve taken them to 
my block-next branch for 7.1:

https://gitlab.com/hreitz/qemu/-/commits/block-next

Thanks!

Hanna


Re: [PATCH v3 0/4] Improve integration of iotests in the meson test harness
Posted by Paolo Bonzini 2 years, 2 months ago
On 2/23/22 10:38, Thomas Huth wrote:
> This way, we can now finally get the output of failed
> tests on the console again (unless you're running meson test in verbose
> mode, where meson only puts this to the log file - for incomprehensible
> reasons),

It's a bug (introduced by yours truly, which perhaps makes it less 
incomprehensible) and there's an open pull request to fix it.

Paolo