1. Patchew
  2. QEMU
  3. [Qemu-devel] [PATCH 0/6] tests/qemu-iotests: Run basic iotests during "make check"
  4. View patch

[Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again

Thomas Huth posted 6 patches 6 years, 9 months ago
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@redhat.com>, Li-Wen Hsu <lwhsu@freebsd.org>, Kevin Wolf <kwolf@redhat.com>, Fam Zheng <fam@euphon.net>, Max Reitz <mreitz@redhat.com>, Ed Maste <emaste@freebsd.org>
There is a newer version of this series
[Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Thomas Huth 6 years, 9 months ago 
People often forget to run the iotests before submitting patches or
pull requests - this is likely due to the fact that we do not run the
tests during our mandatory "make check" tests yet. Now that we've got
a new "ci" group of iotests that should be fine to run in every environ-
ment, it should be OK to enable the iotests during "make check" again.
Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
script, and only use the former "quick" group (that contains some tests
that are failing in some environments) when the user decided to run
"make check SPEED=thorough" or something similar.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/Makefile.include      |  2 +-
 tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 36fc73fef5..eb6a7a41e2 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-block: $(patsubst %,check-%, $(check-block-y))
-check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
+check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
 check-clean:
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
 	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
index 0e554bb972..416b3fc48b 100755
--- a/tests/qemu-iotests-quick.sh
+++ b/tests/qemu-iotests-quick.sh
@@ -1,8 +1,23 @@
 #!/bin/sh
 
+# Honor the SPEED environment variable, just like we do it for the qtests.
+# The default is to run all tests that still work fine in a CI environments,
+# but if the user set SPEED=slow or SPEED=thorough, we also run all other
+# tests that are still marked as "quick"
+if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
+    group=quick
+else
+    group=ci
+fi
+
+if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
+    echo "No qemu-system binary available. Skipped qemu-iotests."
+    exit 0
+fi
+
 cd tests/qemu-iotests
 
 ret=0
-TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
+TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g "$group" || ret=1
 
 exit $ret
-- 
2.21.0
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Daniel P. Berrangé 6 years, 9 months ago 
On Wed, Apr 24, 2019 at 12:37:46PM +0200, Thomas Huth wrote:
> People often forget to run the iotests before submitting patches or
> pull requests - this is likely due to the fact that we do not run the
> tests during our mandatory "make check" tests yet. Now that we've got
> a new "ci" group of iotests that should be fine to run in every environ-
> ment, it should be OK to enable the iotests during "make check" again.
> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
> script, and only use the former "quick" group (that contains some tests
> that are failing in some environments) when the user decided to run
> "make check SPEED=thorough" or something similar.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/Makefile.include      |  2 +-
>  tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>  2 files changed, 17 insertions(+), 2 deletions(-)

> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
> index 0e554bb972..416b3fc48b 100755
> --- a/tests/qemu-iotests-quick.sh
> +++ b/tests/qemu-iotests-quick.sh

s/-quick//  perhaps, since it no longer only runs the "quick" tests.

> @@ -1,8 +1,23 @@
>  #!/bin/sh
>  
> +# Honor the SPEED environment variable, just like we do it for the qtests.
> +# The default is to run all tests that still work fine in a CI environments,
> +# but if the user set SPEED=slow or SPEED=thorough, we also run all other
> +# tests that are still marked as "quick"
> +if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
> +    group=quick

When user asks for "slow" tests we run the "quick" tests, which
presumably implies...

> +else
> +    group=ci

..."ci" is quicker than the "quick"  tests !

Confused ? We certainly try to ensure that you will be :-)

> +fi
> +
> +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> +    echo "No qemu-system binary available. Skipped qemu-iotests."
> +    exit 0
> +fi
> +
>  cd tests/qemu-iotests
>  
>  ret=0
> -TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
> +TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g "$group" || ret=1


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Thomas Huth 6 years, 9 months ago 
On 24/04/2019 12.48, Daniel P. Berrangé wrote:
> On Wed, Apr 24, 2019 at 12:37:46PM +0200, Thomas Huth wrote:
>> People often forget to run the iotests before submitting patches or
>> pull requests - this is likely due to the fact that we do not run the
>> tests during our mandatory "make check" tests yet. Now that we've got
>> a new "ci" group of iotests that should be fine to run in every environ-
>> ment, it should be OK to enable the iotests during "make check" again.
>> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
>> script, and only use the former "quick" group (that contains some tests
>> that are failing in some environments) when the user decided to run
>> "make check SPEED=thorough" or something similar.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  tests/Makefile.include      |  2 +-
>>  tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>>  2 files changed, 17 insertions(+), 2 deletions(-)
> 
>> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
>> index 0e554bb972..416b3fc48b 100755
>> --- a/tests/qemu-iotests-quick.sh
>> +++ b/tests/qemu-iotests-quick.sh
> 
> s/-quick//  perhaps, since it no longer only runs the "quick" tests.
> 
>> @@ -1,8 +1,23 @@
>>  #!/bin/sh
>>  
>> +# Honor the SPEED environment variable, just like we do it for the qtests.
>> +# The default is to run all tests that still work fine in a CI environments,
>> +# but if the user set SPEED=slow or SPEED=thorough, we also run all other
>> +# tests that are still marked as "quick"
>> +if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
>> +    group=quick
> 
> When user asks for "slow" tests we run the "quick" tests, which
> presumably implies...
> 
>> +else
>> +    group=ci
> 
> ..."ci" is quicker than the "quick"  tests !
> 
> Confused ? We certainly try to ensure that you will be :-)

... and there is also tests/check-block.sh (which is not used by "make
check-block", by the way, but still talks about "make check-block"),
which is even slower than the "ci" and "quick" group...

I guess that could be cleaned up, too (why do we need two scripts?) ...
just not sure how the final result should look like and if it should be
part of this patch series or rather something separate. Max, Kevin, what
do you think?

Maybe we should merge the two scripts, and depending on the SPEED
variable, we run:

 quick => "ci" group
 thorough => "quick" group
 slow => old behaviour of the check-block.sh script

?

 Thomas
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Vladimir Sementsov-Ogievskiy 6 years, 9 months ago 
24.04.2019 13:59, Thomas Huth wrote:
> On 24/04/2019 12.48, Daniel P. Berrangé wrote:
>> On Wed, Apr 24, 2019 at 12:37:46PM +0200, Thomas Huth wrote:
>>> People often forget to run the iotests before submitting patches or
>>> pull requests - this is likely due to the fact that we do not run the
>>> tests during our mandatory "make check" tests yet. Now that we've got
>>> a new "ci" group of iotests that should be fine to run in every environ-
>>> ment, it should be OK to enable the iotests during "make check" again.
>>> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
>>> script, and only use the former "quick" group (that contains some tests
>>> that are failing in some environments) when the user decided to run
>>> "make check SPEED=thorough" or something similar.
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>   tests/Makefile.include      |  2 +-
>>>   tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>>>   2 files changed, 17 insertions(+), 2 deletions(-)
>>
>>> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
>>> index 0e554bb972..416b3fc48b 100755
>>> --- a/tests/qemu-iotests-quick.sh
>>> +++ b/tests/qemu-iotests-quick.sh
>>
>> s/-quick//  perhaps, since it no longer only runs the "quick" tests.
>>
>>> @@ -1,8 +1,23 @@
>>>   #!/bin/sh
>>>   
>>> +# Honor the SPEED environment variable, just like we do it for the qtests.
>>> +# The default is to run all tests that still work fine in a CI environments,
>>> +# but if the user set SPEED=slow or SPEED=thorough, we also run all other
>>> +# tests that are still marked as "quick"
>>> +if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
>>> +    group=quick
>>
>> When user asks for "slow" tests we run the "quick" tests, which
>> presumably implies...
>>
>>> +else
>>> +    group=ci
>>
>> ..."ci" is quicker than the "quick"  tests !
>>
>> Confused ? We certainly try to ensure that you will be :-)
> 
> ... and there is also tests/check-block.sh (which is not used by "make
> check-block", by the way, but still talks about "make check-block"),
> which is even slower than the "ci" and "quick" group...
> 
> I guess that could be cleaned up, too (why do we need two scripts?) ...
> just not sure how the final result should look like and if it should be
> part of this patch series or rather something separate. Max, Kevin, what
> do you think?
> 
> Maybe we should merge the two scripts, and depending on the SPEED
> variable, we run:
> 
>   quick => "ci" group
>   thorough => "quick" group
>   slow => old behaviour of the check-block.sh script

May be, instead of SPEED, introduce GROUP variable? To make it plain
ci -> ci
quick -> quick
nothing -> default old behavior
?

> 
> ?
> 
>   Thomas
> 


-- 
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Daniel P. Berrangé 6 years, 9 months ago 
On Wed, Apr 24, 2019 at 12:02:29PM +0000, Vladimir Sementsov-Ogievskiy wrote:
> 24.04.2019 13:59, Thomas Huth wrote:
> > On 24/04/2019 12.48, Daniel P. Berrangé wrote:
> >> On Wed, Apr 24, 2019 at 12:37:46PM +0200, Thomas Huth wrote:
> >>> People often forget to run the iotests before submitting patches or
> >>> pull requests - this is likely due to the fact that we do not run the
> >>> tests during our mandatory "make check" tests yet. Now that we've got
> >>> a new "ci" group of iotests that should be fine to run in every environ-
> >>> ment, it should be OK to enable the iotests during "make check" again.
> >>> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
> >>> script, and only use the former "quick" group (that contains some tests
> >>> that are failing in some environments) when the user decided to run
> >>> "make check SPEED=thorough" or something similar.
> >>>
> >>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> >>> ---
> >>>   tests/Makefile.include      |  2 +-
> >>>   tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
> >>>   2 files changed, 17 insertions(+), 2 deletions(-)
> >>
> >>> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
> >>> index 0e554bb972..416b3fc48b 100755
> >>> --- a/tests/qemu-iotests-quick.sh
> >>> +++ b/tests/qemu-iotests-quick.sh
> >>
> >> s/-quick//  perhaps, since it no longer only runs the "quick" tests.
> >>
> >>> @@ -1,8 +1,23 @@
> >>>   #!/bin/sh
> >>>   
> >>> +# Honor the SPEED environment variable, just like we do it for the qtests.
> >>> +# The default is to run all tests that still work fine in a CI environments,
> >>> +# but if the user set SPEED=slow or SPEED=thorough, we also run all other
> >>> +# tests that are still marked as "quick"
> >>> +if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
> >>> +    group=quick
> >>
> >> When user asks for "slow" tests we run the "quick" tests, which
> >> presumably implies...
> >>
> >>> +else
> >>> +    group=ci
> >>
> >> ..."ci" is quicker than the "quick"  tests !
> >>
> >> Confused ? We certainly try to ensure that you will be :-)
> > 
> > ... and there is also tests/check-block.sh (which is not used by "make
> > check-block", by the way, but still talks about "make check-block"),
> > which is even slower than the "ci" and "quick" group...
> > 
> > I guess that could be cleaned up, too (why do we need two scripts?) ...
> > just not sure how the final result should look like and if it should be
> > part of this patch series or rather something separate. Max, Kevin, what
> > do you think?
> > 
> > Maybe we should merge the two scripts, and depending on the SPEED
> > variable, we run:
> > 
> >   quick => "ci" group
> >   thorough => "quick" group
> >   slow => old behaviour of the check-block.sh script
> 
> May be, instead of SPEED, introduce GROUP variable? To make it plain

"$SPEED" is something used/defined by the gtester harness, so other bits
run by "make check" already honour that, hence it is desirable fro the
iotests todo the same.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Vladimir Sementsov-Ogievskiy 6 years, 9 months ago 
24.04.2019 15:13, Daniel P. Berrangé wrote:
> On Wed, Apr 24, 2019 at 12:02:29PM +0000, Vladimir Sementsov-Ogievskiy wrote:
>> 24.04.2019 13:59, Thomas Huth wrote:
>>> On 24/04/2019 12.48, Daniel P. Berrangé wrote:
>>>> On Wed, Apr 24, 2019 at 12:37:46PM +0200, Thomas Huth wrote:
>>>>> People often forget to run the iotests before submitting patches or
>>>>> pull requests - this is likely due to the fact that we do not run the
>>>>> tests during our mandatory "make check" tests yet. Now that we've got
>>>>> a new "ci" group of iotests that should be fine to run in every environ-
>>>>> ment, it should be OK to enable the iotests during "make check" again.
>>>>> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
>>>>> script, and only use the former "quick" group (that contains some tests
>>>>> that are failing in some environments) when the user decided to run
>>>>> "make check SPEED=thorough" or something similar.
>>>>>
>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>> ---
>>>>>    tests/Makefile.include      |  2 +-
>>>>>    tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>>>>>    2 files changed, 17 insertions(+), 2 deletions(-)
>>>>
>>>>> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
>>>>> index 0e554bb972..416b3fc48b 100755
>>>>> --- a/tests/qemu-iotests-quick.sh
>>>>> +++ b/tests/qemu-iotests-quick.sh
>>>>
>>>> s/-quick//  perhaps, since it no longer only runs the "quick" tests.
>>>>
>>>>> @@ -1,8 +1,23 @@
>>>>>    #!/bin/sh
>>>>>    
>>>>> +# Honor the SPEED environment variable, just like we do it for the qtests.
>>>>> +# The default is to run all tests that still work fine in a CI environments,
>>>>> +# but if the user set SPEED=slow or SPEED=thorough, we also run all other
>>>>> +# tests that are still marked as "quick"
>>>>> +if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
>>>>> +    group=quick
>>>>
>>>> When user asks for "slow" tests we run the "quick" tests, which
>>>> presumably implies...
>>>>
>>>>> +else
>>>>> +    group=ci
>>>>
>>>> ..."ci" is quicker than the "quick"  tests !
>>>>
>>>> Confused ? We certainly try to ensure that you will be :-)
>>>
>>> ... and there is also tests/check-block.sh (which is not used by "make
>>> check-block", by the way, but still talks about "make check-block"),
>>> which is even slower than the "ci" and "quick" group...
>>>
>>> I guess that could be cleaned up, too (why do we need two scripts?) ...
>>> just not sure how the final result should look like and if it should be
>>> part of this patch series or rather something separate. Max, Kevin, what
>>> do you think?
>>>
>>> Maybe we should merge the two scripts, and depending on the SPEED
>>> variable, we run:
>>>
>>>    quick => "ci" group
>>>    thorough => "quick" group
>>>    slow => old behaviour of the check-block.sh script
>>
>> May be, instead of SPEED, introduce GROUP variable? To make it plain
> 
> "$SPEED" is something used/defined by the gtester harness, so other bits
> run by "make check" already honour that, hence it is desirable fro the
> iotests todo the same.
> 

OK, sorry, didn't know.


-- 
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Wainer dos Santos Moschetta 6 years, 9 months ago 
Hi Thomas,


On 04/24/2019 07:37 AM, Thomas Huth wrote:
> People often forget to run the iotests before submitting patches or
> pull requests - this is likely due to the fact that we do not run the
> tests during our mandatory "make check" tests yet. Now that we've got
> a new "ci" group of iotests that should be fine to run in every environ-
> ment, it should be OK to enable the iotests during "make check" again.
> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
> script, and only use the former "quick" group (that contains some tests
> that are failing in some environments) when the user decided to run
> "make check SPEED=thorough" or something similar.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/Makefile.include      |  2 +-
>   tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>   2 files changed, 17 insertions(+), 2 deletions(-)

One of Patchew's runners failed with this patch series:
https://patchew.org/QEMU/20190424103747.10173-1-thuth@redhat.com/

I encountered same failures with `make check` locally on Fedora 29 
x86_64 (pulled latest qemu, and configured with defaults and 
x86_64-softmmu target):
---
Not run: 233
Failures: 069 103 114 133 140 143 197 215 226 244
Failed 10 of 91 tests
---

Let me know if you need more details.

- Wainer

>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 36fc73fef5..eb6a7a41e2 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -1168,7 +1168,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
>   check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi
>   check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>   check-block: $(patsubst %,check-%, $(check-block-y))
> -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree
> +check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-block
>   check-clean:
>   	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>   	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
> diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh
> index 0e554bb972..416b3fc48b 100755
> --- a/tests/qemu-iotests-quick.sh
> +++ b/tests/qemu-iotests-quick.sh
> @@ -1,8 +1,23 @@
>   #!/bin/sh
>   
> +# Honor the SPEED environment variable, just like we do it for the qtests.
> +# The default is to run all tests that still work fine in a CI environments,
> +# but if the user set SPEED=slow or SPEED=thorough, we also run all other
> +# tests that are still marked as "quick"
> +if [ "$SPEED" = "slow" -o "$SPEED" = "thorough" ]; then
> +    group=quick
> +else
> +    group=ci
> +fi
> +
> +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then
> +    echo "No qemu-system binary available. Skipped qemu-iotests."
> +    exit 0
> +fi
> +
>   cd tests/qemu-iotests
>   
>   ret=0
> -TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1
> +TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g "$group" || ret=1
>   
>   exit $ret
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Thomas Huth 6 years, 9 months ago 
On 24/04/2019 22.42, Wainer dos Santos Moschetta wrote:
> Hi Thomas,
> 
> 
> On 04/24/2019 07:37 AM, Thomas Huth wrote:
>> People often forget to run the iotests before submitting patches or
>> pull requests - this is likely due to the fact that we do not run the
>> tests during our mandatory "make check" tests yet. Now that we've got
>> a new "ci" group of iotests that should be fine to run in every environ-
>> ment, it should be OK to enable the iotests during "make check" again.
>> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
>> script, and only use the former "quick" group (that contains some tests
>> that are failing in some environments) when the user decided to run
>> "make check SPEED=thorough" or something similar.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>   tests/Makefile.include      |  2 +-
>>   tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>>   2 files changed, 17 insertions(+), 2 deletions(-)
> 
> One of Patchew's runners failed with this patch series:
> https://patchew.org/QEMU/20190424103747.10173-1-thuth@redhat.com/
> 
> I encountered same failures with `make check` locally on Fedora 29
> x86_64 (pulled latest qemu, and configured with defaults and
> x86_64-softmmu target):
> ---
> Not run: 233
> Failures: 069 103 114 133 140 143 197 215 226 244

Ok, thanks for the info. Looks like many tests are failing because qemu-io suddenly prints its program name in front of the error messages? E.g.:

--- /tmp/qemu-test/src/tests/qemu-iotests/069.out	2019-04-24 16:52:31.000000000 +0000
+++ /tmp/qemu-test/build/tests/qemu-iotests/069.out.bad	2019-04-24 16:59:13.310226424 +0000
@@ -4,5 +4,5 @@
 
 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=131072
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=131072 backing_file=TEST_DIR/t.IMGFMT.base
-can't open device TEST_DIR/t.IMGFMT: Could not open backing file: Could not open 'TEST_DIR/t.IMGFMT.base': No such file or directory
+qemu-io: can't open device TEST_DIR/t.IMGFMT: Could not open backing file: Could not open 'TEST_DIR/t.IMGFMT.base': No such file or directory

Does anybody from the block folks has a clue what might be going wrong here?

 Thomas
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Thomas Huth 6 years, 9 months ago 
On 25/04/2019 07.57, Thomas Huth wrote:
> On 24/04/2019 22.42, Wainer dos Santos Moschetta wrote:
>> Hi Thomas,
>>
>>
>> On 04/24/2019 07:37 AM, Thomas Huth wrote:
>>> People often forget to run the iotests before submitting patches or
>>> pull requests - this is likely due to the fact that we do not run the
>>> tests during our mandatory "make check" tests yet. Now that we've got
>>> a new "ci" group of iotests that should be fine to run in every environ-
>>> ment, it should be OK to enable the iotests during "make check" again.
>>> Thus we now run the "ci" tests by default from the qemu-iotests-quick.sh
>>> script, and only use the former "quick" group (that contains some tests
>>> that are failing in some environments) when the user decided to run
>>> "make check SPEED=thorough" or something similar.
>>>
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>   tests/Makefile.include      |  2 +-
>>>   tests/qemu-iotests-quick.sh | 17 ++++++++++++++++-
>>>   2 files changed, 17 insertions(+), 2 deletions(-)
>>
>> One of Patchew's runners failed with this patch series:
>> https://patchew.org/QEMU/20190424103747.10173-1-thuth@redhat.com/
>>
>> I encountered same failures with `make check` locally on Fedora 29
>> x86_64 (pulled latest qemu, and configured with defaults and
>> x86_64-softmmu target):
>> ---
>> Not run: 233
>> Failures: 069 103 114 133 140 143 197 215 226 244
> 
> Ok, thanks for the info. Looks like many tests are failing because qemu-io suddenly prints its program name in front of the error messages? E.g.:
> 
> --- /tmp/qemu-test/src/tests/qemu-iotests/069.out	2019-04-24 16:52:31.000000000 +0000
> +++ /tmp/qemu-test/build/tests/qemu-iotests/069.out.bad	2019-04-24 16:59:13.310226424 +0000
> @@ -4,5 +4,5 @@
>  
>  Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=131072
>  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=131072 backing_file=TEST_DIR/t.IMGFMT.base
> -can't open device TEST_DIR/t.IMGFMT: Could not open backing file: Could not open 'TEST_DIR/t.IMGFMT.base': No such file or directory
> +qemu-io: can't open device TEST_DIR/t.IMGFMT: Could not open backing file: Could not open 'TEST_DIR/t.IMGFMT.base': No such file or directory
> 
> Does anybody from the block folks has a clue what might be going wrong here?

It's a regression in the current master branch, not caused by my series.
The iotests also fail with vanilla master, e.g. reproducible with:

 cd tests/qemu-iotests
 ./check -qcow2 069

It's caused by Christophe's commit here:

 99e98d7c9fc1a1639fad2c638733b02f4b43aebe
 qemu-io: Use error_[gs]et_progname()

Christophe, could you please have a look?

 Thanks,
  Thomas
Re: [Qemu-devel] [PATCH 5/6] tests: Run the iotests during "make check" again
Posted by Eric Blake 6 years, 9 months ago 
On 4/25/19 5:24 AM, Thomas Huth wrote:

>> Ok, thanks for the info. Looks like many tests are failing because qemu-io suddenly prints its program name in front of the error messages? E.g.:
>>
>> --- /tmp/qemu-test/src/tests/qemu-iotests/069.out	2019-04-24 16:52:31.000000000 +0000
>> +++ /tmp/qemu-test/build/tests/qemu-iotests/069.out.bad	2019-04-24 16:59:13.310226424 +0000
>> @@ -4,5 +4,5 @@
>>  
>>  Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=131072
>>  Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=131072 backing_file=TEST_DIR/t.IMGFMT.base
>> -can't open device TEST_DIR/t.IMGFMT: Could not open backing file: Could not open 'TEST_DIR/t.IMGFMT.base': No such file or directory
>> +qemu-io: can't open device TEST_DIR/t.IMGFMT: Could not open backing file: Could not open 'TEST_DIR/t.IMGFMT.base': No such file or directory
>>
>> Does anybody from the block folks has a clue what might be going wrong here?
> 
> It's a regression in the current master branch, not caused by my series.
> The iotests also fail with vanilla master, e.g. reproducible with:
> 
>  cd tests/qemu-iotests
>  ./check -qcow2 069
> 
> It's caused by Christophe's commit here:
> 
>  99e98d7c9fc1a1639fad2c638733b02f4b43aebe
>  qemu-io: Use error_[gs]et_progname()
> 
> Christophe, could you please have a look?

The change in output is desirable, but yes, it does mean that iotests
need to have .out files updated to match.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Patchew 2.1-devel-b67e4c2

© 2016 - 2026 Red Hat, Inc.