1. Patchew
  2. QEMU
  3. View series

[RFC PATCH] tests/docker: allow user to override check target

Alex Bennée posted 1 patch 1 year, 5 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20221107145227.1210499-1-alex.bennee@linaro.org
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Thomas Huth <thuth@redhat.com>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Beraldo Leal <bleal@redhat.com>
tests/docker/Makefile.include | 2 ++
tests/docker/common.rc        | 6 +++---
2 files changed, 5 insertions(+), 3 deletions(-)
[RFC PATCH] tests/docker: allow user to override check target
Posted by Alex Bennée 1 year, 5 months ago 
This is useful when trying to bisect a particular failing test behind
a docker run. For example:

  make docker-test-clang@fedora \
    TARGET_LIST=arm-softmmu \
    CHECK_TARGET="meson test qtest-arm/qos-test" \
    J=9 V=1

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 2 ++
 tests/docker/common.rc        | 6 +++---
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index c87f14477a..ece0aa77df 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -184,6 +184,7 @@ docker:
 	@echo '    TARGET_LIST=a,b,c    Override target list in builds.'
 	@echo '    EXTRA_CONFIGURE_OPTS="..."'
 	@echo '                         Extra configure options.'
+	@echo '    CHECK_TARGET="..."   Override the default `make check` target '
 	@echo '    IMAGES="a b c ..":   Restrict available images to subset.'
 	@echo '    TESTS="x y z .."     Restrict available tests to subset.'
 	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
@@ -230,6 +231,7 @@ docker-run: docker-qemu-src
 			$(if $(NETWORK),$(if $(subst $(NETWORK),,1),--net=$(NETWORK)),--net=none) \
 			-e TARGET_LIST=$(subst $(SPACE),$(COMMA),$(TARGET_LIST))	\
 			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
+			-e CHECK_TARGET="$(CHECK_TARGET)" 		\
 			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
 			-e SHOW_ENV=$(SHOW_ENV) 			\
 			$(if $(NOUSER),,				\
diff --git a/tests/docker/common.rc b/tests/docker/common.rc
index e6f8cee0d6..f2769c1ff6 100755
--- a/tests/docker/common.rc
+++ b/tests/docker/common.rc
@@ -63,12 +63,12 @@ check_qemu()
 {
     # default to make check unless the caller specifies
     if [ $# = 0 ]; then
-        INVOCATION="check"
+        INVOCATION="${CHECK_TARGET:-make $MAKEFLAGS check}"
     else
-        INVOCATION="$@"
+        INVOCATION="make $MAKEFLAGS $@"
     fi
 
-    make $MAKEFLAGS $INVOCATION
+    $INVOCATION
 }
 
 test_fail()
-- 
2.34.1
Re: [RFC PATCH] tests/docker: allow user to override check target
Posted by Philippe Mathieu-Daudé 1 year, 5 months ago 
On 7/11/22 15:52, Alex Bennée wrote:
> This is useful when trying to bisect a particular failing test behind
> a docker run. For example:
> 
>    make docker-test-clang@fedora \
>      TARGET_LIST=arm-softmmu \
>      CHECK_TARGET="meson test qtest-arm/qos-test" \
>      J=9 V=1
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   tests/docker/Makefile.include | 2 ++
>   tests/docker/common.rc        | 6 +++---
>   2 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index c87f14477a..ece0aa77df 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -184,6 +184,7 @@ docker:
>   	@echo '    TARGET_LIST=a,b,c    Override target list in builds.'
>   	@echo '    EXTRA_CONFIGURE_OPTS="..."'
>   	@echo '                         Extra configure options.'
> +	@echo '    CHECK_TARGET="..."   Override the default `make check` target '

s/target /target./

>   	@echo '    IMAGES="a b c ..":   Restrict available images to subset.'
>   	@echo '    TESTS="x y z .."     Restrict available tests to subset.'
>   	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
> @@ -230,6 +231,7 @@ docker-run: docker-qemu-src
>   			$(if $(NETWORK),$(if $(subst $(NETWORK),,1),--net=$(NETWORK)),--net=none) \
>   			-e TARGET_LIST=$(subst $(SPACE),$(COMMA),$(TARGET_LIST))	\
>   			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
> +			-e CHECK_TARGET="$(CHECK_TARGET)" 		\
>   			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
>   			-e SHOW_ENV=$(SHOW_ENV) 			\
>   			$(if $(NOUSER),,				\
> diff --git a/tests/docker/common.rc b/tests/docker/common.rc
> index e6f8cee0d6..f2769c1ff6 100755
> --- a/tests/docker/common.rc
> +++ b/tests/docker/common.rc
> @@ -63,12 +63,12 @@ check_qemu()
>   {
>       # default to make check unless the caller specifies
>       if [ $# = 0 ]; then
> -        INVOCATION="check"
> +        INVOCATION="${CHECK_TARGET:-make $MAKEFLAGS check}"

Why pass MAKEFLAGS here?

>       else
> -        INVOCATION="$@"
> +        INVOCATION="make $MAKEFLAGS $@"
>       fi
>   
> -    make $MAKEFLAGS $INVOCATION
> +    $INVOCATION
>   }
>   
>   test_fail()
Re: [RFC PATCH] tests/docker: allow user to override check target
Posted by Alex Bennée 1 year, 5 months ago 
Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> On 7/11/22 15:52, Alex Bennée wrote:
>> This is useful when trying to bisect a particular failing test behind
>> a docker run. For example:
>>    make docker-test-clang@fedora \
>>      TARGET_LIST=arm-softmmu \
>>      CHECK_TARGET="meson test qtest-arm/qos-test" \
>>      J=9 V=1
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>   tests/docker/Makefile.include | 2 ++
>>   tests/docker/common.rc        | 6 +++---
>>   2 files changed, 5 insertions(+), 3 deletions(-)
>> diff --git a/tests/docker/Makefile.include
>> b/tests/docker/Makefile.include
>> index c87f14477a..ece0aa77df 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -184,6 +184,7 @@ docker:
>>   	@echo '    TARGET_LIST=a,b,c    Override target list in builds.'
>>   	@echo '    EXTRA_CONFIGURE_OPTS="..."'
>>   	@echo '                         Extra configure options.'
>> +	@echo '    CHECK_TARGET="..."   Override the default `make check` target '
>
> s/target /target./
>
>>   	@echo '    IMAGES="a b c ..":   Restrict available images to subset.'
>>   	@echo '    TESTS="x y z .."     Restrict available tests to subset.'
>>   	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
>> @@ -230,6 +231,7 @@ docker-run: docker-qemu-src
>>   			$(if $(NETWORK),$(if $(subst $(NETWORK),,1),--net=$(NETWORK)),--net=none) \
>>   			-e TARGET_LIST=$(subst $(SPACE),$(COMMA),$(TARGET_LIST))	\
>>   			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
>> +			-e CHECK_TARGET="$(CHECK_TARGET)" 		\
>>   			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
>>   			-e SHOW_ENV=$(SHOW_ENV) 			\
>>   			$(if $(NOUSER),,				\
>> diff --git a/tests/docker/common.rc b/tests/docker/common.rc
>> index e6f8cee0d6..f2769c1ff6 100755
>> --- a/tests/docker/common.rc
>> +++ b/tests/docker/common.rc
>> @@ -63,12 +63,12 @@ check_qemu()
>>   {
>>       # default to make check unless the caller specifies
>>       if [ $# = 0 ]; then
>> -        INVOCATION="check"
>> +        INVOCATION="${CHECK_TARGET:-make $MAKEFLAGS check}"
>
> Why pass MAKEFLAGS here?

That was just preserving previous behaviour. That said I think MAKEFLAGS
only ever has J in it and perhaps for check_qemu we never want to parallise?

>
>>       else
>> -        INVOCATION="$@"
>> +        INVOCATION="make $MAKEFLAGS $@"
>>       fi
>>   -    make $MAKEFLAGS $INVOCATION
>> +    $INVOCATION
>>   }
>>     test_fail()


-- 
Alex Bennée
Re: [RFC PATCH] tests/docker: allow user to override check target
Posted by Philippe Mathieu-Daudé 1 year, 5 months ago 
On 7/11/22 18:40, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <philmd@linaro.org> writes:
> 
>> On 7/11/22 15:52, Alex Bennée wrote:
>>> This is useful when trying to bisect a particular failing test behind
>>> a docker run. For example:
>>>     make docker-test-clang@fedora \
>>>       TARGET_LIST=arm-softmmu \
>>>       CHECK_TARGET="meson test qtest-arm/qos-test" \
>>>       J=9 V=1
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> ---
>>>    tests/docker/Makefile.include | 2 ++
>>>    tests/docker/common.rc        | 6 +++---
>>>    2 files changed, 5 insertions(+), 3 deletions(-)
>>> diff --git a/tests/docker/Makefile.include
>>> b/tests/docker/Makefile.include
>>> index c87f14477a..ece0aa77df 100644
>>> --- a/tests/docker/Makefile.include
>>> +++ b/tests/docker/Makefile.include
>>> @@ -184,6 +184,7 @@ docker:
>>>    	@echo '    TARGET_LIST=a,b,c    Override target list in builds.'
>>>    	@echo '    EXTRA_CONFIGURE_OPTS="..."'
>>>    	@echo '                         Extra configure options.'
>>> +	@echo '    CHECK_TARGET="..."   Override the default `make check` target '
>>
>> s/target /target./
>>
>>>    	@echo '    IMAGES="a b c ..":   Restrict available images to subset.'
>>>    	@echo '    TESTS="x y z .."     Restrict available tests to subset.'
>>>    	@echo '    J=[0..9]*            Overrides the -jN parameter for make commands'
>>> @@ -230,6 +231,7 @@ docker-run: docker-qemu-src
>>>    			$(if $(NETWORK),$(if $(subst $(NETWORK),,1),--net=$(NETWORK)),--net=none) \
>>>    			-e TARGET_LIST=$(subst $(SPACE),$(COMMA),$(TARGET_LIST))	\
>>>    			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
>>> +			-e CHECK_TARGET="$(CHECK_TARGET)" 		\
>>>    			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
>>>    			-e SHOW_ENV=$(SHOW_ENV) 			\
>>>    			$(if $(NOUSER),,				\
>>> diff --git a/tests/docker/common.rc b/tests/docker/common.rc
>>> index e6f8cee0d6..f2769c1ff6 100755
>>> --- a/tests/docker/common.rc
>>> +++ b/tests/docker/common.rc
>>> @@ -63,12 +63,12 @@ check_qemu()
>>>    {
>>>        # default to make check unless the caller specifies
>>>        if [ $# = 0 ]; then
>>> -        INVOCATION="check"
>>> +        INVOCATION="${CHECK_TARGET:-make $MAKEFLAGS check}"
>>
>> Why pass MAKEFLAGS here?
> 
> That was just preserving previous behaviour. That said I think MAKEFLAGS
> only ever has J in it and perhaps for check_qemu we never want to parallise?

If we ever can, we certainly want to use it on CI :) And keep the 
non-parallelized version for manual runs. Anyhow not something to worry as
of today, since IIRC 'make check' + -jX  doesn't work well together.

OK, now I see you changed the full command invoked, so CHECK_TARGET
isn't a Make rule anymore, but a full command. Maybe rename it
TEST_COMMAND?

> 
>>
>>>        else
>>> -        INVOCATION="$@"
>>> +        INVOCATION="make $MAKEFLAGS $@"
>>>        fi
>>>    -    make $MAKEFLAGS $INVOCATION
>>> +    $INVOCATION
>>>    }
>>>      test_fail()
> 
>

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.