tests are run sequentially to avoid mixed results output.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index c9ac741afc..dbbb11617d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -66,7 +66,7 @@ before_script:
   - ./configure ${CONFIG}
 script:
   - make -j${PARALLEL_JOBS}
-  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
+  - if [ "${RUN_TESTS}" == "yes" ]; then make -j${PARALLEL_JOBS} check-compile && make check; fi
 matrix:
   include:
     # Test with CLang for compile portability
-- 
2.11.0

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> tests are run sequentially to avoid mixed results output.

I'm not so sure we are worried about mixed results output. I don't think
anyone can compare the entire log and they tend to scroll to the end
where the error will be.

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index c9ac741afc..dbbb11617d 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -66,7 +66,7 @@ before_script:
>    - ./configure ${CONFIG}
>  script:
>    - make -j${PARALLEL_JOBS}
> -  - if [ "${RUN_TESTS}" == "yes" ]; then make check; fi
> +  - if [ "${RUN_TESTS}" == "yes" ]; then make -j${PARALLEL_JOBS} check-compile && make check; fi
>  matrix:
>    include:
>      # Test with CLang for compile portability


--
Alex Bennée

On 19 June 2017 at 11:58, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>
>> tests are run sequentially to avoid mixed results output.
>
> I'm not so sure we are worried about mixed results output. I don't think
> anyone can compare the entire log and they tend to scroll to the end
> where the error will be.

Indeed, and make's --output-sync flag should be enough to keep
individual test output distinct.

What is perhaps a problem is that I'm not sure our tests all support
being run in parallel with each other without tripping each other up by
using the same temporary file / TCP port / etc at once. To the
extent that this doesn't work it's a bug, but we should start
by doing a solid soak test of the test suite to make sure -j<bignum>
works reliably.

thanks
-- PMM

On 06/19/2017 08:12 AM, Peter Maydell wrote:

> What is perhaps a problem is that I'm not sure our tests all support
> being run in parallel with each other without tripping each other up by
> using the same temporary file / TCP port / etc at once. To the
> extent that this doesn't work it's a bug, but we should start
> by doing a solid soak test of the test suite to make sure -j<bignum>
> works reliably.

I might have caught one:

$ gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

$ export MAKEFLAGS="-j ${PARALLEL_JOBS}"

$ echo "Using ${PARALLEL_JOBS} parallel jobs"
Using 3 parallel jobs

$ ${TEST_CMD}
[...]
   GTESTER check-qtest-unicore32
   GTESTER check-qtest-x86_64
   GTESTER check-qtest-xtensa
   GTESTER check-qtest-xtensaeb
   GTESTER tests/test-qobject-output-visitor
   GTESTER tests/test-clone-visitor
   GTESTER tests/test-qobject-input-visitor
   GTESTER tests/test-qmp-commands
   GTESTER tests/test-string-input-visitor
   GTESTER tests/test-string-output-visitor
   GTESTER tests/test-qmp-event
   GTESTER tests/test-opts-visitor
   GTESTER tests/test-qht-par
Could not access KVM kernel module: No such file or directory
failed to initialize KVM: No such file or directory
Back to tcg accelerator.
**
ERROR:tests/vhost-user-test.c:807:test_connect_fail: child process 
(/x86_64/vhost-user/connect-fail/subprocess [2642]) failed unexpectedly
qemu-system-x86_64: Failed to set msg fds.
qemu-system-x86_64: vhost VQ 0 ring restore failed: -1: Resource 
temporarily unavailable (11)
qemu-system-x86_64: Failed to set msg fds.
qemu-system-x86_64: vhost VQ 1 ring restore failed: -1: Resource 
temporarily unavailable (11)
GTester: last random seed: R02S8c0c7c003144320339c3c86cfa72ba62
make: *** [check-qtest-x86_64] Error 1

(build: https://travis-ci.org/philmd/qemu/jobs/246199793)