[v2] travis: speedup to reduce failures

[Qemu-devel] [PATCH v2 12/21] travis: build using all available cores

Posted by Philippe Mathieu-Daudé 8 years, 4 months ago

Few improvement as of today, but if Travis release their limit on the opensource
plan or upgrade their hardware, new builds will get some benefit.

Restrict to as most 4 cores to respect Travis Open Source plan.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 777914f759..a7512f3ab0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -62,7 +62,14 @@ notifications:
     on_failure: always
 env:
   global:
-    - TEST_CMD="make check"
+    # jobs
+      # detects cores on the hosting hardware, rather than the VM (https://docs.travis-ci.com/user/languages/c#hints)
+    - VM_CORES="4" HW_CORES="$(getconf _NPROCESSORS_ONLN)"
+      # be kind with Travis free plan, restrict to at most 4 cores
+    - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
+    - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
+    # tests
+    - TEST_CMD="make -j${PARALLEL_JOBS} check"
   matrix:
     - CONFIG=""
     - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
@@ -78,9 +85,11 @@ before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
   - git submodule update --init --recursive
 before_script:
+  - echo "Using ${PARALLEL_JOBS} parallel jobs"
   - ./configure ${CONFIG}
 script:
-  - make -j3 && ${TEST_CMD}
+  - make -j${PARALLEL_JOBS}
+  - ${TEST_CMD}
 matrix:
   include:
     # Test with CLang for compile portability
@@ -234,7 +243,7 @@ matrix:
       group: edge
       env:
         - COVERITY=1
-        - COVERITY_BUILD_CMD="make -j3"
+        - COVERITY_BUILD_CMD="make -j${PARALLEL_JOBS}"
         - COVERITY_EMAIL=peter.maydell@linaro.org
         # This 'secure' setting sets COVERITY_TOKEN=<secret token>
         # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...
-- 
2.11.0

Re: [Qemu-devel] [PATCH v2 12/21] travis: build using all available cores

Posted by Alex Bennée 8 years, 4 months ago

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Few improvement as of today, but if Travis release their limit on the opensource
> plan or upgrade their hardware, new builds will get some benefit.
>
> Restrict to as most 4 cores to respect Travis Open Source plan.

You need to mention you are also enabling a parallel make check here.

>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .travis.yml | 15 ++++++++++++---
>  1 file changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 777914f759..a7512f3ab0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -62,7 +62,14 @@ notifications:
>      on_failure: always
>  env:
>    global:
> -    - TEST_CMD="make check"
> +    # jobs
> +      # detects cores on the hosting hardware, rather than the VM (https://docs.travis-ci.com/user/languages/c#hints)
> +    - VM_CORES="4" HW_CORES="$(getconf _NPROCESSORS_ONLN)"
> +      # be kind with Travis free plan, restrict to at most 4 cores
> +    - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
> +    - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> +    # tests
> +    - TEST_CMD="make -j${PARALLEL_JOBS} check"
>    matrix:
>      - CONFIG=""
>      - CONFIG="--enable-debug --enable-debug-tcg --enable-trace-backends=log"
> @@ -78,9 +85,11 @@ before_install:
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
>    - git submodule update --init --recursive
>  before_script:
> +  - echo "Using ${PARALLEL_JOBS} parallel jobs"
>    - ./configure ${CONFIG}
>  script:
> -  - make -j3 && ${TEST_CMD}
> +  - make -j${PARALLEL_JOBS}
> +  - ${TEST_CMD}
>  matrix:
>    include:
>      # Test with CLang for compile portability
> @@ -234,7 +243,7 @@ matrix:
>        group: edge
>        env:
>          - COVERITY=1
> -        - COVERITY_BUILD_CMD="make -j3"
> +        - COVERITY_BUILD_CMD="make -j${PARALLEL_JOBS}"
>          - COVERITY_EMAIL=peter.maydell@linaro.org
>          # This 'secure' setting sets COVERITY_TOKEN=<secret token>
>          # and was created with travis encrypt -r qemu/qemu COVERITY_TOKEN=...


--
Alex Bennée