[v2] travis: speedup to reduce failures

[Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

Posted by Philippe Mathieu-Daudé 8 years, 7 months ago

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index a7512f3ab0..96ddc16cfd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -68,6 +68,8 @@ env:
       # be kind with Travis free plan, restrict to at most 4 cores
     - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
     - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
+    # ccache
+    - CCACHE_SLOPPINESS="file_macro,time_macros"
     # tests
     - TEST_CMD="make -j${PARALLEL_JOBS} check"
   matrix:
@@ -82,14 +84,21 @@ git:
   submodules: false
 before_install:
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
+  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
   - git submodule update --init --recursive
 before_script:
+  # https://docs.travis-ci.com/user/caching#ccache-on-os-x
+  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
+  - if [ "$CC" == "clang" ]; then export CCACHE_CPP2=1 ; fi
+  - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
   - echo "Using ${PARALLEL_JOBS} parallel jobs"
+  - export HOST_CC="${CC}"
   - ./configure ${CONFIG}
 script:
-  - make -j${PARALLEL_JOBS}
+  - make -j${PARALLEL_JOBS} && ccache -s
   - ${TEST_CMD}
+before_cache:
+  - ccache -s
 matrix:
   include:
     # Test with CLang for compile portability
@@ -112,6 +121,7 @@ matrix:
            TEST_CMD=""
       compiler: gcc
     - env: CONFIG=""
+           CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
       os: osx
       compiler: clang
     # Plain Trusty System Build
-- 
2.11.0

Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

Posted by Alex Bennée 8 years, 7 months ago

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Can you expand the commit message to breifly explain what all these
extra settings change. What is CCACHE_SLOPINESS and ccache -s doing?

> ---
>  .travis.yml | 14 ++++++++++++--
>  1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index a7512f3ab0..96ddc16cfd 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -68,6 +68,8 @@ env:
>        # be kind with Travis free plan, restrict to at most 4 cores
>      - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
>      - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> +    # ccache
> +    - CCACHE_SLOPPINESS="file_macro,time_macros"
>      # tests
>      - TEST_CMD="make -j${PARALLEL_JOBS} check"
>    matrix:
> @@ -82,14 +84,21 @@ git:
>    submodules: false
>  before_install:
>    - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update ; fi
> -  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ; fi
> +  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew install libffi gettext glib pixman ccache ; fi
>    - git submodule update --init --recursive
>  before_script:
> +  # https://docs.travis-ci.com/user/caching#ccache-on-os-x
> +  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export PATH=/usr/local/opt/ccache/libexec:$PATH ; fi
> +  - if [ "$CC" == "clang" ]; then export CCACHE_CPP2=1 ; fi
> +  - if [ -x "$(command -v ccache 2>/dev/null)" ]; then export CC="ccache ${CC}"; ccache -s -M 0 -z; fi
>    - echo "Using ${PARALLEL_JOBS} parallel jobs"
> +  - export HOST_CC="${CC}"
>    - ./configure ${CONFIG}
>  script:
> -  - make -j${PARALLEL_JOBS}
> +  - make -j${PARALLEL_JOBS} && ccache -s
>    - ${TEST_CMD}
> +before_cache:
> +  - ccache -s
>  matrix:
>    include:
>      # Test with CLang for compile portability
> @@ -112,6 +121,7 @@ matrix:
>             TEST_CMD=""
>        compiler: gcc
>      - env: CONFIG=""
> +           CCACHE_SLOPPINESS="file_macro,time_macros,include_file_ctime,include_file_mtime"
>        os: osx
>        compiler: clang
>      # Plain Trusty System Build


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

Posted by Philippe Mathieu-Daudé 8 years, 7 months ago

On 06/22/2017 07:22 AM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 
> Can you expand the commit message to breifly explain what all these
> extra settings change. What is CCACHE_SLOPINESS and ccache -s doing?

I'm pretty sure I wrote something here ... probably lost using git Fu :(

Re: [Qemu-devel] [PATCH v2 13/21] travis: improve ccache use

Posted by Paolo Bonzini 8 years, 7 months ago

On 22/06/2017 05:32, Philippe Mathieu-Daudé wrote:
> diff --git a/.travis.yml b/.travis.yml
> index a7512f3ab0..96ddc16cfd 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -68,6 +68,8 @@ env:
>        # be kind with Travis free plan, restrict to at most 4 cores
>      - PARALLEL_JOBS="$(($(test ${HW_CORES} -gt ${VM_CORES} && echo ${VM_CORES} || echo ${HW_CORES}) + 1))"
>      - HOMEBREW_MAKE_JOBS="${PARALLEL_JOBS}"
> +    # ccache
> +    - CCACHE_SLOPPINESS="file_macro,time_macros"

Does this actually have any effect?  We should not be using time macros.

include_file_ctime/mtime makes sense though.

Paolo

>      # tests
>      - TEST_CMD="make -j${PARALLEL_JOBS} check"