travis: Improve log browseability

[PATCH 1/2] travis.yml: avocado: Print logs of non-pass tests only

Posted by Wainer dos Santos Moschetta 6 years, 1 month ago

The acceptance tests build on Travis is configured to print
the entire Avocado's job log in case any test fail. Usually one is
interested on failed tests only though. So this change the Travis
configuration in order to show the log of tests which status is
different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests
will have the log printed too because it can help to debug some
condition on CI environment which is not being fulfilled.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
---
 .travis.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 376b7d6dfa..e7fdcb238c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -262,8 +262,8 @@ matrix:
     - env:
         - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu"
         - TEST_CMD="make check-acceptance"
-      after_failure:
-        - cat tests/results/latest/job.log
+      after_script:
+        - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat
       addons:
         apt:
           packages:
-- 
2.23.0

Re: [PATCH 1/2] travis.yml: avocado: Print logs of non-pass tests only

Posted by Alex Bennée 6 years, 1 month ago

Wainer dos Santos Moschetta <wainersm@redhat.com> writes:

> The acceptance tests build on Travis is configured to print
> the entire Avocado's job log in case any test fail. Usually one is
> interested on failed tests only though. So this change the Travis
> configuration in order to show the log of tests which status is
> different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests
> will have the log printed too because it can help to debug some
> condition on CI environment which is not being fulfilled.
>
> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> ---
>  .travis.yml | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 376b7d6dfa..e7fdcb238c 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -262,8 +262,8 @@ matrix:
>      - env:
>          - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu"
>          - TEST_CMD="make check-acceptance"
> -      after_failure:
> -        - cat tests/results/latest/job.log
> +      after_script:
> +        - python3 -c 'import json; r =
> json.load(open("tests/results/latest/results.json"));
> [print(t["logfile"]) for t in r["tests"] if t["status"] not in
> ("PASS", "SKIP")]' | xargs cat

Shame there is no scriptlet for this sort of thing packaged with
avocado. Anyway I'll give it a spin and see if any failures come up.


>        addons:
>          apt:
>            packages:


-- 
Alex Bennée

Re: [PATCH 1/2] travis.yml: avocado: Print logs of non-pass tests only

Posted by Wainer dos Santos Moschetta 6 years, 1 month ago

On 1/7/20 9:20 AM, Alex Bennée wrote:
> Wainer dos Santos Moschetta <wainersm@redhat.com> writes:
>
>> The acceptance tests build on Travis is configured to print
>> the entire Avocado's job log in case any test fail. Usually one is
>> interested on failed tests only though. So this change the Travis
>> configuration in order to show the log of tests which status is
>> different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests
>> will have the log printed too because it can help to debug some
>> condition on CI environment which is not being fulfilled.
>>
>> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>> ---
>>   .travis.yml | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 376b7d6dfa..e7fdcb238c 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -262,8 +262,8 @@ matrix:
>>       - env:
>>           - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu"
>>           - TEST_CMD="make check-acceptance"
>> -      after_failure:
>> -        - cat tests/results/latest/job.log
>> +      after_script:
>> +        - python3 -c 'import json; r =
>> json.load(open("tests/results/latest/results.json"));
>> [print(t["logfile"]) for t in r["tests"] if t["status"] not in
>> ("PASS", "SKIP")]' | xargs cat
> Shame there is no scriptlet for this sort of thing packaged with
> avocado. Anyway I'll give it a spin and see if any failures come up.

Good point. I'm working on such as scriptlet ('plugin' on Avocado 
jargon) but I think we can stick with this change.

Thanks!

- Wainer

>
>>         addons:
>>           apt:
>>             packages:
>

Re: [PATCH 1/2] travis.yml: avocado: Print logs of non-pass tests only

Posted by Philippe Mathieu-Daudé 6 years, 1 month ago

On 12/30/19 7:43 PM, Wainer dos Santos Moschetta wrote:
> The acceptance tests build on Travis is configured to print
> the entire Avocado's job log in case any test fail. Usually one is
> interested on failed tests only though. So this change the Travis
> configuration in order to show the log of tests which status is
> different from 'PASS' and 'SKIP' only. Note that 'CANCEL'-ed tests
> will have the log printed too because it can help to debug some
> condition on CI environment which is not being fulfilled.

Nice!

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Tested with a broken test:
https://travis-ci.org/philmd/qemu/jobs/631341382

Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> ---
>   .travis.yml | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 376b7d6dfa..e7fdcb238c 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -262,8 +262,8 @@ matrix:
>       - env:
>           - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,sparc-softmmu"
>           - TEST_CMD="make check-acceptance"
> -      after_failure:
> -        - cat tests/results/latest/job.log
> +      after_script:
> +        - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat
>         addons:
>           apt:
>             packages:
>