docker: Update Travis-CI image to run acceptance tests

[Qemu-devel] [PATCH 3/4] tests/docker: Add test-acceptance runner

Posted by Philippe Mathieu-Daudé 6 years, 2 months ago

Add a runner script to be able to run acceptance tests within
Docker images. We can now reproduce Travis CI builds locally (and
debug  them!).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/docker/test-acceptance | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100755 tests/docker/test-acceptance

diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
new file mode 100755
index 0000000000..84edaa676c
--- /dev/null
+++ b/tests/docker/test-acceptance
@@ -0,0 +1,21 @@
+#!/bin/bash -e
+#
+# Compile default Travis-CI target and run Avocado acceptance tests
+#
+# Copyright (c) 2019 Red Hat Inc.
+#
+# Authors:
+#  Philippe Mathieu-Daudé <philmd@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+check_qemu check-acceptance
-- 
2.20.1

Re: [Qemu-devel] [PATCH 3/4] tests/docker: Add test-acceptance runner

Posted by Cleber Rosa 6 years, 2 months ago

On Mon, Aug 19, 2019 at 01:18:26AM +0200, Philippe Mathieu-Daudé wrote:
> Add a runner script to be able to run acceptance tests within
> Docker images. We can now reproduce Travis CI builds locally (and
> debug  them!).
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/docker/test-acceptance | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>  create mode 100755 tests/docker/test-acceptance
> 
> diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
> new file mode 100755
> index 0000000000..84edaa676c
> --- /dev/null
> +++ b/tests/docker/test-acceptance
> @@ -0,0 +1,21 @@
> +#!/bin/bash -e
> +#
> +# Compile default Travis-CI target and run Avocado acceptance tests
> +#
> +# Copyright (c) 2019 Red Hat Inc.
> +#
> +# Authors:
> +#  Philippe Mathieu-Daudé <philmd@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +. common.rc
> +
> +cd "$BUILD_DIR"
> +
> +DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
> +build_qemu
> +check_qemu check-acceptance
> -- 
> 2.20.1
> 
> 

I'm currently seeing test errors when running in a container:

  MKDIR   /tmp/qemu-test/build/tests/results
  AVOCADO tests/acceptance
JOB ID     : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
JOB LOG    : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
 (01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc:  ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
Interrupting job (failfast).
RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 0.26 s
/tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
make: *** [check-acceptance] Error 9

That being said, I'm not running it under docker, but under podman,
although I'm not convinced yet that is the defining issue.  I'll try
to identify what's going here.

Cheers,
- Cleber.

Re: [Qemu-devel] [PATCH 3/4] tests/docker: Add test-acceptance runner

Posted by Alex Bennée 5 years, 11 months ago

Cleber Rosa <crosa@redhat.com> writes:

> On Mon, Aug 19, 2019 at 01:18:26AM +0200, Philippe Mathieu-Daudé wrote:
>> Add a runner script to be able to run acceptance tests within
>> Docker images. We can now reproduce Travis CI builds locally (and
>> debug  them!).
>> 
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>>  tests/docker/test-acceptance | 21 +++++++++++++++++++++
>>  1 file changed, 21 insertions(+)
>>  create mode 100755 tests/docker/test-acceptance
>> 
>> diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
>> new file mode 100755
>> index 0000000000..84edaa676c
>> --- /dev/null
>> +++ b/tests/docker/test-acceptance
>> @@ -0,0 +1,21 @@
>> +#!/bin/bash -e
>> +#
>> +# Compile default Travis-CI target and run Avocado acceptance tests
>> +#
>> +# Copyright (c) 2019 Red Hat Inc.
>> +#
>> +# Authors:
>> +#  Philippe Mathieu-Daudé <philmd@redhat.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2
>> +# or (at your option) any later version. See the COPYING file in
>> +# the top-level directory.
>> +
>> +. common.rc
>> +
>> +cd "$BUILD_DIR"
>> +
>> +DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>> +build_qemu
>> +check_qemu check-acceptance
>> -- 
>> 2.20.1
>> 
>> 
>
> I'm currently seeing test errors when running in a container:
>
>   MKDIR   /tmp/qemu-test/build/tests/results
>   AVOCADO tests/acceptance
> JOB ID     : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
> JOB LOG    : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
>  (01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc:  ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
> Interrupting job (failfast).
> RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
> JOB TIME   : 0.26 s
> /tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
> make: *** [check-acceptance] Error 9
>
> That being said, I'm not running it under docker, but under podman,
> although I'm not convinced yet that is the defining issue.  I'll try
> to identify what's going here.

Was there a conclusion to the discussion about this series?

-- 
Alex Bennée

Re: [Qemu-devel] [PATCH 3/4] tests/docker: Add test-acceptance runner

Posted by Philippe Mathieu-Daudé 5 years, 8 months ago

On 12/3/19 2:41 PM, Alex Bennée wrote:
> Cleber Rosa <crosa@redhat.com> writes:
>> On Mon, Aug 19, 2019 at 01:18:26AM +0200, Philippe Mathieu-Daudé wrote:
>>> Add a runner script to be able to run acceptance tests within
>>> Docker images. We can now reproduce Travis CI builds locally (and
>>> debug  them!).
>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>>> ---
>>>   tests/docker/test-acceptance | 21 +++++++++++++++++++++
>>>   1 file changed, 21 insertions(+)
>>>   create mode 100755 tests/docker/test-acceptance
>>>
>>> diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
>>> new file mode 100755
>>> index 0000000000..84edaa676c
>>> --- /dev/null
>>> +++ b/tests/docker/test-acceptance
>>> @@ -0,0 +1,21 @@
>>> +#!/bin/bash -e
>>> +#
>>> +# Compile default Travis-CI target and run Avocado acceptance tests
>>> +#
>>> +# Copyright (c) 2019 Red Hat Inc.
>>> +#
>>> +# Authors:
>>> +#  Philippe Mathieu-Daudé <philmd@redhat.com>
>>> +#
>>> +# This work is licensed under the terms of the GNU GPL, version 2
>>> +# or (at your option) any later version. See the COPYING file in
>>> +# the top-level directory.
>>> +
>>> +. common.rc
>>> +
>>> +cd "$BUILD_DIR"
>>> +
>>> +DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
>>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>>> +build_qemu
>>> +check_qemu check-acceptance
>>> -- 
>>> 2.20.1
>>>
>>>
>>
>> I'm currently seeing test errors when running in a container:
>>
>>    MKDIR   /tmp/qemu-test/build/tests/results
>>    AVOCADO tests/acceptance
>> JOB ID     : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
>> JOB LOG    : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
>>   (01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc:  ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
>> Interrupting job (failfast).
>> RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
>> JOB TIME   : 0.26 s
>> /tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
>> make: *** [check-acceptance] Error 9
>>
>> That being said, I'm not running it under docker, but under podman,
>> although I'm not convinced yet that is the defining issue.  I'll try
>> to identify what's going here.
> 
> Was there a conclusion to the discussion about this series?

No sure if there was a conclusion...

tests/docker/ is listed as 'Maintained' but the listed maintainers only 
use Docker. If Podman is supported, then maybe we need another 
maintainer for Podman? We are not testing it so far.