1. Patchew
  2. QEMU
  3. View series

[Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job

Philippe Mathieu-Daudé posted 6 patches 7 years, 7 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20180622004435.10291-1-f4bug@amsat.org
Test checkpatch passed
Test docker-mingw@fedora passed
Test docker-quick@centos7 passed
Test s390x passed
.travis.yml                               |  8 +++++
scripts/qemu.py                           |  6 ++--
tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
4 files changed, 67 insertions(+), 8 deletions(-)
[Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Philippe Mathieu-Daudé 7 years, 7 months ago 
Hi,

Another neanderthal approach to add multi-arch acceptance tests using
Avocado.

Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
at using Avocado in QEMU.
Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
a bit further:
- we can now run other archs
- Avocado is integrated into Travis CI

I'm not sure Travis is the best CI available for those tests, but it runs
quite fast:

      ...
      LINK    x86_64-softmmu/qemu-system-x86_64
    JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
    JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
     (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
     (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
     (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
     (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
     (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
     (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
     (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
    RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
    JOB TIME   : 6.21 s

    Done. Your build exited with 0.

    Job ran for 5 min 34 sec

I couldn't add the Alpha tests because all the vmlinux kernels I found
online are gzipped, so I'm waiting some upstream Avocado merges before.

Cleber: I hope you can help me pythonizing this series :p

Regards,

Phil.

[0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html
[1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html

Philippe Mathieu-Daudé (6):
  avocado: Add a Test.arch property
  tests/acceptance: Rename the x86-64 specific BootLinuxConsole test
  tests/acceptance: Improve the Avocado tags
  tests/acceptance: Add a BootLinuxConsoleMips test
  tests/acceptance: Add a kludge to not use the default console
  travis: Add Avocado tests

 .travis.yml                               |  8 +++++
 scripts/qemu.py                           |  6 ++--
 tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
 tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
 4 files changed, 67 insertions(+), 8 deletions(-)

-- 
2.18.0.rc2
Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Alex Bennée 7 years, 7 months ago 
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Hi,
>
> Another neanderthal approach to add multi-arch acceptance tests using
> Avocado.
>
> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
> at using Avocado in QEMU.
> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
> a bit further:
> - we can now run other archs
> - Avocado is integrated into Travis CI
>
> I'm not sure Travis is the best CI available for those tests, but it runs
> quite fast:
>
>       ...
>       LINK    x86_64-softmmu/qemu-system-x86_64
>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>     JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>      (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
>      (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
>      (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
>      (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>     JOB TIME   : 6.21 s

Hmm however if I do:

   pip install --user avocado-framework
   avocado run tests/acceptance

I get:

   Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',)
   Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',)
   Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied')
   Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params'
   Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi'
   You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log

Which I seem to remember from last time I looked into this. I think
avocado makes a bunch of assumptions about the filesystem and OS layout
which is hard to gloss over for something you should be able to run
straight out of the source tree.


>
>     Done. Your build exited with 0.
>
>     Job ran for 5 min 34 sec
>
> I couldn't add the Alpha tests because all the vmlinux kernels I found
> online are gzipped, so I'm waiting some upstream Avocado merges before.
>
> Cleber: I hope you can help me pythonizing this series :p
>
> Regards,
>
> Phil.
>
> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html
> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html
>
> Philippe Mathieu-Daudé (6):
>   avocado: Add a Test.arch property
>   tests/acceptance: Rename the x86-64 specific BootLinuxConsole test
>   tests/acceptance: Improve the Avocado tags
>   tests/acceptance: Add a BootLinuxConsoleMips test
>   tests/acceptance: Add a kludge to not use the default console
>   travis: Add Avocado tests
>
>  .travis.yml                               |  8 +++++
>  scripts/qemu.py                           |  6 ++--
>  tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
>  tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
>  4 files changed, 67 insertions(+), 8 deletions(-)


--
Alex Bennée
Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Cleber Rosa 7 years, 7 months ago 

On 06/28/2018 11:46 AM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
>> Hi,
>>
>> Another neanderthal approach to add multi-arch acceptance tests using
>> Avocado.
>>
>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
>> at using Avocado in QEMU.
>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
>> a bit further:
>> - we can now run other archs
>> - Avocado is integrated into Travis CI
>>
>> I'm not sure Travis is the best CI available for those tests, but it runs
>> quite fast:
>>
>>       ...
>>       LINK    x86_64-softmmu/qemu-system-x86_64
>>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>>     JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>>      (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
>>      (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
>>      (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
>>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
>>      (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
>>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
>>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>     JOB TIME   : 6.21 s
> 
> Hmm however if I do:
> 
>    pip install --user avocado-framework
>    avocado run tests/acceptance
> 
> I get:
> 
>    Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',)
>    Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',)
>    Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied')

Alex,

These are errors related to your installation of Avocado-*VT*[1].

>    Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params'
>    Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi'
>    You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log
> 

And these from Avocado-*Virt*[2].  Neither of those are used in the
tests under "tests/acceptance".

> Which I seem to remember from last time I looked into this. I think
> avocado makes a bunch of assumptions about the filesystem and OS layout
> which is hard to gloss over for something you should be able to run
> straight out of the source tree.
> 
> 

These Avocado-*VT* issues deserve a better look indeed.  Would you
please describe (if not here, under a GH issue) your environment?

Thanks,
- Cleber.

[1] - https://github.com/avocado-framework/avocado-vt
[2] - https://github.com/avocado-framework/avocado-virt

>>
>>     Done. Your build exited with 0.
>>
>>     Job ran for 5 min 34 sec
>>
>> I couldn't add the Alpha tests because all the vmlinux kernels I found
>> online are gzipped, so I'm waiting some upstream Avocado merges before.
>>
>> Cleber: I hope you can help me pythonizing this series :p
>>
>> Regards,
>>
>> Phil.
>>
>> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html
>> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html
>>
>> Philippe Mathieu-Daudé (6):
>>   avocado: Add a Test.arch property
>>   tests/acceptance: Rename the x86-64 specific BootLinuxConsole test
>>   tests/acceptance: Improve the Avocado tags
>>   tests/acceptance: Add a BootLinuxConsoleMips test
>>   tests/acceptance: Add a kludge to not use the default console
>>   travis: Add Avocado tests
>>
>>  .travis.yml                               |  8 +++++
>>  scripts/qemu.py                           |  6 ++--
>>  tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
>>  tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
>>  4 files changed, 67 insertions(+), 8 deletions(-)
> 
> 
> --
> Alex Bennée
> 

-- 
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]
Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Alex Bennée 7 years, 7 months ago 
Cleber Rosa <crosa@redhat.com> writes:

> On 06/28/2018 11:46 AM, Alex Bennée wrote:
>>
>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>>
>>> Hi,
>>>
>>> Another neanderthal approach to add multi-arch acceptance tests using
>>> Avocado.
>>>
>>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
>>> at using Avocado in QEMU.
>>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
>>> a bit further:
>>> - we can now run other archs
>>> - Avocado is integrated into Travis CI
>>>
>>> I'm not sure Travis is the best CI available for those tests, but it runs
>>> quite fast:
>>>
>>>       ...
>>>       LINK    x86_64-softmmu/qemu-system-x86_64
>>>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>>>     JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>>>      (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
>>>      (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
>>>      (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
>>>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>>>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
>>>      (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
>>>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
>>>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>>     JOB TIME   : 6.21 s
>>
>> Hmm however if I do:
>>
>>    pip install --user avocado-framework
>>    avocado run tests/acceptance
>>
>> I get:
>>
>>    Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',)
>>    Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',)
>>    Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied')
>
> Alex,
>
> These are errors related to your installation of Avocado-*VT*[1].
>
>>    Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params'
>>    Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi'
>>    You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log
>>
>
> And these from Avocado-*Virt*[2].  Neither of those are used in the
> tests under "tests/acceptance".
>
>> Which I seem to remember from last time I looked into this. I think
>> avocado makes a bunch of assumptions about the filesystem and OS layout
>> which is hard to gloss over for something you should be able to run
>> straight out of the source tree.
>>
>>
>
> These Avocado-*VT* issues deserve a better look indeed.  Would you
> please describe (if not here, under a GH issue) your environment?

Ahh probably from last time I tried to get things up and running. I
rm -rf ~/.local and things seem to be working well now.

However I was under the impression that VT or virt was needed for this
testing. Or is this an old assumption?

>
> Thanks,
> - Cleber.
>
> [1] - https://github.com/avocado-framework/avocado-vt
> [2] - https://github.com/avocado-framework/avocado-virt
>
>>>
>>>     Done. Your build exited with 0.
>>>
>>>     Job ran for 5 min 34 sec
>>>
>>> I couldn't add the Alpha tests because all the vmlinux kernels I found
>>> online are gzipped, so I'm waiting some upstream Avocado merges before.
>>>
>>> Cleber: I hope you can help me pythonizing this series :p
>>>
>>> Regards,
>>>
>>> Phil.
>>>
>>> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html
>>> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html
>>>
>>> Philippe Mathieu-Daudé (6):
>>>   avocado: Add a Test.arch property
>>>   tests/acceptance: Rename the x86-64 specific BootLinuxConsole test
>>>   tests/acceptance: Improve the Avocado tags
>>>   tests/acceptance: Add a BootLinuxConsoleMips test
>>>   tests/acceptance: Add a kludge to not use the default console
>>>   travis: Add Avocado tests
>>>
>>>  .travis.yml                               |  8 +++++
>>>  scripts/qemu.py                           |  6 ++--
>>>  tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
>>>  tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
>>>  4 files changed, 67 insertions(+), 8 deletions(-)
>>
>>
>> --
>> Alex Bennée
>>


--
Alex Bennée
Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Philippe Mathieu-Daudé 7 years, 7 months ago 
On 06/28/2018 02:13 PM, Alex Bennée wrote:
> 
> Cleber Rosa <crosa@redhat.com> writes:
> 
>> On 06/28/2018 11:46 AM, Alex Bennée wrote:
>>>
>>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>>>
>>>> Hi,
>>>>
>>>> Another neanderthal approach to add multi-arch acceptance tests using
>>>> Avocado.
>>>>
>>>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
>>>> at using Avocado in QEMU.
>>>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
>>>> a bit further:
>>>> - we can now run other archs
>>>> - Avocado is integrated into Travis CI
>>>>
>>>> I'm not sure Travis is the best CI available for those tests, but it runs
>>>> quite fast:
>>>>
>>>>       ...
>>>>       LINK    x86_64-softmmu/qemu-system-x86_64
>>>>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>>>>     JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>>>>      (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
>>>>      (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
>>>>      (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
>>>>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>>>>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
>>>>      (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
>>>>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
>>>>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>>>     JOB TIME   : 6.21 s
>>>
>>> Hmm however if I do:
>>>
>>>    pip install --user avocado-framework
>>>    avocado run tests/acceptance
>>>
>>> I get:
>>>
>>>    Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',)
>>>    Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',)
>>>    Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied')
>>
>> Alex,
>>
>> These are errors related to your installation of Avocado-*VT*[1].
>>
>>>    Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params'
>>>    Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi'
>>>    You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log
>>>
>>
>> And these from Avocado-*Virt*[2].  Neither of those are used in the
>> tests under "tests/acceptance".
>>
>>> Which I seem to remember from last time I looked into this. I think
>>> avocado makes a bunch of assumptions about the filesystem and OS layout
>>> which is hard to gloss over for something you should be able to run
>>> straight out of the source tree.
>>>
>>>
>>
>> These Avocado-*VT* issues deserve a better look indeed.  Would you
>> please describe (if not here, under a GH issue) your environment?
> 
> Ahh probably from last time I tried to get things up and running. I
> rm -rf ~/.local and things seem to be working well now.

Should we check for a prerequisite version?

> 
> However I was under the impression that VT or virt was needed for this
> testing. Or is this an old assumption?

I don't use neither of these for my qemu-system testing, but I'm more
interested in testing system targets that are different from my host.

I think the VT/virt make sens when you test on a same (target, host)
tuple and worry about perfs.

<snip>
Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Alex Bennée 7 years, 7 months ago 
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Hi,
>
> Another neanderthal approach to add multi-arch acceptance tests using
> Avocado.
>
> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
> at using Avocado in QEMU.
> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
> a bit further:
> - we can now run other archs
> - Avocado is integrated into Travis CI
>
> I'm not sure Travis is the best CI available for those tests, but it runs
> quite fast:
>
>       ...
>       LINK    x86_64-softmmu/qemu-system-x86_64
>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>     JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>      (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
>      (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
>      (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
>      (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 |
>       INTERRUPT 0 | CANCEL 0

That said on my AArch64 server it fairs better:

15:59:19 [alex.bennee@libgomp-d05-dev:~/l/qemu.git] review/avocado-v2 + avocado run tests/acceptance
JOB ID     : f40e814204ff007c58073ca75abb38bd575f39f2
JOB LOG    : /home/alex.bennee/avocado/job-results/job-2018-06-28T15.59-f40e814/job.log
 (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: CANCEL (0.00 s)
 (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (2.03 s)
 (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: CANCEL (0.00 s)
 (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: CANCEL (0.00 s)
 (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: CANCEL (0.00 s)
 (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: CANCEL (0.00 s)
 (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: CANCEL (0.00 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 6
JOB TIME   : 2.47 s

Although given the errors in the logs I'm surprised at ERROR 0

>     JOB TIME   : 6.21 s
>
>     Done. Your build exited with 0.
>
>     Job ran for 5 min 34 sec
>
> I couldn't add the Alpha tests because all the vmlinux kernels I found
> online are gzipped, so I'm waiting some upstream Avocado merges before.
>
> Cleber: I hope you can help me pythonizing this series :p
>
> Regards,
>
> Phil.
>
> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html
> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html
>
> Philippe Mathieu-Daudé (6):
>   avocado: Add a Test.arch property
>   tests/acceptance: Rename the x86-64 specific BootLinuxConsole test
>   tests/acceptance: Improve the Avocado tags
>   tests/acceptance: Add a BootLinuxConsoleMips test
>   tests/acceptance: Add a kludge to not use the default console
>   travis: Add Avocado tests
>
>  .travis.yml                               |  8 +++++
>  scripts/qemu.py                           |  6 ++--
>  tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
>  tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
>  4 files changed, 67 insertions(+), 8 deletions(-)


--
Alex Bennée
Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Philippe Mathieu-Daudé 7 years, 6 months ago 
Hi Alex,

On 06/28/2018 01:01 PM, Alex Bennée wrote:
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>>
>> Another neanderthal approach to add multi-arch acceptance tests using
>> Avocado.
>>
>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
>> at using Avocado in QEMU.
>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
>> a bit further:
>> - we can now run other archs
>> - Avocado is integrated into Travis CI
>>
>> I'm not sure Travis is the best CI available for those tests, but it runs
>> quite fast:
>>
>>       ...
>>       LINK    x86_64-softmmu/qemu-system-x86_64
>>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>>     JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>>      (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
>>      (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
>>      (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
>>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
>>      (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
>>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
>>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 |
>>       INTERRUPT 0 | CANCEL 0
> 
> That said on my AArch64 server it fairs better:
> 
> 15:59:19 [alex.bennee@libgomp-d05-dev:~/l/qemu.git] review/avocado-v2 + avocado run tests/acceptance
> JOB ID     : f40e814204ff007c58073ca75abb38bd575f39f2
> JOB LOG    : /home/alex.bennee/avocado/job-results/job-2018-06-28T15.59-f40e814/job.log
>  (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: CANCEL (0.00 s)
>  (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (2.03 s)
>  (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: CANCEL (0.00 s)
>  (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: CANCEL (0.00 s)
>  (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: CANCEL (0.00 s)
>  (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: CANCEL (0.00 s)
>  (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: CANCEL (0.00 s)
> RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 6
> JOB TIME   : 2.47 s
> 
> Although given the errors in the logs I'm surprised at ERROR 0

Why are you surprised? You were expecting the BootLinuxConsoleMips to fail?

> 
>>     JOB TIME   : 6.21 s
>>
>>     Done. Your build exited with 0.
>>
>>     Job ran for 5 min 34 sec
>>
>> I couldn't add the Alpha tests because all the vmlinux kernels I found
>> online are gzipped, so I'm waiting some upstream Avocado merges before.
>>
>> Cleber: I hope you can help me pythonizing this series :p
>>
>> Regards,
>>
>> Phil.
>>
>> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html
>> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html
>>
>> Philippe Mathieu-Daudé (6):
>>   avocado: Add a Test.arch property
>>   tests/acceptance: Rename the x86-64 specific BootLinuxConsole test
>>   tests/acceptance: Improve the Avocado tags
>>   tests/acceptance: Add a BootLinuxConsoleMips test
>>   tests/acceptance: Add a kludge to not use the default console
>>   travis: Add Avocado tests
>>
>>  .travis.yml                               |  8 +++++
>>  scripts/qemu.py                           |  6 ++--
>>  tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
>>  tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
>>  4 files changed, 67 insertions(+), 8 deletions(-)
> 
> 
> --
> Alex Bennée
>
Re: [Qemu-devel] [PATCH v2 0/6] Avocado: start multi-arch tests, add a Travis job
Posted by Alex Bennée 7 years, 6 months ago 
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Hi Alex,
>
> On 06/28/2018 01:01 PM, Alex Bennée wrote:
>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>>>
>>> Another neanderthal approach to add multi-arch acceptance tests using
>>> Avocado.
>>>
>>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1]
>>> at using Avocado in QEMU.
>>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes
>>> a bit further:
>>> - we can now run other archs
>>> - Avocado is integrated into Travis CI
>>>
>>> I'm not sure Travis is the best CI available for those tests, but it runs
>>> quite fast:
>>>
>>>       ...
>>>       LINK    x86_64-softmmu/qemu-system-x86_64
>>>     JOB ID     : a3a56af3c02d193e862ce660aae1d9c72926dcb6
>>>     JOB LOG    : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log
>>>      (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test:  PASS (3.89 s)
>>>      (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test:  PASS (1.83 s)
>>>      (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version:  PASS (0.05 s)
>>>      (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc:  PASS (0.04 s)
>>>      (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password:  PASS (0.04 s)
>>>      (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password:  PASS (0.05 s)
>>>      (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password:  PASS (0.05 s)
>>>     RESULTS    : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 |
>>>       INTERRUPT 0 | CANCEL 0
>>
>> That said on my AArch64 server it fairs better:
>>
>> 15:59:19 [alex.bennee@libgomp-d05-dev:~/l/qemu.git] review/avocado-v2 + avocado run tests/acceptance
>> JOB ID     : f40e814204ff007c58073ca75abb38bd575f39f2
>> JOB LOG    : /home/alex.bennee/avocado/job-results/job-2018-06-28T15.59-f40e814/job.log
>>  (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: CANCEL (0.00 s)
>>  (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (2.03 s)
>>  (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: CANCEL (0.00 s)
>>  (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: CANCEL (0.00 s)
>>  (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: CANCEL (0.00 s)
>>  (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: CANCEL (0.00 s)
>>  (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: CANCEL (0.00 s)
>> RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 6
>> JOB TIME   : 2.47 s
>>
>> Although given the errors in the logs I'm surprised at ERROR 0
>
> Why are you surprised? You were expecting the BootLinuxConsoleMips to
> fail?

No but a bunch have CANCELed... maybe they are skipping but if the test
is run and it fails to complete I would expect that to be an error.

>
>>
>>>     JOB TIME   : 6.21 s
>>>
>>>     Done. Your build exited with 0.
>>>
>>>     Job ran for 5 min 34 sec
>>>
>>> I couldn't add the Alpha tests because all the vmlinux kernels I found
>>> online are gzipped, so I'm waiting some upstream Avocado merges before.
>>>
>>> Cleber: I hope you can help me pythonizing this series :p
>>>
>>> Regards,
>>>
>>> Phil.
>>>
>>> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html
>>> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html
>>>
>>> Philippe Mathieu-Daudé (6):
>>>   avocado: Add a Test.arch property
>>>   tests/acceptance: Rename the x86-64 specific BootLinuxConsole test
>>>   tests/acceptance: Improve the Avocado tags
>>>   tests/acceptance: Add a BootLinuxConsoleMips test
>>>   tests/acceptance: Add a kludge to not use the default console
>>>   travis: Add Avocado tests
>>>
>>>  .travis.yml                               |  8 +++++
>>>  scripts/qemu.py                           |  6 ++--
>>>  tests/acceptance/avocado_qemu/__init__.py | 17 ++++++---
>>>  tests/acceptance/boot_linux_console.py    | 44 +++++++++++++++++++++--
>>>  4 files changed, 67 insertions(+), 8 deletions(-)
>>
>>
>> --
>> Alex Bennée
>>


--
Alex Bennée

Patchew 2.1-devel-b67e4c2

© 2016 - 2026 Red Hat, Inc.