.travis.yml | 8 +++++ scripts/qemu.py | 6 ++-- tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- 4 files changed, 67 insertions(+), 8 deletions(-)
Hi, Another neanderthal approach to add multi-arch acceptance tests using Avocado. Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] at using Avocado in QEMU. Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes a bit further: - we can now run other archs - Avocado is integrated into Travis CI I'm not sure Travis is the best CI available for those tests, but it runs quite fast: ... LINK x86_64-softmmu/qemu-system-x86_64 JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 6.21 s Done. Your build exited with 0. Job ran for 5 min 34 sec I couldn't add the Alpha tests because all the vmlinux kernels I found online are gzipped, so I'm waiting some upstream Avocado merges before. Cleber: I hope you can help me pythonizing this series :p Regards, Phil. [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html Philippe Mathieu-Daudé (6): avocado: Add a Test.arch property tests/acceptance: Rename the x86-64 specific BootLinuxConsole test tests/acceptance: Improve the Avocado tags tests/acceptance: Add a BootLinuxConsoleMips test tests/acceptance: Add a kludge to not use the default console travis: Add Avocado tests .travis.yml | 8 +++++ scripts/qemu.py | 6 ++-- tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- 4 files changed, 67 insertions(+), 8 deletions(-) -- 2.18.0.rc2
Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > Hi, > > Another neanderthal approach to add multi-arch acceptance tests using > Avocado. > > Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] > at using Avocado in QEMU. > Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes > a bit further: > - we can now run other archs > - Avocado is integrated into Travis CI > > I'm not sure Travis is the best CI available for those tests, but it runs > quite fast: > > ... > LINK x86_64-softmmu/qemu-system-x86_64 > JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 > JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log > (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) > (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) > (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) > (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) > (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) > (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) > (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) > RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 > JOB TIME : 6.21 s Hmm however if I do: pip install --user avocado-framework avocado run tests/acceptance I get: Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',) Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',) Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied') Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params' Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi' You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log Which I seem to remember from last time I looked into this. I think avocado makes a bunch of assumptions about the filesystem and OS layout which is hard to gloss over for something you should be able to run straight out of the source tree. > > Done. Your build exited with 0. > > Job ran for 5 min 34 sec > > I couldn't add the Alpha tests because all the vmlinux kernels I found > online are gzipped, so I'm waiting some upstream Avocado merges before. > > Cleber: I hope you can help me pythonizing this series :p > > Regards, > > Phil. > > [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html > [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html > > Philippe Mathieu-Daudé (6): > avocado: Add a Test.arch property > tests/acceptance: Rename the x86-64 specific BootLinuxConsole test > tests/acceptance: Improve the Avocado tags > tests/acceptance: Add a BootLinuxConsoleMips test > tests/acceptance: Add a kludge to not use the default console > travis: Add Avocado tests > > .travis.yml | 8 +++++ > scripts/qemu.py | 6 ++-- > tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- > tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- > 4 files changed, 67 insertions(+), 8 deletions(-) -- Alex Bennée
On 06/28/2018 11:46 AM, Alex Bennée wrote: > > Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > >> Hi, >> >> Another neanderthal approach to add multi-arch acceptance tests using >> Avocado. >> >> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] >> at using Avocado in QEMU. >> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes >> a bit further: >> - we can now run other archs >> - Avocado is integrated into Travis CI >> >> I'm not sure Travis is the best CI available for those tests, but it runs >> quite fast: >> >> ... >> LINK x86_64-softmmu/qemu-system-x86_64 >> JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 >> JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log >> (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) >> (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) >> (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) >> (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) >> (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) >> (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) >> (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) >> RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 >> JOB TIME : 6.21 s > > Hmm however if I do: > > pip install --user avocado-framework > avocado run tests/acceptance > > I get: > > Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',) > Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',) > Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied') Alex, These are errors related to your installation of Avocado-*VT*[1]. > Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params' > Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi' > You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log > And these from Avocado-*Virt*[2]. Neither of those are used in the tests under "tests/acceptance". > Which I seem to remember from last time I looked into this. I think > avocado makes a bunch of assumptions about the filesystem and OS layout > which is hard to gloss over for something you should be able to run > straight out of the source tree. > > These Avocado-*VT* issues deserve a better look indeed. Would you please describe (if not here, under a GH issue) your environment? Thanks, - Cleber. [1] - https://github.com/avocado-framework/avocado-vt [2] - https://github.com/avocado-framework/avocado-virt >> >> Done. Your build exited with 0. >> >> Job ran for 5 min 34 sec >> >> I couldn't add the Alpha tests because all the vmlinux kernels I found >> online are gzipped, so I'm waiting some upstream Avocado merges before. >> >> Cleber: I hope you can help me pythonizing this series :p >> >> Regards, >> >> Phil. >> >> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html >> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html >> >> Philippe Mathieu-Daudé (6): >> avocado: Add a Test.arch property >> tests/acceptance: Rename the x86-64 specific BootLinuxConsole test >> tests/acceptance: Improve the Avocado tags >> tests/acceptance: Add a BootLinuxConsoleMips test >> tests/acceptance: Add a kludge to not use the default console >> travis: Add Avocado tests >> >> .travis.yml | 8 +++++ >> scripts/qemu.py | 6 ++-- >> tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- >> tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- >> 4 files changed, 67 insertions(+), 8 deletions(-) > > > -- > Alex Bennée > -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ] [ 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3 ]
Cleber Rosa <crosa@redhat.com> writes: > On 06/28/2018 11:46 AM, Alex Bennée wrote: >> >> Philippe Mathieu-Daudé <f4bug@amsat.org> writes: >> >>> Hi, >>> >>> Another neanderthal approach to add multi-arch acceptance tests using >>> Avocado. >>> >>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] >>> at using Avocado in QEMU. >>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes >>> a bit further: >>> - we can now run other archs >>> - Avocado is integrated into Travis CI >>> >>> I'm not sure Travis is the best CI available for those tests, but it runs >>> quite fast: >>> >>> ... >>> LINK x86_64-softmmu/qemu-system-x86_64 >>> JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 >>> JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log >>> (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) >>> (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) >>> (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) >>> (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) >>> (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) >>> (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) >>> (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) >>> RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 >>> JOB TIME : 6.21 s >> >> Hmm however if I do: >> >> pip install --user avocado-framework >> avocado run tests/acceptance >> >> I get: >> >> Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',) >> Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',) >> Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied') > > Alex, > > These are errors related to your installation of Avocado-*VT*[1]. > >> Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params' >> Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi' >> You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log >> > > And these from Avocado-*Virt*[2]. Neither of those are used in the > tests under "tests/acceptance". > >> Which I seem to remember from last time I looked into this. I think >> avocado makes a bunch of assumptions about the filesystem and OS layout >> which is hard to gloss over for something you should be able to run >> straight out of the source tree. >> >> > > These Avocado-*VT* issues deserve a better look indeed. Would you > please describe (if not here, under a GH issue) your environment? Ahh probably from last time I tried to get things up and running. I rm -rf ~/.local and things seem to be working well now. However I was under the impression that VT or virt was needed for this testing. Or is this an old assumption? > > Thanks, > - Cleber. > > [1] - https://github.com/avocado-framework/avocado-vt > [2] - https://github.com/avocado-framework/avocado-virt > >>> >>> Done. Your build exited with 0. >>> >>> Job ran for 5 min 34 sec >>> >>> I couldn't add the Alpha tests because all the vmlinux kernels I found >>> online are gzipped, so I'm waiting some upstream Avocado merges before. >>> >>> Cleber: I hope you can help me pythonizing this series :p >>> >>> Regards, >>> >>> Phil. >>> >>> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html >>> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html >>> >>> Philippe Mathieu-Daudé (6): >>> avocado: Add a Test.arch property >>> tests/acceptance: Rename the x86-64 specific BootLinuxConsole test >>> tests/acceptance: Improve the Avocado tags >>> tests/acceptance: Add a BootLinuxConsoleMips test >>> tests/acceptance: Add a kludge to not use the default console >>> travis: Add Avocado tests >>> >>> .travis.yml | 8 +++++ >>> scripts/qemu.py | 6 ++-- >>> tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- >>> tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- >>> 4 files changed, 67 insertions(+), 8 deletions(-) >> >> >> -- >> Alex Bennée >> -- Alex Bennée
On 06/28/2018 02:13 PM, Alex Bennée wrote: > > Cleber Rosa <crosa@redhat.com> writes: > >> On 06/28/2018 11:46 AM, Alex Bennée wrote: >>> >>> Philippe Mathieu-Daudé <f4bug@amsat.org> writes: >>> >>>> Hi, >>>> >>>> Another neanderthal approach to add multi-arch acceptance tests using >>>> Avocado. >>>> >>>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] >>>> at using Avocado in QEMU. >>>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes >>>> a bit further: >>>> - we can now run other archs >>>> - Avocado is integrated into Travis CI >>>> >>>> I'm not sure Travis is the best CI available for those tests, but it runs >>>> quite fast: >>>> >>>> ... >>>> LINK x86_64-softmmu/qemu-system-x86_64 >>>> JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 >>>> JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log >>>> (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) >>>> (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) >>>> (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) >>>> (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) >>>> (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) >>>> (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) >>>> (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) >>>> RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 >>>> JOB TIME : 6.21 s >>> >>> Hmm however if I do: >>> >>> pip install --user avocado-framework >>> avocado run tests/acceptance >>> >>> I get: >>> >>> Failed to load plugin from module "avocado_vt.plugins.vt_list": ImportError('No module named netaddr',) >>> Failed to load plugin from module "avocado_vt.plugins.vt": ImportError('No module named netaddr',) >>> Failed to load plugin from module "avocado_vt.plugins.vt_bootstrap": OSError(13, 'Permission denied') >> >> Alex, >> >> These are errors related to your installation of Avocado-*VT*[1]. >> >>> Error running method "run" of plugin "virt": 'Namespace' object has no attribute 'default_avocado_params' >>> Avocado crashed unexpectedly: [Errno 13] Permission denied: '/var/lib/libvirt/images/avocado_sg6NRi' >>> You can find details in /var/tmp/avocado-traceback-2018-06-28_16:44:25-DGb1CG.log >>> >> >> And these from Avocado-*Virt*[2]. Neither of those are used in the >> tests under "tests/acceptance". >> >>> Which I seem to remember from last time I looked into this. I think >>> avocado makes a bunch of assumptions about the filesystem and OS layout >>> which is hard to gloss over for something you should be able to run >>> straight out of the source tree. >>> >>> >> >> These Avocado-*VT* issues deserve a better look indeed. Would you >> please describe (if not here, under a GH issue) your environment? > > Ahh probably from last time I tried to get things up and running. I > rm -rf ~/.local and things seem to be working well now. Should we check for a prerequisite version? > > However I was under the impression that VT or virt was needed for this > testing. Or is this an old assumption? I don't use neither of these for my qemu-system testing, but I'm more interested in testing system targets that are different from my host. I think the VT/virt make sens when you test on a same (target, host) tuple and worry about perfs. <snip>
Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > Hi, > > Another neanderthal approach to add multi-arch acceptance tests using > Avocado. > > Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] > at using Avocado in QEMU. > Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes > a bit further: > - we can now run other archs > - Avocado is integrated into Travis CI > > I'm not sure Travis is the best CI available for those tests, but it runs > quite fast: > > ... > LINK x86_64-softmmu/qemu-system-x86_64 > JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 > JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log > (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) > (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) > (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) > (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) > (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) > (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) > (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) > RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | > INTERRUPT 0 | CANCEL 0 That said on my AArch64 server it fairs better: 15:59:19 [alex.bennee@libgomp-d05-dev:~/l/qemu.git] review/avocado-v2 + avocado run tests/acceptance JOB ID : f40e814204ff007c58073ca75abb38bd575f39f2 JOB LOG : /home/alex.bennee/avocado/job-results/job-2018-06-28T15.59-f40e814/job.log (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: CANCEL (0.00 s) (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (2.03 s) (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: CANCEL (0.00 s) (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: CANCEL (0.00 s) (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: CANCEL (0.00 s) (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: CANCEL (0.00 s) (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: CANCEL (0.00 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 6 JOB TIME : 2.47 s Although given the errors in the logs I'm surprised at ERROR 0 > JOB TIME : 6.21 s > > Done. Your build exited with 0. > > Job ran for 5 min 34 sec > > I couldn't add the Alpha tests because all the vmlinux kernels I found > online are gzipped, so I'm waiting some upstream Avocado merges before. > > Cleber: I hope you can help me pythonizing this series :p > > Regards, > > Phil. > > [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html > [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html > > Philippe Mathieu-Daudé (6): > avocado: Add a Test.arch property > tests/acceptance: Rename the x86-64 specific BootLinuxConsole test > tests/acceptance: Improve the Avocado tags > tests/acceptance: Add a BootLinuxConsoleMips test > tests/acceptance: Add a kludge to not use the default console > travis: Add Avocado tests > > .travis.yml | 8 +++++ > scripts/qemu.py | 6 ++-- > tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- > tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- > 4 files changed, 67 insertions(+), 8 deletions(-) -- Alex Bennée
Hi Alex, On 06/28/2018 01:01 PM, Alex Bennée wrote: > Philippe Mathieu-Daudé <f4bug@amsat.org> writes: >> >> Another neanderthal approach to add multi-arch acceptance tests using >> Avocado. >> >> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] >> at using Avocado in QEMU. >> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes >> a bit further: >> - we can now run other archs >> - Avocado is integrated into Travis CI >> >> I'm not sure Travis is the best CI available for those tests, but it runs >> quite fast: >> >> ... >> LINK x86_64-softmmu/qemu-system-x86_64 >> JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 >> JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log >> (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) >> (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) >> (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) >> (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) >> (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) >> (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) >> (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) >> RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | >> INTERRUPT 0 | CANCEL 0 > > That said on my AArch64 server it fairs better: > > 15:59:19 [alex.bennee@libgomp-d05-dev:~/l/qemu.git] review/avocado-v2 + avocado run tests/acceptance > JOB ID : f40e814204ff007c58073ca75abb38bd575f39f2 > JOB LOG : /home/alex.bennee/avocado/job-results/job-2018-06-28T15.59-f40e814/job.log > (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: CANCEL (0.00 s) > (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (2.03 s) > (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: CANCEL (0.00 s) > (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: CANCEL (0.00 s) > (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: CANCEL (0.00 s) > (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: CANCEL (0.00 s) > (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: CANCEL (0.00 s) > RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 6 > JOB TIME : 2.47 s > > Although given the errors in the logs I'm surprised at ERROR 0 Why are you surprised? You were expecting the BootLinuxConsoleMips to fail? > >> JOB TIME : 6.21 s >> >> Done. Your build exited with 0. >> >> Job ran for 5 min 34 sec >> >> I couldn't add the Alpha tests because all the vmlinux kernels I found >> online are gzipped, so I'm waiting some upstream Avocado merges before. >> >> Cleber: I hope you can help me pythonizing this series :p >> >> Regards, >> >> Phil. >> >> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html >> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html >> >> Philippe Mathieu-Daudé (6): >> avocado: Add a Test.arch property >> tests/acceptance: Rename the x86-64 specific BootLinuxConsole test >> tests/acceptance: Improve the Avocado tags >> tests/acceptance: Add a BootLinuxConsoleMips test >> tests/acceptance: Add a kludge to not use the default console >> travis: Add Avocado tests >> >> .travis.yml | 8 +++++ >> scripts/qemu.py | 6 ++-- >> tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- >> tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- >> 4 files changed, 67 insertions(+), 8 deletions(-) > > > -- > Alex Bennée >
Philippe Mathieu-Daudé <f4bug@amsat.org> writes: > Hi Alex, > > On 06/28/2018 01:01 PM, Alex Bennée wrote: >> Philippe Mathieu-Daudé <f4bug@amsat.org> writes: >>> >>> Another neanderthal approach to add multi-arch acceptance tests using >>> Avocado. >>> >>> Since Cleber Rosa work got merged [0], I can restart my previous attempt [1] >>> at using Avocado in QEMU. >>> Cleber Rosa rewrote my previous v1 [1], in good python, now this v2 pushes >>> a bit further: >>> - we can now run other archs >>> - Avocado is integrated into Travis CI >>> >>> I'm not sure Travis is the best CI available for those tests, but it runs >>> quite fast: >>> >>> ... >>> LINK x86_64-softmmu/qemu-system-x86_64 >>> JOB ID : a3a56af3c02d193e862ce660aae1d9c72926dcb6 >>> JOB LOG : /home/travis/avocado/job-results/job-2018-06-21T23.49-a3a56af/job.log >>> (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: PASS (3.89 s) >>> (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (1.83 s) >>> (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.05 s) >>> (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.04 s) >>> (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s) >>> (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.05 s) >>> (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.05 s) >>> RESULTS : PASS 7 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | >>> INTERRUPT 0 | CANCEL 0 >> >> That said on my AArch64 server it fairs better: >> >> 15:59:19 [alex.bennee@libgomp-d05-dev:~/l/qemu.git] review/avocado-v2 + avocado run tests/acceptance >> JOB ID : f40e814204ff007c58073ca75abb38bd575f39f2 >> JOB LOG : /home/alex.bennee/avocado/job-results/job-2018-06-28T15.59-f40e814/job.log >> (1/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleX86_64.test: CANCEL (0.00 s) >> (2/7) tests/acceptance/boot_linux_console.py:BootLinuxConsoleMips.test: PASS (2.03 s) >> (3/7) tests/acceptance/version.py:Version.test_qmp_human_info_version: CANCEL (0.00 s) >> (4/7) tests/acceptance/vnc.py:Vnc.test_no_vnc: CANCEL (0.00 s) >> (5/7) tests/acceptance/vnc.py:Vnc.test_no_vnc_change_password: CANCEL (0.00 s) >> (6/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: CANCEL (0.00 s) >> (7/7) tests/acceptance/vnc.py:Vnc.test_vnc_change_password: CANCEL (0.00 s) >> RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 6 >> JOB TIME : 2.47 s >> >> Although given the errors in the logs I'm surprised at ERROR 0 > > Why are you surprised? You were expecting the BootLinuxConsoleMips to > fail? No but a bunch have CANCELed... maybe they are skipping but if the test is run and it fails to complete I would expect that to be an error. > >> >>> JOB TIME : 6.21 s >>> >>> Done. Your build exited with 0. >>> >>> Job ran for 5 min 34 sec >>> >>> I couldn't add the Alpha tests because all the vmlinux kernels I found >>> online are gzipped, so I'm waiting some upstream Avocado merges before. >>> >>> Cleber: I hope you can help me pythonizing this series :p >>> >>> Regards, >>> >>> Phil. >>> >>> [0]: http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg04531.html >>> [1]: http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03076.html >>> >>> Philippe Mathieu-Daudé (6): >>> avocado: Add a Test.arch property >>> tests/acceptance: Rename the x86-64 specific BootLinuxConsole test >>> tests/acceptance: Improve the Avocado tags >>> tests/acceptance: Add a BootLinuxConsoleMips test >>> tests/acceptance: Add a kludge to not use the default console >>> travis: Add Avocado tests >>> >>> .travis.yml | 8 +++++ >>> scripts/qemu.py | 6 ++-- >>> tests/acceptance/avocado_qemu/__init__.py | 17 ++++++--- >>> tests/acceptance/boot_linux_console.py | 44 +++++++++++++++++++++-- >>> 4 files changed, 67 insertions(+), 8 deletions(-) >> >> >> -- >> Alex Bennée >> -- Alex Bennée
© 2016 - 2024 Red Hat, Inc.