This patch series converts the tests/avocado/tuxrun_baselines.py
to the new functional test framework. While converting the sh4 test,
I noticed that the Avocado test was completely broken, so I included
a fix (revert) for that problem in this series, too.

Thomas Huth (17):
  tests/functional: Add a base class for the TuxRun tests
  tests/functional: Convert the Avocado ppc64 tuxrun tests
  tests/functional: Convert the Avocado aarch64 tuxrun tests
  tests/functional: Convert the Avocado sparc64 tuxrun test
  tests/functional: Convert the Avocado s390x tuxrun test
  tests/functional: Convert the Avocado arm tuxrun tests
  tests/functional: Convert the Avocado riscv32 tuxrun tests
  tests/functional: Convert the Avocado riscv64 tuxrun tests
  tests/functional: Convert the Avocado i386 tuxrun test
  tests/functional: Convert the Avocado x86_64 tuxrun test
  tests/functional: Convert the Avocado mips tuxrun test
  tests/functional: Convert the Avocado mipsel tuxrun test
  tests/functional: Convert the Avocado mips64 tuxrun test
  tests/functional: Convert the Avocado mips64el tuxrun test
  tests/functional: Convert the Avocado ppc32 tuxrun test
  Revert "hw/sh4/r2d: Realize IDE controller before accessing it"
  tests/functional: Convert the Avocado sh4 tuxrun test

 MAINTAINERS                              |   4 +-
 hw/sh4/r2d.c                             |   2 +-
 tests/avocado/tuxrun_baselines.py        | 620 -----------------------
 tests/functional/meson.build             |  28 +
 tests/functional/qemu_test/tuxruntest.py | 158 ++++++
 tests/functional/test_aarch64_tuxrun.py  |  50 ++
 tests/functional/test_arm_tuxrun.py      |  70 +++
 tests/functional/test_i386_tuxrun.py     |  35 ++
 tests/functional/test_mips64_tuxrun.py   |  35 ++
 tests/functional/test_mips64el_tuxrun.py |  35 ++
 tests/functional/test_mips_tuxrun.py     |  36 ++
 tests/functional/test_mipsel_tuxrun.py   |  36 ++
 tests/functional/test_ppc64_tuxrun.py    | 110 ++++
 tests/functional/test_ppc_tuxrun.py      |  35 ++
 tests/functional/test_riscv32_tuxrun.py  |  38 ++
 tests/functional/test_riscv64_tuxrun.py  |  38 ++
 tests/functional/test_s390x_tuxrun.py    |  34 ++
 tests/functional/test_sh4_tuxrun.py      |  57 +++
 tests/functional/test_sparc64_tuxrun.py  |  34 ++
 tests/functional/test_x86_64_tuxrun.py   |  36 ++
 20 files changed, 869 insertions(+), 622 deletions(-)
 delete mode 100644 tests/avocado/tuxrun_baselines.py
 create mode 100644 tests/functional/qemu_test/tuxruntest.py
 create mode 100755 tests/functional/test_aarch64_tuxrun.py
 create mode 100755 tests/functional/test_arm_tuxrun.py
 create mode 100755 tests/functional/test_i386_tuxrun.py
 create mode 100755 tests/functional/test_mips64_tuxrun.py
 create mode 100755 tests/functional/test_mips64el_tuxrun.py
 create mode 100755 tests/functional/test_mips_tuxrun.py
 create mode 100755 tests/functional/test_mipsel_tuxrun.py
 create mode 100755 tests/functional/test_ppc64_tuxrun.py
 create mode 100755 tests/functional/test_ppc_tuxrun.py
 create mode 100755 tests/functional/test_riscv32_tuxrun.py
 create mode 100755 tests/functional/test_riscv64_tuxrun.py
 create mode 100755 tests/functional/test_s390x_tuxrun.py
 create mode 100755 tests/functional/test_sh4_tuxrun.py
 create mode 100755 tests/functional/test_sparc64_tuxrun.py
 create mode 100755 tests/functional/test_x86_64_tuxrun.py

-- 
2.46.1

Thomas Huth <thuth@redhat.com> writes:

> This patch series converts the tests/avocado/tuxrun_baselines.py
> to the new functional test framework. While converting the sh4 test,
> I noticed that the Avocado test was completely broken, so I included
> a fix (revert) for that problem in this series, too.

How can I run them from meson? I don't see them in list (but make
check-functional works). Shouldn't I be able to do:

  ./pyvenv/bin/meson test qemu:func-thorough

?
>
> Thomas Huth (17):
>   tests/functional: Add a base class for the TuxRun tests
>   tests/functional: Convert the Avocado ppc64 tuxrun tests
>   tests/functional: Convert the Avocado aarch64 tuxrun tests
>   tests/functional: Convert the Avocado sparc64 tuxrun test
>   tests/functional: Convert the Avocado s390x tuxrun test
>   tests/functional: Convert the Avocado arm tuxrun tests
>   tests/functional: Convert the Avocado riscv32 tuxrun tests
>   tests/functional: Convert the Avocado riscv64 tuxrun tests
>   tests/functional: Convert the Avocado i386 tuxrun test
>   tests/functional: Convert the Avocado x86_64 tuxrun test
>   tests/functional: Convert the Avocado mips tuxrun test
>   tests/functional: Convert the Avocado mipsel tuxrun test
>   tests/functional: Convert the Avocado mips64 tuxrun test
>   tests/functional: Convert the Avocado mips64el tuxrun test
>   tests/functional: Convert the Avocado ppc32 tuxrun test
>   Revert "hw/sh4/r2d: Realize IDE controller before accessing it"
>   tests/functional: Convert the Avocado sh4 tuxrun test
>
>  MAINTAINERS                              |   4 +-
>  hw/sh4/r2d.c                             |   2 +-
>  tests/avocado/tuxrun_baselines.py        | 620 -----------------------
>  tests/functional/meson.build             |  28 +
>  tests/functional/qemu_test/tuxruntest.py | 158 ++++++
>  tests/functional/test_aarch64_tuxrun.py  |  50 ++
>  tests/functional/test_arm_tuxrun.py      |  70 +++
>  tests/functional/test_i386_tuxrun.py     |  35 ++
>  tests/functional/test_mips64_tuxrun.py   |  35 ++
>  tests/functional/test_mips64el_tuxrun.py |  35 ++
>  tests/functional/test_mips_tuxrun.py     |  36 ++
>  tests/functional/test_mipsel_tuxrun.py   |  36 ++
>  tests/functional/test_ppc64_tuxrun.py    | 110 ++++
>  tests/functional/test_ppc_tuxrun.py      |  35 ++
>  tests/functional/test_riscv32_tuxrun.py  |  38 ++
>  tests/functional/test_riscv64_tuxrun.py  |  38 ++
>  tests/functional/test_s390x_tuxrun.py    |  34 ++
>  tests/functional/test_sh4_tuxrun.py      |  57 +++
>  tests/functional/test_sparc64_tuxrun.py  |  34 ++
>  tests/functional/test_x86_64_tuxrun.py   |  36 ++
>  20 files changed, 869 insertions(+), 622 deletions(-)
>  delete mode 100644 tests/avocado/tuxrun_baselines.py
>  create mode 100644 tests/functional/qemu_test/tuxruntest.py
>  create mode 100755 tests/functional/test_aarch64_tuxrun.py
>  create mode 100755 tests/functional/test_arm_tuxrun.py
>  create mode 100755 tests/functional/test_i386_tuxrun.py
>  create mode 100755 tests/functional/test_mips64_tuxrun.py
>  create mode 100755 tests/functional/test_mips64el_tuxrun.py
>  create mode 100755 tests/functional/test_mips_tuxrun.py
>  create mode 100755 tests/functional/test_mipsel_tuxrun.py
>  create mode 100755 tests/functional/test_ppc64_tuxrun.py
>  create mode 100755 tests/functional/test_ppc_tuxrun.py
>  create mode 100755 tests/functional/test_riscv32_tuxrun.py
>  create mode 100755 tests/functional/test_riscv64_tuxrun.py
>  create mode 100755 tests/functional/test_s390x_tuxrun.py
>  create mode 100755 tests/functional/test_sh4_tuxrun.py
>  create mode 100755 tests/functional/test_sparc64_tuxrun.py
>  create mode 100755 tests/functional/test_x86_64_tuxrun.py

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

On 13/10/2024 17.27, Alex Bennée wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> This patch series converts the tests/avocado/tuxrun_baselines.py
>> to the new functional test framework. While converting the sh4 test,
>> I noticed that the Avocado test was completely broken, so I included
>> a fix (revert) for that problem in this series, too.
> 
> How can I run them from meson? I don't see them in list (but make
> check-functional works). Shouldn't I be able to do:
> 
>    ./pyvenv/bin/meson test qemu:func-thorough
> 
> ?

Did you regenerate the meson files before running this command?

Anyway, the "official" way to run the tests is:

  make check-functional

This should take care of
1) Regenerating the meson files if necessary
2) Precaching the assets if necessary
3) Running the tests

Hope that works,
   Thomas

On Mon, Oct 14, 2024 at 08:12:39AM +0200, Thomas Huth wrote:
> On 13/10/2024 17.27, Alex Bennée wrote:
> > Thomas Huth <thuth@redhat.com> writes:
> > 
> > > This patch series converts the tests/avocado/tuxrun_baselines.py
> > > to the new functional test framework. While converting the sh4 test,
> > > I noticed that the Avocado test was completely broken, so I included
> > > a fix (revert) for that problem in this series, too.
> > 
> > How can I run them from meson? I don't see them in list (but make
> > check-functional works). Shouldn't I be able to do:
> > 
> >    ./pyvenv/bin/meson test qemu:func-thorough
> > 
> > ?
> 
> Did you regenerate the meson files before running this command?
> 
> Anyway, the "official" way to run the tests is:
> 
>  make check-functional
> 
> This should take care of
> 1) Regenerating the meson files if necessary
> 2) Precaching the assets if necessary
> 3) Running the tests

Or if you just want to run a single test, directly execute the python
file eg  ./tests/functional/testname.py

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On 11/10/2024 15.19, Thomas Huth wrote:
> This patch series converts the tests/avocado/tuxrun_baselines.py
> to the new functional test framework. While converting the sh4 test,
> I noticed that the Avocado test was completely broken, so I included
> a fix (revert) for that problem in this series, too.
> 
> Thomas Huth (17):
>    tests/functional: Add a base class for the TuxRun tests
>    tests/functional: Convert the Avocado ppc64 tuxrun tests
>    tests/functional: Convert the Avocado aarch64 tuxrun tests
>    tests/functional: Convert the Avocado sparc64 tuxrun test
>    tests/functional: Convert the Avocado s390x tuxrun test
>    tests/functional: Convert the Avocado arm tuxrun tests
>    tests/functional: Convert the Avocado riscv32 tuxrun tests
>    tests/functional: Convert the Avocado riscv64 tuxrun tests
>    tests/functional: Convert the Avocado i386 tuxrun test
>    tests/functional: Convert the Avocado x86_64 tuxrun test
>    tests/functional: Convert the Avocado mips tuxrun test
>    tests/functional: Convert the Avocado mipsel tuxrun test
>    tests/functional: Convert the Avocado mips64 tuxrun test
>    tests/functional: Convert the Avocado mips64el tuxrun test
>    tests/functional: Convert the Avocado ppc32 tuxrun test
>    Revert "hw/sh4/r2d: Realize IDE controller before accessing it"
>    tests/functional: Convert the Avocado sh4 tuxrun test
> 
>   MAINTAINERS                              |   4 +-
>   hw/sh4/r2d.c                             |   2 +-
>   tests/avocado/tuxrun_baselines.py        | 620 -----------------------
>   tests/functional/meson.build             |  28 +
>   tests/functional/qemu_test/tuxruntest.py | 158 ++++++
>   tests/functional/test_aarch64_tuxrun.py  |  50 ++
>   tests/functional/test_arm_tuxrun.py      |  70 +++
>   tests/functional/test_i386_tuxrun.py     |  35 ++
>   tests/functional/test_mips64_tuxrun.py   |  35 ++
>   tests/functional/test_mips64el_tuxrun.py |  35 ++
>   tests/functional/test_mips_tuxrun.py     |  36 ++
>   tests/functional/test_mipsel_tuxrun.py   |  36 ++
>   tests/functional/test_ppc64_tuxrun.py    | 110 ++++
>   tests/functional/test_ppc_tuxrun.py      |  35 ++
>   tests/functional/test_riscv32_tuxrun.py  |  38 ++
>   tests/functional/test_riscv64_tuxrun.py  |  38 ++
>   tests/functional/test_s390x_tuxrun.py    |  34 ++
>   tests/functional/test_sh4_tuxrun.py      |  57 +++
>   tests/functional/test_sparc64_tuxrun.py  |  34 ++
>   tests/functional/test_x86_64_tuxrun.py   |  36 ++
>   20 files changed, 869 insertions(+), 622 deletions(-)
>   delete mode 100644 tests/avocado/tuxrun_baselines.py
>   create mode 100644 tests/functional/qemu_test/tuxruntest.py
>   create mode 100755 tests/functional/test_aarch64_tuxrun.py
>   create mode 100755 tests/functional/test_arm_tuxrun.py
>   create mode 100755 tests/functional/test_i386_tuxrun.py
>   create mode 100755 tests/functional/test_mips64_tuxrun.py
>   create mode 100755 tests/functional/test_mips64el_tuxrun.py
>   create mode 100755 tests/functional/test_mips_tuxrun.py
>   create mode 100755 tests/functional/test_mipsel_tuxrun.py
>   create mode 100755 tests/functional/test_ppc64_tuxrun.py
>   create mode 100755 tests/functional/test_ppc_tuxrun.py
>   create mode 100755 tests/functional/test_riscv32_tuxrun.py
>   create mode 100755 tests/functional/test_riscv64_tuxrun.py
>   create mode 100755 tests/functional/test_s390x_tuxrun.py
>   create mode 100755 tests/functional/test_sh4_tuxrun.py
>   create mode 100755 tests/functional/test_sparc64_tuxrun.py
>   create mode 100755 tests/functional/test_x86_64_tuxrun.py

Ok, if there are no objections, I'll include this patch series in my next 
pull request, with this small fix squashed into the final patch:

diff --git a/MAINTAINERS b/MAINTAINERS
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4071,7 +4071,8 @@ F: scripts/ci/
  F: tests/docker/
  F: tests/vm/
  F: tests/lcitool/
-F: tests/avocado/tuxrun_baselines.py
+F: tests/functional/qemu_test/tuxruntest.py
+F: tests/functional/test_*_tuxrun.py
  F: scripts/archive-source.sh
  F: docs/devel/testing.rst
  W: https://gitlab.com/qemu-project/qemu/pipelines

  Thomas