docs/system: Fix the information on how to run certain functional tests

[PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Thomas Huth 3 weeks ago

The tests have been converted to the functional framework, so
we should not talk about Avocado here anymore.

Fixes: f7d6b772200 ("tests/functional: Convert BananaPi tests to the functional framework")
Fixes: 380f7268b7b ("tests/functional: Convert the OrangePi tests to the functional framework")
Fixes: 4c0a2df81c9 ("tests/functional: Convert some tests that download files via fetch_asset()")
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/system/arm/bananapi_m2u.rst | 5 ++---
 docs/system/arm/orangepi.rst     | 6 +++---
 docs/system/devices/igb.rst      | 4 ++--
 3 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/docs/system/arm/bananapi_m2u.rst b/docs/system/arm/bananapi_m2u.rst
index 587b4886553..d30db8d04c3 100644
--- a/docs/system/arm/bananapi_m2u.rst
+++ b/docs/system/arm/bananapi_m2u.rst
@@ -135,6 +135,5 @@ provide the following command:
 .. code-block:: bash
 
   $ cd qemu-build-dir
-  $ AVOCADO_ALLOW_LARGE_STORAGE=yes tests/venv/bin/avocado \
-    --verbose --show=app,console run -t machine:bpim2u \
-    ../tests/avocado/boot_linux_console.py
+  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
+    meson test --suite thorough func-arm-arm_bpim2u
diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst
index db87e81fec4..8b9448ca7b0 100644
--- a/docs/system/arm/orangepi.rst
+++ b/docs/system/arm/orangepi.rst
@@ -257,9 +257,9 @@ Orange Pi PC integration tests
 
 The Orange Pi PC machine has several integration tests included.
 To run the whole set of tests, build QEMU from source and simply
-provide the following command:
+provide the following command from the build directory:
 
 .. code-block:: bash
 
-  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
-     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
+  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
+    meson test --suite thorough func-arm-arm_orangepi
diff --git a/docs/system/devices/igb.rst b/docs/system/devices/igb.rst
index 04e79dfe549..9145af5c757 100644
--- a/docs/system/devices/igb.rst
+++ b/docs/system/devices/igb.rst
@@ -57,11 +57,11 @@ directory:
   meson test qtest-x86_64/qos-test
 
 ethtool can test register accesses, interrupts, etc. It is automated as an
-Avocado test and can be ran with the following command:
+functional test and can be ran with the following command:
 
 .. code:: shell
 
-  make check-avocado AVOCADO_TESTS=tests/avocado/netdev-ethtool.py
+  meson test --suite thorough func-x86_64-netdev_ethtool
 
 References
 ==========
-- 
2.48.1

Re: [PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Niek Linnenbank 2 weeks, 2 days ago

Hi Thomas,

On Tue, Mar 11, 2025 at 5:08 PM Thomas Huth <thuth@redhat.com> wrote:

> The tests have been converted to the functional framework, so
> we should not talk about Avocado here anymore.
>
> Fixes: f7d6b772200 ("tests/functional: Convert BananaPi tests to the
> functional framework")
> Fixes: 380f7268b7b ("tests/functional: Convert the OrangePi tests to the
> functional framework")
> Fixes: 4c0a2df81c9 ("tests/functional: Convert some tests that download
> files via fetch_asset()")
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  docs/system/arm/bananapi_m2u.rst | 5 ++---
>  docs/system/arm/orangepi.rst     | 6 +++---
>  docs/system/devices/igb.rst      | 4 ++--
>  3 files changed, 7 insertions(+), 8 deletions(-)
>
> diff --git a/docs/system/arm/bananapi_m2u.rst
> b/docs/system/arm/bananapi_m2u.rst
> index 587b4886553..d30db8d04c3 100644
> --- a/docs/system/arm/bananapi_m2u.rst
> +++ b/docs/system/arm/bananapi_m2u.rst
> @@ -135,6 +135,5 @@ provide the following command:
>  .. code-block:: bash
>
>    $ cd qemu-build-dir
> -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes tests/venv/bin/avocado \
> -    --verbose --show=app,console run -t machine:bpim2u \
> -    ../tests/avocado/boot_linux_console.py
> +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
> +    meson test --suite thorough func-arm-arm_bpim2u
> diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst
> index db87e81fec4..8b9448ca7b0 100644
> --- a/docs/system/arm/orangepi.rst
> +++ b/docs/system/arm/orangepi.rst
> @@ -257,9 +257,9 @@ Orange Pi PC integration tests
>
Perhaps for consistency, we can also rename to 'functional tests' here.


>
>  The Orange Pi PC machine has several integration tests included.
>
And same on this line.


>  To run the whole set of tests, build QEMU from source and simply
> -provide the following command:
> +provide the following command from the build directory:
>
>  .. code-block:: bash
>
> -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
> -     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
> +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
> +    meson test --suite thorough func-arm-arm_orangepi
>
I've tried to run on my Ubuntu 24.04.1 LTS based system using this exact
same command, but got this error:

$ QEMU_TEST_ALLOW_LARGE_STORAGE=1 meson test --suite thorough
func-arm-arm_orangepi

ERROR: Build data file '/home/user/qemu/build/meson-private/build.dat'
references functions or classes that don't exist. This probably means that
it was generated with an old version of meson. Consider reconfiguring the
directory with "meson setup --reconfigure".

The meson version I have installed via apt-get is 1.3.2-1ubuntu1. Only when
running using the 'meson' command from the pyvenv, it runs OK:

$ QEMU_TEST_ALLOW_LARGE_STORAGE=1 ./pyvenv/bin/meson test --suite thorough
func-arm-arm_orangepi
ninja: Entering directory `/home/user/qemu/build'
[1/6] Generating qemu-version.h with a custom command (wrapped by meson to
capture output)
1/1 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_orangepi
     OK             165.81s   5 subtests passed



> diff --git a/docs/system/devices/igb.rst b/docs/system/devices/igb.rst
> index 04e79dfe549..9145af5c757 100644
> --- a/docs/system/devices/igb.rst
> +++ b/docs/system/devices/igb.rst
> @@ -57,11 +57,11 @@ directory:
>    meson test qtest-x86_64/qos-test
>
>  ethtool can test register accesses, interrupts, etc. It is automated as an
> -Avocado test and can be ran with the following command:
> +functional test and can be ran with the following command:
>
>  .. code:: shell
>
> -  make check-avocado AVOCADO_TESTS=tests/avocado/netdev-ethtool.py
> +  meson test --suite thorough func-x86_64-netdev_ethtool
>
>  References
>  ==========
> --
> 2.48.1
>
>

-- 
Niek Linnenbank

Re: [PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Thomas Huth 2 weeks, 2 days ago

  Hi!

On 16/03/2025 21.21, Niek Linnenbank wrote:
> Hi Thomas,
> 
> On Tue, Mar 11, 2025 at 5:08 PM Thomas Huth <thuth@redhat.com 
> <mailto:thuth@redhat.com>> wrote:
> 
>     The tests have been converted to the functional framework, so
>     we should not talk about Avocado here anymore.
...
>     diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst
>     index db87e81fec4..8b9448ca7b0 100644
>     --- a/docs/system/arm/orangepi.rst
>     +++ b/docs/system/arm/orangepi.rst
>     @@ -257,9 +257,9 @@ Orange Pi PC integration tests
> 
> Perhaps for consistency, we can also rename to 'functional tests' here.

Agreed, we don't use the term "integration tests" for this anymore.

> 
>       The Orange Pi PC machine has several integration tests included.
> 
> And same on this line.
> 
>       To run the whole set of tests, build QEMU from source and simply
>     -provide the following command:
>     +provide the following command from the build directory:
> 
>       .. code-block:: bash
> 
>     -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
>     -     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
>     +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
>     +    meson test --suite thorough func-arm-arm_orangepi
> 
> I've tried to run on my Ubuntu 24.04.1 LTS based system using this exact 
> same command, but got this error:
> 
> $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 meson test --suite thorough func-arm- 
> arm_orangepi
> 
> ERROR: Build data file '/home/user/qemu/build/meson-private/build.dat' 
> references functions or classes that don't exist. This probably means that 
> it was generated with an old version of meson. Consider reconfiguring the 
> directory with "meson setup --reconfigure".
> 
> The meson version I have installed via apt-get is 1.3.2-1ubuntu1. Only when 
> running using the 'meson' command from the pyvenv, it runs OK:
> 
> $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 ./pyvenv/bin/meson test --suite thorough 
> func-arm-arm_orangepi

Oh, you're right! Thanks for catching it!

I guess I still had a "export PYTHONPATH=$HOME/qemu/python" in my 
environment, so I did not notice. Would you like to send a patch, or want me 
to do it?

  Thomas

Re: [PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Niek Linnenbank 2 weeks, 1 day ago

Hi Thomas,

On Mon, Mar 17, 2025 at 7:50 AM Thomas Huth <thuth@redhat.com> wrote:

>   Hi!
>
> On 16/03/2025 21.21, Niek Linnenbank wrote:
> > Hi Thomas,
> >
> > On Tue, Mar 11, 2025 at 5:08 PM Thomas Huth <thuth@redhat.com
> > <mailto:thuth@redhat.com>> wrote:
> >
> >     The tests have been converted to the functional framework, so
> >     we should not talk about Avocado here anymore.
> ...
> >     diff --git a/docs/system/arm/orangepi.rst
> b/docs/system/arm/orangepi.rst
> >     index db87e81fec4..8b9448ca7b0 100644
> >     --- a/docs/system/arm/orangepi.rst
> >     +++ b/docs/system/arm/orangepi.rst
> >     @@ -257,9 +257,9 @@ Orange Pi PC integration tests
> >
> > Perhaps for consistency, we can also rename to 'functional tests' here.
>
> Agreed, we don't use the term "integration tests" for this anymore.
>
> >
> >       The Orange Pi PC machine has several integration tests included.
> >
> > And same on this line.
> >
> >       To run the whole set of tests, build QEMU from source and simply
> >     -provide the following command:
> >     +provide the following command from the build directory:
> >
> >       .. code-block:: bash
> >
> >     -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
> >     -     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
> >     +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
> >     +    meson test --suite thorough func-arm-arm_orangepi
> >
> > I've tried to run on my Ubuntu 24.04.1 LTS based system using this exact
> > same command, but got this error:
> >
> > $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 meson test --suite thorough func-arm-
> > arm_orangepi
> >
> > ERROR: Build data file '/home/user/qemu/build/meson-private/build.dat'
> > references functions or classes that don't exist. This probably means
> that
> > it was generated with an old version of meson. Consider reconfiguring
> the
> > directory with "meson setup --reconfigure".
> >
> > The meson version I have installed via apt-get is 1.3.2-1ubuntu1. Only
> when
> > running using the 'meson' command from the pyvenv, it runs OK:
> >
> > $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 ./pyvenv/bin/meson test --suite
> thorough
> > func-arm-arm_orangepi
>
> Oh, you're right! Thanks for catching it!
>
> I guess I still had a "export PYTHONPATH=$HOME/qemu/python" in my
> environment, so I did not notice. Would you like to send a patch, or want
> me
> to do it?
>

Yes please feel free to go ahead with the patch. With the above minor
remarks resolved, it looks fine to me:

Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>

Regards,
Niek


>
>   Thomas
>
>

-- 
Niek Linnenbank