docs/system: Fix the information on how to run certain functional tests

[PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Thomas Huth 11 months ago

The tests have been converted to the functional framework, so
we should not talk about Avocado here anymore.

Fixes: f7d6b772200 ("tests/functional: Convert BananaPi tests to the functional framework")
Fixes: 380f7268b7b ("tests/functional: Convert the OrangePi tests to the functional framework")
Fixes: 4c0a2df81c9 ("tests/functional: Convert some tests that download files via fetch_asset()")
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/system/arm/bananapi_m2u.rst | 5 ++---
 docs/system/arm/orangepi.rst     | 6 +++---
 docs/system/devices/igb.rst      | 4 ++--
 3 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/docs/system/arm/bananapi_m2u.rst b/docs/system/arm/bananapi_m2u.rst
index 587b4886553..d30db8d04c3 100644
--- a/docs/system/arm/bananapi_m2u.rst
+++ b/docs/system/arm/bananapi_m2u.rst
@@ -135,6 +135,5 @@ provide the following command:
 .. code-block:: bash
 
   $ cd qemu-build-dir
-  $ AVOCADO_ALLOW_LARGE_STORAGE=yes tests/venv/bin/avocado \
-    --verbose --show=app,console run -t machine:bpim2u \
-    ../tests/avocado/boot_linux_console.py
+  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
+    meson test --suite thorough func-arm-arm_bpim2u
diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst
index db87e81fec4..8b9448ca7b0 100644
--- a/docs/system/arm/orangepi.rst
+++ b/docs/system/arm/orangepi.rst
@@ -257,9 +257,9 @@ Orange Pi PC integration tests
 
 The Orange Pi PC machine has several integration tests included.
 To run the whole set of tests, build QEMU from source and simply
-provide the following command:
+provide the following command from the build directory:
 
 .. code-block:: bash
 
-  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
-     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
+  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
+    meson test --suite thorough func-arm-arm_orangepi
diff --git a/docs/system/devices/igb.rst b/docs/system/devices/igb.rst
index 04e79dfe549..9145af5c757 100644
--- a/docs/system/devices/igb.rst
+++ b/docs/system/devices/igb.rst
@@ -57,11 +57,11 @@ directory:
   meson test qtest-x86_64/qos-test
 
 ethtool can test register accesses, interrupts, etc. It is automated as an
-Avocado test and can be ran with the following command:
+functional test and can be ran with the following command:
 
 .. code:: shell
 
-  make check-avocado AVOCADO_TESTS=tests/avocado/netdev-ethtool.py
+  meson test --suite thorough func-x86_64-netdev_ethtool
 
 References
 ==========
-- 
2.48.1

Re: [PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Niek Linnenbank 10 months, 4 weeks ago

Hi Thomas,

On Tue, Mar 11, 2025 at 5:08 PM Thomas Huth <thuth@redhat.com> wrote:

> The tests have been converted to the functional framework, so
> we should not talk about Avocado here anymore.
>
> Fixes: f7d6b772200 ("tests/functional: Convert BananaPi tests to the
> functional framework")
> Fixes: 380f7268b7b ("tests/functional: Convert the OrangePi tests to the
> functional framework")
> Fixes: 4c0a2df81c9 ("tests/functional: Convert some tests that download
> files via fetch_asset()")
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  docs/system/arm/bananapi_m2u.rst | 5 ++---
>  docs/system/arm/orangepi.rst     | 6 +++---
>  docs/system/devices/igb.rst      | 4 ++--
>  3 files changed, 7 insertions(+), 8 deletions(-)
>
> diff --git a/docs/system/arm/bananapi_m2u.rst
> b/docs/system/arm/bananapi_m2u.rst
> index 587b4886553..d30db8d04c3 100644
> --- a/docs/system/arm/bananapi_m2u.rst
> +++ b/docs/system/arm/bananapi_m2u.rst
> @@ -135,6 +135,5 @@ provide the following command:
>  .. code-block:: bash
>
>    $ cd qemu-build-dir
> -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes tests/venv/bin/avocado \
> -    --verbose --show=app,console run -t machine:bpim2u \
> -    ../tests/avocado/boot_linux_console.py
> +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
> +    meson test --suite thorough func-arm-arm_bpim2u
> diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst
> index db87e81fec4..8b9448ca7b0 100644
> --- a/docs/system/arm/orangepi.rst
> +++ b/docs/system/arm/orangepi.rst
> @@ -257,9 +257,9 @@ Orange Pi PC integration tests
>
Perhaps for consistency, we can also rename to 'functional tests' here.


>
>  The Orange Pi PC machine has several integration tests included.
>
And same on this line.


>  To run the whole set of tests, build QEMU from source and simply
> -provide the following command:
> +provide the following command from the build directory:
>
>  .. code-block:: bash
>
> -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
> -     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
> +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
> +    meson test --suite thorough func-arm-arm_orangepi
>
I've tried to run on my Ubuntu 24.04.1 LTS based system using this exact
same command, but got this error:

$ QEMU_TEST_ALLOW_LARGE_STORAGE=1 meson test --suite thorough
func-arm-arm_orangepi

ERROR: Build data file '/home/user/qemu/build/meson-private/build.dat'
references functions or classes that don't exist. This probably means that
it was generated with an old version of meson. Consider reconfiguring the
directory with "meson setup --reconfigure".

The meson version I have installed via apt-get is 1.3.2-1ubuntu1. Only when
running using the 'meson' command from the pyvenv, it runs OK:

$ QEMU_TEST_ALLOW_LARGE_STORAGE=1 ./pyvenv/bin/meson test --suite thorough
func-arm-arm_orangepi
ninja: Entering directory `/home/user/qemu/build'
[1/6] Generating qemu-version.h with a custom command (wrapped by meson to
capture output)
1/1 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_orangepi
     OK             165.81s   5 subtests passed



> diff --git a/docs/system/devices/igb.rst b/docs/system/devices/igb.rst
> index 04e79dfe549..9145af5c757 100644
> --- a/docs/system/devices/igb.rst
> +++ b/docs/system/devices/igb.rst
> @@ -57,11 +57,11 @@ directory:
>    meson test qtest-x86_64/qos-test
>
>  ethtool can test register accesses, interrupts, etc. It is automated as an
> -Avocado test and can be ran with the following command:
> +functional test and can be ran with the following command:
>
>  .. code:: shell
>
> -  make check-avocado AVOCADO_TESTS=tests/avocado/netdev-ethtool.py
> +  meson test --suite thorough func-x86_64-netdev_ethtool
>
>  References
>  ==========
> --
> 2.48.1
>
>

-- 
Niek Linnenbank

Re: [PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Thomas Huth 10 months, 3 weeks ago

  Hi!

On 16/03/2025 21.21, Niek Linnenbank wrote:
> Hi Thomas,
> 
> On Tue, Mar 11, 2025 at 5:08 PM Thomas Huth <thuth@redhat.com 
> <mailto:thuth@redhat.com>> wrote:
> 
>     The tests have been converted to the functional framework, so
>     we should not talk about Avocado here anymore.
...
>     diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst
>     index db87e81fec4..8b9448ca7b0 100644
>     --- a/docs/system/arm/orangepi.rst
>     +++ b/docs/system/arm/orangepi.rst
>     @@ -257,9 +257,9 @@ Orange Pi PC integration tests
> 
> Perhaps for consistency, we can also rename to 'functional tests' here.

Agreed, we don't use the term "integration tests" for this anymore.

> 
>       The Orange Pi PC machine has several integration tests included.
> 
> And same on this line.
> 
>       To run the whole set of tests, build QEMU from source and simply
>     -provide the following command:
>     +provide the following command from the build directory:
> 
>       .. code-block:: bash
> 
>     -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
>     -     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
>     +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
>     +    meson test --suite thorough func-arm-arm_orangepi
> 
> I've tried to run on my Ubuntu 24.04.1 LTS based system using this exact 
> same command, but got this error:
> 
> $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 meson test --suite thorough func-arm- 
> arm_orangepi
> 
> ERROR: Build data file '/home/user/qemu/build/meson-private/build.dat' 
> references functions or classes that don't exist. This probably means that 
> it was generated with an old version of meson. Consider reconfiguring the 
> directory with "meson setup --reconfigure".
> 
> The meson version I have installed via apt-get is 1.3.2-1ubuntu1. Only when 
> running using the 'meson' command from the pyvenv, it runs OK:
> 
> $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 ./pyvenv/bin/meson test --suite thorough 
> func-arm-arm_orangepi

Oh, you're right! Thanks for catching it!

I guess I still had a "export PYTHONPATH=$HOME/qemu/python" in my 
environment, so I did not notice. Would you like to send a patch, or want me 
to do it?

  Thomas

Re: [PATCH] docs/system: Fix the information on how to run certain functional tests

Posted by Niek Linnenbank 10 months, 3 weeks ago

Hi Thomas,

On Mon, Mar 17, 2025 at 7:50 AM Thomas Huth <thuth@redhat.com> wrote:

>   Hi!
>
> On 16/03/2025 21.21, Niek Linnenbank wrote:
> > Hi Thomas,
> >
> > On Tue, Mar 11, 2025 at 5:08 PM Thomas Huth <thuth@redhat.com
> > <mailto:thuth@redhat.com>> wrote:
> >
> >     The tests have been converted to the functional framework, so
> >     we should not talk about Avocado here anymore.
> ...
> >     diff --git a/docs/system/arm/orangepi.rst
> b/docs/system/arm/orangepi.rst
> >     index db87e81fec4..8b9448ca7b0 100644
> >     --- a/docs/system/arm/orangepi.rst
> >     +++ b/docs/system/arm/orangepi.rst
> >     @@ -257,9 +257,9 @@ Orange Pi PC integration tests
> >
> > Perhaps for consistency, we can also rename to 'functional tests' here.
>
> Agreed, we don't use the term "integration tests" for this anymore.
>
> >
> >       The Orange Pi PC machine has several integration tests included.
> >
> > And same on this line.
> >
> >       To run the whole set of tests, build QEMU from source and simply
> >     -provide the following command:
> >     +provide the following command from the build directory:
> >
> >       .. code-block:: bash
> >
> >     -  $ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
> >     -     -t machine:orangepi-pc tests/avocado/boot_linux_console.py
> >     +  $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 \
> >     +    meson test --suite thorough func-arm-arm_orangepi
> >
> > I've tried to run on my Ubuntu 24.04.1 LTS based system using this exact
> > same command, but got this error:
> >
> > $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 meson test --suite thorough func-arm-
> > arm_orangepi
> >
> > ERROR: Build data file '/home/user/qemu/build/meson-private/build.dat'
> > references functions or classes that don't exist. This probably means
> that
> > it was generated with an old version of meson. Consider reconfiguring
> the
> > directory with "meson setup --reconfigure".
> >
> > The meson version I have installed via apt-get is 1.3.2-1ubuntu1. Only
> when
> > running using the 'meson' command from the pyvenv, it runs OK:
> >
> > $ QEMU_TEST_ALLOW_LARGE_STORAGE=1 ./pyvenv/bin/meson test --suite
> thorough
> > func-arm-arm_orangepi
>
> Oh, you're right! Thanks for catching it!
>
> I guess I still had a "export PYTHONPATH=$HOME/qemu/python" in my
> environment, so I did not notice. Would you like to send a patch, or want
> me
> to do it?
>

Yes please feel free to go ahead with the patch. With the above minor
remarks resolved, it looks fine to me:

Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>

Regards,
Niek


>
>   Thomas
>
>

-- 
Niek Linnenbank