The current riscv64 smoke test (qemu-smoke-riscv64.sh) boots Xen under QEMU
and waits for "All set up" with expect. That is enough for one string on
one console, but RISC-V dom0less, in flight on xen-devel, would appreciate
having more: a matrix of pCPU counts, schedulers and domU layouts, and
checks on several consoles (Xen's and each domU's).

This series adds a small pytest framework built on AMD's QTB (QEMU Test
Bench) Python package [1]. It was written as part of the Xen safety
initiative and drives a QEMU instance over qtest, QMP and GDB, so a test
has full access to the machine while the emulation runs: read and write the
consoles, inspect registers and memory, inject interrupts and faults. QTB
is not upstream in QEMU yet, but is planned to be. Using QTB for the
riscv64 tests was suggested by Stefano Stabellini.

The pytest framework the series adds is a small wrapper on top of the QTB
framework that allows us to test the output of many different Xen
configurations with a simple pytest function. The aim is first to add more
coverage, and also to have a modular and easy way to add test logic. For
example, we plan to use it to add IRQ tests, checking that an interrupt
reaches the expected guest based on its SMP affinity.

Tests are plain pytest functions and the machine they boot is a
fixture. The per-test console logs and a JUnit report are kept as CI
artifacts, so GitLab shows per-test results. The README added in patch 4
is the best place to start reading.

Coverage is unchanged: the only test boots Xen alone on one pCPU and
waits for "All set up". The new job runs next to the expect-based one,
which patch 6 drops separately, so that patch can wait until the QTB job
has proven itself. The domU console tests are ready and will follow as
a separate series once RISC-V dom0less is merged, as they cannot pass
on staging before that.

This series:
    1: CI: add a Debian 13 riscv64 QTB container
    2-3: automation/qtb: add the QTB machine for riscv64, and the
         console-test type with a Xen boot test
    4: automation/qtb: add QTB framework README
    5: CI: add a riscv64 smoke test job using the QTB console-test
    6: CI: drop the riscv64 expect-based smoke test

Tested in GitLab CI, with the container's Debian qemu-system-riscv64 and
the OpenSBI it bundles.

Test CI pipeline:
https://gitlab.com/xen-project/people/baptleduc/xen/-/pipelines/TODO

[1] https://gitlab.com/xen-project/people/amd/qemu/-/tree/24cd7e5c0d7e4db8e651f091d0b9da2e7c58dbab/python

---
Changes in v3:
- Rework the framework by using pytest: tests are pytest functions
  parametrizing a `machine` fixture, run all at once with a JUnit
  report. The CLI, config.yaml, the per-type YAML files, the test class
  hierarchy and the attempts/retry logic are gone, so the v2 remarks on
  them no longer apply.
- Split into patches under 200 lines each.
- Move the container to xen.git (automation/build/), named
  debian:13-riscv64-qtb, with offline wheels from the fork, pytest, and
  a comment that the pinned commit is from the safety branch.
- Binaries and logs directories are given with --binaries-dir and
  --log-dir, no environment variable.
- Mirror Xen's console to stdout, so pytest shows it in the report of a
  failing test.
- Take the per-string timeout from TEST_TIMEOUT_OVERRIDE/TEST_TIMEOUT,
  like console.exp.
- Map pexpect timeouts and EOF to AssertionError (a pytest failure)
  instead of a dedicated TestFailure exception.
- Keep the host at 2GB: the follow-up domU tests boot two 512MB domUs.
- Drop the unit tests patch: nothing in CI runs them.
- Drop the Jinja2 host device tree. The follow-up domU series generates
  it from QEMU's own (-machine dumpdtb=) with fdtput, as the Arm tests
  do, and the Xen-only test here needs none.
- Keep the expect-based test, drop it in its own patch.
- Rename the CI template and job with "qtb".
- Link to v2: https://patch.msgid.link/1787823334.8631fc262581453bbf619ec5b2062170.1a04293255c000c4f3@vates.tech

To: xen-devel@lists.xenproject.org
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony PERARD <anthony.perard@vates.tech>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Alistair Francis <alistair.francis@wdc.com>
Cc: Connor Davis <connojdavis@gmail.com>
Cc: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Cc: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
Cc: Zhang Zheng <zhangzheng@iscas.ac.cn>

---
Baptiste Le Duc (6):
      CI: add a Debian 13 riscv64 QTB container
      automation/qtb: add the QTB machine for riscv64
      automation/qtb: add the console-test type with a Xen boot test
      automation/qtb: add QTB framework README
      CI: add a riscv64 smoke test job using the QTB console-test
      CI: drop the riscv64 expect-based smoke test

 automation/build/debian/13-riscv64-qtb.dockerfile  |  47 ++++++++
 automation/gitlab-ci/test.yaml                     |  25 ++--
 automation/scripts/containerize                    |   1 +
 automation/scripts/qemu-smoke-riscv64.sh           |  19 ---
 automation/scripts/qtb/__init__.py                 |   2 +
 automation/scripts/qtb/env.py                      |  15 +++
 automation/scripts/qtb/riscv/README.md             | 127 +++++++++++++++++++++
 automation/scripts/qtb/riscv/__init__.py           |   2 +
 automation/scripts/qtb/riscv/machine.py            |  92 +++++++++++++++
 automation/scripts/qtb/riscv/tests/__init__.py     |   2 +
 automation/scripts/qtb/riscv/tests/conftest.py     |  65 +++++++++++
 automation/scripts/qtb/riscv/tests/test_console.py |  35 ++++++
 automation/scripts/qtb/utils.py                    |  15 +++
 13 files changed, 419 insertions(+), 28 deletions(-)
---
base-commit: d8e0282815561ff67ebe6b5dfd582a586dafff48
change-id: 20261103-qtb-framework-pytest-f3ed28e4bd78

Best regards,
--  
Baptiste Le Duc <baptiste.le-duc@vates.tech>