GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/836855908
           (All green. phew.)

(This is a quick V2 RFC to freshen the patches on the list and
consolidate some fixes made so far. See changes below.)

This patch series creates a mandatory venv during configure time and
uses it to install meson.

The eventual goal of all of this is to ensure that the Python used to
run meson is the same Python used to run Sphinx, tests, and any
build-time scripting we have. As it stands, meson and sphinx (and their
extensions) may run in a different Python environment than the one
configured and chosen by the user at configure/build time.

We can ensure that everything uses the same Python and has the same
packages available with a bit of up-front work. See
https://www.qemu.org/2023/03/24/python/ for more information.

Changes in V2:

- Each workaround in mkvenv is added in a separate patch now to better
  isolate parts of the design and what they were added to address.
- Introduction into configure is split into two parts, the first of
  which is quite a bit cleaner and introduces a lot less logic into the
  configure file.
- Avocado tests now use the configure-time venv.
- Python 3.7 now works unconditionally without new dependencies, though
  we might drop it in favor of 3.8 soon. "Just in case", I suppose.
- msys2 CI jobs now pass - there was a bug in the os.chmod call under
  windows.
- NetBSD now works; mkvenv.py has better diagnostics for detecting
  NetBSD edge cases.

Changes yet to come for further revisions to this series:

- Need to add Sphinx support.
- Need to add nested venv support.
- PyPI support? (To drop meson git submodule?)
- Installing meson from vendored .whl files for tarballs?
  (To drop --no-use-pep517 hack?)

John Snow (10):
  python: add mkvenv.py
  tests: add python3-venv dependency
  mkvenv: Add better error message for missing pyexapt module
  tests/vm: Configure netbsd to use Python 3.10
  tests/vm: add py310-expat to NetBSD
  mkvenv: generate console entry shims from inside the venv
  mkvenv: work around broken pip installations on Debian 10
  configure: create a python venv unconditionally
  configure: remove --meson=; install meson to the pyvenv
  tests: Use configure-provided pyvenv for tests

 docs/devel/acpi-bits.rst                      |   6 +-
 docs/devel/testing.rst                        |  14 +-
 configure                                     | 138 ++++--
 .gitlab-ci.d/buildtest-template.yml           |   4 +-
 .gitlab-ci.d/buildtest.yml                    |   6 +-
 python/scripts/mkvenv.py                      | 466 ++++++++++++++++++
 .../org.centos/stream/8/x86_64/test-avocado   |   4 +-
 scripts/device-crash-test                     |   2 +-
 tests/Makefile.include                        |  10 +-
 .../dockerfiles/debian-all-test-cross.docker  |   3 +-
 .../dockerfiles/debian-hexagon-cross.docker   |   3 +-
 .../dockerfiles/debian-riscv64-cross.docker   |   3 +-
 .../dockerfiles/debian-tricore-cross.docker   |   3 +-
 tests/requirements.txt                        |   7 +-
 tests/vm/netbsd                               |   2 +
 15 files changed, 590 insertions(+), 81 deletions(-)
 create mode 100644 python/scripts/mkvenv.py

-- 
2.39.2