[PATCH 0/8] Python: Fix 'make check-dev' and modernize to 3.9+

John Snow posted 8 patches 6 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20250519182153.3835722-1-jsnow@redhat.com
Maintainers: John Snow <jsnow@redhat.com>, Peter Maydell <peter.maydell@linaro.org>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, Markus Armbruster <armbru@redhat.com>, Michael Roth <michael.roth@amd.com>, Cleber Rosa <crosa@redhat.com>, Eduardo Habkost <eduardo@habkost.net>, Maksim Davydov <davydov-max@yandex-team.ru>, Manos Pitsidianakis <manos.pitsidianakis@linaro.org>, Thomas Huth <thuth@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, "Daniel P. Berrangé" <berrange@redhat.com>, Ani Sinha <anisinha@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>
There is a newer version of this series
docs/sphinx/compat.py                         |  11 +-
docs/sphinx/dbusdoc.py                        |  13 +-
docs/sphinx/dbusdomain.py                     |  32 ++--
docs/sphinx/fakedbusdoc.py                    |   4 +-
docs/sphinx/qapi_domain.py                    |  77 ++++----
docs/sphinx/qapidoc.py                        |  16 +-
python/README.rst                             |  33 ++--
python/Makefile                               |  18 +-
python/pyproject.toml                         |   8 +
python/qemu/machine/console_socket.py         |   4 +-
python/qemu/machine/machine.py                |  32 ++--
python/qemu/machine/qtest.py                  |  13 +-
python/qemu/qmp/error.py                      |   7 +-
python/qemu/qmp/events.py                     |  72 +++++--
python/qemu/qmp/legacy.py                     |  31 +--
python/qemu/qmp/message.py                    |  38 ++--
python/qemu/qmp/models.py                     |  26 ++-
python/qemu/qmp/protocol.py                   | 179 ++++++++++-------
python/qemu/qmp/qmp_client.py                 | 147 +++++++++-----
python/qemu/qmp/qmp_shell.py                  | 180 ++++++++++++------
python/qemu/qmp/qmp_tui.py                    |  55 +++---
python/qemu/qmp/util.py                       | 116 +----------
python/qemu/utils/accel.py                    |   4 +-
python/qemu/utils/qemu_ga_client.py           |  11 +-
python/qemu/utils/qom_common.py               |  17 +-
python/qemu/utils/qom_fuse.py                 |  12 +-
python/scripts/mkvenv.py                      |  38 ++--
python/setup.cfg                              |   7 +-
python/setup.py                               |  40 ----
python/tests/minreqs.txt                      |   4 +-
python/tests/protocol.py                      |  10 +-
scripts/block-coroutine-wrapper.py            |   2 +-
.../codeconverter/codeconverter/patching.py   |  50 ++---
.../codeconverter/codeconverter/qom_macros.py |  36 ++--
.../codeconverter/qom_type_info.py            |  16 +-
scripts/compare-machine-types.py              |  41 ++--
scripts/qapi/commands.py                      |  13 +-
scripts/qapi/common.py                        |  20 +-
scripts/qapi/events.py                        |   8 +-
scripts/qapi/expr.py                          |  28 ++-
scripts/qapi/features.py                      |   2 +-
scripts/qapi/gen.py                           |  13 +-
scripts/qapi/introspect.py                    |  44 ++---
scripts/qapi/parser.py                        |  42 ++--
scripts/qapi/schema.py                        | 117 ++++++------
scripts/qapi/source.py                        |  10 +-
scripts/qapi/types.py                         |  20 +-
scripts/qapi/visit.py                         |  14 +-
scripts/rust/rustc_args.py                    |   7 +-
scripts/u2f-setup-gen.py                      |   3 +-
tests/functional/qemu_test/ports.py           |   3 +-
tests/functional/test_acpi_bits.py            |   9 +-
tests/qemu-iotests/151                        |   4 +-
tests/qemu-iotests/297                        |   5 +-
tests/qemu-iotests/300                        |   4 +-
tests/qemu-iotests/fat16.py                   |  11 +-
tests/qemu-iotests/findtests.py               |  13 +-
tests/qemu-iotests/iotests.py                 |  38 ++--
tests/qemu-iotests/linters.py                 |   7 +-
tests/qemu-iotests/testenv.py                 |  11 +-
tests/qemu-iotests/testrunner.py              |  19 +-
.../qemu-iotests/tests/stream-under-throttle  |   3 +-
62 files changed, 925 insertions(+), 943 deletions(-)
create mode 100644 python/pyproject.toml
delete mode 100755 python/setup.py
[PATCH 0/8] Python: Fix 'make check-dev' and modernize to 3.9+
Posted by John Snow 6 months ago
This series does a couple things that I'll probably end up splitting out
into smaller series if history is any guide, but either way, here it
goes:

A) Convert qemu.git/python/ to a PEP517/pyproject.toml
package. Ultimately this means deleting setup.py and fully migrating to
newer python infrastructure. I think this should be safe to do by now,
but admittedly I am not *confident* as it relies on setuptools versions
in the wild, not python versions. My motivation for trying it is to fix
"make check-dev", which has been broken for the last two Fedora releases
under newer setuptools which have started removing support for the
pre-PEP517 packaging formats, which will only continue to get worse from
here on out.

B) Sync changes from the qemu.qmp package back over to qemu.git. I know
I need to decouple this badly, but in order to do so, I need to make
sure they're synchronized to be assured that the switch to the
standalone version won't break anything, so this is a necessary
step. It's happening here because of the 3.6+ compat crud we are still
carrying in qemu.git that has since been removed from the standalone
library.

C) Move us to 3.9+ style type hints. I know it feels like we just got
the type hints settled and we're already changing them. Sorry about
that. They are deprecated in 3.9, and *could* be removed at any time. I
figured now was a good time as any to get rid of them before they become
a problem randomly some day in the future.

D) Update the mypy configuration to check under multiple Python versions
more effectively and thoroughly.

Whew.
--js

John Snow (8):
  python: convert packages to PEP517/pyproject.toml
  python: update pylint ignores
  python: sync changes from external qemu.qmp package
  python: use 3.9+ builtin type hints
  python: convert remaining deprecated type hints for 3.9+
  python: clean up requirements for 3.9+
  python: update mkvenv to type-check under different python versions
  python: remove version restriction for mypy

 docs/sphinx/compat.py                         |  11 +-
 docs/sphinx/dbusdoc.py                        |  13 +-
 docs/sphinx/dbusdomain.py                     |  32 ++--
 docs/sphinx/fakedbusdoc.py                    |   4 +-
 docs/sphinx/qapi_domain.py                    |  77 ++++----
 docs/sphinx/qapidoc.py                        |  16 +-
 python/README.rst                             |  33 ++--
 python/Makefile                               |  18 +-
 python/pyproject.toml                         |   8 +
 python/qemu/machine/console_socket.py         |   4 +-
 python/qemu/machine/machine.py                |  32 ++--
 python/qemu/machine/qtest.py                  |  13 +-
 python/qemu/qmp/error.py                      |   7 +-
 python/qemu/qmp/events.py                     |  72 +++++--
 python/qemu/qmp/legacy.py                     |  31 +--
 python/qemu/qmp/message.py                    |  38 ++--
 python/qemu/qmp/models.py                     |  26 ++-
 python/qemu/qmp/protocol.py                   | 179 ++++++++++-------
 python/qemu/qmp/qmp_client.py                 | 147 +++++++++-----
 python/qemu/qmp/qmp_shell.py                  | 180 ++++++++++++------
 python/qemu/qmp/qmp_tui.py                    |  55 +++---
 python/qemu/qmp/util.py                       | 116 +----------
 python/qemu/utils/accel.py                    |   4 +-
 python/qemu/utils/qemu_ga_client.py           |  11 +-
 python/qemu/utils/qom_common.py               |  17 +-
 python/qemu/utils/qom_fuse.py                 |  12 +-
 python/scripts/mkvenv.py                      |  38 ++--
 python/setup.cfg                              |   7 +-
 python/setup.py                               |  40 ----
 python/tests/minreqs.txt                      |   4 +-
 python/tests/protocol.py                      |  10 +-
 scripts/block-coroutine-wrapper.py            |   2 +-
 .../codeconverter/codeconverter/patching.py   |  50 ++---
 .../codeconverter/codeconverter/qom_macros.py |  36 ++--
 .../codeconverter/qom_type_info.py            |  16 +-
 scripts/compare-machine-types.py              |  41 ++--
 scripts/qapi/commands.py                      |  13 +-
 scripts/qapi/common.py                        |  20 +-
 scripts/qapi/events.py                        |   8 +-
 scripts/qapi/expr.py                          |  28 ++-
 scripts/qapi/features.py                      |   2 +-
 scripts/qapi/gen.py                           |  13 +-
 scripts/qapi/introspect.py                    |  44 ++---
 scripts/qapi/parser.py                        |  42 ++--
 scripts/qapi/schema.py                        | 117 ++++++------
 scripts/qapi/source.py                        |  10 +-
 scripts/qapi/types.py                         |  20 +-
 scripts/qapi/visit.py                         |  14 +-
 scripts/rust/rustc_args.py                    |   7 +-
 scripts/u2f-setup-gen.py                      |   3 +-
 tests/functional/qemu_test/ports.py           |   3 +-
 tests/functional/test_acpi_bits.py            |   9 +-
 tests/qemu-iotests/151                        |   4 +-
 tests/qemu-iotests/297                        |   5 +-
 tests/qemu-iotests/300                        |   4 +-
 tests/qemu-iotests/fat16.py                   |  11 +-
 tests/qemu-iotests/findtests.py               |  13 +-
 tests/qemu-iotests/iotests.py                 |  38 ++--
 tests/qemu-iotests/linters.py                 |   7 +-
 tests/qemu-iotests/testenv.py                 |  11 +-
 tests/qemu-iotests/testrunner.py              |  19 +-
 .../qemu-iotests/tests/stream-under-throttle  |   3 +-
 62 files changed, 925 insertions(+), 943 deletions(-)
 create mode 100644 python/pyproject.toml
 delete mode 100755 python/setup.py

-- 
2.48.1

Re: [PATCH 0/8] Python: Fix 'make check-dev' and modernize to 3.9+
Posted by Markus Armbruster 5 months, 4 weeks ago
John Snow <jsnow@redhat.com> writes:

> This series does a couple things that I'll probably end up splitting out
> into smaller series if history is any guide, but either way, here it
> goes:
>
> A) Convert qemu.git/python/ to a PEP517/pyproject.toml
> package. Ultimately this means deleting setup.py and fully migrating to
> newer python infrastructure. I think this should be safe to do by now,
> but admittedly I am not *confident* as it relies on setuptools versions
> in the wild, not python versions. My motivation for trying it is to fix
> "make check-dev", which has been broken for the last two Fedora releases
> under newer setuptools which have started removing support for the
> pre-PEP517 packaging formats, which will only continue to get worse from
> here on out.
>
> B) Sync changes from the qemu.qmp package back over to qemu.git. I know
> I need to decouple this badly, but in order to do so, I need to make
> sure they're synchronized to be assured that the switch to the
> standalone version won't break anything, so this is a necessary
> step. It's happening here because of the 3.6+ compat crud we are still
> carrying in qemu.git that has since been removed from the standalone
> library.
>
> C) Move us to 3.9+ style type hints. I know it feels like we just got
> the type hints settled and we're already changing them. Sorry about
> that. They are deprecated in 3.9, and *could* be removed at any time. I
> figured now was a good time as any to get rid of them before they become
> a problem randomly some day in the future.
>
> D) Update the mypy configuration to check under multiple Python versions
> more effectively and thoroughly.
>
> Whew.
> --js

Appears to be

Based-on: <20250519170055.3693275-1-jsnow@redhat.com>