[PATCH 00/23] Python: delete qemu.qmp package

John Snow posted 23 patches 1 week, 6 days ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20211124192617.3396403-1-jsnow@redhat.com
Maintainers: Cleber Rosa <crosa@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Eduardo Habkost <ehabkost@redhat.com>, Markus Armbruster <armbru@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>, John Snow <jsnow@redhat.com>
python/PACKAGE.rst                           |   4 +-
python/README.rst                            |   4 +-
python/qemu/qmp/README.rst                   |   9 -
python/qemu/aqmp/__init__.py                 |  10 +-
python/qemu/aqmp/aqmp_tui.py                 |   2 +-
python/qemu/aqmp/legacy.py                   | 203 ++++++++-
python/qemu/aqmp/protocol.py                 |  16 +-
python/qemu/aqmp/qmp_client.py               |   8 +-
python/qemu/{qmp => aqmp}/qmp_shell.py       |  31 +-
python/qemu/machine/machine.py               |  18 +-
python/qemu/machine/qtest.py                 |   2 +-
python/qemu/qmp/__init__.py                  | 422 -------------------
python/qemu/qmp/py.typed                     |   0
python/qemu/{qmp => utils}/qemu_ga_client.py |  24 +-
python/qemu/{qmp => utils}/qom.py            |   5 +-
python/qemu/{qmp => utils}/qom_common.py     |   7 +-
python/qemu/{qmp => utils}/qom_fuse.py       |  11 +-
python/setup.cfg                             |  21 +-
scripts/cpu-x86-uarch-abi.py                 |   7 +-
scripts/qmp/qemu-ga-client                   |   2 +-
scripts/qmp/qmp-shell                        |   2 +-
scripts/qmp/qom-fuse                         |   2 +-
scripts/qmp/qom-get                          |   2 +-
scripts/qmp/qom-list                         |   2 +-
scripts/qmp/qom-set                          |   2 +-
scripts/qmp/qom-tree                         |   2 +-
scripts/render_block_graph.py                |   8 +-
scripts/simplebench/bench_block_job.py       |   3 +-
tests/qemu-iotests/iotests.py                |   2 +-
tests/qemu-iotests/tests/mirror-top-perms    |   7 +-
30 files changed, 300 insertions(+), 538 deletions(-)
delete mode 100644 python/qemu/qmp/README.rst
rename python/qemu/{qmp => aqmp}/qmp_shell.py (96%)
delete mode 100644 python/qemu/qmp/__init__.py
delete mode 100644 python/qemu/qmp/py.typed
rename python/qemu/{qmp => utils}/qemu_ga_client.py (94%)
rename python/qemu/{qmp => utils}/qom.py (98%)
rename python/qemu/{qmp => utils}/qom_common.py (96%)
rename python/qemu/{qmp => utils}/qom_fuse.py (97%)

[PATCH 00/23] Python: delete qemu.qmp package

Posted by John Snow 1 week, 6 days ago
GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-qmp-legacy-switch
CI: https://gitlab.com/jsnow/qemu/-/pipelines/415801786

NOT intended for 6.2.

This series swaps out qemu.qmp for qemu.aqmp permanently, instead of
hiding it behind an environment variable toggle. This leaves us with
just one QMP library to worry about.

The series is organized like this:

- 01-02: Fixes and improvements to Async QMP
- 03-11: Switch python/* users to use AQMP exclusively
- 12-17: Switch other users to use AQMP exclusively
- 18-23: Finalize the switchover, delete python/qemu/qmp.

Optional notes about the broader process of moving Python infrastructure
onto PyPI are below, though it isn't required reading for reviewing this
series. Consider it a newsletter from the Python dungeon:

I was asked what the timeline for actually uploading anything to PyPI
was. This series is part of my answer, but the steps look like this:

Phase I:
- Refactor everything in-tree to be a bona-fide python package  [Done]
- Add unit testing and CI for all QEMU python packages          [Done]
- Develop a version of QMP intended for public support via PyPI [Done]

Phase II:
- Switch machine.py and iotests to using async QMP by default   [Done]
- Fix bugs in qemu.aqmp discovered during RC testing           [Ongoing]
- Remove qemu.qmp in favor of qemu.aqmp                    [This Series]
- Rename qemu.aqmp back to qemu.qmp
- Add a proper "sync" version of qemu.aqmp.QMPClient       [In Progress]
  designed to be more supportable via PyPI
  current status: it's functional, but there are some FIXMEs.
- Pivot in-tree users of qemu.(a)qmp.legacy to qemu.qmp.sync,
  -OR- move the "legacy" wrapper outside of the qmp package and into utils.
  (The goal is simply to avoid uploading the legacy wrapper to PyPI.)

Phase III:
- Fork python/qemu/qmp into its own git repo, with its own pkg version
- Add sphinx doc generation to qemu.qmp repo and add readthedocs integration
  [Doc generation is 95% done on a branch, needs polish. RTD is untouched.]
- Convert in-tree users of qemu.qmp to pull the dependency from either
  PyPI or a git URL. I think I'd like to avoid using git submodules ...

That's broadly it. There's some code to do for the sync bridge to make
the design tidier, but the goal there is to move a lot of the QMP event
wrangling functions we have scattered across qmp, machine, and even
iotests into a more central location with much stronger support.

A lot of this will hopefully move pretty fast once the tree re-opens.

One of the remaining skeletons in the closet that I have not yet fully
addressed is how I will be moving remaining in-tree users of the QMP
package onto a PyPI dependency. That's probably where most of the work
will actually be; adding a python virtual environment to iotests et al.

John Snow (23):
  python/aqmp: add __del__ method to legacy interface
  python/aqmp: handle asyncio.TimeoutError on execute()
  python/aqmp: copy type definitions from qmp
  python/aqmp: add SocketAddrT to package root
  python/qemu-ga-client: update instructions to newer CLI syntax
  python/qmp: switch qemu-ga-client to AQMP
  python/qmp: switch qom tools to AQMP
  python/qmp: switch qmp-shell to AQMP
  python: move qmp utilities to python/qemu/utils
  python: move qmp-shell under the AQMP package
  python/machine: permanently switch to AQMP
  scripts/cpu-x86-uarch-abi: fix CLI parsing
  scripts/cpu-x86-uarch-abi: switch to AQMP
  scripts/render-block-graph: switch to AQMP
  scripts/bench-block-job: switch to AQMP
  iotests/mirror-top-perms: switch to AQMP
  iotests: switch to AQMP
  python: temporarily silence pylint duplicate-code warnings
  python/aqmp: take QMPBadPortError and parse_address from qemu.qmp
  python/aqmp: fully separate from qmp.QEMUMonitorProtocol
  python/aqmp: copy qmp docstrings to qemu.aqmp.legacy
  python: remove the old QMP package
  python: re-enable pylint duplicate-code warnings

 python/PACKAGE.rst                           |   4 +-
 python/README.rst                            |   4 +-
 python/qemu/qmp/README.rst                   |   9 -
 python/qemu/aqmp/__init__.py                 |  10 +-
 python/qemu/aqmp/aqmp_tui.py                 |   2 +-
 python/qemu/aqmp/legacy.py                   | 203 ++++++++-
 python/qemu/aqmp/protocol.py                 |  16 +-
 python/qemu/aqmp/qmp_client.py               |   8 +-
 python/qemu/{qmp => aqmp}/qmp_shell.py       |  31 +-
 python/qemu/machine/machine.py               |  18 +-
 python/qemu/machine/qtest.py                 |   2 +-
 python/qemu/qmp/__init__.py                  | 422 -------------------
 python/qemu/qmp/py.typed                     |   0
 python/qemu/{qmp => utils}/qemu_ga_client.py |  24 +-
 python/qemu/{qmp => utils}/qom.py            |   5 +-
 python/qemu/{qmp => utils}/qom_common.py     |   7 +-
 python/qemu/{qmp => utils}/qom_fuse.py       |  11 +-
 python/setup.cfg                             |  21 +-
 scripts/cpu-x86-uarch-abi.py                 |   7 +-
 scripts/qmp/qemu-ga-client                   |   2 +-
 scripts/qmp/qmp-shell                        |   2 +-
 scripts/qmp/qom-fuse                         |   2 +-
 scripts/qmp/qom-get                          |   2 +-
 scripts/qmp/qom-list                         |   2 +-
 scripts/qmp/qom-set                          |   2 +-
 scripts/qmp/qom-tree                         |   2 +-
 scripts/render_block_graph.py                |   8 +-
 scripts/simplebench/bench_block_job.py       |   3 +-
 tests/qemu-iotests/iotests.py                |   2 +-
 tests/qemu-iotests/tests/mirror-top-perms    |   7 +-
 30 files changed, 300 insertions(+), 538 deletions(-)
 delete mode 100644 python/qemu/qmp/README.rst
 rename python/qemu/{qmp => aqmp}/qmp_shell.py (96%)
 delete mode 100644 python/qemu/qmp/__init__.py
 delete mode 100644 python/qemu/qmp/py.typed
 rename python/qemu/{qmp => utils}/qemu_ga_client.py (94%)
 rename python/qemu/{qmp => utils}/qom.py (98%)
 rename python/qemu/{qmp => utils}/qom_common.py (96%)
 rename python/qemu/{qmp => utils}/qom_fuse.py (97%)

-- 
2.31.1