[PULL 00/19] Python patches for 5.1

Philippe Mathieu-Daudé posted 19 patches 3 years, 9 months ago
Test checkpatch passed
Test docker-mingw@fedora passed
Test FreeBSD passed
Test docker-quick@centos7 passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20200714222132.10815-1-philmd@redhat.com
python/qemu/machine.py                   | 176 +++++++++++++++++------
python/qemu/qmp.py                       |  67 +++++++--
scripts/performance/dissect.py           | 166 +++++++++++++++++++++
scripts/render_block_graph.py            |   7 +-
tests/acceptance/boot_linux_console.py   |  14 +-
tests/acceptance/linux_ssh_mips_malta.py |   2 +
tests/qemu-iotests/iotests.py            |   9 +-
7 files changed, 369 insertions(+), 72 deletions(-)
create mode 100755 scripts/performance/dissect.py
[PULL 00/19] Python patches for 5.1
Posted by Philippe Mathieu-Daudé 3 years, 9 months ago
The following changes since commit 1a53dfee92284d3016a579ef31d53367e84d9dd8:

  Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-07-13' into staging (2020-07-14 13:52:10 +0100)

are available in the Git repository at:

  https://gitlab.com/philmd/qemu.git tags/python-next-20200714

for you to fetch changes up to 84dcdf0887cdaaba7300442482c99e5064865a2d:

  python/qmp.py: add QMPProtocolError (2020-07-14 22:22:22 +0200)

----------------------------------------------------------------
Python patches for 5.1

- Reduce race conditions on QEMUMachine::shutdown()

 1. Remove the "bare except" pattern in the existing shutdown code,
    which can mask problems and make debugging difficult.
 2. Ensure that post-shutdown cleanup is always performed, even when
    graceful termination fails.
 3. Unify cleanup paths such that no matter how the VM is terminated,
    the same functions and steps are always taken to reset the object
    state.
 4. Rewrite shutdown() such that any error encountered when attempting
    a graceful shutdown will be raised as an AbnormalShutdown exception.
    The pythonic idiom is to allow the caller to decide if this is a
    problem or not.

- Modify part of the python/qemu library to comply with:

  . mypy --strict
  . pylint
  . flake8

- Script for the TCG Continuous Benchmarking project that uses
  callgrind to dissect QEMU execution into three main phases:

  . code generation
  . JIT execution
  . helpers execution

CI jobs results:
. https://cirrus-ci.com/build/5421349961203712
. https://gitlab.com/philmd/qemu/-/pipelines/166556001
. https://travis-ci.org/github/philmd/qemu/builds/708102347
----------------------------------------------------------------

Ahmed Karaman (1):
  scripts/performance: Add dissect.py script

John Snow (18):
  python/machine.py: consolidate _post_shutdown()
  python/machine.py: Close QMP socket in cleanup
  python/machine.py: Add _early_cleanup hook
  python/machine.py: Perform early cleanup for wait() calls, too
  python/machine.py: Prohibit multiple shutdown() calls
  python/machine.py: Add a configurable timeout to shutdown()
  python/machine.py: Make wait() call shutdown()
  tests/acceptance: wait() instead of shutdown() where appropriate
  tests/acceptance: Don't test reboot on cubieboard
  python/machine.py: split shutdown into hard and soft flavors
  python/machine.py: re-add sigkill warning suppression
  python/machine.py: change default wait timeout to 3 seconds
  python/qmp.py: Define common types
  iotests.py: use qemu.qmp type aliases
  python/qmp.py: re-absorb MonitorResponseError
  python/qmp.py: Do not return None from cmd_obj
  python/qmp.py: add casts to JSON deserialization
  python/qmp.py: add QMPProtocolError

 python/qemu/machine.py                   | 176 +++++++++++++++++------
 python/qemu/qmp.py                       |  67 +++++++--
 scripts/performance/dissect.py           | 166 +++++++++++++++++++++
 scripts/render_block_graph.py            |   7 +-
 tests/acceptance/boot_linux_console.py   |  14 +-
 tests/acceptance/linux_ssh_mips_malta.py |   2 +
 tests/qemu-iotests/iotests.py            |   9 +-
 7 files changed, 369 insertions(+), 72 deletions(-)
 create mode 100755 scripts/performance/dissect.py

-- 
2.21.3


Re: [PULL 00/19] Python patches for 5.1
Posted by Peter Maydell 3 years, 9 months ago
On Tue, 14 Jul 2020 at 23:23, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> The following changes since commit 1a53dfee92284d3016a579ef31d53367e84d9dd8:
>
>   Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-07-13' into staging (2020-07-14 13:52:10 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/philmd/qemu.git tags/python-next-20200714
>
> for you to fetch changes up to 84dcdf0887cdaaba7300442482c99e5064865a2d:
>
>   python/qmp.py: add QMPProtocolError (2020-07-14 22:22:22 +0200)
>
> ----------------------------------------------------------------
> Python patches for 5.1
>
> - Reduce race conditions on QEMUMachine::shutdown()
>
>  1. Remove the "bare except" pattern in the existing shutdown code,
>     which can mask problems and make debugging difficult.
>  2. Ensure that post-shutdown cleanup is always performed, even when
>     graceful termination fails.
>  3. Unify cleanup paths such that no matter how the VM is terminated,
>     the same functions and steps are always taken to reset the object
>     state.
>  4. Rewrite shutdown() such that any error encountered when attempting
>     a graceful shutdown will be raised as an AbnormalShutdown exception.
>     The pythonic idiom is to allow the caller to decide if this is a
>     problem or not.
>
> - Modify part of the python/qemu library to comply with:
>
>   . mypy --strict
>   . pylint
>   . flake8
>
> - Script for the TCG Continuous Benchmarking project that uses
>   callgrind to dissect QEMU execution into three main phases:
>
>   . code generation
>   . JIT execution
>   . helpers execution
>
> CI jobs results:
> . https://cirrus-ci.com/build/5421349961203712
> . https://gitlab.com/philmd/qemu/-/pipelines/166556001
> . https://travis-ci.org/github/philmd/qemu/builds/708102347


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM