[Qemu-devel] [PATCH v5 00/20] monitor: add asynchronous command type

Marc-André Lureau posted 20 patches 4 years, 8 months ago
Test FreeBSD failed
Test asan passed
Test docker-clang@ubuntu passed
Test checkpatch failed
Test s390x passed
Test docker-mingw@fedora passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190715191001.1188-1-marcandre.lureau@redhat.com
Maintainers: Eric Blake <eblake@redhat.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Markus Armbruster <armbru@redhat.com>, Gerd Hoffmann <kraxel@redhat.com>, Michael Roth <mdroth@linux.vnet.ibm.com>
There is a newer version of this series
hmp-commands.hx                         |   3 +-
hw/display/qxl-render.c                 |   9 +-
hw/display/qxl.c                        |   1 +
include/monitor/monitor.h               |   3 +
include/qapi/qmp/dispatch.h             |  89 +++++++++-
include/qapi/qmp/json-parser.h          |   7 +-
include/ui/console.h                    |   4 +
monitor/hmp-cmds.c                      |   6 +-
monitor/hmp.c                           | 110 +++++++++++-
monitor/misc.c                          |  46 +----
monitor/monitor-internal.h              |  12 +-
monitor/monitor.c                       |   2 +-
monitor/qmp.c                           |  79 ++++-----
qapi/misc.json                          |   3 +-
qapi/qmp-dispatch.c                     | 214 +++++++++++++++++++-----
qapi/qmp-registry.c                     |  33 +++-
qapi/ui.json                            |   3 +-
qga/commands.c                          |   2 +-
qga/main.c                              |  51 ++----
qobject/json-lexer.c                    |   5 +-
qobject/json-streamer.c                 |   3 +-
scripts/qapi/commands.py                | 151 ++++++++++++++---
scripts/qapi/common.py                  |  15 +-
scripts/qapi/doc.py                     |   3 +-
scripts/qapi/introspect.py              |   3 +-
tests/qapi-schema/qapi-schema-test.json |   5 +
tests/qapi-schema/qapi-schema-test.out  |   8 +
tests/qapi-schema/test-qapi.py          |   8 +-
tests/test-qmp-cmds.c                   | 206 +++++++++++++++++++----
ui/console.c                            | 100 +++++++++--
30 files changed, 908 insertions(+), 276 deletions(-)
[Qemu-devel] [PATCH v5 00/20] monitor: add asynchronous command type
Posted by Marc-André Lureau 4 years, 8 months ago
Hi,

HMP and QMP commands are handled synchronously in qemu today. But
there are benefits allowing the command handler to re-enter the main
loop if the command cannot be handled synchronously, or if it is
long-lasting. Some bugs such as rhbz#1230527 are difficult to solve
without it.

The common solution is to use a pair of command+event in this case.
But this approach has a number of issues:
- you can't "fix" an existing command: you need a new API, and ad-hoc
  documentation for that command+signal association, and old/broken
  command deprecation
- since the reply event is broadcasted and 'id' is used for matching the
  request, it may conflict with other clients request 'id' space
- it is arguably less efficient and elegant (weird API, useless return
  in most cases, broadcast reply, no cancelling on disconnect etc)

The following series implements an async command solution instead. By
introducing a session context and a command return handler, it can:
- defer the return, allowing the mainloop to reenter
- return only to the caller (instead of broadcast events for reply)
- optionnally allow cancellation when the client is gone
- track on-going qapi command(s) per client/session

and without introduction of new QMP APIs or client visible change.

Existing qemu commands can be gradually replaced by async:true
variants when needed, while carefully reviewing the concurrency
aspects. The async:true commands marshaller helpers are splitted in
half, the calling and return functions. The command is called with a
QmpReturn context, that can return immediately or later, using the
generated return helper, which allows for a step-by-step conversion.

The screendump command is converted to an async:true version to solve
rhbz#1230527. The command shows basic cancellation (this could be
extended if needed). It could be further improved to do asynchronous
IO writes as well.

v5:
- rebased

v4:
- rebased, mostly adapting to new OOB code
  (there was not much feedback in v3 for the async command part,
   but preliminary patches got merged!)
- drop the RFC status

v3:
- complete rework, dropping the asynchronous commands visibility from
  the protocol side entirely (until there is a real need for it)
- rebased, with a few preliminary cleanup patches
- teach asynchronous commands to HMP

v2:
- documentation fixes and improvements
- fix calling async commands sync without id
- fix bad hmp monitor assert
- add a few extra asserts
- add async with no-id failure and screendump test

Marc-André Lureau (20):
  qmp: constify QmpCommand and list
  json-lexer: make it safe to call destroy multiple times
  qmp: add QmpSession
  QmpSession: add a return callback
  QmpSession: add json parser and use it in qga
  monitor: use qmp session to parse json feed
  qga: simplify dispatch_return_cb
  QmpSession: introduce QmpReturn
  qmp: simplify qmp_return_error()
  QmpSession: keep a queue of pending commands
  QmpSession: return orderly
  qmp: introduce asynchronous command type
  scripts: learn 'async' qapi commands
  qmp: add qmp_return_is_cancelled()
  monitor: add qmp_return_get_monitor()
  console: add graphic_hw_update_done()
  console: make screendump asynchronous
  monitor: start making qmp_human_monitor_command() asynchronous
  monitor: teach HMP about asynchronous commands
  hmp: call the asynchronous QMP screendump to fix outdated/glitches

 hmp-commands.hx                         |   3 +-
 hw/display/qxl-render.c                 |   9 +-
 hw/display/qxl.c                        |   1 +
 include/monitor/monitor.h               |   3 +
 include/qapi/qmp/dispatch.h             |  89 +++++++++-
 include/qapi/qmp/json-parser.h          |   7 +-
 include/ui/console.h                    |   4 +
 monitor/hmp-cmds.c                      |   6 +-
 monitor/hmp.c                           | 110 +++++++++++-
 monitor/misc.c                          |  46 +----
 monitor/monitor-internal.h              |  12 +-
 monitor/monitor.c                       |   2 +-
 monitor/qmp.c                           |  79 ++++-----
 qapi/misc.json                          |   3 +-
 qapi/qmp-dispatch.c                     | 214 +++++++++++++++++++-----
 qapi/qmp-registry.c                     |  33 +++-
 qapi/ui.json                            |   3 +-
 qga/commands.c                          |   2 +-
 qga/main.c                              |  51 ++----
 qobject/json-lexer.c                    |   5 +-
 qobject/json-streamer.c                 |   3 +-
 scripts/qapi/commands.py                | 151 ++++++++++++++---
 scripts/qapi/common.py                  |  15 +-
 scripts/qapi/doc.py                     |   3 +-
 scripts/qapi/introspect.py              |   3 +-
 tests/qapi-schema/qapi-schema-test.json |   5 +
 tests/qapi-schema/qapi-schema-test.out  |   8 +
 tests/qapi-schema/test-qapi.py          |   8 +-
 tests/test-qmp-cmds.c                   | 206 +++++++++++++++++++----
 ui/console.c                            | 100 +++++++++--
 30 files changed, 908 insertions(+), 276 deletions(-)

-- 
2.22.0.428.g6d5b264208