[PATCH v3 0/5] monitor: add dynamic QMP monitor hotplug support

Christian Brauner posted 5 patches 4 days, 9 hours ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20260407-work-qmp-monitor-hotplug-v3-0-cb259800fffb@kernel.org
Maintainers: Markus Armbruster <armbru@redhat.com>, "Dr. David Alan Gilbert" <dave@treblig.org>, Eric Blake <eblake@redhat.com>, Thomas Huth <th.huth+qemu@posteo.eu>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, "Daniel P. Berrangé" <berrange@redhat.com>, Fabiano Rosas <farosas@suse.de>, Laurent Vivier <lvivier@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>
There is a newer version of this series
include/monitor/monitor.h                        |   3 +-
monitor/monitor-internal.h                       |  11 ++
monitor/monitor.c                                |  70 +++++++--
monitor/qmp-cmds-control.c                       | 111 ++++++++++++++
monitor/qmp.c                                    |  77 +++++++++-
qapi/control.json                                | 104 +++++++++++++
tests/functional/generic/meson.build             |   1 +
tests/functional/generic/test_monitor_hotplug.py | 170 +++++++++++++++++++++
tests/qtest/qmp-test.c                           | 186 +++++++++++++++++++++++
9 files changed, 717 insertions(+), 16 deletions(-)
[PATCH v3 0/5] monitor: add dynamic QMP monitor hotplug support
Posted by Christian Brauner 4 days, 9 hours ago
QEMU supports runtime hotplug for chardevs, devices, block backends,
and netdevs. Monitors are the only major subsystem that lacks this --
all QMP monitors must be configured at launch via -mon or -qmp CLI
options.

This series adds monitor-add, monitor-remove, and query-monitors QMP
commands so that management tools can create independent QMP sessions
on demand at runtime.

I've implemented a QMP-to-Varlink bridge in systemd. This allows
systemd-vmspawn to control virtual machines and containers through a
unified Varlink interface. Varlink allows protocol upgrades. For
example, it is possible to switch from Varlink to say http. I'm
allowing Varlink clients to upgrade from the Varlink interface to the
native QMP interface. Such clients get a new monitor assigned that
allows them to manage the virtual machine directly via QMP. The main
monitor remains unaffected and tied to the generic Varlink interface. We
can't pre-allocate monitors as we have no control over how many protocol
upgrades we actually get but it won't just be one. And having unused
monitors around really isn't ideal either.

Having the ability to hotplug monitors would really be helpful. I'm not
yet super well-versed in qemu internals so this might be done wrong but
testing works so far.

My systemd patch that triggered this is at
https://github.com/systemd/systemd/pull/41449.

The usage pattern mirrors chardev hotplug:

  -> chardev-add id=qmp-extra backend=socket,...
  -> monitor-add id=extra-qmp chardev=qmp-extra
  [client connects to socket, gets QMP greeting, negotiates, sends commands]
  -> monitor-remove id=extra-qmp
  -> chardev-remove id=qmp-extra

Patches 1-2 add the data model (id/dynamic/dead fields in Monitor) and
the infrastructure for safe per-monitor destruction without shutting
down the shared dispatcher coroutine.

Patch 3 adds the QAPI schema and command handlers.

Patches 4-5 add qtest unit tests and a functional e2e test that
performs a full hotplug -> connect -> handshake -> unplug cycle.

Signed-off-by: Christian Brauner <brauner@kernel.org>
---
Changes in v3:
- Use SPDX license identifier in functional test.
- Use framework's socket_dir() helper instead of manual tempfile
  handling for socket paths in the functional test, drop tearDown().
- Tighten struct field comments in monitor-internal.h.
- Wrap long qtest_add_func() registration line.
- Link to v2: https://patch.msgid.link/20260405-work-qmp-monitor-hotplug-v2-0-ad5bedd2917a@kernel.org

Changes in v2:
- Fix use-after-free in self-removal path: skip monitor_resume() when
  the monitor is dead to avoid scheduling a BH against a monitor that
  is about to be freed by monitor_qmp_destroy().
- Hold monitor_lock in monitor_find_by_id() to prevent races with
  the I/O thread BH that appends to mon_list.
- Deduplicate monitor-remove commit message: trim the gcontext/out_watch
  explanation that repeated the infrastructure commit, reference
  monitor_cancel_out_watch() instead.
- Add missing test descriptions to patch 4 (chardev-in-use, CLI monitor
  rejection, HMP monitor rejection) and patch 5 (self-removal, large
  response, events).
- Fix cover letter wording.
- Link to v1: https://patch.msgid.link/20260402-work-qmp-monitor-hotplug-v1-0-6313a5cdd574@kernel.org

---
Christian Brauner (5):
      monitor: store monitor id and dynamic flag in Monitor struct
      monitor/qmp: add infrastructure for safe dynamic monitor removal
      qapi: add monitor-add, monitor-remove, query-monitors commands
      tests/qtest: add tests for dynamic monitor add/remove
      tests/functional: add e2e test for dynamic QMP monitor hotplug

 include/monitor/monitor.h                        |   3 +-
 monitor/monitor-internal.h                       |  11 ++
 monitor/monitor.c                                |  70 +++++++--
 monitor/qmp-cmds-control.c                       | 111 ++++++++++++++
 monitor/qmp.c                                    |  77 +++++++++-
 qapi/control.json                                | 104 +++++++++++++
 tests/functional/generic/meson.build             |   1 +
 tests/functional/generic/test_monitor_hotplug.py | 170 +++++++++++++++++++++
 tests/qtest/qmp-test.c                           | 186 +++++++++++++++++++++++
 9 files changed, 717 insertions(+), 16 deletions(-)
---
base-commit: b6a7d06213e5d2f7d124d16418bc289c4a8a4b82
change-id: 20260402-work-qmp-monitor-hotplug-fba7c618e3db
Re: [PATCH v3 0/5] monitor: add dynamic QMP monitor hotplug support
Posted by Daniel P. Berrangé 4 days, 6 hours ago
On Tue, Apr 07, 2026 at 09:32:44AM +0200, Christian Brauner wrote:
> QEMU supports runtime hotplug for chardevs, devices, block backends,
> and netdevs. Monitors are the only major subsystem that lacks this --
> all QMP monitors must be configured at launch via -mon or -qmp CLI
> options.
> 
> This series adds monitor-add, monitor-remove, and query-monitors QMP
> commands so that management tools can create independent QMP sessions
> on demand at runtime.
> 
> I've implemented a QMP-to-Varlink bridge in systemd. This allows
> systemd-vmspawn to control virtual machines and containers through a
> unified Varlink interface. Varlink allows protocol upgrades. For
> example, it is possible to switch from Varlink to say http. I'm
> allowing Varlink clients to upgrade from the Varlink interface to the
> native QMP interface. Such clients get a new monitor assigned that
> allows them to manage the virtual machine directly via QMP. The main
> monitor remains unaffected and tied to the generic Varlink interface. We
> can't pre-allocate monitors as we have no control over how many protocol
> upgrades we actually get but it won't just be one. And having unused
> monitors around really isn't ideal either.
> 
> Having the ability to hotplug monitors would really be helpful. I'm not
> yet super well-versed in qemu internals so this might be done wrong but
> testing works so far.

Can you say if AI agents/LLMs were used in creating these patches ? 

> 
> My systemd patch that triggered this is at
> https://github.com/systemd/systemd/pull/41449.
> 
> The usage pattern mirrors chardev hotplug:
> 
>   -> chardev-add id=qmp-extra backend=socket,...
>   -> monitor-add id=extra-qmp chardev=qmp-extra
>   [client connects to socket, gets QMP greeting, negotiates, sends commands]
>   -> monitor-remove id=extra-qmp
>   -> chardev-remove id=qmp-extra

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
Re: [PATCH v3 0/5] monitor: add dynamic QMP monitor hotplug support
Posted by Christian Brauner 3 days, 20 hours ago
On Tue, Apr 07, 2026 at 12:00:38PM +0100, Daniel P. Berrangé wrote:
> On Tue, Apr 07, 2026 at 09:32:44AM +0200, Christian Brauner wrote:
> > QEMU supports runtime hotplug for chardevs, devices, block backends,
> > and netdevs. Monitors are the only major subsystem that lacks this --
> > all QMP monitors must be configured at launch via -mon or -qmp CLI
> > options.
> > 
> > This series adds monitor-add, monitor-remove, and query-monitors QMP
> > commands so that management tools can create independent QMP sessions
> > on demand at runtime.
> > 
> > I've implemented a QMP-to-Varlink bridge in systemd. This allows
> > systemd-vmspawn to control virtual machines and containers through a
> > unified Varlink interface. Varlink allows protocol upgrades. For
> > example, it is possible to switch from Varlink to say http. I'm
> > allowing Varlink clients to upgrade from the Varlink interface to the
> > native QMP interface. Such clients get a new monitor assigned that
> > allows them to manage the virtual machine directly via QMP. The main
> > monitor remains unaffected and tied to the generic Varlink interface. We
> > can't pre-allocate monitors as we have no control over how many protocol
> > upgrades we actually get but it won't just be one. And having unused
> > monitors around really isn't ideal either.
> > 
> > Having the ability to hotplug monitors would really be helpful. I'm not
> > yet super well-versed in qemu internals so this might be done wrong but
> > testing works so far.
> 
> Can you say if AI agents/LLMs were used in creating these patches ? 

In exploring the QEMU codebase and how the QMP protocol is used in other
code-bases. So specifically I used it to research:

* Does monitor hotplug already work?
* Has the idea of monitor hotplug been discussed before?
* Is there an obvious fundamental reason this cannot work?
* What does the lifetime of a monitor look like?
* I used it to review the code I wrote and point out bugs that I then
  fixed myself.
* I had various research questions around how io-thread interactions
  work and common synchronization primitives in the codebase.

The code hasn't been written by any coding tool in line with what is
requested in the contribution guidelines. I used it to speed up my
understanding of what's going on and to debug things without using the
output.

Re: [PATCH v3 0/5] monitor: add dynamic QMP monitor hotplug support
Posted by Daniel P. Berrangé 3 days, 7 hours ago
On Tue, Apr 07, 2026 at 10:57:26PM +0200, Christian Brauner wrote:
> On Tue, Apr 07, 2026 at 12:00:38PM +0100, Daniel P. Berrangé wrote:
> > On Tue, Apr 07, 2026 at 09:32:44AM +0200, Christian Brauner wrote:
> > > QEMU supports runtime hotplug for chardevs, devices, block backends,
> > > and netdevs. Monitors are the only major subsystem that lacks this --
> > > all QMP monitors must be configured at launch via -mon or -qmp CLI
> > > options.
> > > 
> > > This series adds monitor-add, monitor-remove, and query-monitors QMP
> > > commands so that management tools can create independent QMP sessions
> > > on demand at runtime.
> > > 
> > > I've implemented a QMP-to-Varlink bridge in systemd. This allows
> > > systemd-vmspawn to control virtual machines and containers through a
> > > unified Varlink interface. Varlink allows protocol upgrades. For
> > > example, it is possible to switch from Varlink to say http. I'm
> > > allowing Varlink clients to upgrade from the Varlink interface to the
> > > native QMP interface. Such clients get a new monitor assigned that
> > > allows them to manage the virtual machine directly via QMP. The main
> > > monitor remains unaffected and tied to the generic Varlink interface. We
> > > can't pre-allocate monitors as we have no control over how many protocol
> > > upgrades we actually get but it won't just be one. And having unused
> > > monitors around really isn't ideal either.
> > > 
> > > Having the ability to hotplug monitors would really be helpful. I'm not
> > > yet super well-versed in qemu internals so this might be done wrong but
> > > testing works so far.
> > 
> > Can you say if AI agents/LLMs were used in creating these patches ? 
> 
> In exploring the QEMU codebase and how the QMP protocol is used in other
> code-bases. So specifically I used it to research:
> 
> * Does monitor hotplug already work?
> * Has the idea of monitor hotplug been discussed before?
> * Is there an obvious fundamental reason this cannot work?
> * What does the lifetime of a monitor look like?
> * I used it to review the code I wrote and point out bugs that I then
>   fixed myself.
> * I had various research questions around how io-thread interactions
>   work and common synchronization primitives in the codebase.
> 
> The code hasn't been written by any coding tool in line with what is
> requested in the contribution guidelines. I used it to speed up my
> understanding of what's going on and to debug things without using the
> output.

That's all fine, thanks for confirming. I was too suprised by the
thoroughness of the patches, given your caveat that you were not
familiar with QEMU internals !

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|


Re: [PATCH v3 0/5] monitor: add dynamic QMP monitor hotplug support
Posted by Christian Brauner 2 days, 20 hours ago
On Wed, Apr 08, 2026 at 10:52:25AM +0100, Daniel P. Berrangé wrote:
> On Tue, Apr 07, 2026 at 10:57:26PM +0200, Christian Brauner wrote:
> > On Tue, Apr 07, 2026 at 12:00:38PM +0100, Daniel P. Berrangé wrote:
> > > On Tue, Apr 07, 2026 at 09:32:44AM +0200, Christian Brauner wrote:
> > > > QEMU supports runtime hotplug for chardevs, devices, block backends,
> > > > and netdevs. Monitors are the only major subsystem that lacks this --
> > > > all QMP monitors must be configured at launch via -mon or -qmp CLI
> > > > options.
> > > > 
> > > > This series adds monitor-add, monitor-remove, and query-monitors QMP
> > > > commands so that management tools can create independent QMP sessions
> > > > on demand at runtime.
> > > > 
> > > > I've implemented a QMP-to-Varlink bridge in systemd. This allows
> > > > systemd-vmspawn to control virtual machines and containers through a
> > > > unified Varlink interface. Varlink allows protocol upgrades. For
> > > > example, it is possible to switch from Varlink to say http. I'm
> > > > allowing Varlink clients to upgrade from the Varlink interface to the
> > > > native QMP interface. Such clients get a new monitor assigned that
> > > > allows them to manage the virtual machine directly via QMP. The main
> > > > monitor remains unaffected and tied to the generic Varlink interface. We
> > > > can't pre-allocate monitors as we have no control over how many protocol
> > > > upgrades we actually get but it won't just be one. And having unused
> > > > monitors around really isn't ideal either.
> > > > 
> > > > Having the ability to hotplug monitors would really be helpful. I'm not
> > > > yet super well-versed in qemu internals so this might be done wrong but
> > > > testing works so far.
> > > 
> > > Can you say if AI agents/LLMs were used in creating these patches ? 
> > 
> > In exploring the QEMU codebase and how the QMP protocol is used in other
> > code-bases. So specifically I used it to research:
> > 
> > * Does monitor hotplug already work?
> > * Has the idea of monitor hotplug been discussed before?
> > * Is there an obvious fundamental reason this cannot work?
> > * What does the lifetime of a monitor look like?
> > * I used it to review the code I wrote and point out bugs that I then
> >   fixed myself.
> > * I had various research questions around how io-thread interactions
> >   work and common synchronization primitives in the codebase.
> > 
> > The code hasn't been written by any coding tool in line with what is
> > requested in the contribution guidelines. I used it to speed up my
> > understanding of what's going on and to debug things without using the
> > output.
> 
> That's all fine, thanks for confirming. I was too suprised by the
> thoroughness of the patches, given your caveat that you were not
> familiar with QEMU internals !

Well, please don't look at the first version then. :)