[PATCH 00/26] integrate auto-shutdown of VMs with daemons

Daniel P. Berrangé posted 26 patches 1 week, 5 days ago
include/libvirt/libvirt-admin.h     |  13 ++
include/libvirt/libvirt-domain.h    |   4 +
src/admin/admin_protocol.x          |  11 +-
src/admin/admin_server_dispatch.c   |  13 ++
src/admin/libvirt-admin.c           |  33 ++++
src/admin/libvirt_admin_public.syms |   5 +
src/bhyve/bhyve_driver.c            |  53 ++----
src/conf/domain_conf.c              |   6 +-
src/conf/domain_conf.h              |   1 +
src/conf/virdomainobjlist.c         |   7 +-
src/driver-hypervisor.h             |  10 ++
src/hypervisor/domain_driver.c      | 250 ++++++++++++++++++++++++++++
src/hypervisor/domain_driver.h      |  42 +++++
src/libvirt-domain.c                |  87 ++++++++++
src/libvirt_private.syms            |  10 +-
src/libvirt_public.syms             |   6 +
src/libvirt_remote.syms             |   2 +-
src/libxl/libxl_driver.c            |  36 ++--
src/lxc/lxc_driver.c                |  13 +-
src/lxc/lxc_process.c               |  18 +-
src/lxc/lxc_process.h               |   2 +
src/qemu/libvirtd_qemu.aug          |   7 +
src/qemu/qemu.conf.in               |  59 +++++++
src/qemu/qemu_conf.c                |  63 +++++++
src/qemu/qemu_conf.h                |   7 +
src/qemu/qemu_driver.c              | 203 +++++++++++++---------
src/qemu/test_libvirtd_qemu.aug.in  |   7 +
src/remote/libvirtd.service.in      |   2 +-
src/remote/remote_daemon.c          |  78 +++------
src/remote/remote_driver.c          |   2 +
src/remote/remote_protocol.x        |  30 +++-
src/remote_protocol-structs         |  12 ++
src/rpc/gendispatch.pl              |   4 +-
src/rpc/virnetdaemon.c              | 212 +++++++++++++++++++----
src/rpc/virnetdaemon.h              |  20 ++-
src/util/virsystemd.c               |  41 ++++-
src/util/virsystemd.h               |   6 +-
src/virtd.service.in                |   2 +-
tools/virsh-domain-monitor.c        |   5 +
tools/virsh-domain.c                |  39 ++++-
tools/virt-admin.c                  |  41 +++++
41 files changed, 1181 insertions(+), 281 deletions(-)
[PATCH 00/26] integrate auto-shutdown of VMs with daemons
Posted by Daniel P. Berrangé 1 week, 5 days ago
This series starts the work needed to obsolete the libvirt-guests.sh
script which has grown a surprisingly large amount of functionality.

Currently the virt daemons will acquire inhibitors to delay OS shutdown
when VMs are running. The libvirt-guests.service unit can be used to
call libvirt-guests.sh to shutdown running VMs on system shutdown.

This split is a bad architecture because libvirt-guests.service will
only run once the system has decided to initiate the shutdown sequence.

When the user requests as shutdown while inhibitors are present, logind
will emit a "PrepareForShutdown" signal over dbus. Applications are
supposed to respond to this by preserving state & releasing their
inhibitors, which in turns allows shutdown to be initiated.

The remote daemon already has support for listening for the
"PrepareForShutdown" signal, but only does this for session instances,
not system instances.

This series essentially takes that logic and expands it to run in the
system instances too, thus conceptually making libvirt-guests.service
obsolete.

It is slightly more complicated than that though for many reasons...

Saving running VMs can take a very long time. The inhibitor delay
can be as low as 5 seconds, and when killing a service, systemd may
not wait very long for it to terminate. libvirt-guests.service deals
with this by setting TimeoutStopSecs=0 to make systemd wait forever.

This is undesirable to set in libvirtd.service though, as we would
like systemd to kill the daemon aggressively if it hangs. The series
thus uses the notification protocol to request systemd give it more
time to shutdown, as long as we're in the phase of saving running
VMs. A bug in this code will still result in systemd waiting forever,
which is no different from libvirt-guests.service, but a bug in any
other part of the libvirt daemon shutdown code will result in systemd
killing us.

The existing logic for saving VMs in the session daemons had many
feature gaps compared to libvirt-guests.sh. Thus there is code to
add support

 * Requesting graceful OS shutdown if managed save failed
 * Force poweroff of VMs if no other action worked
 * Optionally enabling/disabling use of managed save,
   graceful shutdown and force poweroff, which is more flexible
   than ON_SHUTDOWN=nnn, as we can try the whole sequence of
   options
 * Ability to bypass cache in managed save
 * Support for one-time autostart of VMs as an official API

To aid in testing this logic, virt-admin gains a new command

 'virt-admin daemon-shutdown --preserve'

All this new functionality is wired up into the QEMU driver, and is
made easily accessible to other hypervisor drivers, so would easily
be extendable to Xen, CH, LXC drivers, but this is not done in this
series. IOW, libvirt-guests.service is not yet fully obsolete.

The new functionality is also not enabled by default for the system
daemon, it requires explicit admin changes to /etc/libvirt/qemu.conf
to enable it. This is because it would clash with execution of the
libvirt-guests.service if both were enabled.

It is highly desirable that we enable this by default though, so we
need to figure out a upgrade story wrt libvirt-guests.service.

The only libvirt-guests.sh features not implemented are:

 * PARALLEL_SHUTDOWN=nn.

   When doing a graceful shutdown we initiate it on every single VM
   at once, and then monitor progress of all of them in parallel.

 * SYNC_TIME=nn

   When make not attempt to sync guest time when restoring from
   managed save. This ought to be fixed

Daniel P. Berrangé (26):
  util: add APIs for more systemd notifications
  remote: notify systemd when reloading config
  hypervisor: introduce helper for autostart
  src: convert drivers over to use new autostart helper
  hypervisor: add support for delay interval during autostart
  qemu: add 'auto_start_delay' configuration parameter
  hypervisor: move support for auto-shutdown out of QEMU driver
  remote: always invoke virStateStop for all daemons
  hypervisor: expand available shutdown actions
  hypervisor: custom shutdown actions for transient vs persistent VMs
  qemu: support automatic VM managed save in system daemon
  qemu: improve shutdown defaults for session daemon
  qemu: configurable delay for shutdown before poweroff
  hypervisor: support bypassing cache for managed save
  qemu: add config parameter to control auto-save bypass cache
  src: add new APIs for marking a domain to autostart once
  conf: implement support for autostart once feature
  hypervisor: wire up support for auto restore of running domains
  qemu: wire up support for once only autostart
  qemu: add config to control if auto-shutdown VMs are restored
  rpc: move state stop into virNetDaemon class
  rpc: don't unconditionally quit after preserving state
  rpc: fix shutdown sequence when preserving state
  admin: add 'daemon-shutdown' command
  rpc: don't let systemd shutdown daemon while saving VMs
  hypervisor: send systemd status messages while saving

 include/libvirt/libvirt-admin.h     |  13 ++
 include/libvirt/libvirt-domain.h    |   4 +
 src/admin/admin_protocol.x          |  11 +-
 src/admin/admin_server_dispatch.c   |  13 ++
 src/admin/libvirt-admin.c           |  33 ++++
 src/admin/libvirt_admin_public.syms |   5 +
 src/bhyve/bhyve_driver.c            |  53 ++----
 src/conf/domain_conf.c              |   6 +-
 src/conf/domain_conf.h              |   1 +
 src/conf/virdomainobjlist.c         |   7 +-
 src/driver-hypervisor.h             |  10 ++
 src/hypervisor/domain_driver.c      | 250 ++++++++++++++++++++++++++++
 src/hypervisor/domain_driver.h      |  42 +++++
 src/libvirt-domain.c                |  87 ++++++++++
 src/libvirt_private.syms            |  10 +-
 src/libvirt_public.syms             |   6 +
 src/libvirt_remote.syms             |   2 +-
 src/libxl/libxl_driver.c            |  36 ++--
 src/lxc/lxc_driver.c                |  13 +-
 src/lxc/lxc_process.c               |  18 +-
 src/lxc/lxc_process.h               |   2 +
 src/qemu/libvirtd_qemu.aug          |   7 +
 src/qemu/qemu.conf.in               |  59 +++++++
 src/qemu/qemu_conf.c                |  63 +++++++
 src/qemu/qemu_conf.h                |   7 +
 src/qemu/qemu_driver.c              | 203 +++++++++++++---------
 src/qemu/test_libvirtd_qemu.aug.in  |   7 +
 src/remote/libvirtd.service.in      |   2 +-
 src/remote/remote_daemon.c          |  78 +++------
 src/remote/remote_driver.c          |   2 +
 src/remote/remote_protocol.x        |  30 +++-
 src/remote_protocol-structs         |  12 ++
 src/rpc/gendispatch.pl              |   4 +-
 src/rpc/virnetdaemon.c              | 212 +++++++++++++++++++----
 src/rpc/virnetdaemon.h              |  20 ++-
 src/util/virsystemd.c               |  41 ++++-
 src/util/virsystemd.h               |   6 +-
 src/virtd.service.in                |   2 +-
 tools/virsh-domain-monitor.c        |   5 +
 tools/virsh-domain.c                |  39 ++++-
 tools/virt-admin.c                  |  41 +++++
 41 files changed, 1181 insertions(+), 281 deletions(-)

-- 
2.47.1