[libvirt] [PATCH 00/29] Split the libvirtd daemon into per-driver daemons

Daniel P. Berrangé posted 29 patches 13 weeks ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20190711160516.2130-1-berrange@redhat.com
.gitignore                                    |  12 +
build-aux/augeas-gentest.pl                   |   2 +-
libvirt.spec.in                               |  10 +
src/bhyve/Makefile.inc.am                     |  14 +
src/bhyve/bhyve_driver.c                      |   2 +-
src/driver.h                                  |   2 +
src/interface/Makefile.inc.am                 |  14 +
src/interface/interface_backend_netcf.c       |   6 +-
src/interface/interface_backend_udev.c        |   6 +-
src/libvirt.c                                 |  24 ++
src/libvirt_remote.syms                       |   1 +
src/libxl/Makefile.inc.am                     |  14 +
src/libxl/libxl_driver.c                      |   2 +-
src/locking/lock_daemon.c                     |   2 +-
src/logging/log_daemon.c                      |   2 +-
src/lxc/Makefile.inc.am                       |  15 +
src/lxc/lxc_driver.c                          |   2 +-
src/network/Makefile.inc.am                   |  14 +
src/network/leaseshelper.c                    |   2 +-
src/node_device/Makefile.inc.am               |  14 +
src/node_device/node_device_hal.c             |   2 +-
src/node_device/node_device_udev.c            |   2 +-
src/nwfilter/Makefile.inc.am                  |  14 +
src/nwfilter/nwfilter_driver.c                |   2 +-
src/qemu/Makefile.inc.am                      |  14 +
src/qemu/qemu_driver.c                        |   2 +-
src/remote/Makefile.inc.am                    | 153 ++++---
src/remote/{libvirtd.aug => libvirtd.aug.in}  |  24 +-
.../{libvirtd.conf => libvirtd.conf.in}       |  42 +-
src/remote/remote_daemon.c                    | 221 +++++++---
src/remote/remote_daemon_config.c             |  41 +-
src/remote/remote_daemon_config.h             |   9 +-
src/remote/remote_daemon_dispatch.c           |  82 +++-
src/remote/remote_driver.c                    | 391 ++++++++++++------
src/remote/remote_driver.h                    |   4 -
src/remote/test_libvirtd.aug.in               |  16 +-
src/rpc/virnetserver.c                        |  17 +
src/rpc/virnetserver.h                        |   3 +
src/secret/Makefile.inc.am                    |  14 +
src/secret/secret_driver.c                    |   2 +-
src/storage/Makefile.inc.am                   |  14 +
src/util/virlog.c                             |  20 +-
src/vbox/Makefile.inc.am                      |  14 +
src/vz/Makefile.inc.am                        |  14 +
src/vz/vz_driver.c                            |   2 +-
45 files changed, 938 insertions(+), 341 deletions(-)
rename src/remote/{libvirtd.aug => libvirtd.aug.in} (88%)
rename src/remote/{libvirtd.conf => libvirtd.conf.in} (95%)

[libvirt] [PATCH 00/29] Split the libvirtd daemon into per-driver daemons

Posted by Daniel P. Berrangé 13 weeks ago
This is what all the driver refactoring I've done has been about
enabling.

We gain new daemons for each driver, for the primary virt drivers:

  virtlibxld
  virtlxcd
  virtqemud
  virtvboxd
  virtvzd

And again for the secondary drivers

  virtinterfaced
  virtnetworkd
  virtnodedevd
  virtnwfilterd
  virtsecretd
  virtstoraged

Finally to support IP connectivity, and also the legacy lbivirtd UNIX
domain socket (for the old libvirt remote driver SSH tunnelling):

  virtproxyd

The the sake of facilitating upgrades, the existing libvirtd still
exists and works the same way it always has.

You either run libvirtd, or you run the per-driver daemons, never both.

The remote driver will look to see whether libvirtd is running to figure
out whether to connect to libvirtd or the new per-driver daemons.

When auto-spawning daemons for nonroot users, we default to spawning the
per-driver daemons.

This can be controlled with a UR parameter "?mode=direct|legacy|auto",
where 'direct' means per-driver and 'legacy' means libvirtd (or indirect
via virtproxyd if that's running).

Still todo

 - Add systemd unit files for the new daemons
 - Make it possible to disable build of libvirtd, or of the per-driver
   daemons so downstream vendors can decide which to ship
 - Tuning of the daemon defaults for worker threads to better suit
   the fact that we have per-driver daemons
 - More work on RPM packaging to allow install of per-driver daemosn
   without pulling in libvirtd too
 - A bunch of stuff that doesn't occurr to me right now
 - Identify & fix more bugs I've created here

Daniel P. Berrangé (29):
  rpc: add API for checking whether an auth scheme is in use on a server
  remote: simplify libvirtd code for deciding if SASL auth is needed
  logging: pass binary name not logfile name when enabling logging
  remote: conditionalize socket names in libvirtd daemon
  remote: conditionalize daemon name in libvirtd daemon
  remote: conditionalize driver loading in libvirtd daemon
  remote: conditionalize IP socket usage in libvirtd daemon
  remote: conditionalize IP socket config in libvirtd.conf
  remote: conditionalize IP socket config in augeas definitions
  remote: refactor & rename variables for building libvirtd
  secret: introduce virtsecretd daemon
  network: introduce virtnetworkd daemon
  interface: introduce virtinterfaced daemon
  storage: introduce virtstoraged daemon
  nodedev: introduce virtnodedevd daemon
  nwfilter: introduce virtnwfilterd daemon
  libxl: introduce virtlibxld daemon
  qemu: introduce virtqemud daemon
  lxc: introduce virtlxcd daemon
  vbox: introduce virtvboxd daemon
  bhyve: introduce virtbhyved daemon
  vz: introduce virtvzd daemon
  remote: introduce virtproxyd daemon to handle IP connectivity
  remote: open secondary drivers via remote driver if needed
  remote: use enum helpers for parsing remote driver transport
  remote: refactor the code for choosing the UNIX socket path
  remote: switch to connect to per-driver daemons by default
  all: don't wait for driver lock during startup
  interface: fix driver name in state directory path

 .gitignore                                    |  12 +
 build-aux/augeas-gentest.pl                   |   2 +-
 libvirt.spec.in                               |  10 +
 src/bhyve/Makefile.inc.am                     |  14 +
 src/bhyve/bhyve_driver.c                      |   2 +-
 src/driver.h                                  |   2 +
 src/interface/Makefile.inc.am                 |  14 +
 src/interface/interface_backend_netcf.c       |   6 +-
 src/interface/interface_backend_udev.c        |   6 +-
 src/libvirt.c                                 |  24 ++
 src/libvirt_remote.syms                       |   1 +
 src/libxl/Makefile.inc.am                     |  14 +
 src/libxl/libxl_driver.c                      |   2 +-
 src/locking/lock_daemon.c                     |   2 +-
 src/logging/log_daemon.c                      |   2 +-
 src/lxc/Makefile.inc.am                       |  15 +
 src/lxc/lxc_driver.c                          |   2 +-
 src/network/Makefile.inc.am                   |  14 +
 src/network/leaseshelper.c                    |   2 +-
 src/node_device/Makefile.inc.am               |  14 +
 src/node_device/node_device_hal.c             |   2 +-
 src/node_device/node_device_udev.c            |   2 +-
 src/nwfilter/Makefile.inc.am                  |  14 +
 src/nwfilter/nwfilter_driver.c                |   2 +-
 src/qemu/Makefile.inc.am                      |  14 +
 src/qemu/qemu_driver.c                        |   2 +-
 src/remote/Makefile.inc.am                    | 153 ++++---
 src/remote/{libvirtd.aug => libvirtd.aug.in}  |  24 +-
 .../{libvirtd.conf => libvirtd.conf.in}       |  42 +-
 src/remote/remote_daemon.c                    | 221 +++++++---
 src/remote/remote_daemon_config.c             |  41 +-
 src/remote/remote_daemon_config.h             |   9 +-
 src/remote/remote_daemon_dispatch.c           |  82 +++-
 src/remote/remote_driver.c                    | 391 ++++++++++++------
 src/remote/remote_driver.h                    |   4 -
 src/remote/test_libvirtd.aug.in               |  16 +-
 src/rpc/virnetserver.c                        |  17 +
 src/rpc/virnetserver.h                        |   3 +
 src/secret/Makefile.inc.am                    |  14 +
 src/secret/secret_driver.c                    |   2 +-
 src/storage/Makefile.inc.am                   |  14 +
 src/util/virlog.c                             |  20 +-
 src/vbox/Makefile.inc.am                      |  14 +
 src/vz/Makefile.inc.am                        |  14 +
 src/vz/vz_driver.c                            |   2 +-
 45 files changed, 938 insertions(+), 341 deletions(-)
 rename src/remote/{libvirtd.aug => libvirtd.aug.in} (88%)
 rename src/remote/{libvirtd.conf => libvirtd.conf.in} (95%)

-- 
2.21.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH 00/29] Split the libvirtd daemon into per-driver daemons

Posted by Michal Privoznik 13 weeks ago
On 7/11/19 6:04 PM, Daniel P. Berrangé wrote:
> This is what all the driver refactoring I've done has been about
> enabling.
> 
> We gain new daemons for each driver, for the primary virt drivers:
> 
>    virtlibxld
>    virtlxcd
>    virtqemud
>    virtvboxd
>    virtvzd
> 
> And again for the secondary drivers
> 
>    virtinterfaced
>    virtnetworkd
>    virtnodedevd
>    virtnwfilterd
>    virtsecretd
>    virtstoraged
> 
> Finally to support IP connectivity, and also the legacy lbivirtd UNIX
> domain socket (for the old libvirt remote driver SSH tunnelling):
> 
>    virtproxyd
> 
> The the sake of facilitating upgrades, the existing libvirtd still
> exists and works the same way it always has.
> 
> You either run libvirtd, or you run the per-driver daemons, never both.
> 
> The remote driver will look to see whether libvirtd is running to figure
> out whether to connect to libvirtd or the new per-driver daemons.
> 
> When auto-spawning daemons for nonroot users, we default to spawning the
> per-driver daemons.
> 
> This can be controlled with a UR parameter "?mode=direct|legacy|auto",
> where 'direct' means per-driver and 'legacy' means libvirtd (or indirect
> via virtproxyd if that's running).
> 
> Still todo
> 
>   - Add systemd unit files for the new daemons
>   - Make it possible to disable build of libvirtd, or of the per-driver
>     daemons so downstream vendors can decide which to ship
>   - Tuning of the daemon defaults for worker threads to better suit
>     the fact that we have per-driver daemons
>   - More work on RPM packaging to allow install of per-driver daemosn
>     without pulling in libvirtd too
>   - A bunch of stuff that doesn't occurr to me right now
>   - Identify & fix more bugs I've created here

Teach virt-admin how to connect to individual daemons.

> 
> Daniel P. Berrangé (29):
>    rpc: add API for checking whether an auth scheme is in use on a server
>    remote: simplify libvirtd code for deciding if SASL auth is needed
>    logging: pass binary name not logfile name when enabling logging
>    remote: conditionalize socket names in libvirtd daemon
>    remote: conditionalize daemon name in libvirtd daemon
>    remote: conditionalize driver loading in libvirtd daemon
>    remote: conditionalize IP socket usage in libvirtd daemon
>    remote: conditionalize IP socket config in libvirtd.conf
>    remote: conditionalize IP socket config in augeas definitions
>    remote: refactor & rename variables for building libvirtd
>    secret: introduce virtsecretd daemon
>    network: introduce virtnetworkd daemon
>    interface: introduce virtinterfaced daemon
>    storage: introduce virtstoraged daemon
>    nodedev: introduce virtnodedevd daemon
>    nwfilter: introduce virtnwfilterd daemon
>    libxl: introduce virtlibxld daemon
>    qemu: introduce virtqemud daemon
>    lxc: introduce virtlxcd daemon
>    vbox: introduce virtvboxd daemon
>    bhyve: introduce virtbhyved daemon
>    vz: introduce virtvzd daemon
>    remote: introduce virtproxyd daemon to handle IP connectivity
>    remote: open secondary drivers via remote driver if needed
>    remote: use enum helpers for parsing remote driver transport
>    remote: refactor the code for choosing the UNIX socket path
>    remote: switch to connect to per-driver daemons by default
>    all: don't wait for driver lock during startup
>    interface: fix driver name in state directory path
> 
>   .gitignore                                    |  12 +
>   build-aux/augeas-gentest.pl                   |   2 +-
>   libvirt.spec.in                               |  10 +
>   src/bhyve/Makefile.inc.am                     |  14 +
>   src/bhyve/bhyve_driver.c                      |   2 +-
>   src/driver.h                                  |   2 +
>   src/interface/Makefile.inc.am                 |  14 +
>   src/interface/interface_backend_netcf.c       |   6 +-
>   src/interface/interface_backend_udev.c        |   6 +-
>   src/libvirt.c                                 |  24 ++
>   src/libvirt_remote.syms                       |   1 +
>   src/libxl/Makefile.inc.am                     |  14 +
>   src/libxl/libxl_driver.c                      |   2 +-
>   src/locking/lock_daemon.c                     |   2 +-
>   src/logging/log_daemon.c                      |   2 +-
>   src/lxc/Makefile.inc.am                       |  15 +
>   src/lxc/lxc_driver.c                          |   2 +-
>   src/network/Makefile.inc.am                   |  14 +
>   src/network/leaseshelper.c                    |   2 +-
>   src/node_device/Makefile.inc.am               |  14 +
>   src/node_device/node_device_hal.c             |   2 +-
>   src/node_device/node_device_udev.c            |   2 +-
>   src/nwfilter/Makefile.inc.am                  |  14 +
>   src/nwfilter/nwfilter_driver.c                |   2 +-
>   src/qemu/Makefile.inc.am                      |  14 +
>   src/qemu/qemu_driver.c                        |   2 +-
>   src/remote/Makefile.inc.am                    | 153 ++++---
>   src/remote/{libvirtd.aug => libvirtd.aug.in}  |  24 +-
>   .../{libvirtd.conf => libvirtd.conf.in}       |  42 +-
>   src/remote/remote_daemon.c                    | 221 +++++++---
>   src/remote/remote_daemon_config.c             |  41 +-
>   src/remote/remote_daemon_config.h             |   9 +-
>   src/remote/remote_daemon_dispatch.c           |  82 +++-
>   src/remote/remote_driver.c                    | 391 ++++++++++++------
>   src/remote/remote_driver.h                    |   4 -
>   src/remote/test_libvirtd.aug.in               |  16 +-
>   src/rpc/virnetserver.c                        |  17 +
>   src/rpc/virnetserver.h                        |   3 +
>   src/secret/Makefile.inc.am                    |  14 +
>   src/secret/secret_driver.c                    |   2 +-
>   src/storage/Makefile.inc.am                   |  14 +
>   src/util/virlog.c                             |  20 +-
>   src/vbox/Makefile.inc.am                      |  14 +
>   src/vz/Makefile.inc.am                        |  14 +
>   src/vz/vz_driver.c                            |   2 +-
>   45 files changed, 938 insertions(+), 341 deletions(-)
>   rename src/remote/{libvirtd.aug => libvirtd.aug.in} (88%)
>   rename src/remote/{libvirtd.conf => libvirtd.conf.in} (95%)
> 

Patches look good, but they have couple of small flaws. I don't want to 
make you post v2, so you can count on my ACK if you fix all issues I've 
raised. But if you'd like to post v2 feel free to do so.

Michal

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list