[PATCH RFC 0/6] spec: Decompose the daemon subpackage

Jim Fehlig posted 6 patches 1 year, 4 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20221123231155.31612-1-jfehlig@suse.com
There is a newer version of this series
libvirt.spec.in | 406 ++++++++++++++++++++++++++----------------------
1 file changed, 219 insertions(+), 187 deletions(-)
[PATCH RFC 0/6] spec: Decompose the daemon subpackage
Posted by Jim Fehlig 1 year, 4 months ago
Currently it is not possible to install a modular daemon subpackage without
also installing the monolithic daemon

https://listman.redhat.com/archives/libvir-list/2022-September/234554.html

This series is an initial attempt at moving common daemons, utilities, and
files from the daemon subpackage to a new daemon-core subpackage. The
monolithic and modular daemons can then depend on the new subpackage.

libvirt-guests is moved to a new libvirt-guests subpackage, which is
recommended by the daemon subpackage to provide smoother upgrade.

I've likely overlooked several items, but before continuing down this
path too far I first wanted to gauge interest and see if this work is
worth pursuing. If so, any comments on the RFC are appreciated!

Note that patches 1-3 are things I noticed while working on the others
and could be pushed independently.

Jim Fehlig (6):
  spec: Remove redundant with_libxl
  spec: Use more %{name} macro
  spec: Remove daemon postun trigger
  spec: Move common daemons to a separate subpackage
  spec: Move more files to the daemon-core subpackage
  spec: Move libvirt-guests to guests subpackage

 libvirt.spec.in | 406 ++++++++++++++++++++++++++----------------------
 1 file changed, 219 insertions(+), 187 deletions(-)

-- 
2.37.3
Re: [PATCH RFC 0/6] spec: Decompose the daemon subpackage
Posted by Jim Fehlig 1 year, 4 months ago
On 11/23/22 16:11, Jim Fehlig wrote:
> Currently it is not possible to install a modular daemon subpackage without
> also installing the monolithic daemon
> 
> https://listman.redhat.com/archives/libvir-list/2022-September/234554.html
> 
> This series is an initial attempt at moving common daemons, utilities, and
> files from the daemon subpackage to a new daemon-core subpackage. The
> monolithic and modular daemons can then depend on the new subpackage.
> 
> libvirt-guests is moved to a new libvirt-guests subpackage, which is
> recommended by the daemon subpackage to provide smoother upgrade.
> 
> I've likely overlooked several items, but before continuing down this
> path too far I first wanted to gauge interest and see if this work is
> worth pursuing. If so, any comments on the RFC are appreciated!

I have a small bit of time for this task today and have started reworking the 
series as Daniel suggested.

> Note that patches 1-3 are things I noticed while working on the others
> and could be pushed independently.

But I forgot about these patches, which are unrelated. Patch 1 could nearly be 
pushed under the trivial rule. Any comments on 2 or 3?

Regards,
Jim

> 
> Jim Fehlig (6):
>    spec: Remove redundant with_libxl
>    spec: Use more %{name} macro
>    spec: Remove daemon postun trigger
>    spec: Move common daemons to a separate subpackage
>    spec: Move more files to the daemon-core subpackage
>    spec: Move libvirt-guests to guests subpackage
> 
>   libvirt.spec.in | 406 ++++++++++++++++++++++++++----------------------
>   1 file changed, 219 insertions(+), 187 deletions(-)
>
Re: [PATCH RFC 0/6] spec: Decompose the daemon subpackage
Posted by Daniel P. Berrangé 1 year, 4 months ago
On Wed, Nov 23, 2022 at 04:11:49PM -0700, Jim Fehlig wrote:
> Currently it is not possible to install a modular daemon subpackage without
> also installing the monolithic daemon
> 
> https://listman.redhat.com/archives/libvir-list/2022-September/234554.html
> 
> This series is an initial attempt at moving common daemons, utilities, and
> files from the daemon subpackage to a new daemon-core subpackage. The
> monolithic and modular daemons can then depend on the new subpackage.
> 
> libvirt-guests is moved to a new libvirt-guests subpackage, which is
> recommended by the daemon subpackage to provide smoother upgrade.
> 
> I've likely overlooked several items, but before continuing down this
> path too far I first wanted to gauge interest and see if this work is
> worth pursuing. If so, any comments on the RFC are appreciated!

With this refactoring the two big questions

 - What is the desired end state
 - Can we ensure a clean upgrade path

Let me ignore everything except the QEMU driver and the nodedev driver,
for sake of illustration.

Today we've got a few install approaches recommended

 - Ask for 'libvirt', which gives you

     - libvirt-daemon
     - libvirt-daemon-driver-qemu
     - libvirt-daemon-driver-nodedev

   All the libvirt pieces, but not the hypervisor itself


 - Ask for 'libvirt-daemon-kvm', which also gives you
 
     - libvirt-daemon
     - libvirt-daemon-driver-qemu
     - libvirt-daemon-driver-nodedev
     - qemu-kvm

   All the recommended libvirt pieces for QEMU, including
   the hypervisor itself


 - Ask for 'libvirt-daemon-driver-qemu', 'libvirt-daemon-driver-nodedev',
   which also gives you

     - libvirt-daemon

   The bare minimum libvirt pieces, but not the hypervisor itself


In terms of package contents

  - libvirt-daemon-driver-qemu

     /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so
     /usr/sbin/virtqemud

  - libvirt-daemon

     /usr/sbin/libvirtd
     /usr/sbin/virtlogd
     /usr/sbin/virtlockd
     /usr/sbin/virtproxyd


Our goal is the ability to install only the module daemon, without
libvirtd. This implies that we need to cut the dependency from
libvirt-daemon-driver-qemu, to libvirt-daemon.

I didn't do that originally because it would break apps that
installed libvirt-daemon-driver-qemu, expecting to get libvirtd.
We also needed it for the virtlogd/virtproxyd dep.

That decision, however, was made when libvirtd was still the
default active daemon.  Since we changed the systemd presets
in Fedora/RHEL to activate modular daemons, we don't need the
deps to pull in libvirtd by default.

We do not want to force virlockd as that's an opt-in feature,
and also don't want to force virtproxyd as that's only needed
for back compat with old remote libvirt clients, or for exposing
over TCP. 

virtlogd is more difficult, it is possible to turn that off
and if people do that they might prefer not to have it installed.
Or they may want to use it, but have it installed in a different
container from virtqemud. So on balance I thin we need to sever
the dependancy there too.

So I'd say we need to have

  - libvirt-daemon-lock
  
      /usr/sbin/virtlockd

  - libvirt-daemon-log
  
      /usr/sbin/virtlogd

  - libvirt-daemon-proxy
  
      /usr/sbin/virtproxyd

Probably a 'libvirt-daemon-plugin-lockd'  for the lock-driver/lockd.so
as that really ought to be  an optional component too.

A 'libvirt-daemon-client' package for 'virt-admin'

The libvirt-daemon package would need to hold

     /usr/sbin/libvirtd

and would also need dependencies for the upgrade path on

    libvirt-daemon-lock
    libvirt-daemon-log
    libvirt-daemon-proxy
    libvirt-daemon-pugin-lockd
    libvirt-daemon-client

I'm not sure if we'll still need a 'libvirt-daemon-core' too as your
patches suggest, for some of the common files, or if they all end up
in one of the other sub-RPMs above.

The libvirt-daemon-driver-XXX packages would then have *no* dependency
on anything. If you are asking for libvirt-daemon-driver-XXXX packages
of any kind, you're responsible for pickin the exact set of pieces you
want to have present.

Installing 'libvirt-daemon' will give you libvirtd, virtproxyd, virtlockd
etc, and you can ask for libvirt-daemon-driver-XXX on top.

The 'libvirt-daemon-kvm' would give you the sensible set of pieces,
*not* included libvirt-daemon any more though on distro versions which
have switched to module daemons.

The only thing we can't achieve this way is to install libvirtd and
QEMU, without having the module daemons present. I'm not sure that
matters though, if we aim to discontinue shipping libvirtd long term.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH RFC 0/6] spec: Decompose the daemon subpackage
Posted by Jim Fehlig 1 year, 4 months ago
On 11/24/22 03:57, Daniel P. Berrangé wrote:
> On Wed, Nov 23, 2022 at 04:11:49PM -0700, Jim Fehlig wrote:
>> Currently it is not possible to install a modular daemon subpackage without
>> also installing the monolithic daemon
>>
>> https://listman.redhat.com/archives/libvir-list/2022-September/234554.html
>>
>> This series is an initial attempt at moving common daemons, utilities, and
>> files from the daemon subpackage to a new daemon-core subpackage. The
>> monolithic and modular daemons can then depend on the new subpackage.
>>
>> libvirt-guests is moved to a new libvirt-guests subpackage, which is
>> recommended by the daemon subpackage to provide smoother upgrade.
>>
>> I've likely overlooked several items, but before continuing down this
>> path too far I first wanted to gauge interest and see if this work is
>> worth pursuing. If so, any comments on the RFC are appreciated!
> 
> With this refactoring the two big questions
> 
>   - What is the desired end state
>   - Can we ensure a clean upgrade path
> 
> Let me ignore everything except the QEMU driver and the nodedev driver,
> for sake of illustration.
> 
> Today we've got a few install approaches recommended
> 
>   - Ask for 'libvirt', which gives you
> 
>       - libvirt-daemon
>       - libvirt-daemon-driver-qemu
>       - libvirt-daemon-driver-nodedev
> 
>     All the libvirt pieces, but not the hypervisor itself
> 
> 
>   - Ask for 'libvirt-daemon-kvm', which also gives you
>   
>       - libvirt-daemon
>       - libvirt-daemon-driver-qemu
>       - libvirt-daemon-driver-nodedev
>       - qemu-kvm
> 
>     All the recommended libvirt pieces for QEMU, including
>     the hypervisor itself
> 
> 
>   - Ask for 'libvirt-daemon-driver-qemu', 'libvirt-daemon-driver-nodedev',
>     which also gives you
> 
>       - libvirt-daemon
> 
>     The bare minimum libvirt pieces, but not the hypervisor itself

Why not the hypervisor in this scenario?

> In terms of package contents
> 
>    - libvirt-daemon-driver-qemu
> 
>       /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so
>       /usr/sbin/virtqemud
> 
>    - libvirt-daemon
> 
>       /usr/sbin/libvirtd
>       /usr/sbin/virtlogd
>       /usr/sbin/virtlockd
>       /usr/sbin/virtproxyd
> 
> 
> Our goal is the ability to install only the module daemon, without
> libvirtd. This implies that we need to cut the dependency from
> libvirt-daemon-driver-qemu, to libvirt-daemon.
> 
> I didn't do that originally because it would break apps that
> installed libvirt-daemon-driver-qemu, expecting to get libvirtd.
> We also needed it for the virtlogd/virtproxyd dep.
> 
> That decision, however, was made when libvirtd was still the
> default active daemon.  Since we changed the systemd presets
> in Fedora/RHEL to activate modular daemons, we don't need the
> deps to pull in libvirtd by default.
> 
> We do not want to force virlockd as that's an opt-in feature,
> and also don't want to force virtproxyd as that's only needed
> for back compat with old remote libvirt clients, or for exposing
> over TCP.
> 
> virtlogd is more difficult, it is possible to turn that off
> and if people do that they might prefer not to have it installed.
> Or they may want to use it, but have it installed in a different
> container from virtqemud. So on balance I thin we need to sever
> the dependancy there too.
> 
> So I'd say we need to have
> 
>    - libvirt-daemon-lock
>    
>        /usr/sbin/virtlockd
> 
>    - libvirt-daemon-log
>    
>        /usr/sbin/virtlogd
> 
>    - libvirt-daemon-proxy
>    
>        /usr/sbin/virtproxyd
> 

I considered splitting these as you suggest, but wasn't sure if the 
proliferation of subpackages would receive a warm welcome :-).

> Probably a 'libvirt-daemon-plugin-lockd'  for the lock-driver/lockd.so
> as that really ought to be  an optional component too.
> 
> A 'libvirt-daemon-client' package for 'virt-admin'
> 
> The libvirt-daemon package would need to hold
> 
>       /usr/sbin/libvirtd
> 
> and would also need dependencies for the upgrade path on
> 
>      libvirt-daemon-lock
>      libvirt-daemon-log
>      libvirt-daemon-proxy
>      libvirt-daemon-pugin-lockd
>      libvirt-daemon-client
> 
> I'm not sure if we'll still need a 'libvirt-daemon-core' too as your
> patches suggest, for some of the common files, or if they all end up
> in one of the other sub-RPMs above.

I'll see if they can be placed in other subpackages while reworking this series.

> The libvirt-daemon-driver-XXX packages would then have *no* dependency
> on anything. If you are asking for libvirt-daemon-driver-XXXX packages
> of any kind, you're responsible for pickin the exact set of pieces you
> want to have present.
> 
> Installing 'libvirt-daemon' will give you libvirtd, virtproxyd, virtlockd
> etc, and you can ask for libvirt-daemon-driver-XXX on top.
> 
> The 'libvirt-daemon-kvm' would give you the sensible set of pieces,
> *not* included libvirt-daemon any more though on distro versions which
> have switched to module daemons.
> 
> The only thing we can't achieve this way is to install libvirtd and
> QEMU, without having the module daemons present. I'm not sure that
> matters though, if we aim to discontinue shipping libvirtd long term.

Could be avoided by moving the qemu dependency to libvirt-daemon-driver-qemu right?

Regards,
Jim
Re: [PATCH RFC 0/6] spec: Decompose the daemon subpackage
Posted by Daniel P. Berrangé 1 year, 4 months ago
On Mon, Nov 28, 2022 at 10:53:52AM -0700, Jim Fehlig wrote:
> On 11/24/22 03:57, Daniel P. Berrangé wrote:
> > On Wed, Nov 23, 2022 at 04:11:49PM -0700, Jim Fehlig wrote:
> > > Currently it is not possible to install a modular daemon subpackage without
> > > also installing the monolithic daemon
> > > 
> > > https://listman.redhat.com/archives/libvir-list/2022-September/234554.html
> > > 
> > > This series is an initial attempt at moving common daemons, utilities, and
> > > files from the daemon subpackage to a new daemon-core subpackage. The
> > > monolithic and modular daemons can then depend on the new subpackage.
> > > 
> > > libvirt-guests is moved to a new libvirt-guests subpackage, which is
> > > recommended by the daemon subpackage to provide smoother upgrade.
> > > 
> > > I've likely overlooked several items, but before continuing down this
> > > path too far I first wanted to gauge interest and see if this work is
> > > worth pursuing. If so, any comments on the RFC are appreciated!
> > 
> > With this refactoring the two big questions
> > 
> >   - What is the desired end state
> >   - Can we ensure a clean upgrade path
> > 
> > Let me ignore everything except the QEMU driver and the nodedev driver,
> > for sake of illustration.
> > 
> > Today we've got a few install approaches recommended
> > 
> >   - Ask for 'libvirt', which gives you
> > 
> >       - libvirt-daemon
> >       - libvirt-daemon-driver-qemu
> >       - libvirt-daemon-driver-nodedev
> > 
> >     All the libvirt pieces, but not the hypervisor itself
> > 
> > 
> >   - Ask for 'libvirt-daemon-kvm', which also gives you
> >       - libvirt-daemon
> >       - libvirt-daemon-driver-qemu
> >       - libvirt-daemon-driver-nodedev
> >       - qemu-kvm
> > 
> >     All the recommended libvirt pieces for QEMU, including
> >     the hypervisor itself
> > 
> > 
> >   - Ask for 'libvirt-daemon-driver-qemu', 'libvirt-daemon-driver-nodedev',
> >     which also gives you
> > 
> >       - libvirt-daemon
> > 
> >     The bare minimum libvirt pieces, but not the hypervisor itself
> 
> Why not the hypervisor in this scenario?

'qemu-kvm' is a "typical installation" of QEMU packages on Fedora.

People wanting minimal installations want to fine tune exactly
which qemu-kvm-XXXX sub packages they install. So we don't force
any dep from  'libvirt-daemon-driver-qemu', let the user choose
exactly what they want.


> > So I'd say we need to have
> > 
> >    - libvirt-daemon-lock
> >        /usr/sbin/virtlockd
> > 
> >    - libvirt-daemon-log
> >        /usr/sbin/virtlogd
> > 
> >    - libvirt-daemon-proxy
> >        /usr/sbin/virtproxyd
> > 
> 
> I considered splitting these as you suggest, but wasn't sure if the
> proliferation of subpackages would receive a warm welcome :-).

We lost the non-proliferation war years ago ;-P Three more is
merely noise 


> > The libvirt-daemon-driver-XXX packages would then have *no* dependency
> > on anything. If you are asking for libvirt-daemon-driver-XXXX packages
> > of any kind, you're responsible for pickin the exact set of pieces you
> > want to have present.
> > 
> > Installing 'libvirt-daemon' will give you libvirtd, virtproxyd, virtlockd
> > etc, and you can ask for libvirt-daemon-driver-XXX on top.
> > 
> > The 'libvirt-daemon-kvm' would give you the sensible set of pieces,
> > *not* included libvirt-daemon any more though on distro versions which
> > have switched to module daemons.
> > 
> > The only thing we can't achieve this way is to install libvirtd and
> > QEMU, without having the module daemons present. I'm not sure that
> > matters though, if we aim to discontinue shipping libvirtd long term.
> 
> Could be avoided by moving the qemu dependency to libvirt-daemon-driver-qemu right?

See above for why we don't have a qemu dep from there.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH RFC 0/6] spec: Decompose the daemon subpackage
Posted by Andrea Bolognani 1 year, 4 months ago
On Mon, Nov 28, 2022 at 06:03:29PM +0000, Daniel P. Berrangé wrote:
> On Mon, Nov 28, 2022 at 10:53:52AM -0700, Jim Fehlig wrote:
> > On 11/24/22 03:57, Daniel P. Berrangé wrote:
> > >   - Ask for 'libvirt-daemon-driver-qemu', 'libvirt-daemon-driver-nodedev',
> > >     which also gives you
> > >
> > >       - libvirt-daemon
> > >
> > >     The bare minimum libvirt pieces, but not the hypervisor itself
> >
> > Why not the hypervisor in this scenario?
>
> 'qemu-kvm' is a "typical installation" of QEMU packages on Fedora.
>
> People wanting minimal installations want to fine tune exactly
> which qemu-kvm-XXXX sub packages they install. So we don't force
> any dep from  'libvirt-daemon-driver-qemu', let the user choose
> exactly what they want.

I wonder if we could use

  Requires: qemu-kvm-core
  Recommends: qemu-kvm

here. The idea being that default installations would get a
fully-featured QEMU binary, whiel those aiming for a minimal install
would be able to disable recommends and get a stripped down, but
still functional, QEMU binary.

> > > The only thing we can't achieve this way is to install libvirtd and
> > > QEMU, without having the module daemons present. I'm not sure that
> > > matters though, if we aim to discontinue shipping libvirtd long term.

That's already the case today, so I'd say there's no need to worry
about it.

-- 
Andrea Bolognani / Red Hat / Virtualization