[libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries

Pavel Hrdina posted 21 patches 3 years ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/cover.1618851847.git.phrdina@redhat.com
There is a newer version of this series
libvirt.spec.in                               |  31 ---
meson.build                                   | 214 ++++++------------
src/bhyve/bhyve_command.c                     |   4 +
src/libvirt_private.syms                      |   6 +-
src/locking/lock_driver_lockd.c               |  12 +-
src/network/bridge_driver.c                   |   8 +-
src/node_device/node_device_driver.c          |   2 +
src/qemu/qemu_conf.c                          |  23 +-
src/qemu/qemu_domain.c                        |   3 +-
src/storage/storage_backend_fs.c              |  24 +-
src/storage/storage_backend_logical.c         |  13 ++
src/storage/storage_backend_sheepdog.c        |   2 +
src/storage/storage_backend_zfs.c             |   3 +
src/storage/storage_util.c                    |   2 +
src/storage/storage_util.h                    |   6 +
src/util/virdnsmasq.c                         |  56 +----
src/util/virdnsmasq.h                         |   8 +-
src/util/virfile.c                            |  16 +-
src/util/virfile.h                            |   6 +-
src/util/virfirewall.c                        |   4 +-
src/util/virfirewall.h                        |   4 +
src/util/viriscsi.h                           |   2 +
src/util/virkmod.h                            |   3 +
src/util/virnetdev.c                          |  46 ----
src/util/virnetdev.h                          |   4 -
src/util/virnetdevbandwidth.c                 |  50 ++++
src/util/virnetdevbandwidth.h                 |   6 +
src/util/virnetdevip.c                        |   2 +
src/util/virnetdevmidonet.c                   |   2 +
src/util/virnetdevopenvswitch.c               |   2 +
src/util/virnuma.c                            |   1 +
src/util/virsysinfo.c                         |   1 +
src/util/virutil.c                            |   2 +
.../bhyvexml2argv-acpiapic.args               |   2 +-
.../bhyvexml2argv-acpiapic.ldargs             |   2 +-
...ml2argv-addr-isa-controller-on-slot-1.args |   2 +-
...2argv-addr-isa-controller-on-slot-1.ldargs |   2 +-
...l2argv-addr-isa-controller-on-slot-31.args |   2 +-
...argv-addr-isa-controller-on-slot-31.ldargs |   2 +-
...xml2argv-addr-more-than-32-sata-disks.args |   2 +-
...l2argv-addr-more-than-32-sata-disks.ldargs |   2 +-
...hyvexml2argv-addr-multiple-sata-disks.args |   2 +-
...vexml2argv-addr-multiple-sata-disks.ldargs |   2 +-
...vexml2argv-addr-multiple-virtio-disks.args |   2 +-
...xml2argv-addr-multiple-virtio-disks.ldargs |   2 +-
...rgv-addr-no32devs-multiple-sata-disks.args |   2 +-
...v-addr-no32devs-multiple-sata-disks.ldargs |   2 +-
...l2argv-addr-no32devs-single-sata-disk.args |   2 +-
...argv-addr-no32devs-single-sata-disk.ldargs |   2 +-
...rgv-addr-non-isa-controller-on-slot-1.args |   2 +-
.../bhyvexml2argv-addr-single-sata-disk.args  |   2 +-
...bhyvexml2argv-addr-single-sata-disk.ldargs |   2 +-
...bhyvexml2argv-addr-single-virtio-disk.args |   2 +-
...yvexml2argv-addr-single-virtio-disk.ldargs |   2 +-
.../bhyvexml2argvdata/bhyvexml2argv-base.args |   2 +-
.../bhyvexml2argv-base.ldargs                 |   2 +-
.../bhyvexml2argv-bhyveload-bootorder.args    |   2 +-
.../bhyvexml2argv-bhyveload-bootorder.ldargs  |   2 +-
.../bhyvexml2argv-bhyveload-bootorder1.args   |   2 +-
.../bhyvexml2argv-bhyveload-bootorder1.ldargs |   2 +-
.../bhyvexml2argv-bhyveload-bootorder3.args   |   2 +-
.../bhyvexml2argv-bhyveload-bootorder3.ldargs |   2 +-
.../bhyvexml2argv-bhyveload-explicitargs.args |   2 +-
...hyvexml2argv-bhyveload-explicitargs.ldargs |   2 +-
.../bhyvexml2argv-commandline.args            |   2 +-
.../bhyvexml2argv-commandline.ldargs          |   2 +-
...gv-console-master-slave-not-specified.args |   2 +-
...-console-master-slave-not-specified.ldargs |   2 +-
.../bhyvexml2argv-console.args                |   2 +-
.../bhyvexml2argv-console.ldargs              |   2 +-
.../bhyvexml2argv-cputopology.args            |   2 +-
.../bhyvexml2argv-cputopology.ldargs          |   2 +-
.../bhyvexml2argv-custom-loader.args          |   2 +-
.../bhyvexml2argv-custom-loader.ldargs        |   2 +-
.../bhyvexml2argv-disk-cdrom-grub.args        |   2 +-
.../bhyvexml2argv-disk-cdrom-grub.ldargs      |   2 +-
.../bhyvexml2argv-disk-cdrom.args             |   2 +-
.../bhyvexml2argv-disk-cdrom.ldargs           |   2 +-
.../bhyvexml2argv-disk-virtio.args            |   2 +-
.../bhyvexml2argv-disk-virtio.ldargs          |   2 +-
.../bhyvexml2argv-firmware-efi.args           |   2 +-
.../bhyvexml2argv-fs-9p-readonly.args         |   2 +-
.../bhyvexml2argv-fs-9p-readonly.ldargs       |   2 +-
.../bhyvexml2argv-fs-9p.args                  |   2 +-
.../bhyvexml2argv-fs-9p.ldargs                |   2 +-
.../bhyvexml2argv-grub-bootorder.args         |   2 +-
.../bhyvexml2argv-grub-bootorder.ldargs       |   2 +-
.../bhyvexml2argv-grub-bootorder2.args        |   2 +-
.../bhyvexml2argv-grub-bootorder2.ldargs      |   2 +-
.../bhyvexml2argv-grub-defaults.args          |   2 +-
.../bhyvexml2argv-grub-defaults.ldargs        |   2 +-
.../bhyvexml2argv-input-xhci-tablet.args      |   2 +-
.../bhyvexml2argv-input-xhci-tablet.ldargs    |   2 +-
.../bhyvexml2argv-isa-controller.args         |   2 +-
.../bhyvexml2argv-isa-controller.ldargs       |   2 +-
.../bhyvexml2argv-localtime.args              |   2 +-
.../bhyvexml2argv-localtime.ldargs            |   2 +-
.../bhyvexml2argv-macaddr.args                |   2 +-
.../bhyvexml2argv-macaddr.ldargs              |   2 +-
.../bhyvexml2argvdata/bhyvexml2argv-msrs.args |   2 +-
.../bhyvexml2argv-msrs.ldargs                 |   2 +-
.../bhyvexml2argv-net-e1000.args              |   2 +-
.../bhyvexml2argv-net-e1000.ldargs            |   2 +-
.../bhyvexml2argv-serial-grub-nocons.args     |   2 +-
.../bhyvexml2argv-serial-grub-nocons.ldargs   |   2 +-
.../bhyvexml2argv-serial-grub.args            |   2 +-
.../bhyvexml2argv-serial-grub.ldargs          |   2 +-
.../bhyvexml2argv-serial.args                 |   2 +-
.../bhyvexml2argv-serial.ldargs               |   2 +-
.../bhyvexml2argv-sound.args                  |   2 +-
.../bhyvexml2argv-sound.ldargs                |   2 +-
.../bhyvexml2argvdata/bhyvexml2argv-uefi.args |   2 +-
.../bhyvexml2argv-vnc-autoport.args           |   2 +-
.../bhyvexml2argv-vnc-password.args           |   2 +-
.../bhyvexml2argv-vnc-resolution.args         |   2 +-
.../bhyvexml2argv-vnc-vgaconf-io.args         |   2 +-
.../bhyvexml2argv-vnc-vgaconf-off.args        |   2 +-
.../bhyvexml2argv-vnc-vgaconf-on.args         |   2 +-
.../bhyvexml2argvdata/bhyvexml2argv-vnc.args  |   2 +-
.../bhyvexml2argv-wired.args                  |   2 +-
.../bhyvexml2argv-wired.ldargs                |   2 +-
tests/bhyvexml2argvtest.c                     |   4 +-
tests/domaincapsmock.c                        |  17 ++
tests/meson.build                             |   1 +
tests/networkxml2conftest.c                   |   6 +-
tests/networkxml2firewalltest.c               |  16 +-
tests/nwfilterebiptablestest.c                |  15 +-
tests/nwfilterxml2firewalltest.c              |  14 +-
tests/qemuxml2argvmock.c                      |   5 +-
tests/testutilsqemu.c                         |  15 --
tests/virfirewallmock.c                       |  35 +++
tests/virfirewalltest.c                       |  15 +-
132 files changed, 403 insertions(+), 484 deletions(-)
create mode 100644 tests/virfirewallmock.c
[libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Pavel Hrdina 3 years ago
Recent attempt to add a lot of meson options to specify different
runtime paths motivated me enough to cleanup this from meson.

Changes in v2:
    - split and rework patch 16/17 to address review comments
    - added a new patch to cleanup libvirt.spec.in file

Pavel Hrdina (21):
  bridge_driver: fix comment about dnsmasqCaps
  virdnsmasq: drop unused dnsmasqCapsNewFromFile function
  virdnsmasq: drop unused dnsmasqCapsRefresh function
  virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBuffer
  virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBinary
  virfirewall: use virFindFileInPath instead of virFileIsExecutable
  tests: introduce virfirewallmock
  tests: use virfirewallmock instead of hasNetfilterTools
  virnetdev: move virNetDevSetRootQDisc to virnetdevbandwidth
  tests: testutilsqemu: move virFindFileInPath into domaincapsmock
  meson: don't check collie as program for sheepdog
  bhyvexml2argvtest: use virCommandToStringFull to strip command path
  storage: use virFindFileInPath to validate presence of mkfs
  virfile: introduce virFindFileInPathFull()
  qemu_conf: use virFindFileInPathFull for runtime binaries
  meson: drop check for runtime binary dependencies
  meson: move iscsiadm check into storage_iscsi condition
  meson: stop setting runtime binaries defines during compilation
  meson: use runtime binaries to only resolve features with "auto" value
  meson: optional_programs should be used only for building libvirt
  libvirt.spec: drop no longer required build dependencies

 libvirt.spec.in                               |  31 ---
 meson.build                                   | 214 ++++++------------
 src/bhyve/bhyve_command.c                     |   4 +
 src/libvirt_private.syms                      |   6 +-
 src/locking/lock_driver_lockd.c               |  12 +-
 src/network/bridge_driver.c                   |   8 +-
 src/node_device/node_device_driver.c          |   2 +
 src/qemu/qemu_conf.c                          |  23 +-
 src/qemu/qemu_domain.c                        |   3 +-
 src/storage/storage_backend_fs.c              |  24 +-
 src/storage/storage_backend_logical.c         |  13 ++
 src/storage/storage_backend_sheepdog.c        |   2 +
 src/storage/storage_backend_zfs.c             |   3 +
 src/storage/storage_util.c                    |   2 +
 src/storage/storage_util.h                    |   6 +
 src/util/virdnsmasq.c                         |  56 +----
 src/util/virdnsmasq.h                         |   8 +-
 src/util/virfile.c                            |  16 +-
 src/util/virfile.h                            |   6 +-
 src/util/virfirewall.c                        |   4 +-
 src/util/virfirewall.h                        |   4 +
 src/util/viriscsi.h                           |   2 +
 src/util/virkmod.h                            |   3 +
 src/util/virnetdev.c                          |  46 ----
 src/util/virnetdev.h                          |   4 -
 src/util/virnetdevbandwidth.c                 |  50 ++++
 src/util/virnetdevbandwidth.h                 |   6 +
 src/util/virnetdevip.c                        |   2 +
 src/util/virnetdevmidonet.c                   |   2 +
 src/util/virnetdevopenvswitch.c               |   2 +
 src/util/virnuma.c                            |   1 +
 src/util/virsysinfo.c                         |   1 +
 src/util/virutil.c                            |   2 +
 .../bhyvexml2argv-acpiapic.args               |   2 +-
 .../bhyvexml2argv-acpiapic.ldargs             |   2 +-
 ...ml2argv-addr-isa-controller-on-slot-1.args |   2 +-
 ...2argv-addr-isa-controller-on-slot-1.ldargs |   2 +-
 ...l2argv-addr-isa-controller-on-slot-31.args |   2 +-
 ...argv-addr-isa-controller-on-slot-31.ldargs |   2 +-
 ...xml2argv-addr-more-than-32-sata-disks.args |   2 +-
 ...l2argv-addr-more-than-32-sata-disks.ldargs |   2 +-
 ...hyvexml2argv-addr-multiple-sata-disks.args |   2 +-
 ...vexml2argv-addr-multiple-sata-disks.ldargs |   2 +-
 ...vexml2argv-addr-multiple-virtio-disks.args |   2 +-
 ...xml2argv-addr-multiple-virtio-disks.ldargs |   2 +-
 ...rgv-addr-no32devs-multiple-sata-disks.args |   2 +-
 ...v-addr-no32devs-multiple-sata-disks.ldargs |   2 +-
 ...l2argv-addr-no32devs-single-sata-disk.args |   2 +-
 ...argv-addr-no32devs-single-sata-disk.ldargs |   2 +-
 ...rgv-addr-non-isa-controller-on-slot-1.args |   2 +-
 .../bhyvexml2argv-addr-single-sata-disk.args  |   2 +-
 ...bhyvexml2argv-addr-single-sata-disk.ldargs |   2 +-
 ...bhyvexml2argv-addr-single-virtio-disk.args |   2 +-
 ...yvexml2argv-addr-single-virtio-disk.ldargs |   2 +-
 .../bhyvexml2argvdata/bhyvexml2argv-base.args |   2 +-
 .../bhyvexml2argv-base.ldargs                 |   2 +-
 .../bhyvexml2argv-bhyveload-bootorder.args    |   2 +-
 .../bhyvexml2argv-bhyveload-bootorder.ldargs  |   2 +-
 .../bhyvexml2argv-bhyveload-bootorder1.args   |   2 +-
 .../bhyvexml2argv-bhyveload-bootorder1.ldargs |   2 +-
 .../bhyvexml2argv-bhyveload-bootorder3.args   |   2 +-
 .../bhyvexml2argv-bhyveload-bootorder3.ldargs |   2 +-
 .../bhyvexml2argv-bhyveload-explicitargs.args |   2 +-
 ...hyvexml2argv-bhyveload-explicitargs.ldargs |   2 +-
 .../bhyvexml2argv-commandline.args            |   2 +-
 .../bhyvexml2argv-commandline.ldargs          |   2 +-
 ...gv-console-master-slave-not-specified.args |   2 +-
 ...-console-master-slave-not-specified.ldargs |   2 +-
 .../bhyvexml2argv-console.args                |   2 +-
 .../bhyvexml2argv-console.ldargs              |   2 +-
 .../bhyvexml2argv-cputopology.args            |   2 +-
 .../bhyvexml2argv-cputopology.ldargs          |   2 +-
 .../bhyvexml2argv-custom-loader.args          |   2 +-
 .../bhyvexml2argv-custom-loader.ldargs        |   2 +-
 .../bhyvexml2argv-disk-cdrom-grub.args        |   2 +-
 .../bhyvexml2argv-disk-cdrom-grub.ldargs      |   2 +-
 .../bhyvexml2argv-disk-cdrom.args             |   2 +-
 .../bhyvexml2argv-disk-cdrom.ldargs           |   2 +-
 .../bhyvexml2argv-disk-virtio.args            |   2 +-
 .../bhyvexml2argv-disk-virtio.ldargs          |   2 +-
 .../bhyvexml2argv-firmware-efi.args           |   2 +-
 .../bhyvexml2argv-fs-9p-readonly.args         |   2 +-
 .../bhyvexml2argv-fs-9p-readonly.ldargs       |   2 +-
 .../bhyvexml2argv-fs-9p.args                  |   2 +-
 .../bhyvexml2argv-fs-9p.ldargs                |   2 +-
 .../bhyvexml2argv-grub-bootorder.args         |   2 +-
 .../bhyvexml2argv-grub-bootorder.ldargs       |   2 +-
 .../bhyvexml2argv-grub-bootorder2.args        |   2 +-
 .../bhyvexml2argv-grub-bootorder2.ldargs      |   2 +-
 .../bhyvexml2argv-grub-defaults.args          |   2 +-
 .../bhyvexml2argv-grub-defaults.ldargs        |   2 +-
 .../bhyvexml2argv-input-xhci-tablet.args      |   2 +-
 .../bhyvexml2argv-input-xhci-tablet.ldargs    |   2 +-
 .../bhyvexml2argv-isa-controller.args         |   2 +-
 .../bhyvexml2argv-isa-controller.ldargs       |   2 +-
 .../bhyvexml2argv-localtime.args              |   2 +-
 .../bhyvexml2argv-localtime.ldargs            |   2 +-
 .../bhyvexml2argv-macaddr.args                |   2 +-
 .../bhyvexml2argv-macaddr.ldargs              |   2 +-
 .../bhyvexml2argvdata/bhyvexml2argv-msrs.args |   2 +-
 .../bhyvexml2argv-msrs.ldargs                 |   2 +-
 .../bhyvexml2argv-net-e1000.args              |   2 +-
 .../bhyvexml2argv-net-e1000.ldargs            |   2 +-
 .../bhyvexml2argv-serial-grub-nocons.args     |   2 +-
 .../bhyvexml2argv-serial-grub-nocons.ldargs   |   2 +-
 .../bhyvexml2argv-serial-grub.args            |   2 +-
 .../bhyvexml2argv-serial-grub.ldargs          |   2 +-
 .../bhyvexml2argv-serial.args                 |   2 +-
 .../bhyvexml2argv-serial.ldargs               |   2 +-
 .../bhyvexml2argv-sound.args                  |   2 +-
 .../bhyvexml2argv-sound.ldargs                |   2 +-
 .../bhyvexml2argvdata/bhyvexml2argv-uefi.args |   2 +-
 .../bhyvexml2argv-vnc-autoport.args           |   2 +-
 .../bhyvexml2argv-vnc-password.args           |   2 +-
 .../bhyvexml2argv-vnc-resolution.args         |   2 +-
 .../bhyvexml2argv-vnc-vgaconf-io.args         |   2 +-
 .../bhyvexml2argv-vnc-vgaconf-off.args        |   2 +-
 .../bhyvexml2argv-vnc-vgaconf-on.args         |   2 +-
 .../bhyvexml2argvdata/bhyvexml2argv-vnc.args  |   2 +-
 .../bhyvexml2argv-wired.args                  |   2 +-
 .../bhyvexml2argv-wired.ldargs                |   2 +-
 tests/bhyvexml2argvtest.c                     |   4 +-
 tests/domaincapsmock.c                        |  17 ++
 tests/meson.build                             |   1 +
 tests/networkxml2conftest.c                   |   6 +-
 tests/networkxml2firewalltest.c               |  16 +-
 tests/nwfilterebiptablestest.c                |  15 +-
 tests/nwfilterxml2firewalltest.c              |  14 +-
 tests/qemuxml2argvmock.c                      |   5 +-
 tests/testutilsqemu.c                         |  15 --
 tests/virfirewallmock.c                       |  35 +++
 tests/virfirewalltest.c                       |  15 +-
 132 files changed, 403 insertions(+), 484 deletions(-)
 create mode 100644 tests/virfirewallmock.c

-- 
2.30.2

Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Neal Gompa 3 years ago
On Mon, Apr 19, 2021 at 1:14 PM Pavel Hrdina <phrdina@redhat.com> wrote:
>
> Recent attempt to add a lot of meson options to specify different
> runtime paths motivated me enough to cleanup this from meson.
>
> Changes in v2:
>     - split and rework patch 16/17 to address review comments
>     - added a new patch to cleanup libvirt.spec.in file
>
> Pavel Hrdina (21):
>   bridge_driver: fix comment about dnsmasqCaps
>   virdnsmasq: drop unused dnsmasqCapsNewFromFile function
>   virdnsmasq: drop unused dnsmasqCapsRefresh function
>   virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBuffer
>   virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBinary
>   virfirewall: use virFindFileInPath instead of virFileIsExecutable
>   tests: introduce virfirewallmock
>   tests: use virfirewallmock instead of hasNetfilterTools
>   virnetdev: move virNetDevSetRootQDisc to virnetdevbandwidth
>   tests: testutilsqemu: move virFindFileInPath into domaincapsmock
>   meson: don't check collie as program for sheepdog
>   bhyvexml2argvtest: use virCommandToStringFull to strip command path
>   storage: use virFindFileInPath to validate presence of mkfs
>   virfile: introduce virFindFileInPathFull()
>   qemu_conf: use virFindFileInPathFull for runtime binaries
>   meson: drop check for runtime binary dependencies
>   meson: move iscsiadm check into storage_iscsi condition
>   meson: stop setting runtime binaries defines during compilation
>   meson: use runtime binaries to only resolve features with "auto" value
>   meson: optional_programs should be used only for building libvirt
>   libvirt.spec: drop no longer required build dependencies
>
>  libvirt.spec.in                               |  31 ---
>  meson.build                                   | 214 ++++++------------
>  src/bhyve/bhyve_command.c                     |   4 +
>  src/libvirt_private.syms                      |   6 +-
>  src/locking/lock_driver_lockd.c               |  12 +-
>  src/network/bridge_driver.c                   |   8 +-
>  src/node_device/node_device_driver.c          |   2 +
>  src/qemu/qemu_conf.c                          |  23 +-
>  src/qemu/qemu_domain.c                        |   3 +-
>  src/storage/storage_backend_fs.c              |  24 +-
>  src/storage/storage_backend_logical.c         |  13 ++
>  src/storage/storage_backend_sheepdog.c        |   2 +
>  src/storage/storage_backend_zfs.c             |   3 +
>  src/storage/storage_util.c                    |   2 +
>  src/storage/storage_util.h                    |   6 +
>  src/util/virdnsmasq.c                         |  56 +----
>  src/util/virdnsmasq.h                         |   8 +-
>  src/util/virfile.c                            |  16 +-
>  src/util/virfile.h                            |   6 +-
>  src/util/virfirewall.c                        |   4 +-
>  src/util/virfirewall.h                        |   4 +
>  src/util/viriscsi.h                           |   2 +
>  src/util/virkmod.h                            |   3 +
>  src/util/virnetdev.c                          |  46 ----
>  src/util/virnetdev.h                          |   4 -
>  src/util/virnetdevbandwidth.c                 |  50 ++++
>  src/util/virnetdevbandwidth.h                 |   6 +
>  src/util/virnetdevip.c                        |   2 +
>  src/util/virnetdevmidonet.c                   |   2 +
>  src/util/virnetdevopenvswitch.c               |   2 +
>  src/util/virnuma.c                            |   1 +
>  src/util/virsysinfo.c                         |   1 +
>  src/util/virutil.c                            |   2 +
>  .../bhyvexml2argv-acpiapic.args               |   2 +-
>  .../bhyvexml2argv-acpiapic.ldargs             |   2 +-
>  ...ml2argv-addr-isa-controller-on-slot-1.args |   2 +-
>  ...2argv-addr-isa-controller-on-slot-1.ldargs |   2 +-
>  ...l2argv-addr-isa-controller-on-slot-31.args |   2 +-
>  ...argv-addr-isa-controller-on-slot-31.ldargs |   2 +-
>  ...xml2argv-addr-more-than-32-sata-disks.args |   2 +-
>  ...l2argv-addr-more-than-32-sata-disks.ldargs |   2 +-
>  ...hyvexml2argv-addr-multiple-sata-disks.args |   2 +-
>  ...vexml2argv-addr-multiple-sata-disks.ldargs |   2 +-
>  ...vexml2argv-addr-multiple-virtio-disks.args |   2 +-
>  ...xml2argv-addr-multiple-virtio-disks.ldargs |   2 +-
>  ...rgv-addr-no32devs-multiple-sata-disks.args |   2 +-
>  ...v-addr-no32devs-multiple-sata-disks.ldargs |   2 +-
>  ...l2argv-addr-no32devs-single-sata-disk.args |   2 +-
>  ...argv-addr-no32devs-single-sata-disk.ldargs |   2 +-
>  ...rgv-addr-non-isa-controller-on-slot-1.args |   2 +-
>  .../bhyvexml2argv-addr-single-sata-disk.args  |   2 +-
>  ...bhyvexml2argv-addr-single-sata-disk.ldargs |   2 +-
>  ...bhyvexml2argv-addr-single-virtio-disk.args |   2 +-
>  ...yvexml2argv-addr-single-virtio-disk.ldargs |   2 +-
>  .../bhyvexml2argvdata/bhyvexml2argv-base.args |   2 +-
>  .../bhyvexml2argv-base.ldargs                 |   2 +-
>  .../bhyvexml2argv-bhyveload-bootorder.args    |   2 +-
>  .../bhyvexml2argv-bhyveload-bootorder.ldargs  |   2 +-
>  .../bhyvexml2argv-bhyveload-bootorder1.args   |   2 +-
>  .../bhyvexml2argv-bhyveload-bootorder1.ldargs |   2 +-
>  .../bhyvexml2argv-bhyveload-bootorder3.args   |   2 +-
>  .../bhyvexml2argv-bhyveload-bootorder3.ldargs |   2 +-
>  .../bhyvexml2argv-bhyveload-explicitargs.args |   2 +-
>  ...hyvexml2argv-bhyveload-explicitargs.ldargs |   2 +-
>  .../bhyvexml2argv-commandline.args            |   2 +-
>  .../bhyvexml2argv-commandline.ldargs          |   2 +-
>  ...gv-console-master-slave-not-specified.args |   2 +-
>  ...-console-master-slave-not-specified.ldargs |   2 +-
>  .../bhyvexml2argv-console.args                |   2 +-
>  .../bhyvexml2argv-console.ldargs              |   2 +-
>  .../bhyvexml2argv-cputopology.args            |   2 +-
>  .../bhyvexml2argv-cputopology.ldargs          |   2 +-
>  .../bhyvexml2argv-custom-loader.args          |   2 +-
>  .../bhyvexml2argv-custom-loader.ldargs        |   2 +-
>  .../bhyvexml2argv-disk-cdrom-grub.args        |   2 +-
>  .../bhyvexml2argv-disk-cdrom-grub.ldargs      |   2 +-
>  .../bhyvexml2argv-disk-cdrom.args             |   2 +-
>  .../bhyvexml2argv-disk-cdrom.ldargs           |   2 +-
>  .../bhyvexml2argv-disk-virtio.args            |   2 +-
>  .../bhyvexml2argv-disk-virtio.ldargs          |   2 +-
>  .../bhyvexml2argv-firmware-efi.args           |   2 +-
>  .../bhyvexml2argv-fs-9p-readonly.args         |   2 +-
>  .../bhyvexml2argv-fs-9p-readonly.ldargs       |   2 +-
>  .../bhyvexml2argv-fs-9p.args                  |   2 +-
>  .../bhyvexml2argv-fs-9p.ldargs                |   2 +-
>  .../bhyvexml2argv-grub-bootorder.args         |   2 +-
>  .../bhyvexml2argv-grub-bootorder.ldargs       |   2 +-
>  .../bhyvexml2argv-grub-bootorder2.args        |   2 +-
>  .../bhyvexml2argv-grub-bootorder2.ldargs      |   2 +-
>  .../bhyvexml2argv-grub-defaults.args          |   2 +-
>  .../bhyvexml2argv-grub-defaults.ldargs        |   2 +-
>  .../bhyvexml2argv-input-xhci-tablet.args      |   2 +-
>  .../bhyvexml2argv-input-xhci-tablet.ldargs    |   2 +-
>  .../bhyvexml2argv-isa-controller.args         |   2 +-
>  .../bhyvexml2argv-isa-controller.ldargs       |   2 +-
>  .../bhyvexml2argv-localtime.args              |   2 +-
>  .../bhyvexml2argv-localtime.ldargs            |   2 +-
>  .../bhyvexml2argv-macaddr.args                |   2 +-
>  .../bhyvexml2argv-macaddr.ldargs              |   2 +-
>  .../bhyvexml2argvdata/bhyvexml2argv-msrs.args |   2 +-
>  .../bhyvexml2argv-msrs.ldargs                 |   2 +-
>  .../bhyvexml2argv-net-e1000.args              |   2 +-
>  .../bhyvexml2argv-net-e1000.ldargs            |   2 +-
>  .../bhyvexml2argv-serial-grub-nocons.args     |   2 +-
>  .../bhyvexml2argv-serial-grub-nocons.ldargs   |   2 +-
>  .../bhyvexml2argv-serial-grub.args            |   2 +-
>  .../bhyvexml2argv-serial-grub.ldargs          |   2 +-
>  .../bhyvexml2argv-serial.args                 |   2 +-
>  .../bhyvexml2argv-serial.ldargs               |   2 +-
>  .../bhyvexml2argv-sound.args                  |   2 +-
>  .../bhyvexml2argv-sound.ldargs                |   2 +-
>  .../bhyvexml2argvdata/bhyvexml2argv-uefi.args |   2 +-
>  .../bhyvexml2argv-vnc-autoport.args           |   2 +-
>  .../bhyvexml2argv-vnc-password.args           |   2 +-
>  .../bhyvexml2argv-vnc-resolution.args         |   2 +-
>  .../bhyvexml2argv-vnc-vgaconf-io.args         |   2 +-
>  .../bhyvexml2argv-vnc-vgaconf-off.args        |   2 +-
>  .../bhyvexml2argv-vnc-vgaconf-on.args         |   2 +-
>  .../bhyvexml2argvdata/bhyvexml2argv-vnc.args  |   2 +-
>  .../bhyvexml2argv-wired.args                  |   2 +-
>  .../bhyvexml2argv-wired.ldargs                |   2 +-
>  tests/bhyvexml2argvtest.c                     |   4 +-
>  tests/domaincapsmock.c                        |  17 ++
>  tests/meson.build                             |   1 +
>  tests/networkxml2conftest.c                   |   6 +-
>  tests/networkxml2firewalltest.c               |  16 +-
>  tests/nwfilterebiptablestest.c                |  15 +-
>  tests/nwfilterxml2firewalltest.c              |  14 +-
>  tests/qemuxml2argvmock.c                      |   5 +-
>  tests/testutilsqemu.c                         |  15 --
>  tests/virfirewallmock.c                       |  35 +++
>  tests/virfirewalltest.c                       |  15 +-
>  132 files changed, 403 insertions(+), 484 deletions(-)
>  create mode 100644 tests/virfirewallmock.c
>
> --
> 2.30.2
>

In case my reply on the v1 thread is missed, I'll say it again here,
since it still applies here.

So, I think you *shouldn't* do it that way. The problem with doing
this is that we can wind up with mismatched capabilities and
non-functional libvirt build based on the assumption that things will
just "be there" with no check that they will actually be there.

In other words, runtime executable dependencies should be treated
*exactly* as they are now, because we have no other avenue to
guarantee that things work for a given installation.

The idea that binaries would randomly get replaced because of
differences in system and session could also lead to other unexpected
issues, but I think those are considerably less likely. I don't think
Daniel's idea of this being supported is a good one though. It's
remarkably easy to accidentally break things that way, and since GNOME
Boxes is a very prominent user of the session connection, I'd rather
not see that get broken by accident.

To make it clear, I still NACK the series, because I think this is
fundamentally a bad idea.


--
真実はいつも一つ!/ Always, there's only one truth!


Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Pavel Hrdina 3 years ago
On Tue, Apr 20, 2021 at 05:30:09AM -0400, Neal Gompa wrote:
> On Mon, Apr 19, 2021 at 1:14 PM Pavel Hrdina <phrdina@redhat.com> wrote:
> >
> > Recent attempt to add a lot of meson options to specify different
> > runtime paths motivated me enough to cleanup this from meson.
> >
> > Changes in v2:
> >     - split and rework patch 16/17 to address review comments
> >     - added a new patch to cleanup libvirt.spec.in file
> >
> > Pavel Hrdina (21):
> >   bridge_driver: fix comment about dnsmasqCaps
> >   virdnsmasq: drop unused dnsmasqCapsNewFromFile function
> >   virdnsmasq: drop unused dnsmasqCapsRefresh function
> >   virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBuffer
> >   virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBinary
> >   virfirewall: use virFindFileInPath instead of virFileIsExecutable
> >   tests: introduce virfirewallmock
> >   tests: use virfirewallmock instead of hasNetfilterTools
> >   virnetdev: move virNetDevSetRootQDisc to virnetdevbandwidth
> >   tests: testutilsqemu: move virFindFileInPath into domaincapsmock
> >   meson: don't check collie as program for sheepdog
> >   bhyvexml2argvtest: use virCommandToStringFull to strip command path
> >   storage: use virFindFileInPath to validate presence of mkfs
> >   virfile: introduce virFindFileInPathFull()
> >   qemu_conf: use virFindFileInPathFull for runtime binaries
> >   meson: drop check for runtime binary dependencies
> >   meson: move iscsiadm check into storage_iscsi condition
> >   meson: stop setting runtime binaries defines during compilation
> >   meson: use runtime binaries to only resolve features with "auto" value
> >   meson: optional_programs should be used only for building libvirt
> >   libvirt.spec: drop no longer required build dependencies
> >
> >  libvirt.spec.in                               |  31 ---
> >  meson.build                                   | 214 ++++++------------
> >  src/bhyve/bhyve_command.c                     |   4 +
> >  src/libvirt_private.syms                      |   6 +-
> >  src/locking/lock_driver_lockd.c               |  12 +-
> >  src/network/bridge_driver.c                   |   8 +-
> >  src/node_device/node_device_driver.c          |   2 +
> >  src/qemu/qemu_conf.c                          |  23 +-
> >  src/qemu/qemu_domain.c                        |   3 +-
> >  src/storage/storage_backend_fs.c              |  24 +-
> >  src/storage/storage_backend_logical.c         |  13 ++
> >  src/storage/storage_backend_sheepdog.c        |   2 +
> >  src/storage/storage_backend_zfs.c             |   3 +
> >  src/storage/storage_util.c                    |   2 +
> >  src/storage/storage_util.h                    |   6 +
> >  src/util/virdnsmasq.c                         |  56 +----
> >  src/util/virdnsmasq.h                         |   8 +-
> >  src/util/virfile.c                            |  16 +-
> >  src/util/virfile.h                            |   6 +-
> >  src/util/virfirewall.c                        |   4 +-
> >  src/util/virfirewall.h                        |   4 +
> >  src/util/viriscsi.h                           |   2 +
> >  src/util/virkmod.h                            |   3 +
> >  src/util/virnetdev.c                          |  46 ----
> >  src/util/virnetdev.h                          |   4 -
> >  src/util/virnetdevbandwidth.c                 |  50 ++++
> >  src/util/virnetdevbandwidth.h                 |   6 +
> >  src/util/virnetdevip.c                        |   2 +
> >  src/util/virnetdevmidonet.c                   |   2 +
> >  src/util/virnetdevopenvswitch.c               |   2 +
> >  src/util/virnuma.c                            |   1 +
> >  src/util/virsysinfo.c                         |   1 +
> >  src/util/virutil.c                            |   2 +
> >  .../bhyvexml2argv-acpiapic.args               |   2 +-
> >  .../bhyvexml2argv-acpiapic.ldargs             |   2 +-
> >  ...ml2argv-addr-isa-controller-on-slot-1.args |   2 +-
> >  ...2argv-addr-isa-controller-on-slot-1.ldargs |   2 +-
> >  ...l2argv-addr-isa-controller-on-slot-31.args |   2 +-
> >  ...argv-addr-isa-controller-on-slot-31.ldargs |   2 +-
> >  ...xml2argv-addr-more-than-32-sata-disks.args |   2 +-
> >  ...l2argv-addr-more-than-32-sata-disks.ldargs |   2 +-
> >  ...hyvexml2argv-addr-multiple-sata-disks.args |   2 +-
> >  ...vexml2argv-addr-multiple-sata-disks.ldargs |   2 +-
> >  ...vexml2argv-addr-multiple-virtio-disks.args |   2 +-
> >  ...xml2argv-addr-multiple-virtio-disks.ldargs |   2 +-
> >  ...rgv-addr-no32devs-multiple-sata-disks.args |   2 +-
> >  ...v-addr-no32devs-multiple-sata-disks.ldargs |   2 +-
> >  ...l2argv-addr-no32devs-single-sata-disk.args |   2 +-
> >  ...argv-addr-no32devs-single-sata-disk.ldargs |   2 +-
> >  ...rgv-addr-non-isa-controller-on-slot-1.args |   2 +-
> >  .../bhyvexml2argv-addr-single-sata-disk.args  |   2 +-
> >  ...bhyvexml2argv-addr-single-sata-disk.ldargs |   2 +-
> >  ...bhyvexml2argv-addr-single-virtio-disk.args |   2 +-
> >  ...yvexml2argv-addr-single-virtio-disk.ldargs |   2 +-
> >  .../bhyvexml2argvdata/bhyvexml2argv-base.args |   2 +-
> >  .../bhyvexml2argv-base.ldargs                 |   2 +-
> >  .../bhyvexml2argv-bhyveload-bootorder.args    |   2 +-
> >  .../bhyvexml2argv-bhyveload-bootorder.ldargs  |   2 +-
> >  .../bhyvexml2argv-bhyveload-bootorder1.args   |   2 +-
> >  .../bhyvexml2argv-bhyveload-bootorder1.ldargs |   2 +-
> >  .../bhyvexml2argv-bhyveload-bootorder3.args   |   2 +-
> >  .../bhyvexml2argv-bhyveload-bootorder3.ldargs |   2 +-
> >  .../bhyvexml2argv-bhyveload-explicitargs.args |   2 +-
> >  ...hyvexml2argv-bhyveload-explicitargs.ldargs |   2 +-
> >  .../bhyvexml2argv-commandline.args            |   2 +-
> >  .../bhyvexml2argv-commandline.ldargs          |   2 +-
> >  ...gv-console-master-slave-not-specified.args |   2 +-
> >  ...-console-master-slave-not-specified.ldargs |   2 +-
> >  .../bhyvexml2argv-console.args                |   2 +-
> >  .../bhyvexml2argv-console.ldargs              |   2 +-
> >  .../bhyvexml2argv-cputopology.args            |   2 +-
> >  .../bhyvexml2argv-cputopology.ldargs          |   2 +-
> >  .../bhyvexml2argv-custom-loader.args          |   2 +-
> >  .../bhyvexml2argv-custom-loader.ldargs        |   2 +-
> >  .../bhyvexml2argv-disk-cdrom-grub.args        |   2 +-
> >  .../bhyvexml2argv-disk-cdrom-grub.ldargs      |   2 +-
> >  .../bhyvexml2argv-disk-cdrom.args             |   2 +-
> >  .../bhyvexml2argv-disk-cdrom.ldargs           |   2 +-
> >  .../bhyvexml2argv-disk-virtio.args            |   2 +-
> >  .../bhyvexml2argv-disk-virtio.ldargs          |   2 +-
> >  .../bhyvexml2argv-firmware-efi.args           |   2 +-
> >  .../bhyvexml2argv-fs-9p-readonly.args         |   2 +-
> >  .../bhyvexml2argv-fs-9p-readonly.ldargs       |   2 +-
> >  .../bhyvexml2argv-fs-9p.args                  |   2 +-
> >  .../bhyvexml2argv-fs-9p.ldargs                |   2 +-
> >  .../bhyvexml2argv-grub-bootorder.args         |   2 +-
> >  .../bhyvexml2argv-grub-bootorder.ldargs       |   2 +-
> >  .../bhyvexml2argv-grub-bootorder2.args        |   2 +-
> >  .../bhyvexml2argv-grub-bootorder2.ldargs      |   2 +-
> >  .../bhyvexml2argv-grub-defaults.args          |   2 +-
> >  .../bhyvexml2argv-grub-defaults.ldargs        |   2 +-
> >  .../bhyvexml2argv-input-xhci-tablet.args      |   2 +-
> >  .../bhyvexml2argv-input-xhci-tablet.ldargs    |   2 +-
> >  .../bhyvexml2argv-isa-controller.args         |   2 +-
> >  .../bhyvexml2argv-isa-controller.ldargs       |   2 +-
> >  .../bhyvexml2argv-localtime.args              |   2 +-
> >  .../bhyvexml2argv-localtime.ldargs            |   2 +-
> >  .../bhyvexml2argv-macaddr.args                |   2 +-
> >  .../bhyvexml2argv-macaddr.ldargs              |   2 +-
> >  .../bhyvexml2argvdata/bhyvexml2argv-msrs.args |   2 +-
> >  .../bhyvexml2argv-msrs.ldargs                 |   2 +-
> >  .../bhyvexml2argv-net-e1000.args              |   2 +-
> >  .../bhyvexml2argv-net-e1000.ldargs            |   2 +-
> >  .../bhyvexml2argv-serial-grub-nocons.args     |   2 +-
> >  .../bhyvexml2argv-serial-grub-nocons.ldargs   |   2 +-
> >  .../bhyvexml2argv-serial-grub.args            |   2 +-
> >  .../bhyvexml2argv-serial-grub.ldargs          |   2 +-
> >  .../bhyvexml2argv-serial.args                 |   2 +-
> >  .../bhyvexml2argv-serial.ldargs               |   2 +-
> >  .../bhyvexml2argv-sound.args                  |   2 +-
> >  .../bhyvexml2argv-sound.ldargs                |   2 +-
> >  .../bhyvexml2argvdata/bhyvexml2argv-uefi.args |   2 +-
> >  .../bhyvexml2argv-vnc-autoport.args           |   2 +-
> >  .../bhyvexml2argv-vnc-password.args           |   2 +-
> >  .../bhyvexml2argv-vnc-resolution.args         |   2 +-
> >  .../bhyvexml2argv-vnc-vgaconf-io.args         |   2 +-
> >  .../bhyvexml2argv-vnc-vgaconf-off.args        |   2 +-
> >  .../bhyvexml2argv-vnc-vgaconf-on.args         |   2 +-
> >  .../bhyvexml2argvdata/bhyvexml2argv-vnc.args  |   2 +-
> >  .../bhyvexml2argv-wired.args                  |   2 +-
> >  .../bhyvexml2argv-wired.ldargs                |   2 +-
> >  tests/bhyvexml2argvtest.c                     |   4 +-
> >  tests/domaincapsmock.c                        |  17 ++
> >  tests/meson.build                             |   1 +
> >  tests/networkxml2conftest.c                   |   6 +-
> >  tests/networkxml2firewalltest.c               |  16 +-
> >  tests/nwfilterebiptablestest.c                |  15 +-
> >  tests/nwfilterxml2firewalltest.c              |  14 +-
> >  tests/qemuxml2argvmock.c                      |   5 +-
> >  tests/testutilsqemu.c                         |  15 --
> >  tests/virfirewallmock.c                       |  35 +++
> >  tests/virfirewalltest.c                       |  15 +-
> >  132 files changed, 403 insertions(+), 484 deletions(-)
> >  create mode 100644 tests/virfirewallmock.c
> >
> > --
> > 2.30.2
> >
> 
> In case my reply on the v1 thread is missed, I'll say it again here,
> since it still applies here.
> 
> So, I think you *shouldn't* do it that way. The problem with doing
> this is that we can wind up with mismatched capabilities and
> non-functional libvirt build based on the assumption that things will
> just "be there" with no check that they will actually be there.
>
> In other words, runtime executable dependencies should be treated
> *exactly* as they are now, because we have no other avenue to
> guarantee that things work for a given installation.

I disagree, let me explain why using several possible scenarios:

1) libvirt is compiled by package maintainers to be shipped in some
   distribution

   In this case it is responsibility of the package maintainer to ensure
   that the build dependencies and runtime dependencies are correctly
   defined. Users usually don't have an option to remove any dependency
   without removing the dependent package as well so the assumption that
   things will just "be there" is correct.

2) libvirt is compiled directly by user from git/tarball

   In this situation there is nothing ensuring the correct dependencies
   are installed for compilation or runtime so everything can breake
   with or without this series. For example without this series users
   can easily install the necessary binaries, compile libvirt and
   uninstall the binaries again which would result in non-functional
   libvirt features using these binaries.

   Another point for this series is that by default (keeping all libvirt
   defined meson features set to "auto") will behave exactly the same as
   before this series. The difference comes only once user explicitly
   enables some libvirt feature without required binaries.

The following list of binaries was always optional and if missing in the
system only the name was defined so libvirt would always figure out the
full path during runtime:

    dmidecode
    dnsmasq
    ebtables
    ip
    ip6tables
    iptables
    mdevctl
    mm-ctl
    modprobe
    ovs-vsctl
    radvd
    rmmod
    scrub
    tc
    udevadm

    qemu-bridge-helper
    qemu-pr-helper
    slirp-helper
    dbus-daemon

    showmount

Before this series if these are available during compilation libvirt
would set the full path to be used during runtime but if they are not
available it would silently use only the names. After the series it will
always use only the names. So the possibility to end up with
non-functional libvirt is the same as libvirt assumes that things will
just "be there" if they are not during compilation.

For remaining binaries affected by this series we change the behavior
to not error out if they are missing but as I've already pointed out we
keep the default behavior to not enable the libvirt features requiring
these binaries.

    bhyve, bhyveclt, bhyveload (driver_bhyve)

    parted (storage_disk)
    mount, umount, mkfs (storage_fs)
    lvm aliases (storage_lvm)
    dog (storage_sheepdog)
    zfs, zpool (storage_zfs)

    numad (numad)

Here we will use the same behavior as the previous list of optional
binaries.

> The idea that binaries would randomly get replaced because of
> differences in system and session could also lead to other unexpected
> issues, but I think those are considerably less likely. I don't think
> Daniel's idea of this being supported is a good one though. It's
> remarkably easy to accidentally break things that way, and since GNOME
> Boxes is a very prominent user of the session connection, I'd rather
> not see that get broken by accident.

Looking at the list of binaries that will be affected by this series
I don't see any that would be used by GNOME Boxes.

> To make it clear, I still NACK the series, because I think this is
> fundamentally a bad idea.

It was already pointed out that the series have other patches fixing
issues not related to the binary path detection, NACKing the whole
series is ignoring the fact.

To summarize it I don't see any strong reason why this series shouldn't
be accepted.

Pavel
Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Daniel P. Berrangé 3 years ago
On Tue, Apr 20, 2021 at 12:50:51PM +0200, Pavel Hrdina wrote:
> On Tue, Apr 20, 2021 at 05:30:09AM -0400, Neal Gompa wrote:
> > On Mon, Apr 19, 2021 at 1:14 PM Pavel Hrdina <phrdina@redhat.com> wrote:
> > >
> > > Recent attempt to add a lot of meson options to specify different
> > > runtime paths motivated me enough to cleanup this from meson.
> > >
> > > Changes in v2:
> > >     - split and rework patch 16/17 to address review comments
> > >     - added a new patch to cleanup libvirt.spec.in file
> > >
> > > Pavel Hrdina (21):
> > >   bridge_driver: fix comment about dnsmasqCaps
> > >   virdnsmasq: drop unused dnsmasqCapsNewFromFile function
> > >   virdnsmasq: drop unused dnsmasqCapsRefresh function
> > >   virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBuffer
> > >   virdnsmasq: remove binaryPath argument from dnsmasqCapsNewFromBinary
> > >   virfirewall: use virFindFileInPath instead of virFileIsExecutable
> > >   tests: introduce virfirewallmock
> > >   tests: use virfirewallmock instead of hasNetfilterTools
> > >   virnetdev: move virNetDevSetRootQDisc to virnetdevbandwidth
> > >   tests: testutilsqemu: move virFindFileInPath into domaincapsmock
> > >   meson: don't check collie as program for sheepdog
> > >   bhyvexml2argvtest: use virCommandToStringFull to strip command path
> > >   storage: use virFindFileInPath to validate presence of mkfs
> > >   virfile: introduce virFindFileInPathFull()
> > >   qemu_conf: use virFindFileInPathFull for runtime binaries
> > >   meson: drop check for runtime binary dependencies
> > >   meson: move iscsiadm check into storage_iscsi condition
> > >   meson: stop setting runtime binaries defines during compilation
> > >   meson: use runtime binaries to only resolve features with "auto" value
> > >   meson: optional_programs should be used only for building libvirt
> > >   libvirt.spec: drop no longer required build dependencies


> > >
> > > --
> > > 2.30.2
> > >
> > 
> > In case my reply on the v1 thread is missed, I'll say it again here,
> > since it still applies here.
> > 
> > So, I think you *shouldn't* do it that way. The problem with doing
> > this is that we can wind up with mismatched capabilities and
> > non-functional libvirt build based on the assumption that things will
> > just "be there" with no check that they will actually be there.
> >
> > In other words, runtime executable dependencies should be treated
> > *exactly* as they are now, because we have no other avenue to
> > guarantee that things work for a given installation.
> 
> I disagree, let me explain why using several possible scenarios:
> 
> 1) libvirt is compiled by package maintainers to be shipped in some
>    distribution
> 
>    In this case it is responsibility of the package maintainer to ensure
>    that the build dependencies and runtime dependencies are correctly
>    defined. Users usually don't have an option to remove any dependency
>    without removing the dependent package as well so the assumption that
>    things will just "be there" is correct.
> 
> 2) libvirt is compiled directly by user from git/tarball
> 
>    In this situation there is nothing ensuring the correct dependencies
>    are installed for compilation or runtime so everything can breake
>    with or without this series. For example without this series users
>    can easily install the necessary binaries, compile libvirt and
>    uninstall the binaries again which would result in non-functional
>    libvirt features using these binaries.
> 
>    Another point for this series is that by default (keeping all libvirt
>    defined meson features set to "auto") will behave exactly the same as
>    before this series. The difference comes only once user explicitly
>    enables some libvirt feature without required binaries.
> 
> The following list of binaries was always optional and if missing in the
> system only the name was defined so libvirt would always figure out the
> full path during runtime:
> 
>     dmidecode
>     dnsmasq
>     ebtables
>     ip
>     ip6tables
>     iptables
>     mdevctl
>     mm-ctl
>     modprobe
>     ovs-vsctl
>     radvd
>     rmmod
>     scrub
>     tc
>     udevadm
> 
>     qemu-bridge-helper
>     qemu-pr-helper
>     slirp-helper
>     dbus-daemon
> 
>     showmount
> 
> Before this series if these are available during compilation libvirt
> would set the full path to be used during runtime but if they are not
> available it would silently use only the names. After the series it will
> always use only the names. So the possibility to end up with
> non-functional libvirt is the same as libvirt assumes that things will
> just "be there" if they are not during compilation.
> 
> For remaining binaries affected by this series we change the behavior
> to not error out if they are missing but as I've already pointed out we
> keep the default behavior to not enable the libvirt features requiring
> these binaries.
> 
>     bhyve, bhyveclt, bhyveload (driver_bhyve)
> 
>     parted (storage_disk)
>     mount, umount, mkfs (storage_fs)
>     lvm aliases (storage_lvm)
>     dog (storage_sheepdog)
>     zfs, zpool (storage_zfs)
> 
>     numad (numad)
> 
> Here we will use the same behavior as the previous list of optional
> binaries.

I would note that over the years users have fairly frequent hit problems
where they didn't have a binary present during build, but installed it
later, and then complained to us that libvirt didn't use it. This is
why alot of the binaries you list in the previous set we changed to
be optional at configure time, relying on $PATH at runtime.


> > The idea that binaries would randomly get replaced because of
> > differences in system and session could also lead to other unexpected
> > issues, but I think those are considerably less likely. I don't think
> > Daniel's idea of this being supported is a good one though. It's
> > remarkably easy to accidentally break things that way, and since GNOME
> > Boxes is a very prominent user of the session connection, I'd rather
> > not see that get broken by accident.
> 
> Looking at the list of binaries that will be affected by this series
> I don't see any that would be used by GNOME Boxes.

Even if it does though, I don't think this is  a reason not to do the
change. If an unprivileged user is replacing system binaries then it
is their responsiblity if their replacements break stuff. I think this
is not likely to be a significant problem in the real world though.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Andrea Bolognani 3 years ago
On Mon, 2021-04-19 at 19:14 +0200, Pavel Hrdina wrote:
> Recent attempt to add a lot of meson options to specify different
> runtime paths motivated me enough to cleanup this from meson.
> 
> Changes in v2:
>     - split and rework patch 16/17 to address review comments
>     - added a new patch to cleanup libvirt.spec.in file

Overall I like this change, if nothing else because it finally cleans
up the mess where some optional programs would be detected at build
time while others would be detected at runtime, with no apparent
reason for going one way rather than the other.

The only concern I have, which is one I have expressed already in the
past, is that it will now be harder for users and packagers alike to
figure out that they don't have a certain optional program installed:
AFAICT, after your patches the only way would be to try using each
feature and look out for errors.

Can we still check for the availability of these commands at build
time and simply report their presence? I'm thinking of something like

  optional_runtime_programs = [
    'dmidecode',
    'dnsmasq',
    ...
  ]

which would be reported in the Meson summary as

  Optional runtime programs:
            dmidecode: YES
              dnsmasq: NO
              ...

Then, for each of these commands, we'd also create a macro
definition in meson-config.h like

  #define PROGRAM_DMIDECODE "dmidecode"
  #define PROGRAM_DNSMASQ "dnsmasq"
  ...

and use those in the various C functions instead of the ad-hoc,
private defines that are currently sprinkled throughout the code.

Note that said macro would *not* be defined conditionally based on
whether the optional program was found at build time: the intention
is simply to have a single location where information about all these
optional runtime programs are stored, instead of forcing users and
packagers to go hunting for them across all of libvirt, potentially
missing a bunch in the process.

What do you think?

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Daniel P. Berrangé 3 years ago
On Tue, Apr 20, 2021 at 03:05:26PM +0200, Andrea Bolognani wrote:
> On Mon, 2021-04-19 at 19:14 +0200, Pavel Hrdina wrote:
> > Recent attempt to add a lot of meson options to specify different
> > runtime paths motivated me enough to cleanup this from meson.
> > 
> > Changes in v2:
> >     - split and rework patch 16/17 to address review comments
> >     - added a new patch to cleanup libvirt.spec.in file
> 
> Overall I like this change, if nothing else because it finally cleans
> up the mess where some optional programs would be detected at build
> time while others would be detected at runtime, with no apparent
> reason for going one way rather than the other.
> 
> The only concern I have, which is one I have expressed already in the
> past, is that it will now be harder for users and packagers alike to
> figure out that they don't have a certain optional program installed:
> AFAICT, after your patches the only way would be to try using each
> feature and look out for errors.

I'd say that running "meson" and looking at its output is a poor
way for users to learn what the pre-requisites are, both before
and after this change.

Ideally we should document the list of packages we depend on.
We have it sorta documented in the many docker files.

I wonder if we should include the list of mapping names from
libvirt-ci as an explicit doc item in README.md or somewhere
else that could be useful. Or just tell tem to look at the
dockerfiles. Even if their distro isn't covered, it'll be
enough info for them to get most of the way there.

> Note that said macro would *not* be defined conditionally based on
> whether the optional program was found at build time: the intention
> is simply to have a single location where information about all these
> optional runtime programs are stored, instead of forcing users and
> packagers to go hunting for them across all of libvirt, potentially
> missing a bunch in the process.
> 
> What do you think?
> 
> -- 
> Andrea Bolognani / Red Hat / Virtualization
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Pavel Hrdina 3 years ago
On Tue, Apr 20, 2021 at 02:14:59PM +0100, Daniel P. Berrangé wrote:
> On Tue, Apr 20, 2021 at 03:05:26PM +0200, Andrea Bolognani wrote:
> > On Mon, 2021-04-19 at 19:14 +0200, Pavel Hrdina wrote:
> > > Recent attempt to add a lot of meson options to specify different
> > > runtime paths motivated me enough to cleanup this from meson.
> > > 
> > > Changes in v2:
> > >     - split and rework patch 16/17 to address review comments
> > >     - added a new patch to cleanup libvirt.spec.in file
> > 
> > Overall I like this change, if nothing else because it finally cleans
> > up the mess where some optional programs would be detected at build
> > time while others would be detected at runtime, with no apparent
> > reason for going one way rather than the other.
> > 
> > The only concern I have, which is one I have expressed already in the
> > past, is that it will now be harder for users and packagers alike to
> > figure out that they don't have a certain optional program installed:
> > AFAICT, after your patches the only way would be to try using each
> > feature and look out for errors.
> 
> I'd say that running "meson" and looking at its output is a poor
> way for users to learn what the pre-requisites are, both before
> and after this change.
> 
> Ideally we should document the list of packages we depend on.
> We have it sorta documented in the many docker files.
> 
> I wonder if we should include the list of mapping names from
> libvirt-ci as an explicit doc item in README.md or somewhere
> else that could be useful. Or just tell tem to look at the
> dockerfiles. Even if their distro isn't covered, it'll be
> enough info for them to get most of the way there.

Agreed, instead of having it in meson I would also prefer having it
documented somewhere. Ideally we should not document only what optional
programs we use but every single dependency we use ideally with links to
upstream projects and possibly examples of downstream packages which is
as Dan pointed out stored in libvirt-ci.

Pavel

> > Note that said macro would *not* be defined conditionally based on
> > whether the optional program was found at build time: the intention
> > is simply to have a single location where information about all these
> > optional runtime programs are stored, instead of forcing users and
> > packagers to go hunting for them across all of libvirt, potentially
> > missing a bunch in the process.
> > 
> > What do you think?
> > 
> > -- 
> > Andrea Bolognani / Red Hat / Virtualization
> > 
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
> 
Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Andrea Bolognani 3 years ago
On Tue, 2021-04-20 at 15:20 +0200, Pavel Hrdina wrote:
> On Tue, Apr 20, 2021 at 02:14:59PM +0100, Daniel P. Berrangé wrote:
> > On Tue, Apr 20, 2021 at 03:05:26PM +0200, Andrea Bolognani wrote:
> > > The only concern I have, which is one I have expressed already in the
> > > past, is that it will now be harder for users and packagers alike to
> > > figure out that they don't have a certain optional program installed:
> > > AFAICT, after your patches the only way would be to try using each
> > > feature and look out for errors.
> > 
> > I'd say that running "meson" and looking at its output is a poor
> > way for users to learn what the pre-requisites are, both before
> > and after this change.
> > 
> > Ideally we should document the list of packages we depend on.
> > We have it sorta documented in the many docker files.
> > 
> > I wonder if we should include the list of mapping names from
> > libvirt-ci as an explicit doc item in README.md or somewhere
> > else that could be useful. Or just tell tem to look at the
> > dockerfiles. Even if their distro isn't covered, it'll be
> > enough info for them to get most of the way there.
> 
> Agreed, instead of having it in meson I would also prefer having it
> documented somewhere. Ideally we should not document only what optional
> programs we use but every single dependency we use ideally with links to
> upstream projects and possibly examples of downstream packages which is
> as Dan pointed out stored in libvirt-ci.

Documentation is great and all, but the problem with it is that it's
extremely easy for it to become outdated as new optional runtime
dependencies are added; the advantage of having this list stored in
Meson is that the person who's adding a new optional runtime
dependency is likely to look around for how the existing ones are
handled, and thus automatically do the right thing.

The point about libvirt-ci storing all the dependencies is a good
one, but note that we only keep track of *build time* dependencies
for projects: that is, once e.g. dnsmasq is no longer required to be
available in the build environment in order to produce a
fully-featured libvirt build, we'll simply stop including it. This
will happen with almost all other optional runtime dependencies too,
so really the mappings that are maintained as part of the libvirt-ci
project doesn't solve the issue of keeping track of optional runtime
dependencies at all.

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH v2 00/21] cleanup meson checks for runtime binaries
Posted by Pavel Hrdina 3 years ago
On Tue, Apr 20, 2021 at 03:36:55PM +0200, Andrea Bolognani wrote:
> On Tue, 2021-04-20 at 15:20 +0200, Pavel Hrdina wrote:
> > On Tue, Apr 20, 2021 at 02:14:59PM +0100, Daniel P. Berrangé wrote:
> > > On Tue, Apr 20, 2021 at 03:05:26PM +0200, Andrea Bolognani wrote:
> > > > The only concern I have, which is one I have expressed already in the
> > > > past, is that it will now be harder for users and packagers alike to
> > > > figure out that they don't have a certain optional program installed:
> > > > AFAICT, after your patches the only way would be to try using each
> > > > feature and look out for errors.
> > > 
> > > I'd say that running "meson" and looking at its output is a poor
> > > way for users to learn what the pre-requisites are, both before
> > > and after this change.
> > > 
> > > Ideally we should document the list of packages we depend on.
> > > We have it sorta documented in the many docker files.
> > > 
> > > I wonder if we should include the list of mapping names from
> > > libvirt-ci as an explicit doc item in README.md or somewhere
> > > else that could be useful. Or just tell tem to look at the
> > > dockerfiles. Even if their distro isn't covered, it'll be
> > > enough info for them to get most of the way there.
> > 
> > Agreed, instead of having it in meson I would also prefer having it
> > documented somewhere. Ideally we should not document only what optional
> > programs we use but every single dependency we use ideally with links to
> > upstream projects and possibly examples of downstream packages which is
> > as Dan pointed out stored in libvirt-ci.
> 
> Documentation is great and all, but the problem with it is that it's
> extremely easy for it to become outdated as new optional runtime
> dependencies are added; the advantage of having this list stored in
> Meson is that the person who's adding a new optional runtime
> dependency is likely to look around for how the existing ones are
> handled, and thus automatically do the right thing.

I would say the same applies to the documentation. If the person who's
adding a new dependency will "git grep" for any already existing binary
to see what needs to be done they will find the documentation as well.
At the same time there is no guarantee that having it in meson would
make sure that we will not miss any new dependency if it's only
optional.

The only way to make sure the list is not outdated in meson or in
documentation is to add a check which would compare the list with what
we actually have in the code, like for example we do for po/POTFILES.in
to ensure that every source code file using translation is listed there.

I don't see this as a strong argument for having the list in meson.

> The point about libvirt-ci storing all the dependencies is a good
> one, but note that we only keep track of *build time* dependencies
> for projects: that is, once e.g. dnsmasq is no longer required to be
> available in the build environment in order to produce a
> fully-featured libvirt build, we'll simply stop including it. This
> will happen with almost all other optional runtime dependencies too,
> so really the mappings that are maintained as part of the libvirt-ci
> project doesn't solve the issue of keeping track of optional runtime
> dependencies at all.

Good point, but as a starting point libvirt-ci can be used to get at
least build dependencies.

Pavel