libvirt depends on a ton of packages, so trying to install them
all by using the classic approach of repeatedly running configure
and reacting to each failure by installing the corresponding
missing package will inevitably lead to frustration.

Luckily there's an easy solution to get most dependencies
installed in one fell swoop, and we just need to document it.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 CONTRIBUTING.rst | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 68c7b547c6..f476700fdd 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -17,3 +17,22 @@ your git clone run:
   $ make
 
 You'll find the freshly-built document in ``docs/contribute.html``.
+
+If ``configure`` fails because of missing dependencies, you can set
+up your system by calling
+
+::
+
+  $ sudo dnf builddep libvirt
+
+if you're on a RHEL-based distribution or
+
+::
+
+  $ sudo apt-get build-dep libvirt
+
+if you're on a Debian-based one.
+
+You might still be missing some dependencies if your distribution is
+shipping an old libvirt version, but that will get you much closer to
+where you need to be to build successfully from source.
-- 
2.25.3

On 4/20/20 6:54 AM, Andrea Bolognani wrote:
> libvirt depends on a ton of packages, so trying to install them
> all by using the classic approach of repeatedly running configure
> and reacting to each failure by installing the corresponding
> missing package will inevitably lead to frustration.
>
> Luckily there's an easy solution to get most dependencies
> installed in one fell swoop, and we just need to document it.
>
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>


Reviewed-by: Laine Stump <laine@redhat.com>


> ---
>   CONTRIBUTING.rst | 19 +++++++++++++++++++
>   1 file changed, 19 insertions(+)
>
> diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
> index 68c7b547c6..f476700fdd 100644
> --- a/CONTRIBUTING.rst
> +++ b/CONTRIBUTING.rst
> @@ -17,3 +17,22 @@ your git clone run:
>     $ make
>   
>   You'll find the freshly-built document in ``docs/contribute.html``.
> +
> +If ``configure`` fails because of missing dependencies, you can set
> +up your system by calling
> +
> +::
> +
> +  $ sudo dnf builddep libvirt
> +
> +if you're on a RHEL-based distribution or
> +
> +::
> +
> +  $ sudo apt-get build-dep libvirt
> +
> +if you're on a Debian-based one.
> +
> +You might still be missing some dependencies if your distribution is
> +shipping an old libvirt version, but that will get you much closer to
> +where you need to be to build successfully from source.

On 4/21/20 1:16 PM, Laine Stump wrote:
> On 4/20/20 6:54 AM, Andrea Bolognani wrote:
>> libvirt depends on a ton of packages, so trying to install them
>> all by using the classic approach of repeatedly running configure
>> and reacting to each failure by installing the corresponding
>> missing package will inevitably lead to frustration.
>>
>> Luckily there's an easy solution to get most dependencies
>> installed in one fell swoop, and we just need to document it.
>>
>> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
>
>
> Reviewed-by: Laine Stump <laine@redhat.com>
>
>
>> ---
>>   CONTRIBUTING.rst | 19 +++++++++++++++++++
>>   1 file changed, 19 insertions(+)
>>
>> diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
>> index 68c7b547c6..f476700fdd 100644
>> --- a/CONTRIBUTING.rst
>> +++ b/CONTRIBUTING.rst
>> @@ -17,3 +17,22 @@ your git clone run:
>>     $ make
>>     You'll find the freshly-built document in ``docs/contribute.html``.
>> +
>> +If ``configure`` fails because of missing dependencies, you can set
>> +up your system by calling
>> +
>> +::
>> +
>> +  $ sudo dnf builddep libvirt
>> +
>> +if you're on a RHEL-based distribution or
>> +
>> +::
>> +
>> +  $ sudo apt-get build-dep libvirt
>> +
>> +if you're on a Debian-based one.
>> +
>> +You might still be missing some dependencies if your distribution is
>> +shipping an old libvirt version, but that will get you much closer to
>> +where you need to be to build successfully from source.
>

BTW, I just went through this on a new Fedora 31 machine (AMD Ryzen 
3950x! w00t!!) that I had installed with "Fedora Workstation", and 
thought it might be useful to list exactly what was still missing for 
certain "developer build" tasks after running "dnf builddep libvirt" on 
a clean OS install. I found that I also had to install the following:


1) dnf install make


This was required (duh!) to build from source tar, but not a part of the 
base OS install nor in Build-Requires (I guess it makes sense - what 
self-respecting Linux distro doesn't have basic build tools like make in 
the base install?)


2) dnf install autoconf automake libtool


These three were required (but not in base OS + "dnf builddep libvirt" 
to do a successful "./autogen.sh --system && make check" (i.e. build 
from a fresh git clone of the tree).


Again, I can see why autoconf, automake, and libtool wouldn't be in the 
specfile Build-Requires:, since they *aren't* required when building 
from a source tar, which already includes the configure script and 
Makefile.in's that are generated using autotools.


3) dnf install rpm-build


This one was required (again - duh!) to run "make rpm". I guess it makes 
sense that it's not in the Build-Requires in some ways, since it isn't 
required to build the binaries, only to build the rpm file. (But on the 
other hand, the entire purpose of libvirt.spec is to build rpms. 
although, on the *other* other hand, you'd think that rpm-build would be 
included in any minimal build environment anyway (and apparently it *is* 
- e.g. the buildroot for Fedora koji).


4) dnf install cppi dwarves python3-flake8


These three were *not* required to successfully complete any build 
operation, but parts of "make syntax-check" were skipped because they 
weren't present (and so not having them might result in a developer 
believing that their patches had passed "make syntax-check, when in fact 
they had not).


At least cppi and python3-flake8 can't be Build-Required: in the 
specfile for building because they aren't available on RHEL8/CentOS8 (at 
least not without EPEL - haven't checked there yet, but of course EPEL 
packages aren't available in the official build environment, so it's 
kind of irrelevant)(oh, and also make syntax-check isn't run as part of 
building the rpms, so those programs are never run during an official 
build anyway, i.e. adding them to the Build-Requires: of the specfile 
would be a lie :-)


===============


Anyway, does anyone think it's worth adding a short bit to this file 
about these extra packages? Or should we keep this file simple and 
rather let newcomers (and old timers who've forgotten and are setting up 
a new machine) figure it out for themselves?

On Thu, 2020-04-23 at 21:30 -0400, Laine Stump wrote:
> BTW, I just went through this on a new Fedora 31 machine (AMD Ryzen 
> 3950x! w00t!!) that I had installed with "Fedora Workstation", and 
> thought it might be useful to list exactly what was still missing for 
> certain "developer build" tasks after running "dnf builddep libvirt" on 
> a clean OS install. I found that I also had to install the following:
> 
> 
> 1) dnf install make
> 
> 
> This was required (duh!) to build from source tar, but not a part of the 
> base OS install nor in Build-Requires (I guess it makes sense - what 
> self-respecting Linux distro doesn't have basic build tools like make in 
> the base install?)

Plenty! Especially in the age of containers, where everyone is
scrambling to bring down the size of the base install. And it makes
sense, too: if you're going to run Apache or something like that on
the system, it's really not necessary to have make available.

> 2) dnf install autoconf automake libtool
> 
> 
> These three were required (but not in base OS + "dnf builddep libvirt" 
> to do a successful "./autogen.sh --system && make check" (i.e. build 
> from a fresh git clone of the tree).
> 
> 
> Again, I can see why autoconf, automake, and libtool wouldn't be in the 
> specfile Build-Requires:, since they *aren't* required when building 
> from a source tar, which already includes the configure script and 
> Makefile.in's that are generated using autotools.

That is not entirely true: if you look at our spec file, you'll find

  # Default to skipping autoreconf.  Distros can change just this one line
  # (or provide a command-line override) if they backport any patches that
  # touch configure.ac or Makefile.am.
  %{!?enable_autotools:%global enable_autotools 0}

  %if 0%{?enable_autotools}
  BuildRequires: autoconf
  BuildRequires: automake
  BuildRequires: gettext-devel
  BuildRequires: libtool
  %endif

  %if 0%{?enable_autotools}
   autoreconf -if
  %endif

so we clearly anticipate the possibility of running autoreconf at
build time. Debian, for example, while obviously not using our spec
file, mandates this behavior for all its packages...

I think some packages are left out of build dependencies because they
are used by so many that you'd see them repeated all over the place.
Debian has a special package defined for this very purpose:

  https://packages.debian.org/sid/build-essential

As you can see, it includes gcc and make plus dpkg-dev (roughly
equivalent to rpmbuild, although to build most packages currently in
the archive you'll also want debhelper at the very least in addition
to that), but not autotools.

For Fedora, the closest I've found is

  sudo dnf groupinfo "C Development Tools and Libraries"
  Group: C Development Tools and Libraries
   Description: These tools include core development tools such as automake, gcc and debuggers.
   Mandatory Packages:
     autoconf
     automake
     binutils
     bison
     flex
     gcc
     gcc-c++
     gdb
     glibc-devel
     libtool
     make
     pkgconf
     strace
   Default Packages:
     ...
   Optional Packages:
     ...

which includes autotools but not rpmbuild, and has a very
inconvenient name :)

> 4) dnf install cppi dwarves python3-flake8
> 
> 
> These three were *not* required to successfully complete any build 
> operation, but parts of "make syntax-check" were skipped because they 
> weren't present (and so not having them might result in a developer 
> believing that their patches had passed "make syntax-check, when in fact 
> they had not).
> 
> 
> At least cppi and python3-flake8 can't be Build-Required: in the 
> specfile for building because they aren't available on RHEL8/CentOS8 (at 
> least not without EPEL - haven't checked there yet, but of course EPEL 
> packages aren't available in the official build environment, so it's 
> kind of irrelevant)(oh, and also make syntax-check isn't run as part of 
> building the rpms, so those programs are never run during an official 
> build anyway, i.e. adding them to the Build-Requires: of the specfile 
> would be a lie :-)

Yeah, we could easily list them as requirements on Fedora only, but
since we don't actually run syntax-check at package build time it
doesn't feel right.

> Anyway, does anyone think it's worth adding a short bit to this file 
> about these extra packages? Or should we keep this file simple and 
> rather let newcomers (and old timers who've forgotten and are setting up 
> a new machine) figure it out for themselves?

I'll post a follow-up that adds instructions for installing autotools
as a prerequisite. That will surely age well, now that the switch to
Meson is looming O:-)

I think I'll leave out the set of packages in 4), though. I don't
want the instructions to become too long, and I think even without
those packages we'll still catch most issues.

-- 
Andrea Bolognani / Red Hat / Virtualization