[libvirt-ci PATCH] lcitool: Set make parallelism dynamically

Andrea Bolognani posted 1 patch 3 years, 10 months ago
Failed in applying to current master (apply log)
guests/playbooks/update/templates/bashrc.j2 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
[libvirt-ci PATCH] lcitool: Set make parallelism dynamically
Posted by Andrea Bolognani 3 years, 10 months ago
We're about to make some changes to lcitool that would make it
more annoying to get access to the number of install-time vCPUs
from 'lcitool update', so let's use getconf instead.

As a side effect, this also copes better with the situation where
the builder was installed with a certain number of vCPUs but the
amount has since been modified.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
 guests/playbooks/update/templates/bashrc.j2 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guests/playbooks/update/templates/bashrc.j2 b/guests/playbooks/update/templates/bashrc.j2
index 8775864..05a86a1 100644
--- a/guests/playbooks/update/templates/bashrc.j2
+++ b/guests/playbooks/update/templates/bashrc.j2
@@ -4,7 +4,7 @@ export MAKE="{{ paths.make }}"
 export NINJA="{{ paths.ninja }}"
 export PYTHON="{{ paths.python }}"
 
-export MAKEFLAGS="-j{{ install_vcpus|int + 1 }}"
+export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
 export CCACHE_MAXSIZE="2G"
 
 export VIRT_PREFIX="$HOME/build/libvirt"
-- 
2.25.4

Re: [libvirt-ci PATCH] lcitool: Set make parallelism dynamically
Posted by Erik Skultety 3 years, 10 months ago
On Thu, May 07, 2020 at 06:53:12PM +0200, Andrea Bolognani wrote:
> We're about to make some changes to lcitool that would make it
> more annoying to get access to the number of install-time vCPUs
> from 'lcitool update', so let's use getconf instead.
> 
> As a side effect, this also copes better with the situation where
> the builder was installed with a certain number of vCPUs but the
> amount has since been modified.
> 
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---

Reviewed-by: Erik Skultety <eskultet@redhat.com>

Re: [libvirt-ci PATCH] lcitool: Set make parallelism dynamically
Posted by Erik Skultety 3 years, 10 months ago
On Thu, May 07, 2020 at 06:53:12PM +0200, Andrea Bolognani wrote:
> We're about to make some changes to lcitool that would make it
> more annoying to get access to the number of install-time vCPUs
> from 'lcitool update', so let's use getconf instead.
> 
> As a side effect, this also copes better with the situation where
> the builder was installed with a certain number of vCPUs but the
> amount has since been modified.
> 
> Signed-off-by: Andrea Bolognani <abologna@redhat.com>
> ---
>  guests/playbooks/update/templates/bashrc.j2 | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/guests/playbooks/update/templates/bashrc.j2 b/guests/playbooks/update/templates/bashrc.j2
> index 8775864..05a86a1 100644
> --- a/guests/playbooks/update/templates/bashrc.j2
> +++ b/guests/playbooks/update/templates/bashrc.j2
> @@ -4,7 +4,7 @@ export MAKE="{{ paths.make }}"
>  export NINJA="{{ paths.ninja }}"
>  export PYTHON="{{ paths.python }}"
>  
> -export MAKEFLAGS="-j{{ install_vcpus|int + 1 }}"
> +export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"

By the way, do you happen to know where the _NPROCESSORS_ONLN came from? I mean
the only trace of it was the system config constant _SC_NPROCESSORS_ONLN which
the getconf one will be translated to, I was just confused where shorter name
came from, since it's not documented in any of the man pages (or it may be, but
I don't have that specific man page installed).

Erik

Re: [libvirt-ci PATCH] lcitool: Set make parallelism dynamically
Posted by Andrea Bolognani 3 years, 10 months ago
On Mon, 2020-05-11 at 10:20 +0200, Erik Skultety wrote:
> On Thu, May 07, 2020 at 06:53:12PM +0200, Andrea Bolognani wrote:
> > +export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> 
> By the way, do you happen to know where the _NPROCESSORS_ONLN came from? I mean
> the only trace of it was the system config constant _SC_NPROCESSORS_ONLN which
> the getconf one will be translated to, I was just confused where shorter name
> came from, since it's not documented in any of the man pages (or it may be, but
> I don't have that specific man page installed).

No idea, but it's used quite extensively in our GitLab CI
configurations and I've verified it also works on FreeBSD...

If you find the other variant is preferable for whatever reason,
we can always change it :)

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt-ci PATCH] lcitool: Set make parallelism dynamically
Posted by Erik Skultety 3 years, 10 months ago
On Mon, May 11, 2020 at 02:02:50PM +0200, Andrea Bolognani wrote:
> On Mon, 2020-05-11 at 10:20 +0200, Erik Skultety wrote:
> > On Thu, May 07, 2020 at 06:53:12PM +0200, Andrea Bolognani wrote:
> > > +export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> > 
> > By the way, do you happen to know where the _NPROCESSORS_ONLN came from? I mean
> > the only trace of it was the system config constant _SC_NPROCESSORS_ONLN which
> > the getconf one will be translated to, I was just confused where shorter name
> > came from, since it's not documented in any of the man pages (or it may be, but
> > I don't have that specific man page installed).
> 
> No idea, but it's used quite extensively in our GitLab CI
> configurations and I've verified it also works on FreeBSD...
> 
> If you find the other variant is preferable for whatever reason,
> we can always change it :)

We cant', because getconf doesn't accept it in the system constant form, that's
the puzzling part for me, your patch is the correct approach.

-- 
Erik Skultety