[libvirt PATCH 4/4] ci: use 'needs' more often

Ján Tomko posted 4 patches 5 years, 6 months ago
[libvirt PATCH 4/4] ci: use 'needs' more often
Posted by Ján Tomko 5 years, 6 months ago
Make the pipeline chart more interesting.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
---
 .gitlab-ci.yml | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 077465e436..7d23ddfdf9 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -268,41 +268,49 @@ mingw64-fedora-rawhide-container:
 
 x64-debian-10:
   <<: *native_build_job_definition
+  needs: ["x64-debian-10-container"]
   variables:
     NAME: debian-10
 
 x64-debian-sid:
   <<: *native_build_job_definition
+  needs: ["x64-debian-sid-container"]
   variables:
     NAME: debian-sid
 
 x64-centos-7:
   <<: *native_build_job_definition
+  needs: ["x64-centos-7-container"]
   variables:
     NAME: centos-7
 
 x64-centos-8:
   <<: *native_build_job_definition
+  needs: ["x64-centos-8-container"]
   variables:
     NAME: centos-8
 
 x64-centos-stream:
   <<: *native_build_job_definition
+  needs: ["x64-centos-stream-container"]
   variables:
     NAME: centos-stream
 
 x64-fedora-31:
   <<: *native_build_job_definition
+  needs: ["x64-fedora-31-container"]
   variables:
     NAME: fedora-31
 
 x64-fedora-32:
   <<: *native_build_job_definition
+  needs: ["x64-fedora-32-container"]
   variables:
     NAME: fedora-32
 
 x64-fedora-rawhide:
   <<: *native_build_job_definition
+  needs: ["x64-fedora-rawhide-container"]
   variables:
     NAME: fedora-rawhide
 
@@ -322,6 +330,7 @@ x64-fedora-rawhide-clang:
 
 x64-opensuse-151:
   <<: *native_build_job_definition
+  needs: ["x64-opensuse-151-container"]
   variables:
     NAME: opensuse-151
 
@@ -433,6 +442,7 @@ mingw64-fedora-rawhide:
 website:
   stage: builds
   image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
+  needs: ["x64-centos-8-container"]
   before_script:
     - *script_variables
   script:
@@ -455,6 +465,7 @@ website:
 codestyle:
   stage: builds
   image: $CI_REGISTRY_IMAGE/ci-opensuse-151:latest
+  needs: ["x64-opensuse-151-container"]
   before_script:
     - *script_variables
   script:
-- 
2.26.2

Re: [libvirt PATCH 4/4] ci: use 'needs' more often
Posted by Andrea Bolognani 5 years, 6 months ago
On Wed, 2020-07-29 at 01:36 +0200, Ján Tomko wrote:
> Make the pipeline chart more interesting.

Maybe include a short summary of the actual motivation here :)

> +++ b/.gitlab-ci.yml
> @@ -268,41 +268,49 @@ mingw64-fedora-rawhide-container:
>  
>  x64-debian-10:
>    <<: *native_build_job_definition
> +  needs: ["x64-debian-10-container"]
>    variables:
>      NAME: debian-10

Why are you limiting this change to a subset of the build jobs? It
seems to me that this approach can be used for all of them, both
native and cross.

And please use the

  needs:
    - x64-debian-10-container

syntax, since that's what we use for lists everywhere else.

I also thought we could make this much nicer by using something
like

  .native_build_job_template: &native_build_job_definition
    stage: builds
    image: $CI_REGISTRY_IMAGE/ci-$NAME:latest
    needs:
      - x64-$NAME-container
    ...

  .cross_build_default_job_template: &cross_build_job_definition
    stage: builds
    image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest
    needs:
      - $CROSS-$NAME-container
    ...

  x64-debian-10:
    <<: *native_build_job_definition
    variables:
      NAME: debian-10

  aarch64-debian-10:
    <<: *cross_build_job_definition
    variables:
      NAME: debian-10
      CROSS: aarch64

but it turns out that GitLab doesn't support variable expansion
inside of 'needs', so that's unfortunately not viable :(

-- 
Andrea Bolognani / Red Hat / Virtualization

Re: [libvirt PATCH 4/4] ci: use 'needs' more often
Posted by Ján Tomko 5 years, 6 months ago
On a Wednesday in 2020, Andrea Bolognani wrote:
>On Wed, 2020-07-29 at 01:36 +0200, Ján Tomko wrote:
>> Make the pipeline chart more interesting.
>
>Maybe include a short summary of the actual motivation here :)
>
>> +++ b/.gitlab-ci.yml
>> @@ -268,41 +268,49 @@ mingw64-fedora-rawhide-container:
>>
>>  x64-debian-10:
>>    <<: *native_build_job_definition
>> +  needs: ["x64-debian-10-container"]
>>    variables:
>>      NAME: debian-10
>
>Why are you limiting this change to a subset of the build jobs?

It goes in line with the actual motivation mentioned in the commit
message ;)

That is: I wanted to get some parallelism running quickly without
figuring out what the other jobs depend on or how to put it to
writing efficiently in this beautiful, whitespace-based language.

>It
>seems to me that this approach can be used for all of them, both
>native and cross.
>
>And please use the
>
>  needs:
>    - x64-debian-10-container
>
>syntax, since that's what we use for lists everywhere else.
>

Oh, okay.

>I also thought we could make this much nicer by using something
>like
>
>  .native_build_job_template: &native_build_job_definition
>    stage: builds
>    image: $CI_REGISTRY_IMAGE/ci-$NAME:latest
>    needs:
>      - x64-$NAME-container
>    ...
>
>  .cross_build_default_job_template: &cross_build_job_definition
>    stage: builds
>    image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest
>    needs:
>      - $CROSS-$NAME-container
>    ...
>
>  x64-debian-10:
>    <<: *native_build_job_definition
>    variables:
>      NAME: debian-10
>
>  aarch64-debian-10:
>    <<: *cross_build_job_definition
>    variables:
>      NAME: debian-10
>      CROSS: aarch64
>
>but it turns out that GitLab doesn't support variable expansion
>inside of 'needs', so that's unfortunately not viable :(

Thanks, now I feel better about being too lazy
to try it in the first place.

Jano
Re: [libvirt PATCH 4/4] ci: use 'needs' more often
Posted by Andrea Bolognani 5 years, 6 months ago
On Wed, 2020-07-29 at 11:17 +0200, Ján Tomko wrote:
> On a Wednesday in 2020, Andrea Bolognani wrote:
> > Why are you limiting this change to a subset of the build jobs?
> 
> It goes in line with the actual motivation mentioned in the commit
> message ;)
> 
> That is: I wanted to get some parallelism running quickly without
> figuring out what the other jobs depend on or how to put it to
> writing efficiently in this beautiful, whitespace-based language.

Each build job depends on the corresponding container build. You've
already taken care of the few exceptions to this rule, so extending
this pattern to all build job should be entirely mechanical. Please
do so in v2.

-- 
Andrea Bolognani / Red Hat / Virtualization