1. Patchew
  2. QEMU
  3. View series

[PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch

Philippe Mathieu-Daudé posted 1 patch 4 years, 9 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210215192814.989441-1-philmd@redhat.com
.gitlab-ci.d/containers.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
[PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch
Posted by Philippe Mathieu-Daudé 4 years, 9 months ago 
While we are interested in building docker images in different
branches, it only makes sense to push 'latest' to the registry
when this is the project default branch (usually 'master').

Else when pushing different branches concurrently we might have
inconsistent image state between branches.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 .gitlab-ci.d/containers.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index 90fac85ce46..52a915f4141 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -17,7 +17,7 @@
           -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
           -r $CI_REGISTRY_IMAGE
     - docker tag "qemu/$NAME" "$TAG"
-    - docker push "$TAG"
+    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push "$TAG"
   after_script:
     - docker logout
   rules:
-- 
2.26.2
Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch
Posted by Thomas Huth 4 years, 9 months ago 
On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
> While we are interested in building docker images in different
> branches, it only makes sense to push 'latest' to the registry
> when this is the project default branch (usually 'master').
> 
> Else when pushing different branches concurrently we might have
> inconsistent image state between branches.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>   .gitlab-ci.d/containers.yml | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
> index 90fac85ce46..52a915f4141 100644
> --- a/.gitlab-ci.d/containers.yml
> +++ b/.gitlab-ci.d/containers.yml
> @@ -17,7 +17,7 @@
>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
>             -r $CI_REGISTRY_IMAGE
>       - docker tag "qemu/$NAME" "$TAG"
> -    - docker push "$TAG"
> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push "$TAG"

So does that mean that the following stages in the CI (i.e. build, test) are 
only always (i.e. also for the non-master branches) going to use containers 
that have been build on the master branch?

  Thomas
Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch
Posted by Philippe Mathieu-Daudé 4 years, 9 months ago 
On 2/16/21 7:55 AM, Thomas Huth wrote:
> On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
>> While we are interested in building docker images in different
>> branches, it only makes sense to push 'latest' to the registry
>> when this is the project default branch (usually 'master').
>>
>> Else when pushing different branches concurrently we might have
>> inconsistent image state between branches.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>>   .gitlab-ci.d/containers.yml | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
>> index 90fac85ce46..52a915f4141 100644
>> --- a/.gitlab-ci.d/containers.yml
>> +++ b/.gitlab-ci.d/containers.yml
>> @@ -17,7 +17,7 @@
>>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
>>             -r $CI_REGISTRY_IMAGE
>>       - docker tag "qemu/$NAME" "$TAG"
>> -    - docker push "$TAG"
>> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push
>> "$TAG"
> 
> So does that mean that the following stages in the CI (i.e. build, test)
> are only always (i.e. also for the non-master branches) going to use
> containers that have been build on the master branch?

Hmm good point. Should we use "$CI_COMMIT_BRANCH" instead of "latest"?
Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch
Posted by Daniel P. Berrangé 4 years, 9 months ago 
On Tue, Feb 16, 2021 at 08:05:09AM +0100, Philippe Mathieu-Daudé wrote:
> On 2/16/21 7:55 AM, Thomas Huth wrote:
> > On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
> >> While we are interested in building docker images in different
> >> branches, it only makes sense to push 'latest' to the registry
> >> when this is the project default branch (usually 'master').
> >>
> >> Else when pushing different branches concurrently we might have
> >> inconsistent image state between branches.
> >>
> >> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> >> ---
> >>   .gitlab-ci.d/containers.yml | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
> >> index 90fac85ce46..52a915f4141 100644
> >> --- a/.gitlab-ci.d/containers.yml
> >> +++ b/.gitlab-ci.d/containers.yml
> >> @@ -17,7 +17,7 @@
> >>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
> >>             -r $CI_REGISTRY_IMAGE
> >>       - docker tag "qemu/$NAME" "$TAG"
> >> -    - docker push "$TAG"
> >> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push
> >> "$TAG"
> > 
> > So does that mean that the following stages in the CI (i.e. build, test)
> > are only always (i.e. also for the non-master branches) going to use
> > containers that have been build on the master branch?
> 
> Hmm good point. Should we use "$CI_COMMIT_BRANCH" instead of "latest"?

This doesn't work because git branch names are not guaranteed to be
valid docker tag names. Please see my thread here:

  https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg03535.html

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH] gitlab-ci: Only push Docker 'latest' image when building default branch
Posted by Philippe Mathieu-Daudé 4 years, 9 months ago 
On 2/16/21 10:52 AM, Daniel P. Berrangé wrote:
> On Tue, Feb 16, 2021 at 08:05:09AM +0100, Philippe Mathieu-Daudé wrote:
>> On 2/16/21 7:55 AM, Thomas Huth wrote:
>>> On 15/02/2021 20.28, Philippe Mathieu-Daudé wrote:
>>>> While we are interested in building docker images in different
>>>> branches, it only makes sense to push 'latest' to the registry
>>>> when this is the project default branch (usually 'master').
>>>>
>>>> Else when pushing different branches concurrently we might have
>>>> inconsistent image state between branches.
>>>>
>>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>>>> ---
>>>>   .gitlab-ci.d/containers.yml | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
>>>> index 90fac85ce46..52a915f4141 100644
>>>> --- a/.gitlab-ci.d/containers.yml
>>>> +++ b/.gitlab-ci.d/containers.yml
>>>> @@ -17,7 +17,7 @@
>>>>             -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
>>>>             -r $CI_REGISTRY_IMAGE
>>>>       - docker tag "qemu/$NAME" "$TAG"
>>>> -    - docker push "$TAG"
>>>> +    - test "$CI_COMMIT_BRANCH" = "$CI_DEFAULT_BRANCH" && docker push
>>>> "$TAG"
>>>
>>> So does that mean that the following stages in the CI (i.e. build, test)
>>> are only always (i.e. also for the non-master branches) going to use
>>> containers that have been build on the master branch?
>>
>> Hmm good point. Should we use "$CI_COMMIT_BRANCH" instead of "latest"?
> 
> This doesn't work because git branch names are not guaranteed to be
> valid docker tag names. Please see my thread here:
> 
>   https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg03535.html

Indeed this thread describes the problem I am having. I'll see if there
are follow up on your thread.

Another way of fixing this is to find a way to only allow 1 pipeline at
a time. I haven't tried it yet because I don't like the idea of making
CI slower, but this is still better than debugging inconsistent
pipelines when Docker images are rebuilt.

Regards,

Phil.

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.