.gitlab-ci.d/static_checks.yml | 8 ++++---- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 5 insertions(+), 4 deletions(-)
The 'check-patch' and 'check-dco' jobs only need Python and git for
checking the patches, so it's not really necessary to use a container
here that has all the other build dependencies installed. By installing
"git" in the python container, we can use this light-weight container
for these jobs instead.
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
Mark as RFC since I'm not sure whether we want to have "git" in
the python container or not?
.gitlab-ci.d/static_checks.yml | 8 ++++----
tests/docker/dockerfiles/python.docker | 1 +
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 5e955540d3..0e080bd0a0 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -1,8 +1,8 @@
check-patch:
stage: build
- image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+ image: $CI_REGISTRY_IMAGE/qemu/python:latest
needs:
- job: amd64-centos8-container
+ job: python-container
script:
- .gitlab-ci.d/check-patch.py
variables:
@@ -15,9 +15,9 @@ check-patch:
check-dco:
stage: build
- image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+ image: $CI_REGISTRY_IMAGE/qemu/python:latest
needs:
- job: amd64-centos8-container
+ job: python-container
script: .gitlab-ci.d/check-dco.py
variables:
GIT_DEPTH: 1000
diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker
index 56d88417df..b2fb3a306d 100644
--- a/tests/docker/dockerfiles/python.docker
+++ b/tests/docker/dockerfiles/python.docker
@@ -6,6 +6,7 @@ MAINTAINER John Snow <jsnow@redhat.com>
# Please keep this list sorted alphabetically
ENV PACKAGES \
gcc \
+ git \
make \
pipenv \
python3 \
--
2.27.0On Tue, May 03, 2022 at 10:36:21PM +0200, Thomas Huth wrote:
> The 'check-patch' and 'check-dco' jobs only need Python and git for
> checking the patches, so it's not really necessary to use a container
> here that has all the other build dependencies installed. By installing
> "git" in the python container, we can use this light-weight container
> for these jobs instead.
Our python container is far from light-weight....
$ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/python
real 1m52.717s
user 1m32.327s
sys 0m19.453s
vs
$ time podman pull python:3.8-alpine
real 0m4.509s
user 0m3.780s
sys 0m1.052s
It is quicker to use the alpine python container and then just
install 'git' on every job, than it is to use the pre-built
qemu python container
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
> Mark as RFC since I'm not sure whether we want to have "git" in
> the python container or not?
>
> .gitlab-ci.d/static_checks.yml | 8 ++++----
> tests/docker/dockerfiles/python.docker | 1 +
> 2 files changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
> index 5e955540d3..0e080bd0a0 100644
> --- a/.gitlab-ci.d/static_checks.yml
> +++ b/.gitlab-ci.d/static_checks.yml
> @@ -1,8 +1,8 @@
> check-patch:
> stage: build
> - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
> + image: $CI_REGISTRY_IMAGE/qemu/python:latest
> needs:
> - job: amd64-centos8-container
> + job: python-container
> script:
> - .gitlab-ci.d/check-patch.py
> variables:
> @@ -15,9 +15,9 @@ check-patch:
>
> check-dco:
> stage: build
> - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
> + image: $CI_REGISTRY_IMAGE/qemu/python:latest
> needs:
> - job: amd64-centos8-container
> + job: python-container
> script: .gitlab-ci.d/check-dco.py
> variables:
> GIT_DEPTH: 1000
IOW this is sufficient:
image: python:3.8-alpine
needs: []
before_script:
- apk update
- apk add git
script: ./gitlab-ci.d/check-dco.py
I expect the same would work for check-patch.py container
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 04/05/2022 11.12, Daniel P. Berrangé wrote: > On Tue, May 03, 2022 at 10:36:21PM +0200, Thomas Huth wrote: >> The 'check-patch' and 'check-dco' jobs only need Python and git for >> checking the patches, so it's not really necessary to use a container >> here that has all the other build dependencies installed. By installing >> "git" in the python container, we can use this light-weight container >> for these jobs instead. > > Our python container is far from light-weight.... > > $ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/python > > real 1m52.717s > user 1m32.327s > sys 0m19.453s > > vs > > $ time podman pull python:3.8-alpine > > real 0m4.509s > user 0m3.780s > sys 0m1.052s > > > It is quicker to use the alpine python container and then just > install 'git' on every job, than it is to use the pre-built > qemu python container > >> >> Signed-off-by: Thomas Huth <thuth@redhat.com> >> --- >> Mark as RFC since I'm not sure whether we want to have "git" in >> the python container or not? >> >> .gitlab-ci.d/static_checks.yml | 8 ++++---- >> tests/docker/dockerfiles/python.docker | 1 + >> 2 files changed, 5 insertions(+), 4 deletions(-) >> >> diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml >> index 5e955540d3..0e080bd0a0 100644 >> --- a/.gitlab-ci.d/static_checks.yml >> +++ b/.gitlab-ci.d/static_checks.yml >> @@ -1,8 +1,8 @@ >> check-patch: >> stage: build >> - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest >> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >> needs: >> - job: amd64-centos8-container >> + job: python-container >> script: >> - .gitlab-ci.d/check-patch.py >> variables: >> @@ -15,9 +15,9 @@ check-patch: >> >> check-dco: >> stage: build >> - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest >> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >> needs: >> - job: amd64-centos8-container >> + job: python-container >> script: .gitlab-ci.d/check-dco.py >> variables: >> GIT_DEPTH: 1000 > > IOW this is sufficient: > > image: python:3.8-alpine > needs: [] > before_script: > - apk update > - apk add git > script: ./gitlab-ci.d/check-dco.py > > > I expect the same would work for check-patch.py container ... or would it make sense to switch tests/docker/dockerfiles/python.docker to use alpine instead of fedora? Thomas
On Wed, May 04, 2022 at 11:18:30AM +0200, Thomas Huth wrote: > On 04/05/2022 11.12, Daniel P. Berrangé wrote: > > On Tue, May 03, 2022 at 10:36:21PM +0200, Thomas Huth wrote: > > > The 'check-patch' and 'check-dco' jobs only need Python and git for > > > checking the patches, so it's not really necessary to use a container > > > here that has all the other build dependencies installed. By installing > > > "git" in the python container, we can use this light-weight container > > > for these jobs instead. > > > > Our python container is far from light-weight.... > > > > $ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/python > > > > real 1m52.717s > > user 1m32.327s > > sys 0m19.453s > > > > vs > > > > $ time podman pull python:3.8-alpine > > > > real 0m4.509s > > user 0m3.780s > > sys 0m1.052s > > > > > > It is quicker to use the alpine python container and then just > > install 'git' on every job, than it is to use the pre-built > > qemu python container > > > > > > > > Signed-off-by: Thomas Huth <thuth@redhat.com> > > > --- > > > Mark as RFC since I'm not sure whether we want to have "git" in > > > the python container or not? > > > > > > .gitlab-ci.d/static_checks.yml | 8 ++++---- > > > tests/docker/dockerfiles/python.docker | 1 + > > > 2 files changed, 5 insertions(+), 4 deletions(-) > > > > > > diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml > > > index 5e955540d3..0e080bd0a0 100644 > > > --- a/.gitlab-ci.d/static_checks.yml > > > +++ b/.gitlab-ci.d/static_checks.yml > > > @@ -1,8 +1,8 @@ > > > check-patch: > > > stage: build > > > - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest > > > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > > > needs: > > > - job: amd64-centos8-container > > > + job: python-container > > > script: > > > - .gitlab-ci.d/check-patch.py > > > variables: > > > @@ -15,9 +15,9 @@ check-patch: > > > check-dco: > > > stage: build > > > - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest > > > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > > > needs: > > > - job: amd64-centos8-container > > > + job: python-container > > > script: .gitlab-ci.d/check-dco.py > > > variables: > > > GIT_DEPTH: 1000 > > > > IOW this is sufficient: > > > > image: python:3.8-alpine > > needs: [] > > before_script: > > - apk update > > - apk add git > > script: ./gitlab-ci.d/check-dco.py > > > > > > I expect the same would work for check-patch.py container > > ... or would it make sense to switch tests/docker/dockerfiles/python.docker > to use alpine instead of fedora? That could work too With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 04/05/2022 11.20, Daniel P. Berrangé wrote: > On Wed, May 04, 2022 at 11:18:30AM +0200, Thomas Huth wrote: >> On 04/05/2022 11.12, Daniel P. Berrangé wrote: >>> On Tue, May 03, 2022 at 10:36:21PM +0200, Thomas Huth wrote: >>>> The 'check-patch' and 'check-dco' jobs only need Python and git for >>>> checking the patches, so it's not really necessary to use a container >>>> here that has all the other build dependencies installed. By installing >>>> "git" in the python container, we can use this light-weight container >>>> for these jobs instead. >>> >>> Our python container is far from light-weight.... >>> >>> $ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/python >>> >>> real 1m52.717s >>> user 1m32.327s >>> sys 0m19.453s >>> >>> vs >>> >>> $ time podman pull python:3.8-alpine >>> >>> real 0m4.509s >>> user 0m3.780s >>> sys 0m1.052s >>> >>> >>> It is quicker to use the alpine python container and then just >>> install 'git' on every job, than it is to use the pre-built >>> qemu python container >>> >>>> >>>> Signed-off-by: Thomas Huth <thuth@redhat.com> >>>> --- >>>> Mark as RFC since I'm not sure whether we want to have "git" in >>>> the python container or not? >>>> >>>> .gitlab-ci.d/static_checks.yml | 8 ++++---- >>>> tests/docker/dockerfiles/python.docker | 1 + >>>> 2 files changed, 5 insertions(+), 4 deletions(-) >>>> >>>> diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml >>>> index 5e955540d3..0e080bd0a0 100644 >>>> --- a/.gitlab-ci.d/static_checks.yml >>>> +++ b/.gitlab-ci.d/static_checks.yml >>>> @@ -1,8 +1,8 @@ >>>> check-patch: >>>> stage: build >>>> - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest >>>> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >>>> needs: >>>> - job: amd64-centos8-container >>>> + job: python-container >>>> script: >>>> - .gitlab-ci.d/check-patch.py >>>> variables: >>>> @@ -15,9 +15,9 @@ check-patch: >>>> check-dco: >>>> stage: build >>>> - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest >>>> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >>>> needs: >>>> - job: amd64-centos8-container >>>> + job: python-container >>>> script: .gitlab-ci.d/check-dco.py >>>> variables: >>>> GIT_DEPTH: 1000 >>> >>> IOW this is sufficient: >>> >>> image: python:3.8-alpine >>> needs: [] >>> before_script: >>> - apk update >>> - apk add git >>> script: ./gitlab-ci.d/check-dco.py >>> >>> >>> I expect the same would work for check-patch.py container >> >> ... or would it make sense to switch tests/docker/dockerfiles/python.docker >> to use alpine instead of fedora? > > That could work too I just tried it, and it did not work out as expected: The check-python-pipenv job needs a more sophisticated environment than the alpine container can provide easily (e.g. it still needs python-3.6), so I updated my patch with your idea instead and used the python:alpine container as a base there now. Thomas
© 2016 - 2024 Red Hat, Inc.