Instead of using pre-built containers hosted on Quay, build
containers as part of the GitLab CI pipeline and upload them to the
GitLab container registry for later use.
This will not significantly slow down builds, because containers are
only rebuilt when the corresponding Dockerfile has been modified.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
---
.gitlab-ci.yml | 234 +++++++++++++++++-
ci/containers/README.rst | 14 ++
ci/containers/ci-centos-7.Dockerfile | 137 ++++++++++
ci/containers/ci-centos-8.Dockerfile | 108 ++++++++
.../ci-debian-10-cross-aarch64.Dockerfile | 122 +++++++++
.../ci-debian-10-cross-armv6l.Dockerfile | 120 +++++++++
.../ci-debian-10-cross-armv7l.Dockerfile | 121 +++++++++
.../ci-debian-10-cross-i686.Dockerfile | 121 +++++++++
.../ci-debian-10-cross-mips.Dockerfile | 121 +++++++++
.../ci-debian-10-cross-mips64el.Dockerfile | 121 +++++++++
.../ci-debian-10-cross-mipsel.Dockerfile | 121 +++++++++
.../ci-debian-10-cross-ppc64le.Dockerfile | 121 +++++++++
.../ci-debian-10-cross-s390x.Dockerfile | 121 +++++++++
ci/containers/ci-debian-10.Dockerfile | 112 +++++++++
.../ci-debian-9-cross-aarch64.Dockerfile | 126 ++++++++++
.../ci-debian-9-cross-armv6l.Dockerfile | 124 ++++++++++
.../ci-debian-9-cross-armv7l.Dockerfile | 125 ++++++++++
.../ci-debian-9-cross-mips.Dockerfile | 125 ++++++++++
.../ci-debian-9-cross-mips64el.Dockerfile | 125 ++++++++++
.../ci-debian-9-cross-mipsel.Dockerfile | 125 ++++++++++
.../ci-debian-9-cross-ppc64le.Dockerfile | 125 ++++++++++
.../ci-debian-9-cross-s390x.Dockerfile | 125 ++++++++++
ci/containers/ci-debian-9.Dockerfile | 116 +++++++++
.../ci-debian-sid-cross-aarch64.Dockerfile | 122 +++++++++
.../ci-debian-sid-cross-armv6l.Dockerfile | 120 +++++++++
.../ci-debian-sid-cross-armv7l.Dockerfile | 121 +++++++++
.../ci-debian-sid-cross-i686.Dockerfile | 121 +++++++++
.../ci-debian-sid-cross-mips.Dockerfile | 121 +++++++++
.../ci-debian-sid-cross-mips64el.Dockerfile | 121 +++++++++
.../ci-debian-sid-cross-mipsel.Dockerfile | 120 +++++++++
.../ci-debian-sid-cross-ppc64le.Dockerfile | 121 +++++++++
.../ci-debian-sid-cross-s390x.Dockerfile | 121 +++++++++
ci/containers/ci-debian-sid.Dockerfile | 112 +++++++++
ci/containers/ci-fedora-31.Dockerfile | 109 ++++++++
ci/containers/ci-fedora-32.Dockerfile | 109 ++++++++
...ci-fedora-rawhide-cross-mingw32.Dockerfile | 129 ++++++++++
...ci-fedora-rawhide-cross-mingw64.Dockerfile | 129 ++++++++++
ci/containers/ci-fedora-rawhide.Dockerfile | 110 ++++++++
ci/containers/ci-opensuse-151.Dockerfile | 109 ++++++++
ci/containers/ci-ubuntu-1804.Dockerfile | 117 +++++++++
ci/containers/ci-ubuntu-2004.Dockerfile | 113 +++++++++
ci/containers/refresh | 43 ++++
42 files changed, 4973 insertions(+), 5 deletions(-)
create mode 100644 ci/containers/README.rst
create mode 100644 ci/containers/ci-centos-7.Dockerfile
create mode 100644 ci/containers/ci-centos-8.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-aarch64.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-armv6l.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-armv7l.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-i686.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-mips.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-mips64el.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-mipsel.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-ppc64le.Dockerfile
create mode 100644 ci/containers/ci-debian-10-cross-s390x.Dockerfile
create mode 100644 ci/containers/ci-debian-10.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-aarch64.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-armv6l.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-armv7l.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-mips.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-mips64el.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-mipsel.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-ppc64le.Dockerfile
create mode 100644 ci/containers/ci-debian-9-cross-s390x.Dockerfile
create mode 100644 ci/containers/ci-debian-9.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-aarch64.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-armv6l.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-armv7l.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-i686.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-mips.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-mips64el.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-mipsel.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-ppc64le.Dockerfile
create mode 100644 ci/containers/ci-debian-sid-cross-s390x.Dockerfile
create mode 100644 ci/containers/ci-debian-sid.Dockerfile
create mode 100644 ci/containers/ci-fedora-31.Dockerfile
create mode 100644 ci/containers/ci-fedora-32.Dockerfile
create mode 100644 ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile
create mode 100644 ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile
create mode 100644 ci/containers/ci-fedora-rawhide.Dockerfile
create mode 100644 ci/containers/ci-opensuse-151.Dockerfile
create mode 100644 ci/containers/ci-ubuntu-1804.Dockerfile
create mode 100644 ci/containers/ci-ubuntu-2004.Dockerfile
create mode 100755 ci/containers/refresh
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 8a5b3372de..0e7917d6cd 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -3,6 +3,7 @@ variables:
stages:
- prebuild
+ - containers
- native_build
- cross_build
- other
@@ -16,10 +17,35 @@ stages:
# Common templates
+.container_job_template: &container_job_definition
+ image: docker:stable
+ stage: containers
+ services:
+ - docker:dind
+ before_script:
+ - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest"
+ - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest"
+ - docker info
+ - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
+ script:
+ - docker pull "$TAG" || docker pull "$COMMON_TAG" || true
+ - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/ci-$NAME.Dockerfile" ci/containers
+ - docker push "$TAG"
+ after_script:
+ - docker logout
+
+# We build many containers which can be useful to debug problems but are not
+# needed for the pipeline itself to complete: those sometimes fail, and when
+# that happens it's mostly because of temporary issues with Debian sid. We
+# don't want those failures to affect the overall pipeline status
+.container_optional_job_template: &container_optional_job_definition
+ <<: *container_job_definition
+ allow_failure: true
+
# Default native build jobs that are always run
.native_build_default_job_template: &native_build_default_job_definition
stage: native_build
- image: quay.io/libvirt/buildenv-libvirt-$NAME:latest
+ image: $CI_REGISTRY_IMAGE/ci-$NAME:latest
cache:
paths:
- ccache/
@@ -44,7 +70,7 @@ stages:
# Default cross build jobs that are always run
.cross_build_default_job_template: &cross_build_default_job_definition
stage: cross_build
- image: quay.io/libvirt/buildenv-libvirt-$NAME-cross-$CROSS:latest
+ image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest
cache:
paths:
- ccache/
@@ -66,6 +92,204 @@ stages:
- /^ci-full-.*$/
+# Container build jobs
+
+centos-7-container:
+ <<: *container_job_definition
+ variables:
+ NAME: centos-7
+
+centos-8-container:
+ <<: *container_job_definition
+ variables:
+ NAME: centos-8
+
+debian-9-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-9
+
+debian-9-cross-aarch64-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-9-cross-aarch64
+
+debian-9-cross-armv6l-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-9-cross-armv6l
+
+debian-9-cross-armv7l-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-9-cross-armv7l
+
+debian-9-cross-mips-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-9-cross-mips
+
+debian-9-cross-mips64el-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-9-cross-mips64el
+
+debian-9-cross-mipsel-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-9-cross-mipsel
+
+debian-9-cross-ppc64le-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-9-cross-ppc64le
+
+debian-9-cross-s390x-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-9-cross-s390x
+
+debian-10-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-10
+
+debian-10-cross-aarch64-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-10-cross-aarch64
+
+debian-10-cross-armv6l-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-10-cross-armv6l
+
+debian-10-cross-armv7l-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-10-cross-armv7l
+
+debian-10-cross-i686-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-10-cross-i686
+
+debian-10-cross-mips-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-10-cross-mips
+
+debian-10-cross-mips64el-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-10-cross-mips64el
+
+debian-10-cross-mipsel-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-10-cross-mipsel
+
+debian-10-cross-ppc64le-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-10-cross-ppc64le
+
+debian-10-cross-s390x-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-10-cross-s390x
+
+debian-sid-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-sid
+
+debian-sid-cross-aarch64-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-sid-cross-aarch64
+
+debian-sid-cross-armv6l-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-sid-cross-armv6l
+
+debian-sid-cross-armv7l-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-sid-cross-armv7l
+
+debian-sid-cross-i686-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-sid-cross-i686
+
+debian-sid-cross-mips-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-sid-cross-mips
+
+debian-sid-cross-mips64el-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-sid-cross-mips64el
+
+debian-sid-cross-mipsel-container:
+ <<: *container_job_definition
+ variables:
+ NAME: debian-sid-cross-mipsel
+
+debian-sid-cross-ppc64le-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-sid-cross-ppc64le
+
+debian-sid-cross-s390x-container:
+ <<: *container_optional_job_definition
+ variables:
+ NAME: debian-sid-cross-s390x
+
+fedora-31-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora-31
+
+fedora-32-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora-32
+
+fedora-rawhide-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora-rawhide
+
+fedora-rawhide-cross-mingw32-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora-rawhide-cross-mingw32
+
+fedora-rawhide-cross-mingw64-container:
+ <<: *container_job_definition
+ variables:
+ NAME: fedora-rawhide-cross-mingw64
+
+opensuse-151-container:
+ <<: *container_job_definition
+ variables:
+ NAME: opensuse-151
+ RPM: skip
+
+ubuntu-1804-container:
+ <<: *container_job_definition
+ variables:
+ NAME: ubuntu-1804
+
+ubuntu-2004-container:
+ <<: *container_job_definition
+ variables:
+ NAME: ubuntu-2004
+
# Native architecture build + test jobs
x64-debian-9:
@@ -198,6 +422,7 @@ mingw64-fedora-rawhide:
# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=website
website:
stage: other
+ image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
before_script:
- *script_variables
script:
@@ -208,7 +433,6 @@ website:
- $MAKE -C docs install
- cd ..
- mv vroot/share/doc/libvirt/html/ website
- image: quay.io/libvirt/buildenv-libvirt-centos-8:latest
artifacts:
expose_as: 'Website'
name: 'website'
@@ -220,6 +444,7 @@ website:
codestyle:
stage: other
+ image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
before_script:
- *script_variables
script:
@@ -227,7 +452,6 @@ codestyle:
- cd build
- ../autogen.sh || (cat config.log && exit 1)
- $MAKE syntax-check
- image: quay.io/libvirt/buildenv-libvirt-centos-8:latest
# This artifact published by this job is downloaded to push to Weblate
@@ -235,6 +459,7 @@ codestyle:
# https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=potfile
potfile:
stage: prebuild
+ image: $CI_REGISTRY_IMAGE/ci-centos-8:latest
only:
- master
before_script:
@@ -247,7 +472,6 @@ potfile:
- $MAKE -C po libvirt.pot
- cd ..
- mv build/po/libvirt.pot libvirt.pot
- image: quay.io/libvirt/buildenv-libvirt-centos-8:latest
artifacts:
expose_as: 'Potfile'
name: 'potfile'
diff --git a/ci/containers/README.rst b/ci/containers/README.rst
new file mode 100644
index 0000000000..530897e311
--- /dev/null
+++ b/ci/containers/README.rst
@@ -0,0 +1,14 @@
+CI job assets
+=============
+
+This directory contains assets used in the automated CI jobs, most
+notably the Dockerfiles used to build container images in which the
+CI jobs then run.
+
+The ``refresh`` script is used to re-create the Dockerfiles using the
+``lcitool`` command that is provided by repo
+https://gitlab.com/libvirt/libvirt-ci
+
+The containers are built during the CI process and cached in the GitLab
+container registry of the project doing the build. The cached containers
+can be deleted at any time and will be correctly rebuilt.
diff --git a/ci/containers/ci-centos-7.Dockerfile b/ci/containers/ci-centos-7.Dockerfile
new file mode 100644
index 0000000000..abbbdcc47a
--- /dev/null
+++ b/ci/containers/ci-centos-7.Dockerfile
@@ -0,0 +1,137 @@
+FROM centos:7
+
+RUN echo -e '[openvz]\n\
+name=OpenVZ addons\n\
+baseurl=https://download.openvz.org/virtuozzo/releases/openvz-7.0.11-235/x86_64/os/\n\
+enabled=1\n\
+gpgcheck=1\n\
+skip_if_unavailable=0\n\
+metadata_expire=6h\n\
+priority=90\n\
+includepkgs=libprl*' > /etc/yum.repos.d/openvz.repo && \
+ echo -e '-----BEGIN PGP PUBLIC KEY BLOCK-----\n\
+Version: GnuPG v2.0.22 (GNU/Linux)\n\
+\n\
+mI0EVl80nQEEAKrEeyeTCwrzS9kYedZ/sAc/GUqlb81C7pA9SaR3fyck5mVw1Ogk\n\
+YdmNBPM2kY7QDxR9F0EpSpnxSCAXZXugsQ8KzZ0DRLVeBDQyGs9IGK5hI0zzxIil\n\
+BzfvIexLiQQhLy7YlIi8Jt/uUqKkW0pIMNMGcduY97VATtczpncpkmSzABEBAAG0\n\
+SFZpcnR1b3p6byBUZWFtIChHUEcga2V5IHNpZ25hdHVyZSBmb3IgcGFja2FnZXMp\n\
+IDxzZWN1cml0eUB2aXJ0dW96em8uY29tPoi5BBMBAgAjBQJWXzSdAhsDBwsJCAcD\n\
+AgEGFQgCCQoLBBYCAwECHgECF4AACgkQygt9GUTNrSruIgP/er70Eyo73A1gfrjv\n\
+oPUkyo4rslVRZu3qqCwoMFtJc/Z/UxWgEka1buorlcGLa6eO/EZ49c0n+KGa4Kvt\n\
+EUboIq0yEu5i0FyAj92ifm+hNhoAbGfm0cZ4/fD0oGr3l8OsQo4+iHX4xAPwFe7Y\n\
+zABuB8I1ZDZ4OIp5tDfTTuF2LT24jQRWXzSdAQQAog2Aqb+Ptl68O7cQhWLjVGkj\n\
+yyigZrdeReLx3HloKJPBeQ/kA6uvMJc/IYS3uppMWXv9v+QenS6uhP1TUJ2k9FvM\n\
+t94MQZfALN7Vpf8AF+UeWu4Ru+y4BNzcFhrPhIFNFChOR2QqW6FkgE57D9I177NC\n\
+oJMyrlNe8wcGa178An8AEQEAAYifBBgBAgAJBQJWXzSdAhsMAAoJEMoLfRlEza0q\n\
+bKwD/3+OFVIEXnIv5XgdGRNX5fHggsUN1bb8gva7HANRlKdd4LD8foDM3F/yv/3V\n\
+igG14D5EjKz56SaBDNgiI4++hOzb2M8jhAsR86jxkXFrrP1U3ZNRKg6av9DPFAPS\n\
+WEiJKtQrZDJloqtyi/mmRa1VsV7RYR0VPJjhK/R8EQ7Ysshy\n\
+=fRMg\n\
+-----END PGP PUBLIC KEY BLOCK-----' > /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \
+ rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \
+ yum install -y epel-release && \
+ yum update -y && \
+ yum install -y \
+ audit-libs-devel \
+ augeas \
+ autoconf \
+ automake \
+ avahi-devel \
+ bash \
+ bash-completion \
+ ca-certificates \
+ ccache \
+ chrony \
+ cyrus-sasl-devel \
+ dbus-devel \
+ device-mapper-devel \
+ dnsmasq \
+ dwarves \
+ ebtables \
+ fuse-devel \
+ gcc \
+ gdb \
+ gettext \
+ gettext-devel \
+ git \
+ glib2-devel \
+ glibc-common \
+ glibc-devel \
+ glusterfs-api-devel \
+ gnutls-devel \
+ iproute \
+ iscsi-initiator-utils \
+ kmod \
+ libacl-devel \
+ libattr-devel \
+ libblkid-devel \
+ libcap-ng-devel \
+ libcurl-devel \
+ libiscsi-devel \
+ libnl3-devel \
+ libpcap-devel \
+ libpciaccess-devel \
+ libprlsdk-devel \
+ librbd1-devel \
+ libselinux-devel \
+ libssh-devel \
+ libssh2-devel \
+ libtirpc-devel \
+ libtool \
+ libudev-devel \
+ libwsman-devel \
+ libxml2 \
+ libxml2-devel \
+ libxslt \
+ lsof \
+ lvm2 \
+ make \
+ ncurses-devel \
+ net-tools \
+ netcf-devel \
+ nfs-utils \
+ ninja-build \
+ numactl-devel \
+ numad \
+ parted \
+ parted-devel \
+ patch \
+ perl \
+ pkgconfig \
+ polkit \
+ python3 \
+ python3-pip \
+ python3-setuptools \
+ python3-wheel \
+ python36-docutils \
+ qemu-img \
+ radvd \
+ readline-devel \
+ rpm-build \
+ sanlock-devel \
+ screen \
+ scrub \
+ strace \
+ sudo \
+ systemtap-sdt-devel \
+ vim \
+ wireshark-devel \
+ xfsprogs-devel \
+ yajl-devel && \
+ yum autoremove -y && \
+ yum clean all -y && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+RUN pip3 install \
+ meson==0.49.0
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja-build"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
[... many more Dockerfiles here ...]
diff --git a/ci/containers/refresh b/ci/containers/refresh
new file mode 100755
index 0000000000..8c00363ae1
--- /dev/null
+++ b/ci/containers/refresh
@@ -0,0 +1,43 @@
+#!/bin/sh
+
+if test -z "$1"
+then
+ echo "syntax: $0 PATH-TO-LCITOOL"
+ exit 1
+fi
+
+LCITOOL=$1
+
+if ! test -x "$LCITOOL"
+then
+ echo "$LCITOOL is not executable"
+ exit 1
+fi
+
+HOSTS=$($LCITOOL hosts | grep -v freebsd)
+
+for host in $HOSTS
+do
+ name=${host#libvirt-}
+
+ case "$name" in
+ fedora-rawhide)
+ for cross in mingw32 mingw64
+ do
+ $LCITOOL dockerfile $host libvirt --cross $cross >ci-$name-cross-$cross.Dockerfile
+ done
+ ;;
+ debian-*)
+ for cross in aarch64 armv6l armv7l i686 mips mips64el mipsel ppc64le s390x
+ do
+ if test "$name" = "debian-9" && test "$cross" = "i686"
+ then
+ continue
+ fi
+ $LCITOOL dockerfile $host libvirt --cross $cross >ci-$name-cross-$cross.Dockerfile
+ done
+ ;;
+ esac
+
+ $LCITOOL dockerfile $host libvirt >ci-$name.Dockerfile
+done
--
2.25.4
On Fri, May 29, 2020 at 03:00:42PM +0200, Andrea Bolognani wrote: > Instead of using pre-built containers hosted on Quay, build > containers as part of the GitLab CI pipeline and upload them to the > GitLab container registry for later use. > > This will not significantly slow down builds, because containers are > only rebuilt when the corresponding Dockerfile has been modified. > > Signed-off-by: Andrea Bolognani <abologna@redhat.com> > --- > .gitlab-ci.yml | 234 +++++++++++++++++- > ci/containers/README.rst | 14 ++ > ci/containers/ci-centos-7.Dockerfile | 137 ++++++++++ > ci/containers/ci-centos-8.Dockerfile | 108 ++++++++ > .../ci-debian-10-cross-aarch64.Dockerfile | 122 +++++++++ > .../ci-debian-10-cross-armv6l.Dockerfile | 120 +++++++++ > .../ci-debian-10-cross-armv7l.Dockerfile | 121 +++++++++ > .../ci-debian-10-cross-i686.Dockerfile | 121 +++++++++ > .../ci-debian-10-cross-mips.Dockerfile | 121 +++++++++ > .../ci-debian-10-cross-mips64el.Dockerfile | 121 +++++++++ > .../ci-debian-10-cross-mipsel.Dockerfile | 121 +++++++++ > .../ci-debian-10-cross-ppc64le.Dockerfile | 121 +++++++++ > .../ci-debian-10-cross-s390x.Dockerfile | 121 +++++++++ > ci/containers/ci-debian-10.Dockerfile | 112 +++++++++ > .../ci-debian-9-cross-aarch64.Dockerfile | 126 ++++++++++ > .../ci-debian-9-cross-armv6l.Dockerfile | 124 ++++++++++ > .../ci-debian-9-cross-armv7l.Dockerfile | 125 ++++++++++ > .../ci-debian-9-cross-mips.Dockerfile | 125 ++++++++++ > .../ci-debian-9-cross-mips64el.Dockerfile | 125 ++++++++++ > .../ci-debian-9-cross-mipsel.Dockerfile | 125 ++++++++++ > .../ci-debian-9-cross-ppc64le.Dockerfile | 125 ++++++++++ > .../ci-debian-9-cross-s390x.Dockerfile | 125 ++++++++++ > ci/containers/ci-debian-9.Dockerfile | 116 +++++++++ > .../ci-debian-sid-cross-aarch64.Dockerfile | 122 +++++++++ > .../ci-debian-sid-cross-armv6l.Dockerfile | 120 +++++++++ > .../ci-debian-sid-cross-armv7l.Dockerfile | 121 +++++++++ > .../ci-debian-sid-cross-i686.Dockerfile | 121 +++++++++ > .../ci-debian-sid-cross-mips.Dockerfile | 121 +++++++++ > .../ci-debian-sid-cross-mips64el.Dockerfile | 121 +++++++++ > .../ci-debian-sid-cross-mipsel.Dockerfile | 120 +++++++++ > .../ci-debian-sid-cross-ppc64le.Dockerfile | 121 +++++++++ > .../ci-debian-sid-cross-s390x.Dockerfile | 121 +++++++++ > ci/containers/ci-debian-sid.Dockerfile | 112 +++++++++ > ci/containers/ci-fedora-31.Dockerfile | 109 ++++++++ > ci/containers/ci-fedora-32.Dockerfile | 109 ++++++++ > ...ci-fedora-rawhide-cross-mingw32.Dockerfile | 129 ++++++++++ > ...ci-fedora-rawhide-cross-mingw64.Dockerfile | 129 ++++++++++ > ci/containers/ci-fedora-rawhide.Dockerfile | 110 ++++++++ > ci/containers/ci-opensuse-151.Dockerfile | 109 ++++++++ > ci/containers/ci-ubuntu-1804.Dockerfile | 117 +++++++++ > ci/containers/ci-ubuntu-2004.Dockerfile | 113 +++++++++ > ci/containers/refresh | 43 ++++ > 42 files changed, 4973 insertions(+), 5 deletions(-) > create mode 100644 ci/containers/README.rst > create mode 100644 ci/containers/ci-centos-7.Dockerfile > create mode 100644 ci/containers/ci-centos-8.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-aarch64.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-armv6l.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-armv7l.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-i686.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-mips.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-mips64el.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-mipsel.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-ppc64le.Dockerfile > create mode 100644 ci/containers/ci-debian-10-cross-s390x.Dockerfile > create mode 100644 ci/containers/ci-debian-10.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-aarch64.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-armv6l.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-armv7l.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-mips.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-mips64el.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-mipsel.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-ppc64le.Dockerfile > create mode 100644 ci/containers/ci-debian-9-cross-s390x.Dockerfile > create mode 100644 ci/containers/ci-debian-9.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-aarch64.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-armv6l.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-armv7l.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-i686.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-mips.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-mips64el.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-mipsel.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-ppc64le.Dockerfile > create mode 100644 ci/containers/ci-debian-sid-cross-s390x.Dockerfile > create mode 100644 ci/containers/ci-debian-sid.Dockerfile > create mode 100644 ci/containers/ci-fedora-31.Dockerfile > create mode 100644 ci/containers/ci-fedora-32.Dockerfile > create mode 100644 ci/containers/ci-fedora-rawhide-cross-mingw32.Dockerfile > create mode 100644 ci/containers/ci-fedora-rawhide-cross-mingw64.Dockerfile > create mode 100644 ci/containers/ci-fedora-rawhide.Dockerfile > create mode 100644 ci/containers/ci-opensuse-151.Dockerfile > create mode 100644 ci/containers/ci-ubuntu-1804.Dockerfile > create mode 100644 ci/containers/ci-ubuntu-2004.Dockerfile > create mode 100755 ci/containers/refresh > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 8a5b3372de..0e7917d6cd 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -3,6 +3,7 @@ variables: > > stages: > - prebuild > + - containers > - native_build > - cross_build > - other > @@ -16,10 +17,35 @@ stages: > > # Common templates > > +.container_job_template: &container_job_definition > + image: docker:stable > + stage: containers > + services: > + - docker:dind > + before_script: > + - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" > + - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt/ci-$NAME:latest" > + - docker info > + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" > + script: > + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true > + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/ci-$NAME.Dockerfile" ci/containers > + - docker push "$TAG" > + after_script: > + - docker logout > + > +# We build many containers which can be useful to debug problems but are not > +# needed for the pipeline itself to complete: those sometimes fail, and when > +# that happens it's mostly because of temporary issues with Debian sid. We > +# don't want those failures to affect the overall pipeline status > +.container_optional_job_template: &container_optional_job_definition > + <<: *container_job_definition > + allow_failure: true I don't think we should be building container images that we're not going to be using in any of the jobs, as it can only ever slow down the build overall. > + > # Default native build jobs that are always run > .native_build_default_job_template: &native_build_default_job_definition > stage: native_build > - image: quay.io/libvirt/buildenv-libvirt-$NAME:latest > + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest > cache: > paths: > - ccache/ > @@ -44,7 +70,7 @@ stages: > # Default cross build jobs that are always run > .cross_build_default_job_template: &cross_build_default_job_definition > stage: cross_build > - image: quay.io/libvirt/buildenv-libvirt-$NAME-cross-$CROSS:latest > + image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest > cache: > paths: > - ccache/ > @@ -66,6 +92,204 @@ stages: > - /^ci-full-.*$/ > > > +# Container build jobs > + > +centos-7-container: IMHO we should name these to match the build job. eg arch, then distro x64-centos-7-container > + <<: *container_job_definition > + variables: > + NAME: centos-7 > + > +centos-8-container: > + <<: *container_job_definition > + variables: > + NAME: centos-8 > + > +debian-9-container: > + <<: *container_job_definition > + variables: > + NAME: debian-9 > + > +debian-9-cross-aarch64-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-aarch64 > + > +debian-9-cross-armv6l-container: > + <<: *container_job_definition > + variables: > + NAME: debian-9-cross-armv6l This container, and many others are only used by the "extra" build jobs, so should be subject to the same filtering. > + > +debian-9-cross-armv7l-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-armv7l > + > +debian-9-cross-mips-container: > + <<: *container_job_definition > + variables: > + NAME: debian-9-cross-mips > + > +debian-9-cross-mips64el-container: > + <<: *container_job_definition > + variables: > + NAME: debian-9-cross-mips64el > + > +debian-9-cross-mipsel-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-mipsel > + > +debian-9-cross-ppc64le-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-ppc64le > + > +debian-9-cross-s390x-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-9-cross-s390x > + > +debian-10-container: > + <<: *container_job_definition > + variables: > + NAME: debian-10 > + > +debian-10-cross-aarch64-container: > + <<: *container_job_definition > + variables: > + NAME: debian-10-cross-aarch64 > + > +debian-10-cross-armv6l-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-armv6l > + > +debian-10-cross-armv7l-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-armv7l > + > +debian-10-cross-i686-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-i686 > + > +debian-10-cross-mips-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-mips > + > +debian-10-cross-mips64el-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-mips64el > + > +debian-10-cross-mipsel-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-10-cross-mipsel > + > +debian-10-cross-ppc64le-container: > + <<: *container_job_definition > + variables: > + NAME: debian-10-cross-ppc64le > + > +debian-10-cross-s390x-container: > + <<: *container_job_definition > + variables: > + NAME: debian-10-cross-s390x > + > +debian-sid-container: > + <<: *container_job_definition > + variables: > + NAME: debian-sid > + > +debian-sid-cross-aarch64-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-aarch64 > + > +debian-sid-cross-armv6l-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-armv6l > + > +debian-sid-cross-armv7l-container: > + <<: *container_job_definition > + variables: > + NAME: debian-sid-cross-armv7l > + > +debian-sid-cross-i686-container: > + <<: *container_job_definition > + variables: > + NAME: debian-sid-cross-i686 > + > +debian-sid-cross-mips-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-mips > + > +debian-sid-cross-mips64el-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-mips64el > + > +debian-sid-cross-mipsel-container: > + <<: *container_job_definition > + variables: > + NAME: debian-sid-cross-mipsel > + > +debian-sid-cross-ppc64le-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-ppc64le > + > +debian-sid-cross-s390x-container: > + <<: *container_optional_job_definition > + variables: > + NAME: debian-sid-cross-s390x > + > +fedora-31-container: > + <<: *container_job_definition > + variables: > + NAME: fedora-31 > + > +fedora-32-container: > + <<: *container_job_definition > + variables: > + NAME: fedora-32 > + > +fedora-rawhide-container: > + <<: *container_job_definition > + variables: > + NAME: fedora-rawhide > + > +fedora-rawhide-cross-mingw32-container: > + <<: *container_job_definition > + variables: > + NAME: fedora-rawhide-cross-mingw32 > + > +fedora-rawhide-cross-mingw64-container: > + <<: *container_job_definition > + variables: > + NAME: fedora-rawhide-cross-mingw64 > + > +opensuse-151-container: > + <<: *container_job_definition > + variables: > + NAME: opensuse-151 > + RPM: skip > + > +ubuntu-1804-container: > + <<: *container_job_definition > + variables: > + NAME: ubuntu-1804 > + > +ubuntu-2004-container: > + <<: *container_job_definition > + variables: > + NAME: ubuntu-2004 > + > # Native architecture build + test jobs > > x64-debian-9: > @@ -198,6 +422,7 @@ mingw64-fedora-rawhide: > # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=website > website: > stage: other > + image: $CI_REGISTRY_IMAGE/ci-centos-8:latest > before_script: > - *script_variables > script: > @@ -208,7 +433,6 @@ website: > - $MAKE -C docs install > - cd .. > - mv vroot/share/doc/libvirt/html/ website > - image: quay.io/libvirt/buildenv-libvirt-centos-8:latest > artifacts: > expose_as: 'Website' > name: 'website' > @@ -220,6 +444,7 @@ website: > > codestyle: > stage: other > + image: $CI_REGISTRY_IMAGE/ci-centos-8:latest > before_script: > - *script_variables > script: > @@ -227,7 +452,6 @@ codestyle: > - cd build > - ../autogen.sh || (cat config.log && exit 1) > - $MAKE syntax-check > - image: quay.io/libvirt/buildenv-libvirt-centos-8:latest > > > # This artifact published by this job is downloaded to push to Weblate > @@ -235,6 +459,7 @@ codestyle: > # https://gitlab.com/libvirt/libvirt/-/jobs/artifacts/master/download?job=potfile > potfile: > stage: prebuild > + image: $CI_REGISTRY_IMAGE/ci-centos-8:latest > only: > - master > before_script: > @@ -247,7 +472,6 @@ potfile: > - $MAKE -C po libvirt.pot > - cd .. > - mv build/po/libvirt.pot libvirt.pot > - image: quay.io/libvirt/buildenv-libvirt-centos-8:latest > artifacts: > expose_as: 'Potfile' > name: 'potfile' Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote: > On Fri, May 29, 2020 at 03:00:42PM +0200, Andrea Bolognani wrote: > > +# We build many containers which can be useful to debug problems but are not > > +# needed for the pipeline itself to complete: those sometimes fail, and when > > +# that happens it's mostly because of temporary issues with Debian sid. We > > +# don't want those failures to affect the overall pipeline status > > +.container_optional_job_template: &container_optional_job_definition > > + <<: *container_job_definition > > + allow_failure: true > > I don't think we should be building container images that we're not going > to be using in any of the jobs, as it can only ever slow down the build > overall. These same containers are also available for use outside of CI, eg. with 'make ci-build', so I think we should keep building them. As for slowing down builds, that still only applies to the first build after Dockerfiles are updated, so I don't think it ultimately matters very much. > > +# Container build jobs > > + > > +centos-7-container: > > IMHO we should name these to match the build job. eg arch, then distro > > x64-centos-7-container Okay. > > +debian-9-cross-armv6l-container: > > + <<: *container_job_definition > > + variables: > > + NAME: debian-9-cross-armv6l > > This container, and many others are only used by the "extra" build > jobs, so should be subject to the same filtering. Okay, even though as we discussed separately the whole idea of splitting jobs between regular and extra might be more trouble than it's worth and be more confusing than helpful. -- Andrea Bolognani / Red Hat / Virtualization
On Tue, Jun 02, 2020 at 01:10:08PM +0200, Andrea Bolognani wrote: > On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote: > > On Fri, May 29, 2020 at 03:00:42PM +0200, Andrea Bolognani wrote: > > > +# We build many containers which can be useful to debug problems but are not > > > +# needed for the pipeline itself to complete: those sometimes fail, and when > > > +# that happens it's mostly because of temporary issues with Debian sid. We > > > +# don't want those failures to affect the overall pipeline status > > > +.container_optional_job_template: &container_optional_job_definition > > > + <<: *container_job_definition > > > + allow_failure: true > > > > I don't think we should be building container images that we're not going > > to be using in any of the jobs, as it can only ever slow down the build > > overall. > > These same containers are also available for use outside of CI, eg. > with 'make ci-build', so I think we should keep building them. That only needs them built on the master branch of the main repo though, not every branch in every fork > As for slowing down builds, that still only applies to the first > build after Dockerfiles are updated, so I don't think it ultimately > matters very much. I'd expect a rebiuld if the distro base image changes which could be fairly often for the rawhide like distros. > > > > +# Container build jobs > > > + > > > +centos-7-container: > > > > IMHO we should name these to match the build job. eg arch, then distro > > > > x64-centos-7-container > > Okay. > > > > +debian-9-cross-armv6l-container: > > > + <<: *container_job_definition > > > + variables: > > > + NAME: debian-9-cross-armv6l > > > > This container, and many others are only used by the "extra" build > > jobs, so should be subject to the same filtering. > > Okay, even though as we discussed separately the whole idea of > splitting jobs between regular and extra might be more trouble than > it's worth and be more confusing than helpful. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, 2020-06-02 at 12:23 +0100, Daniel P. Berrangé wrote:
> On Tue, Jun 02, 2020 at 01:10:08PM +0200, Andrea Bolognani wrote:
> > On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote:
> > > I don't think we should be building container images that we're not going
> > > to be using in any of the jobs, as it can only ever slow down the build
> > > overall.
> >
> > These same containers are also available for use outside of CI, eg.
> > with 'make ci-build', so I think we should keep building them.
>
> That only needs them built on the master branch of the main repo
> though, not every branch in every fork
Fair enough. So what you're suggesting is something like
.container_optional_job_template: &container_optional_job_definition
<<: *container_job_definition
allow_failure: true
except:
variables:
- $CI_PROJECT_NAMESPACE != libvirt
only:
- master
correct?
> > As for slowing down builds, that still only applies to the first
> > build after Dockerfiles are updated, so I don't think it ultimately
> > matters very much.
>
> I'd expect a rebiuld if the distro base image changes which could
> be fairly often for the rawhide like distros.
This advantage might be cancelled out by the fact that only a limited
number of shared runners is available, so if for example we have
access to 5 runners, whether we run 6 or 10 jobs will make no
difference in terms of total pipeline completion time.
--
Andrea Bolognani / Red Hat / Virtualization
On Tue, Jun 02, 2020 at 02:45:30PM +0200, Andrea Bolognani wrote:
> On Tue, 2020-06-02 at 12:23 +0100, Daniel P. Berrangé wrote:
> > On Tue, Jun 02, 2020 at 01:10:08PM +0200, Andrea Bolognani wrote:
> > > On Tue, 2020-06-02 at 11:33 +0100, Daniel P. Berrangé wrote:
> > > > I don't think we should be building container images that we're not going
> > > > to be using in any of the jobs, as it can only ever slow down the build
> > > > overall.
> > >
> > > These same containers are also available for use outside of CI, eg.
> > > with 'make ci-build', so I think we should keep building them.
> >
> > That only needs them built on the master branch of the main repo
> > though, not every branch in every fork
>
> Fair enough. So what you're suggesting is something like
>
> .container_optional_job_template: &container_optional_job_definition
> <<: *container_job_definition
> allow_failure: true
> except:
> variables:
> - $CI_PROJECT_NAMESPACE != libvirt
> only:
> - master
>
> correct?
Perhaps just matching what we do with extra builds:
only:
- master
- /^ci-full-.*$/
so users can still get the full set of builds in their fork if they push
to certain branch.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
© 2016 - 2026 Red Hat, Inc.