The python build needs to validate two axis

 - A variety of libvirt versions
 - A variety of python versions

We get coverage for both these axis by running a build against the
distro provided libvirt packages. All that is then missing is a build
against the latest libvirt git master, which only needs to be run on
a single distro, for which CentOS 8 is picked as a stable long life
base.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.yml                | 172 ++++++++++++++++++++++++++++++++++
 ci/libvirt-centos-7.dkr       |  85 +++++++++++++++++
 ci/libvirt-centos-8.dkr       |  64 +++++++++++++
 ci/libvirt-debian-10.dkr      |  55 +++++++++++
 ci/libvirt-debian-9.dkr       |  58 ++++++++++++
 ci/libvirt-debian-sid.dkr     |  55 +++++++++++
 ci/libvirt-fedora-30.dkr      |  52 ++++++++++
 ci/libvirt-fedora-31.dkr      |  52 ++++++++++
 ci/libvirt-fedora-rawhide.dkr |  53 +++++++++++
 ci/libvirt-opensuse-151.dkr   |  54 +++++++++++
 ci/libvirt-ubuntu-1604.dkr    |  58 ++++++++++++
 ci/libvirt-ubuntu-1804.dkr    |  58 ++++++++++++
 ci/refresh                    |  21 +++++
 13 files changed, 837 insertions(+)
 create mode 100644 .gitlab-ci.yml
 create mode 100644 ci/libvirt-centos-7.dkr
 create mode 100644 ci/libvirt-centos-8.dkr
 create mode 100644 ci/libvirt-debian-10.dkr
 create mode 100644 ci/libvirt-debian-9.dkr
 create mode 100644 ci/libvirt-debian-sid.dkr
 create mode 100644 ci/libvirt-fedora-30.dkr
 create mode 100644 ci/libvirt-fedora-31.dkr
 create mode 100644 ci/libvirt-fedora-rawhide.dkr
 create mode 100644 ci/libvirt-opensuse-151.dkr
 create mode 100644 ci/libvirt-ubuntu-1604.dkr
 create mode 100644 ci/libvirt-ubuntu-1804.dkr
 create mode 100755 ci/refresh

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 0000000..bfcd31c
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,172 @@
+
+stages:
+  - prebuild
+  - containers
+  - build
+  - docs
+
+.container_job_template: &container_job_definition
+  image: docker:stable
+  stage: containers
+  services:
+    - docker:dind
+  before_script:
+    - export TAG="${CI_REGISTRY_IMAGE}/buildenv-${NAME}:latest"
+    - export COMMON_TAG="${CI_REGISTRY}/libvirt/libvirt-perl/buildenv-${NAME}:latest"
+    - docker info
+    - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD}
+  script:
+    - docker pull ${TAG} || docker pull ${COMMON_TAG} || true
+    - docker build --cache-from ${TAG} --cache-from ${COMMON_TAG} --tag ${TAG} -f ci/libvirt-${NAME}.dkr ci
+    - docker push ${TAG}
+  after_script:
+    - docker logout
+
+.build_git_job_template: &build_git_job_definition
+  stage: build
+  before_script:
+    - export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
+    - export SCRATCH_DIR="/tmp/scratch"
+    - export VROOT="${SCRATCH_DIR}/vroot"
+    - export LD_LIBRARY_PATH="${VROOT}/lib"
+    - export PATH="${PATH}:${VROOT}/bin"
+    - export PKG_CONFIG_PATH="${VROOT}/lib/pkgconfig"
+  script:
+    - pushd ${PWD}
+    - mkdir -p ${SCRATCH_DIR}
+    - cd ${SCRATCH_DIR}
+    - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git src
+    - mkdir build
+    - cd build
+    - ../src/autogen.sh --prefix=${VROOT} --without-libvirtd
+    - make install
+    - popd
+    - ${PYTHON} setup.py build
+    - ${PYTHON} setup.py test
+
+.build_dist_job_template: &build_dist_job_definition
+  stage: build
+  before_script:
+    - export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
+    - export TEST_MAINTAINER=1
+  script:
+    - ${PYTHON} setup.py build
+    - ${PYTHON} setup.py test
+
+# Check that all commits are signed-off for the DCO.
+# Skip on "libvirt" namespace, since we only need to run
+# this test on developer's personal forks from which
+# merge requests are submitted
+check-dco:
+  stage: prebuild
+  image: registry.gitlab.com/libvirt/libvirt-ci/check-dco:master
+  script:
+    - /check-dco
+  except:
+    variables:
+      - $CI_PROJECT_NAMESPACE == 'libvirt'
+
+ctr-centos-7:
+  <<: *container_job_definition
+  variables:
+    NAME: centos-7
+
+ctr-centos-8:
+  <<: *container_job_definition
+  variables:
+    NAME: centos-8
+
+ctr-debian-9:
+  <<: *container_job_definition
+  variables:
+    NAME: debian-9
+
+ctr-debian-10:
+  <<: *container_job_definition
+  variables:
+    NAME: debian-10
+
+ctr-debian-sid:
+  <<: *container_job_definition
+  variables:
+    NAME: debian-sid
+
+ctr-fedora-30:
+  <<: *container_job_definition
+  variables:
+    NAME: fedora-30
+
+ctr-fedora-31:
+  <<: *container_job_definition
+  variables:
+    NAME: fedora-31
+
+ctr-fedora-rawhide:
+  <<: *container_job_definition
+  variables:
+    NAME: fedora-rawhide
+
+ctr-opensuse-151:
+  <<: *container_job_definition
+  variables:
+    NAME: opensuse-151
+
+ctr-ubuntu-1604:
+  <<: *container_job_definition
+  variables:
+    NAME: ubuntu-1604
+
+ctr-ubuntu-1804:
+  <<: *container_job_definition
+  variables:
+    NAME: ubuntu-1804
+
+
+build-git-centos-8:
+  <<: *build_git_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-centos-8:latest
+
+
+build-dist-centos-7:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-centos-7:latest
+
+build-dist-centos-8:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-centos-8:latest
+
+build-dist-debian-9:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-debian-9:latest
+
+build-dist-debian-10:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-debian-10:latest
+
+build-dist-debian-sid:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-debian-sid:latest
+
+build-dist-fedora-30:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-fedora-30:latest
+
+build-dist-fedora-31:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-fedora-31:latest
+
+build-dist-fedora-rawhide:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-fedora-rawhide:latest
+
+build-dist-opensuse-151:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-opensuse-151:latest
+
+build-dist-ubuntu-1604:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-ubuntu-1604:latest
+
+build-dist-ubuntu-1804:
+  <<: *build_dist_job_definition
+  image: ${CI_REGISTRY_IMAGE}/buildenv-ubuntu-1804:latest
diff --git a/ci/libvirt-centos-7.dkr b/ci/libvirt-centos-7.dkr
new file mode 100644
index 0000000..9ff638b
--- /dev/null
+++ b/ci/libvirt-centos-7.dkr
@@ -0,0 +1,85 @@
+FROM centos:7
+
+RUN echo -e '[openvz]\n\
+name=OpenVZ addons\n\
+baseurl=https://download.openvz.org/virtuozzo/releases/openvz-7.0.11-235/x86_64/os/\n\
+enabled=1\n\
+gpgcheck=1\n\
+skip_if_unavailable=0\n\
+metadata_expire=6h\n\
+priority=90\n\
+includepkgs=libprl*' > /etc/yum.repos.d/openvz.repo && \
+    echo -e '-----BEGIN PGP PUBLIC KEY BLOCK-----\n\
+Version: GnuPG v2.0.22 (GNU/Linux)\n\
+\n\
+mI0EVl80nQEEAKrEeyeTCwrzS9kYedZ/sAc/GUqlb81C7pA9SaR3fyck5mVw1Ogk\n\
+YdmNBPM2kY7QDxR9F0EpSpnxSCAXZXugsQ8KzZ0DRLVeBDQyGs9IGK5hI0zzxIil\n\
+BzfvIexLiQQhLy7YlIi8Jt/uUqKkW0pIMNMGcduY97VATtczpncpkmSzABEBAAG0\n\
+SFZpcnR1b3p6byBUZWFtIChHUEcga2V5IHNpZ25hdHVyZSBmb3IgcGFja2FnZXMp\n\
+IDxzZWN1cml0eUB2aXJ0dW96em8uY29tPoi5BBMBAgAjBQJWXzSdAhsDBwsJCAcD\n\
+AgEGFQgCCQoLBBYCAwECHgECF4AACgkQygt9GUTNrSruIgP/er70Eyo73A1gfrjv\n\
+oPUkyo4rslVRZu3qqCwoMFtJc/Z/UxWgEka1buorlcGLa6eO/EZ49c0n+KGa4Kvt\n\
+EUboIq0yEu5i0FyAj92ifm+hNhoAbGfm0cZ4/fD0oGr3l8OsQo4+iHX4xAPwFe7Y\n\
+zABuB8I1ZDZ4OIp5tDfTTuF2LT24jQRWXzSdAQQAog2Aqb+Ptl68O7cQhWLjVGkj\n\
+yyigZrdeReLx3HloKJPBeQ/kA6uvMJc/IYS3uppMWXv9v+QenS6uhP1TUJ2k9FvM\n\
+t94MQZfALN7Vpf8AF+UeWu4Ru+y4BNzcFhrPhIFNFChOR2QqW6FkgE57D9I177NC\n\
+oJMyrlNe8wcGa178An8AEQEAAYifBBgBAgAJBQJWXzSdAhsMAAoJEMoLfRlEza0q\n\
+bKwD/3+OFVIEXnIv5XgdGRNX5fHggsUN1bb8gva7HANRlKdd4LD8foDM3F/yv/3V\n\
+igG14D5EjKz56SaBDNgiI4++hOzb2M8jhAsR86jxkXFrrP1U3ZNRKg6av9DPFAPS\n\
+WEiJKtQrZDJloqtyi/mmRa1VsV7RYR0VPJjhK/R8EQ7Ysshy\n\
+=fRMg\n\
+-----END PGP PUBLIC KEY BLOCK-----' > /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \
+    rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \
+    yum install -y epel-release && \
+    yum update -y && \
+    yum install -y \
+        autoconf \
+        automake \
+        bash \
+        bash-completion \
+        ca-certificates \
+        ccache \
+        chrony \
+        gcc \
+        gdb \
+        gettext \
+        gettext-devel \
+        git \
+        glibc-common \
+        glibc-devel \
+        libtool \
+        libvirt-devel \
+        lsof \
+        make \
+        net-tools \
+        ninja-build \
+        patch \
+        perl \
+        pkgconfig \
+        python3 \
+        python3-pip \
+        python3-setuptools \
+        python36-devel \
+        python36-lxml \
+        python36-nose \
+        rpm-build \
+        screen \
+        strace \
+        sudo \
+        vim && \
+    yum autoremove -y && \
+    yum clean all -y && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+RUN pip3 install \
+         meson==0.49.0
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja-build"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-centos-8.dkr b/ci/libvirt-centos-8.dkr
new file mode 100644
index 0000000..bf3ec47
--- /dev/null
+++ b/ci/libvirt-centos-8.dkr
@@ -0,0 +1,64 @@
+FROM centos:8
+
+RUN dnf install 'dnf-command(config-manager)' -y && \
+    dnf config-manager --set-enabled PowerTools -y && \
+    dnf install -y epel-release && \
+    dnf update -y && \
+    dnf install -y \
+        autoconf \
+        automake \
+        bash \
+        bash-completion \
+        ca-certificates \
+        ccache \
+        chrony \
+        gcc \
+        gdb \
+        gettext \
+        gettext-devel \
+        git \
+        glib2-devel \
+        glibc-devel \
+        glibc-langpack-en \
+        gnutls-devel \
+        libnl3-devel \
+        libpcap-devel \
+        libtirpc-devel \
+        libtool \
+        libvirt-devel \
+        libxml2 \
+        libxml2-devel \
+        libxslt \
+        lsof \
+        make \
+        meson \
+        net-tools \
+        ninja-build \
+        patch \
+        perl \
+        pkgconfig \
+        python3 \
+        python3-devel \
+        python3-docutils \
+        python3-lxml \
+        python3-nose \
+        python3-setuptools \
+        rpcgen \
+        rpm-build \
+        screen \
+        strace \
+        sudo \
+        vim && \
+    dnf autoremove -y && \
+    dnf clean all -y && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-debian-10.dkr b/ci/libvirt-debian-10.dkr
new file mode 100644
index 0000000..7d98dac
--- /dev/null
+++ b/ci/libvirt-debian-10.dkr
@@ -0,0 +1,55 @@
+FROM debian:10
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get dist-upgrade -y && \
+    apt-get install --no-install-recommends -y \
+            autoconf \
+            automake \
+            autopoint \
+            bash \
+            bash-completion \
+            ca-certificates \
+            ccache \
+            chrony \
+            gcc \
+            gdb \
+            gettext \
+            git \
+            libc6-dev \
+            libtool \
+            libtool-bin \
+            libvirt-dev \
+            locales \
+            lsof \
+            make \
+            meson \
+            net-tools \
+            ninja-build \
+            patch \
+            perl \
+            pkgconf \
+            python3 \
+            python3-dev \
+            python3-lxml \
+            python3-nose \
+            python3-setuptools \
+            screen \
+            strace \
+            sudo \
+            vim && \
+    apt-get autoremove -y && \
+    apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-debian-9.dkr b/ci/libvirt-debian-9.dkr
new file mode 100644
index 0000000..1c565fa
--- /dev/null
+++ b/ci/libvirt-debian-9.dkr
@@ -0,0 +1,58 @@
+FROM debian:9
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get dist-upgrade -y && \
+    apt-get install --no-install-recommends -y \
+            autoconf \
+            automake \
+            autopoint \
+            bash \
+            bash-completion \
+            ca-certificates \
+            ccache \
+            chrony \
+            gcc \
+            gdb \
+            gettext \
+            git \
+            libc6-dev \
+            libtool \
+            libtool-bin \
+            libvirt-dev \
+            locales \
+            lsof \
+            make \
+            net-tools \
+            ninja-build \
+            patch \
+            perl \
+            pkgconf \
+            python3 \
+            python3-dev \
+            python3-lxml \
+            python3-nose \
+            python3-pip \
+            python3-setuptools \
+            screen \
+            strace \
+            sudo \
+            vim && \
+    apt-get autoremove -y && \
+    apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+RUN pip3 install \
+         meson==0.49.0
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-debian-sid.dkr b/ci/libvirt-debian-sid.dkr
new file mode 100644
index 0000000..a9f0ee4
--- /dev/null
+++ b/ci/libvirt-debian-sid.dkr
@@ -0,0 +1,55 @@
+FROM debian:sid
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get dist-upgrade -y && \
+    apt-get install --no-install-recommends -y \
+            autoconf \
+            automake \
+            autopoint \
+            bash \
+            bash-completion \
+            ca-certificates \
+            ccache \
+            chrony \
+            gcc \
+            gdb \
+            gettext \
+            git \
+            libc6-dev \
+            libtool \
+            libtool-bin \
+            libvirt-dev \
+            locales \
+            lsof \
+            make \
+            meson \
+            net-tools \
+            ninja-build \
+            patch \
+            perl \
+            pkgconf \
+            python3 \
+            python3-dev \
+            python3-lxml \
+            python3-nose \
+            python3-setuptools \
+            screen \
+            strace \
+            sudo \
+            vim && \
+    apt-get autoremove -y && \
+    apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-fedora-30.dkr b/ci/libvirt-fedora-30.dkr
new file mode 100644
index 0000000..63cd0f2
--- /dev/null
+++ b/ci/libvirt-fedora-30.dkr
@@ -0,0 +1,52 @@
+FROM fedora:30
+
+RUN dnf update -y && \
+    dnf install -y \
+        autoconf \
+        automake \
+        bash \
+        bash-completion \
+        ca-certificates \
+        ccache \
+        chrony \
+        cppi \
+        gcc \
+        gdb \
+        gettext \
+        gettext-devel \
+        git \
+        glibc-devel \
+        glibc-langpack-en \
+        libtool \
+        libvirt-devel \
+        lsof \
+        make \
+        meson \
+        net-tools \
+        ninja-build \
+        patch \
+        perl \
+        pkgconfig \
+        python3 \
+        python3-devel \
+        python3-lxml \
+        python3-nose \
+        python3-setuptools \
+        rpm-build \
+        screen \
+        strace \
+        sudo \
+        vim && \
+    dnf autoremove -y && \
+    dnf clean all -y && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-fedora-31.dkr b/ci/libvirt-fedora-31.dkr
new file mode 100644
index 0000000..3307e5c
--- /dev/null
+++ b/ci/libvirt-fedora-31.dkr
@@ -0,0 +1,52 @@
+FROM fedora:31
+
+RUN dnf update -y && \
+    dnf install -y \
+        autoconf \
+        automake \
+        bash \
+        bash-completion \
+        ca-certificates \
+        ccache \
+        chrony \
+        cppi \
+        gcc \
+        gdb \
+        gettext \
+        gettext-devel \
+        git \
+        glibc-devel \
+        glibc-langpack-en \
+        libtool \
+        libvirt-devel \
+        lsof \
+        make \
+        meson \
+        net-tools \
+        ninja-build \
+        patch \
+        perl \
+        pkgconfig \
+        python3 \
+        python3-devel \
+        python3-lxml \
+        python3-nose \
+        python3-setuptools \
+        rpm-build \
+        screen \
+        strace \
+        sudo \
+        vim && \
+    dnf autoremove -y && \
+    dnf clean all -y && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-fedora-rawhide.dkr b/ci/libvirt-fedora-rawhide.dkr
new file mode 100644
index 0000000..475c757
--- /dev/null
+++ b/ci/libvirt-fedora-rawhide.dkr
@@ -0,0 +1,53 @@
+FROM fedora:rawhide
+
+RUN dnf update -y --nogpgcheck fedora-gpg-keys && \
+    dnf update -y && \
+    dnf install -y \
+        autoconf \
+        automake \
+        bash \
+        bash-completion \
+        ca-certificates \
+        ccache \
+        chrony \
+        cppi \
+        gcc \
+        gdb \
+        gettext \
+        gettext-devel \
+        git \
+        glibc-devel \
+        glibc-langpack-en \
+        libtool \
+        libvirt-devel \
+        lsof \
+        make \
+        meson \
+        net-tools \
+        ninja-build \
+        patch \
+        perl \
+        pkgconfig \
+        python3 \
+        python3-devel \
+        python3-lxml \
+        python3-nose \
+        python3-setuptools \
+        rpm-build \
+        screen \
+        strace \
+        sudo \
+        vim && \
+    dnf autoremove -y && \
+    dnf clean all -y && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-opensuse-151.dkr b/ci/libvirt-opensuse-151.dkr
new file mode 100644
index 0000000..526aadf
--- /dev/null
+++ b/ci/libvirt-opensuse-151.dkr
@@ -0,0 +1,54 @@
+FROM opensuse/leap:15.1
+
+RUN zypper update -y && \
+    zypper install -y \
+        autoconf \
+        automake \
+        bash \
+        bash-completion \
+        ca-certificates \
+        ccache \
+        chrony \
+        cppi \
+        gcc \
+        gdb \
+        gettext \
+        gettext-devel \
+        git \
+        glibc-devel \
+        glibc-locale \
+        libtool \
+        libvirt-devel \
+        lsof \
+        make \
+        net-tools \
+        ninja \
+        patch \
+        perl \
+        pkgconfig \
+        python3 \
+        python3-devel \
+        python3-lxml \
+        python3-nose \
+        python3-pip \
+        python3-setuptools \
+        rpm-build \
+        screen \
+        strace \
+        sudo \
+        vim && \
+    zypper clean --all && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+RUN pip3 install \
+         meson==0.49.0
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-ubuntu-1604.dkr b/ci/libvirt-ubuntu-1604.dkr
new file mode 100644
index 0000000..9652cf7
--- /dev/null
+++ b/ci/libvirt-ubuntu-1604.dkr
@@ -0,0 +1,58 @@
+FROM ubuntu:16.04
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get dist-upgrade -y && \
+    apt-get install --no-install-recommends -y \
+            autoconf \
+            automake \
+            autopoint \
+            bash \
+            bash-completion \
+            ca-certificates \
+            ccache \
+            chrony \
+            gcc \
+            gdb \
+            gettext \
+            git \
+            libc6-dev \
+            libtool \
+            libtool-bin \
+            libvirt-dev \
+            locales \
+            lsof \
+            make \
+            net-tools \
+            ninja-build \
+            patch \
+            perl \
+            pkgconf \
+            python3 \
+            python3-dev \
+            python3-lxml \
+            python3-nose \
+            python3-pip \
+            python3-setuptools \
+            screen \
+            strace \
+            sudo \
+            vim && \
+    apt-get autoremove -y && \
+    apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+RUN pip3 install \
+         meson==0.49.0
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/libvirt-ubuntu-1804.dkr b/ci/libvirt-ubuntu-1804.dkr
new file mode 100644
index 0000000..7ec35d8
--- /dev/null
+++ b/ci/libvirt-ubuntu-1804.dkr
@@ -0,0 +1,58 @@
+FROM ubuntu:18.04
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+    apt-get update && \
+    apt-get dist-upgrade -y && \
+    apt-get install --no-install-recommends -y \
+            autoconf \
+            automake \
+            autopoint \
+            bash \
+            bash-completion \
+            ca-certificates \
+            ccache \
+            chrony \
+            gcc \
+            gdb \
+            gettext \
+            git \
+            libc6-dev \
+            libtool \
+            libtool-bin \
+            libvirt-dev \
+            locales \
+            lsof \
+            make \
+            net-tools \
+            ninja-build \
+            patch \
+            perl \
+            pkgconf \
+            python3 \
+            python3-dev \
+            python3-lxml \
+            python3-nose \
+            python3-pip \
+            python3-setuptools \
+            screen \
+            strace \
+            sudo \
+            vim && \
+    apt-get autoremove -y && \
+    apt-get autoclean -y && \
+    sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+    dpkg-reconfigure locales && \
+    mkdir -p /usr/libexec/ccache-wrappers && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+    ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc)
+
+RUN pip3 install \
+         meson==0.49.0
+
+ENV LANG "en_US.UTF-8"
+
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
diff --git a/ci/refresh b/ci/refresh
new file mode 100755
index 0000000..2e7b3be
--- /dev/null
+++ b/ci/refresh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+if test -z "$1"
+then
+  echo "syntax: $0 PATH-TO-LCITOOL"
+  exit 1
+fi
+
+LCITOOL=$1
+
+HOSTS=$(${LCITOOL} hosts | grep -v freebsd)
+
+for host in ${HOSTS}
+do
+    if test "$host" == "libvirt-centos-8"
+    then
+	${LCITOOL} dockerfile $host libvirt-minimal,libvirt-dist,libvirt-python > $host.dkr
+    else
+	${LCITOOL} dockerfile $host libvirt-dist,libvirt-python > $host.dkr
+    fi
+done
-- 
2.25.4

On Wed, 2020-04-29 at 14:36 +0100, Daniel P. Berrangé wrote:
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,172 @@
> +.container_job_template: &container_job_definition
> +  image: docker:stable
> +  stage: containers
> +  services:
> +    - docker:dind
> +  before_script:
> +    - export TAG="${CI_REGISTRY_IMAGE}/buildenv-${NAME}:latest"
> +    - export COMMON_TAG="${CI_REGISTRY}/libvirt/libvirt-perl/buildenv-${NAME}:latest"

Since we're now storing images directly under the project's namespace
on registry.gitlab.com and not somewhere more public like Quay, where
people could conceivably look for a container image that can be used
to *run* libvirt and friends instead of build them, I think we can
safely drop the buildenv- prefix.

> +    - docker info
> +    - docker login registry.gitlab.com -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD}
> +  script:
> +    - docker pull ${TAG} || docker pull ${COMMON_TAG} || true
> +    - docker build --cache-from ${TAG} --cache-from ${COMMON_TAG} --tag ${TAG} -f ci/libvirt-${NAME}.dkr ci

Using a .dkr extension for Dockerfiles is very non-standard, and vim
for example doesn't recognize it as a Dockerfile because of that.
Please use either $OS.Dockerfile or, even better, $OS/Dockerfile,
which would avoid the need to use the -f option in the build command.

> +.build_git_job_template: &build_git_job_definition
> +  stage: build
> +  before_script:
> +    - export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> +    - export SCRATCH_DIR="/tmp/scratch"
> +    - export VROOT="${SCRATCH_DIR}/vroot"
> +    - export LD_LIBRARY_PATH="${VROOT}/lib"
> +    - export PATH="${PATH}:${VROOT}/bin"
> +    - export PKG_CONFIG_PATH="${VROOT}/lib/pkgconfig"
> +  script:
> +    - pushd ${PWD}
> +    - mkdir -p ${SCRATCH_DIR}
> +    - cd ${SCRATCH_DIR}
> +    - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git src
> +    - mkdir build
> +    - cd build
> +    - ../src/autogen.sh --prefix=${VROOT} --without-libvirtd
> +    - make install
> +    - popd
> +    - ${PYTHON} setup.py build
> +    - ${PYTHON} setup.py test

Style-wise, all the scripts introduced in this patch contain a lot of
unnecessary curly braces and could use a bit more quoting, so it'd be
nice (but not blocking) if you could clean that up.

> +.build_dist_job_template: &build_dist_job_definition
> +  stage: build
> +  before_script:
> +    - export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> +    - export TEST_MAINTAINER=1

Is TEST_MAINTAINER a thing for libvirt-python? The only reference to
it I could find in the libvirt-ci repo is in the job definition for
libvirt-perl.

> +ctr-centos-7:
> +  <<: *container_job_definition
> +  variables:
> +    NAME: centos-7

Not to keen on the ctr- prefix. I understand you're trying to keep
the job names short so that they all fit in the tiny boxes in the
pipeline overview page, but I don't think that should be an absolute
requirement considering that the first thing you're going to do when
you see a failure there is to click on the name of the job to see
more details, and when you do so the full name will be displayed
quite prominently in the top-right corner in a way that's actually
readable, unlike the cut-off low-constrast version that's on the
pipeline overview page...

> +build-git-centos-8:
> +  <<: *build_git_job_definition
> +  image: ${CI_REGISTRY_IMAGE}/buildenv-centos-8:latest

... and these slightly longer names already get cut off anyway.

[...]
> +++ b/ci/refresh
> @@ -0,0 +1,21 @@
> +#!/bin/sh
> +
> +if test -z "$1"
> +then
> +  echo "syntax: $0 PATH-TO-LCITOOL"
> +  exit 1
> +fi

I'd also check that whatever the path is pointing to is executable.

The indentation is also off - two spaces instead of four.

> +LCITOOL=$1
> +
> +HOSTS=$(${LCITOOL} hosts | grep -v freebsd)
> +
> +for host in ${HOSTS}
> +do
> +    if test "$host" == "libvirt-centos-8"

== is a bashism, use the portable = instead.

> +    then
> +	${LCITOOL} dockerfile $host libvirt-minimal,libvirt-dist,libvirt-python > $host.dkr
> +    else
> +	${LCITOOL} dockerfile $host libvirt-dist,libvirt-python > $host.dkr
> +    fi
> +done

The indentation is off here, I think some tabs sneaked in.


We should also add a short README that tells people what this
directory is all about and how to make changes to it, but we can come
up with something that works for all repos and push it everywhere as
a follow-up enhancement.


Overall, aside from the nits pointed out above, the idea is solid and
the implementation looks good.

-- 
Andrea Bolognani / Red Hat / Virtualization

On Thu, Apr 30, 2020 at 10:16:32AM +0200, Andrea Bolognani wrote:
> On Wed, 2020-04-29 at 14:36 +0100, Daniel P. Berrangé wrote:
> > +++ b/.gitlab-ci.yml
> > @@ -0,0 +1,172 @@
> > +.container_job_template: &container_job_definition
> > +  image: docker:stable
> > +  stage: containers
> > +  services:
> > +    - docker:dind
> > +  before_script:
> > +    - export TAG="${CI_REGISTRY_IMAGE}/buildenv-${NAME}:latest"
> > +    - export COMMON_TAG="${CI_REGISTRY}/libvirt/libvirt-perl/buildenv-${NAME}:latest"
> 
> Since we're now storing images directly under the project's namespace
> on registry.gitlab.com and not somewhere more public like Quay, where
> people could conceivably look for a container image that can be used
> to *run* libvirt and friends instead of build them, I think we can
> safely drop the buildenv- prefix.

The GitLab container registry is just as public, so I think the
rationale still stands. Especially since when users submit a
merge request, these containers are going to magically appear
in their fork. I think can simplify the prefix to "ci-" hough.


> > +.build_dist_job_template: &build_dist_job_definition
> > +  stage: build
> > +  before_script:
> > +    - export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> > +    - export TEST_MAINTAINER=1
> 
> Is TEST_MAINTAINER a thing for libvirt-python? The only reference to
> it I could find in the libvirt-ci repo is in the job definition for
> libvirt-perl.

I have another change for python that uses it, but havent sent
it yet.

> > +ctr-centos-7:
> > +  <<: *container_job_definition
> > +  variables:
> > +    NAME: centos-7
> 
> Not to keen on the ctr- prefix. I understand you're trying to keep
> the job names short so that they all fit in the tiny boxes in the
> pipeline overview page, but I don't think that should be an absolute
> requirement considering that the first thing you're going to do when
> you see a failure there is to click on the name of the job to see
> more details, and when you do so the full name will be displayed
> quite prominently in the top-right corner in a way that's actually
> readable, unlike the cut-off low-constrast version that's on the
> pipeline overview page...

I'll just switch to a "-container" suffix instead, so the most
important info is first.

> 
> > +build-git-centos-8:
> > +  <<: *build_git_job_definition
> > +  image: ${CI_REGISTRY_IMAGE}/buildenv-centos-8:latest
> 
> ... and these slightly longer names already get cut off anyway.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Thu, 2020-04-30 at 12:59 +0100, Daniel P. Berrangé wrote:
> On Thu, Apr 30, 2020 at 10:16:32AM +0200, Andrea Bolognani wrote:
> > On Wed, 2020-04-29 at 14:36 +0100, Daniel P. Berrangé wrote:
> > > +  before_script:
> > > +    - export TAG="${CI_REGISTRY_IMAGE}/buildenv-${NAME}:latest"
> > > +    - export COMMON_TAG="${CI_REGISTRY}/libvirt/libvirt-perl/buildenv-${NAME}:latest"
> > 
> > Since we're now storing images directly under the project's namespace
> > on registry.gitlab.com and not somewhere more public like Quay, where
> > people could conceivably look for a container image that can be used
> > to *run* libvirt and friends instead of build them, I think we can
> > safely drop the buildenv- prefix.
> 
> The GitLab container registry is just as public, so I think the
> rationale still stands. Especially since when users submit a
> merge request, these containers are going to magically appear
> in their fork. I think can simplify the prefix to "ci-" hough.

Sounds good.

> > > +.build_dist_job_template: &build_dist_job_definition
> > > +  stage: build
> > > +  before_script:
> > > +    - export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> > > +    - export TEST_MAINTAINER=1
> > 
> > Is TEST_MAINTAINER a thing for libvirt-python? The only reference to
> > it I could find in the libvirt-ci repo is in the job definition for
> > libvirt-perl.
> 
> I have another change for python that uses it, but havent sent
> it yet.

Okay, so maybe introduce its use in GitLab CI at the same time as
you add support for it in the project proper.

Additional thing that I failed to mention the first time around: you
only set this environment variable in build_dist_job_template, but
you should probably have it in build_git_job_template as well.

> > > +ctr-centos-7:
> > > +  <<: *container_job_definition
> > > +  variables:
> > > +    NAME: centos-7
> > 
> > Not to keen on the ctr- prefix. I understand you're trying to keep
> > the job names short so that they all fit in the tiny boxes in the
> > pipeline overview page, but I don't think that should be an absolute
> > requirement considering that the first thing you're going to do when
> > you see a failure there is to click on the name of the job to see
> > more details, and when you do so the full name will be displayed
> > quite prominently in the top-right corner in a way that's actually
> > readable, unlike the cut-off low-constrast version that's on the
> > pipeline overview page...
> 
> I'll just switch to a "-container" suffix instead, so the most
> important info is first.

Sounds good.

Please rename the build jobs accordingly at the same time.

-- 
Andrea Bolognani / Red Hat / Virtualization