From nobody Thu Apr 25 17:13:59 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1593195659; cv=none; d=zohomail.com; s=zohoarc; b=NSqy0cSTy+YQDeh+sNlNlvQr9skA1xP40etv7vsRSDm2tRuM8Brugvq0Y6be867YRAVz7BibcH8WrtqQ0idAkMvmMeDtzQAcbAAKi+YXD+5+PB4jwhhntRKInmV7BJy3vk8NvmGER2CtlT+/dRO99OZTup6xjNvH4U3M84BvBx4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1593195659; h=Content-Type:Content-Transfer-Encoding:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=TER6QwEPxpNLH/V+90ni1QRmpF/dnkImvWxcQGijSpY=; b=acJW+MSmskMuCNTyLbwpgzGiTfmgjHG0a+EZ8QdN7GeqqwmBdjvfb04K7mDU2zCW5QYKri2yaknTspIS6FqR95FbOpicyBk/HB2PyuQvrS9EQYCjsBCuBebxJfX3noToOcDb/QZPKTMmYqvfJdcV9tgXh9k4URn8KleFlvFWGwk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1593195659455861.9277828045032; Fri, 26 Jun 2020 11:20:59 -0700 (PDT) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-283-Dg8ALQ5APJCNcFiXDPK38w-1; Fri, 26 Jun 2020 14:20:55 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CBC388015F5; Fri, 26 Jun 2020 18:20:49 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 516985D9DA; Fri, 26 Jun 2020 18:20:49 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2C9DA875BA; Fri, 26 Jun 2020 18:20:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 05QIKl3Z024584 for ; Fri, 26 Jun 2020 14:20:47 -0400 Received: by smtp.corp.redhat.com (Postfix) id 0EA951001B07; Fri, 26 Jun 2020 18:20:47 +0000 (UTC) Received: from kinshicho.usersys.redhat.com (unknown [10.40.195.50]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E973610013D2 for ; Fri, 26 Jun 2020 18:20:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593195658; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=TER6QwEPxpNLH/V+90ni1QRmpF/dnkImvWxcQGijSpY=; b=W+T9irirmiB18TcgkFGe/Wpnr5YNQFTm5g3sct+uTrck7mbL79OqebXukTL+DTKoupOjs/ mQbnsPXtX50rAR2iDXsH0e1mxF/5QZstfufW0m3aDGjcRTRtZWOBFt3zbMqnygih6pKVMk 1Ln8BCe31KrSU8d8w27x8dw6vCcefVM= X-MC-Unique: Dg8ALQ5APJCNcFiXDPK38w-1 From: Andrea Bolognani To: libvir-list@redhat.com Subject: [libvirt PATCH] ci: Run all jobs, for all branches, all the time Date: Fri, 26 Jun 2020 20:20:40 +0200 Message-Id: <20200626182040.929942-1-abologna@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: libvir-list@redhat.com X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" After recent changes (increasing the parallelism of the pipeline by reducing the number of stages, introducing FreeBSD builds that take longer than any other job), the difference between running the full pipeline or a reduced one has basically disappeared: in both cases, the completion time is around 25-35 minutes depending on whether containers need to be rebuilt and how many shared runners are available. Reduce the complexity of our .gitlab-ci.yml and make things simpler for contributors by simply always running all jobs. Signed-off-by: Andrea Bolognani Reviewed-by: Daniel P. Berrang=C3=A9 --- .gitlab-ci.yml | 143 ++++++++++++++++++------------------------------- 1 file changed, 53 insertions(+), 90 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6cb910b0fa..e6eb2f9905 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -15,8 +15,7 @@ stages: =20 # Common templates =20 -# Containers that are necessary for the default pipeline -.container_default_job_template: &container_default_job_definition +.container_job_template: &container_job_definition image: docker:stable stage: containers services: @@ -33,23 +32,15 @@ stages: after_script: - docker logout =20 -# Containers that are only needed for the full pipeline -.container_extra_job_template: &container_extra_job_definition - <<: *container_default_job_definition - only: - - master - - /^ci-full-.*$/ - # We build many containers which can be useful to debug problems but are n= ot # needed for the pipeline itself to complete: those sometimes fail, and wh= en # that happens it's mostly because of temporary issues with Debian sid. We # don't want those failures to affect the overall pipeline status .container_optional_job_template: &container_optional_job_definition - <<: *container_extra_job_definition + <<: *container_job_definition allow_failure: true =20 -# Default native build jobs that are always run -.native_build_default_job_template: &native_build_default_job_definition +.native_build_job_template: &native_build_job_definition stage: builds image: $CI_REGISTRY_IMAGE/ci-$NAME:latest cache: @@ -64,18 +55,10 @@ stages: - ../autogen.sh || (cat config.log && exit 1) - $MAKE distcheck =20 -# Extra native build jobs that are only run post-merge, or -# when code is pushed to a branch with "ci-full-" name prefix -.native_build_extra_job_template: &native_build_extra_job_definition - <<: *native_build_default_job_definition - only: - - master - - /^ci-full-.*$/ - # Jobs that we delegate to Cirrus CI because they require an operating # system other than Linux. These jobs will only run if the required # setup has been performed on the GitLab account (see ci/README.rst). -.cirrus_build_default_job_template: &cirrus_build_default_job_definition +.cirrus_build_job_template: &cirrus_build_job_definition stage: builds image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master script: @@ -85,19 +68,7 @@ stages: - $CIRRUS_GITHUB_REPO - $CIRRUS_API_TOKEN =20 -.cirrus_build_extra_job_template: &cirrus_build_extra_job_definition - <<: *cirrus_build_default_job_definition - only: - variables: - - $CIRRUS_GITHUB_REPO - - $CIRRUS_API_TOKEN - refs: - - master - - /^ci-full-.*$/ - - -# Default cross build jobs that are always run -.cross_build_default_job_template: &cross_build_default_job_definition +.cross_build_default_job_template: &cross_build_job_definition stage: builds image: $CI_REGISTRY_IMAGE/ci-$NAME-cross-$CROSS:latest cache: @@ -112,74 +83,66 @@ stages: - ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1) - $MAKE =20 -# Extra cross build jobs that are only run post-merge, or -# when code is pushed to a branch with "ci-full-" name prefix -.cross_build_extra_job_template: &cross_build_extra_job_definition - <<: *cross_build_default_job_definition - only: - - master - - /^ci-full-.*$/ - =20 # Native container build jobs =20 x64-centos-7-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: centos-7 =20 x64-centos-8-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: centos-8 =20 x64-centos-stream-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: centos-stream =20 x64-debian-9-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-9 =20 x64-debian-10-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: debian-10 =20 x64-debian-sid-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-sid =20 x64-fedora-31-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: fedora-31 =20 x64-fedora-32-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: fedora-32 =20 x64-fedora-rawhide-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: fedora-rawhide =20 x64-opensuse-151-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: opensuse-151 =20 x64-ubuntu-1804-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: ubuntu-1804 =20 x64-ubuntu-2004-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: ubuntu-2004 =20 @@ -192,7 +155,7 @@ aarch64-debian-9-container: NAME: debian-9-cross-aarch64 =20 armv6l-debian-9-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-9-cross-armv6l =20 @@ -202,12 +165,12 @@ armv7l-debian-9-container: NAME: debian-9-cross-armv7l =20 mips-debian-9-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-9-cross-mips =20 mips64el-debian-9-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-9-cross-mips64el =20 @@ -227,7 +190,7 @@ s390x-debian-9-container: NAME: debian-9-cross-s390x =20 aarch64-debian-10-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-10-cross-aarch64 =20 @@ -257,7 +220,7 @@ mips64el-debian-10-container: NAME: debian-10-cross-mips64el =20 mipsel-debian-10-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-10-cross-mipsel =20 @@ -267,7 +230,7 @@ ppc64le-debian-10-container: NAME: debian-10-cross-ppc64le =20 s390x-debian-10-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: debian-10-cross-s390x =20 @@ -282,12 +245,12 @@ armv6l-debian-sid-container: NAME: debian-sid-cross-armv6l =20 armv7-ldebian-sid-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: debian-sid-cross-armv7l =20 i686-debian-sid-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-sid-cross-i686 =20 @@ -302,7 +265,7 @@ mipsel-debian-sid-container: NAME: debian-sid-cross-mipsel =20 ppc64le-debian-sid-container: - <<: *container_extra_job_definition + <<: *container_job_definition variables: NAME: debian-sid-cross-ppc64le =20 @@ -312,12 +275,12 @@ s390x-debian-sid-container: NAME: debian-sid-cross-s390x =20 mingw32-fedora-rawhide-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: fedora-rawhide-cross-mingw32 =20 mingw64-fedora-rawhide-container: - <<: *container_default_job_definition + <<: *container_job_definition variables: NAME: fedora-rawhide-cross-mingw64 =20 @@ -325,72 +288,72 @@ mingw64-fedora-rawhide-container: # Native architecture build + test jobs =20 x64-debian-9: - <<: *native_build_extra_job_definition + <<: *native_build_job_definition variables: NAME: debian-9 =20 x64-debian-10: - <<: *native_build_default_job_definition + <<: *native_build_job_definition variables: NAME: debian-10 =20 x64-debian-sid: - <<: *native_build_extra_job_definition + <<: *native_build_job_definition variables: NAME: debian-sid =20 x64-centos-7: - <<: *native_build_default_job_definition + <<: *native_build_job_definition variables: NAME: centos-7 =20 x64-centos-8: - <<: *native_build_extra_job_definition + <<: *native_build_job_definition variables: NAME: centos-8 =20 x64-centos-stream: - <<: *native_build_extra_job_definition + <<: *native_build_job_definition variables: NAME: centos-stream =20 x64-fedora-31: - <<: *native_build_extra_job_definition + <<: *native_build_job_definition variables: NAME: fedora-31 =20 x64-fedora-32: - <<: *native_build_default_job_definition + <<: *native_build_job_definition variables: NAME: fedora-32 =20 x64-fedora-rawhide: - <<: *native_build_default_job_definition + <<: *native_build_job_definition variables: NAME: fedora-rawhide =20 x64-opensuse-151: - <<: *native_build_default_job_definition + <<: *native_build_job_definition variables: NAME: opensuse-151 =20 x64-ubuntu-1804: - <<: *native_build_extra_job_definition + <<: *native_build_job_definition variables: NAME: ubuntu-1804 =20 x64-ubuntu-2004: - <<: *native_build_default_job_definition + <<: *native_build_job_definition variables: NAME: ubuntu-2004 =20 x64-freebsd-12-build: - <<: *cirrus_build_default_job_definition + <<: *cirrus_build_job_definition variables: NAME: freebsd-12 =20 x64-macos-1015-build: - <<: *cirrus_build_default_job_definition + <<: *cirrus_build_job_definition variables: NAME: macos-1015 =20 @@ -398,67 +361,67 @@ x64-macos-1015-build: # Cross compiled build jobs =20 armv6l-debian-9: - <<: *cross_build_extra_job_definition + <<: *cross_build_job_definition variables: NAME: debian-9 CROSS: armv6l =20 mips64el-debian-9: - <<: *cross_build_extra_job_definition + <<: *cross_build_job_definition variables: NAME: debian-9 CROSS: mips64el =20 mips-debian-9: - <<: *cross_build_extra_job_definition + <<: *cross_build_job_definition variables: NAME: debian-9 CROSS: mips =20 aarch64-debian-10: - <<: *cross_build_extra_job_definition + <<: *cross_build_job_definition variables: NAME: debian-10 CROSS: aarch64 =20 mipsel-debian-10: - <<: *cross_build_extra_job_definition + <<: *cross_build_job_definition variables: NAME: debian-10 CROSS: mipsel =20 s390x-debian-10: - <<: *cross_build_default_job_definition + <<: *cross_build_job_definition variables: NAME: debian-10 CROSS: s390x =20 armv7l-debian-sid: - <<: *cross_build_default_job_definition + <<: *cross_build_job_definition variables: NAME: debian-sid CROSS: armv7l =20 i686-debian-sid: - <<: *cross_build_extra_job_definition + <<: *cross_build_job_definition variables: NAME: debian-sid CROSS: i686 =20 ppc64le-debian-sid: - <<: *cross_build_extra_job_definition + <<: *cross_build_job_definition variables: NAME: debian-sid CROSS: ppc64le =20 mingw32-fedora-rawhide: - <<: *cross_build_default_job_definition + <<: *cross_build_job_definition variables: NAME: fedora-rawhide CROSS: mingw32 =20 mingw64-fedora-rawhide: - <<: *cross_build_default_job_definition + <<: *cross_build_job_definition variables: NAME: fedora-rawhide CROSS: mingw64 --=20 2.25.4