From nobody Tue Apr 30 18:53:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592840099; cv=none; d=zohomail.com; s=zohoarc; b=cAaaGLu3EzqxjgcX6tHvu//09R31ure+6mXJ26VwNTM+cnPRbZj2ampU1/kA8td9FlocHBlel/4NSYmbhHc0zyR0x9thj2dKDFYBSZKFcNWz+0ddH93d4/cFuAkkqr3kEPtFKkxq3q7D8/q0JMnHlcYbwAClOyUV99vM0pPwKKw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592840099; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+C3phPrrj7sLxFUe/oknyaioR8bFPciwI/iQ7dAlpo8=; b=MU5fwlcyGcZEBP1Pf6cFu+wvkYuEbb955VLaCDBqSE1YKoQCve2i7r5MI0C2IkvCqcrp/GahQrW26OXcb2PkkTvfoEin5tY0gRBbclBQZA4Pg5R3zw9m+lxc/ckf3NLy1Z2AmQJxLYBdXX2ClOan2ZE00YkrdbJK4ZZ4pnqtofw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 159284009902479.77164894375301; Mon, 22 Jun 2020 08:34:59 -0700 (PDT) Received: from localhost ([::1]:57402 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jnOTI-00057B-Mc for importer@patchew.org; Mon, 22 Jun 2020 11:34:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57222) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jnORv-0004A5-8a for qemu-devel@nongnu.org; Mon, 22 Jun 2020 11:33:31 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:42675 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jnORt-000281-5d for qemu-devel@nongnu.org; Mon, 22 Jun 2020 11:33:30 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-166-QyGARkk-MfiMZsIc9NQhTg-1; Mon, 22 Jun 2020 11:33:26 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A14218585A2; Mon, 22 Jun 2020 15:33:25 +0000 (UTC) Received: from domokun.gsslab.fab.redhat.com (unknown [10.33.8.110]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22D4460BEC; Mon, 22 Jun 2020 15:33:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592840008; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+C3phPrrj7sLxFUe/oknyaioR8bFPciwI/iQ7dAlpo8=; b=SG7uL1K0oGqL+PgZQ+R06VZWhSqQ5OBemInlMWy1ClmzcGhaAs5AvOag2w6PPAKEESrZ3N XyG1ySH2j59q4in+bPVd1rGhCgBcmxuO0PBGX1il72UBirY9buRiiTHEWhOMVvB6uvSMF3 I3qXjjSKM8y93mdqpn2Pg4e+zJrnins= X-MC-Unique: QyGARkk-MfiMZsIc9NQhTg-1 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Subject: [PATCH RFC 1/3] gitlab: introduce explicit "container" and "build" stages Date: Mon, 22 Jun 2020 16:33:16 +0100 Message-Id: <20200622153318.751107-2-berrange@redhat.com> In-Reply-To: <20200622153318.751107-1-berrange@redhat.com> References: <20200622153318.751107-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/22 03:17:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laszlo Ersek , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Alex=20Benn=C3=A9e?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) If no stage is listed, jobs get put in an implicit "test" stage. Some jobs which create container images to be used by later stages are currently listed as in a "build" stages. Signed-off-by: Daniel P. Berrang=C3=A9 Acked-by: Laszlo Ersek Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Thomas Huth --- .gitlab-ci.d/edk2.yml | 3 ++- .gitlab-ci.d/opensbi.yml | 3 ++- .gitlab-ci.yml | 11 +++++++++++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.d/edk2.yml b/.gitlab-ci.d/edk2.yml index 088ba4b43a..d4e7dfcba6 100644 --- a/.gitlab-ci.d/edk2.yml +++ b/.gitlab-ci.d/edk2.yml @@ -1,5 +1,5 @@ docker-edk2: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci-edk2.yml @@ -24,6 +24,7 @@ docker-edk2: - docker push $IMAGE_TAG =20 build-edk2: + stage: build rules: # Only run this job when ... - changes: # ... roms/edk2/ is modified (submodule updated) - roms/edk2/* diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index dd051c0124..ec1c1f4cab 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b/.gitlab-ci.d/opensbi.yml @@ -1,5 +1,5 @@ docker-opensbi: - stage: build + stage: containers rules: # Only run this job when the Dockerfile is modified - changes: - .gitlab-ci-opensbi.yml @@ -24,6 +24,7 @@ docker-opensbi: - docker push $IMAGE_TAG =20 build-opensbi: + stage: build rules: # Only run this job when ... - changes: # ... roms/opensbi/ is modified (submodule updated) - roms/opensbi/* diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 349c77aa58..9fdc752ea6 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,3 +1,7 @@ +stages: + - containers + - build + include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' @@ -17,6 +21,7 @@ include: - JOBS=3D$(expr $(nproc) + 1) =20 build-system1: + stage: build image: ubuntu:19.10 <<: *before_script_apt script: @@ -31,6 +36,7 @@ build-system1: - make -j"$JOBS" check =20 build-system2: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -46,6 +52,7 @@ build-system2: - make -j"$JOBS" check =20 build-disabled: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -62,6 +69,7 @@ build-disabled: - make -j"$JOBS" check-qtest SPEED=3Dslow =20 build-tcg-disabled: + stage: build image: centos:8 <<: *before_script_dnf script: @@ -82,6 +90,7 @@ build-tcg-disabled: 260 261 262 263 264 270 272 273 277 279 =20 build-user: + stage: build <<: *before_script_apt script: - mkdir build @@ -92,6 +101,7 @@ build-user: - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user =20 build-clang: + stage: build image: fedora:latest <<: *before_script_dnf script: @@ -106,6 +116,7 @@ build-clang: - make -j"$JOBS" check =20 build-tci: + stage: build image: centos:8 <<: *before_script_dnf script: --=20 2.24.1 From nobody Tue Apr 30 18:53:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592840113; cv=none; d=zohomail.com; s=zohoarc; b=MrWsNWMv2EHpLBEwC69C8eRNJEOVs6hxtU5zal0TyWNBwlm5gm42Px36ptsoRrbPK8F90W8h3BJnM6C3EUy7/XeJT7za+rUV1Q/4wABxF2VOGqGvZ0/OUEW8uOd4ss2szEEL3NURNT3bk1dBSMLQN+PLP4Ivjjzo6YT+J3mug9M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592840113; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Kqh+q9O0KowAQ02SumcTcblwmOGWmfIZwx77yfwqX0A=; b=Z6AzClgQ+sR8Bz+4bZMRbWlQ0cU5z407fekHRB5i1Ma/oJSo9Xlz0JS+AKKXTxuvLvxwR789mZ0qwT4KMaGi58Su9E0lud4v42LcfFpJv2CGZJnNiAacG3yNJvVP+7tU4Ir4y+yZbAip6PbVQFgMxVueKbjjCnlajyIxjiVMjPM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592840113826158.40652907081244; Mon, 22 Jun 2020 08:35:13 -0700 (PDT) Received: from localhost ([::1]:57940 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jnOTY-0005Og-9c for importer@patchew.org; Mon, 22 Jun 2020 11:35:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jnORy-0004CX-LY for qemu-devel@nongnu.org; Mon, 22 Jun 2020 11:33:34 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:29142 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jnORw-00028j-PF for qemu-devel@nongnu.org; Mon, 22 Jun 2020 11:33:34 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-245--QiVR4gmO6OpFRXEtEulfA-1; Mon, 22 Jun 2020 11:33:30 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 215FF107B279; Mon, 22 Jun 2020 15:33:27 +0000 (UTC) Received: from domokun.gsslab.fab.redhat.com (unknown [10.33.8.110]) by smtp.corp.redhat.com (Postfix) with ESMTP id BDF7D60BEC; Mon, 22 Jun 2020 15:33:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592840011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Kqh+q9O0KowAQ02SumcTcblwmOGWmfIZwx77yfwqX0A=; b=H97AR8XztVfPIxAXq9QqYMj6a0ZjbCuvA+Gl2dLX8vWu1ADpt/JGC4ZSvX/u4flWVvZzSy 2sSQB3D4hfPUaFHEiCcnS32wLa0/hPdvmwGUGg3O9hTw8CimZsu0t58U9rQOG9c2P8FfLg T3zwTrHGOZkYGde6OrWnpD1vS/rn00E= X-MC-Unique: -QiVR4gmO6OpFRXEtEulfA-1 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Subject: [PATCH RFC 2/3] gitlab: build all container images during CI Date: Mon, 22 Jun 2020 16:33:17 +0100 Message-Id: <20200622153318.751107-3-berrange@redhat.com> In-Reply-To: <20200622153318.751107-1-berrange@redhat.com> References: <20200622153318.751107-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=berrange@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/22 02:57:26 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laszlo Ersek , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Alex=20Benn=C3=A9e?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrang=C3=A9 Acked-by: Thomas Huth Reviewed-by: Alex Benn=C3=A9e --- .gitlab-ci.d/containers.yml | 248 ++++++++++++++++++++++++++++++++++++ .gitlab-ci.yml | 3 + 2 files changed, 251 insertions(+) create mode 100644 .gitlab-ci.d/containers.yml diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml new file mode 100644 index 0000000000..ea1edbb196 --- /dev/null +++ b/.gitlab-ci.d/containers.yml @@ -0,0 +1,248 @@ + + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG=3D"$CI_REGISTRY_IMAGE/ci-$NAME:latest" + - export COMMON_TAG=3D"$CI_REGISTRY/qemu-project/qemu/ci-$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGI= STRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - sed -i -e "s,FROM qemu:,FROM $CI_REGISTRY_IMAGE/ci-," tests/docker/d= ockerfiles/$NAME.docker + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$= TAG" -f "tests/docker/dockerfiles/$NAME.docker" tests/docker/dockerfiles + - docker push "$TAG" + after_script: + - docker logout + +amd64-centos7-container: + <<: *container_job_definition + variables: + NAME: centos7 + +amd64-centos8-container: + <<: *container_job_definition + variables: + NAME: centos8 + +amd64-debian10-container: + <<: *container_job_definition + variables: + NAME: debian10 + +amd64-debian11-container: + <<: *container_job_definition + variables: + NAME: debian11 + +amd64-debian9-container: + <<: *container_job_definition + variables: + NAME: debian9 + +amd64-debian9-mxe-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian9-mxe + +alpha-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-alpha-cross + +amd64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64-cross + +amd64-debian-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-amd64 + +arm64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-arm64-cross + +arm64-test-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian11-container'] + variables: + NAME: debian-arm64-test-cross + +armel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armel-cross + +armhf-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-armhf-cross + +hppa-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-hppa-cross + +m68k-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-m68k-cross + +mips64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64-cross + +mips64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips64el-cross + +mips-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mips-cross + +mipsel-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-mipsel-cross + +powerpc-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-powerpc-cross + +ppc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64-cross + +ppc64el-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-ppc64el-cross + +riscv64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-riscv64-cross + +s390x-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-s390x-cross + +sh4-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sh4-cross + +sparc64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian10-container'] + variables: + NAME: debian-sparc64-cross + +tricore-debian-cross-container: + <<: *container_job_definition + stage: containers-layer2 + needs: ['amd64-debian9-container'] + variables: + NAME: debian-tricore-cross + +win32-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win32-cross + +win64-debian-cross-container: + <<: *container_job_definition + stage: containers-layer3 + needs: ['amd64-debian9-mxe-container'] + variables: + NAME: debian-win64-cross + +xtensa-debian-cross-container: + <<: *container_job_definition + variables: + NAME: debian-xtensa-cross + +cris-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-cris-cross + +amd64-fedora-container: + <<: *container_job_definition + variables: + NAME: fedora + +i386-fedora-cross-container: + <<: *container_job_definition + variables: + NAME: fedora-i386-cross + +amd64-ubuntu1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu1804 + +amd64-ubuntu2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu2004 + +amd64-ubuntu-container: + <<: *container_job_definition + variables: + NAME: ubuntu + diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9fdc752ea6..72d688875f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,10 +1,13 @@ stages: - containers + - containers-layer2 + - containers-layer3 - build =20 include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' + - local: '/.gitlab-ci.d/containers.yml' =20 .update_apt_template: &before_script_apt before_script: --=20 2.24.1 From nobody Tue Apr 30 18:53:20 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1592840264; cv=none; d=zohomail.com; s=zohoarc; b=FSZvzA0DxhChoNzrsWE9j+nFoN8Z9WwXp6iLOKeVGZOhV52Z9uqQ1iSLpHVzlBxKZwJ9T1oYeOzWL0QUb/T1bXgh77ir/lv9pR6N/ey1bMVDtwEHT4fYSbDnFmKrWz8qWJ094q4FRPwjt0IJep1NlKEhRPx6tZCi5WSCE6gPbEU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1592840264; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=5p9+mJduQhGS9oftmsPvNAzWw5/XxsIW3qskgtyLetw=; b=gV2yyv/DAMgj1vO06XeyyqL26FHzM8O+msewhuVw5KcSFn2EIunC3DzHbBa8DP4W80OPejfke2MOmZ7MIiCDNFyGds3L2atWr/Y7Mccj/TPkKkZTBefdIsa+JQCcPXs9L8zzsdoDijE+cL+dSVhvRNx15Q87+ylP3AyNSdnqDkU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1592840264504454.43511523348764; Mon, 22 Jun 2020 08:37:44 -0700 (PDT) Received: from localhost ([::1]:35484 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jnOVz-00033p-8R for importer@patchew.org; Mon, 22 Jun 2020 11:37:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57256) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jnORy-0004Cm-RM for qemu-devel@nongnu.org; Mon, 22 Jun 2020 11:33:34 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:24379 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jnORw-00028l-Qz for qemu-devel@nongnu.org; Mon, 22 Jun 2020 11:33:34 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-219-FjBaHrKKPLes33m-xvG-tg-1; Mon, 22 Jun 2020 11:33:29 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BCC7983DA41; Mon, 22 Jun 2020 15:33:28 +0000 (UTC) Received: from domokun.gsslab.fab.redhat.com (unknown [10.33.8.110]) by smtp.corp.redhat.com (Postfix) with ESMTP id 65D3D60BEC; Mon, 22 Jun 2020 15:33:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592840012; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5p9+mJduQhGS9oftmsPvNAzWw5/XxsIW3qskgtyLetw=; b=LadT7Lk6zc39qWlGl/X28+P9BwXMPHE+X3MaiIOtagBdRu7LAVDzIjF2JxPvXNCB+AFAqy YJb3RW1yf8V0pfhPU/rE3xY5GX+Mp5w446Fl3Atzv3XqiQpuVfVO2bSK79YFc/6imFLCGM dVsFfH2bTAewN0p9XHMSkuXyAwHDBrM= X-MC-Unique: FjBaHrKKPLes33m-xvG-tg-1 From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Subject: [PATCH RFC 3/3] gitlab: convert jobs to use custom built containers Date: Mon, 22 Jun 2020 16:33:18 +0100 Message-Id: <20200622153318.751107-4-berrange@redhat.com> In-Reply-To: <20200622153318.751107-1-berrange@redhat.com> References: <20200622153318.751107-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.120; envelope-from=berrange@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/22 02:57:26 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Laszlo Ersek , Wainer dos Santos Moschetta , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , =?UTF-8?q?Alex=20Benn=C3=A9e?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Now that we're building standard container images from dockerfiles in tests/docker/dockerfiles, we can convert the build jobs to use them. The key benefit of this is that a contributor can now more easily replicate the CI environment on their local machine. The container images are cached too, so we are not spending time waiting for the apt-get/dnf package installs to complete. Signed-off-by: Daniel P. Berrang=C3=A9 Acked-by: Thomas Huth --- .gitlab-ci.yml | 187 +++++++++++++++++++++---------------------------- 1 file changed, 81 insertions(+), 106 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 72d688875f..3297a402f7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -9,133 +9,108 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' =20 -.update_apt_template: &before_script_apt - before_script: - - apt-get update -qq - - apt-get install -y -qq git gcc libglib2.0-dev libpixman-1-dev make - genisoimage - - JOBS=3D$(expr $(nproc) + 1) - -.update_dnf_template: &before_script_dnf - before_script: - - dnf update -y - - dnf install -y bzip2 diffutils gcc git genisoimage findutils glib2-dev= el - make python3 perl-podlators perl-Test-Harness pixman-devel zlib-de= vel - - JOBS=3D$(expr $(nproc) + 1) +.native_build_job_template: &native_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/ci-$IMAGE:latest + before_script: + - JOBS=3D$(expr $(nproc) + 1) + script: + - mkdir build + - cd build + - if test -n "$TARGETS"; + then + ../configure --enable-werror $CONFIGURE_ARGS --target-list=3D"$TAR= GETS" ; + else + ../configure --enable-werror $CONFIGURE_ARGS ; + fi + - make -j"$JOBS" + - make -j"$JOBS" $MAKE_CHECK_ARGS =20 build-system1: - stage: build - image: ubuntu:19.10 - <<: *before_script_apt - script: - - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev - libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev libvdepl= ug-dev - - mkdir build - - cd build - - ../configure --enable-werror --target-list=3D"aarch64-softmmu alpha-sof= tmmu - cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-so= ftmmu - mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-soft= mmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-= softmmu + moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc= -softmmu + riscv64-softmmu sparc-softmmu + MAKE_CHECK_ARGS: check =20 build-system2: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y SDL2-devel libgcrypt-devel brlapi-devel libaio-devel - libfdt-devel lzo-devel librdmacm-devel libibverbs-devel libibumad-d= evel - libzstd-devel - - mkdir build - - cd build - - ../configure --enable-werror --target-list=3D"tricore-softmmu unicore32= -softmmu - microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-so= ftmmu - sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-sof= tmmu" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-sof= tmmu + riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-sof= tmmu + xtensa-softmmu nios2-softmmu or1k-softmmu + MAKE_CHECK_ARGS: check =20 build-disabled: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-rdma --disable-slirp --disable-c= url + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --disable-rdma --disable-slirp --disable-curl --disable-capstone --disable-live-block-migration --disable-glusterfs --disable-replication --disable-coroutine-pool --disable-smartcard --disable-guest-agent --disable-curses --disable-libxml2 --disable-t= pm --disable-qom-cast-debug --disable-spice --disable-vhost-vsock --disable-vhost-net --disable-vhost-crypto --disable-vhost-user - --target-list=3D"i386-softmmu ppc64-softmmu mips64-softmmu i386-linu= x-user" - - make -j"$JOBS" - - make -j"$JOBS" check-qtest SPEED=3Dslow + TARGETS: i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user + MAKE_CHECK_ARGS: check-qtest SPEED=3Dslow =20 build-tcg-disabled: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - dnf install -y clang gtk3-devel libusbx-devel libgcrypt-devel - - mkdir build - - cd build - - ../configure --cc=3Dclang --enable-werror --disable-tcg --audio-drv-lis= t=3D"" - - make -j"$JOBS" - - make check-unit - - make check-qapi-schema - - cd tests/qemu-iotests/ - - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048 + <<: *native_build_job_definition + variables: + IMAGE: centos8 + script: + - mkdir build + - cd build + - ../configure --disable-tcg --audio-drv-list=3D"" + - make -j"$JOBS" + - make check-unit + - make check-qapi-schema + - cd tests/qemu-iotests/ + - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033= 048 052 063 077 086 101 104 106 113 148 150 151 152 157 159 160 163 170 171 183 184 192 194 197 208 215 221 222 226 227 236 253 277 - - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 102 = 122 + - ./check -qcow2 028 051 056 057 058 065 067 068 082 085 091 095 096 1= 02 122 124 132 139 142 144 145 151 152 155 157 165 194 196 197 200 202 208 209 215 216 218 222 227 234 246 247 248 250 254 255 257 258 260 261 262 263 264 270 272 273 277 279 =20 build-user: - stage: build - <<: *before_script_apt - script: - - mkdir build - - cd build - - ../configure --enable-werror --disable-system --disable-guest-agent - --disable-capstone --disable-slirp --disable-fdt - - make -j"$JOBS" - - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user + <<: *native_build_job_definition + variables: + IMAGE: ubuntu2004 + CONFIGURE_ARGS: --disable-system --disable-guest-agent + --disable-capstone --disable-slirp --disable-fdt + MAKE_CHECK_ARGS: run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-l= inux-user =20 build-clang: - stage: build - image: fedora:latest - <<: *before_script_dnf - script: - - yum install -y clang SDL2-devel libattr-devel libcap-ng-devel xfsprogs-= devel - libiscsi-devel libnfs-devel libseccomp-devel gnutls-devel librbd-de= vel - - mkdir build - - cd build - - ../configure --cc=3Dclang --cxx=3Dclang++ --enable-werror - --target-list=3D"alpha-softmmu arm-softmmu m68k-softmmu mips64-softm= mu - ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-us= er" - - make -j"$JOBS" - - make -j"$JOBS" check + <<: *native_build_job_definition + variables: + IMAGE: fedora + CONFIGURE_ARGS: --cc=3Dclang --cxx=3Dclang++ + TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu + ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user + MAKE_CHECK_ARGS: check =20 build-tci: - stage: build - image: centos:8 - <<: *before_script_dnf - script: - - TARGETS=3D"aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86= _64" - - mkdir build - - cd build - - ../configure --enable-tcg-interpreter - --target-list=3D"$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; = done)" - - make -j"$JOBS" - - make run-tcg-tests-x86_64-softmmu - - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/px= e-test - - for tg in $TARGETS ; do - export QTEST_QEMU_BINARY=3D"${tg}-softmmu/qemu-system-${tg}" ; - ./tests/qtest/boot-serial-test || exit 1 ; - ./tests/qtest/cdrom-test || exit 1 ; - done - - QTEST_QEMU_BINARY=3D"x86_64-softmmu/qemu-system-x86_64" ./tests/qtest/p= xe-test - - QTEST_QEMU_BINARY=3D"s390x-softmmu/qemu-system-s390x" - ./tests/qtest/pxe-test -m slow + <<: *native_build_job_definition + variables: + IMAGE: fedora + script: + - TARGETS=3D"aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x = x86_64" + - mkdir build + - cd build + - ../configure --enable-tcg-interpreter + --target-list=3D"$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '= ; done)" + - make -j"$JOBS" + - make run-tcg-tests-x86_64-softmmu + - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest= /pxe-test + - for tg in $TARGETS ; do + export QTEST_QEMU_BINARY=3D"${tg}-softmmu/qemu-system-${tg}" ; + ./tests/qtest/boot-serial-test || exit 1 ; + ./tests/qtest/cdrom-test || exit 1 ; + done + - QTEST_QEMU_BINARY=3D"x86_64-softmmu/qemu-system-x86_64" ./tests/qtes= t/pxe-test + - QTEST_QEMU_BINARY=3D"s390x-softmmu/qemu-system-s390x" ./tests/qtest/= pxe-test -m slow --=20 2.24.1