1. Patchew
  2. Xen
  3. View series

[PATCH] automation: optimize build jobs order

Marek Marczykowski-Górecki posted 1 patch 1 year ago
Patches applied successfully (tree, apply log)
git fetch https://gitlab.com/xen-project/patchew/xen tags/patchew/20230429142707.176299-1-marmarek@invisiblethingslab.com
There is a newer version of this series
automation/gitlab-ci/build.yaml | 735 ++++++++++++++++----------------
1 file changed, 370 insertions(+), 365 deletions(-)
[PATCH] automation: optimize build jobs order
Posted by Marek Marczykowski-Górecki 1 year ago 
Put jobs that are needed for (any) test earlier, so the tests can start
running in parallel to builds.
This commits splits only x86 build jobs into two sections (one on top
and one on bottom), but keep ARM build jobs in one section, as most of
them have some test connected and the few that do not are not worth
reducing readability of the file.

And also, put artifacts jobs at the very beginning, not the very end.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
This made the pipeline to complete within 45 minutes. This isn't big
improvement on its own, but should make adding more runners more
beneficial. While looking at it in real time, most jobs were really
waiting for available runners and not stuck on dependencies anymore.
---
 automation/gitlab-ci/build.yaml | 735 ++++++++++++++++----------------
 1 file changed, 370 insertions(+), 365 deletions(-)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index d323c30a8304..32dec45b8b9a 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -236,359 +236,211 @@
   variables:
     <<: *gcc
 
-# Jobs below this line
+## Test artifacts common
 
-archlinux-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: archlinux:current
+.test-jobs-artifact-common:
+  stage: build
+  except: !reference [.test-jobs-common, except]
 
-archlinux-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: archlinux:current
+# Arm test artifacts
 
-centos-7-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: centos:7
+alpine-3.12-arm64-rootfs-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.12-arm64v8
+  script:
+    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
+  artifacts:
+    paths:
+      - binaries/initrd.tar.gz
+  tags:
+    - arm64
 
-centos-7-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: centos:7
+kernel-5.19-arm64-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:5.19-arm64v8
+  script:
+    - mkdir binaries && cp /Image binaries/Image
+  artifacts:
+    paths:
+      - binaries/Image
+  tags:
+    - arm64
 
-debian-stretch-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: debian:stretch
+qemu-system-aarch64-6.0.0-arm64-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/qemu-system-aarch64:6.0.0-arm64v8
+  script:
+    - mkdir binaries && cp /qemu-system-aarch64 binaries/qemu-system-aarch64
+  artifacts:
+    paths:
+      - binaries/qemu-system-aarch64
+  tags:
+    - arm64
 
-debian-stretch-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: debian:stretch
+qemu-system-aarch64-6.0.0-arm32-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/qemu-system-aarch64:6.0.0-arm64v8
+  script:
+    - mkdir binaries && cp /qemu-system-arm binaries/qemu-system-arm
+  artifacts:
+    paths:
+      - binaries/qemu-system-arm
+  tags:
+    - arm64
 
-debian-stretch-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: debian:stretch
+# x86_64 test artifacts
 
-debian-stretch-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: debian:stretch
+alpine-3.12-rootfs-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.12
+  script:
+    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
+  artifacts:
+    paths:
+      - binaries/initrd.tar.gz
+  tags:
+    - x86_64
 
-debian-stretch-32-clang-debug:
-  extends: .clang-x86-32-build-debug
-  variables:
-    CONTAINER: debian:stretch-i386
+kernel-6.1.19-export:
+  extends: .test-jobs-artifact-common
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
+  script:
+    - mkdir binaries && cp /bzImage binaries/bzImage
+  artifacts:
+    paths:
+      - binaries/bzImage
+  tags:
+    - x86_64
 
-debian-stretch-32-gcc-debug:
-  extends: .gcc-x86-32-build-debug
-  variables:
-    CONTAINER: debian:stretch-i386
+# Jobs below this line
 
-debian-buster-gcc-ibt:
+# Build jobs needed for tests
+
+alpine-3.12-gcc:
   extends: .gcc-x86-64-build
   variables:
-    CONTAINER: debian:buster-gcc-ibt
-    RANDCONFIG: y
-    EXTRA_FIXED_RANDCONFIG: |
-      CONFIG_XEN_IBT=y
+    CONTAINER: alpine:3.12
 
-debian-unstable-clang:
-  extends: .clang-x86-64-build
+alpine-3.12-gcc-debug:
+  extends: .gcc-x86-64-build-debug
   variables:
-    CONTAINER: debian:unstable
+    CONTAINER: alpine:3.12
+
+debian-stretch-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: debian:stretch
 
 debian-unstable-clang-debug:
   extends: .clang-x86-64-build-debug
   variables:
     CONTAINER: debian:unstable
 
-debian-unstable-gcc:
-  extends: .gcc-x86-64-build
+# Arm32 cross-build
+
+debian-unstable-gcc-arm32:
+  extends: .gcc-arm32-cross-build
   variables:
-    CONTAINER: debian:unstable
+    CONTAINER: debian:unstable-arm64v8-arm32-gcc
+    HYPERVISOR_ONLY: y
 
-debian-unstable-gcc-debug:
-  extends: .gcc-x86-64-build-debug
+debian-unstable-gcc-arm32-debug:
+  extends: .gcc-arm32-cross-build-debug
   variables:
-    CONTAINER: debian:unstable
+    CONTAINER: debian:unstable-arm64v8-arm32-gcc
+    HYPERVISOR_ONLY: y
 
-debian-unstable-gcc-randconfig:
-  extends: .gcc-x86-64-build
+debian-unstable-gcc-arm32-randconfig:
+  extends: .gcc-arm32-cross-build
   variables:
-    CONTAINER: debian:unstable
+    CONTAINER: debian:unstable-arm64v8-arm32-gcc
+    HYPERVISOR_ONLY: y
     RANDCONFIG: y
 
-debian-unstable-gcc-debug-randconfig:
-  extends: .gcc-x86-64-build-debug
+debian-unstable-gcc-arm32-debug-randconfig:
+  extends: .gcc-arm32-cross-build-debug
   variables:
-    CONTAINER: debian:unstable
+    CONTAINER: debian:unstable-arm64v8-arm32-gcc
+    HYPERVISOR_ONLY: y
     RANDCONFIG: y
 
-debian-unstable-32-clang-debug:
-  extends: .clang-x86-32-build-debug
+debian-unstable-gcc-arm32-staticmem:
+  extends: .gcc-arm32-cross-build
   variables:
-    CONTAINER: debian:unstable-i386
+    CONTAINER: debian:unstable-arm64v8-arm32-gcc
+    HYPERVISOR_ONLY: y
+    EXTRA_XEN_CONFIG: |
+      CONFIG_EXPERT=y
+      CONFIG_UNSUPPORTED=y
+      CONFIG_STATIC_MEMORY=y
 
-debian-unstable-32-gcc-debug:
-  extends: .gcc-x86-32-build-debug
+debian-unstable-gcc-arm32-debug-staticmem:
+  extends: .gcc-arm32-cross-build-debug
   variables:
-    CONTAINER: debian:unstable-i386
+    CONTAINER: debian:unstable-arm64v8-arm32-gcc
+    HYPERVISOR_ONLY: y
+    EXTRA_XEN_CONFIG: |
+      CONFIG_EXPERT=y
+      CONFIG_UNSUPPORTED=y
+      CONFIG_STATIC_MEMORY=y
 
-fedora-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: fedora:29
+# Arm builds
 
-fedora-gcc-debug:
-  extends: .gcc-x86-64-build-debug
+debian-unstable-gcc-arm64:
+  extends: .gcc-arm64-build
   variables:
-    CONTAINER: fedora:29
-
-# Ubuntu Trusty's Clang is 3.4 while Xen requires 3.5
+    CONTAINER: debian:unstable-arm64v8
 
-ubuntu-trusty-gcc:
-  extends: .gcc-x86-64-build
+debian-unstable-gcc-debug-arm64:
+  extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: ubuntu:trusty
+    CONTAINER: debian:unstable-arm64v8
 
-ubuntu-trusty-gcc-debug:
-  extends: .gcc-x86-64-build-debug
+debian-unstable-gcc-arm64-randconfig:
+  extends: .gcc-arm64-build
   variables:
-    CONTAINER: ubuntu:trusty
+    CONTAINER: debian:unstable-arm64v8
+    RANDCONFIG: y
 
-ubuntu-xenial-clang:
-  extends: .clang-x86-64-build
+debian-unstable-gcc-debug-arm64-randconfig:
+  extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: ubuntu:xenial
+    CONTAINER: debian:unstable-arm64v8
+    RANDCONFIG: y
 
-ubuntu-xenial-clang-debug:
-  extends: .clang-x86-64-build-debug
+alpine-3.12-gcc-arm64:
+  extends: .gcc-arm64-build
   variables:
-    CONTAINER: ubuntu:xenial
+    CONTAINER: alpine:3.12-arm64v8
 
-ubuntu-xenial-gcc:
-  extends: .gcc-x86-64-build
+alpine-3.12-gcc-debug-arm64:
+  extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: ubuntu:xenial
+    CONTAINER: alpine:3.12-arm64v8
 
-ubuntu-xenial-gcc-debug:
-  extends: .gcc-x86-64-build-debug
+alpine-3.12-gcc-arm64-randconfig:
+  extends: .gcc-arm64-build
   variables:
-    CONTAINER: ubuntu:xenial
+    CONTAINER: alpine:3.12-arm64v8
+    RANDCONFIG: y
 
-ubuntu-bionic-clang:
-  extends: .clang-x86-64-build
+alpine-3.12-gcc-debug-arm64-randconfig:
+  extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: ubuntu:bionic
+    CONTAINER: alpine:3.12-arm64v8
+    RANDCONFIG: y
 
-ubuntu-bionic-clang-debug:
-  extends: .clang-x86-64-build-debug
+alpine-3.12-gcc-arm64-staticmem:
+  extends: .gcc-arm64-build
   variables:
-    CONTAINER: ubuntu:bionic
+    CONTAINER: alpine:3.12-arm64v8
+    EXTRA_XEN_CONFIG: |
+      CONFIG_EXPERT=y
+      CONFIG_UNSUPPORTED=y
+      CONFIG_STATIC_MEMORY=y
 
-ubuntu-bionic-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: ubuntu:bionic
-
-ubuntu-bionic-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:bionic
-
-ubuntu-focal-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: ubuntu:focal
-
-ubuntu-focal-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:focal
-
-ubuntu-focal-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: ubuntu:focal
-
-ubuntu-focal-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:focal
-
-opensuse-leap-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: suse:opensuse-leap
-
-opensuse-leap-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: suse:opensuse-leap
-
-opensuse-leap-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: suse:opensuse-leap
-
-opensuse-leap-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: suse:opensuse-leap
-
-opensuse-tumbleweed-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: suse:opensuse-tumbleweed
-  allow_failure: true
-
-opensuse-tumbleweed-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: suse:opensuse-tumbleweed
-  allow_failure: true
-
-opensuse-tumbleweed-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: suse:opensuse-tumbleweed
-  allow_failure: true
-
-opensuse-tumbleweed-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: suse:opensuse-tumbleweed
-  allow_failure: true
-
-alpine-3.12-gcc:
-  extends: .gcc-x86-64-build
-  variables:
-    CONTAINER: alpine:3.12
-
-alpine-3.12-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: alpine:3.12
-
-alpine-3.12-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: alpine:3.12
-
-alpine-3.12-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: alpine:3.12
-
-# Arm32 cross-build
-
-debian-unstable-gcc-arm32:
-  extends: .gcc-arm32-cross-build
-  variables:
-    CONTAINER: debian:unstable-arm64v8-arm32-gcc
-    HYPERVISOR_ONLY: y
-
-debian-unstable-gcc-arm32-debug:
-  extends: .gcc-arm32-cross-build-debug
-  variables:
-    CONTAINER: debian:unstable-arm64v8-arm32-gcc
-    HYPERVISOR_ONLY: y
-
-debian-unstable-gcc-arm32-randconfig:
-  extends: .gcc-arm32-cross-build
-  variables:
-    CONTAINER: debian:unstable-arm64v8-arm32-gcc
-    HYPERVISOR_ONLY: y
-    RANDCONFIG: y
-
-debian-unstable-gcc-arm32-debug-randconfig:
-  extends: .gcc-arm32-cross-build-debug
-  variables:
-    CONTAINER: debian:unstable-arm64v8-arm32-gcc
-    HYPERVISOR_ONLY: y
-    RANDCONFIG: y
-
-debian-unstable-gcc-arm32-staticmem:
-  extends: .gcc-arm32-cross-build
-  variables:
-    CONTAINER: debian:unstable-arm64v8-arm32-gcc
-    HYPERVISOR_ONLY: y
-    EXTRA_XEN_CONFIG: |
-      CONFIG_EXPERT=y
-      CONFIG_UNSUPPORTED=y
-      CONFIG_STATIC_MEMORY=y
-
-debian-unstable-gcc-arm32-debug-staticmem:
-  extends: .gcc-arm32-cross-build-debug
-  variables:
-    CONTAINER: debian:unstable-arm64v8-arm32-gcc
-    HYPERVISOR_ONLY: y
-    EXTRA_XEN_CONFIG: |
-      CONFIG_EXPERT=y
-      CONFIG_UNSUPPORTED=y
-      CONFIG_STATIC_MEMORY=y
-
-# Arm builds
-
-debian-unstable-gcc-arm64:
-  extends: .gcc-arm64-build
-  variables:
-    CONTAINER: debian:unstable-arm64v8
-
-debian-unstable-gcc-debug-arm64:
-  extends: .gcc-arm64-build-debug
-  variables:
-    CONTAINER: debian:unstable-arm64v8
-
-debian-unstable-gcc-arm64-randconfig:
-  extends: .gcc-arm64-build
-  variables:
-    CONTAINER: debian:unstable-arm64v8
-    RANDCONFIG: y
-
-debian-unstable-gcc-debug-arm64-randconfig:
-  extends: .gcc-arm64-build-debug
-  variables:
-    CONTAINER: debian:unstable-arm64v8
-    RANDCONFIG: y
-
-alpine-3.12-gcc-arm64:
-  extends: .gcc-arm64-build
-  variables:
-    CONTAINER: alpine:3.12-arm64v8
-
-alpine-3.12-gcc-debug-arm64:
-  extends: .gcc-arm64-build-debug
-  variables:
-    CONTAINER: alpine:3.12-arm64v8
-
-alpine-3.12-gcc-arm64-randconfig:
-  extends: .gcc-arm64-build
-  variables:
-    CONTAINER: alpine:3.12-arm64v8
-    RANDCONFIG: y
-
-alpine-3.12-gcc-debug-arm64-randconfig:
-  extends: .gcc-arm64-build-debug
-  variables:
-    CONTAINER: alpine:3.12-arm64v8
-    RANDCONFIG: y
-
-alpine-3.12-gcc-arm64-staticmem:
-  extends: .gcc-arm64-build
-  variables:
-    CONTAINER: alpine:3.12-arm64v8
-    EXTRA_XEN_CONFIG: |
-      CONFIG_EXPERT=y
-      CONFIG_UNSUPPORTED=y
-      CONFIG_STATIC_MEMORY=y
-
-alpine-3.12-gcc-debug-arm64-staticmem:
-  extends: .gcc-arm64-build-debug
+alpine-3.12-gcc-debug-arm64-staticmem:
+  extends: .gcc-arm64-build-debug
   variables:
     CONTAINER: alpine:3.12-arm64v8
     EXTRA_XEN_CONFIG: |
@@ -706,78 +558,231 @@ debian-unstable-gcc-arm64-cppcheck:
     CPPCHECK: y
     HYPERVISOR_ONLY: y
 
-## Test artifacts common
+# Build jobs not needed for tests
 
-.test-jobs-artifact-common:
-  stage: build
-  except: !reference [.test-jobs-common, except]
+alpine-3.12-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: alpine:3.12
 
-# Arm test artifacts
+alpine-3.12-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: alpine:3.12
 
-alpine-3.12-arm64-rootfs-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.12-arm64v8
-  script:
-    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
-  artifacts:
-    paths:
-      - binaries/initrd.tar.gz
-  tags:
-    - arm64
+archlinux-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: archlinux:current
 
-kernel-5.19-arm64-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:5.19-arm64v8
-  script:
-    - mkdir binaries && cp /Image binaries/Image
-  artifacts:
-    paths:
-      - binaries/Image
-  tags:
-    - arm64
+archlinux-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: archlinux:current
 
-qemu-system-aarch64-6.0.0-arm64-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/qemu-system-aarch64:6.0.0-arm64v8
-  script:
-    - mkdir binaries && cp /qemu-system-aarch64 binaries/qemu-system-aarch64
-  artifacts:
-    paths:
-      - binaries/qemu-system-aarch64
-  tags:
-    - arm64
+centos-7-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: centos:7
 
-qemu-system-aarch64-6.0.0-arm32-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/qemu-system-aarch64:6.0.0-arm64v8
-  script:
-    - mkdir binaries && cp /qemu-system-arm binaries/qemu-system-arm
-  artifacts:
-    paths:
-      - binaries/qemu-system-arm
-  tags:
-    - arm64
+centos-7-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: centos:7
 
-# x86_64 test artifacts
+debian-stretch-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: debian:stretch
 
-alpine-3.12-rootfs-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.12
-  script:
-    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
-  artifacts:
-    paths:
-      - binaries/initrd.tar.gz
-  tags:
-    - x86_64
+debian-stretch-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: debian:stretch
+
+debian-stretch-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: debian:stretch
+
+debian-stretch-32-clang-debug:
+  extends: .clang-x86-32-build-debug
+  variables:
+    CONTAINER: debian:stretch-i386
+
+debian-stretch-32-gcc-debug:
+  extends: .gcc-x86-32-build-debug
+  variables:
+    CONTAINER: debian:stretch-i386
+
+debian-buster-gcc-ibt:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: debian:buster-gcc-ibt
+    RANDCONFIG: y
+    EXTRA_FIXED_RANDCONFIG: |
+      CONFIG_XEN_IBT=y
+
+debian-unstable-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: debian:unstable
+
+debian-unstable-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: debian:unstable
+
+debian-unstable-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: debian:unstable
+
+debian-unstable-gcc-randconfig:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: debian:unstable
+    RANDCONFIG: y
+
+debian-unstable-gcc-debug-randconfig:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: debian:unstable
+    RANDCONFIG: y
+
+debian-unstable-32-clang-debug:
+  extends: .clang-x86-32-build-debug
+  variables:
+    CONTAINER: debian:unstable-i386
+
+debian-unstable-32-gcc-debug:
+  extends: .gcc-x86-32-build-debug
+  variables:
+    CONTAINER: debian:unstable-i386
+
+fedora-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: fedora:29
+
+fedora-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: fedora:29
+
+# Ubuntu Trusty's Clang is 3.4 while Xen requires 3.5
+
+ubuntu-trusty-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: ubuntu:trusty
+
+ubuntu-trusty-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:trusty
+
+ubuntu-xenial-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: ubuntu:xenial
+
+ubuntu-xenial-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:xenial
+
+ubuntu-xenial-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: ubuntu:xenial
+
+ubuntu-xenial-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:xenial
+
+ubuntu-bionic-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: ubuntu:bionic
+
+ubuntu-bionic-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:bionic
+
+ubuntu-bionic-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: ubuntu:bionic
+
+ubuntu-bionic-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:bionic
+
+ubuntu-focal-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: ubuntu:focal
+
+ubuntu-focal-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:focal
+
+ubuntu-focal-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: ubuntu:focal
+
+ubuntu-focal-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:focal
+
+opensuse-leap-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: suse:opensuse-leap
+
+opensuse-leap-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: suse:opensuse-leap
+
+opensuse-leap-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: suse:opensuse-leap
+
+opensuse-leap-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: suse:opensuse-leap
+
+opensuse-tumbleweed-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+  allow_failure: true
+
+opensuse-tumbleweed-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+  allow_failure: true
+
+opensuse-tumbleweed-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+  allow_failure: true
+
+opensuse-tumbleweed-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+  allow_failure: true
 
-kernel-6.1.19-export:
-  extends: .test-jobs-artifact-common
-  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:6.1.19
-  script:
-    - mkdir binaries && cp /bzImage binaries/bzImage
-  artifacts:
-    paths:
-      - binaries/bzImage
-  tags:
-    - x86_64
-- 
2.39.2
Re: [PATCH] automation: optimize build jobs order
Posted by Andrew Cooper 1 year ago 
On 29/04/2023 3:27 pm, Marek Marczykowski-Górecki wrote:
> Put jobs that are needed for (any) test earlier, so the tests can start
> running in parallel to builds.
> This commits splits only x86 build jobs into two sections (one on top
> and one on bottom), but keep ARM build jobs in one section, as most of
> them have some test connected and the few that do not are not worth
> reducing readability of the file.
>
> And also, put artifacts jobs at the very beginning, not the very end.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
> This made the pipeline to complete within 45 minutes. This isn't big
> improvement on its own, but should make adding more runners more
> beneficial. While looking at it in real time, most jobs were really
> waiting for available runners and not stuck on dependencies anymore.

That's still 1/4 better than before.  I'd say that's a good improvement
all on its own.

As for the patch, Its not the easiest to review.

The test artefacts section is new, and is just moving various jobs
forwards in the file.

I suspect that if you split the patch into two, first forming the test
artefacts section, and second rearranging the existing x86 tests, the
result might be readable (or at least, more readable).

The key (I think) will be to keep the # Jobs below this line, and
following archlinux tests unmodified in patch 1, at which point the diff
ought to render as one block insertion, then scattered deletions.

I suspect the second patch is going to be a mess however you try to
rearrange it, so I wouldn't worry too much if this approach doesn't work.

~Andrew

Patchew 2.1-devel-0870f84

© 2016 - 2024 Red Hat, Inc.