[PATCH 13/16] gitlab: use template for ubuntu-24.04-s390x jobs

Alex Bennée posted 16 patches 1 month ago
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Thomas Huth <thuth@redhat.com>, Ed Maste <emaste@freebsd.org>, Li-Wen Hsu <lwhsu@freebsd.org>, Kohei Tokunaga <ktokunaga.mail@gmail.com>, Peter Maydell <peter.maydell@linaro.org>, Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>, Dmitry Osipenko <dmitry.osipenko@collabora.com>, David Woodhouse <dwmw2@infradead.org>, Paul Durrant <paul@xen.org>, Paolo Bonzini <pbonzini@redhat.com>, Zhao Liu <zhao1.liu@intel.com>, Sriram Yagnaraman <sriram.yagnaraman@ericsson.com>
[PATCH 13/16] gitlab: use template for ubuntu-24.04-s390x jobs
Posted by Alex Bennée 1 month ago
Most of the test is pure boilerplate so to save ourselves from
repetition move all the main bits into a minimal copy of
native_build_job_template but without the caching.

We keep all the current allow_fail and configure setups but do take
the opportunity to replace the inline nproc calls to using a common
JOBS variable.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .../custom-runners/ubuntu-24.04-s390x.yml     | 181 +++++++-----------
 1 file changed, 66 insertions(+), 115 deletions(-)

diff --git a/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml b/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml
index 45dbee17880..ef8db2750ac 100644
--- a/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml
+++ b/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml
@@ -2,129 +2,80 @@
 # setup by the scripts/ci/setup/ubuntu/build-environment.yml task
 # "Install basic packages to build QEMU on Ubuntu 24.04"
 
+.ubuntu_s390x_template:
+  extends: .custom_runner_template
+  needs: []
+  stage: build
+  tags:
+    - ubuntu_24.04
+    - s390x
+  rules:
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+    - if: "$S390X_RUNNER_AVAILABLE"
+  before_script:
+    - source scripts/ci/gitlab-ci-section
+    - section_start setup "Pre-script setup"
+    - JOBS=$(expr $(nproc) - 1)
+    - section_end setup
+  script:
+    - mkdir build
+    - cd build
+    - section_start configure "Running configure"
+    - ../configure $CONFIGURE_ARGS ||
+        { cat config.log meson-logs/meson-log.txt; exit 1; }
+    - section_end configure
+    - section_start build "Building QEMU"
+    - make --output-sync -j"$JOBS"
+    - section_end build
+    - section_start test "Running tests"
+    - if test -n "$MAKE_CHECK_ARGS";
+      then
+        make -j"$JOBS" $MAKE_CHECK_ARGS ;
+      fi
+    - section_end test
+
 ubuntu-24.04-s390x-all-linux:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - s390x
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
- - if: "$S390X_RUNNER_AVAILABLE"
- script:
- - mkdir build
- - cd build
- - ../configure --enable-debug --disable-system --disable-tools --disable-docs
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc`
- - make --output-sync check-tcg
- - make --output-sync -j`nproc` check
+  extends: .ubuntu_s390x_template
+  needs: []
+  variables:
+    CONFIGURE_ARGS: --enable-debug --disable-system --disable-tools --disable-docs
+    MAKE_CHECK_ARGS: check-tcg check
 
 ubuntu-24.04-s390x-all-system:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - s390x
- timeout: 75m
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   allow_failure: true
- - if: "$S390X_RUNNER_AVAILABLE"
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --disable-user
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check
+  extends: .ubuntu_s390x_template
+  needs: []
+  variables:
+    CONFIGURE_ARGS: --disable-user
+    MAKE_CHECK_ARGS: check
+  allow_failure: true
 
 ubuntu-24.04-s390x-alldbg:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - s390x
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$S390X_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --enable-debug
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make clean
- - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check
+  extends: .ubuntu_s390x_template
+  needs: []
+  variables:
+    CONFIGURE_ARGS: --enable-debug
+    MAKE_CHECK_ARGS: check
+  allow_failure: true
 
 ubuntu-24.04-s390x-clang:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - s390x
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$S390X_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --cc=clang --cxx=clang++ --enable-ubsan
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check
+  extends: .ubuntu_s390x_template
+  needs: []
+  variables:
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-ubsan
+    MAKE_CHECK_ARGS: check
+  allow_failure: true
 
 ubuntu-24.04-s390x-tci:
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - s390x
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$S390X_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --enable-tcg-interpreter
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc`
+  extends: .ubuntu_s390x_template
+  needs: []
+  variables:
+    CONFIGURE_ARGS: --enable-tcg-interpreter
+  allow_failure: true
 
 ubuntu-24.04-s390x-notcg:
- extends: .custom_runner_template
- needs: []
- stage: build
- tags:
- - ubuntu_24.04
- - s390x
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
-   when: manual
-   allow_failure: true
- - if: "$S390X_RUNNER_AVAILABLE"
-   when: manual
-   allow_failure: true
- script:
- - mkdir build
- - cd build
- - ../configure --disable-tcg
-   || { cat config.log meson-logs/meson-log.txt; exit 1; }
- - make --output-sync -j`nproc`
- - make --output-sync -j`nproc` check
+  extends: .ubuntu_s390x_template
+  needs: []
+  variables:
+    CONFIGURE_ARGS: --disable-tcg
+    MAKE_CHECK_ARGS: check
+  allow_failure: true
-- 
2.47.3


Re: [PATCH 13/16] gitlab: use template for ubuntu-24.04-s390x jobs
Posted by Daniel P. Berrangé 1 month ago
On Tue, Nov 11, 2025 at 05:17:20PM +0000, Alex Bennée wrote:
> Most of the test is pure boilerplate so to save ourselves from
> repetition move all the main bits into a minimal copy of
> native_build_job_template but without the caching.
> 
> We keep all the current allow_fail and configure setups but do take
> the opportunity to replace the inline nproc calls to using a common
> JOBS variable.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .../custom-runners/ubuntu-24.04-s390x.yml     | 181 +++++++-----------
>  1 file changed, 66 insertions(+), 115 deletions(-)
> 
> diff --git a/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml b/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml
> index 45dbee17880..ef8db2750ac 100644
> --- a/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml
> +++ b/.gitlab-ci.d/custom-runners/ubuntu-24.04-s390x.yml
> @@ -2,129 +2,80 @@
>  # setup by the scripts/ci/setup/ubuntu/build-environment.yml task
>  # "Install basic packages to build QEMU on Ubuntu 24.04"
>  
> +.ubuntu_s390x_template:
> +  extends: .custom_runner_template
> +  needs: []
> +  stage: build
> +  tags:
> +    - ubuntu_24.04
> +    - s390x
> +  rules:
> +    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> +    - if: "$S390X_RUNNER_AVAILABLE"

These rules mean the jobs always run if those three envs
are satisified, however....


>  ubuntu-24.04-s390x-alldbg:
> - extends: .custom_runner_template
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_24.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> -   when: manual
> -   allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> -   when: manual
> -   allow_failure: true

...this job...

>  ubuntu-24.04-s390x-clang:
> - extends: .custom_runner_template
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_24.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> -   when: manual
> -   allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> -   when: manual
> -   allow_failure: true

...this job..


>  ubuntu-24.04-s390x-tci:
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_24.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> -   when: manual
> -   allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> -   when: manual
> -   allow_failure: true

...this job...

>  ubuntu-24.04-s390x-notcg:
> - extends: .custom_runner_template
> - needs: []
> - stage: build
> - tags:
> - - ubuntu_24.04
> - - s390x
> - rules:
> - - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
> -   when: manual
> -   allow_failure: true
> - - if: "$S390X_RUNNER_AVAILABLE"
> -   when: manual
> -   allow_failure: true

...and this job, would never get run by default even if the env vars
were satisfied. It would require someone to hit the 'run' button in
the CI web pages. Now with this commit, these 4 manual jobs are run
unconditionally every time. IOW, pipelines now trigger 6 s390x
jobs instead of 2 jobs, and 4 opt-in jobs.

If that is intentional, IMHO, there should have been a prior commit
that removed the 'when: manual' stanza from all these jobs, so that
this commit was pure re-factoring with no functional change in job
behaviour.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|