1. Patchew
  2. Xen
  3. [PATCH test-artifacts v1 0/5] Changes for building arbitrary Linux branch and for hw12 runner
  4. View patch

[PATCH test-artifacts v1 4/5] Support building arbitrary Linux branch/tag/commit

Marek Marczykowski-Górecki posted 5 patches 4 months, 1 week ago
[PATCH test-artifacts v1 4/5] Support building arbitrary Linux branch/tag/commit
Posted by Marek Marczykowski-Górecki 4 months, 1 week ago 
If LINUX_URL is set, fetch LINUX_VERSION from there. Go with "git
init" + "git fetch" instead of "git clone" to support any of
branch/tag/commit.

This also defines optional linux-git-* jobs which will build the thing
if LINUX_GIT_VERSION and LINUX_GIT_URL variables are provided for the
pipeline.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
The script variable and job variable need to have different names, so a
pipeline variable won't override it for all jobs. While LINUX_VERSION /
LINUX_GIT_VERSION is IMO okay, I'm not very happy about LINUX_URL /
LINUX_GIT_URL. Any better ideas?
---
 .gitlab-ci.yml         | 22 ++++++++++++++++++++++
 scripts/build-linux.sh | 18 +++++++++++++-----
 2 files changed, 35 insertions(+), 5 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ad44fb4..60af072 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,5 +1,9 @@
 variables:
   REGISTRY: registry.gitlab.com/xen-project/hardware/test-artifacts
+  LINUX_GIT_VERSION:
+    description: "branch/tag/commit for the linux-git jobs"
+  LINUX_GIT_URL:
+    description: "git url for the linux-git jobs"
 
 stages:
   - build
@@ -46,6 +50,15 @@ linux-6.6.86-arm64:
   variables:
     LINUX_VERSION: 6.6.86
 
+linux-git-arm64:
+  extends: .arm64-artifacts
+  script: ./scripts/build-linux.sh
+  variables:
+    LINUX_VERSION: $LINUX_GIT_VERSION
+    LINUX_URL: $LINUX_GIT_URL
+  rules:
+  - if: $LINUX_GIT_VERSION && $LINUX_GIT_URL
+
 #
 # x86_64 artifacts
 #
@@ -70,6 +83,15 @@ linux-6.12.34-x86_64:
   variables:
     LINUX_VERSION: 6.12.34
 
+linux-git-x86_64:
+  extends: .x86_64-artifacts
+  script: ./scripts/build-linux.sh
+  variables:
+    LINUX_VERSION: $LINUX_GIT_VERSION
+    LINUX_URL: $LINUX_GIT_URL
+  rules:
+  - if: $LINUX_GIT_VERSION && $LINUX_GIT_URL
+
 microcode-x86:
   extends: .x86_64-artifacts
   script: ./scripts/x86-microcode.sh
diff --git a/scripts/build-linux.sh b/scripts/build-linux.sh
index cf0e744..1fc96d1 100755
--- a/scripts/build-linux.sh
+++ b/scripts/build-linux.sh
@@ -12,11 +12,19 @@ COPYDIR="${WORKDIR}/binaries"
 UNAME=$(uname -m)
 
 # Build Linux
-MAJOR=${LINUX_VERSION%%.*}
-curl -fsSLO \
-    https://cdn.kernel.org/pub/linux/kernel/v"${MAJOR}".x/linux-"${LINUX_VERSION}".tar.xz
-tar xf linux-"${LINUX_VERSION}".tar.xz
-cd linux-"${LINUX_VERSION}"
+if [[ -n "${LINUX_URL}" ]]; then
+    mkdir linux
+    cd linux
+    git init
+    git fetch --depth=1 "${LINUX_URL}" "${LINUX_VERSION}"
+    git checkout FETCH_HEAD
+else
+    MAJOR=${LINUX_VERSION%%.*}
+    curl -fsSLO \
+        https://cdn.kernel.org/pub/linux/kernel/v"${MAJOR}".x/linux-"${LINUX_VERSION}".tar.xz
+    tar xf linux-"${LINUX_VERSION}".tar.xz
+    cd linux-"${LINUX_VERSION}"
+fi
 
 make defconfig
 ./scripts/config --enable BRIDGE
-- 
git-series 0.9.1
Re: [PATCH test-artifacts v1 4/5] Support building arbitrary Linux branch/tag/commit
Posted by Stefano Stabellini 4 months, 1 week ago 
On Mon, 23 Jun 2025, Marek Marczykowski-Górecki wrote:
> If LINUX_URL is set, fetch LINUX_VERSION from there. Go with "git
> init" + "git fetch" instead of "git clone" to support any of
> branch/tag/commit.
> 
> This also defines optional linux-git-* jobs which will build the thing
> if LINUX_GIT_VERSION and LINUX_GIT_URL variables are provided for the
> pipeline.
> 
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
> The script variable and job variable need to have different names, so a
> pipeline variable won't override it for all jobs. While LINUX_VERSION /
> LINUX_GIT_VERSION is IMO okay, I'm not very happy about LINUX_URL /
> LINUX_GIT_URL. Any better ideas?

I understand the usefulness of these two jobs for testing purposes (I
mean personal testing during development, not the CI-loop) and based on
that alone I would accept this.

However, I thought I would mention that for personal testing purposes
these 2 jobs don't actually need to be part of the master branch. The
person could add them to their own branch and git push.

That said, I am not opposed to having them in the tree for convenience.

I can also see you are using the two variables in your xen.git patch
series. Is that idea that it would allow for a much wider Linux versions
testing? If we are going to only test one version, this is not required.


> ---
>  .gitlab-ci.yml         | 22 ++++++++++++++++++++++
>  scripts/build-linux.sh | 18 +++++++++++++-----
>  2 files changed, 35 insertions(+), 5 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index ad44fb4..60af072 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,5 +1,9 @@
>  variables:
>    REGISTRY: registry.gitlab.com/xen-project/hardware/test-artifacts
> +  LINUX_GIT_VERSION:
> +    description: "branch/tag/commit for the linux-git jobs"
> +  LINUX_GIT_URL:
> +    description: "git url for the linux-git jobs"
>  
>  stages:
>    - build
> @@ -46,6 +50,15 @@ linux-6.6.86-arm64:
>    variables:
>      LINUX_VERSION: 6.6.86
>  
> +linux-git-arm64:
> +  extends: .arm64-artifacts
> +  script: ./scripts/build-linux.sh
> +  variables:
> +    LINUX_VERSION: $LINUX_GIT_VERSION
> +    LINUX_URL: $LINUX_GIT_URL
> +  rules:
> +  - if: $LINUX_GIT_VERSION && $LINUX_GIT_URL
> +
>  #
>  # x86_64 artifacts
>  #
> @@ -70,6 +83,15 @@ linux-6.12.34-x86_64:
>    variables:
>      LINUX_VERSION: 6.12.34
>  
> +linux-git-x86_64:
> +  extends: .x86_64-artifacts
> +  script: ./scripts/build-linux.sh
> +  variables:
> +    LINUX_VERSION: $LINUX_GIT_VERSION
> +    LINUX_URL: $LINUX_GIT_URL
> +  rules:
> +  - if: $LINUX_GIT_VERSION && $LINUX_GIT_URL
> +
>  microcode-x86:
>    extends: .x86_64-artifacts
>    script: ./scripts/x86-microcode.sh
> diff --git a/scripts/build-linux.sh b/scripts/build-linux.sh
> index cf0e744..1fc96d1 100755
> --- a/scripts/build-linux.sh
> +++ b/scripts/build-linux.sh
> @@ -12,11 +12,19 @@ COPYDIR="${WORKDIR}/binaries"
>  UNAME=$(uname -m)
>  
>  # Build Linux
> -MAJOR=${LINUX_VERSION%%.*}
> -curl -fsSLO \
> -    https://cdn.kernel.org/pub/linux/kernel/v"${MAJOR}".x/linux-"${LINUX_VERSION}".tar.xz
> -tar xf linux-"${LINUX_VERSION}".tar.xz
> -cd linux-"${LINUX_VERSION}"
> +if [[ -n "${LINUX_URL}" ]]; then
> +    mkdir linux
> +    cd linux
> +    git init
> +    git fetch --depth=1 "${LINUX_URL}" "${LINUX_VERSION}"
> +    git checkout FETCH_HEAD
> +else
> +    MAJOR=${LINUX_VERSION%%.*}
> +    curl -fsSLO \
> +        https://cdn.kernel.org/pub/linux/kernel/v"${MAJOR}".x/linux-"${LINUX_VERSION}".tar.xz
> +    tar xf linux-"${LINUX_VERSION}".tar.xz
> +    cd linux-"${LINUX_VERSION}"
> +fi
>  
>  make defconfig
>  ./scripts/config --enable BRIDGE
> -- 
> git-series 0.9.1
>

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.