[PATCH 7/8] Infrastructure for arm64 linux builds

Andrew Cooper posted 8 patches 8 months, 1 week ago
There is a newer version of this series
[PATCH 7/8] Infrastructure for arm64 linux builds
Posted by Andrew Cooper 8 months, 1 week ago
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
CC: Anthony PERARD <anthony.perard@vates.tech>
---
 .gitlab-ci.yml                            |  7 ++++++
 containerize                              |  1 +
 images/alpine/3.18-arm64-build.dockerfile | 27 +++++++++++++++++++++++
 scripts/build-linux.sh                    | 10 +++++++++
 4 files changed, 45 insertions(+)
 create mode 100644 images/alpine/3.18-arm64-build.dockerfile

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ff8dce7be05d..6e38c2f2a108 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -19,6 +19,13 @@ stages:
     exclude:
       - binaries/.gitignore
 
+.arm64-artifacts:
+  extends: .artifacts
+  tags:
+    - arm64
+  variables:
+    CONTAINER: alpine:3.18-arm64-build
+
 .x86_64-artifacts:
   extends: .artifacts
   tags:
diff --git a/containerize b/containerize
index c32e40f5a6d9..6664ef9821d0 100755
--- a/containerize
+++ b/containerize
@@ -24,6 +24,7 @@ die() {
 #
 BASE="registry.gitlab.com/xen-project/hardware/test-artifacts"
 case "_${CONTAINER}" in
+    _alpine-3.18-arm64-build) CONTAINER="${BASE}/alpine:3.18-arm64-build" ;;
     _alpine-x86_64-base) CONTAINER="${BASE}/alpine:x86_64-base" ;;
     _alpine-x86_64-build|_) CONTAINER="${BASE}/alpine:x86_64-build" ;;
 esac
diff --git a/images/alpine/3.18-arm64-build.dockerfile b/images/alpine/3.18-arm64-build.dockerfile
new file mode 100644
index 000000000000..c76ac435e8dd
--- /dev/null
+++ b/images/alpine/3.18-arm64-build.dockerfile
@@ -0,0 +1,27 @@
+# syntax=docker/dockerfile:1
+FROM --platform=linux/arm64/v8 alpine:3.18
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+RUN apk --no-cache add bash
+
+RUN bash -ex <<EOF
+      adduser -D user --shell bash
+
+      DEPS=(# Base environment
+            musl-dev
+            build-base
+            curl
+
+            # Linux build deps
+            flex
+            bison
+            perl
+            openssl-dev
+      )
+
+      apk add --no-cache "\${DEPS[@]}"
+EOF
+
+USER user
+WORKDIR /build
diff --git a/scripts/build-linux.sh b/scripts/build-linux.sh
index 5e25e958c0f3..1be20a38071d 100755
--- a/scripts/build-linux.sh
+++ b/scripts/build-linux.sh
@@ -21,6 +21,7 @@ cd linux-"${LINUX_VERSION}"
 make defconfig
 ./scripts/config --enable BRIDGE
 ./scripts/config --enable IGC
+./scripts/config --enable IPV6
 ./scripts/config --enable TUN
 
 case $UNAME in
@@ -32,6 +33,10 @@ case $UNAME in
             | grep '=m' \
             | sed 's/=m/=y/g' >> .config
         ;;
+
+    aarch64)
+        ./scripts/config --enable XEN_NETDEV_BACKEND
+        ;;
 esac
 
 make olddefconfig
@@ -41,4 +46,9 @@ case $UNAME in
         make -j$(nproc) bzImage
         cp arch/x86/boot/bzImage "${COPYDIR}"
         ;;
+
+    aarch64)
+        make -j$(nproc) Image
+        cp arch/arm64/boot/Image "${COPYDIR}"
+        ;;
 esac
-- 
2.39.5


Re: [PATCH 7/8] Infrastructure for arm64 linux builds
Posted by Andrew Cooper 8 months, 1 week ago
On 09/04/2025 5:37 pm, Andrew Cooper wrote:
> diff --git a/images/alpine/3.18-arm64-build.dockerfile b/images/alpine/3.18-arm64-build.dockerfile
> new file mode 100644
> index 000000000000..c76ac435e8dd
> --- /dev/null
> +++ b/images/alpine/3.18-arm64-build.dockerfile
> @@ -0,0 +1,27 @@
> +# syntax=docker/dockerfile:1
> +FROM --platform=linux/arm64/v8 alpine:3.18
> +LABEL maintainer.name="The Xen Project" \
> +      maintainer.email="xen-devel@lists.xenproject.org"
> +
> +RUN apk --no-cache add bash
> +
> +RUN bash -ex <<EOF
> +      adduser -D user --shell bash
> +
> +      DEPS=(# Base environment
> +            musl-dev
> +            build-base
> +            curl

It turns out we don't need musl-dev.  build-base depends on libc-dev
which musl-dev provides.  This was a bad dependency brought across from
the form we have in xen.git right now.

~Andrew

Re: [PATCH 7/8] Infrastructure for arm64 linux builds
Posted by Marek Marczykowski-Górecki 8 months, 1 week ago
On Wed, Apr 09, 2025 at 05:37:01PM +0100, Andrew Cooper wrote:
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> CC: Anthony PERARD <anthony.perard@vates.tech>
> ---
>  .gitlab-ci.yml                            |  7 ++++++
>  containerize                              |  1 +
>  images/alpine/3.18-arm64-build.dockerfile | 27 +++++++++++++++++++++++
>  scripts/build-linux.sh                    | 10 +++++++++
>  4 files changed, 45 insertions(+)
>  create mode 100644 images/alpine/3.18-arm64-build.dockerfile
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index ff8dce7be05d..6e38c2f2a108 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -19,6 +19,13 @@ stages:
>      exclude:
>        - binaries/.gitignore
>  
> +.arm64-artifacts:
> +  extends: .artifacts
> +  tags:
> +    - arm64
> +  variables:
> +    CONTAINER: alpine:3.18-arm64-build

Arm64 kernel used to be built in a Bookworm container. The build in alpine
has "find: unrecognized: -printf" in the middle of the build. It doesn't
fail outright, but something might be broken. I guess it's related to
initramfs - maybe some option can be disabled, to avoid this message?

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
Re: [PATCH 7/8] Infrastructure for arm64 linux builds
Posted by Andrew Cooper 8 months, 1 week ago
On 09/04/2025 6:11 pm, Marek Marczykowski-Górecki wrote:
> On Wed, Apr 09, 2025 at 05:37:01PM +0100, Andrew Cooper wrote:
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>> CC: Roger Pau Monné <roger.pau@citrix.com>
>> CC: Stefano Stabellini <sstabellini@kernel.org>
>> CC: Michal Orzel <michal.orzel@amd.com>
>> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>> CC: Anthony PERARD <anthony.perard@vates.tech>
>> ---
>>  .gitlab-ci.yml                            |  7 ++++++
>>  containerize                              |  1 +
>>  images/alpine/3.18-arm64-build.dockerfile | 27 +++++++++++++++++++++++
>>  scripts/build-linux.sh                    | 10 +++++++++
>>  4 files changed, 45 insertions(+)
>>  create mode 100644 images/alpine/3.18-arm64-build.dockerfile
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index ff8dce7be05d..6e38c2f2a108 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -19,6 +19,13 @@ stages:
>>      exclude:
>>        - binaries/.gitignore
>>  
>> +.arm64-artifacts:
>> +  extends: .artifacts
>> +  tags:
>> +    - arm64
>> +  variables:
>> +    CONTAINER: alpine:3.18-arm64-build
> Arm64 kernel used to be built in a Bookworm container.

So did x86 before they were moved across.

>  The build in alpine
> has "find: unrecognized: -printf" in the middle of the build. It doesn't
> fail outright, but something might be broken. I guess it's related to
> initramfs - maybe some option can be disabled, to avoid this message?

Nothing seems to break...

There's also a bad awk regex.  Alpine seem to fix this by using mawk,
except they then have to patch the Linux build system to take the
override properly.

I can't see anything obvious they do to fix this -printf warning.

~Andrew

Re: [PATCH 7/8] Infrastructure for arm64 linux builds
Posted by Andrew Cooper 8 months, 1 week ago
On 09/04/2025 6:15 pm, Andrew Cooper wrote:
> On 09/04/2025 6:11 pm, Marek Marczykowski-Górecki wrote:
>> On Wed, Apr 09, 2025 at 05:37:01PM +0100, Andrew Cooper wrote:
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> ---
>>> CC: Roger Pau Monné <roger.pau@citrix.com>
>>> CC: Stefano Stabellini <sstabellini@kernel.org>
>>> CC: Michal Orzel <michal.orzel@amd.com>
>>> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>>> CC: Anthony PERARD <anthony.perard@vates.tech>
>>> ---
>>>  .gitlab-ci.yml                            |  7 ++++++
>>>  containerize                              |  1 +
>>>  images/alpine/3.18-arm64-build.dockerfile | 27 +++++++++++++++++++++++
>>>  scripts/build-linux.sh                    | 10 +++++++++
>>>  4 files changed, 45 insertions(+)
>>>  create mode 100644 images/alpine/3.18-arm64-build.dockerfile
>>>
>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>> index ff8dce7be05d..6e38c2f2a108 100644
>>> --- a/.gitlab-ci.yml
>>> +++ b/.gitlab-ci.yml
>>> @@ -19,6 +19,13 @@ stages:
>>>      exclude:
>>>        - binaries/.gitignore
>>>  
>>> +.arm64-artifacts:
>>> +  extends: .artifacts
>>> +  tags:
>>> +    - arm64
>>> +  variables:
>>> +    CONTAINER: alpine:3.18-arm64-build
>> Arm64 kernel used to be built in a Bookworm container.
> So did x86 before they were moved across.
>
>>  The build in alpine
>> has "find: unrecognized: -printf" in the middle of the build. It doesn't
>> fail outright, but something might be broken. I guess it's related to
>> initramfs - maybe some option can be disabled, to avoid this message?
> Nothing seems to break...
>
> There's also a bad awk regex.  Alpine seem to fix this by using mawk,
> except they then have to patch the Linux build system to take the
> override properly.
>
> I can't see anything obvious they do to fix this -printf warning.

Yes I can.  We want findutils too.

x86 also wants diffutils too. (as I'm fixing that side of things too)

~Andrew

Re: [PATCH 7/8] Infrastructure for arm64 linux builds
Posted by Andrew Cooper 8 months, 1 week ago
On 10/04/2025 11:41 am, Andrew Cooper wrote:
> On 09/04/2025 6:15 pm, Andrew Cooper wrote:
>> On 09/04/2025 6:11 pm, Marek Marczykowski-Górecki wrote:
>>> On Wed, Apr 09, 2025 at 05:37:01PM +0100, Andrew Cooper wrote:
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>> ---
>>>> CC: Roger Pau Monné <roger.pau@citrix.com>
>>>> CC: Stefano Stabellini <sstabellini@kernel.org>
>>>> CC: Michal Orzel <michal.orzel@amd.com>
>>>> CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>>>> CC: Anthony PERARD <anthony.perard@vates.tech>
>>>> ---
>>>>  .gitlab-ci.yml                            |  7 ++++++
>>>>  containerize                              |  1 +
>>>>  images/alpine/3.18-arm64-build.dockerfile | 27 +++++++++++++++++++++++
>>>>  scripts/build-linux.sh                    | 10 +++++++++
>>>>  4 files changed, 45 insertions(+)
>>>>  create mode 100644 images/alpine/3.18-arm64-build.dockerfile
>>>>
>>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>>> index ff8dce7be05d..6e38c2f2a108 100644
>>>> --- a/.gitlab-ci.yml
>>>> +++ b/.gitlab-ci.yml
>>>> @@ -19,6 +19,13 @@ stages:
>>>>      exclude:
>>>>        - binaries/.gitignore
>>>>  
>>>> +.arm64-artifacts:
>>>> +  extends: .artifacts
>>>> +  tags:
>>>> +    - arm64
>>>> +  variables:
>>>> +    CONTAINER: alpine:3.18-arm64-build
>>> Arm64 kernel used to be built in a Bookworm container.
>> So did x86 before they were moved across.
>>
>>>  The build in alpine
>>> has "find: unrecognized: -printf" in the middle of the build. It doesn't
>>> fail outright, but something might be broken. I guess it's related to
>>> initramfs - maybe some option can be disabled, to avoid this message?
>> Nothing seems to break...
>>
>> There's also a bad awk regex.  Alpine seem to fix this by using mawk,
>> except they then have to patch the Linux build system to take the
>> override properly.
>>
>> I can't see anything obvious they do to fix this -printf warning.
> Yes I can.  We want findutils too.
>
> x86 also wants diffutils too. (as I'm fixing that side of things too)

Excellent.  Both x86 and ARM are clean with these changes:

https://gitlab.com/xen-project/hardware/test-artifacts/-/jobs/9682944850
https://gitlab.com/xen-project/hardware/test-artifacts/-/jobs/9682944867

The awk complaint has gone away too, so I suppose that's a knock-on effect.

~Andrew

Re: [PATCH 7/8] Infrastructure for arm64 linux builds
Posted by Marek Marczykowski-Górecki 8 months, 1 week ago
On Thu, Apr 10, 2025 at 11:57:23AM +0100, Andrew Cooper wrote:
> The awk complaint has gone away too, so I suppose that's a knock-on effect.

It isn't? 
https://gitlab.com/xen-project/hardware/test-artifacts/-/jobs/9683454411#L106

Anyway, hopefully harmless.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab