1. Patchew
  2. QEMU
  3. [PATCH for 10.1 00/13] documentation updates and test tweaks
  4. View patch

[PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross

Alex Bennée posted 13 patches 3 months, 3 weeks ago
Maintainers: Paolo Bonzini <pbonzini@redhat.com>, "Alex Bennée" <alex.bennee@linaro.org>, Thomas Huth <thuth@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>
[PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross
Posted by Alex Bennée 3 months, 3 weeks ago 
When building on non-x86 we get a bunch but not all of the compilers.
Handle this in the Dockerfile by probing the arch and expanding the
list available.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .../dockerfiles/debian-all-test-cross.docker  | 31 ++++++++++---------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
index 5aa43749ebe..16a83241270 100644
--- a/tests/docker/dockerfiles/debian-all-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
@@ -23,7 +23,9 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         bison \
         ccache \
         clang  \
+        dpkg-dev \
         flex \
+        gcc \
         git \
         libclang-rt-dev \
         ninja-build \
@@ -33,16 +35,11 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         python3-venv \
         python3-wheel
 
-RUN DEBIAN_FRONTEND=noninteractive eatmydata \
-        apt install -y --no-install-recommends \
-        gcc-aarch64-linux-gnu \
+# All the generally available compilers
+ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu \
         libc6-dev-arm64-cross \
         gcc-arm-linux-gnueabihf \
         libc6-dev-armhf-cross \
-        gcc-hppa-linux-gnu \
-        libc6-dev-hppa-cross \
-        gcc-m68k-linux-gnu \
-        libc6-dev-m68k-cross \
         gcc-mips-linux-gnu \
         libc6-dev-mips-cross \
         gcc-mips64-linux-gnuabi64 \
@@ -51,18 +48,24 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         libc6-dev-mips64el-cross \
         gcc-mipsel-linux-gnu \
         libc6-dev-mipsel-cross \
-        gcc-powerpc-linux-gnu \
-        libc6-dev-powerpc-cross \
-        gcc-powerpc64-linux-gnu \
-        libc6-dev-ppc64-cross \
         gcc-powerpc64le-linux-gnu \
         libc6-dev-ppc64el-cross \
         gcc-riscv64-linux-gnu \
         libc6-dev-riscv64-cross \
         gcc-s390x-linux-gnu \
-        libc6-dev-s390x-cross \
-        gcc-sparc64-linux-gnu \
-        libc6-dev-sparc64-cross && \
+        libc6-dev-s390x-cross
+
+RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; fi
+RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; fi
+RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; fi
+RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; fi
+RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; fi
+
+RUN echo "compilers: ${AVAILABLE_COMPILERS}"
+
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+        apt install -y --no-install-recommends \
+        ${AVAILABLE_COMPILERS} && \
         dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
 
 
-- 
2.47.2
Re: [PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross
Posted by Manos Pitsidianakis 3 months, 3 weeks ago 
On Thu, Jul 24, 2025 at 2:00 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> When building on non-x86 we get a bunch but not all of the compilers.
> Handle this in the Dockerfile by probing the arch and expanding the
> list available.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .../dockerfiles/debian-all-test-cross.docker  | 31 ++++++++++---------
>  1 file changed, 17 insertions(+), 14 deletions(-)
>
> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
> index 5aa43749ebe..16a83241270 100644
> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
> @@ -23,7 +23,9 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>          bison \
>          ccache \
>          clang  \
> +        dpkg-dev \
>          flex \
> +        gcc \
>          git \
>          libclang-rt-dev \
>          ninja-build \
> @@ -33,16 +35,11 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>          python3-venv \
>          python3-wheel
>
> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> -        apt install -y --no-install-recommends \
> -        gcc-aarch64-linux-gnu \
> +# All the generally available compilers
> +ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu \
>          libc6-dev-arm64-cross \
>          gcc-arm-linux-gnueabihf \
>          libc6-dev-armhf-cross \
> -        gcc-hppa-linux-gnu \
> -        libc6-dev-hppa-cross \
> -        gcc-m68k-linux-gnu \
> -        libc6-dev-m68k-cross \
>          gcc-mips-linux-gnu \
>          libc6-dev-mips-cross \
>          gcc-mips64-linux-gnuabi64 \
> @@ -51,18 +48,24 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>          libc6-dev-mips64el-cross \
>          gcc-mipsel-linux-gnu \
>          libc6-dev-mipsel-cross \
> -        gcc-powerpc-linux-gnu \
> -        libc6-dev-powerpc-cross \
> -        gcc-powerpc64-linux-gnu \
> -        libc6-dev-ppc64-cross \
>          gcc-powerpc64le-linux-gnu \
>          libc6-dev-ppc64el-cross \
>          gcc-riscv64-linux-gnu \
>          libc6-dev-riscv64-cross \
>          gcc-s390x-linux-gnu \
> -        libc6-dev-s390x-cross \
> -        gcc-sparc64-linux-gnu \
> -        libc6-dev-sparc64-cross && \
> +        libc6-dev-s390x-cross
> +
> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; fi
> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; fi
> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; fi
> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; fi
> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; fi
> +
> +RUN echo "compilers: ${AVAILABLE_COMPILERS}"

Nitpick, each `RUN` command will create a new cached layer for the
container build. It makes more sense to fold them in a single `RUN`
step to avoid unnecessary layers. Does not make a big difference so
feel free to ignore.

Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>

> +
> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> +        apt install -y --no-install-recommends \
> +        ${AVAILABLE_COMPILERS} && \
>          dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
>
>
> --
> 2.47.2
>
>
Re: [PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross
Posted by Alex Bennée 3 months, 3 weeks ago 
Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:

> On Thu, Jul 24, 2025 at 2:00 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> When building on non-x86 we get a bunch but not all of the compilers.
>> Handle this in the Dockerfile by probing the arch and expanding the
>> list available.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  .../dockerfiles/debian-all-test-cross.docker  | 31 ++++++++++---------
>>  1 file changed, 17 insertions(+), 14 deletions(-)
>>
>> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> index 5aa43749ebe..16a83241270 100644
>> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
>> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> @@ -23,7 +23,9 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>          bison \
>>          ccache \
>>          clang  \
>> +        dpkg-dev \
>>          flex \
>> +        gcc \
>>          git \
>>          libclang-rt-dev \
>>          ninja-build \
>> @@ -33,16 +35,11 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>          python3-venv \
>>          python3-wheel
>>
>> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> -        apt install -y --no-install-recommends \
>> -        gcc-aarch64-linux-gnu \
>> +# All the generally available compilers
>> +ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu \
>>          libc6-dev-arm64-cross \
>>          gcc-arm-linux-gnueabihf \
>>          libc6-dev-armhf-cross \
>> -        gcc-hppa-linux-gnu \
>> -        libc6-dev-hppa-cross \
>> -        gcc-m68k-linux-gnu \
>> -        libc6-dev-m68k-cross \
>>          gcc-mips-linux-gnu \
>>          libc6-dev-mips-cross \
>>          gcc-mips64-linux-gnuabi64 \
>> @@ -51,18 +48,24 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>>          libc6-dev-mips64el-cross \
>>          gcc-mipsel-linux-gnu \
>>          libc6-dev-mipsel-cross \
>> -        gcc-powerpc-linux-gnu \
>> -        libc6-dev-powerpc-cross \
>> -        gcc-powerpc64-linux-gnu \
>> -        libc6-dev-ppc64-cross \
>>          gcc-powerpc64le-linux-gnu \
>>          libc6-dev-ppc64el-cross \
>>          gcc-riscv64-linux-gnu \
>>          libc6-dev-riscv64-cross \
>>          gcc-s390x-linux-gnu \
>> -        libc6-dev-s390x-cross \
>> -        gcc-sparc64-linux-gnu \
>> -        libc6-dev-sparc64-cross && \
>> +        libc6-dev-s390x-cross
>> +
>> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; fi
>> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; fi
>> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; fi
>> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; fi
>> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; fi
>> +
>> +RUN echo "compilers: ${AVAILABLE_COMPILERS}"
>
> Nitpick, each `RUN` command will create a new cached layer for the
> container build. It makes more sense to fold them in a single `RUN`
> step to avoid unnecessary layers. Does not make a big difference so
> feel free to ignore.

I did try to figure out how to do a multi-line shell with an env
expansion but wasn't able to get the escaping right. If you can suggest
the right runes please do ;-)

>
> Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
>
>> +
>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> +        apt install -y --no-install-recommends \
>> +        ${AVAILABLE_COMPILERS} && \
>>          dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
>>
>>
>> --
>> 2.47.2
>>
>>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro
Re: [PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross
Posted by Manos Pitsidianakis 3 months, 3 weeks ago 
On Thu, Jul 24, 2025 at 3:36 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:
>
> > On Thu, Jul 24, 2025 at 2:00 PM Alex Bennée <alex.bennee@linaro.org> wrote:
> >>
> >> When building on non-x86 we get a bunch but not all of the compilers.
> >> Handle this in the Dockerfile by probing the arch and expanding the
> >> list available.
> >>
> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> >> ---
> >>  .../dockerfiles/debian-all-test-cross.docker  | 31 ++++++++++---------
> >>  1 file changed, 17 insertions(+), 14 deletions(-)
> >>
> >> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
> >> index 5aa43749ebe..16a83241270 100644
> >> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
> >> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
> >> @@ -23,7 +23,9 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >>          bison \
> >>          ccache \
> >>          clang  \
> >> +        dpkg-dev \
> >>          flex \
> >> +        gcc \
> >>          git \
> >>          libclang-rt-dev \
> >>          ninja-build \
> >> @@ -33,16 +35,11 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >>          python3-venv \
> >>          python3-wheel
> >>
> >> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >> -        apt install -y --no-install-recommends \
> >> -        gcc-aarch64-linux-gnu \
> >> +# All the generally available compilers
> >> +ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu \
> >>          libc6-dev-arm64-cross \
> >>          gcc-arm-linux-gnueabihf \
> >>          libc6-dev-armhf-cross \
> >> -        gcc-hppa-linux-gnu \
> >> -        libc6-dev-hppa-cross \
> >> -        gcc-m68k-linux-gnu \
> >> -        libc6-dev-m68k-cross \
> >>          gcc-mips-linux-gnu \
> >>          libc6-dev-mips-cross \
> >>          gcc-mips64-linux-gnuabi64 \
> >> @@ -51,18 +48,24 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >>          libc6-dev-mips64el-cross \
> >>          gcc-mipsel-linux-gnu \
> >>          libc6-dev-mipsel-cross \
> >> -        gcc-powerpc-linux-gnu \
> >> -        libc6-dev-powerpc-cross \
> >> -        gcc-powerpc64-linux-gnu \
> >> -        libc6-dev-ppc64-cross \
> >>          gcc-powerpc64le-linux-gnu \
> >>          libc6-dev-ppc64el-cross \
> >>          gcc-riscv64-linux-gnu \
> >>          libc6-dev-riscv64-cross \
> >>          gcc-s390x-linux-gnu \
> >> -        libc6-dev-s390x-cross \
> >> -        gcc-sparc64-linux-gnu \
> >> -        libc6-dev-sparc64-cross && \
> >> +        libc6-dev-s390x-cross
> >> +
> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; fi
> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; fi
> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; fi
> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; fi
> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; fi
> >> +
> >> +RUN echo "compilers: ${AVAILABLE_COMPILERS}"
> >
> > Nitpick, each `RUN` command will create a new cached layer for the
> > container build. It makes more sense to fold them in a single `RUN`
> > step to avoid unnecessary layers. Does not make a big difference so
> > feel free to ignore.
>
> I did try to figure out how to do a multi-line shell with an env
> expansion but wasn't able to get the escaping right. If you can suggest
> the right runes please do ;-)

Like this?

RUN if dpkg-architecture -e amd64; then \
  export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
gcc-hppa-linux-gnu libc6-dev-hppa-cross"; \
  export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
gcc-m68k-linux-gnu libc6-dev-m68k-cross"; \
  export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; \
  export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; \
  export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; \
  fi && \
  echo "compilers: ${AVAILABLE_COMPILERS}"


>
> >
> > Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> >
> >> +
> >> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >> +        apt install -y --no-install-recommends \
> >> +        ${AVAILABLE_COMPILERS} && \
> >>          dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
> >>
> >>
> >> --
> >> 2.47.2
> >>
> >>
>
> --
> Alex Bennée
> Virtualisation Tech Lead @ Linaro

-- 
Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro Ltd
Re: [PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross
Posted by Alex Bennée 3 months, 3 weeks ago 
Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:

> On Thu, Jul 24, 2025 at 3:36 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:
>>
>> > On Thu, Jul 24, 2025 at 2:00 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>> >>
>> >> When building on non-x86 we get a bunch but not all of the compilers.
>> >> Handle this in the Dockerfile by probing the arch and expanding the
>> >> list available.
>> >>
>> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> >> ---
>> >>  .../dockerfiles/debian-all-test-cross.docker  | 31 ++++++++++---------
>> >>  1 file changed, 17 insertions(+), 14 deletions(-)
>> >>
>> >> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> >> index 5aa43749ebe..16a83241270 100644
>> >> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
>> >> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
>> >> @@ -23,7 +23,9 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >>          bison \
>> >>          ccache \
>> >>          clang  \
>> >> +        dpkg-dev \
>> >>          flex \
>> >> +        gcc \
>> >>          git \
>> >>          libclang-rt-dev \
>> >>          ninja-build \
>> >> @@ -33,16 +35,11 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >>          python3-venv \
>> >>          python3-wheel
>> >>
>> >> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >> -        apt install -y --no-install-recommends \
>> >> -        gcc-aarch64-linux-gnu \
>> >> +# All the generally available compilers
>> >> +ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu \
>> >>          libc6-dev-arm64-cross \
>> >>          gcc-arm-linux-gnueabihf \
>> >>          libc6-dev-armhf-cross \
>> >> -        gcc-hppa-linux-gnu \
>> >> -        libc6-dev-hppa-cross \
>> >> -        gcc-m68k-linux-gnu \
>> >> -        libc6-dev-m68k-cross \
>> >>          gcc-mips-linux-gnu \
>> >>          libc6-dev-mips-cross \
>> >>          gcc-mips64-linux-gnuabi64 \
>> >> @@ -51,18 +48,24 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >>          libc6-dev-mips64el-cross \
>> >>          gcc-mipsel-linux-gnu \
>> >>          libc6-dev-mipsel-cross \
>> >> -        gcc-powerpc-linux-gnu \
>> >> -        libc6-dev-powerpc-cross \
>> >> -        gcc-powerpc64-linux-gnu \
>> >> -        libc6-dev-ppc64-cross \
>> >>          gcc-powerpc64le-linux-gnu \
>> >>          libc6-dev-ppc64el-cross \
>> >>          gcc-riscv64-linux-gnu \
>> >>          libc6-dev-riscv64-cross \
>> >>          gcc-s390x-linux-gnu \
>> >> -        libc6-dev-s390x-cross \
>> >> -        gcc-sparc64-linux-gnu \
>> >> -        libc6-dev-sparc64-cross && \
>> >> +        libc6-dev-s390x-cross
>> >> +
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; fi
>> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; fi
>> >> +
>> >> +RUN echo "compilers: ${AVAILABLE_COMPILERS}"
>> >
>> > Nitpick, each `RUN` command will create a new cached layer for the
>> > container build. It makes more sense to fold them in a single `RUN`
>> > step to avoid unnecessary layers. Does not make a big difference so
>> > feel free to ignore.
>>
>> I did try to figure out how to do a multi-line shell with an env
>> expansion but wasn't able to get the escaping right. If you can suggest
>> the right runes please do ;-)
>
> Like this?
>
> RUN if dpkg-architecture -e amd64; then \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-hppa-linux-gnu libc6-dev-hppa-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-m68k-linux-gnu libc6-dev-m68k-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; \
>   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; \
>   fi && \
>   echo "compilers: ${AVAILABLE_COMPILERS}"
>

Nope, the continuation has to run for all the lines of the RUN
statement.

  Processing triggers for libc-bin (2.36-9+deb12u10) ...
  --> 703d5c21c6e4
  STEP 5/18: ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu         libc6-dev-arm64-cross         gcc-arm-linux-gnueabihf         libc6-dev-armhf-cross         gcc-mips-linux-gnu         libc6-dev-mips-cross         gcc-mips64-linux-gnuabi64         libc6-dev-mips64-cross         gcc-mips64el-linux-gnuabi64         libc6-dev-mips64el-cross         gcc-mipsel-linux-gnu         libc6-dev-mipsel-cross         gcc-powerpc64le-linux-gnu         libc6-dev-ppc64el-cross         gcc-riscv64-linux-gnu         libc6-dev-riscv64-cross         gcc-s390x-linux-gnu         libc6-dev-s390x-cross
  --> e6fa99881c94
  STEP 6/18: RUN if dpkg-architecture -e amd64; then   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
  /bin/sh: 1: Syntax error: Unterminated quoted string
  Error: building at STEP "RUN if dpkg-architecture -e amd64; then   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}": while running runtime: exit status 2
  make: *** [tests/docker/Makefile.include:40: docker-image-debian-all-test-cross] Error 2

>
>>
>> >
>> > Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
>> >
>> >> +
>> >> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
>> >> +        apt install -y --no-install-recommends \
>> >> +        ${AVAILABLE_COMPILERS} && \
>> >>          dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
>> >>
>> >>
>> >> --
>> >> 2.47.2
>> >>
>> >>
>>
>> --
>> Alex Bennée
>> Virtualisation Tech Lead @ Linaro

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro
Re: [PATCH for 10.1 13/13] tests/docker: handle host-arch selection for all-test-cross
Posted by Manos Pitsidianakis 3 months, 3 weeks ago 
On Fri, Jul 25, 2025 at 2:57 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:
>
> > On Thu, Jul 24, 2025 at 3:36 PM Alex Bennée <alex.bennee@linaro.org> wrote:
> >>
> >> Manos Pitsidianakis <manos.pitsidianakis@linaro.org> writes:
> >>
> >> > On Thu, Jul 24, 2025 at 2:00 PM Alex Bennée <alex.bennee@linaro.org> wrote:
> >> >>
> >> >> When building on non-x86 we get a bunch but not all of the compilers.
> >> >> Handle this in the Dockerfile by probing the arch and expanding the
> >> >> list available.
> >> >>
> >> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> >> >> ---
> >> >>  .../dockerfiles/debian-all-test-cross.docker  | 31 ++++++++++---------
> >> >>  1 file changed, 17 insertions(+), 14 deletions(-)
> >> >>
> >> >> diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
> >> >> index 5aa43749ebe..16a83241270 100644
> >> >> --- a/tests/docker/dockerfiles/debian-all-test-cross.docker
> >> >> +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
> >> >> @@ -23,7 +23,9 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >> >>          bison \
> >> >>          ccache \
> >> >>          clang  \
> >> >> +        dpkg-dev \
> >> >>          flex \
> >> >> +        gcc \
> >> >>          git \
> >> >>          libclang-rt-dev \
> >> >>          ninja-build \
> >> >> @@ -33,16 +35,11 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >> >>          python3-venv \
> >> >>          python3-wheel
> >> >>
> >> >> -RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >> >> -        apt install -y --no-install-recommends \
> >> >> -        gcc-aarch64-linux-gnu \
> >> >> +# All the generally available compilers
> >> >> +ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu \
> >> >>          libc6-dev-arm64-cross \
> >> >>          gcc-arm-linux-gnueabihf \
> >> >>          libc6-dev-armhf-cross \
> >> >> -        gcc-hppa-linux-gnu \
> >> >> -        libc6-dev-hppa-cross \
> >> >> -        gcc-m68k-linux-gnu \
> >> >> -        libc6-dev-m68k-cross \
> >> >>          gcc-mips-linux-gnu \
> >> >>          libc6-dev-mips-cross \
> >> >>          gcc-mips64-linux-gnuabi64 \
> >> >> @@ -51,18 +48,24 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >> >>          libc6-dev-mips64el-cross \
> >> >>          gcc-mipsel-linux-gnu \
> >> >>          libc6-dev-mipsel-cross \
> >> >> -        gcc-powerpc-linux-gnu \
> >> >> -        libc6-dev-powerpc-cross \
> >> >> -        gcc-powerpc64-linux-gnu \
> >> >> -        libc6-dev-ppc64-cross \
> >> >>          gcc-powerpc64le-linux-gnu \
> >> >>          libc6-dev-ppc64el-cross \
> >> >>          gcc-riscv64-linux-gnu \
> >> >>          libc6-dev-riscv64-cross \
> >> >>          gcc-s390x-linux-gnu \
> >> >> -        libc6-dev-s390x-cross \
> >> >> -        gcc-sparc64-linux-gnu \
> >> >> -        libc6-dev-sparc64-cross && \
> >> >> +        libc6-dev-s390x-cross
> >> >> +
> >> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-hppa-linux-gnu libc6-dev-hppa-cross"; fi
> >> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-m68k-linux-gnu libc6-dev-m68k-cross"; fi
> >> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; fi
> >> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; fi
> >> >> +RUN if dpkg-architecture -e amd64; then export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS} gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; fi
> >> >> +
> >> >> +RUN echo "compilers: ${AVAILABLE_COMPILERS}"
> >> >
> >> > Nitpick, each `RUN` command will create a new cached layer for the
> >> > container build. It makes more sense to fold them in a single `RUN`
> >> > step to avoid unnecessary layers. Does not make a big difference so
> >> > feel free to ignore.
> >>
> >> I did try to figure out how to do a multi-line shell with an env
> >> expansion but wasn't able to get the escaping right. If you can suggest
> >> the right runes please do ;-)
> >
> > Like this?
> >
> > RUN if dpkg-architecture -e amd64; then \
> >   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> > gcc-hppa-linux-gnu libc6-dev-hppa-cross"; \
> >   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> > gcc-m68k-linux-gnu libc6-dev-m68k-cross"; \
> >   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> > gcc-powerpc-linux-gnu libc6-dev-powerpc-cross"; \
> >   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> > gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross"; \
> >   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
> > gcc-sparc64-linux-gnu libc6-dev-sparc64-cross"; \
> >   fi && \
> >   echo "compilers: ${AVAILABLE_COMPILERS}"
> >
>
> Nope, the continuation has to run for all the lines of the RUN
> statement.

Sorry, it seems Gmail textwrapped the lines. The ones without
continuation should be merged with the following one.


>
>   Processing triggers for libc-bin (2.36-9+deb12u10) ...
>   --> 703d5c21c6e4
>   STEP 5/18: ENV AVAILABLE_COMPILERS gcc-aarch64-linux-gnu         libc6-dev-arm64-cross         gcc-arm-linux-gnueabihf         libc6-dev-armhf-cross         gcc-mips-linux-gnu         libc6-dev-mips-cross         gcc-mips64-linux-gnuabi64         libc6-dev-mips64-cross         gcc-mips64el-linux-gnuabi64         libc6-dev-mips64el-cross         gcc-mipsel-linux-gnu         libc6-dev-mipsel-cross         gcc-powerpc64le-linux-gnu         libc6-dev-ppc64el-cross         gcc-riscv64-linux-gnu         libc6-dev-riscv64-cross         gcc-s390x-linux-gnu         libc6-dev-s390x-cross
>   --> e6fa99881c94
>   STEP 6/18: RUN if dpkg-architecture -e amd64; then   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}
>   /bin/sh: 1: Syntax error: Unterminated quoted string
>   Error: building at STEP "RUN if dpkg-architecture -e amd64; then   export AVAILABLE_COMPILERS="${AVAILABLE_COMPILERS}": while running runtime: exit status 2
>   make: *** [tests/docker/Makefile.include:40: docker-image-debian-all-test-cross] Error 2
>
> >
> >>
> >> >
> >> > Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
> >> >
> >> >> +
> >> >> +RUN DEBIAN_FRONTEND=noninteractive eatmydata \
> >> >> +        apt install -y --no-install-recommends \
> >> >> +        ${AVAILABLE_COMPILERS} && \
> >> >>          dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
> >> >>
> >> >>
> >> >> --
> >> >> 2.47.2
> >> >>
> >> >>
> >>
> >> --
> >> Alex Bennée
> >> Virtualisation Tech Lead @ Linaro
>
> --
> Alex Bennée
> Virtualisation Tech Lead @ Linaro

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.