[PATCH v2 1/5] automation: Add a Dockerfile for running FVP_Base jobs

Henry Wang posted 5 patches 2 years, 2 months ago
Only 3 patches received!
[PATCH v2 1/5] automation: Add a Dockerfile for running FVP_Base jobs
Posted by Henry Wang 2 years, 2 months ago
Fixed Virtual Platforms (FVPs) are complete simulations of an Arm
system, including processor, memory and peripherals. These are set
out in a "programmer's view", which gives programmers a comprehensive
model on which to build and test software. FVP can be configured to
different setups by its cmdline parameters, and hence having the FVP
in CI will provide us with the flexibility to test Arm features and
setups that we find difficult to use real hardware or emulators.

This commit adds a Dockerfile for the new arm64v8 container with
FVP installed, based on the debian bookworm-arm64v8 image. This
container will be used to run the FVP test jobs. Compared to the
debian bookworm-arm64v8 image, the packages in the newly added FVP
container does not contain the `u-boot-qemu`, and adds the `expect`
to run expect scripts introduced by following commits, `telnet` to
connect to FVP, and `tftpd-hpa` to provide the TFTP service for
the FVP.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
v2:
- Add Stefano's Reviewed-by tag.
---
 .../debian/bookworm-arm64v8-fvp.dockerfile    | 64 +++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 automation/build/debian/bookworm-arm64v8-fvp.dockerfile

diff --git a/automation/build/debian/bookworm-arm64v8-fvp.dockerfile b/automation/build/debian/bookworm-arm64v8-fvp.dockerfile
new file mode 100644
index 0000000000..3b87dc5a5b
--- /dev/null
+++ b/automation/build/debian/bookworm-arm64v8-fvp.dockerfile
@@ -0,0 +1,64 @@
+FROM --platform=linux/arm64/v8 debian:bookworm
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ARG FVP_BASE_VERSION="11.23_9_Linux64_armv8l"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        zlib1g-dev \
+        libncurses5-dev \
+        libssl-dev \
+        python3-dev \
+        python3-setuptools \
+        xorg-dev \
+        uuid-dev \
+        libyajl-dev \
+        libaio-dev \
+        libglib2.0-dev \
+        clang \
+        libpixman-1-dev \
+        pkg-config \
+        flex \
+        bison \
+        acpica-tools \
+        libfdt-dev \
+        bin86 \
+        bcc \
+        liblzma-dev \
+        libnl-3-dev \
+        ocaml-nox \
+        libfindlib-ocaml-dev \
+        markdown \
+        transfig \
+        pandoc \
+        checkpolicy \
+        wget \
+        git \
+        nasm \
+        # for test phase, fvp-smoke-* jobs
+        u-boot-tools \
+        expect \
+        device-tree-compiler \
+        curl \
+        cpio \
+        busybox-static \
+        telnet \
+        tftpd-hpa \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
+
+RUN wget https://developer.arm.com/-/media/Files/downloads/ecosystem-models/FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz && \
+    mkdir -p /FVP/FVP_Base_RevC-2xAEMvA && \
+    tar -xvzf FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz -C /FVP/FVP_Base_RevC-2xAEMvA && \
+    rm FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz
-- 
2.25.1
Re: [PATCH v2 1/5] automation: Add a Dockerfile for running FVP_Base jobs
Posted by Julien Grall 2 years, 2 months ago
Hi,

On 08/12/2023 05:46, Henry Wang wrote:
> Fixed Virtual Platforms (FVPs) are complete simulations of an Arm
> system, including processor, memory and peripherals. These are set
> out in a "programmer's view", which gives programmers a comprehensive
> model on which to build and test software. FVP can be configured to
> different setups by its cmdline parameters, and hence having the FVP
> in CI will provide us with the flexibility to test Arm features and
> setups that we find difficult to use real hardware or emulators.
> 
> This commit adds a Dockerfile for the new arm64v8 container with
> FVP installed, based on the debian bookworm-arm64v8 image. This
> container will be used to run the FVP test jobs. Compared to the
> debian bookworm-arm64v8 image, the packages in the newly added FVP
> container does not contain the `u-boot-qemu`, and adds the `expect`
> to run expect scripts introduced by following commits, `telnet` to
> connect to FVP, and `tftpd-hpa` to provide the TFTP service for
> the FVP.
> 
> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> v2:
> - Add Stefano's Reviewed-by tag.
> ---
>   .../debian/bookworm-arm64v8-fvp.dockerfile    | 64 +++++++++++++++++++
>   1 file changed, 64 insertions(+)
>   create mode 100644 automation/build/debian/bookworm-arm64v8-fvp.dockerfile
> 
> diff --git a/automation/build/debian/bookworm-arm64v8-fvp.dockerfile b/automation/build/debian/bookworm-arm64v8-fvp.dockerfile
> new file mode 100644
> index 0000000000..3b87dc5a5b
> --- /dev/null
> +++ b/automation/build/debian/bookworm-arm64v8-fvp.dockerfile
> @@ -0,0 +1,64 @@
> +FROM --platform=linux/arm64/v8 debian:bookworm
> +LABEL maintainer.name="The Xen Project" \
> +      maintainer.email="xen-devel@lists.xenproject.org"
> +
> +ARG FVP_BASE_VERSION="11.23_9_Linux64_armv8l"
> +
> +ENV DEBIAN_FRONTEND=noninteractive
> +ENV USER root
> +
> +RUN mkdir /build
> +WORKDIR /build
> +
> +# build depends
> +RUN apt-get update && \
> +    apt-get --quiet --yes install \
> +        build-essential \
> +        zlib1g-dev \
> +        libncurses5-dev \
> +        libssl-dev \
> +        python3-dev \
> +        python3-setuptools \
> +        xorg-dev \
> +        uuid-dev \
> +        libyajl-dev \
> +        libaio-dev \
> +        libglib2.0-dev \
> +        clang \
> +        libpixman-1-dev \
> +        pkg-config \
> +        flex \
> +        bison \
> +        acpica-tools \
> +        libfdt-dev \
> +        bin86 \
> +        bcc \
> +        liblzma-dev \
> +        libnl-3-dev \
> +        ocaml-nox \
> +        libfindlib-ocaml-dev \
> +        markdown \
> +        transfig \
> +        pandoc \
> +        checkpolicy \
> +        wget \
> +        git \
> +        nasm \
> +        # for test phase, fvp-smoke-* jobs
> +        u-boot-tools \
> +        expect \
> +        device-tree-compiler \
> +        curl \
> +        cpio \
> +        busybox-static \
> +        telnet \
> +        tftpd-hpa \
> +        && \
> +        apt-get autoremove -y && \
> +        apt-get clean && \
> +        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
> +
> +RUN wget https://developer.arm.com/-/media/Files/downloads/ecosystem-models/FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz && \

I vaguely recall some discussions on whether it was ok for us to publish 
a container with the FVP model due to the license agreement.

I guess this has now been resolved because the download can be done 
without sign-in to the account. Can you confirm?

It would also be good that the commit message indicates whether there is 
any implicit license agreement from Xen Project (or any user that decide 
to use our scripts).

Cheers,

-- 
Julien Grall
Re: [PATCH v2 1/5] automation: Add a Dockerfile for running FVP_Base jobs
Posted by Henry Wang 2 years, 2 months ago
Hi Julien,

> On Dec 8, 2023, at 20:30, Julien Grall <julien@xen.org> wrote:
> 
> Hi,
> 
> On 08/12/2023 05:46, Henry Wang wrote:
>> Fixed Virtual Platforms (FVPs) are complete simulations of an Arm
>> system, including processor, memory and peripherals. These are set
>> out in a "programmer's view", which gives programmers a comprehensive
>> model on which to build and test software. FVP can be configured to
>> different setups by its cmdline parameters, and hence having the FVP
>> in CI will provide us with the flexibility to test Arm features and
>> setups that we find difficult to use real hardware or emulators.
>> This commit adds a Dockerfile for the new arm64v8 container with
>> FVP installed, based on the debian bookworm-arm64v8 image. This
>> container will be used to run the FVP test jobs. Compared to the
>> debian bookworm-arm64v8 image, the packages in the newly added FVP
>> container does not contain the `u-boot-qemu`, and adds the `expect`
>> to run expect scripts introduced by following commits, `telnet` to
>> connect to FVP, and `tftpd-hpa` to provide the TFTP service for
>> the FVP.
>> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>> ---
>> v2:
>> - Add Stefano's Reviewed-by tag.
>> ---
>> +
>> +RUN wget https://developer.arm.com/-/media/Files/downloads/ecosystem-models/FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz && \
> 
> I vaguely recall some discussions on whether it was ok for us to publish a container with the FVP model due to the license agreement.
> 
> I guess this has now been resolved because the download can be done without sign-in to the account. Can you confirm?

Yes, quoting some words from the people we asked internally:
(the page referred to is https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms):

"All the FVPs referenced on this page that you are interested in are licensed under
lightweight Eco System EULA that has no restrictions on the redistribution.”

"So, yes, we can ship container images containing the FVP and the license on the FVP will remain as is.”

"No issues with redistributing the model package in a Docker container, as long as the EULA in included."

> It would also be good that the commit message indicates whether there is any implicit license agreement from Xen Project (or any user that decide to use our scripts).

I think it is the “END USER LICENSE AGREEMENT FOR ARM ECOSYSTEM MODELS”?

Kind regards,
Henry

> 
> Cheers,
> 
> -- 
> Julien Grall

Re: [PATCH v2 1/5] automation: Add a Dockerfile for running FVP_Base jobs
Posted by Julien Grall 2 years, 2 months ago

On 08/12/2023 12:59, Henry Wang wrote:
> Hi Julien,

Hi,

>> On Dec 8, 2023, at 20:30, Julien Grall <julien@xen.org> wrote:
>>
>> Hi,
>>
>> On 08/12/2023 05:46, Henry Wang wrote:
>>> Fixed Virtual Platforms (FVPs) are complete simulations of an Arm
>>> system, including processor, memory and peripherals. These are set
>>> out in a "programmer's view", which gives programmers a comprehensive
>>> model on which to build and test software. FVP can be configured to
>>> different setups by its cmdline parameters, and hence having the FVP
>>> in CI will provide us with the flexibility to test Arm features and
>>> setups that we find difficult to use real hardware or emulators.
>>> This commit adds a Dockerfile for the new arm64v8 container with
>>> FVP installed, based on the debian bookworm-arm64v8 image. This
>>> container will be used to run the FVP test jobs. Compared to the
>>> debian bookworm-arm64v8 image, the packages in the newly added FVP
>>> container does not contain the `u-boot-qemu`, and adds the `expect`
>>> to run expect scripts introduced by following commits, `telnet` to
>>> connect to FVP, and `tftpd-hpa` to provide the TFTP service for
>>> the FVP.
>>> Signed-off-by: Henry Wang <Henry.Wang@arm.com>
>>> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
>>> ---
>>> v2:
>>> - Add Stefano's Reviewed-by tag.
>>> ---
>>> +
>>> +RUN wget https://developer.arm.com/-/media/Files/downloads/ecosystem-models/FVP_Base_RevC-2xAEMvA_${FVP_BASE_VERSION}.tgz && \
>>
>> I vaguely recall some discussions on whether it was ok for us to publish a container with the FVP model due to the license agreement.
>>
>> I guess this has now been resolved because the download can be done without sign-in to the account. Can you confirm?
> 
> Yes, quoting some words from the people we asked internally:
> (the page referred to is https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms):
> 
> "All the FVPs referenced on this page that you are interested in are licensed under
> lightweight Eco System EULA that has no restrictions on the redistribution.”
> 
> "So, yes, we can ship container images containing the FVP and the license on the FVP will remain as is.”
> 
> "No issues with redistributing the model package in a Docker container, as long as the EULA in included."

Thanks for checking. In the current form, I don't think it is easy to 
know that the FVP has a specific license. I think this should be written 
down at the top of the container file. Something:

"The FVP is license under... Please read the file in ... for more details".

> 
>> It would also be good that the commit message indicates whether there is any implicit license agreement from Xen Project (or any user that decide to use our scripts).
> 
> I think it is the “END USER LICENSE AGREEMENT FOR ARM ECOSYSTEM MODELS”?

It looks like it.

Cheers,

-- 
Julien Grall