On Mon, 16 Jan 2023 at 18:41, Oliver Steffen <osteffen@redhat.com> wrote:
>
> Update CI, run all Linux (aka Ubuntu-GCC5) based jobs in custom
> containers. This decouples the CI environment from the virtual machine
> images that Azure DevOps provides. The currently used ubuntu-18.04 image
> has been deprecated for a while now and will be removed soon.
>
> The container image provides the required compiler toolchains and Qemu
> for the supported architectures. These are then no longer downloaded at
> runtime, avoiding CI failures due to download errors. This approach also
> makes it easier to switch to other or newer compilers. It makes the CI
> setup independent from the default images that Azure DevOps provides.
> It can also help debugging CI problems, because the CI environment
> can be reproduced on a local machine.
>
> The container images are hosted on ghcr.io and are automatically
> generated using GitHub Actions. The Dockerfiles are maintained in the
> Tianocore "containers" repository:
> https://github.com/tianocore/containers.
>
> The current image is based on Fedora 35, with gcc 11. Fedora was chosen
> because of its fast release cycle which makes it easy to keep the
> toolchains up-to-date.
>
> Some further possible changes not included in this series:
> - The Tianocore/containers repository provides stack of layered images.
> One image for general purpose (build+test) and build-only jobs.
> The build+test image is based on the build-only one and adds Qemu,
> for the testing job that involve Qemu. The work in the image side
> is done, we just need to change the CI setup accordingly.
> This patch set uses the build+test images for all jobs.
> - Further reduce the number of external dependencies that need to be
> downloaded at runtime. Candidates are iasl and nasm, which are already
> included in the image but not used yet.
>
> PR: https://github.com/tianocore/edk2/pull/3904
>
> v11:
> - Rebase to current master branch
> - Use lasted image, includes Qemu 7.2
> - Set container image URL in a central place (the defaults.yml template)
>
> v10:
> - Split commits by package
> - Use ubuntu-22.04 as vm_image (instead of the floating ubuntu-latest)
> - Some fixups around the Python version template argument and defaults.yml
> - Fix some commit messages
> - Add license statement to defaults.yml
>
> v9:
> - Drop the "Don't install cspell" patch
> - Use explicitly use ubuntu-22.04 as vm_image
>
> v8:
> - Use updated container image that contains gcc for LoongArch64.
> - Remove ext_dep files for the LoongArch64 gcc.
> - Don't change the scopes in .pytool/CISettings.py if running Linux CI
> it.
> - Split commits that touch multiple packages.
> - Use the smaller "build" image for jobs that allow it.
> - Add a CI template file as a central place to define the default Python version
> and use it where needed.
>
> v7:
> - Rebase to latest master branch.
> - Use latest Fedora 35 CI image.
> - Stop using the ubuntu-18.04 vm_image since this will no longer be available
> after Dec 1st. Use ubuntu-latest instead.
>
> v6:
> - Include suggestions by Chris Fernald.
> - Added a parameter for the container image to the job template, makes usage
> of containers optional.
> - Added a parameter to configure the Python version to download. Allows
> using Python from the VM/container image also.
> - Restructure the commits (no further functional changes).
>
> v5:
> - Update image
>
> v4:
> - Use the latest image from the tianocode/containers repository which
> - does not include acpica-tools
> - includes Pyhton 3.10
>
> v3:
> - Use the latest image from the tianocode/containers repository which
> pins down version numbers of gcc, iasl, and nasm in the Dockerfile.
>
> v2:
> - Images are now hosted under the Tianocore Organization
> https://github.com/tianocore/containers
>
> v1:
> - Thread: https://edk2.groups.io/g/devel/message/89058
> - Images were hosted at https://github.com/osteffenrh/edk2-build-images
>
> Signed-off-by: Oliver Steffen <osteffen@redhat.com>
> Acked-by: Ard Biesheuvel <ardb@kernel.org>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Reviewed-by: Chris Fernald <chfernal@microsoft.com>
>
> Oliver Steffen (17):
> CI: make Python version configurable
> ArmVirtPkg: CI: use Python version from defaults template
> EmulatorPkg: CI: use Python version from defaults template
> OvmfPkg: CI: use Python version from defaults template
> CI: add ~/.local/bin to PATH (Linux only)
> CI: Allow running in a container.
> CI: Use Fedora 35 container (Linux only)
> ArmVirtPkg: CI: Use Fedora 35 container (Linux only)
> EmulatorPkg: CI: Use Fedora 35 container (Linux only)
> OvmfPkg: CI: Use Fedora 35 container (Linux only)
> .pytool: CISettings.py: don't add scopes for GCC
> BaseTools: remove ext_dep files for gcc
> ArmVirtPkg: CI: use ubuntu-22.04 vm_image (Linux only)
> EmulatorPgk: CI: use ubuntu-22.04 vm_image (Linux only)
> OvmfPkg: CI: use ubuntu-22.04 vm_image (Linux only)
> CI: use ubuntu-22.04 image (Linux only)
Thanks
I have created a PR here
https://github.com/tianocore/edk2/pull/3913
with all the acks added.
> OfmvPkg: CI: Qemu tests: set CPU to Broadwell
>
... and I omitted this one (as discussed on the thread)
> .azurepipelines/Ubuntu-GCC5.yml | 12 +++++-----
> .azurepipelines/Windows-VS2019.yml | 4 ++++
> .../templates/basetools-build-steps.yml | 9 --------
> .azurepipelines/templates/defaults.yml | 12 ++++++++++
> .../templates/platform-build-run-steps.yml | 12 +++++++++-
> .../templates/pr-gate-build-job.yml | 6 +++++
> .azurepipelines/templates/pr-gate-steps.yml | 12 ++++++++--
> .pytool/CISettings.py | 9 --------
> .../.azurepipelines/Ubuntu-GCC5.yml | 12 +++++-----
> BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------
> BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------
> ...gcc_loongarch64_unknown_linux_ext_dep.yaml | 22 -------------------
> .../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 -------------------
> .../.azurepipelines/Ubuntu-GCC5.yml | 9 +++++++-
> .../.azurepipelines/Windows-VS2019.yml | 4 ++++
> .../.azurepipelines/Ubuntu-GCC5.yml | 12 +++++-----
> .../.azurepipelines/Windows-VS2019.yml | 5 +++++
> OvmfPkg/PlatformCI/PlatformBuildLib.py | 1 +
> 18 files changed, 82 insertions(+), 123 deletions(-)
> create mode 100644 .azurepipelines/templates/defaults.yml
> delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
> delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
> delete mode 100644 BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
> delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
>
> --
> 2.39.0
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#98727): https://edk2.groups.io/g/devel/message/98727
Mute This Topic: https://groups.io/mt/96311815/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-