Update CI, run all Linux (aka Ubuntu-GCC5) based jobs in custom
containers. This decouples the CI environment from the virtual machine
images that Azure DevOps provides. The currently used ubuntu-18.04 image
has been deprecated for a while now and will finally be removed on
Dec 1st 2022.
The container image provides the required compiler toolchains and Qemu
for the supported architectures. These are then no longer downloaded at
runtime, avoiding CI failures due to download errors. This approach also
makes it easier to switch to other or newer compilers. It makes the CI
setup independent from the default images that Azure DevOps provides.
It can also help debugging CI problems, because the CI environment
can be reproduced on a local machine.
The container images are hosted on ghcr.io and are automatically
generated using GitHub Actions. The Dockerfiles are maintained in the
Tianocore "containers" repository:
https://github.com/tianocore/containers.
The current image is based on Fedora 35, with gcc 11. Fedora was chosen
because of its fast release cycle which makes it easy to keep the
toolchains up-to-date.
Some further possible changes not included in this series:
- The Tianocore/containers repository provides stack of layered images.
One image for general purpose (build+test) and build-only jobs.
The build+test image is based on the build-only one and adds Qemu,
for the testing job that involve Qemu. The work in the image side
is done, we just need to change the CI setup accordingly.
This patch set uses the build+test images for all jobs.
- Further reduce the number of external dependencies that need to be
downloaded at runtime. Candidates are iasl and nasm, which are already
included in the image but not used yet.
PR: https://github.com/tianocore/edk2/pull/3696
v10:
- Split commits by package
- use ubuntu-22.04 as vm_image (instread of the floating ubuntu-latest)
- some fixups around the Python version template argument and defaults.yml
- fix some commit messages
- add license statement to defaults.yml
v9:
- Drop the "Don't install cspell" patch
- Use explicitly use ubuntu-22.04 as vm_image
v8:
- Use updated container image that contains gcc for LoongArch64.
- Remove ext_dep files for the LoongArch64 gcc.
- Don't change the scopes in .pytool/CISettings.py if running Linux CI
it.
- Split commits that touch multiple packages.
- Use the smaller "build" image for jobs that allow it.
- Add a CI template file as a central place to define the default Python version
and use it where needed.
v7:
- Rebase to latest master branch.
- Use latest Fedora 35 CI image.
- Stop using the ubuntu-18.04 vm_image since this will no longer be available
after Dec 1st. Use ubuntu-latest instead.
v6:
- Include suggestions by Chris Fernald.
- Added a parameter for the container image to the job template, makes usage
of containers optional.
- Added a parameter to configure the Python version to download. Allows
using Python from the VM/container image also.
- Restructure the commits (no further functional changes).
v5:
- Update image
v4:
- Use the latest image from the tianocode/containers repository which
- does not include acpica-tools
- includes Pyhton 3.10
v3:
- Use the latest image from the tianocode/containers repository which
pins down version numbers of gcc, iasl, and nasm in the Dockerfile.
v2:
- Images are now hosted under the Tianocore Organization
https://github.com/tianocore/containers
v1:
- Thread: https://edk2.groups.io/g/devel/message/89058
- Images were hosted at https://github.com/osteffenrh/edk2-build-images
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Chris Fernald <chfernal@microsoft.com>
Oliver Steffen (17):
CI: make Python version configurable
ArmVirtPkg: CI: use Python version from defaults template
EmulatorPkg: CI: use Python version from defaults template
OvmfPkg: CI: use Python version from defaults template
CI: add ~/.local/bin to PATH (Linux only)
CI: Allow running in a container.
CI: Use Fedora 35 container (Linux only)
ArmVirtPkg: CI: Use Fedora 35 container (Linux only)
EmulatorPkg: CI: Use Fedora 35 container (Linux only)
OvmfPkg: CI: Use Fedora 35 container (Linux only)
.pytool: CISettings.py: don't add scopes for GCC
BaseTools: remove ext_dep files for gcc
ArmVirtPkg: CI: use ubuntu-22.04 vm_image (Linux only)
EmulatorPgk: CI: use ubuntu-22.04 vm_image (Linux only)
OvmfPkg: CI: use ubuntu-22.04 vm_image (Linux only)
CI: use ubuntu-22.04 image (Linux only)
do not merge: modify *.dsc to trigger CI
ArmPkg/ArmPkg.dsc | 1 +
.../ArmCrashDumpDxe/ArmCrashDumpDxe.dsc | 1 +
ArmPlatformPkg/ArmPlatformPkg.dsc | 1 +
ArmVirtPkg/ArmVirtCloudHv.dsc | 1 +
ArmVirtPkg/ArmVirtKvmTool.dsc | 1 +
ArmVirtPkg/ArmVirtQemu.dsc | 1 +
ArmVirtPkg/ArmVirtQemuKernel.dsc | 1 +
ArmVirtPkg/ArmVirtXen.dsc | 1 +
CryptoPkg/CryptoPkg.dsc | 1 +
CryptoPkg/Test/CryptoPkgHostUnitTest.dsc | 1 +
DynamicTablesPkg/DynamicTablesPkg.dsc | 1 +
EmbeddedPkg/EmbeddedPkg.dsc | 1 +
EmulatorPkg/EmulatorPkg.dsc | 1 +
FatPkg/FatPkg.dsc | 1 +
FmpDevicePkg/FmpDevicePkg.dsc | 1 +
FmpDevicePkg/Test/FmpDeviceHostPkgTest.dsc | 1 +
IntelFsp2Pkg/IntelFsp2Pkg.dsc | 1 +
IntelFsp2Pkg/Tools/Tests/QemuFspPkg.dsc | 1 +
IntelFsp2WrapperPkg/IntelFsp2WrapperPkg.dsc | 1 +
MdeModulePkg/MdeModulePkg.dsc | 1 +
MdeModulePkg/Test/MdeModulePkgHostTest.dsc | 1 +
MdePkg/MdePkg.dsc | 1 +
MdePkg/Test/MdePkgHostTest.dsc | 1 +
NetworkPkg/NetworkPkg.dsc | 1 +
OvmfPkg/AmdSev/AmdSevX64.dsc | 1 +
OvmfPkg/Bhyve/BhyveX64.dsc | 1 +
OvmfPkg/CloudHv/CloudHvX64.dsc | 1 +
OvmfPkg/IntelTdx/IntelTdxX64.dsc | 1 +
OvmfPkg/Microvm/MicrovmX64.dsc | 1 +
OvmfPkg/OvmfPkgIa32.dsc | 1 +
OvmfPkg/OvmfPkgIa32X64.dsc | 1 +
OvmfPkg/OvmfPkgX64.dsc | 1 +
OvmfPkg/OvmfXen.dsc | 1 +
PcAtChipsetPkg/PcAtChipsetPkg.dsc | 1 +
PrmPkg/PrmPkg.dsc | 1 +
PrmPkg/Test/PrmPkgHostTest.dsc | 1 +
RedfishPkg/RedfishPkg.dsc | 1 +
SecurityPkg/SecurityPkg.dsc | 1 +
SecurityPkg/Test/SecurityPkgHostTest.dsc | 1 +
ShellPkg/ShellPkg.dsc | 1 +
SignedCapsulePkg/SignedCapsulePkg.dsc | 1 +
SourceLevelDebugPkg/SourceLevelDebugPkg.dsc | 1 +
StandaloneMmPkg/StandaloneMmPkg.dsc | 1 +
UefiCpuPkg/Test/UefiCpuPkgHostTest.dsc | 1 +
UefiCpuPkg/UefiCpuPkg.dsc | 1 +
UefiPayloadPkg/UefiPayloadPkg.dsc | 1 +
.../Test/UnitTestFrameworkPkgHostTest.dsc | 1 +
UnitTestFrameworkPkg/UnitTestFrameworkPkg.dsc | 1 +
.azurepipelines/Ubuntu-GCC5.yml | 4 +++-
.azurepipelines/Windows-VS2019.yml | 4 ++++
.../templates/basetools-build-steps.yml | 9 --------
.azurepipelines/templates/defaults.yml | 11 ++++++++++
.../templates/platform-build-run-steps.yml | 12 +++++++++-
.../templates/pr-gate-build-job.yml | 6 +++++
.azurepipelines/templates/pr-gate-steps.yml | 12 ++++++++--
.pytool/CISettings.py | 9 --------
.../.azurepipelines/Ubuntu-GCC5.yml | 9 ++++----
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------
...gcc_loongarch64_unknown_linux_ext_dep.yaml | 22 -------------------
.../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 -------------------
.../.azurepipelines/Ubuntu-GCC5.yml | 5 ++++-
.../.azurepipelines/Windows-VS2019.yml | 4 ++++
.../.azurepipelines/Ubuntu-GCC5.yml | 9 ++++----
.../.azurepipelines/Windows-VS2019.yml | 5 +++++
65 files changed, 114 insertions(+), 119 deletions(-)
create mode 100644 .azurepipelines/templates/defaults.yml
delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml
delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml
--
2.38.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#96845): https://edk2.groups.io/g/devel/message/96845
Mute This Topic: https://groups.io/mt/95391913/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-