From nobody Tue Feb 10 04:16:54 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+78238+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+78238+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=arm.com ARC-Seal: i=1; a=rsa-sha256; t=1627404659; cv=none; d=zohomail.com; s=zohoarc; b=m8FmgYnKtMmSp+FkVw6tQg8Uy5D/pVPFxHJs3rW0OtmqxuQGsvknmZuIDsWWJS0Wdn4pLsjc48HS9XFe9PsXSCtvj72ldqFMe/1M+9mHciIVDfFIOvYoHc+IU+rcDQn7Op+ZmaiVujNP8YR2GEdnsc3U7kbqcRLZzGxHtqSRMmA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1627404659; h=Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To; bh=+x0EirpEaaYdvYsHj+IWpIEOO9w5F7m7GgCH+Eh0TCk=; b=DgJ+8vPDr9g2LL55UYY0TAQCLhvQ55k7IvGjaPtgirNZIgf1yuA4MAMUV+SEckc+9/PhVs0KqKBpd7PRT7qg5EHoB67JpwS60JNstpqwm9nUKMdcb37FWZUIdIYOnVS/P/lT0CkWmSZX3wgyEZihWN/1NM7gldPdkT7UCOmBVrU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+78238+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1627404659935655.0114079809949; Tue, 27 Jul 2021 09:50:59 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id IyhrYY1788612xmFxjoNLrh1; Tue, 27 Jul 2021 09:50:59 -0700 X-Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.3975.1627404658373310557 for ; Tue, 27 Jul 2021 09:50:58 -0700 X-Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E07BE139F; Tue, 27 Jul 2021 09:50:57 -0700 (PDT) X-Received: from e120189.home (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 522543F70D; Tue, 27 Jul 2021 09:50:56 -0700 (PDT) From: "PierreGondois" To: devel@edk2.groups.io, Sean Brogan , Bret Barkelew , Michael D Kinney , Liming Gao , Sami Mujawar Subject: [edk2-devel] [PATCH edk2-platforms v2 4/6] .azurepipelines: Add Azure Pipelines YML configuration files Date: Tue, 27 Jul 2021 17:50:02 +0100 Message-Id: <20210727165004.9470-5-Pierre.Gondois@arm.com> In-Reply-To: <20210727165004.9470-1-Pierre.Gondois@arm.com> References: <20210727165004.9470-1-Pierre.Gondois@arm.com> Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,pierre.gondois@arm.com X-Gm-Message-State: tOpNiGYPX0149fze5Bb7pbrOx1787277AA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1627404659; bh=0iTt45L2GlROCa/g/P2kcqhQsHPByLM5/bC+mKkAra0=; h=Date:From:Reply-To:Subject:To; b=jHY4DHUBZIQlqD34UD9QxEtK6BEcoTbykcVlqQQtRUcHV7t/4U4hMReQKfNWJfNb4Qe eCL6W1AERZc47edsRG5Aq4a+TZHLjwPT3tZLVFDFLr7modh6KbAf+/lgJguMwtzKpYt6G R0MqSazELoPRKyOQEDnDD0Jd7F8d5AUBk6Q= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1627404661886100002 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Pierre Gondois To enable CI support of the tianocore/edk2-platforms repository, add YML configuration files used to run Continuous Integration (CI) checks on Azure Pipelines agents. These files are copies of the files from the tianocore/edk2 repository. Any modification to the tianocore/edk2 files must be reflected on the tianocore/edk2-platforms copies. The following files have been modified: - .azurepipelines/templates/platform-build-run-steps.yml - .azurepipelines/templates/pr-gate-build-job.yml - .azurepipelines/templates/pr-gate-steps.yml The sections modified are marked with the following comments: -EDK2_PLATFORMS_MODIF_START -EDK2_PLATFORMS_MODIF_END Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3509 Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael D Kinney Cc: Liming Gao Cc: Sami Mujawar Signed-off-by: Sami Mujawar Signed-off-by: Pierre Gondois --- .azurepipelines/ReadMe.md | 50 ++++++ .azurepipelines/Ubuntu-GCC5.yml | 21 +++ .azurepipelines/Ubuntu-PatchCheck.yml | 36 +++++ .azurepipelines/Windows-VS2019.yml | 20 +++ .azurepipelines/templates/ReadMe.md | 59 +++++++ .../templates/basetools-build-steps.yml | 37 +++++ .../templates/platform-build-run-steps.yml | 144 ++++++++++++++++++ .../templates/pr-gate-build-job.yml | 63 ++++++++ .azurepipelines/templates/pr-gate-steps.yml | 142 +++++++++++++++++ .../templates/spell-check-prereq-steps.yml | 22 +++ 10 files changed, 594 insertions(+) create mode 100644 .azurepipelines/ReadMe.md create mode 100644 .azurepipelines/Ubuntu-GCC5.yml create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml create mode 100644 .azurepipelines/Windows-VS2019.yml create mode 100644 .azurepipelines/templates/ReadMe.md create mode 100644 .azurepipelines/templates/basetools-build-steps.yml create mode 100644 .azurepipelines/templates/platform-build-run-steps.yml create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml create mode 100644 .azurepipelines/templates/pr-gate-steps.yml create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml diff --git a/.azurepipelines/ReadMe.md b/.azurepipelines/ReadMe.md new file mode 100644 index 000000000000..cf57d282c197 --- /dev/null +++ b/.azurepipelines/ReadMe.md @@ -0,0 +1,50 @@ +# Azure DevOps Pipelines + +These yml files are used to provide CI builds using the Azure DevOps Pipel= ine Service. +Most of the CI leverages edk2-pytools to support cross platform building a= nd execution. + +## Core CI + +Focused on building and testing all packages in Edk2 without an actual tar= get platform. + +See `.pytools/ReadMe.py` for more details + +## Platform CI + +Focused on building a single target platform and confirming functionality = on that platform. + +## Conventions + +* Files extension should be *.yml. *.yaml is also supported but in Edk2 w= e use those for our package configuration. +* Platform CI files should be in the `/.azurepipelines` folde= r. +* Core CI files are in the root folder. +* Shared templates are in the `templates` folder. +* Top level CI files should be named `-.yml` + +## Links + +* Basic Azure Landing Site - https://docs.microsoft.com/en-us/azure/devops= /pipelines/?view=3Dazure-devops +* Pipeline jobs - https://docs.microsoft.com/en-us/azure/devops/pipelines/= process/phases?view=3Dazure-devops&tabs=3Dyaml +* Pipeline yml scheme - https://docs.microsoft.com/en-us/azure/devops/pipe= lines/yaml-schema?view=3Dazure-devops&tabs=3Dschema%2Cparameter-schema +* Pipeline expression - https://docs.microsoft.com/en-us/azure/devops/pipe= lines/process/expressions?view=3Dazure-devops +* PyTools - https://github.com/tianocore/edk2-pytool-extensions and https:= //github.com/tianocore/edk2-pytool-library + +## Lessons Learned + +### Templates and parameters + +They are great but evil. If they are used as part of determining the step= s of a build they must resolve before the build starts. They can not use v= ariables set in a yml or determined as part of a matrix. If they are used = in a step then they can be bound late. + +### File matching patterns + +On Linux this can hang if there are too many files in the search list. + +### Templates and file splitting + +Suggestion is to do one big yaml file that does what you want for one of y= our targets. Then do the second one and find the deltas. From that you ca= n start to figure out the right split of files, steps, jobs. + +### Conditional steps + +If you want the step to show up in the log but not run, use a step conditi= onal. This is great when a platform doesn't currently support a feature but= you want the builders to know that the features exists and maybe someday i= t will. + +If you want the step to not show up use a template step conditional wrappe= r. Beware this will be evaluated early (at build start). This can hide th= ings not needed on a given OS for example. diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.= yml new file mode 100644 index 000000000000..3760c6efe105 --- /dev/null +++ b/.azurepipelines/Ubuntu-GCC5.yml @@ -0,0 +1,21 @@ +## @file +# Azure Pipeline build file for a build using ubuntu and GCC5 +# +# Copyright (c) Microsoft Corporation. +# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All right= s reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +## +trigger: +- master +- stable/* +pr: +- master +- stable/* + +jobs: +- template: templates/pr-gate-build-job.yml + parameters: + tool_chain_tag: 'GCC5' + vm_image: 'ubuntu-latest' + arch_list: "IA32,X64,ARM,AARCH64,RISCV64" + diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu= -PatchCheck.yml new file mode 100644 index 000000000000..4de453bf9db8 --- /dev/null +++ b/.azurepipelines/Ubuntu-PatchCheck.yml @@ -0,0 +1,36 @@ +## @file +# Azure Pipielines YML file that evalues the patch series in a PR using the +# python script BaseTools/Scripts/PatchCheck.py. +# +# NOTE: This example monitors pull requests against the edk2-ci branch. M= ost +# environments would replace 'edk2-ci' with 'master'. +# +# Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +# +# https://github.com/tianocore +# +## + +trigger: none + +pr: +- master +- stable/* + +pool: + vmImage: 'ubuntu-latest' + +steps: +- checkout: self + clean: true + +- task: UsePythonVersion@0 + inputs: + versionSpec: '3.7.x' + architecture: 'x64' + +- script: | + git fetch origin $(System.PullRequest.TargetBranch):$(System.PullReque= st.TargetBranch) + python BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBran= ch)..$(System.PullRequest.SourceCommitId) + displayName: 'Use PatchCheck.py to verify patch series in pull request' diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-V= S2019.yml new file mode 100644 index 000000000000..22f2d88c2c6a --- /dev/null +++ b/.azurepipelines/Windows-VS2019.yml @@ -0,0 +1,20 @@ +## @file +# Azure Pipeline build file for a build using Windows and VS2019 +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +trigger: +- master +- stable/* + +pr: +- master +- stable/* + +jobs: +- template: templates/pr-gate-build-job.yml + parameters: + tool_chain_tag: 'VS2019' + vm_image: 'windows-latest' + arch_list: "IA32,X64" diff --git a/.azurepipelines/templates/ReadMe.md b/.azurepipelines/template= s/ReadMe.md new file mode 100644 index 000000000000..fa433e3ef597 --- /dev/null +++ b/.azurepipelines/templates/ReadMe.md @@ -0,0 +1,59 @@ +# CI Templates + +This folder contains azure pipeline yml templates for "Core" and "Platform= " Continuous Integration and PR validation. + +## Common CI templates + +### basetools-build-steps.yml + +This template compiles the Edk2 basetools from source. The steps in this = template are +conditional and will only run if variable `pkg_count` is greater than 0. + +It also has two conditional steps only used when the toolchain contains GC= C. These two steps +use `apt` to update the system packages and add those necessary for Edk2 b= uilds. + +## Core CI templates + +### pr-gate-build-job.yml + +This templates contains the jobs and most importantly the matrix of which = packages and +targets to run for Core CI. + +### pr-gate-steps.yml + +This template is the main Core CI template. It controls all the steps run= and is responsible for most functionality of the Core CI process. This te= mplate sets +the `pkg_count` variable using the `stuart_pr_eval` tool when the +build type is "pull request" + +### spell-check-prereq-steps.yml + +This template installs the node based tools used by the spell checker plug= in. The steps +in this template are conditional and will only run if variable `pkg_count`= is greater than 0. + +## Platform CI templates + +### platform-build-run-steps.yml + +This template makes heavy use of pytools to build and run a platform in th= e Edk2 repo + +Also uses basetools-build-steps.yml to compile basetools + +#### Special Notes + +* For a build type of pull request it will conditionally build if the patc= hes change files that impact the platform. + * uses `stuart_pr_eval` to determine impact +* For manual builds or CI builds it will always build the platform +* It compiles basetools from source +* Will use `stuart_build --FlashOnly` to attempt to run the built image if= the `Run` parameter is set. +* See the parameters block for expected configuration options +* Parameter `extra_install_step` allows the caller to insert extra steps. = This is useful if additional dependencies, tools, or other things need to = be installed. Here is an example of installing qemu on Windows. + + ``` yaml + steps: + - template: ../../.azurepipelines/templates/build-run-steps.yml + parameters: + extra_install_step: + - powershell: choco install qemu; Write-Host "##vso[task.prependpa= th]c:\Program Files\qemu" + displayName: Install QEMU and Set QEMU on path # friendly name d= isplayed in the UI + condition: and(gt(variables.pkg_count, 0), succeeded()) + ``` diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepi= pelines/templates/basetools-build-steps.yml new file mode 100644 index 000000000000..d8c108c6e212 --- /dev/null +++ b/.azurepipelines/templates/basetools-build-steps.yml @@ -0,0 +1,37 @@ +## @file +# File templates/basetools-build-job.yml +# +# template file to build basetools +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +parameters: + tool_chain_tag: '' + +steps: +- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}: + - bash: sudo apt-get update + displayName: Update apt + condition: and(gt(variables.pkg_count, 0), succeeded()) + + - bash: sudo apt-get install gcc g++ make uuid-dev + displayName: Install required tools + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CmdLine@1 + displayName: Build Base Tools from source + inputs: + filename: python + arguments: BaseTools/Edk2ToolsBuild.py -t ${{ parameters.tool_chain_ta= g }} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CopyFiles@2 + displayName: "Copy base tools build log" + inputs: + targetFolder: '$(Build.ArtifactStagingDirectory)' + SourceFolder: 'BaseTools/BaseToolsBuild' + contents: | + BASETOOLS_BUILD*.* + flattenFolders: true + condition: and(gt(variables.pkg_count, 0), succeededOrFailed()) diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azur= epipelines/templates/platform-build-run-steps.yml new file mode 100644 index 000000000000..2a07e387754b --- /dev/null +++ b/.azurepipelines/templates/platform-build-run-steps.yml @@ -0,0 +1,144 @@ + +## @file +# File steps.yml +# +# template file containing the steps to build +# +# Copyright (c) 2021, Arm Limited. All rights reserved.
+# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: +- name: tool_chain_tag + type: string + default: '' +- name: build_pkg + type: string + default: '' +- name: build_target + type: string + default: '' +- name: build_arch + type: string + default: '' +- name: build_file + type: string + default: '' +- name: build_flags + type: string + default: '' +- name: run_flags + type: string + default: '' + +- name: extra_install_step + type: stepList + default: [] + +steps: +- checkout: self + clean: true + fetchDepth: 1 + +- task: UsePythonVersion@0 + inputs: + versionSpec: "3.8.x" + architecture: "x64" + +- script: pip install -r pip-requirements.txt --upgrade + displayName: 'Install/Upgrade pip modules' + +# Set default +- bash: echo "##vso[task.setvariable variable=3Dpkg_count]${{ 1 }}" + +# Fetch the target branch so that pr_eval can diff them. +# Seems like azure pipelines/github changed checkout process in nov 2020. +- script: git fetch origin $(System.PullRequest.targetBranch) + displayName: fetch target branch + condition: eq(variables['Build.Reason'], 'PullRequest') + +# trim the package list if this is a PR +- task: CmdLine@1 + displayName: Check if ${{ parameters.build_pkg }} need testing + inputs: + filename: stuart_pr_eval + arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_tar= get}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullReques= t.targetBranch) --output-count-format-string "##vso[task.setvariable variab= le=3Dpkg_count;isOutpout=3Dtrue]{pkgcount}" + condition: eq(variables['Build.Reason'], 'PullRequest') + + # Setup repo +- task: CmdLine@1 + displayName: Setup + inputs: + filename: stuart_setup + arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parame= ters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.bu= ild_arch}} ${{ parameters.build_flags}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# Stuart Update +- task: CmdLine@1 + displayName: Update + inputs: + filename: stuart_update + arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parame= ters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.bu= ild_arch}} ${{ parameters.build_flags}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# EDK2_PLATFORMS_MODIF_START: +# The base tools are imported in .pytool/CISettings.py via the 'edk2baseto= ols' python module. +# # build basetools +# # do this after setup and update so that code base dependencies +# # are all resolved. +# - template: basetools-build-steps.yml +# parameters: +# tool_chain_tag: ${{ parameters.tool_chain_tag }} +# EDK2_PLATFORMS_MODIF_END + +# Potential Extra steps +- ${{ parameters.extra_install_step }} + +# Build +- task: CmdLine@1 + displayName: Build + inputs: + filename: stuart_build + arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parame= ters.tool_chain_tag}} TARGET=3D${{ parameters.build_target}} -a ${{ paramet= ers.build_arch}} ${{ parameters.build_flags}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# Run +- task: CmdLine@1 + displayName: Run to shell + inputs: + filename: stuart_build + arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parame= ters.tool_chain_tag}} TARGET=3D${{ parameters.build_target}} -a ${{ paramet= ers.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} -= -FlashOnly + condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variable= s['Run'], true)) + timeoutInMinutes: 1 + +# Copy the build logs to the artifact staging directory +- task: CopyFiles@2 + displayName: "Copy build logs" + inputs: + targetFolder: "$(Build.ArtifactStagingDirectory)" + SourceFolder: "Build" + contents: | + BUILDLOG_*.txt + BUILDLOG_*.md + CI_*.txt + CI_*.md + CISETUP.txt + SETUPLOG.txt + UPDATE_LOG.txt + PREVALLOG.txt + TestSuites.xml + **/BUILD_TOOLS_REPORT.html + **/OVERRIDELOG.TXT + BASETOOLS_BUILD*.* + flattenFolders: true + condition: succeededOrFailed() + +# Publish build artifacts to Azure Artifacts/TFS or a file share +- task: PublishBuildArtifacts@1 + continueOnError: true + displayName: "Publish build logs" + inputs: + pathtoPublish: "$(Build.ArtifactStagingDirectory)" + artifactName: "Build Logs $(System.JobName)" + condition: succeededOrFailed() diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipeli= nes/templates/pr-gate-build-job.yml new file mode 100644 index 000000000000..e436618a91f2 --- /dev/null +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -0,0 +1,63 @@ +## @file +# File templates/pr-gate-build-job.yml +# +# template file used to build supported packages. +# +# Copyright (c) Microsoft Corporation. +# Copyright (c) 2020 - 2021, ARM Limited. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + tool_chain_tag: '' + vm_image: '' + arch_list: '' + +# Build step +jobs: + +- job: Build_${{ parameters.tool_chain_tag }} + +# EDK2_PLATFORMS_MODIF_START: +# Build edk2-platforms packages + #Use matrix to speed up the build process + strategy: + matrix: + TARGET_ARM_PLATFORMS: + Build.Pkgs: 'JunoPkg' + Build.Targets: 'DEBUG,RELEASE,NO-TARGET,NOOPT' +# EDK2_PLATFORMS_MODIF_END + + workspace: + clean: all + + pool: + vmImage: ${{ parameters.vm_image }} + + steps: + - template: pr-gate-steps.yml + parameters: + tool_chain_tag: ${{ parameters.tool_chain_tag }} + build_pkgs: $(Build.Pkgs) + build_targets: $(Build.Targets) + build_archs: ${{ parameters.arch_list }} + +- job: FINISHED + dependsOn: Build_${{ parameters.tool_chain_tag }} + condition: succeeded() + steps: + - checkout: none + - script: | + echo FINISHED + sleep 10 + displayName: FINISHED + +- job: FAILED + dependsOn: Build_${{ parameters.tool_chain_tag }} + condition: failed() + steps: + - checkout: none + - script: | + echo FAILED + sleep 10 + displayName: FAILED diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/= templates/pr-gate-steps.yml new file mode 100644 index 000000000000..0a8a0e1e09a3 --- /dev/null +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -0,0 +1,142 @@ +## @file +# File templates/pr-gate-steps.yml +# +# template file containing the steps to build +# +# Copyright (c) 2021, Arm Limited. All rights reserved.
+# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + tool_chain_tag: '' + build_pkgs: '' + build_targets: '' + build_archs: '' + +steps: +- checkout: self + clean: true + fetchDepth: 1 + +- task: UsePythonVersion@0 + inputs: + versionSpec: '3.8.x' + architecture: 'x64' + +- script: pip install -r pip-requirements.txt --upgrade + displayName: 'Install/Upgrade pip modules' + +# Set default +- bash: | + echo "##vso[task.setvariable variable=3Dpkgs_to_build]${{ parameters.b= uild_pkgs }}" + echo "##vso[task.setvariable variable=3Dpkg_count]${{ 1 }}" + +# Fetch the target branch so that pr_eval can diff them. +# Seems like azure pipelines/github changed checkout process in nov 2020. +- script: git fetch origin $(System.PullRequest.targetBranch) + displayName: fetch target branch + condition: eq(variables['Build.Reason'], 'PullRequest') + +# trim the package list if this is a PR +- task: CmdLine@1 + displayName: Check if ${{ parameters.build_pkgs }} need testing + inputs: + filename: stuart_pr_eval + arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --= pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-str= ing "##vso[task.setvariable variable=3Dpkgs_to_build;isOutpout=3Dtrue]{pkgc= sv}" --output-count-format-string "##vso[task.setvariable variable=3Dpkg_co= unt;isOutpout=3Dtrue]{pkgcount}" + condition: eq(variables['Build.Reason'], 'PullRequest') + +# install spell check prereqs +- template: spell-check-prereq-steps.yml + +# Build repo +- task: CmdLine@1 + displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_arc= hs}} + inputs: + filename: stuart_setup + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ paramet= ers.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=3D${{ pa= rameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- task: CmdLine@1 + displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_ar= chs}} + inputs: + filename: stuart_update + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ paramet= ers.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=3D${{ pa= rameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# build basetools +# do this after setup and update so that code base dependencies +# are all resolved. +# EDK2_PLATFORMS_MODIF_START: +# The base tools are imported in .pytool/CISettings.py via the 'edk2baseto= ols' python module. +# - template: basetools-build-steps.yml +# parameters: +# tool_chain_tag: ${{ parameters.tool_chain_tag }} +# EDK2_PLATFORMS_MODIF_END + +- task: CmdLine@1 + displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.= build_archs}} + inputs: + filename: stuart_ci_build + arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ paramet= ers.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=3D${{ pa= rameters.tool_chain_tag}} + condition: and(gt(variables.pkg_count, 0), succeeded()) + +# Publish Test Results to Azure Pipelines/TFS +- task: PublishTestResults@2 + displayName: 'Publish junit test results' + continueOnError: true + condition: and( succeededOrFailed(),gt(variables.pkg_count, 0)) + inputs: + testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit + testResultsFiles: 'Build/TestSuites.xml' + #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional + mergeTestResults: true # Optional + testRunTitle: $(System.JobName) # Optional + #buildPlatform: # Optional + #buildConfiguration: # Optional + publishRunAttachments: true # Optional + +# Publish Test Results to Azure Pipelines/TFS +- task: PublishTestResults@2 + displayName: 'Publish host based test results for $(System.JobName)' + continueOnError: true + condition: and( succeededOrFailed(), gt(variables.pkg_count, 0)) + inputs: + testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit + testResultsFiles: 'Build/**/*.result.xml' + #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional + mergeTestResults: false # Optional + testRunTitle: ${{ parameters.build_pkgs }} # Optional + #buildPlatform: # Optional + #buildConfiguration: # Optional + publishRunAttachments: true # Optional + +# Copy the build logs to the artifact staging directory +- task: CopyFiles@2 + displayName: "Copy build logs" + inputs: + targetFolder: '$(Build.ArtifactStagingDirectory)' + SourceFolder: 'Build' + contents: | + BUILDLOG_*.txt + BUILDLOG_*.md + CI_*.txt + CI_*.md + CISETUP.txt + SETUPLOG.txt + UPDATE_LOG.txt + PREVALLOG.txt + TestSuites.xml + **/BUILD_TOOLS_REPORT.html + **/OVERRIDELOG.TXT + flattenFolders: true + condition: succeededOrFailed() + +# Publish build artifacts to Azure Artifacts/TFS or a file share +- task: PublishBuildArtifacts@1 + continueOnError: true + displayName: "Publish build logs" + inputs: + pathtoPublish: '$(Build.ArtifactStagingDirectory)' + artifactName: 'Build Logs $(System.JobName)' + condition: succeededOrFailed() diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azur= epipelines/templates/spell-check-prereq-steps.yml new file mode 100644 index 000000000000..e1570d4f2aac --- /dev/null +++ b/.azurepipelines/templates/spell-check-prereq-steps.yml @@ -0,0 +1,22 @@ +## @file +# File templates/spell-check-prereq-steps.yml +# +# template file used to install spell checking prerequisits +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +parameters: + none: '' + +steps: +- task: NodeTool@0 + inputs: + versionSpec: '10.x' + #checkLatest: false # Optional + condition: and(gt(variables.pkg_count, 0), succeeded()) + +- script: npm install -g cspell + displayName: 'Install cspell npm' + condition: and(gt(variables.pkg_count, 0), succeeded()) --=20 2.17.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#78238): https://edk2.groups.io/g/devel/message/78238 Mute This Topic: https://groups.io/mt/84486155/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-