From nobody Sun Feb 8 21:28:47 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+50340+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+50340+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1573461681; cv=none; d=zoho.com; s=zohoarc; b=hSkAMxQiNHPZHvezrspaleiBZqvHDPsObzcnbkZ7scj0T7QrqKPr971z7nL2mgVTRV57/AhKr0FEXNXaikCq9f5nCV6Zck1JCbDoy0DbIJddBaSVWpTreR77WdtMNThyTVR1rJTZTIt0r7le/0lrLZwFTkljfEleFk6fky7nlU4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573461681; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=XALQdRCXmDf0MlVuw0lfhrLyyqNJyc8IvuYfJAJnOPw=; b=Zuw9EOgbOgkqcOhuC8WFGBdER8LMjdCd9h4prSOikWC/xUAXxBG3S4SjG1xEGERS6qkb8FLTLTbVJb8eBrfoM1xDJ+tiUH1c2+VOt5ab6EocSM9nNm8P2V2VxxAyp7jcM3GZdy6Dm8qa2jMl1rc5LK2u3qPBnnw5DSLowDa/Gzs= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+50340+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1573461681261515.3654495429521; Mon, 11 Nov 2019 00:41:21 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id EFz9YY1788612xKd1rO0714s; Mon, 11 Nov 2019 00:41:20 -0800 X-Received: from mga01.intel.com (mga01.intel.com []) by mx.groups.io with SMTP id smtpd.web10.1539.1573461678251400806 for ; Mon, 11 Nov 2019 00:41:19 -0800 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Nov 2019 00:41:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,292,1569308400"; d="scan'208";a="234403645" X-Received: from mdkinney-mobl2.amr.corp.intel.com ([10.255.231.122]) by fmsmga002.fm.intel.com with ESMTP; 11 Nov 2019 00:41:12 -0800 From: "Michael D Kinney" To: devel@edk2.groups.io Cc: Sean Brogan , Bob Feng , Liming Gao Subject: [edk2-devel] [Patch v5 05/22] BaseTools: Add YAML files with path env and tool extdeps Date: Mon, 11 Nov 2019 00:40:50 -0800 Message-Id: <20191111084107.5368-6-michael.d.kinney@intel.com> In-Reply-To: <20191111084107.5368-1-michael.d.kinney@intel.com> References: <20191111084107.5368-1-michael.d.kinney@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,michael.d.kinney@intel.com X-Gm-Message-State: aL4vt3DAk3zjJCoUey2fIpg4x1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1573461680; bh=sEnN0b9DFtjcb/4gSKaKh+rKASDhf+LPZJNa05MU/AU=; h=Cc:Date:From:Reply-To:Subject:To; b=LFDGSwsD3QgegT4Cq3CwYqfE7tvuCn403FVY4ZfveJc1RVC4UuZq9yQFfwnmN7HYln+ epNlj++SBkwWg/iFKeTSZzlztGRCz7ICCOhfNHHv/O47pPVXh2p6xb6RLGDz0xPE0O2uq ZcZNdy7EVRvjpxgA857106/b1LwNV9rxNsk= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" From: Sean Brogan Cc: Bob Feng Cc: Liming Gao Signed-off-by: Michael D Kinney Reviewed-by: Liming Gao --- BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 +++ BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 +++ BaseTools/Bin/iasl_ext_dep.yaml | 21 +++ BaseTools/Bin/nasm_ext_dep.yaml | 18 ++ .../BinWrappers/PosixLike/posix_path_env.yaml | 10 ++ .../WindowsLike/win_build_tools_path_env.yaml | 10 ++ BaseTools/Edk2ToolsBuild.py | 163 ++++++++++++++++++ .../basetool_tiano_python_path_env.yaml | 11 ++ BaseTools/basetools_calling_path_env.yaml | 11 ++ BaseTools/basetools_path_env.yaml | 11 ++ 10 files changed, 297 insertions(+) create mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml create mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml create mode 100644 BaseTools/Bin/iasl_ext_dep.yaml create mode 100644 BaseTools/Bin/nasm_ext_dep.yaml create mode 100644 BaseTools/BinWrappers/PosixLike/posix_path_env.yaml create mode 100644 BaseTools/BinWrappers/WindowsLike/win_build_tools_path_= env.yaml create mode 100644 BaseTools/Edk2ToolsBuild.py create mode 100644 BaseTools/Source/Python/basetool_tiano_python_path_env.= yaml create mode 100644 BaseTools/basetools_calling_path_env.yaml create mode 100644 BaseTools/basetools_path_env.yaml diff --git a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml b/BaseTools/Bin/g= cc_aarch64_linux_ext_dep.yaml new file mode 100644 index 0000000000..bf31cbd8d1 --- /dev/null +++ b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml @@ -0,0 +1,21 @@ +## @file +# Download GCC AARCH64 compiler from Linaro's release site +# Set shell variable GCC5_AARCH64_INSTALL to this folder +# +# This is only downloaded when a build activates scope gcc_aarch64_linux +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "gcc_aarch64_linux", + "type": "web", + "name": "gcc_aarch64_linux", + "source": "http://releases.linaro.org/components/toolchain/binaries/7.4-= 2019.02/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu= .tar.xz", + "version": "7.4.1", + "sha256": "27f1dc2c491ed61ae8f0d4b0c11de59cd2f7dd9c94761ee7153006fcac1bf= 9ab", + "compression_type": "tar", + "internal_path": "/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu/", + "flags": ["set_shell_var", ], + "var_name": "GCC5_AARCH64_INSTALL" +} diff --git a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml b/BaseTools/Bin/gcc_a= rm_linux_ext_dep.yaml new file mode 100644 index 0000000000..4105f5e4aa --- /dev/null +++ b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml @@ -0,0 +1,21 @@ +## @file +# Download GCC ARM compiler from Linaro's release site +# Set shell variable GCC5_ARM_INSTALL to this folder +# +# This is only downloaded when a build activates scope gcc_arm_linux +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "gcc_arm_linux", + "type": "web", + "name": "gcc_arm_linux", + "source": "https://releases.linaro.org/components/toolchain/binaries/7.4= -2019.02/arm-linux-gnueabihf/gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnue= abihf.tar.xz", + "version": "7.4.1", + "sha256": "3C951CF1941D0FA06D64CC0D5E88612B209D8123B273FA26C16D70BD7BC6B= 163", + "compression_type": "tar", + "internal_path": "/gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabihf/", + "flags": ["set_shell_var", ], + "var_name": "GCC5_ARM_INSTALL" +} diff --git a/BaseTools/Bin/iasl_ext_dep.yaml b/BaseTools/Bin/iasl_ext_dep.y= aml new file mode 100644 index 0000000000..7168eafe47 --- /dev/null +++ b/BaseTools/Bin/iasl_ext_dep.yaml @@ -0,0 +1,21 @@ +## @file +# Download iasl executable tool from a nuget.org package +# - package contains different binaries based on host +# Add the folder with the tool to the path +# +# This is only downloaded for scope cibuild thus +# should have no impact on the asl compiler used by any +# given platform to build. +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "id": "iasl-ci-1", + "scope": "cibuild", + "type": "nuget", + "name": "iasl", + "source": "https://api.nuget.org/v3/index.json", + "version": "20190215.0.0", + "flags": ["set_path", "host_specific"] +} diff --git a/BaseTools/Bin/nasm_ext_dep.yaml b/BaseTools/Bin/nasm_ext_dep.y= aml new file mode 100644 index 0000000000..cc3156662e --- /dev/null +++ b/BaseTools/Bin/nasm_ext_dep.yaml @@ -0,0 +1,18 @@ +## @file +# Download nasm x86 assembler executable tool from a nuget.org package +# - package contains different binaries based on host +# Put on the tool on the path +# +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "id": "nasm-1", + "scope": "edk2-build", + "type": "nuget", + "name": "mu_nasm", + "source": "https://api.nuget.org/v3/index.json", + "version": "2.14.02", + "flags": ["set_path", "host_specific"] +} diff --git a/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml b/BaseTool= s/BinWrappers/PosixLike/posix_path_env.yaml new file mode 100644 index 0000000000..ab5187efc8 --- /dev/null +++ b/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml @@ -0,0 +1,10 @@ +## @file +# Set this folder on the path for all linux builds +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "global-nix", + "flags": ["set_path"] +} diff --git a/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yam= l b/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml new file mode 100644 index 0000000000..1e6dd21b70 --- /dev/null +++ b/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml @@ -0,0 +1,10 @@ +## @file +# Add this folder to the path on Windows +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "global-win", + "flags": ["set_path"] +} diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py new file mode 100644 index 0000000000..057d2e9e06 --- /dev/null +++ b/BaseTools/Edk2ToolsBuild.py @@ -0,0 +1,163 @@ +# @file Edk2ToolsBuild.py +# Invocable class that builds the basetool c files. +# +# Supports VS2017, VS2019, and GCC5 +## +# Copyright (c) Microsoft Corporation +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +import os +import sys +import logging +import argparse +from edk2toolext import edk2_logging +from edk2toolext.environment import self_describing_environment +from edk2toolext.base_abstract_invocable import BaseAbstractInvocable +from edk2toollib.utility_functions import RunCmd +from edk2toollib.windows.locate_tools import QueryVcVariables + + +class Edk2ToolsBuild(BaseAbstractInvocable): + + def ParseCommandLineOptions(self): + ''' parse arguments ''' + ParserObj =3D argparse.ArgumentParser() + ParserObj.add_argument("-t", "--tool_chain_tag", dest=3D"tct", def= ault=3D"VS2017", + help=3D"Set the toolchain used to compile t= he build tools") + args =3D ParserObj.parse_args() + self.tool_chain_tag =3D args.tct + + def GetWorkspaceRoot(self): + ''' Return the workspace root for initializing the SDE ''' + + # this is the bastools dir...not the traditional EDK2 workspace ro= ot + return os.path.dirname(os.path.abspath(__file__)) + + def GetActiveScopes(self): + ''' return tuple containing scopes that should be active for this = process ''' + + # for now don't use scopes + return ('global',) + + def GetLoggingLevel(self, loggerType): + ''' Get the logging level for a given type (return Logging.Level) + base =3D=3D lowest logging level supported + con =3D=3D Screen logging + txt =3D=3D plain text file logging + md =3D=3D markdown file logging + ''' + if(loggerType =3D=3D "con"): + return logging.ERROR + else: + return logging.DEBUG + + def GetLoggingFolderRelativeToRoot(self): + ''' Return a path to folder for log files ''' + return "BaseToolsBuild" + + def GetVerifyCheckRequired(self): + ''' Will call self_describing_environment.VerifyEnvironment if thi= s returns True ''' + return True + + def GetLoggingFileName(self, loggerType): + ''' Get the logging file name for the type. + Return None if the logger shouldn't be created + + base =3D=3D lowest logging level supported + con =3D=3D Screen logging + txt =3D=3D plain text file logging + md =3D=3D markdown file logging + ''' + return "BASETOOLS_BUILD" + + def WritePathEnvFile(self, OutputDir): + ''' Write a PyTool path env file for future PyTool based edk2 buil= ds''' + content =3D '''## +# Set shell variable EDK_TOOLS_BIN to this folder +# +# Autogenerated by Edk2ToolsBuild.py +# +# Copyright (c), Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "id": "You-Built-BaseTools", + "scope": "edk2-build", + "flags": ["set_shell_var", "set_path"], + "var_name": "EDK_TOOLS_BIN" +} +''' + with open(os.path.join(OutputDir, "basetoolsbin_path_env.yaml"), "= w") as f: + f.write(content) + + def Go(self): + logging.info("Running Python version: " + str(sys.version_info)) + + (build_env, shell_env) =3D self_describing_environment.BootstrapEn= vironment( + self.GetWorkspaceRoot(), self.GetActiveScopes()) + + # # Bind our current execution environment into the shell vars. + ph =3D os.path.dirname(sys.executable) + if " " in ph: + ph =3D '"' + ph + '"' + shell_env.set_shell_var("PYTHON_HOME", ph) + # PYTHON_COMMAND is required to be set for using edk2 python build= s. + pc =3D sys.executable + if " " in pc: + pc =3D '"' + pc + '"' + shell_env.set_shell_var("PYTHON_COMMAND", pc) + + if self.tool_chain_tag.lower().startswith("vs"): + + # # Update environment with required VC vars. + interesting_keys =3D ["ExtensionSdkDir", "INCLUDE", "LIB"] + interesting_keys.extend( + ["LIBPATH", "Path", "UniversalCRTSdkDir", "UCRTVersion", "= WindowsLibPath", "WindowsSdkBinPath"]) + interesting_keys.extend( + ["WindowsSdkDir", "WindowsSdkVerBinPath", "WindowsSDKVersi= on", "VCToolsInstallDir"]) + vc_vars =3D QueryVcVariables( + interesting_keys, 'x86', vs_version=3Dself.tool_chain_tag.= lower()) + for key in vc_vars.keys(): + logging.debug(f"Var - {key} =3D {vc_vars[key]}") + if key.lower() =3D=3D 'path': + shell_env.insert_path(vc_vars[key]) + else: + shell_env.set_shell_var(key, vc_vars[key]) + + self.OutputDir =3D os.path.join( + shell_env.get_shell_var("EDK_TOOLS_PATH"), "Bin", "Win32") + + # compiled tools need to be added to path because antlr is ref= erenced + shell_env.insert_path(self.OutputDir) + + # Actually build the tools. + ret =3D RunCmd('nmake.exe', None, + workingdir=3Dshell_env.get_shell_var("EDK_TOOLS_P= ATH")) + if ret !=3D 0: + raise Exception("Failed to build.") + + self.WritePathEnvFile(self.OutputDir) + return ret + + elif self.tool_chain_tag.lower().startswith("gcc"): + ret =3D RunCmd("make", "-C .", workingdir=3Dshell_env.get_shel= l_var("EDK_TOOLS_PATH")) + if ret !=3D 0: + raise Exception("Failed to build.") + + self.OutputDir =3D os.path.join( + shell_env.get_shell_var("EDK_TOOLS_PATH"), "Source", "C", = "bin") + + self.WritePathEnvFile(self.OutputDir) + return ret + + logging.critical("Tool Chain not supported") + return -1 + + +def main(): + Edk2ToolsBuild().Invoke() + + +if __name__ =3D=3D "__main__": + main() diff --git a/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml b/= BaseTools/Source/Python/basetool_tiano_python_path_env.yaml new file mode 100644 index 0000000000..12d517d32c --- /dev/null +++ b/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml @@ -0,0 +1,11 @@ +## @file +# Add this folder to the pypath so modules can be easily +# loaded +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "global", + "flags": ["set_pypath"] +} diff --git a/BaseTools/basetools_calling_path_env.yaml b/BaseTools/basetool= s_calling_path_env.yaml new file mode 100644 index 0000000000..2e444cd480 --- /dev/null +++ b/BaseTools/basetools_calling_path_env.yaml @@ -0,0 +1,11 @@ +## @file +# Set shell variable BASE_TOOL_PATH to this folder +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "global", + "flags": ["set_shell_var"], + "var_name": "BASE_TOOLS_PATH" +} diff --git a/BaseTools/basetools_path_env.yaml b/BaseTools/basetools_path_e= nv.yaml new file mode 100644 index 0000000000..23ce5533ba --- /dev/null +++ b/BaseTools/basetools_path_env.yaml @@ -0,0 +1,11 @@ +## @file +# Set shell variable EDK_TOOL_PATH to this folder +# +# Copyright (c) 2019, Microsoft Corporation +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "global", + "flags": ["set_shell_var", "set_build_var"], + "var_name": "EDK_TOOLS_PATH" +} --=20 2.21.0.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#50340): https://edk2.groups.io/g/devel/message/50340 Mute This Topic: https://groups.io/mt/52458137/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-