From nobody Tue Feb 3 16:34:04 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+53459+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+53459+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1579831838; cv=none; d=zohomail.com; s=zohoarc; b=G1ZBLaY+E1b2d4Y2Er00wQJuWd5qDuXP+2bqkUUt8gGJ/Qm+3slyZEHxuejrZjbVC+J/ie62H/pVhIk2s5drqroO4f6OjxjWbRzOW9brz5i7xySBGhQoo/2t4ZFl23r5JgCfRg63CFI1d6OSDZlJkbYe7jhGx0XCQyyZZLScr7o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1579831838; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=o/S+SPh+k5cjn0lDp16hF3rQkTazZJLIclNLk1kysJ8=; b=DioFkoWM4NbR1luftOj8gMV/1kLHL+GxRpCc3yQct8lZ4777F5eF6ZsRFzw6ifKkddjFTjzFELzTFQxPEkzNDm5/2wXdxkty5olbqUdUyuCG0otBGq6tMUxvnPfmdhQePU/x1cDiTUSunE7IYBbG1Zes5OXPZbrzakmTt4hOGZg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+53459+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1579831838231306.0609559873517; Thu, 23 Jan 2020 18:10:38 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id PVAIYY1788612xq2OCCQ8zhD; Thu, 23 Jan 2020 18:10:37 -0800 X-Received: from mga09.intel.com (mga09.intel.com []) by mx.groups.io with SMTP id smtpd.web11.7301.1579831835785711151 for ; Thu, 23 Jan 2020 18:10:36 -0800 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Jan 2020 18:10:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,356,1574150400"; d="scan'208";a="375395287" X-Received: from unknown (HELO mdkinney-MOBL2.amr.corp.intel.com) ([10.241.98.74]) by orsmga004.jf.intel.com with ESMTP; 23 Jan 2020 18:10:35 -0800 From: "Michael D Kinney" To: devel@edk2.groups.io Cc: Sean Brogan , Bret Barkelew , Liming Gao Subject: [edk2-devel] [Patch 01/11] .pytool: Add CI support for host based unit tests with results Date: Thu, 23 Jan 2020 18:10:22 -0800 Message-Id: <20200124021032.13808-2-michael.d.kinney@intel.com> In-Reply-To: <20200124021032.13808-1-michael.d.kinney@intel.com> References: <20200124021032.13808-1-michael.d.kinney@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,michael.d.kinney@intel.com X-Gm-Message-State: iVOgOtaKaMLWHjYEz4oOJdljx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1579831837; bh=BIOqNVUAEwcyQ4woKZaim7O31rVhuprzkumxX1a1eHE=; h=Cc:Date:From:Reply-To:Subject:To; b=HJXTxqBA2o9WqbPPr+e8Jo78pDeEM7AtYB54uKhR/WnYP2aj/Gd4bg5aOZlnD8tB1fn T1BGm1FPoBCo+EYpOW1QVO0l37drV3shSaUphG253Lc0ilFbkoiRhivLSbEnHSJQiFoBC AFbjmlEXoOfZevl6UW2woFmMp8Nhq0d5EoU= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" * Add plugin to build and run host based unit tests * Add plugin that performs a DSC complete check DSC files used to build host based tests * Update DscCompleteCheck plugin to ignore module INFs with a MODULE_TYPE of HOST_APPLICATION and library INFs that only support a module type of HOST_APPLICATION. * Fix issues in XML reports from checkers. Cc: Sean Brogan Cc: Bret Barkelew Cc: Liming Gao Signed-off-by: Michael D Kinney Reviewed-by: Bret Barkelew --- .pytool/CISettings.py | 22 ++++-- .../CharEncodingCheck/CharEncodingCheck.py | 2 +- .../Plugin/CompilerPlugin/CompilerPlugin.py | 4 +- .../Plugin/DependencyCheck/DependencyCheck.py | 2 +- .../DscCompleteCheck/DscCompleteCheck.py | 35 ++++++--- .pytool/Plugin/DscCompleteCheck/Readme.md | 7 +- .pytool/Plugin/GuidCheck/GuidCheck.py | 2 +- .../HostUnitTestCompilerPlugin.py} | 71 +++++++++++++++---- .../HostUnitTestCompiler_plug_in.yaml | 12 ++++ .../HostUnitTestCompilerPlugin/Readme.md | 24 +++++++ .../HostUnitTestDscCompleteCheck.py} | 58 ++++++++++----- .../HostUnitTestDscCompleteCheck_plug_in.yaml | 12 ++++ .../HostUnitTestDscCompleteCheck/Readme.md | 32 +++++++++ .../LibraryClassCheck/LibraryClassCheck.py | 2 +- 14 files changed, 231 insertions(+), 54 deletions(-) copy .pytool/Plugin/{CompilerPlugin/CompilerPlugin.py =3D> HostUnitTestCom= pilerPlugin/HostUnitTestCompilerPlugin.py} (54%) create mode 100644 .pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestC= ompiler_plug_in.yaml create mode 100644 .pytool/Plugin/HostUnitTestCompilerPlugin/Readme.md copy .pytool/Plugin/{DscCompleteCheck/DscCompleteCheck.py =3D> HostUnitTes= tDscCompleteCheck/HostUnitTestDscCompleteCheck.py} (62%) create mode 100644 .pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTes= tDscCompleteCheck_plug_in.yaml create mode 100644 .pytool/Plugin/HostUnitTestDscCompleteCheck/Readme.md diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py index ce177937e1..79593d9dc5 100644 --- a/.pytool/CISettings.py +++ b/.pytool/CISettings.py @@ -48,7 +48,8 @@ class Settings(CiBuildSettingsManager, UpdateSettingsMana= ger, SetupSettingsManag "FmpDevicePkg", "ShellPkg", "FatPkg", - "CryptoPkg" + "CryptoPkg", + "UnitTestFrameworkPkg" ) =20 def GetArchitecturesSupported(self): @@ -117,10 +118,13 @@ class Settings(CiBuildSettingsManager, UpdateSettings= Manager, SetupSettingsManag =20 def GetActiveScopes(self): ''' return tuple containing scopes that should be active for this = process ''' - scopes =3D ("cibuild","edk2-build") + scopes =3D ("cibuild", "edk2-build", "host-based-test") =20 self.ActualToolChainTag =3D shell_environment.GetBuildVars().GetVa= lue("TOOL_CHAIN_TAG", "") =20 + if GetHostInfo().os.upper() =3D=3D "WINDOWS": + scopes +=3D ('host-test-win',) + if GetHostInfo().os.upper() =3D=3D "LINUX" and self.ActualToolChai= nTag.upper().startswith("GCC"): if "AARCH64" in self.ActualArchitectures: scopes +=3D ("gcc_aarch64_linux",) @@ -133,18 +137,21 @@ class Settings(CiBuildSettingsManager, UpdateSettings= Manager, SetupSettingsManag ''' return iterable containing RequiredSubmodule objects. If no RequiredSubmodules return an empty iterable ''' - rs=3D[] + rs =3D [] rs.append(RequiredSubmodule( "ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3", False)) rs.append(RequiredSubmodule( "CryptoPkg/Library/OpensslLib/openssl", False)) + rs.append(RequiredSubmodule( + "UnitTestFrameworkPkg/Library/CmockaLib/cmocka", False)) return rs =20 def GetName(self): return "Edk2" =20 def GetDependencies(self): - return [] + return [ + ] =20 def GetPackagesPath(self): return () @@ -155,10 +162,11 @@ class Settings(CiBuildSettingsManager, UpdateSettings= Manager, SetupSettingsManag =20 def FilterPackagesToTest(self, changedFilesList: list, potentialPackag= esList: list) -> list: ''' Filter potential packages to test based on changed files. ''' - build_these_packages=3D[] - possible_packages=3DpotentialPackagesList.copy() + build_these_packages =3D [] + possible_packages =3D potentialPackagesList.copy() for f in changedFilesList: - nodes=3Df.split("/") # split each part of path for comparison= later + # split each part of path for comparison later + nodes =3D f.split("/") =20 # python file change in .pytool folder causes building all if f.endswith(".py") and ".pytool" in nodes: diff --git a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py b/.pytoo= l/Plugin/CharEncodingCheck/CharEncodingCheck.py index 02f25ab19f..1496e1f249 100644 --- a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py +++ b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py @@ -100,7 +100,7 @@ class CharEncodingCheck(ICiBuildPlugin): overall_status +=3D 1 =20 tc.LogStdOut("Tested Encoding on {0} files".format(files_tested)) - if overall_status is not 0: + if overall_status !=3D 0: tc.SetFailed("CharEncoding {0} Failed. Errors {1}".format(pac= kagename, overall_status), "CHAR_ENCODING_CHECK_FAILED") else: tc.SetSuccess() diff --git a/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py b/.pytool/Plug= in/CompilerPlugin/CompilerPlugin.py index 3b6f7c7698..e8657940d7 100644 --- a/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py +++ b/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py @@ -1,4 +1,4 @@ -# @file HostUnitTestCompiler_plugin.py +# @file CompilerPlugin.py ## # Copyright (c) Microsoft Corporation. # SPDX-License-Identifier: BSD-2-Clause-Patent @@ -42,7 +42,7 @@ class CompilerPlugin(ICiBuildPlugin): return ["DEBUG", "RELEASE"] =20 ## - # External function of plugin. This function is used to perform the t= ask of the MuBuild Plugin + # External function of plugin. This function is used to perform the t= ask of the ICiBuildPlugin Plugin # # - package is the edk2 path to package. This means workspace/packa= gepath relative. # - edk2path object configured with workspace and packages path diff --git a/.pytool/Plugin/DependencyCheck/DependencyCheck.py b/.pytool/Pl= ugin/DependencyCheck/DependencyCheck.py index 2c3d8baf69..db154d769a 100644 --- a/.pytool/Plugin/DependencyCheck/DependencyCheck.py +++ b/.pytool/Plugin/DependencyCheck/DependencyCheck.py @@ -113,7 +113,7 @@ class DependencyCheck(ICiBuildPlugin): overall_status +=3D 1 =20 # If XML object exists, add results - if overall_status is not 0: + if overall_status !=3D 0: tc.SetFailed("Failed with {0} errors".format(overall_status), = "DEPENDENCYCHECK_FAILED") else: tc.SetSuccess() diff --git a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py b/.pytool/= Plugin/DscCompleteCheck/DscCompleteCheck.py index 9af4f72c8d..c613cd5233 100644 --- a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py +++ b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py @@ -54,12 +54,15 @@ class DscCompleteCheck(ICiBuildPlugin): # Parse the config for required DscPath element if "DscPath" not in pkgconfig: tc.SetSkipped() - tc.LogStdError("DscPath not found in config file. Nothing to = check.") + tc.LogStdError( + "DscPath not found in config file. Nothing to check.") return -1 =20 - abs_pkg_path =3D Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2Rel= ativePath(packagename) + abs_pkg_path =3D Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2Rel= ativePath( + packagename) abs_dsc_path =3D os.path.join(abs_pkg_path, pkgconfig["DscPath"].s= trip()) - wsr_dsc_path =3D Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(a= bs_dsc_path) + wsr_dsc_path =3D Edk2pathObj.GetEdk2RelativePathFromAbsolutePath( + abs_dsc_path) =20 if abs_dsc_path is None or wsr_dsc_path =3D=3D "" or not os.path.i= sfile(abs_dsc_path): tc.SetSkipped() @@ -68,7 +71,8 @@ class DscCompleteCheck(ICiBuildPlugin): =20 # Get INF Files INFFiles =3D self.WalkDirectoryForExtension([".inf"], abs_pkg_path) - INFFiles =3D [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) f= or x in INFFiles] # make edk2relative path so can compare with DSC + INFFiles =3D [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath( + x) for x in INFFiles] # make edk2relative path so can compare= with DSC =20 # remove ignores =20 @@ -79,8 +83,10 @@ class DscCompleteCheck(ICiBuildPlugin): tc.LogStdOut("Ignoring INF {0}".format(a)) INFFiles.remove(a) except: - tc.LogStdError("DscCompleteCheck.IgnoreInf -> {0} not = found in filesystem. Invalid ignore file".format(a)) - logging.info("DscCompleteCheck.IgnoreInf -> {0} not fo= und in filesystem. Invalid ignore file".format(a)) + tc.LogStdError( + "DscCompleteCheck.IgnoreInf -> {0} not found in fi= lesystem. Invalid ignore file".format(a)) + logging.info( + "DscCompleteCheck.IgnoreInf -> {0} not found in fi= lesystem. Invalid ignore file".format(a)) =20 # DSC Parser dp =3D DscParser() @@ -99,11 +105,19 @@ class DscCompleteCheck(ICiBuildPlugin): infp.SetPackagePaths(Edk2pathObj.PackagePathList) infp.ParseFile(INF) if("MODULE_TYPE" not in infp.Dict): - tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYP= E {0}".format(INF)) + tc.LogStdOut( + "Ignoring INF. Missing key for MODULE_TYPE {0}".fo= rmat(INF)) continue =20 if(infp.Dict["MODULE_TYPE"] =3D=3D "HOST_APPLICATION"): - tc.LogStdOut("Ignoring INF. Module type is HOST_APPLI= CATION {0}".format(INF)) + tc.LogStdOut( + "Ignoring INF. Module type is HOST_APPLICATION {0= }".format(INF)) + continue + + if len(infp.SupportedPhases) =3D=3D 1 and \ + "HOST_APPLICATION" in infp.SupportedPhases: + tc.LogStdOut( + "Ignoring Library INF due to only supporting type = HOST_APPLICATION {0}".format(INF)) continue =20 logging.critical(INF + " not in " + wsr_dsc_path) @@ -111,8 +125,9 @@ class DscCompleteCheck(ICiBuildPlugin): overall_status =3D overall_status + 1 =20 # If XML object exists, add result - if overall_status is not 0: - tc.SetFailed("DscCompleteCheck {0} Failed. Errors {1}".format= (wsr_dsc_path, overall_status), "CHECK_FAILED") + if overall_status !=3D 0: + tc.SetFailed("DscCompleteCheck {0} Failed. Errors {1}".format( + wsr_dsc_path, overall_status), "CHECK_FAILED") else: tc.SetSuccess() return overall_status diff --git a/.pytool/Plugin/DscCompleteCheck/Readme.md b/.pytool/Plugin/Dsc= CompleteCheck/Readme.md index eefbb9894d..8aaa4f76ee 100644 --- a/.pytool/Plugin/DscCompleteCheck/Readme.md +++ b/.pytool/Plugin/DscCompleteCheck/Readme.md @@ -7,6 +7,11 @@ that it would not be built if the package were built). Thi= s is critical because much of the CI infrastructure assumes that all modules will be listed in t= he DSC and compiled. =20 +This test will ignore INFs in the following cases: + +1. When MODULE_TYPE =3D HOST_APPLICATION +2. When a Library instance **only** supports the HOST_APPLICATION environm= ent + ## Configuration =20 The plugin has a few configuration options to support the UEFI codebase. @@ -14,7 +19,7 @@ The plugin has a few configuration options to support the= UEFI codebase. ``` yaml "DscCompleteCheck": { "DscPath": "", # Path to dsc from root of package - "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc + "IgnoreInf": [] # Ignore INF if found in filesystem but not dsc } ``` =20 diff --git a/.pytool/Plugin/GuidCheck/GuidCheck.py b/.pytool/Plugin/GuidChe= ck/GuidCheck.py index f0b10beb1e..61fdc77911 100644 --- a/.pytool/Plugin/GuidCheck/GuidCheck.py +++ b/.pytool/Plugin/GuidCheck/GuidCheck.py @@ -221,7 +221,7 @@ class GuidCheck(ICiBuildPlugin): =20 # add result to test case overall_status =3D len(Errors) - if overall_status is not 0: + if overall_status !=3D 0: tc.SetFailed("GuidCheck {0} Failed. Errors {1}".format( packagename, overall_status), "CHECK_FAILED") else: diff --git a/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py b/.pytool/Plug= in/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.py similarity index 54% copy from .pytool/Plugin/CompilerPlugin/CompilerPlugin.py copy to .pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugi= n.py index 3b6f7c7698..f21b40caf2 100644 --- a/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py +++ b/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.= py @@ -1,4 +1,4 @@ -# @file HostUnitTestCompiler_plugin.py +# @file HostUnitTestCompilerPlugin.py ## # Copyright (c) Microsoft Corporation. # SPDX-License-Identifier: BSD-2-Clause-Patent @@ -12,21 +12,25 @@ from edk2toolext.environment.plugintypes.ci_build_plugi= n import ICiBuildPlugin from edk2toolext.environment.uefi_build import UefiBuilder from edk2toolext import edk2_logging from edk2toolext.environment.var_dict import VarDict +from edk2toollib.utility_functions import GetHostInfo =20 =20 -class CompilerPlugin(ICiBuildPlugin): +class HostUnitTestCompilerPlugin(ICiBuildPlugin): """ - A CiBuildPlugin that compiles the package dsc - from the package being tested. + A CiBuildPlugin that compiles the dsc for host based unit test apps. + An IUefiBuildPlugin may be attached to this plugin that will run the + unit tests and collect the results after successful compilation. =20 Configuration options: - "CompilerPlugin": { + "HostUnitTestCompilerPlugin": { "DscPath": "" } """ =20 def GetTestName(self, packagename: str, environment: VarDict) -> tuple: """ Provide the testcase name and classname for use in reporting + testclassname: a descriptive string for the testcase can inclu= de whitespace + classname: should be patterned .. =20 Args: packagename: string containing name of package to build @@ -35,14 +39,41 @@ class CompilerPlugin(ICiBuildPlugin): a tuple containing the testcase name and the classname (testcasename, classname) """ - target =3D environment.GetValue("TARGET") - return ("Compile " + packagename + " " + target, packagename + ".C= ompiler." + target) + num,types =3D self.__GetHostUnitTestArch(environment) + types =3D types.replace(" ", "_") + + return ("Compile and Run Host-Based UnitTests for " + packagename = + " on arch " + types, + packagename + ".HostUnitTestCompiler." + types) =20 def RunsOnTargetList(self): - return ["DEBUG", "RELEASE"] + return ["NOOPT"] + + # + # Find the intersection of application types that can run on this host + # and the TARGET_ARCH being build in this request. + # + # return tuple with (number of UEFI arch types, space separated string) + def __GetHostUnitTestArch(self, environment): + requested =3D environment.GetValue("TARGET_ARCH").split(' ') + host =3D [] + if GetHostInfo().arch =3D=3D 'x86': + #assume 64bit can handle 64 and 32 + #assume 32bit can only handle 32 + ## change once IA32 issues resolved host.append("IA32") + if GetHostInfo().bit =3D=3D '64': + host.append("X64") + elif GetHostInfo().arch =3D=3D 'ARM': + if GetHostInfo().bit =3D=3D '64': + host.append("AARCH64") + elif GetHostInfo().bit =3D=3D '32': + host.append("ARM") + + willrun =3D set(requested) & set(host) + return (len(willrun), " ".join(willrun)) + =20 ## - # External function of plugin. This function is used to perform the t= ask of the MuBuild Plugin + # External function of plugin. This function is used to perform the t= ask of the ICiBuildPlugin Plugin # # - package is the edk2 path to package. This means workspace/packa= gepath relative. # - edk2path object configured with workspace and packages path @@ -54,11 +85,12 @@ class CompilerPlugin(ICiBuildPlugin): # - output_stream the StringIO output stream from this plugin via lo= gging def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environm= ent, PLM, PLMHelper, tc, output_stream=3DNone): self._env =3D environment + environment.SetValue("CI_BUILD_TYPE", "host_unit_test", "Set in Ho= stUnitTestCompilerPlugin") =20 # Parse the config for required DscPath element if "DscPath" not in pkgconfig: tc.SetSkipped() - tc.LogStdError("DscPath not found in config file. Nothing to = compile.") + tc.LogStdError("DscPath not found in config file. Nothing to = compile for HostBasedUnitTests.") return -1 =20 AP =3D Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(= packagename) @@ -67,11 +99,26 @@ class CompilerPlugin(ICiBuildPlugin): AP_Path =3D Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(APDSC) if AP is None or AP_Path is None or not os.path.isfile(APDSC): tc.SetSkipped() - tc.LogStdError("Package Dsc not found.") + tc.LogStdError("Package HostBasedUnitTest Dsc not found.") return -1 =20 logging.info("Building {0}".format(AP_Path)) self._env.SetValue("ACTIVE_PLATFORM", AP_Path, "Set in Compiler Pl= ugin") + num, RUNNABLE_ARCHITECTURES =3D self.__GetHostUnitTestArch(environ= ment) + if(num =3D=3D 0): + tc.SetSkipped() + tc.LogStdError("No host architecture compatibility") + return -1 + + if not environment.SetValue("TARGET_ARCH", + RUNNABLE_ARCHITECTURES, + "Update Target Arch based on Host Supp= ort"): + #use AllowOverride function since this is a controlled attempt= to change + environment.AllowOverride("TARGET_ARCH") + if not environment.SetValue("TARGET_ARCH", + RUNNABLE_ARCHITECTURES, + "Update Target Arch based on Host = Support"): + raise RuntimeError("Can't Change TARGET_ARCH as required") =20 # Parse DSC to check for SUPPORTED_ARCHITECTURES dp =3D DscParser() @@ -85,7 +132,7 @@ class CompilerPlugin(ICiBuildPlugin): # Skip if there is no intersection between SUPPORTED_ARCHITECT= URES and TARGET_ARCHITECTURES if len(set(SUPPORTED_ARCHITECTURES) & set(TARGET_ARCHITECTURES= )) =3D=3D 0: tc.SetSkipped() - tc.LogStdError("No supported architecutres to build") + tc.LogStdError("No supported architecutres to build for ho= st unit tests") return -1 =20 uefiBuilder =3D UefiBuilder() diff --git a/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompiler= _plug_in.yaml b/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompi= ler_plug_in.yaml new file mode 100644 index 0000000000..3cecf0af9a --- /dev/null +++ b/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompiler_plug_i= n.yaml @@ -0,0 +1,12 @@ +## +# CiBuildPlugin used to build anything that identifies +# as a unit test. +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "host-based-test", + "name": "Host Unit Test Compiler Plugin", + "module": "HostUnitTestCompilerPlugin" +} diff --git a/.pytool/Plugin/HostUnitTestCompilerPlugin/Readme.md b/.pytool/= Plugin/HostUnitTestCompilerPlugin/Readme.md new file mode 100644 index 0000000000..3eeebb4b16 --- /dev/null +++ b/.pytool/Plugin/HostUnitTestCompilerPlugin/Readme.md @@ -0,0 +1,24 @@ +# Host UnitTest Compiler Plugin + +A CiBuildPlugin that compiles the dsc for host based unit test apps. +An IUefiBuildPlugin may be attached to this plugin that will run the unit = tests and collect the results after successful compilation. + +## Configuration + +The package relative path of the DSC file to build. + +``` yaml +"HostUnitTestCompilerPlugin": { + "DscPath": "" +} +``` + +### DscPath + +Package relative path to the DSC file to build. + +## Copyright + +Copyright (c) Microsoft Corporation. +SPDX-License-Identifier: BSD-2-Clause-Patent + diff --git a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py b/.pytool/= Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCheck.py similarity index 62% copy from .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py copy to .pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscComplete= Check.py index 9af4f72c8d..66bdecacfb 100644 --- a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py +++ b/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCh= eck.py @@ -1,4 +1,7 @@ -# @file DscCompleteCheck.py +# @file HostUnitTestDscCompleteCheck.py +# +# This is a copy of DscCompleteCheck with different filtering logic. +# It should be discussed if this should be one plugin # # Copyright (c) Microsoft Corporation. # SPDX-License-Identifier: BSD-2-Clause-Patent @@ -11,15 +14,15 @@ from edk2toollib.uefi.edk2.parsers.inf_parser import In= fParser from edk2toolext.environment.var_dict import VarDict =20 =20 -class DscCompleteCheck(ICiBuildPlugin): +class HostUnitTestDscCompleteCheck(ICiBuildPlugin): """ - A CiBuildPlugin that scans the package dsc file and confirms all modul= es (inf files) are + A CiBuildPlugin that scans the package Host Unit Test dsc file and con= firms all Host application modules (inf files) are listed in the components sections. =20 Configuration options: - "DscCompleteCheck": { - "DscPath": "" - "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc + "HostUnitTestDscCompleteCheck": { + "DscPath": "", # Path to Host based unit test DSC file + "IgnoreInf": [] # Ignore INF if found in filesystem but not dsc } """ =20 @@ -35,7 +38,7 @@ class DscCompleteCheck(ICiBuildPlugin): testclassname: a descriptive string for the testcase can i= nclude whitespace classname: should be patterned .. """ - return ("Check the " + packagename + " DSC for a being complete", = packagename + ".DscCompleteCheck") + return ("Check the " + packagename + " Host Unit Test DSC for a be= ing complete", packagename + ".HostUnitTestDscCompleteCheck") =20 ## # External function of plugin. This function is used to perform the t= ask of the MuBuild Plugin @@ -54,21 +57,25 @@ class DscCompleteCheck(ICiBuildPlugin): # Parse the config for required DscPath element if "DscPath" not in pkgconfig: tc.SetSkipped() - tc.LogStdError("DscPath not found in config file. Nothing to = check.") + tc.LogStdError( + "DscPath not found in config file. Nothing to check.") return -1 =20 - abs_pkg_path =3D Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2Rel= ativePath(packagename) + abs_pkg_path =3D Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2Rel= ativePath( + packagename) abs_dsc_path =3D os.path.join(abs_pkg_path, pkgconfig["DscPath"].s= trip()) - wsr_dsc_path =3D Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(a= bs_dsc_path) + wsr_dsc_path =3D Edk2pathObj.GetEdk2RelativePathFromAbsolutePath( + abs_dsc_path) =20 if abs_dsc_path is None or wsr_dsc_path =3D=3D "" or not os.path.i= sfile(abs_dsc_path): tc.SetSkipped() - tc.LogStdError("Package Dsc not found") + tc.LogStdError("Package Host Unit Test Dsc not found") return 0 =20 # Get INF Files INFFiles =3D self.WalkDirectoryForExtension([".inf"], abs_pkg_path) - INFFiles =3D [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) f= or x in INFFiles] # make edk2relative path so can compare with DSC + INFFiles =3D [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath( + x) for x in INFFiles] # make edk2relative path so can compare= with DSC =20 # remove ignores =20 @@ -79,8 +86,10 @@ class DscCompleteCheck(ICiBuildPlugin): tc.LogStdOut("Ignoring INF {0}".format(a)) INFFiles.remove(a) except: - tc.LogStdError("DscCompleteCheck.IgnoreInf -> {0} not = found in filesystem. Invalid ignore file".format(a)) - logging.info("DscCompleteCheck.IgnoreInf -> {0} not fo= und in filesystem. Invalid ignore file".format(a)) + tc.LogStdError( + "HostUnitTestDscCompleteCheck.IgnoreInf -> {0} not= found in filesystem. Invalid ignore file".format(a)) + logging.info( + "HostUnitTestDscCompleteCheck.IgnoreInf -> {0} not= found in filesystem. Invalid ignore file".format(a)) =20 # DSC Parser dp =3D DscParser() @@ -99,11 +108,23 @@ class DscCompleteCheck(ICiBuildPlugin): infp.SetPackagePaths(Edk2pathObj.PackagePathList) infp.ParseFile(INF) if("MODULE_TYPE" not in infp.Dict): - tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYP= E {0}".format(INF)) + tc.LogStdOut( + "Ignoring INF. Missing key for MODULE_TYPE {0}".fo= rmat(INF)) continue =20 if(infp.Dict["MODULE_TYPE"] =3D=3D "HOST_APPLICATION"): - tc.LogStdOut("Ignoring INF. Module type is HOST_APPLI= CATION {0}".format(INF)) + # should compile test a library that is declared type = HOST_APPLICATION + pass + + elif len(infp.SupportedPhases) > 0 and \ + "HOST_APPLICATION" in infp.SupportedPhases: + # should compile test a library that supports HOST_APP= LICATION but + # require it to be an explicit opt-in + pass + + else: + tc.LogStdOut( + "Ignoring INF. MODULE_TYPE or suppored phases not = HOST_APPLICATION {0}".format(INF)) continue =20 logging.critical(INF + " not in " + wsr_dsc_path) @@ -111,8 +132,9 @@ class DscCompleteCheck(ICiBuildPlugin): overall_status =3D overall_status + 1 =20 # If XML object exists, add result - if overall_status is not 0: - tc.SetFailed("DscCompleteCheck {0} Failed. Errors {1}".format= (wsr_dsc_path, overall_status), "CHECK_FAILED") + if overall_status !=3D 0: + tc.SetFailed("HostUnitTestDscCompleteCheck {0} Failed. Errors= {1}".format( + wsr_dsc_path, overall_status), "CHECK_FAILED") else: tc.SetSuccess() return overall_status diff --git a/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCom= pleteCheck_plug_in.yaml b/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostU= nitTestDscCompleteCheck_plug_in.yaml new file mode 100644 index 0000000000..82cebd7667 --- /dev/null +++ b/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCh= eck_plug_in.yaml @@ -0,0 +1,12 @@ +## +# CiBuildPlugin used to confirm all INFs are listed in +# the components section of package dsc +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +{ + "scope": "host-based-test", + "name": "Host Unit Test Dsc Complete Check Test", + "module": "HostUnitTestDscCompleteCheck" + } diff --git a/.pytool/Plugin/HostUnitTestDscCompleteCheck/Readme.md b/.pytoo= l/Plugin/HostUnitTestDscCompleteCheck/Readme.md new file mode 100644 index 0000000000..d77a1f2af1 --- /dev/null +++ b/.pytool/Plugin/HostUnitTestDscCompleteCheck/Readme.md @@ -0,0 +1,32 @@ +# Host Unit Test Dsc Complete Check Plugin + +This CiBuildPlugin scans all INF files from a package for those related to= host +based unit tests confirms they are listed in the unit test DSC file for th= e package. +The test considers it an error if any INF meeting the requirements does no= t appear +in the `Components` section of the unit test DSC. This is critical because +much of the CI infrastructure assumes that modules will be listed in the = DSC +and compiled. + +This test will only require INFs in the following cases: + +1. When MODULE_TYPE =3D HOST_APPLICATION +2. When a Library instance supports the HOST_APPLICATION environment + +## Configuration + +The plugin has a few configuration options to support the UEFI codebase. + +``` yaml +"HostUnitTestDscCompleteCheck": { + "DscPath": "", # Path to Host based unit test DSC file + "IgnoreInf": [] # Ignore INF if found in filesystem but not dsc +} +``` + +### DscPath + +Path to DSC to consider platform dsc + +### IgnoreInf + +Ignore error if Inf file is not listed in DSC file diff --git a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py b/.pytoo= l/Plugin/LibraryClassCheck/LibraryClassCheck.py index a62a7e912b..20d87f13f5 100644 --- a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py +++ b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py @@ -146,7 +146,7 @@ class LibraryClassCheck(ICiBuildPlugin): =20 =20 # If XML object exists, add result - if overall_status is not 0: + if overall_status !=3D 0: tc.SetFailed("LibraryClassCheck {0} Failed. Errors {1}".forma= t(wsr_dec_path, overall_status), "CHECK_FAILED") else: tc.SetSuccess() --=20 2.21.0.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#53459): https://edk2.groups.io/g/devel/message/53459 Mute This Topic: https://groups.io/mt/70061160/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-