From nobody Tue Apr 23 09:11:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+96548+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96548+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1669117679; cv=none; d=zohomail.com; s=zohoarc; b=KidrI+8E+sVS8ueKdFW7GNJ/JDWK89KbnDlTa1v3VJK1nqQk38Pwucurq3sbtVDoj0+h/6i7dHWhop1NS27VVn8GgpCGY6BZplH4/oWwne99r+CqMqCVij+eWSNlBof+H8DFjQbAqBigy31r2sFonnk9z5tiWR1H5VoWxfPwtN8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1669117679; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=xzXeITDgLLHBeoGQuoyvN+WhTIdu/y+toZ2FiLpsBLQ=; b=ZNFrjYY3Fmw3wYefKKnXuk/WUu077lpgElSiO8oHLialTor2cIZvmDaIpesTPZXMzmU/nQlCLx7J9Rg36Xp9qBecc5uLKUBZ2kfPSQ4ASlzOFM3gMHijGsetpWIVrdLq2BU3lCWHY7E76gO6035trZBzrTDA0x/pssG91Sms7mA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96548+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1669117679749700.6639388873816; Tue, 22 Nov 2022 03:47:59 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id L9mtYY1788612xIxxlXPsGnJ; Tue, 22 Nov 2022 03:47:59 -0800 X-Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mx.groups.io with SMTP id smtpd.web11.16865.1669117674960617993 for ; Tue, 22 Nov 2022 03:47:59 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10538"; a="293505479" X-IronPort-AV: E=Sophos;i="5.96,183,1665471600"; d="scan'208";a="293505479" X-Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 03:47:58 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10538"; a="592118784" X-IronPort-AV: E=Sophos;i="5.96,183,1665471600"; d="scan'208";a="592118784" X-Received: from shwdeopenlab702.ccr.corp.intel.com ([10.239.56.220]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 03:47:57 -0800 From: "duntan" To: devel@edk2.groups.io Cc: Ard Biesheuvel , Jiewen Yao , Jordan Justen , Gerd Hoffmann , Ray Ni Subject: [edk2-devel] [PATCH 1/3] OvmfPkg/PlatformCI: Expand Ovmf PlatformCI for Shell UnitTest Date: Tue, 22 Nov 2022 19:47:35 +0800 Message-Id: <20221122114737.393-2-dun.tan@intel.com> In-Reply-To: <20221122114737.393-1-dun.tan@intel.com> References: <20221122114737.393-1-dun.tan@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,dun.tan@intel.com X-Gm-Message-State: zUdFT9j9UWtJ3GwufvJCu4B2x1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1669117679; bh=jKqgWGxgHtctgamK0Jl7MLqJay0XUoKmP2O0jNOFEwE=; h=Cc:Date:From:Reply-To:Subject:To; b=hl/ZfT84FCNgEp9KZ6ce6t14GLs93JdyB9YYunrLaM5ej5qWLCpG/bs4AH+ILJDfAqt rY8et9PqaOj0pk4vHFARTSKTS1dqCcYO9H+9BT4KtfJqo/jj5GwNX4dPOzxyWWH512d9h BpHDb2iruaW1baJFoZPO+Ku6I9NjrMpNa7M= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1669117681495100001 Content-Type: text/plain; charset="utf-8" Expand Ovmf PlatformBuild.py and PlatformBuildLib.py to support building and running specific Shell target UnitTest modules. In the new CommonPlatform class: It provides new class attributes and new methods to support build and run specific Shell Unit Test modules. In the new SettingsManager class for stuart_pr_eval: It calls new API in CommonPlatform to updates PackagesSupported based on -u ShellUnitTestList input from cmd. The package which contains the module in ShellUnitTestList will be added into PackagesSupported for further evaluation. In the new PlatformBuilder class for stuart_build: 1.In PlatformPostBuild(), it conditionally calls new API in CommonPlatform to build -u ShellUnitTestList in -p PkgsToBuild and copy them to VirtualDrive folder. If no -p option, all the modules in -u ShellUnitTestList will be built. 2.In FlashRomImage(), it conditionally calls the new API in CommonPlatform to write all efi files name in VirtualDrive into startup.nsh and output UnitTest log into ShellUnitTestLog. 3. After the boot process, it conditionally calls new API in CommonPlatform to check the UnitTest boot log. Signed-off-by: Dun Tan Cc: Ard Biesheuvel Cc: Jiewen Yao Cc: Jordan Justen Cc: Gerd Hoffmann Cc: Ray Ni --- OvmfPkg/PlatformCI/PlatformBuild.py | 112 +++++++++++++++++++++++++++++= +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= ++++++++ OvmfPkg/PlatformCI/PlatformBuildLib.py | 51 +++++++++++++++++++++++++++++= ++++++++++++++++------ 2 files changed, 157 insertions(+), 6 deletions(-) diff --git a/OvmfPkg/PlatformCI/PlatformBuild.py b/OvmfPkg/PlatformCI/Platf= ormBuild.py index 6c541cdea4..72cb7e0e9e 100644 --- a/OvmfPkg/PlatformCI/PlatformBuild.py +++ b/OvmfPkg/PlatformCI/PlatformBuild.py @@ -6,6 +6,11 @@ ## import os import sys +import shutil +import logging +import re +from edk2toolext.environment import shell_environment +from edk2toolext.environment.multiple_workspace import MultipleWorkspace =20 sys.path.append(os.path.dirname(os.path.abspath(__file__))) from PlatformBuildLib import SettingsManager @@ -24,6 +29,10 @@ class CommonPlatform(): Scopes =3D ('ovmf', 'edk2-build') WorkspaceRoot =3D os.path.realpath(os.path.join( os.path.dirname(os.path.abspath(__file__)), "..", "..")) + # Support build and run Shell Unit Test modules + UnitTestModuleList =3D {} + RunShellUnitTest =3D False + ShellUnitTestLog =3D '' =20 @classmethod def GetDscName(cls, ArchCsv: str) -> str: @@ -39,5 +48,108 @@ class CommonPlatform(): dsc +=3D ".dsc" return dsc =20 + @classmethod + def UpdatePackagesSupported(cls, ShellUnitTestListArg): + ''' Update PackagesSupported by -u ShellUnitTestList from cmd line= . ''' + UnitTestModuleListStr =3D ','.join(ShellUnitTestListArg) + if not re.search(r'.+.inf:.+.dsc', UnitTestModuleListStr): + raise Exception('No valid ModulePath:DscPath in the -u {}'.for= mat(UnitTestModuleListStr)) + UnitTestModuleList =3D UnitTestModuleListStr.split(',') + PackagesSupported =3D [] + for KeyValue in UnitTestModuleList: + PkgName =3D KeyValue.split("Pkg")[0] + 'Pkg' + if PkgName not in PackagesSupported: + PackagesSupported.append(PkgName) + cls.PackagesSupported =3D tuple(PackagesSupported) + print('PackagesSupported for UnitTest is {}'.format(cls.PackagesSu= pported)) + + @classmethod + def UpdateUnitTestConfig(cls, args): + ''' Update UnitTest config by -u ShellUnitTestList and -p PkgsToBu= ildForUT from cmd line. + ShellUnitTestList is in this format: {module1:dsc1, module2:ds= c2, module3:dsc2...}. + Only the modules which are in the PkgsToBuildForUT list are ad= ded into self.UnitTestModuleList. + ''' + UnitTestModuleListStr =3D ','.join(args.ShellUnitTestList) + if not re.search(r'.+.inf:.+.dsc', UnitTestModuleListStr): + raise Exception('No valid ModulePath:DscPath in the -u {}'.for= mat(args.ShellUnitTestList)) + UnitTestModuleList =3D UnitTestModuleListStr.split(',') + if args.PkgsToBuildForUT is None or all(['Pkg' not in Pkg for Pkg = in args.PkgsToBuildForUT]): + # No invalid Pkgs from input. Build all modules in -u UnitTest= ModuleList. + for KeyValue in UnitTestModuleList: + UnitTestPath =3D os.path.normpath(KeyValue.split(":")[0]) + DscPath =3D os.path.normpath(KeyValue.split(":")[1]) + cls.UnitTestModuleList[UnitTestPath] =3D DscPath + else: + PkgsToBuildForUT =3D ','.join(args.PkgsToBuildForUT).split(',') + for KeyValue in UnitTestModuleList: + UnitTestPath =3D os.path.normpath(KeyValue.split(":")[0]) + DscPath =3D os.path.normpath(KeyValue.split(":")[1]) + PkgName =3D UnitTestPath.split("Pkg")[0] + 'Pkg' + if PkgName in PkgsToBuildForUT: + cls.UnitTestModuleList[UnitTestPath] =3D DscPath + if len(cls.UnitTestModuleList) > 0: + cls.RunShellUnitTest =3D True + cls.ShellUnitTestLog =3D os.path.join(cls.WorkspaceRoot, 'Buil= d', "BUILDLOG_UnitTest.txt") + print('UnitTestModuleList is {}'.format(cls.UnitTestModuleList= )) + + def BuildUnitTest(self): + ''' Build specific DSC for modules in UnitTestModuleList ''' + self.env =3D shell_environment.GetBuildVars() + self.ws =3D PlatformBuilder.GetWorkspaceRoot(self) + self.mws =3D MultipleWorkspace() + self.pp =3D '' + VirtualDrive =3D os.path.join(self.env.GetValue("BUILD_OUTPUT_BASE= "), "VirtualDrive") + os.makedirs(VirtualDrive, exist_ok=3DTrue) + + # DSC by self.GetDscName() should have been built in BUILD process. + BuiltDsc =3D [CommonPlatform.GetDscName(",".join(self.env.GetValue= ("TARGET_ARCH").split(' ')))] + for UnitTestPath, DscPath in CommonPlatform.UnitTestModuleList.ite= ms(): + if DscPath not in BuiltDsc: + ModuleName =3D os.path.split(UnitTestPath)[1].split('.inf'= )[0] + logging.info('Build {0} for {1}'.format(DscPath, ModuleNam= e)) + BuiltDsc.append(DscPath) + Arch =3D self.env.GetValue("TARGET_ARCH").split(" ") + if 'X64' in Arch: + UTArch =3D 'X64' + else: + UTArch =3D 'IA32' + self.env.AllowOverride("ACTIVE_PLATFORM") + self.env.SetValue("ACTIVE_PLATFORM", DscPath, "For UnitTes= t") + self.env.AllowOverride("TARGET_ARCH") + self.env.SetValue("TARGET_ARCH", UTArch, "For UnitTest") #= Set UnitTest arch the same as Ovmf Shell module. + ret =3D PlatformBuilder.Build(self) = # Build specific dsc for UnitTest modules + if (ret !=3D 0): + return ret + ret =3D PlatformBuilder.ParseDscFile(self) # Parse OUTPUT_DIRE= CTORY from dsc files + if(ret !=3D 0): + return ret + OutputPath =3D os.path.normpath(os.path.join(self.ws, self.env= .GetValue("OUTPUT_DIRECTORY"))) + EfiPath =3D os.path.join(OutputPath, self.env.GetValue("TAR= GET") + "_" + self.env.GetValue("TOOL_CHAIN_TAG"), + UTArch, UnitTestPath.split('.inf')[0], "OUTPUT", = ModuleName + '.efi') + logging.info('Copy {0}.efi from:{1}'.format(ModuleName, EfiPat= h)) + shutil.copy(EfiPath, VirtualDrive) + return 0 + + @staticmethod + def WriteEfiToStartup(EfiFolder, FileObj): + ''' Write all the .efi files' name in VirtualDrive into Startup.ns= h ''' + for Root,Dirs,Files in os.walk(EfiFolder): + for File in Files: + if os.path.splitext(File)[1] =3D=3D '.efi': + FileObj.write("{0} \n".format(File)) + + @classmethod + def CheckUnitTestLog(cls): + ''' Check the boot log for UnitTest ''' + File =3D open(cls.ShellUnitTestLog, "r") + FileContent =3D File.readlines() + logging.info('Check the UnitTest boot log:{0}'.format(cls.ShellUni= tTestLog)) + for Index in range(len(FileContent)): + if 'FAILURE MESSAGE:' in FileContent[Index]: + if FileContent[Index + 1].strip() !=3D '': + FailureMessage =3D FileContent[Index + 1] + FileConten= t[Index + 2] + return FailureMessage + return 0 + import PlatformBuildLib PlatformBuildLib.CommonPlatform =3D CommonPlatform diff --git a/OvmfPkg/PlatformCI/PlatformBuildLib.py b/OvmfPkg/PlatformCI/Pl= atformBuildLib.py index bfef9849c7..b42235b2ac 100644 --- a/OvmfPkg/PlatformCI/PlatformBuildLib.py +++ b/OvmfPkg/PlatformCI/PlatformBuildLib.py @@ -103,15 +103,28 @@ class SettingsManager(UpdateSettingsManager, SetupSet= tingsManager, PrEvalSetting =20 return build_these_packages =20 + def AddCommandLineOptions(self, parserObj): + parserObj.add_argument('-u', '--UnitTest', dest=3D'ShellUnitTestLi= st', type=3Dstr, + help=3D'Optional - Key:Value that contains Shell UnitTest list= and corresponding DscPath you want to test.(workspace relative)' + 'Can list multiple by doing -u , or -u -u ', + action=3D"append", default=3DNone) + + def RetrieveCommandLineOptions(self, args): + if args.ShellUnitTestList: + CommonPlatform.UpdatePackagesSupported(args.ShellUnitTestList) + def GetPlatformDscAndConfig(self) -> tuple: ''' If a platform desires to provide its DSC then Policy 4 will ev= aluate if any of the changes will be built in the dsc. =20 The tuple should be (, ) + This Policy 4 can only be applied when PackagesSupported only cont= ains OvmfPkg Since it doesn't support + mutiple packages evaluation. ''' - dsc =3D CommonPlatform.GetDscName(",".join(self.ActualArchitecture= s)) - return (f"OvmfPkg/{dsc}", {}) - + if (len(CommonPlatform.PackagesSupported) =3D=3D 1) and (CommonPla= tform.PackagesSupported[0] =3D=3D 'OvmfPkg'): + dsc =3D CommonPlatform.GetDscName(",".join(self.ActualArchitec= tures)) + return (f"OvmfPkg/{dsc}", {}) + return None =20 # ####################################################################= ################### # # Actual Configuration for Platform Build = # @@ -126,6 +139,14 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManag= er): help=3D"Optional - CSV of architecture to build. IA32 will us= e IA32 for Pei & Dxe. " "X64 will use X64 for both PEI and DXE. IA32,X64 will use IA3= 2 for PEI and " "X64 for DXE. default is IA32,X64") + parserObj.add_argument('-p', '--pkg', '--pkg-dir', dest=3D'PkgsToB= uildForUT', type=3Dstr, + help=3D'Optional - Package list you want to build for UnitTest= .efi. (workspace relative).' + 'Can list multiple by doing -p , or -p -p <= pkg4>.If no valid input -p, build and run all -u UnitTest', + action=3D"append", default=3DNone) + parserObj.add_argument('-u', '--UnitTest', dest=3D'ShellUnitTestLi= st', type=3Dstr, + help=3D'Optional - Key:Value that contains Shell UnitTest list= and corresponding DscPath you want to test.(workspace relative)' + 'Can list multiple by doing -u , or -u -u ', + action=3D"append", default=3DNone) =20 def RetrieveCommandLineOptions(self, args): ''' Retrieve command line options from the argparser ''' @@ -133,6 +154,10 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManag= er): shell_environment.GetBuildVars().SetValue("TARGET_ARCH"," ".join(a= rgs.build_arch.upper().split(",")), "From CmdLine") dsc =3D CommonPlatform.GetDscName(args.build_arch) shell_environment.GetBuildVars().SetValue("ACTIVE_PLATFORM", f"Ovm= fPkg/{dsc}", "From CmdLine") + self.RunShellUnitTest =3D False + if args.ShellUnitTestList: + CommonPlatform.UpdateUnitTestConfig(args) + self.RunShellUnitTest =3D CommonPlatform.RunShellUnitTest =20 def GetWorkspaceRoot(self): ''' get WorkspacePath ''' @@ -176,6 +201,11 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManag= er): return 0 =20 def PlatformPostBuild(self): + if self.RunShellUnitTest: + ret =3D CommonPlatform.BuildUnitTest(self) + if ret !=3D0: + logging.critical("Build UnitTest failed") + return ret return 0 =20 def FlashRomImage(self): @@ -210,9 +240,13 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManag= er): else: args +=3D " -pflash " + os.path.join(OutputPath_FV, "OVMF.fd")= # path to firmware =20 - if (self.env.GetValue("MAKE_STARTUP_NSH").upper() =3D=3D "TRUE"): f =3D open(os.path.join(VirtualDrive, "startup.nsh"), "w") + if self.RunShellUnitTest: + # When RunShellUnitTest is True, write all efi files name = into startup.nsh. + CommonPlatform.WriteEfiToStartup(VirtualDrive, f) + # Output UnitTest log into ShellUnitTestLog. + args +=3D " -serial file:{}".format(CommonPlatform.ShellUn= itTestLog) f.write("BOOT SUCCESS !!! \n") ## add commands here f.write("reset -s\n") @@ -222,6 +256,11 @@ class PlatformBuilder( UefiBuilder, BuildSettingsManag= er): =20 if ret =3D=3D 0xc0000005: #for some reason getting a c0000005 on successful return - return 0 - + ret =3D 0 + if self.RunShellUnitTest and ret =3D=3D 0: + # Check the UnitTest boot log. + UnitTestResult =3D CommonPlatform.CheckUnitTestLog() + if (UnitTestResult): + logging.info("UnitTest failed with this FAILURE MESSAGE:\n= {}".format(UnitTestResult)) + return UnitTestResult return ret --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96548): https://edk2.groups.io/g/devel/message/96548 Mute This Topic: https://groups.io/mt/95193613/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Tue Apr 23 09:11:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+96549+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96549+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1669117683; cv=none; d=zohomail.com; s=zohoarc; b=QtgYNQA01y4C8NOq5x3Zu/uMcEuL4vUMiHkCwmLqtAuU515QTE4bDxKwdJzG+HiSivF6OWjy4kqRYqRNspunMfOu9Wjx0OJDvt7R3FgoBrv8Vg1WsjDQJhdvwEQwNFJmNGinPQz4xrzZud7e8i9b3NVRooiaeClrQEofQbrGvLk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1669117683; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=r/eChDN98dHdRWsq3xPvZQoIs31+ebO8gKQ2qT/2SPs=; b=RM+xikHiWxBPFDsGheTRWGFPh9EQytwd4KHKabSv/pFuVAXhLSMrCsvzanwwG0SZAvSVkcA7JAMpHvXWH4tWzM1/N75Qn3o0+KjZu6nu3p1E0IJWtuyn5AolTSmllIW0onqg/f5jD+KeYBB9YYAcXHJIS27Vhrq+iwU6nADX1vI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96549+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1669117683874439.31263815293323; Tue, 22 Nov 2022 03:48:03 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id arMkYY1788612xdNMnK7S49K; Tue, 22 Nov 2022 03:48:03 -0800 X-Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mx.groups.io with SMTP id smtpd.web11.16865.1669117674960617993 for ; Tue, 22 Nov 2022 03:48:03 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10538"; a="293505499" X-IronPort-AV: E=Sophos;i="5.96,183,1665471600"; d="scan'208";a="293505499" X-Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 03:48:03 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10538"; a="592118835" X-IronPort-AV: E=Sophos;i="5.96,183,1665471600"; d="scan'208";a="592118835" X-Received: from shwdeopenlab702.ccr.corp.intel.com ([10.239.56.220]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 03:48:01 -0800 From: "duntan" To: devel@edk2.groups.io Cc: Ard Biesheuvel , Jiewen Yao , Jordan Justen , Gerd Hoffmann , Ray Ni Subject: [edk2-devel] [PATCH 2/3] OvmfPkg/PlatformCI: Add new JOB in .yml of OvmfPkg PlatformCI Date: Tue, 22 Nov 2022 19:47:36 +0800 Message-Id: <20221122114737.393-3-dun.tan@intel.com> In-Reply-To: <20221122114737.393-1-dun.tan@intel.com> References: <20221122114737.393-1-dun.tan@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,dun.tan@intel.com X-Gm-Message-State: ItSJPj7Qvtug6ykb2CECXSayx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1669117683; bh=9f1PZqPwvfRh5XCCotIRmrDhEjmBvJhTJAfsG5Wh1tk=; h=Cc:Date:From:Reply-To:Subject:To; b=d8wSBHtRxtnjUZf5RqWfy+PE6I0klLMDQdGSNsOEXkT9n6daAI9pRUPDbG16jInBOJD 7yaPgCN8TT5V+FLjMGBWqX+NKJfhAAy83hJpKbNVYoXmsDlwNIPNN6iONmSS7HwKne64G 0t+KOb0loJXM+KFKDacHVBI68z/CFnokbEk= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1669117685373100001 Content-Type: text/plain; charset="utf-8" Add new job like OVMF_X64_DEBUG_UNIT_TEST in OvmfPkg PlatformCI .yml file. New parameter unit_test_list is used to specify Shell Unit Test list which needs to build and run. Format for this input should be:'-u ModulePath1:DscPath1,ModulePath2:DscPath2' or '-u ModulePath1:DscPath1 -u ModulePath2:DscPath2'. (Path is edk2 workspace relative) Signed-off-by: Dun Tan Cc: Ard Biesheuvel Cc: Jiewen Yao Cc: Jordan Justen Cc: Gerd Hoffmann Cc: Ray Ni --- OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 11 +++++++++++ OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 11 +++++++++++ 2 files changed, 22 insertions(+) diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/P= latformCI/.azurepipelines/Ubuntu-GCC5.yml index 7160d95f7e..2242ffebb5 100644 --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -22,6 +22,7 @@ jobs: vm_image: 'ubuntu-18.04' should_run: true run_flags: "MAKE_STARTUP_NSH=3DTRUE QEMU_HEADLESS=3DTRUE" + unit_test_list: '' =20 #Use matrix to speed up the build process strategy: @@ -55,6 +56,15 @@ jobs: Build.Target: "DEBUG" Run.Flags: $(run_flags) Run: $(should_run) + OVMF_X64_DEBUG_UNIT_TEST: + Build.File: "$(package)/PlatformCI/PlatformBuild.py" + Build.Arch: "X64" + Build.Flags: "" + Build.Target: "DEBUG" + Run.Flags: $(run_flags) + Run: $(should_run) + # unit_test_list should be the format: '-u ModulePath1:DscPath= 1,ModulePath2:DscPath2' or '-u ModulePath1:DscPath1 -u ModulePath2:DscPath2= '.(Path is workspace relative) + unit_test_list: '' OVMF_X64_RELEASE: Build.File: "$(package)/PlatformCI/PlatformBuild.py" Build.Arch: "X64" @@ -187,6 +197,7 @@ jobs: build_file: $(Build.File) build_flags: $(Build.Flags) run_flags: $(Run.Flags) + unit_test_list: $(unit_test_list) extra_install_step: - bash: sudo apt-get install qemu displayName: Install qemu diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPk= g/PlatformCI/.azurepipelines/Windows-VS2019.yml index 7d6344d638..881db9eb27 100644 --- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml +++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml @@ -21,6 +21,7 @@ jobs: vm_image: 'windows-2019' should_run: true run_flags: "MAKE_STARTUP_NSH=3DTRUE QEMU_HEADLESS=3DTRUE" + unit_test_list : '' =20 #Use matrix to speed up the build process strategy: @@ -54,6 +55,15 @@ jobs: Build.Target: "DEBUG" Run.Flags: $(run_flags) Run: $(should_run) + OVMF_X64_DEBUG_UNIT_TEST: + Build.File: "$(package)/PlatformCI/PlatformBuild.py" + Build.Arch: "X64" + Build.Flags: "" + Build.Target: "DEBUG" + Run.Flags: $(run_flags) + Run: $(should_run) + # unit_test_list should be the format: '-u ModulePath1:DscPath= 1,ModulePath2:DscPath2' or '-u ModulePath1:DscPath1 -u ModulePath2:DscPath2= '.(Path is workspace relative) + unit_test_list: '' OVMF_X64_RELEASE: Build.File: "$(package)/PlatformCI/PlatformBuild.py" Build.Arch: "X64" @@ -133,6 +143,7 @@ jobs: build_file: $(Build.File) build_flags: $(Build.Flags) run_flags: $(Run.Flags) + unit_test_list: $(unit_test_list) extra_install_step: - powershell: choco install qemu --version=3D2021.5.5; Write-Host = "##vso[task.prependpath]c:\Program Files\qemu" displayName: Install QEMU and Set QEMU on path # friendly name d= isplayed in the UI --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96549): https://edk2.groups.io/g/devel/message/96549 Mute This Topic: https://groups.io/mt/95193614/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Tue Apr 23 09:11:49 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+96550+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96550+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1669117688; cv=none; d=zohomail.com; s=zohoarc; b=cLpfbY3tfmz3VMq3GAdQTxWNSq/9HCPPSu9rybP4QENO7T332DYqUu0f0W9vba3XtOrAknniv/XHJaXdEEFkDjIRVxE87XzA3vOefXR8oEdhyVd9Li7xgNrf0vt+yc5ns5xl5KiTZ1r2Wt7XC4m8BO5U4CQ1LgbztZ3AKVz4sl0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1669117688; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=8moTpmqaGDNssQZPUKem786w0ZkhRe+WpTQNwbUQL6A=; b=A3P/E/DPyAZEAwytwrCRfdAJWyOv6XGQYIp6ziIMmO+NxuyWMrxfxr+pNQKF3tli5pSx+v8cI1zuMxrKh6KT1kMlO5KcVFhZYC/goUrdZwhm7HyQ1uvMBqixo6b5qiJH9TfNms7uxp0eHOwwaEiG/WWJ9luDoXLwKwH07Z4eol8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96550+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1669117688880680.1528090297444; Tue, 22 Nov 2022 03:48:08 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id KJxGYY1788612xPJ8AmvPZKI; Tue, 22 Nov 2022 03:48:08 -0800 X-Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mx.groups.io with SMTP id smtpd.web11.16865.1669117674960617993 for ; Tue, 22 Nov 2022 03:48:08 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10538"; a="293505523" X-IronPort-AV: E=Sophos;i="5.96,183,1665471600"; d="scan'208";a="293505523" X-Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 03:48:07 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10538"; a="592118864" X-IronPort-AV: E=Sophos;i="5.96,183,1665471600"; d="scan'208";a="592118864" X-Received: from shwdeopenlab702.ccr.corp.intel.com ([10.239.56.220]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 03:48:06 -0800 From: "duntan" To: devel@edk2.groups.io Cc: Sean Brogan , Michael Kubacki , Michael D Kinney , Liming Gao , Ray Ni Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest Date: Tue, 22 Nov 2022 19:47:37 +0800 Message-Id: <20221122114737.393-4-dun.tan@intel.com> In-Reply-To: <20221122114737.393-1-dun.tan@intel.com> References: <20221122114737.393-1-dun.tan@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,dun.tan@intel.com X-Gm-Message-State: qiIFM8kjKZzDjj7hFhnV5vTKx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1669117688; bh=IynlNv5wSrdyZYGxOzq1izN3ENJW29ZBZpF3emc37ZU=; h=Cc:Date:From:Reply-To:Subject:To; b=DZyt0J2D57WMp6YjoSWePcwAnJyVeC69sjOs15niYPZotZpnWMKbHgp39Ke4LyfwTDu Bq7Z/LGcEXd5wv27d9HStwas9p4QS/tbaVjVavd6zm7FnvgOa/sB8y7VO3siB9KCFUHKP KdP9hdsYnTeg+PqYYO+l69/AeqWsUU8BPOw= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1669117689460100001 Content-Type: text/plain; charset="utf-8" Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules. In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule. In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build. In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell. Signed-off-by: Dun Tan Cc: Sean Brogan Cc: Michael Kubacki Cc: Michael D Kinney Cc: Liming Gao Cc: Ray Ni --- .azurepipelines/templates/platform-build-run-steps.yml | 21 ++++++++++++++= +++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azur= epipelines/templates/platform-build-run-steps.yml index 40a31a509f..51503287c4 100644 --- a/.azurepipelines/templates/platform-build-run-steps.yml +++ b/.azurepipelines/templates/platform-build-run-steps.yml @@ -30,6 +30,9 @@ parameters: - name: run_flags type: string default: '' +- name: unit_test_list + type: string + default: '' =20 - name: extra_install_step type: stepList @@ -49,7 +52,9 @@ steps: displayName: 'Install/Upgrade pip modules' =20 # Set default -- bash: echo "##vso[task.setvariable variable=3Dpkg_count]${{ 1 }}" +- bash: | + echo "##vso[task.setvariable variable=3Dpkg_count]${{ 1 }}" + echo "##vso[task.setvariable variable=3Dpkgs_to_build]${{ 'all' }}" =20 # Fetch the target branch so that pr_eval can diff them. # Seems like azure pipelines/github changed checkout process in nov 2020. @@ -62,7 +67,7 @@ steps: displayName: Check if ${{ parameters.build_pkg }} need testing inputs: filename: stuart_pr_eval - arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_tar= get}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullReques= t.targetBranch) --output-count-format-string "##vso[task.setvariable variab= le=3Dpkg_count;isOutpout=3Dtrue]{pkgcount}" + arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_tar= get}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullReques= t.targetBranch) --output-count-format-string "##vso[task.setvariable variab= le=3Dpkg_count;isOutpout=3Dtrue]{pkgcount}" --output-csv-format-string "##v= so[task.setvariable variable=3Dpkgs_to_build]{pkgcsv}" ${{ parameters.unit_= test_list}} condition: eq(variables['Build.Reason'], 'PullRequest') =20 # Setup repo @@ -97,14 +102,22 @@ steps: inputs: filename: stuart_build arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parame= ters.tool_chain_tag}} TARGET=3D${{ parameters.build_target}} -a ${{ paramet= ers.build_arch}} ${{ parameters.build_flags}} - condition: and(gt(variables.pkg_count, 0), succeeded()) + condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variable= s.unit_test_list, '')) + +# Build specific pkg for UnitTest +- task: CmdLine@1 + displayName: Build UnitTest + inputs: + filename: stuart_build + arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build) -c ${{ = parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parameters.tool_chain_tag}} T= ARGET=3D${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ pa= rameters.build_flags}} + condition: and(and(gt(variables.pkg_count, 0), succeeded()), not(eq(vari= ables.unit_test_list, ''))) =20 # Run - task: CmdLine@1 displayName: Run to shell inputs: filename: stuart_build - arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parame= ters.tool_chain_tag}} TARGET=3D${{ parameters.build_target}} -a ${{ paramet= ers.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} -= -FlashOnly + arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=3D${{ parame= ters.tool_chain_tag}} TARGET=3D${{ parameters.build_target}} -a ${{ paramet= ers.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} -= -FlashOnly ${{ parameters.unit_test_list}} condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variable= s['Run'], true)) timeoutInMinutes: 1 =20 --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96550): https://edk2.groups.io/g/devel/message/96550 Mute This Topic: https://groups.io/mt/95193616/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-