From nobody Sun May 19 04:56:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+98874+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+98874+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=kernel.org ARC-Seal: i=1; a=rsa-sha256; t=1674116527; cv=none; d=zohomail.com; s=zohoarc; b=a6RnvfyxGi6E1wLiK3cOWAgoauz5OxrfH15YO47tPanNmuNe90Hp/iNY/VEksZbsL1NmBRBzsOmy+ci4l7KbPoeSgvuWpPeqXRk/Ph3wa2U3Stx4nhfjN0qyck+nBoFtBKdP5g8uxJ9HNYe50FRwA5ZxNKLtf47881z3uE8Ojrs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674116527; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=h6dKJd4KBuctIv1uVE1hytLWctz+YQpnLHzEjoZ0Ke8=; b=FKHlGZAkf+VtR4XZzEX4RIUAQplQpdvU3WoyPz3ThyFEOEuoJ8hi90h97EdUVEVARCuitUhk3HvcRPPB4UeUcdmvQMg3I1vJrWpMptVxOl2J2vfbQByup0uDwXHqW1MPGzK5jhqQ2Y/hK5cIEbPe4Cvc8oLKqHikusUU3KDSKs4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+98874+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1674116527286128.920042508576; Thu, 19 Jan 2023 00:22:07 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id t4DnYY1788612xwe1qBawd6p; Thu, 19 Jan 2023 00:22:07 -0800 X-Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mx.groups.io with SMTP id smtpd.web10.40754.1674116526507237338 for ; Thu, 19 Jan 2023 00:22:06 -0800 X-Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F3CD761209; Thu, 19 Jan 2023 08:22:05 +0000 (UTC) X-Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41093C433F1; Thu, 19 Jan 2023 08:22:04 +0000 (UTC) From: "Ard Biesheuvel" To: devel@edk2.groups.io Cc: Ard Biesheuvel , Gerd Hoffmann , Michael Kubacki , Oliver Steffen Subject: [edk2-devel] [PATCH 1/3] ArmVirtPkg/PrePi: Ensure timely execution of library constructors Date: Thu, 19 Jan 2023 09:21:51 +0100 Message-Id: <20230119082153.1520196-2-ardb@kernel.org> In-Reply-To: <20230119082153.1520196-1-ardb@kernel.org> References: <20230119082153.1520196-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ardb@kernel.org X-Gm-Message-State: XkGGItt6Gt1t941B1omHAs2ex1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1674116527; bh=U7XYsWLlNG0NmBIIbZmFk+CPI9wfbQsoCL2LWzrHRTM=; h=Cc:Date:From:Reply-To:Subject:To; b=Blxf3aZ/sAeXlsg5dkLnERzapj+Vxhi3ov9VGFJKEmSx9vw5lAoeMJvc0dO4ztBGpzr ARjIFdP/kl8fqthIkXoAdWlK+Xbxes7HaHXV8ae6jvEptne2V9hQ80WHu1opKMs9ZW9yW 9TXq+e0diU5v3HxRgD+t++rAejrEZivXL/E= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1674116529273100006 Content-Type: text/plain; charset="utf-8" PrePi has a bare metal entry point, and so it is in charge of calling the library constructors once the C runtime has been initialized sufficiently. However, we are now relying on a HOB to have been constructed by the time the MMU code runs, and so the constructors should be run before that. Signed-off-by: Ard Biesheuvel --- ArmVirtPkg/PrePi/PrePi.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ArmVirtPkg/PrePi/PrePi.c b/ArmVirtPkg/PrePi/PrePi.c index c15dc305fced..3d943b2138d3 100755 --- a/ArmVirtPkg/PrePi/PrePi.c +++ b/ArmVirtPkg/PrePi/PrePi.c @@ -60,6 +60,9 @@ PrePiMain ( // InvalidateDataCacheRange ((VOID *)(UINTN)PcdGet64 (PcdFdBaseAddress), Pc= dGet32 (PcdFdSize)); =20 + // SEC phase needs to run library constructors by hand. + ProcessLibraryConstructorList (); + // Initialize MMU and Memory HOBs (Resource Descriptor HOBs) Status =3D MemoryPeim (UefiMemoryBase, FixedPcdGet32 (PcdSystemMemoryUef= iRegionSize)); ASSERT_EFI_ERROR (Status); @@ -93,9 +96,6 @@ PrePiMain ( // Now, the HOB List has been initialized, we can register performance i= nformation PERF_START (NULL, "PEI", NULL, StartTimeStamp); =20 - // SEC phase needs to run library constructors by hand. - ProcessLibraryConstructorList (); - // Assume the FV that contains the SEC (our code) also contains a compre= ssed FV. Status =3D DecompressFirstFv (); ASSERT_EFI_ERROR (Status); --=20 2.39.0 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#98874): https://edk2.groups.io/g/devel/message/98874 Mute This Topic: https://groups.io/mt/96373698/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun May 19 04:56:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+98875+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+98875+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=kernel.org ARC-Seal: i=1; a=rsa-sha256; t=1674116529; cv=none; d=zohomail.com; s=zohoarc; b=Z4EqEpAo1iGkfVZ6MNGPFsC71/UolV88QjCmJqp8h3xj4WNdhD+kk8SXdS7gSlBHCFV2SAkl18moxdTztUrOEaQzLx/rv0vZT1Lnt/1RapAzGEsw1tiatGEPRM9YJEOE28RnPHGhkqvV5oKRUZ+VqjK2ZV92jf/QdviA/RD3WPM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674116529; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=y/jPC6IkzrIQ0XeRj4f4tZwN7/LD+9SxbehasifT6AI=; b=RNotXN66mmV4EBVSbTR+q6FT57+vkkHKDVfWCEJQVEc1fxOBDbQ4DoL7ZOHuJiko6Jz8G75Lr6iCt5Z/2Uk/neyoFbUFN+1YQGnaYkjZNbSInk1tk1Frj7ooaCMjz8uI6SfcvjVNhIjHdWagLOxWknVoYXXM8l/hWmUW+/eSt4A= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+98875+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1674116529352436.64854813456134; Thu, 19 Jan 2023 00:22:09 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id ptWeYY1788612xeZ7NtegvuX; Thu, 19 Jan 2023 00:22:08 -0800 X-Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mx.groups.io with SMTP id smtpd.web11.41013.1674116528077567672 for ; Thu, 19 Jan 2023 00:22:08 -0800 X-Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8AAAC611FB; Thu, 19 Jan 2023 08:22:07 +0000 (UTC) X-Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE34BC433EF; Thu, 19 Jan 2023 08:22:05 +0000 (UTC) From: "Ard Biesheuvel" To: devel@edk2.groups.io Cc: Ard Biesheuvel , Gerd Hoffmann , Michael Kubacki , Oliver Steffen Subject: [edk2-devel] [PATCH 2/3] ArmVirtPkg: CI: Perform build and boot test of ArmVirtQemuKernel Date: Thu, 19 Jan 2023 09:21:52 +0100 Message-Id: <20230119082153.1520196-3-ardb@kernel.org> In-Reply-To: <20230119082153.1520196-1-ardb@kernel.org> References: <20230119082153.1520196-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ardb@kernel.org X-Gm-Message-State: 4SI008ft5wz2lvmg78iKItd4x1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1674116528; bh=uzUufCGyQGRfxIYqOpkRna1Jv+Ouu18+5pd8C4JOl+4=; h=Cc:Date:From:Reply-To:Subject:To; b=O50AfpzElFKQH9W6MBpv44xwLjtrHqLWm28QsTlaFPEj9FbvYD5HXNlHF1SSBZS9eEB IWStSeddumq07x9aSc1IfcX8iEGowG4izybnn2R/nlyy6pTymq+QSK5r5JLikAGXKS2z2 +GUMxAw6nP7WCfdzJ+PHQFTr0H1oy8oakZc= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1674116531273100010 Content-Type: text/plain; charset="utf-8" Signed-off-by: Ard Biesheuvel --- ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 43 ++++ ArmVirtPkg/PlatformCI/QemuKernelBuild.py | 268 ++++++++++++++= ++++++ 2 files changed, 311 insertions(+) diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVir= tPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index 5fa7518d2c5e..f39d52aed541 100644 --- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -72,6 +72,49 @@ jobs: Run.Flags: $(run_flags) Run: $(should_run) =20 + QEMU_KERNEL_AARCH64_DEBUG: + Build.File: "$(package)/PlatformCI/QemuKernelBuild.py" + Build.Arch: "AARCH64" + Build.Flags: "" + Build.Target: "DEBUG" + Run.Flags: $(run_flags) + Run: $(should_run) + QEMU_KERNEL_AARCH64_RELEASE: + Build.File: "$(package)/PlatformCI/QemuKernelBuild.py" + Build.Arch: "AARCH64" + Build.Flags: "" + Build.Target: "RELEASE" + Run.Flags: $(run_flags) + Run: $(should_run) + QEMU_KERNEL_AARCH64_NOOPT: + Build.File: "$(package)/PlatformCI/QemuKernelBuild.py" + Build.Arch: "AARCH64" + Build.Flags: "" + Build.Target: "NOOPT" + Run.Flags: $(run_flags) + Run: $(should_run) + QEMU_KERNEL_ARM_DEBUG: + Build.File: "$(package)/PlatformCI/QemuKernelBuild.py" + Build.Arch: "ARM" + Build.Flags: "" + Build.Target: "DEBUG" + Run.Flags: $(run_flags) + Run: $(should_run) + QEMU_KERNEL_ARM_RELEASE: + Build.File: "$(package)/PlatformCI/QemuKernelBuild.py" + Build.Arch: "ARM" + Build.Flags: "" + Build.Target: "RELEASE" + Run.Flags: $(run_flags) + Run: $(should_run) + QEMU_KERNEL_ARM_NOOPT: + Build.File: "$(package)/PlatformCI/QemuKernelBuild.py" + Build.Arch: "ARM" + Build.Flags: "" + Build.Target: "NOOPT" + Run.Flags: $(run_flags) + Run: $(should_run) + workspace: clean: all =20 diff --git a/ArmVirtPkg/PlatformCI/QemuKernelBuild.py b/ArmVirtPkg/Platform= CI/QemuKernelBuild.py new file mode 100644 index 000000000000..1c652478cb41 --- /dev/null +++ b/ArmVirtPkg/PlatformCI/QemuKernelBuild.py @@ -0,0 +1,268 @@ +# @file +# Script to Build ArmVirtPkg UEFI firmware +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +import os +import logging +import io + +from edk2toolext.environment import shell_environment +from edk2toolext.environment.uefi_build import UefiBuilder +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, Requir= edSubmodule +from edk2toolext.invocables.edk2_update import UpdateSettingsManager +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager +from edk2toollib.utility_functions import RunCmd +from edk2toollib.utility_functions import GetHostInfo + +# ########################################################################= ############### # +# Common Configuration = # +# ########################################################################= ############### # + + +class CommonPlatform(): + ''' Common settings for this platform. Define static data here and use + for the different parts of stuart + ''' + PackagesSupported =3D ("ArmVirtPkg",) + ArchSupported =3D ("AARCH64", "ARM") + TargetsSupported =3D ("DEBUG", "RELEASE", "NOOPT") + Scopes =3D ('armvirt', 'edk2-build') + WorkspaceRoot =3D os.path.realpath(os.path.join( + os.path.dirname(os.path.abspath(__file__)), "..", "..")) + + # ####################################################################= ################### # + # Configuration for Update & Setup = # + # ####################################################################= ################### # + + +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalS= ettingsManager): + + def GetPackagesSupported(self): + ''' return iterable of edk2 packages supported by this build. + These should be edk2 workspace relative paths ''' + return CommonPlatform.PackagesSupported + + def GetArchitecturesSupported(self): + ''' return iterable of edk2 architectures supported by this build = ''' + return CommonPlatform.ArchSupported + + def GetTargetsSupported(self): + ''' return iterable of edk2 target tags supported by this build ''' + return CommonPlatform.TargetsSupported + + def GetRequiredSubmodules(self): + ''' return iterable containing RequiredSubmodule objects. + If no RequiredSubmodules return an empty iterable + ''' + rs =3D [] + + # intentionally declare this one with recursive false to avoid ove= rhead + rs.append(RequiredSubmodule( + "CryptoPkg/Library/OpensslLib/openssl", False)) + + # To avoid maintenance of this file for every new submodule + # lets just parse the .gitmodules and add each if not already in l= ist. + # The GetRequiredSubmodules is designed to allow a build to optimi= ze + # the desired submodules but it isn't necessary for this repositor= y. + result =3D io.StringIO() + ret =3D RunCmd("git", "config --file .gitmodules --get-regexp path= ", workingdir=3Dself.GetWorkspaceRoot(), outstream=3Dresult) + # Cmd output is expected to look like: + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Li= brary/OpensslLib/openssl + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley= -softfloat-3 + if ret =3D=3D 0: + for line in result.getvalue().splitlines(): + _, _, path =3D line.partition(" ") + if path is not None: + if path not in [x.path for x in rs]: + rs.append(RequiredSubmodule(path, True)) # add it = with recursive since we don't know + return rs + + def SetArchitectures(self, list_of_requested_architectures): + ''' Confirm the requests architecture list is valid and configure = SettingsManager + to run only the requested architectures. + + Raise Exception if a list_of_requested_architectures is not suppor= ted + ''' + unsupported =3D set(list_of_requested_architectures) - \ + set(self.GetArchitecturesSupported()) + if(len(unsupported) > 0): + errorString =3D ( + "Unsupported Architecture Requested: " + " ".join(unsuppor= ted)) + logging.critical(errorString) + raise Exception(errorString) + self.ActualArchitectures =3D list_of_requested_architectures + + def GetWorkspaceRoot(self): + ''' get WorkspacePath ''' + return CommonPlatform.WorkspaceRoot + + def GetActiveScopes(self): + ''' return tuple containing scopes that should be active for this = process ''' + + scopes =3D CommonPlatform.Scopes + ActualToolChainTag =3D shell_environment.GetBuildVars().GetValue("= TOOL_CHAIN_TAG", "") + + if GetHostInfo().os.upper() =3D=3D "LINUX" and ActualToolChainTag.= upper().startswith("GCC"): + if "AARCH64" in self.ActualArchitectures: + scopes +=3D ("gcc_aarch64_linux",) + if "ARM" in self.ActualArchitectures: + scopes +=3D ("gcc_arm_linux",) + return scopes + + def FilterPackagesToTest(self, changedFilesList: list, potentialPackag= esList: list) -> list: + ''' Filter other cases that this package should be built + based on changed files. This should cover things that can't + be detected as dependencies. ''' + build_these_packages =3D [] + possible_packages =3D potentialPackagesList.copy() + for f in changedFilesList: + # BaseTools files that might change the build + if "BaseTools" in f: + if os.path.splitext(f) not in [".txt", ".md"]: + build_these_packages =3D possible_packages + break + + # if the azure pipeline platform template file changed + if "platform-build-run-steps.yml" in f: + build_these_packages =3D possible_packages + break + + + return build_these_packages + + def GetPlatformDscAndConfig(self) -> tuple: + ''' If a platform desires to provide its DSC then Policy 4 will ev= aluate if + any of the changes will be built in the dsc. + + The tuple should be (, ) + ''' + return (os.path.join("ArmVirtPkg", "ArmVirtQemuKernel.dsc"), {}) + + + # ####################################################################= ################### # + # Actual Configuration for Platform Build = # + # ####################################################################= ################### # + + +class PlatformBuilder(UefiBuilder, BuildSettingsManager): + def __init__(self): + UefiBuilder.__init__(self) + + def AddCommandLineOptions(self, parserObj): + ''' Add command line options to the argparser ''' + parserObj.add_argument('-a', "--arch", dest=3D"build_arch", type= =3Dstr, default=3D"AARCH64", + help=3D"Optional - Architecture to build. = Default =3D AARCH64") + + def RetrieveCommandLineOptions(self, args): + ''' Retrieve command line options from the argparser ''' + + shell_environment.GetBuildVars().SetValue( + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine") + + shell_environment.GetBuildVars().SetValue( + "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemuKernel.dsc", "From C= mdLine") + + def GetWorkspaceRoot(self): + ''' get WorkspacePath ''' + return CommonPlatform.WorkspaceRoot + + def GetPackagesPath(self): + ''' Return a list of workspace relative paths that should be mappe= d as edk2 PackagesPath ''' + return () + + def GetActiveScopes(self): + ''' return tuple containing scopes that should be active for this = process ''' + scopes =3D CommonPlatform.Scopes + ActualToolChainTag =3D shell_environment.GetBuildVars().GetValue("= TOOL_CHAIN_TAG", "") + Arch =3D shell_environment.GetBuildVars().GetValue("TARGET_ARCH", = "") + + if GetHostInfo().os.upper() =3D=3D "LINUX" and ActualToolChainTag.= upper().startswith("GCC"): + if "AARCH64" =3D=3D Arch: + scopes +=3D ("gcc_aarch64_linux",) + elif "ARM" =3D=3D Arch: + scopes +=3D ("gcc_arm_linux",) + return scopes + + def GetName(self): + ''' Get the name of the repo, platform, or product being build ''' + ''' Used for naming the log file, among others ''' + # check the startup nsh flag and if set then rename the log file. + # this helps in CI so we don't overwrite the build log since runni= ng + # uses the stuart_build command. + if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "= FALSE") =3D=3D "TRUE"): + return "ArmVirtPkg_With_Run" + return "ArmVirtPkg" + + def GetLoggingLevel(self, loggerType): + ''' Get the logging level for a given type + base =3D=3D lowest logging level supported + con =3D=3D Screen logging + txt =3D=3D plain text file logging + md =3D=3D markdown file logging + ''' + return logging.DEBUG + + def SetPlatformEnv(self): + logging.debug("PlatformBuilder SetPlatformEnv") + self.env.SetValue("PRODUCT_NAME", "ArmVirtQemuKernel", "Platform H= ardcoded") + self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false") + self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false") + return 0 + + def PlatformPreBuild(self): + return 0 + + def PlatformPostBuild(self): + return 0 + + def FlashRomImage(self): + VirtualDrive =3D os.path.join(self.env.GetValue( + "BUILD_OUTPUT_BASE"), "VirtualDrive") + os.makedirs(VirtualDrive, exist_ok=3DTrue) + OutputPath_FV =3D os.path.join( + self.env.GetValue("BUILD_OUTPUT_BASE"), "FV") + Built_FV =3D os.path.join(OutputPath_FV, "QEMU_EFI.fd") + + # Unique Command and Args parameters per ARCH + if (self.env.GetValue("TARGET_ARCH").upper() =3D=3D "AARCH64"): + cmd =3D "qemu-system-aarch64" + args =3D "-M virt" + args +=3D " -cpu cortex-a57" = # emulate cpu + elif(self.env.GetValue("TARGET_ARCH").upper() =3D=3D "ARM"): + cmd =3D "qemu-system-arm" + args =3D "-M virt,highmem=3Doff" + args +=3D " -cpu cortex-a15" = # emulate cpu + else: + raise NotImplementedError() + + # Common Args + args +=3D " -kernel " + Built_FV = # path to fw + args +=3D " -m 1024" = # 1gb memory + # turn off network + args +=3D " -net none" + # Serial messages out + args +=3D " -serial stdio" + # Mount disk with startup.nsh + args +=3D f" -drive file=3Dfat:rw:{VirtualDrive},format=3Draw,medi= a=3Ddisk" + + # Conditional Args + if (self.env.GetValue("QEMU_HEADLESS").upper() =3D=3D "TRUE"): + args +=3D " -display none" # no graphics + + if (self.env.GetValue("MAKE_STARTUP_NSH").upper() =3D=3D "TRUE"): + f =3D open(os.path.join(VirtualDrive, "startup.nsh"), "w") + f.write("BOOT SUCCESS !!! \n") + # add commands here + f.write("reset -s\n") + f.close() + + ret =3D RunCmd(cmd, args) + + if ret =3D=3D 0xc0000005: + # for some reason getting a c0000005 on successful return + return 0 + + return ret --=20 2.39.0 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#98875): https://edk2.groups.io/g/devel/message/98875 Mute This Topic: https://groups.io/mt/96373699/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun May 19 04:56:04 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+98876+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+98876+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=kernel.org ARC-Seal: i=1; a=rsa-sha256; t=1674116531; cv=none; d=zohomail.com; s=zohoarc; b=jaAWfVUMLVseNY1fuh31mdoYOBVRKiwTjWXzH73aH7cEmgHr1Jy8dpg/nl4BDaFZTcbDGLWV2bH0hZTlZ9bOHBa7wzThYqBnSqOWWgzqy1uvgfdEdYPztWSOizBeJfF3Y4HeGrrF0imchwzEt5OWSV9LNgfubCMq6853pCyHTBg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1674116531; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=SPzPWB6NHf62DD3P5KCQE5+ZIVdMsriHTn0WWMT5YL4=; b=OdepD+ub20zeIH+EUYDxtddc1KX3o8kI/JVySs5IA3XO2pHqFGReFIbssiW5UL+INJsEC+GuuYPn4mWvda+OUPo5H4YyKu1Pk5PZKvLQbLhOww73PFEmBvydqriiTwjnP9+FQKs6hh0GJcNK0XcH8A0WNKW20ZWIIlsvZotvPro= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+98876+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1674116531551716.2278567157131; Thu, 19 Jan 2023 00:22:11 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id tzngYY1788612xUY5Svu5nho; Thu, 19 Jan 2023 00:22:10 -0800 X-Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mx.groups.io with SMTP id smtpd.web10.40755.1674116529720452793 for ; Thu, 19 Jan 2023 00:22:09 -0800 X-Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 23D6B61245; Thu, 19 Jan 2023 08:22:09 +0000 (UTC) X-Received: by smtp.kernel.org (Postfix) with ESMTPSA id 690FEC433F2; Thu, 19 Jan 2023 08:22:07 +0000 (UTC) From: "Ard Biesheuvel" To: devel@edk2.groups.io Cc: Ard Biesheuvel , Gerd Hoffmann , Michael Kubacki , Oliver Steffen Subject: [edk2-devel] [PATCH 3/3] ArmVirtPkg: CI: Perform build test of ArmVirtKvmTool Date: Thu, 19 Jan 2023 09:21:53 +0100 Message-Id: <20230119082153.1520196-4-ardb@kernel.org> In-Reply-To: <20230119082153.1520196-1-ardb@kernel.org> References: <20230119082153.1520196-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,ardb@kernel.org X-Gm-Message-State: 2UKuIISECmfHtRniAHfGXfzox1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1674116530; bh=jaG8wzutnaGdTb6EgbIJgwM9aPgplTbrXys7DTl1W8g=; h=Cc:Date:From:Reply-To:Subject:To; b=e05JakbSVGBNJ9Yyi+hNCqVPaXXVbRyOihylrKjhjUK2mqZhkFWS+gWJgBCxrf6+W+U rigztu+bYw/T8sLij1tBdFg3/OdeqwY5Wu3xG1FncUw7fHhiBrG810b0c8r/G3Mda5ZOM 4Fqnq/Cahh9OcWE+uhFHddpgKft+LMl04pk= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1674116533274100013 Content-Type: text/plain; charset="utf-8" Signed-off-by: Ard Biesheuvel --- ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 25 +++ ArmVirtPkg/PlatformCI/KvmToolBuild.py | 215 ++++++++++++++= ++++++ 2 files changed, 240 insertions(+) diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVir= tPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index f39d52aed541..304074686ec7 100644 --- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -115,6 +115,31 @@ jobs: Run.Flags: $(run_flags) Run: $(should_run) =20 + KVMTOOL_AARCH64_DEBUG: + Build.File: "$(package)/PlatformCI/KvmToolBuild.py" + Build.Arch: "AARCH64" + Build.Flags: "" + Build.Target: "DEBUG" + Run: false + KVMTOOL_AARCH64_RELEASE: + Build.File: "$(package)/PlatformCI/KvmToolBuild.py" + Build.Arch: "AARCH64" + Build.Flags: "" + Build.Target: "RELEASE" + Run: false + KVMTOOL_ARM_DEBUG: + Build.File: "$(package)/PlatformCI/KvmToolBuild.py" + Build.Arch: "ARM" + Build.Flags: "" + Build.Target: "DEBUG" + Run: false + KVMTOOL_ARM_RELEASE: + Build.File: "$(package)/PlatformCI/KvmToolBuild.py" + Build.Arch: "ARM" + Build.Flags: "" + Build.Target: "RELEASE" + Run: false + workspace: clean: all =20 diff --git a/ArmVirtPkg/PlatformCI/KvmToolBuild.py b/ArmVirtPkg/PlatformCI/= KvmToolBuild.py new file mode 100644 index 000000000000..c2cb42d9a219 --- /dev/null +++ b/ArmVirtPkg/PlatformCI/KvmToolBuild.py @@ -0,0 +1,215 @@ +# @file +# Script to Build ArmVirtPkg UEFI firmware +# +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: BSD-2-Clause-Patent +## +import os +import logging +import io + +from edk2toolext.environment import shell_environment +from edk2toolext.environment.uefi_build import UefiBuilder +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, Requir= edSubmodule +from edk2toolext.invocables.edk2_update import UpdateSettingsManager +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager +from edk2toollib.utility_functions import RunCmd +from edk2toollib.utility_functions import GetHostInfo + +# ########################################################################= ############### # +# Common Configuration = # +# ########################################################################= ############### # + + +class CommonPlatform(): + ''' Common settings for this platform. Define static data here and use + for the different parts of stuart + ''' + PackagesSupported =3D ("ArmVirtPkg",) + ArchSupported =3D ("AARCH64", "ARM") + TargetsSupported =3D ("DEBUG", "RELEASE") + Scopes =3D ('armvirt', 'edk2-build') + WorkspaceRoot =3D os.path.realpath(os.path.join( + os.path.dirname(os.path.abspath(__file__)), "..", "..")) + + # ####################################################################= ################### # + # Configuration for Update & Setup = # + # ####################################################################= ################### # + + +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalS= ettingsManager): + + def GetPackagesSupported(self): + ''' return iterable of edk2 packages supported by this build. + These should be edk2 workspace relative paths ''' + return CommonPlatform.PackagesSupported + + def GetArchitecturesSupported(self): + ''' return iterable of edk2 architectures supported by this build = ''' + return CommonPlatform.ArchSupported + + def GetTargetsSupported(self): + ''' return iterable of edk2 target tags supported by this build ''' + return CommonPlatform.TargetsSupported + + def GetRequiredSubmodules(self): + ''' return iterable containing RequiredSubmodule objects. + If no RequiredSubmodules return an empty iterable + ''' + rs =3D [] + + # intentionally declare this one with recursive false to avoid ove= rhead + rs.append(RequiredSubmodule( + "CryptoPkg/Library/OpensslLib/openssl", False)) + + # To avoid maintenance of this file for every new submodule + # lets just parse the .gitmodules and add each if not already in l= ist. + # The GetRequiredSubmodules is designed to allow a build to optimi= ze + # the desired submodules but it isn't necessary for this repositor= y. + result =3D io.StringIO() + ret =3D RunCmd("git", "config --file .gitmodules --get-regexp path= ", workingdir=3Dself.GetWorkspaceRoot(), outstream=3Dresult) + # Cmd output is expected to look like: + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Li= brary/OpensslLib/openssl + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley= -softfloat-3 + if ret =3D=3D 0: + for line in result.getvalue().splitlines(): + _, _, path =3D line.partition(" ") + if path is not None: + if path not in [x.path for x in rs]: + rs.append(RequiredSubmodule(path, True)) # add it = with recursive since we don't know + return rs + + def SetArchitectures(self, list_of_requested_architectures): + ''' Confirm the requests architecture list is valid and configure = SettingsManager + to run only the requested architectures. + + Raise Exception if a list_of_requested_architectures is not suppor= ted + ''' + unsupported =3D set(list_of_requested_architectures) - \ + set(self.GetArchitecturesSupported()) + if(len(unsupported) > 0): + errorString =3D ( + "Unsupported Architecture Requested: " + " ".join(unsuppor= ted)) + logging.critical(errorString) + raise Exception(errorString) + self.ActualArchitectures =3D list_of_requested_architectures + + def GetWorkspaceRoot(self): + ''' get WorkspacePath ''' + return CommonPlatform.WorkspaceRoot + + def GetActiveScopes(self): + ''' return tuple containing scopes that should be active for this = process ''' + + scopes =3D CommonPlatform.Scopes + ActualToolChainTag =3D shell_environment.GetBuildVars().GetValue("= TOOL_CHAIN_TAG", "") + + if GetHostInfo().os.upper() =3D=3D "LINUX" and ActualToolChainTag.= upper().startswith("GCC"): + if "AARCH64" in self.ActualArchitectures: + scopes +=3D ("gcc_aarch64_linux",) + if "ARM" in self.ActualArchitectures: + scopes +=3D ("gcc_arm_linux",) + return scopes + + def FilterPackagesToTest(self, changedFilesList: list, potentialPackag= esList: list) -> list: + ''' Filter other cases that this package should be built + based on changed files. This should cover things that can't + be detected as dependencies. ''' + build_these_packages =3D [] + possible_packages =3D potentialPackagesList.copy() + for f in changedFilesList: + # BaseTools files that might change the build + if "BaseTools" in f: + if os.path.splitext(f) not in [".txt", ".md"]: + build_these_packages =3D possible_packages + break + + # if the azure pipeline platform template file changed + if "platform-build-run-steps.yml" in f: + build_these_packages =3D possible_packages + break + + + return build_these_packages + + def GetPlatformDscAndConfig(self) -> tuple: + ''' If a platform desires to provide its DSC then Policy 4 will ev= aluate if + any of the changes will be built in the dsc. + + The tuple should be (, ) + ''' + return (os.path.join("ArmVirtPkg", "ArmVirtKvmTool.dsc"), {}) + + + # ####################################################################= ################### # + # Actual Configuration for Platform Build = # + # ####################################################################= ################### # + + +class PlatformBuilder(UefiBuilder, BuildSettingsManager): + def __init__(self): + UefiBuilder.__init__(self) + + def AddCommandLineOptions(self, parserObj): + ''' Add command line options to the argparser ''' + parserObj.add_argument('-a', "--arch", dest=3D"build_arch", type= =3Dstr, default=3D"AARCH64", + help=3D"Optional - Architecture to build. = Default =3D AARCH64") + + def RetrieveCommandLineOptions(self, args): + ''' Retrieve command line options from the argparser ''' + + shell_environment.GetBuildVars().SetValue( + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine") + + shell_environment.GetBuildVars().SetValue( + "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtKvmTool.dsc", "From CmdL= ine") + + def GetWorkspaceRoot(self): + ''' get WorkspacePath ''' + return CommonPlatform.WorkspaceRoot + + def GetPackagesPath(self): + ''' Return a list of workspace relative paths that should be mappe= d as edk2 PackagesPath ''' + return () + + def GetActiveScopes(self): + ''' return tuple containing scopes that should be active for this = process ''' + scopes =3D CommonPlatform.Scopes + ActualToolChainTag =3D shell_environment.GetBuildVars().GetValue("= TOOL_CHAIN_TAG", "") + Arch =3D shell_environment.GetBuildVars().GetValue("TARGET_ARCH", = "") + + if GetHostInfo().os.upper() =3D=3D "LINUX" and ActualToolChainTag.= upper().startswith("GCC"): + if "AARCH64" =3D=3D Arch: + scopes +=3D ("gcc_aarch64_linux",) + elif "ARM" =3D=3D Arch: + scopes +=3D ("gcc_arm_linux",) + return scopes + + def GetName(self): + ''' Get the name of the repo, platform, or product being build ''' + ''' Used for naming the log file, among others ''' + return "ArmVirtPkg" + + def GetLoggingLevel(self, loggerType): + ''' Get the logging level for a given type + base =3D=3D lowest logging level supported + con =3D=3D Screen logging + txt =3D=3D plain text file logging + md =3D=3D markdown file logging + ''' + return logging.DEBUG + + def SetPlatformEnv(self): + logging.debug("PlatformBuilder SetPlatformEnv") + self.env.SetValue("PRODUCT_NAME", "ArmVirtKvmTool", "Platform Hard= coded") + return 0 + + def PlatformPreBuild(self): + return 0 + + def PlatformPostBuild(self): + return 0 + + def FlashRomImage(self): + return 0 --=20 2.39.0 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#98876): https://edk2.groups.io/g/devel/message/98876 Mute This Topic: https://groups.io/mt/96373700/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-