From nobody Mon Feb 9 07:22:30 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+45630+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+45630+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1565806330; cv=none; d=zoho.com; s=zohoarc; b=VkDZAu7G5Jr6zy7nGe2YzZVCSkC4fEpi9FWKj5i41MJBj7B3G+zbcaHSJ80/4MOdNAsfSZUU/tKYgRpM74yIueIIOQNFDQlUUlwSLHwDkGZcpFpdUmsAaB3nLSFIET4lKmzmUCpLimjFDqhFV6lfU8zmjW3VUfRO+YI5K7aBMjs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1565806330; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To:ARC-Authentication-Results; bh=DQvpCBgwh0WJXv6OdEQg4Y5z3eJsfhcBO2/k72OZnFg=; b=WNBc8+DLHvHtk+eE7kBQSFF/tUWGS91bdD9uc7+gJMc72fw+4UaYCpHCvicOubMMe/YPjt897OE8F2eRaeJWgUNh1tloDSuQhaehkwbkeAIgyCDHZg0lG5Vs4+YdF37DQDGDVz2oGTbFRXmaVUUDWjLWh6bzTWTILJ1mOArWKSs= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+45630+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1565806330090860.7553401607803; Wed, 14 Aug 2019 11:12:10 -0700 (PDT) Return-Path: X-Received: from mga18.intel.com (mga18.intel.com []) by groups.io with SMTP; Wed, 14 Aug 2019 11:12:09 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Aug 2019 11:11:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,386,1559545200"; d="scan'208";a="260581455" X-Received: from jshi19-mobl.ccr.corp.intel.com ([10.254.213.161]) by orsmga001.jf.intel.com with ESMTP; 14 Aug 2019 11:11:55 -0700 From: "Steven Shi" To: devel@edk2.groups.io Cc: liming.gao@intel.com, bob.c.feng@intel.com, christian.rodriguez@intel.com, michael.johnson@intel.com, "Shi, Steven" Subject: [edk2-devel] [PATCH v4 3/5] BaseTools: Change the [Arch][Name] module key in Build cache Date: Thu, 15 Aug 2019 02:11:28 +0800 Message-Id: <20190814181130.8020-4-steven.shi@intel.com> In-Reply-To: <20190814181130.8020-1-steven.shi@intel.com> References: <20190814181130.8020-1-steven.shi@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,steven.shi@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1565806329; bh=9jJzATGkjR/U4v1/rN7BLQjaUrb/WzKfuXnqJdf0ZII=; h=Cc:Date:From:Reply-To:Subject:To; b=HpNyJglYTBvKA9xxuwOYrzM7IBX6H6mTb250VV2cc1D+geItXA6pdsYO8xT02dnWW5E RD+zoYpMEhkjTtDZ/vDRVtJXsUEJQM6Glg4t2MHqbgdX8fqiHbof4wSalCD94VAqdQeHF A5slBQT8d2buPmU7JmLz5Y70P/JmVGLIMFE= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D1951 Current build cache use the module's [self.Arch][self.Name] info as the ModuleAutoGen object key in hash list and dictionary. The [self.Arch][self.Name] is not safe as the module key because there could be two modules with same module name and arch name in one platform. E.g. A platform can override a module or library instance in another different path, the overriding module can has the same module name and arch name as the original one. Directly use the ModuleAutoGen obj self as the key, because the obj __hash__ and __repr__ attributes already contain the full path and arch name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 +----- BaseTools/Source/Python/build/build.py | 48 ++++++++++++++++++++------= ---------------------- 2 files changed, 21 insertions(+), 33 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/= Python/AutoGen/GenMake.py index ce047e7f64..3e6ec4085c 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -959,16 +959,12 @@ cleanlib: # Keep the file to be checked headerFileDependencySet.add(aFileName) =20 - # Ensure that gModuleBuildTracking has been initialized per archit= ecture - if self._AutoGenObject.Arch not in GlobalData.gModuleBuildTracking: - GlobalData.gModuleBuildTracking[self._AutoGenObject.Arch] =3D = dict() - # Check if a module dependency header file is missing from the mod= ule's MetaFile for aFile in headerFileDependencySet: if aFile in headerFilesInMetaFileSet: continue if GlobalData.gUseHashCache: - GlobalData.gModuleBuildTracking[self._AutoGenObject.Arch][= self._AutoGenObject] =3D 'FAIL_METAFILE' + GlobalData.gModuleBuildTracking[self._AutoGenObject] =3D '= FAIL_METAFILE' EdkLogger.warn("build","Module MetaFile [Sources] is missing l= ocal header!", ExtraData =3D "Local Header: " + aFile + " not fou= nd in " + self._AutoGenObject.MetaFile.Path ) diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Pyth= on/build/build.py index d7c817b95c..299fa64311 100755 --- a/BaseTools/Source/Python/build/build.py +++ b/BaseTools/Source/Python/build/build.py @@ -630,12 +630,11 @@ class BuildTask: =20 # Set the value used by hash invalidation flow in GlobalData.gModu= leBuildTracking to 'SUCCESS' # If Module or Lib is being tracked, it did not fail header check = test, and built successfully - if (self.BuildItem.BuildObject.Arch in GlobalData.gModuleBuildTrac= king and - self.BuildItem.BuildObject in GlobalData.gModuleBuildTracking[s= elf.BuildItem.BuildObject.Arch] and - GlobalData.gModuleBuildTracking[self.BuildItem.BuildObject.Arch= ][self.BuildItem.BuildObject] !=3D 'FAIL_METAFILE' and + if (self.BuildItem.BuildObject in GlobalData.gModuleBuildTracking = and + GlobalData.gModuleBuildTracking[self.BuildItem.BuildObject] != =3D 'FAIL_METAFILE' and not BuildTask._ErrorFlag.isSet() ): - GlobalData.gModuleBuildTracking[self.BuildItem.BuildObject.Arc= h][self.BuildItem.BuildObject] =3D 'SUCCESS' + GlobalData.gModuleBuildTracking[self.BuildItem.BuildObject] = =3D 'SUCCESS' =20 # indicate there's a thread is available for another build task BuildTask._RunningQueueLock.acquire() @@ -1171,25 +1170,24 @@ class Build(): return =20 # GlobalData.gModuleBuildTracking contains only modules or libs th= at cannot be skipped by hash - for moduleAutoGenObjArch in GlobalData.gModuleBuildTracking.keys(): - for moduleAutoGenObj in GlobalData.gModuleBuildTracking[module= AutoGenObjArch].keys(): - # Skip invalidating for Successful Module/Lib builds - if GlobalData.gModuleBuildTracking[moduleAutoGenObjArch][m= oduleAutoGenObj] =3D=3D 'SUCCESS': - continue + for Ma in GlobalData.gModuleBuildTracking: + # Skip invalidating for Successful Module/Lib builds + if GlobalData.gModuleBuildTracking[Ma] =3D=3D 'SUCCESS': + continue =20 - # The module failed to build, failed to start building, or= failed the header check test from this point on + # The module failed to build, failed to start building, or fai= led the header check test from this point on =20 - # Remove .hash from build - ModuleHashFile =3D os.path.join(moduleAutoGenObj.BuildDir,= moduleAutoGenObj.Name + ".hash") - if os.path.exists(ModuleHashFile): - os.remove(ModuleHashFile) + # Remove .hash from build + ModuleHashFile =3D os.path.join(Ma.BuildDir, Ma.Name + ".hash") + if os.path.exists(ModuleHashFile): + os.remove(ModuleHashFile) =20 - # Remove .hash file from cache - if GlobalData.gBinCacheDest: - FileDir =3D os.path.join(GlobalData.gBinCacheDest, mod= uleAutoGenObj.Arch, moduleAutoGenObj.SourceDir, moduleAutoGenObj.MetaFile.B= aseName) - HashFile =3D os.path.join(FileDir, moduleAutoGenObj.Na= me + '.hash') - if os.path.exists(HashFile): - os.remove(HashFile) + # Remove .hash file from cache + if GlobalData.gBinCacheDest: + FileDir =3D os.path.join(GlobalData.gBinCacheDest, Ma.Plat= formInfo.OutputDir, Ma.BuildTarget + "_" + Ma.ToolChain, Ma.Arch, Ma.Source= Dir, Ma.MetaFile.BaseName) + HashFile =3D os.path.join(FileDir, Ma.Name + '.hash') + if os.path.exists(HashFile): + os.remove(HashFile) =20 ## Build a module or platform # @@ -1889,10 +1887,7 @@ class Build(): =20 self.BuildModules.append(Ma) # Initialize all modules in tracking to 'FAIL' - if Ma.Arch not in GlobalData.gModuleBuildTrack= ing: - GlobalData.gModuleBuildTracking[Ma.Arch] = =3D dict() - if Ma not in GlobalData.gModuleBuildTracking[M= a.Arch]: - GlobalData.gModuleBuildTracking[Ma.Arch][M= a] =3D 'FAIL' + GlobalData.gModuleBuildTracking[Ma] =3D 'FAIL' self.AutoGenTime +=3D int(round((time.time() - AutoGen= Start))) MakeStart =3D time.time() for Ma in self.BuildModules: @@ -2075,10 +2070,7 @@ class Build(): PcdMaList.append(Ma) TotalModules.append(Ma) # Initialize all modules in tracking to 'FAIL' - if Ma.Arch not in GlobalData.gModuleBuildTracking: - GlobalData.gModuleBuildTracking[Ma.Arch] =3D d= ict() - if Ma not in GlobalData.gModuleBuildTracking[Ma.Ar= ch]: - GlobalData.gModuleBuildTracking[Ma.Arch][Ma] = =3D 'FAIL' + GlobalData.gModuleBuildTracking[Ma] =3D 'FAIL' =20 mqueue =3D mp.Queue() for m in Pa.GetAllModuleInfo: --=20 2.17.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#45630): https://edk2.groups.io/g/devel/message/45630 Mute This Topic: https://groups.io/mt/32866764/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-