From nobody Mon Feb 9 09:09:03 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+45629+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+45629+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1565806329; cv=none; d=zoho.com; s=zohoarc; b=G258qnx1bvjIYgcLlveOajCrdsDd4YiwxlP7eEd+O3ZxrMFzW+AmBTvRDELvBzLY6jVRAXHX4wHh5toSrO72Cy2tNpNFGPp28S2pYVHch9xjiCHsR0Hvuoeitx5ojR0gXcGM0ItM42iDfyeWLbq/8hu9dlSKazkoRDV6PYRwJ9c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1565806329; h=Cc:Date:From:In-Reply-To:List-Id:List-Unsubscribe:Message-ID:Reply-To:References:Sender:Subject:To:ARC-Authentication-Results; bh=zrpoBUrcpjk77Zsl+SxXy9LgqPxan/2SgS+a27ausHo=; b=EEJrYUwU4Ay5TZNMZmVvOCGqc/LkksFBRyTf/BYcQvqcRJ8pcDGD1tWXPRFLbeYCmjLAhZ47J1O98++lCTUAN3CukJWS67YLCNTWJ+Af0AxTU68Hf5HUY9LCygEUeENHPSAz8OLL9FlIrFC8PGd7hOTcbWqmrsRneR+R/wkehyg= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+45629+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1565806329640564.1980902395688; Wed, 14 Aug 2019 11:12:09 -0700 (PDT) Return-Path: X-Received: from mga18.intel.com (mga18.intel.com []) by groups.io with SMTP; Wed, 14 Aug 2019 11:12:08 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Aug 2019 11:11:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,386,1559545200"; d="scan'208";a="260581443" X-Received: from jshi19-mobl.ccr.corp.intel.com ([10.254.213.161]) by orsmga001.jf.intel.com with ESMTP; 14 Aug 2019 11:11:53 -0700 From: "Steven Shi" To: devel@edk2.groups.io Cc: liming.gao@intel.com, bob.c.feng@intel.com, christian.rodriguez@intel.com, michael.johnson@intel.com, "Shi, Steven" Subject: [edk2-devel] [PATCH v4 2/5] BaseTools: Print first cache missing file for build cachle Date: Thu, 15 Aug 2019 02:11:27 +0800 Message-Id: <20190814181130.8020-3-steven.shi@intel.com> In-Reply-To: <20190814181130.8020-1-steven.shi@intel.com> References: <20190814181130.8020-1-steven.shi@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,steven.shi@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1565806329; bh=vaU4JlPQXtGHFdtmDtF/6gMqUUiXXiQRYqeF7Jbhvvk=; h=Cc:Date:From:Reply-To:Subject:To; b=pajmD/T56y1wvWwYAoNgo79/YIzpTCqv6albPMwUEDTlPDyjuKYrFdAEW08kroTxfHi UmARJYWAOssQmBpEZsQ7eTyEvC8xa9qquU7c5hnfdwoCqo+yUwufrZ9civIVUXeYDu5pj 4RAiLzmiYqdeKLxJ62I9ITw+eeWEIfkCDFk= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: "Shi, Steven" BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao Cc: Bob Feng Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGenWorker.py | 2 ++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 76 ++++++++++++++++++++= ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+) diff --git a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py b/BaseTools/S= ource/Python/AutoGen/AutoGenWorker.py index a84ed46f2e..30d2f96fc7 100755 --- a/BaseTools/Source/Python/AutoGen/AutoGenWorker.py +++ b/BaseTools/Source/Python/AutoGen/AutoGenWorker.py @@ -246,6 +246,8 @@ class AutoGenWorkerInProcess(mp.Process): Ma.GenMakeHash(GlobalData.gCacheIR) if Ma.CanSkipbyMakeCache(GlobalData.gCacheIR): continue + else: + Ma.PrintFirstMakeCacheMissFile(GlobalData.gCacheIR) except Empty: pass except: diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/S= ource/Python/AutoGen/ModuleAutoGen.py index 613b0d2fb8..6db3b47a91 100755 --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py @@ -2379,6 +2379,82 @@ class ModuleAutoGen(AutoGen): print("[cache hit]: checkpoint_Makefile:", self.MetaFile.Path, sel= f.Arch) return True =20 + ## Show the first file name which causes cache miss + def PrintFirstMakeCacheMissFile(self, gDict): + if not GlobalData.gBinCacheSource: + return + + # skip binary module + if self.IsBinaryModule: + return + + if not (self.MetaFile.Path, self.Arch) in gDict: + return + + # Only print cache miss file for the MakeCache not hit module + if gDict[(self.MetaFile.Path, self.Arch)].MakeCacheHit: + return + + if not gDict[(self.MetaFile.Path, self.Arch)].MakeHashChain: + EdkLogger.quiet("[cache insight]: MakeHashChain is missing for= : %s[%s]" % (self.MetaFile.Path, self.Arch)) + return + + # Find the cache dir name through the .ModuleHashPair file info + FileDir =3D path.join(GlobalData.gBinCacheSource, self.PlatformInf= o.OutputDir, self.BuildTarget + "_" + self.ToolChain, self.Arch, self.Sourc= eDir, self.MetaFile.BaseName) + + ModuleHashPairList =3D [] # tuple list: [tuple(PreMakefileHash, Ma= keHash)] + ModuleHashPair =3D path.join(FileDir, self.Name + ".ModuleHashPair= ") + if not os.path.exists(ModuleHashPair): + EdkLogger.quiet("[cache insight]: Cannot find ModuleHashPair f= ile for module: %s[%s]" % (self.MetaFile.Path, self.Arch)) + return + + try: + f =3D open(ModuleHashPair, 'r') + ModuleHashPairList =3D json.load(f) + f.close() + except: + EdkLogger.quiet("[cache insight]: Cannot load ModuleHashPair f= ile for module: %s[%s]" % (self.MetaFile.Path, self.Arch)) + return + + MakeHashSet =3D set() + for idx, (PreMakefileHash, MakeHash) in enumerate (ModuleHashPairL= ist): + TargetHashDir =3D path.join(FileDir, str(MakeHash)) + if os.path.exists(TargetHashDir): + MakeHashSet.add(MakeHash) + if not MakeHashSet: + EdkLogger.quiet("[cache insight]: Cannot find valid cache dir = for module: %s[%s]" % (self.MetaFile.Path, self.Arch)) + return + + TargetHash =3D list(MakeHashSet)[0] + TargetHashDir =3D path.join(FileDir, str(TargetHash)) + if len(MakeHashSet) > 1 : + EdkLogger.quiet("[cache insight]: found multiple cache dirs fo= r this module, random select dir '%s' to search the first cache miss file: = %s[%s]" % (TargetHash, self.MetaFile.Path, self.Arch)) + + ListFile =3D path.join(TargetHashDir, self.Name + '.MakeHashChain') + if os.path.exists(ListFile): + try: + f =3D open(ListFile, 'r') + CachedList =3D json.load(f) + f.close() + except: + EdkLogger.quiet("[cache insight]: Cannot load MakeHashChai= n file: %s" % ListFile) + return + else: + EdkLogger.quiet("[cache insight]: Cannot find MakeHashChain fi= le: %s" % ListFile) + return + + CurrentList =3D gDict[(self.MetaFile.Path, self.Arch)].MakeHashCha= in + for idx, (file, hash) in enumerate (CurrentList): + (filecached, hashcached) =3D CachedList[idx] + if file !=3D filecached: + EdkLogger.quiet("[cache insight]: first different file in = %s[%s] is %s, the cached one is %s" % (self.MetaFile.Path, self.Arch, file,= filecached)) + break + if hash !=3D hashcached: + EdkLogger.quiet("[cache insight]: first cache miss file in= %s[%s] is %s" % (self.MetaFile.Path, self.Arch, file)) + break + + return True + ## Decide whether we can skip the ModuleAutoGen process def CanSkipbyCache(self, gDict): # Hashing feature is off --=20 2.17.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#45629): https://edk2.groups.io/g/devel/message/45629 Mute This Topic: https://groups.io/mt/32866763/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-