From nobody Tue May 7 22:49:48 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+41470+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+41470+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1559033515; cv=none; d=zoho.com; s=zohoarc; b=Vydq8jFgN5gKo+hBLHYWJlTp/uOkorcuiQCmMxnFM39VnRoTrMef5LzkyYtTjkZRG6KpnDbket193k/+Bo/CiMSpobJ4PfTT5U1VfeHkGRRHXoVUi8z+wDXpomLH5026y2l1S2bnG7HRKUEoksyoaft3M3N/0EFVIlWDt5LOe0Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1559033515; h=Cc:Date:From:List-Id:List-Unsubscribe:Message-ID:Reply-To:Sender:Subject:To:ARC-Authentication-Results; bh=N+H4TlLoN5bUKTyivlEhFEh+aR15X2cpoBtS+qJMHsI=; b=K8SqCA24896obbz+avyKym2boiZJLZcRVumg+GTrQxOrB1z3hOlSJUBSXsOi7jps3fnUpKkAp041rTo2TH7VPuVnbvxAR4iLbyNWA4QImIH5DXvNrIOgEkVYz9gfM5Dv4QvYS5tE7SQ68aN+yH+f+Xy64JHH+FmN14vTog1P6mE= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+41470+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1559033515513802.756314467084; Tue, 28 May 2019 01:51:55 -0700 (PDT) Return-Path: X-Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by groups.io with SMTP; Tue, 28 May 2019 01:51:54 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 May 2019 01:51:53 -0700 X-ExtLoop1: 1 X-Received: from jshi19-mobl.ccr.corp.intel.com ([10.254.209.247]) by fmsmga005.fm.intel.com with ESMTP; 28 May 2019 01:51:49 -0700 From: "Steven Shi" To: devel@edk2.groups.io Cc: liming.gao@intel.com, bob.c.feng@intel.com, christian.rodriguez@intel.com, zhijux.fan@intel.com Subject: [edk2-devel] [PATCH] BaseTools:Extend the binary cache to support library cache Date: Tue, 28 May 2019 16:51:39 +0800 Message-Id: <20190528085139.14716-1-steven.shi@intel.com> Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,steven.shi@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1559033515; bh=7vDBDll5mCDybR5fgGAJ+m9tACqL1U3kFteqTc5F/ps=; h=Cc:Date:From:Reply-To:Subject:To; b=Ww43QPaOh/09dLcwnRt8LFH5yqP+JBej+6V/fvjHdrEDCtgiASt46rOkonh2uamjaWo siO00aLlDO9uKI7/HL/YFSuo+lWe58A8AjlusNJC+nwLtbH6WbNFz3VHWF0spgQR9mfFn h5VzXbe5Zx8xNtbvxFF0ANJMfM7HI88Hk3o= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" https://bugzilla.tianocore.org/show_bug.cgi?id=3D1797 Current binary cache doesn't support to save and restore the library module. If a driver module cache miss happen, all its dependency library modules need rebuild which is very time-consuming. This patch is to entend the binary cache to support library. Cc: Liming Gao Cc: Bob Feng Cc: Christian Rodriguez Signed-off-by: Steven Shi --- BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++++++--- BaseTools/Source/Python/build/build.py | 3 ++- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/= Python/AutoGen/AutoGen.py index a5bef4f7c6..aeb63f52c5 100644 --- a/BaseTools/Source/Python/AutoGen/AutoGen.py +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py @@ -2578,7 +2578,7 @@ class ModuleAutoGen(AutoGen): self.AutoGenDepSet =3D set() self.ReferenceModules =3D [] self.ConstPcd =3D {} - + self.CacheRestored =3D False =20 def __repr__(self): return "%s [%s]" % (self.MetaFile, self.Arch) @@ -3906,6 +3906,12 @@ class ModuleAutoGen(AutoGen): ModuleFile =3D path.join(self.OutputDir, self.Name + '.inf') if os.path.exists(ModuleFile): shutil.copy2(ModuleFile, FileDir) + else: + OutputDir =3D self.OutputDir.replace('\\', '/').strip('/') + DebugDir =3D self.DebugDir.replace('\\', '/').strip('/') + for Item in self.CodaTargetList: + File =3D Item.Target.Path.replace('\\', '/').strip('/').re= place(DebugDir, '').replace(OutputDir, '').strip('/') + self.OutputFile.add(File) if not self.OutputFile: Ma =3D self.BuildDatabase[self.MetaFile, self.Arch, self.Build= Target, self.ToolChain] self.OutputFile =3D Ma.Binaries @@ -3949,6 +3955,7 @@ class ModuleAutoGen(AutoGen): destination_dir =3D os.path.dirname(destin= ation_file) CreateDirectory(destination_dir) shutil.copy2(File, destination_dir) + self.CacheRestored =3D True if self.Name =3D=3D "PcdPeim" or self.Name =3D=3D "Pcd= Dxe": CreatePcdDatabaseCode(self, TemplateString(), Temp= lateString()) return True @@ -3987,7 +3994,9 @@ class ModuleAutoGen(AutoGen): self.GenFfsList =3D GenFfsList if not self.IsLibrary and CreateLibraryMakeFile: for LibraryAutoGen in self.LibraryAutoGenList: - LibraryAutoGen.CreateMakeFile() + # Only create makefile for libraries which have not been r= estored + if not LibraryAutoGen.CacheRestored: + LibraryAutoGen.CreateMakeFile() =20 if self.CanSkip(): return @@ -4030,7 +4039,9 @@ class ModuleAutoGen(AutoGen): =20 if not self.IsLibrary and CreateLibraryCodeFile: for LibraryAutoGen in self.LibraryAutoGenList: - LibraryAutoGen.CreateCodeFile() + # Only create autogen code for libraries which have not be= en restored + if not LibraryAutoGen.CacheRestored: + LibraryAutoGen.CreateCodeFile() =20 if self.CanSkip(): return diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Pyth= on/build/build.py index 80ceb98310..f1f4c07980 100644 --- a/BaseTools/Source/Python/build/build.py +++ b/BaseTools/Source/Python/build/build.py @@ -341,7 +341,8 @@ class ModuleMakeUnit(BuildUnit): # @param Target The build target name, one of gSupportedTarget # def __init__(self, Obj, Target): - Dependency =3D [ModuleMakeUnit(La, Target) for La in Obj.LibraryAu= toGenList] + # Skip the dependency modules which are already restored from cache + Dependency =3D [ModuleMakeUnit(La, Target) for La in Obj.LibraryAu= toGenList if not La.CacheRestored] BuildUnit.__init__(self, Obj, Obj.BuildCommand, Target, Dependency= , Obj.MakeFileDir) if Target in [None, "", "all"]: self.Target =3D "tbuild" --=20 2.17.1.windows.2 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#41470): https://edk2.groups.io/g/devel/message/41470 Mute This Topic: https://groups.io/mt/31819658/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-