[edk2-devel] [PATCH] BaseTools: Support more file types in build cache

Steven Shi posted 1 patch 4 years, 7 months ago
Failed in applying to current master (apply log)
BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 34 ++++++++++++++--------------------
1 file changed, 14 insertions(+), 20 deletions(-)
[edk2-devel] [PATCH] BaseTools: Support more file types in build cache
Posted by Steven Shi 4 years, 7 months ago
From: "Shi, Steven" <steven.shi@intel.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1958

Current build cache does not store and restore all
types file of Hii/vfr, version and dpx. This patch
adds more file types to support them in build cache.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 34 ++++++++++++++--------------------
 1 file changed, 14 insertions(+), 20 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index 2cd0d3859e..4f17dee9a4 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -1267,29 +1267,30 @@ class ModuleAutoGen(AutoGen):
     @cached_property
     def OutputFile(self):
         retVal = set()
+
         OutputDir = self.OutputDir.replace('\\', '/').strip('/')
         DebugDir = self.DebugDir.replace('\\', '/').strip('/')
-        FfsOutputDir = self.FfsOutputDir.replace('\\', '/').rstrip('/')
         for Item in self.CodaTargetList:
             File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
-            retVal.add(File)
-        if self.DepexGenerated:
-            retVal.add(self.Name + '.depex')
+            NewFile = path.join(self.OutputDir, File)
+            retVal.add(NewFile)
 
         Bin = self._GenOffsetBin()
         if Bin:
-            retVal.add(Bin)
+            NewFile = path.join(self.OutputDir, Bin)
+            retVal.add(NewFile)
 
-        for Root, Dirs, Files in os.walk(OutputDir):
+        for Root, Dirs, Files in os.walk(self.OutputDir):
             for File in Files:
-                if File.lower().endswith('.pdb'):
-                    retVal.add(File)
+                # lib file is already added through above CodaTargetList, skip it here
+                if not (File.lower().endswith('.obj') or File.lower().endswith('.lib')):
+                    NewFile = path.join(self.OutputDir, File)
+                    retVal.add(NewFile)
 
-        for Root, Dirs, Files in os.walk(FfsOutputDir):
+        for Root, Dirs, Files in os.walk(self.FfsOutputDir):
             for File in Files:
-                if File.lower().endswith('.ffs') or File.lower().endswith('.offset') or File.lower().endswith('.raw') \
-                    or File.lower().endswith('.raw.txt'):
-                    retVal.add(File)
+                NewFile = path.join(self.FfsOutputDir, File)
+                retVal.add(NewFile)
 
         return retVal
 
@@ -1659,15 +1660,8 @@ class ModuleAutoGen(AutoGen):
             Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget, self.ToolChain]
             self.OutputFile = Ma.Binaries
         for File in self.OutputFile:
-            File = str(File)
-            if not os.path.isabs(File):
-                NewFile = os.path.join(self.OutputDir, File)
-                if not os.path.exists(NewFile):
-                    NewFile = os.path.join(self.FfsOutputDir, File)
-                File = NewFile
             if os.path.exists(File):
-                if File.lower().endswith('.ffs') or File.lower().endswith('.offset') or File.lower().endswith('.raw') \
-                    or File.lower().endswith('.raw.txt'):
+                if File.startswith(os.path.abspath(self.FfsOutputDir)+os.sep):
                     self.CacheCopyFile(FfsDir, self.FfsOutputDir, File)
                 else:
                     self.CacheCopyFile(FileDir, self.OutputDir, File)
-- 
2.17.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#46409): https://edk2.groups.io/g/devel/message/46409
Mute This Topic: https://groups.io/mt/33043035/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH] BaseTools: Support more file types in build cache
Posted by Liming Gao 4 years, 6 months ago
Reviewed-by: Liming Gao <liming.gao@intel.com>

> -----Original Message-----
> From: Shi, Steven
> Sent: Tuesday, August 27, 2019 1:17 PM
> To: devel@edk2.groups.io
> Cc: Gao, Liming <liming.gao@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Shi, Steven <steven.shi@intel.com>
> Subject: [PATCH] BaseTools: Support more file types in build cache
> 
> From: "Shi, Steven" <steven.shi@intel.com>
> 
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1958
> 
> Current build cache does not store and restore all
> types file of Hii/vfr, version and dpx. This patch
> adds more file types to support them in build cache.
> 
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Signed-off-by: Steven Shi <steven.shi@intel.com>
> ---
>  BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 34 ++++++++++++++--------------------
>  1 file changed, 14 insertions(+), 20 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> index 2cd0d3859e..4f17dee9a4 100755
> --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> @@ -1267,29 +1267,30 @@ class ModuleAutoGen(AutoGen):
>      @cached_property
>      def OutputFile(self):
>          retVal = set()
> +
>          OutputDir = self.OutputDir.replace('\\', '/').strip('/')
>          DebugDir = self.DebugDir.replace('\\', '/').strip('/')
> -        FfsOutputDir = self.FfsOutputDir.replace('\\', '/').rstrip('/')
>          for Item in self.CodaTargetList:
>              File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
> -            retVal.add(File)
> -        if self.DepexGenerated:
> -            retVal.add(self.Name + '.depex')
> +            NewFile = path.join(self.OutputDir, File)
> +            retVal.add(NewFile)
> 
>          Bin = self._GenOffsetBin()
>          if Bin:
> -            retVal.add(Bin)
> +            NewFile = path.join(self.OutputDir, Bin)
> +            retVal.add(NewFile)
> 
> -        for Root, Dirs, Files in os.walk(OutputDir):
> +        for Root, Dirs, Files in os.walk(self.OutputDir):
>              for File in Files:
> -                if File.lower().endswith('.pdb'):
> -                    retVal.add(File)
> +                # lib file is already added through above CodaTargetList, skip it here
> +                if not (File.lower().endswith('.obj') or File.lower().endswith('.lib')):
> +                    NewFile = path.join(self.OutputDir, File)
> +                    retVal.add(NewFile)
> 
> -        for Root, Dirs, Files in os.walk(FfsOutputDir):
> +        for Root, Dirs, Files in os.walk(self.FfsOutputDir):
>              for File in Files:
> -                if File.lower().endswith('.ffs') or File.lower().endswith('.offset') or File.lower().endswith('.raw') \
> -                    or File.lower().endswith('.raw.txt'):
> -                    retVal.add(File)
> +                NewFile = path.join(self.FfsOutputDir, File)
> +                retVal.add(NewFile)
> 
>          return retVal
> 
> @@ -1659,15 +1660,8 @@ class ModuleAutoGen(AutoGen):
>              Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget, self.ToolChain]
>              self.OutputFile = Ma.Binaries
>          for File in self.OutputFile:
> -            File = str(File)
> -            if not os.path.isabs(File):
> -                NewFile = os.path.join(self.OutputDir, File)
> -                if not os.path.exists(NewFile):
> -                    NewFile = os.path.join(self.FfsOutputDir, File)
> -                File = NewFile
>              if os.path.exists(File):
> -                if File.lower().endswith('.ffs') or File.lower().endswith('.offset') or File.lower().endswith('.raw') \
> -                    or File.lower().endswith('.raw.txt'):
> +                if File.startswith(os.path.abspath(self.FfsOutputDir)+os.sep):
>                      self.CacheCopyFile(FfsDir, self.FfsOutputDir, File)
>                  else:
>                      self.CacheCopyFile(FileDir, self.OutputDir, File)
> --
> 2.17.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#46653): https://edk2.groups.io/g/devel/message/46653
Mute This Topic: https://groups.io/mt/33043035/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-