[edk2-devel] [Patch V3] BaseTools: Fix checking for Sources section in INF file

Christian Rodriguez posted 1 patch 4 years, 8 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/edk2 tags/patchew/20190812153211.1791-1-christian.rodriguez@intel.com
BaseTools/Source/Python/AutoGen/GenMake.py       | 25 ++++++++++++++++++++++---
BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 15 +++++++++++++++
2 files changed, 37 insertions(+), 3 deletions(-)
[edk2-devel] [Patch V3] BaseTools: Fix checking for Sources section in INF file
Posted by Christian Rodriguez 4 years, 8 months ago
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804

The check to see if [Sources] section lists all the header type
files of a module is missing the exclusion of source files that
fall under the scope of Package includes. This change adds the
exclusions.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
---
 BaseTools/Source/Python/AutoGen/GenMake.py       | 25 ++++++++++++++++++++++---
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 15 +++++++++++++++
 2 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index 5802ae5ae4..499ef82aea 100644
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -906,8 +906,14 @@ cleanlib:
                                     self._AutoGenObject.IncludePathList + self._AutoGenObject.BuildOptionIncPathList
                                     )
 
+        # Get a set of unique package includes from MetaFile
+        parentMetaFileIncludes = set()
+        for aInclude in self._AutoGenObject.PackageIncludePathList:
+            aIncludeName = str(aInclude)
+            parentMetaFileIncludes.add(aIncludeName.lower())
+
         # Check if header files are listed in metafile
-        # Get a list of unique module header source files from MetaFile
+        # Get a set of unique module header source files from MetaFile
         headerFilesInMetaFileSet = set()
         for aFile in self._AutoGenObject.SourceFileList:
             aFileName = str(aFile)
@@ -915,24 +921,37 @@ cleanlib:
                 continue
             headerFilesInMetaFileSet.add(aFileName.lower())
 
-        # Get a list of unique module autogen files
+        # Get a set of unique module autogen files
         localAutoGenFileSet = set()
         for aFile in self._AutoGenObject.AutoGenFileList:
             localAutoGenFileSet.add(str(aFile).lower())
 
-        # Get a list of unique module dependency header files
+        # Get a set of unique module dependency header files
         # Exclude autogen files and files not in the source directory
+        # and files that are under the package include list
         headerFileDependencySet = set()
         localSourceDir = str(self._AutoGenObject.SourceDir).lower()
         for Dependency in FileDependencyDict.values():
             for aFile in Dependency:
                 aFileName = str(aFile).lower()
+                # Exclude non-header files
                 if not aFileName.endswith('.h'):
                     continue
+                # Exclude autogen files
                 if aFileName in localAutoGenFileSet:
                     continue
+                # Exclude include out of local scope
                 if localSourceDir not in aFileName:
                     continue
+                # Exclude files covered by package includes
+                pathNeeded = True
+                for aIncludePath in parentMetaFileIncludes:
+                    if aIncludePath in aFileName:
+                        pathNeeded = False
+                        break
+                if not pathNeeded:
+                    continue
+                # Keep the file to be checked
                 headerFileDependencySet.add(aFileName)
 
         # Ensure that gModuleBuildTracking has been initialized per architecture
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index ed6822334e..aaea8767ef 100644
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -1114,6 +1114,21 @@ class ModuleAutoGen(AutoGen):
     def IncludePathLength(self):
         return sum(len(inc)+1 for inc in self.IncludePathList)
 
+    ## Get the list of include paths from the packages
+    #
+    #   @IncludesList     list             The list path
+    #
+    @cached_property
+    def PackageIncludePathList(self):
+        IncludesList = []
+        for Package in self.Module.Packages:
+            PackageDir = mws.join(self.WorkspaceDir, Package.MetaFile.Dir)
+            IncludesList = Package.Includes
+            if Package._PrivateIncludes:
+                if not self.MetaFile.Path.startswith(PackageDir):
+                    IncludesList = list(set(Package.Includes).difference(set(Package._PrivateIncludes)))
+        return IncludesList
+
     ## Get HII EX PCDs which maybe used by VFR
     #
     #  efivarstore used by VFR may relate with HII EX PCDs
-- 
2.22.0.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45460): https://edk2.groups.io/g/devel/message/45460
Mute This Topic: https://groups.io/mt/32841936/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [Patch V3] BaseTools: Fix checking for Sources section in INF file
Posted by Liming Gao 4 years, 8 months ago
Tested-by: Liming Gao <liming.gao@intel.com>

> -----Original Message-----
> From: Rodriguez, Christian
> Sent: Monday, August 12, 2019 11:32 PM
> To: devel@edk2.groups.io
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>
> Subject: [Patch V3] BaseTools: Fix checking for Sources section in INF file
> 
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804
> 
> The check to see if [Sources] section lists all the header type
> files of a module is missing the exclusion of source files that
> fall under the scope of Package includes. This change adds the
> exclusions.
> 
> Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> ---
>  BaseTools/Source/Python/AutoGen/GenMake.py       | 25 ++++++++++++++++++++++---
>  BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 15 +++++++++++++++
>  2 files changed, 37 insertions(+), 3 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
> index 5802ae5ae4..499ef82aea 100644
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -906,8 +906,14 @@ cleanlib:
>                                      self._AutoGenObject.IncludePathList + self._AutoGenObject.BuildOptionIncPathList
> 
>                                      )
> 
> 
> 
> +        # Get a set of unique package includes from MetaFile
> 
> +        parentMetaFileIncludes = set()
> 
> +        for aInclude in self._AutoGenObject.PackageIncludePathList:
> 
> +            aIncludeName = str(aInclude)
> 
> +            parentMetaFileIncludes.add(aIncludeName.lower())
> 
> +
> 
>          # Check if header files are listed in metafile
> 
> -        # Get a list of unique module header source files from MetaFile
> 
> +        # Get a set of unique module header source files from MetaFile
> 
>          headerFilesInMetaFileSet = set()
> 
>          for aFile in self._AutoGenObject.SourceFileList:
> 
>              aFileName = str(aFile)
> 
> @@ -915,24 +921,37 @@ cleanlib:
>                  continue
> 
>              headerFilesInMetaFileSet.add(aFileName.lower())
> 
> 
> 
> -        # Get a list of unique module autogen files
> 
> +        # Get a set of unique module autogen files
> 
>          localAutoGenFileSet = set()
> 
>          for aFile in self._AutoGenObject.AutoGenFileList:
> 
>              localAutoGenFileSet.add(str(aFile).lower())
> 
> 
> 
> -        # Get a list of unique module dependency header files
> 
> +        # Get a set of unique module dependency header files
> 
>          # Exclude autogen files and files not in the source directory
> 
> +        # and files that are under the package include list
> 
>          headerFileDependencySet = set()
> 
>          localSourceDir = str(self._AutoGenObject.SourceDir).lower()
> 
>          for Dependency in FileDependencyDict.values():
> 
>              for aFile in Dependency:
> 
>                  aFileName = str(aFile).lower()
> 
> +                # Exclude non-header files
> 
>                  if not aFileName.endswith('.h'):
> 
>                      continue
> 
> +                # Exclude autogen files
> 
>                  if aFileName in localAutoGenFileSet:
> 
>                      continue
> 
> +                # Exclude include out of local scope
> 
>                  if localSourceDir not in aFileName:
> 
>                      continue
> 
> +                # Exclude files covered by package includes
> 
> +                pathNeeded = True
> 
> +                for aIncludePath in parentMetaFileIncludes:
> 
> +                    if aIncludePath in aFileName:
> 
> +                        pathNeeded = False
> 
> +                        break
> 
> +                if not pathNeeded:
> 
> +                    continue
> 
> +                # Keep the file to be checked
> 
>                  headerFileDependencySet.add(aFileName)
> 
> 
> 
>          # Ensure that gModuleBuildTracking has been initialized per architecture
> 
> diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> index ed6822334e..aaea8767ef 100644
> --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
> @@ -1114,6 +1114,21 @@ class ModuleAutoGen(AutoGen):
>      def IncludePathLength(self):
> 
>          return sum(len(inc)+1 for inc in self.IncludePathList)
> 
> 
> 
> +    ## Get the list of include paths from the packages
> 
> +    #
> 
> +    #   @IncludesList     list             The list path
> 
> +    #
> 
> +    @cached_property
> 
> +    def PackageIncludePathList(self):
> 
> +        IncludesList = []
> 
> +        for Package in self.Module.Packages:
> 
> +            PackageDir = mws.join(self.WorkspaceDir, Package.MetaFile.Dir)
> 
> +            IncludesList = Package.Includes
> 
> +            if Package._PrivateIncludes:
> 
> +                if not self.MetaFile.Path.startswith(PackageDir):
> 
> +                    IncludesList = list(set(Package.Includes).difference(set(Package._PrivateIncludes)))
> 
> +        return IncludesList
> 
> +
> 
>      ## Get HII EX PCDs which maybe used by VFR
> 
>      #
> 
>      #  efivarstore used by VFR may relate with HII EX PCDs
> 
> --
> 2.22.0.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45461): https://edk2.groups.io/g/devel/message/45461
Mute This Topic: https://groups.io/mt/32841936/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [Patch V3] BaseTools: Fix checking for Sources section in INF file
Posted by Bob Feng 4 years, 8 months ago
 Reviewed-by: Bob Feng <bob.c.feng@intel.com>

-----Original Message-----
From: Rodriguez, Christian 
Sent: Monday, August 12, 2019 11:32 PM
To: devel@edk2.groups.io
Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: [Patch V3] BaseTools: Fix checking for Sources section in INF file

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1804

The check to see if [Sources] section lists all the header type files of a module is missing the exclusion of source files that fall under the scope of Package includes. This change adds the exclusions.

Signed-off-by: Christian Rodriguez <christian.rodriguez@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
---
 BaseTools/Source/Python/AutoGen/GenMake.py       | 25 ++++++++++++++++++++++---
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 15 +++++++++++++++
 2 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index 5802ae5ae4..499ef82aea 100644
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -906,8 +906,14 @@ cleanlib:
                                     self._AutoGenObject.IncludePathList + self._AutoGenObject.BuildOptionIncPathList                                     ) +        # Get a set of unique package includes from MetaFile+        parentMetaFileIncludes = set()+        for aInclude in self._AutoGenObject.PackageIncludePathList:+            aIncludeName = str(aInclude)+            parentMetaFileIncludes.add(aIncludeName.lower())+         # Check if header files are listed in metafile-        # Get a list of unique module header source files from MetaFile+        # Get a set of unique module header source files from MetaFile         headerFilesInMetaFileSet = set()         for aFile in self._AutoGenObject.SourceFileList:             aFileName = str(aFile)@@ -915,24 +921,37 @@ cleanlib:
                 continue             headerFilesInMetaFileSet.add(aFileName.lower()) -        # Get a list of unique module autogen files+        # Get a set of unique module autogen files         localAutoGenFileSet = set()         for aFile in self._AutoGenObject.AutoGenFileList:             localAutoGenFileSet.add(str(aFile).lower()) -        # Get a list of unique module dependency header files+        # Get a set of unique module dependency header files         # Exclude autogen files and files not in the source directory+        # and files that are under the package include list         headerFileDependencySet = set()         localSourceDir = str(self._AutoGenObject.SourceDir).lower()         for Dependency in FileDependencyDict.values():             for aFile in Dependency:                 aFileName = str(aFile).lower()+                # Exclude non-header files                 if not aFileName.endswith('.h'):                     continue+                # Exclude autogen files                 if aFileName in localAutoGenFileSet:                     continue+                # Exclude include out of local scope                 if localSourceDir not in aFileName:                     continue+                # Exclude files covered by package includes+                pathNeeded = True+                for aIncludePath in parentMetaFileIncludes:+                    if aIncludePath in aFileName:+                        pathNeeded = False+                        break+                if not pathNeeded:+                    continue+                # Keep the file to be checked                 headerFileDependencySet.add(aFileName)          # Ensure that gModuleBuildTracking has been initialized per architecturediff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index ed6822334e..aaea8767ef 100644
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -1114,6 +1114,21 @@ class ModuleAutoGen(AutoGen):
     def IncludePathLength(self):         return sum(len(inc)+1 for inc in self.IncludePathList) +    ## Get the list of include paths from the packages+    #+    #   @IncludesList     list             The list path+    #+    @cached_property+    def PackageIncludePathList(self):+        IncludesList = []+        for Package in self.Module.Packages:+            PackageDir = mws.join(self.WorkspaceDir, Package.MetaFile.Dir)+            IncludesList = Package.Includes+            if Package._PrivateIncludes:+                if not self.MetaFile.Path.startswith(PackageDir):+                    IncludesList = list(set(Package.Includes).difference(set(Package._PrivateIncludes)))+        return IncludesList+     ## Get HII EX PCDs which maybe used by VFR     #     #  efivarstore used by VFR may relate with HII EX PCDs-- 
2.22.0.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#45578): https://edk2.groups.io/g/devel/message/45578
Mute This Topic: https://groups.io/mt/32841936/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-