[edk2-devel] [PATCH] BaseTools: Fix an incremental build issue caused by macro in #include

Lin, Derek (HPS SW) posted 1 patch 4 years, 5 months ago
Failed in applying to current master (apply log)
BaseTools/Source/Python/AutoGen/GenMake.py | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
[edk2-devel] [PATCH] BaseTools: Fix an incremental build issue caused by macro in #include
Posted by Lin, Derek (HPS SW) 4 years, 5 months ago
When c/h file use macro after #include, for example,
In this case, GenMake is not able to create a healthy dependency for the c
file. GenMake used to add $(FORCE_REBUILD) dependency in the c file, this
guarantee the c file is always compiled in incremental build. But, this
function is broken since 05217d210e8da37b47d0be58ec363f7af2fa1c18 which
enable /MP for MSVC compiler, in order to compile multiple c files in one
command multi-processing. The fix here is adding '$(FORCE_REBUILD)' back to
retain the original function.

Line number 1728 and 978 are the code pieces which handle this logic.

Signed-off-by: Derek Lin <derek.lin2@hpe.com>
---
 BaseTools/Source/Python/AutoGen/GenMake.py | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index 97ba158ff2..59a01a7f24 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1080,13 +1080,17 @@ cleanlib:
                     else:
                         CmdCppDict[item.Target.SubDir] = ['$(MAKE_FILE)', Path]
                     if CppPath.Path in DependencyDict:
-                        for Temp in DependencyDict[CppPath.Path]:
-                            try:
-                                Path = self.PlaceMacro(Temp.Path, self.Macros)
-                            except:
-                                continue
-                            if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
-                                CmdCppDict[item.Target.SubDir].append(Path)
+                        if '$(FORCE_REBUILD)' in DependencyDict[CppPath.Path]:
+                            if '$(FORCE_REBUILD)' not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
+                                CmdCppDict[item.Target.SubDir].append('$(FORCE_REBUILD)')
+                        else:
+                            for Temp in DependencyDict[CppPath.Path]:
+                                try:
+                                    Path = self.PlaceMacro(Temp.Path, self.Macros)
+                                except:
+                                    continue
+                                if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
+                                    CmdCppDict[item.Target.SubDir].append(Path)
         if T.Commands:
             CommandList = T.Commands[:]
             for Item in CommandList[:]:
-- 
2.20.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#49053): https://edk2.groups.io/g/devel/message/49053
Mute This Topic: https://groups.io/mt/34557472/1787277
Mute #include: https://groups.io/mk?hashtag=include&subid=3901457
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH] BaseTools: Fix an incremental build issue caused by macro in #include
Posted by Philippe Mathieu-Daudé 4 years, 5 months ago
Hi Derek,

On 10/16/19 8:17 AM, Lin, Derek (HPS SW) wrote:
> When c/h file use macro after #include, for example,

Apparently you forgot to write your example.

> In this case, GenMake is not able to create a healthy dependency for the c
> file. GenMake used to add $(FORCE_REBUILD) dependency in the c file, this
> guarantee the c file is always compiled in incremental build. But, this
> function is broken since 05217d210e8da37b47d0be58ec363f7af2fa1c18 which
> enable /MP for MSVC compiler, in order to compile multiple c files in one
> command multi-processing. The fix here is adding '$(FORCE_REBUILD)' back to
> retain the original function.
> 
> Line number 1728 and 978 are the code pieces which handle this logic.
> 
> Signed-off-by: Derek Lin <derek.lin2@hpe.com>
> ---
>   BaseTools/Source/Python/AutoGen/GenMake.py | 18 +++++++++++-------
>   1 file changed, 11 insertions(+), 7 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
> index 97ba158ff2..59a01a7f24 100755
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -1080,13 +1080,17 @@ cleanlib:
>                       else:
>                           CmdCppDict[item.Target.SubDir] = ['$(MAKE_FILE)', Path]
>                       if CppPath.Path in DependencyDict:
> -                        for Temp in DependencyDict[CppPath.Path]:
> -                            try:
> -                                Path = self.PlaceMacro(Temp.Path, self.Macros)
> -                            except:
> -                                continue
> -                            if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
> -                                CmdCppDict[item.Target.SubDir].append(Path)
> +                        if '$(FORCE_REBUILD)' in DependencyDict[CppPath.Path]:
> +                            if '$(FORCE_REBUILD)' not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
> +                                CmdCppDict[item.Target.SubDir].append('$(FORCE_REBUILD)')
> +                        else:
> +                            for Temp in DependencyDict[CppPath.Path]:
> +                                try:
> +                                    Path = self.PlaceMacro(Temp.Path, self.Macros)
> +                                except:
> +                                    continue
> +                                if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
> +                                    CmdCppDict[item.Target.SubDir].append(Path)
>           if T.Commands:
>               CommandList = T.Commands[:]
>               for Item in CommandList[:]:
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#49092): https://edk2.groups.io/g/devel/message/49092
Mute This Topic: https://groups.io/mt/34557472/1787277
Mute #include: https://groups.io/mk?hashtag=include&subid=3901457
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH] BaseTools: Fix an incremental build issue caused by macro in #include
Posted by Bob Feng 4 years, 5 months ago
I think one of the example would be like this:
    #include MACRO(xxx.h)

I think this patch is good. Thanks for caching and fixing this bug.

Reviewed-by: Bob Feng <bob.c.feng@intel.com>

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Philippe Mathieu-Daudé
Sent: Wednesday, October 16, 2019 7:54 PM
To: devel@edk2.groups.io; derek.lin2@hpe.com
Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Fan, ZhijuX <zhijux.fan@intel.com>
Subject: Re: [edk2-devel] [PATCH] BaseTools: Fix an incremental build issue caused by macro in #include

Hi Derek,

On 10/16/19 8:17 AM, Lin, Derek (HPS SW) wrote:
> When c/h file use macro after #include, for example,

Apparently you forgot to write your example.

> In this case, GenMake is not able to create a healthy dependency for 
> the c file. GenMake used to add $(FORCE_REBUILD) dependency in the c 
> file, this guarantee the c file is always compiled in incremental 
> build. But, this function is broken since 
> 05217d210e8da37b47d0be58ec363f7af2fa1c18 which enable /MP for MSVC 
> compiler, in order to compile multiple c files in one command 
> multi-processing. The fix here is adding '$(FORCE_REBUILD)' back to retain the original function.
> 
> Line number 1728 and 978 are the code pieces which handle this logic.
> 
> Signed-off-by: Derek Lin <derek.lin2@hpe.com>
> ---
>   BaseTools/Source/Python/AutoGen/GenMake.py | 18 +++++++++++-------
>   1 file changed, 11 insertions(+), 7 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py 
> b/BaseTools/Source/Python/AutoGen/GenMake.py
> index 97ba158ff2..59a01a7f24 100755
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -1080,13 +1080,17 @@ cleanlib:
>                       else:
>                           CmdCppDict[item.Target.SubDir] = ['$(MAKE_FILE)', Path]
>                       if CppPath.Path in DependencyDict:
> -                        for Temp in DependencyDict[CppPath.Path]:
> -                            try:
> -                                Path = self.PlaceMacro(Temp.Path, self.Macros)
> -                            except:
> -                                continue
> -                            if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
> -                                CmdCppDict[item.Target.SubDir].append(Path)
> +                        if '$(FORCE_REBUILD)' in DependencyDict[CppPath.Path]:
> +                            if '$(FORCE_REBUILD)' not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
> +                                CmdCppDict[item.Target.SubDir].append('$(FORCE_REBUILD)')
> +                        else:
> +                            for Temp in DependencyDict[CppPath.Path]:
> +                                try:
> +                                    Path = self.PlaceMacro(Temp.Path, self.Macros)
> +                                except:
> +                                    continue
> +                                if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
> +                                    
> + CmdCppDict[item.Target.SubDir].append(Path)
>           if T.Commands:
>               CommandList = T.Commands[:]
>               for Item in CommandList[:]:
> 





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#49127): https://edk2.groups.io/g/devel/message/49127
Mute This Topic: https://groups.io/mt/34557472/1787277
Mute #include: https://groups.io/mk?hashtag=include&subid=3901457
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH] BaseTools: Fix an incremental build issue caused by macro in #include
Posted by Lin, Derek (HPS SW) 4 years, 5 months ago
Hi Phillippe,

Yeah, I forgot to fill my example. Thanks for the catch.

It is exactly as Bob mentioned, #include MACRO(xxx.h).

There are silicon code use macro to separate definitions, and use the approach to reuse code. Like:
#define PATH(x) <silicon1/x.h> // in platform 1
#define PATH(x) <silicon2/x.h> // in platform 2
#include PATH(abc.h)   // in common code

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#49135): https://edk2.groups.io/g/devel/message/49135
Mute This Topic: https://groups.io/mt/34557472/1787277
Mute #include: https://groups.io/mk?hashtag=include&subid=3901457
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-