BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875

In order to improve the build performance, we implemented
multiple-processes AutoGen. This change will reduce 20% time
for AutoGen phase.

The design document can be got from:
https://edk2.groups.io/g/devel/files/Designs/2019/0627/Multiple-thread-AutoGen.pdf

This patch serial pass the build of Ovmf, MinKabylake, MinPurley, packages
under Edk2 repository and intel client and server platforms.

V2: 
1. The first version missed autogen related commit 
from e812a812c1a0800c49e11507cb46222351520cc7. V2 add those commit
back.
2. Move CreateAsBuildInf into AutoGenWorker process
3. Save GlobalVar_<platform guid>_<arch>.bin to build folder.
4. Regenerate patches based on master bb824f685d

Feng, Bob C (9):
  BaseTools: Singleton the object to handle build conf file
  BaseTools: Split WorkspaceAutoGen._InitWorker into multiple functions
  BaseTools: Add functions to get platform scope build options
  BaseTools: Decouple AutoGen Objects
  BaseTools: Enable Multiple Process AutoGen
  BaseTools: Add shared data for processes
  BaseTools: Add LogAgent to support multiple process Autogen
  BaseTools: Move BuildOption parser out of build.py
  BaseTools: Add the support for python 2

 BaseTools/Source/Python/AutoGen/AutoGen.py    | 4227 +----------------
 .../Source/Python/AutoGen/AutoGenWorker.py    |  221 +
 .../Source/Python/AutoGen/BuildEngine.py      |   22 +
 BaseTools/Source/Python/AutoGen/DataPipe.py   |  153 +
 BaseTools/Source/Python/AutoGen/GenC.py       |    6 +-
 BaseTools/Source/Python/AutoGen/GenMake.py    |    3 +-
 .../Source/Python/AutoGen/ModuleAutoGen.py    | 1903 ++++++++
 .../Python/AutoGen/ModuleAutoGenHelper.py     |  616 +++
 .../Source/Python/AutoGen/PlatformAutoGen.py  | 1493 ++++++
 .../Source/Python/AutoGen/WorkspaceAutoGen.py |  905 ++++
 BaseTools/Source/Python/Common/EdkLogger.py   |  117 +-
 BaseTools/Source/Python/Common/Misc.py        |    1 -
 .../Python/Common/TargetTxtClassObject.py     |   28 +-
 .../Python/Common/ToolDefClassObject.py       |    6 +-
 BaseTools/Source/Python/GenFds/GenFds.py      |    4 +-
 .../Python/GenFds/GenFdsGlobalVariable.py     |   54 +-
 .../Python/PatchPcdValue/PatchPcdValue.py     |    1 -
 .../Source/Python/Workspace/DscBuildData.py   |   38 +-
 .../Source/Python/Workspace/InfBuildData.py   |   39 +
 .../Python/Workspace/WorkspaceCommon.py       |    4 +
 .../Python/Workspace/WorkspaceDatabase.py     |    3 +
 BaseTools/Source/Python/build/BuildReport.py  |    4 +-
 BaseTools/Source/Python/build/build.py        |  325 +-
 BaseTools/Source/Python/build/buildoptions.py |   92 +
 24 files changed, 5825 insertions(+), 4440 deletions(-)
 create mode 100644 BaseTools/Source/Python/AutoGen/AutoGenWorker.py
 create mode 100644 BaseTools/Source/Python/AutoGen/DataPipe.py
 create mode 100644 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
 create mode 100644 BaseTools/Source/Python/AutoGen/ModuleAutoGenHelper.py
 create mode 100644 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
 create mode 100644 BaseTools/Source/Python/AutoGen/WorkspaceAutoGen.py
 create mode 100644 BaseTools/Source/Python/build/buildoptions.py

-- 
2.20.1.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44123): https://edk2.groups.io/g/devel/message/44123
Mute This Topic: https://groups.io/mt/32556734/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Bob,

On 07/22/19 10:50, Bob Feng wrote:
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875
> 
> In order to improve the build performance, we implemented
> multiple-processes AutoGen. This change will reduce 20% time
> for AutoGen phase.
> 
> The design document can be got from:
> https://edk2.groups.io/g/devel/files/Designs/2019/0627/Multiple-thread-AutoGen.pdf
> 
> This patch serial pass the build of Ovmf, MinKabylake, MinPurley, packages
> under Edk2 repository and intel client and server platforms.
> 
> V2: 
> 1. The first version missed autogen related commit 
> from e812a812c1a0800c49e11507cb46222351520cc7. V2 add those commit
> back.

> 2. Move CreateAsBuildInf into AutoGenWorker process
> 3. Save GlobalVar_<platform guid>_<arch>.bin to build folder.
> 4. Regenerate patches based on master bb824f685d

whenever someone posts a patch set (brand new patch set, or v2, v3, ...
of a previously posted patch set), I suggest capturing a mailing list
URL in the TianoCore Bugzilla, as a new comment.

This way someone interested in the evolution of the feature can approach
the mailing list postings from the Bugzilla, and get links to all
versions ever posted.

I've done this for your v1 posting; please add the v2 and v3 links.

(NB this request is for everyone.)

Thanks
Laszlo

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#44157): https://edk2.groups.io/g/devel/message/44157
Mute This Topic: https://groups.io/mt/32556734/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-