1. Patchew
  2. EDK2
  3. [edk2-devel] [PATCH 0/2] UnitTestFrameworkPkg: Fix Google Test components with multiple files
  4. View patch

[edk2-devel] [PATCH 1/2] UnitTestFrameworkPkg: Fix Google Test components with multiple files

Pedro Falcato posted 2 patches 2 years, 2 months ago
[edk2-devel] [PATCH 1/2] UnitTestFrameworkPkg: Fix Google Test components with multiple files
Posted by Pedro Falcato 2 years, 2 months ago 
Google Test hides test registration in global constructors on global
objects. Global constructors are traditionally implemented by placing
references to the global constructor's symbol in special sections
(traditionally named .ctors or .init_array). These sections are not
explicitly referenced by the linker, and libc only looks at special
start and end symbols (and calls them).

This works fine if you're linking a program manually using

    gcc a.o b.o c.o -o test_suite

but fails miserably when using static libraries (such as what EDK2
does), because traditional static archive symbol resolution rules don't
allow for object files to be pulled in to the link if there isn't an
undefined symbol reference to that .o elsewhere.

Fix it by passing --whole-archive (GCC) and /WHOLEARCHIVE (MSVC). These
options force the linker to pull in the entire static library, thus
including previously-unreferenced constructors and making sure
multi-file gtest EDK2 components work.

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=4610
Signed-off-by: Pedro Falcato <pedro.falcato@gmail.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
---
 Note: /WHOLEARCHIVE should work for VS2015 and newer. I haven't been able to
       test it, so if someone could test on msft it would be much appreciated.
 Testing is as simple as taking this patch set
 (https://openfw.io/edk2-devel/20231130024611.67135-1-pedro.falcato@gmail.com/)
 and removing the #include "TestCheckSum.cpp" in TestBaseLibMain.cpp. If everything worked correctly,
 you should have 4 tests and not 0.

 UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
index 5217de31e491..0f11706e7324 100644
--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
@@ -37,7 +37,7 @@
   # MSFT
   #
   MSFT:*_*_*_CC_FLAGS               = /EHsc
-  MSFT:*_*_*_DLINK_FLAGS            == /out:"$(BIN_DIR)\$(MODULE_NAME_GUID).exe" /pdb:"$(BIN_DIR)\$(MODULE_NAME_GUID).pdb" /IGNORE:4001 /NOLOGO /SUBSYSTEM:CONSOLE /DEBUG /STACK:0x40000,0x40000 /NODEFAULTLIB:libcmt.lib libcmtd.lib
+  MSFT:*_*_*_DLINK_FLAGS            == /out:"$(BIN_DIR)\$(MODULE_NAME_GUID).exe" /pdb:"$(BIN_DIR)\$(MODULE_NAME_GUID).pdb" /IGNORE:4001 /NOLOGO /SUBSYSTEM:CONSOLE /DEBUG /STACK:0x40000,0x40000 /NODEFAULTLIB:libcmt.lib libcmtd.lib /WHOLEARCHIVE
   MSFT:*_*_IA32_DLINK_FLAGS         = /MACHINE:I386
   MSFT:*_*_X64_DLINK_FLAGS          = /MACHINE:AMD64
 
@@ -56,7 +56,12 @@
   #
   GCC:*_*_IA32_DLINK_FLAGS == -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m32 -no-pie
   GCC:*_*_X64_DLINK_FLAGS  == -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m64 -no-pie
-  GCC:*_*_*_DLINK2_FLAGS   == -lgcov -lpthread -lstdc++ -lm
+  #
+  # Surround our static libraries with whole-archive, so constructor-based test registration works properly.
+  # Note that we need to --no-whole-archive before linking system libraries.
+  #
+  GCC:*_*_*_DLINK_FLAGS    = -Wl,--whole-archive
+  GCC:*_*_*_DLINK2_FLAGS   == -Wl,--no-whole-archive -lgcov -lpthread -lstdc++ -lm
 
   #
   # Need to do this link via gcc and not ld as the pathing to libraries changes from OS version to OS version
-- 
2.43.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#111916): https://edk2.groups.io/g/devel/message/111916
Mute This Topic: https://groups.io/mt/102904623/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 1/2] UnitTestFrameworkPkg: Fix Google Test components with multiple files
Posted by Michael Kubacki 2 years, 2 months ago 
Hi Pedro,

Visual Studio NOOPT builds result in linker errors. I combined your 
patch series with the test instruction change in this PR - 
https://github.com/tianocore/edk2/pull/5096.

You can use a PR to test the VS build.

Thanks,
Michael

On 11/30/2023 5:42 PM, Pedro Falcato wrote:
> Google Test hides test registration in global constructors on global
> objects. Global constructors are traditionally implemented by placing
> references to the global constructor's symbol in special sections
> (traditionally named .ctors or .init_array). These sections are not
> explicitly referenced by the linker, and libc only looks at special
> start and end symbols (and calls them).
> 
> This works fine if you're linking a program manually using
> 
>      gcc a.o b.o c.o -o test_suite
> 
> but fails miserably when using static libraries (such as what EDK2
> does), because traditional static archive symbol resolution rules don't
> allow for object files to be pulled in to the link if there isn't an
> undefined symbol reference to that .o elsewhere.
> 
> Fix it by passing --whole-archive (GCC) and /WHOLEARCHIVE (MSVC). These
> options force the linker to pull in the entire static library, thus
> including previously-unreferenced constructors and making sure
> multi-file gtest EDK2 components work.
> 
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=4610
> Signed-off-by: Pedro Falcato <pedro.falcato@gmail.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Michael Kubacki <mikuback@linux.microsoft.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> ---
>   Note: /WHOLEARCHIVE should work for VS2015 and newer. I haven't been able to
>         test it, so if someone could test on msft it would be much appreciated.
>   Testing is as simple as taking this patch set
>   (https://openfw.io/edk2-devel/20231130024611.67135-1-pedro.falcato@gmail.com/)
>   and removing the #include "TestCheckSum.cpp" in TestBaseLibMain.cpp. If everything worked correctly,
>   you should have 4 tests and not 0.
> 
>   UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc | 9 +++++++--
>   1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
> index 5217de31e491..0f11706e7324 100644
> --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
> +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
> @@ -37,7 +37,7 @@
>     # MSFT
>     #
>     MSFT:*_*_*_CC_FLAGS               = /EHsc
> -  MSFT:*_*_*_DLINK_FLAGS            == /out:"$(BIN_DIR)\$(MODULE_NAME_GUID).exe" /pdb:"$(BIN_DIR)\$(MODULE_NAME_GUID).pdb" /IGNORE:4001 /NOLOGO /SUBSYSTEM:CONSOLE /DEBUG /STACK:0x40000,0x40000 /NODEFAULTLIB:libcmt.lib libcmtd.lib
> +  MSFT:*_*_*_DLINK_FLAGS            == /out:"$(BIN_DIR)\$(MODULE_NAME_GUID).exe" /pdb:"$(BIN_DIR)\$(MODULE_NAME_GUID).pdb" /IGNORE:4001 /NOLOGO /SUBSYSTEM:CONSOLE /DEBUG /STACK:0x40000,0x40000 /NODEFAULTLIB:libcmt.lib libcmtd.lib /WHOLEARCHIVE
>     MSFT:*_*_IA32_DLINK_FLAGS         = /MACHINE:I386
>     MSFT:*_*_X64_DLINK_FLAGS          = /MACHINE:AMD64
>   
> @@ -56,7 +56,12 @@
>     #
>     GCC:*_*_IA32_DLINK_FLAGS == -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m32 -no-pie
>     GCC:*_*_X64_DLINK_FLAGS  == -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m64 -no-pie
> -  GCC:*_*_*_DLINK2_FLAGS   == -lgcov -lpthread -lstdc++ -lm
> +  #
> +  # Surround our static libraries with whole-archive, so constructor-based test registration works properly.
> +  # Note that we need to --no-whole-archive before linking system libraries.
> +  #
> +  GCC:*_*_*_DLINK_FLAGS    = -Wl,--whole-archive
> +  GCC:*_*_*_DLINK2_FLAGS   == -Wl,--no-whole-archive -lgcov -lpthread -lstdc++ -lm
>   
>     #
>     # Need to do this link via gcc and not ld as the pathing to libraries changes from OS version to OS version


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#111983): https://edk2.groups.io/g/devel/message/111983
Mute This Topic: https://groups.io/mt/102904623/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 1/2] UnitTestFrameworkPkg: Fix Google Test components with multiple files
Posted by Pedro Falcato 2 years, 2 months ago 
On Fri, Dec 1, 2023 at 5:07 PM Michael Kubacki
<mikuback@linux.microsoft.com> wrote:
>
> Hi Pedro,
>
> Visual Studio NOOPT builds result in linker errors. I combined your
> patch series with the test instruction change in this PR -
> https://github.com/tianocore/edk2/pull/5096.
>
> You can use a PR to test the VS build.

Thanks for the heads up, but I ended up booting Windows to expedite the process.

So, I noticed from the build logs that libcmtd.lib was having issues
doing a /WHOLEARCHIVE link (not unheard of, had the same problems with
Linux system libraries). Then I noticed in MSDN:
"The /WHOLEARCHIVE option forces the linker to include every object
file from either a specified static library, or if no library is
specified, from all static libraries specified to the LINK command"
Note the "from all static libraries specified to the LINK command". So
I noticed libcmtd.lib was being specified manually, and I simply
deleted

/NODEFAULTLIB:libcmt.lib libcmtd.lib

From line 40 of UnitTestFrameworkPkgHost.dsc.inc.
So, before I submit a v2 of this, does anyone know why this was added
manually? Mike?

Note: I tried to add /MTd, but that seems to be a cl.exe option, not link.exe

-- 
Pedro


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#111985): https://edk2.groups.io/g/devel/message/111985
Mute This Topic: https://groups.io/mt/102904623/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Patchew 2.1-devel-b67e4c2

© 2016 - 2026 Red Hat, Inc.