From nobody Mon Feb 9 16:45:45 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+96116+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96116+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1667949185; cv=none; d=zohomail.com; s=zohoarc; b=DAif1WpBnGjI9D+CVUvv8WY1zZhddD0Yxblvtgnff0IHnabc0qqhAH8eY9Gl3lBoUl8TQfvVAe4RUHvrDMnLdHVG/Jb7LjO9FTnQPJ6j6Lzj2DjIZcKqbX8gI+48mlHhYznoNNfIW5cY5rr0y/o0bR77gMW5x3Y1LyIr6L8BinY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1667949185; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=a8y246kBqHATsPJyatv0ubNjVtrOF5Ao0CV+FRWfcHE=; b=Qt2AhuTWQohWt9YAx/C1CxwHL/mNzjqYh619oCwJ8/bCZvWZYVyKHQ5Y0iv5As+gdklo9Ymybx3BIZvZ80ZbR3K8MEGZziNyvPVkjFAbfDRzF7BC7ZSYT8p/vX5z7wz2sHA7m4DqhLMweKt2662U0CSmhEs7DHSJWrks7qm/jAQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+96116+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1667949185505112.05259600348973; Tue, 8 Nov 2022 15:13:05 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id 5T8KYY1788612xk3LdG5cPPF; Tue, 08 Nov 2022 15:13:05 -0800 X-Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web08.21.1667949182063513285 for ; Tue, 08 Nov 2022 15:13:03 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10525"; a="311990221" X-IronPort-AV: E=Sophos;i="5.96,149,1665471600"; d="scan'208";a="311990221" X-Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2022 15:13:00 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10525"; a="636512158" X-IronPort-AV: E=Sophos;i="5.96,149,1665471600"; d="scan'208";a="636512158" X-Received: from mdkinney-mobl2.amr.corp.intel.com ([10.209.46.35]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2022 15:13:00 -0800 From: "Michael D Kinney" To: devel@edk2.groups.io Cc: Michael Kubacki , Sean Brogan , Andrew Fish , Leif Lindholm Subject: [edk2-devel] [Patch v3 3/7] UnitTestFrameworkPkg: Add googletest submodule and GoogleTestLib Date: Tue, 8 Nov 2022 15:12:48 -0800 Message-Id: <20221108231252.1864-4-michael.d.kinney@intel.com> In-Reply-To: <20221108231252.1864-1-michael.d.kinney@intel.com> References: <20221108231252.1864-1-michael.d.kinney@intel.com> MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,michael.d.kinney@intel.com X-Gm-Message-State: H0K9E3wCE2B4ukYzmoYF8ZSox1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1667949185; bh=mAZxvZAb5BzKOBT+ozcIvMFArKDJu7ZWPrHQFlmcuGE=; h=Cc:Date:From:Reply-To:Subject:To; b=qIALErIjTp+0tjCNKRT5z/OclKrPo+4zCaCQzUsfyx8V1qDrYsjpYpEI3/9jWOdJ1Jt Qcr4ZY5QzvRu8GbMUH90QRDc5TRWkJArLntOUP1kwPF1HsdjJaNnd1HgGexwD+02Nuoqp 3Cjekqw6aCwffNmhK/sezjKfQZuaER/9zfw= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1667949187376100019 Content-Type: text/plain; charset="utf-8" REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D4134 Add submodule for googletest and add GoogleTestLib that is required for GoogleTest based unit tests. Add GoogleTest documentation to Readme.md along with a port of the sample unit test to the GoogleTest style. Cc: Michael Kubacki Cc: Sean Brogan Cc: Andrew Fish Cc: Leif Lindholm Signed-off-by: Michael D Kinney Reviewed-by: Michael Kubacki Acked-by: Leif Lindholm --- .gitmodules | 3 + ReadMe.rst | 1 + .../Include/Library/GoogleTestLib.h | 14 + .../Library/GoogleTestLib/GoogleTestLib.inf | 36 +++ .../Library/GoogleTestLib/GoogleTestLib.uni | 14 + .../Library/GoogleTestLib/googletest | 1 + UnitTestFrameworkPkg/ReadMe.md | 255 +++++++++++++++-- .../SampleGoogleTest/SampleGoogleTest.cpp | 263 ++++++++++++++++++ .../SampleGoogleTest/SampleGoogleTestHost.inf | 35 +++ .../Test/UnitTestFrameworkPkgHostTest.dsc | 4 +- .../UnitTestFrameworkPkg.ci.yaml | 4 +- UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec | 8 + .../UnitTestFrameworkPkgHost.dsc.inc | 4 +- 13 files changed, 611 insertions(+), 31 deletions(-) create mode 100644 UnitTestFrameworkPkg/Include/Library/GoogleTestLib.h create mode 100644 UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLi= b.inf create mode 100644 UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLi= b.uni create mode 160000 UnitTestFrameworkPkg/Library/GoogleTestLib/googletest create mode 100644 UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogl= eTest/SampleGoogleTest.cpp create mode 100644 UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogl= eTest/SampleGoogleTestHost.inf diff --git a/.gitmodules b/.gitmodules index b845c9ee3ff0..8011a88d9d25 100644 --- a/.gitmodules +++ b/.gitmodules @@ -20,3 +20,6 @@ [submodule "RedfishPkg/Library/JsonLib/jansson"] path =3D RedfishPkg/Library/JsonLib/jansson url =3D https://github.com/akheron/jansson +[submodule "UnitTestFrameworkPkg/Library/GoogleTestLib/googletest"] + path =3D UnitTestFrameworkPkg/Library/GoogleTestLib/googletest + url =3D https://github.com/google/googletest.git diff --git a/ReadMe.rst b/ReadMe.rst index 8f5db11281bf..497d96355908 100644 --- a/ReadMe.rst +++ b/ReadMe.rst @@ -93,6 +93,7 @@ that are covered by additional licenses. - `MdeModulePkg/Library/BrotliCustomDecompressLib/brotli `__ - `MdeModulePkg/Universal/RegularExpressionDxe/oniguruma `__ - `UnitTestFrameworkPkg/Library/CmockaLib/cmocka `__ +- `UnitTestFrameworkPkg/Library/GoogleTestLib/googletest `__ - `RedfishPkg/Library/JsonLib/jansson `__ =20 The EDK II Project is composed of packages. The maintainers for each packa= ge diff --git a/UnitTestFrameworkPkg/Include/Library/GoogleTestLib.h b/UnitTes= tFrameworkPkg/Include/Library/GoogleTestLib.h new file mode 100644 index 000000000000..ebec766d4cf7 --- /dev/null +++ b/UnitTestFrameworkPkg/Include/Library/GoogleTestLib.h @@ -0,0 +1,14 @@ +/** @file + GoogleTestLib class with APIs from the googletest project + + Copyright (c) 2022, Intel Corporation. All rights reserved.
+ SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef GOOGLE_TEST_LIB_H_ +#define GOOGLE_TEST_LIB_H_ + +#include + +#endif diff --git a/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf b= /UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf new file mode 100644 index 000000000000..68db75d7023f --- /dev/null +++ b/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf @@ -0,0 +1,36 @@ +## @file +# This module provides GoogleTest Library implementation. +# +# Copyright (c) 2022, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + INF_VERSION =3D 0x00010005 + BASE_NAME =3D GoogleTestLib + MODULE_UNI_FILE =3D GoogleTestLib.uni + FILE_GUID =3D A90E4751-AD30-43CC-980B-01E356B49ADF + MODULE_TYPE =3D BASE + VERSION_STRING =3D 0.1 + LIBRARY_CLASS =3D GoogleTestLib|HOST_APPLICATION + +# +# VALID_ARCHITECTURES =3D IA32 X64 ARM AARCH64 +# + +[Sources] + googletest/googletest/src/gtest-all.cc + +[Packages] + UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec + +[BuildOptions] + MSFT:*_*_*_CC_FLAGS =3D=3D /c /EHsc /Zi + MSFT:NOOPT_*_*_CC_FLAGS =3D /Od + + GCC:*_*_*_CC_FLAGS =3D=3D -g -c + + GCC:NOOPT_*_*_CC_FLAGS =3D -O0 + GCC:*_*_IA32_CC_FLAGS =3D -m32 + GCC:*_*_X64_CC_FLAGS =3D -m64 diff --git a/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.uni b= /UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.uni new file mode 100644 index 000000000000..14c862a23744 --- /dev/null +++ b/UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.uni @@ -0,0 +1,14 @@ +// /** @file +// This module provides GoogleTest Library implementation. +// +// This module provides GoogleTest Library implementation. +// +// Copyright (c) 2022, Intel Corporation. All rights reserved.
+// +// SPDX-License-Identifier: BSD-2-Clause-Patent +// +// **/ + +#string STR_MODULE_ABSTRACT #language en-US "GoogleTest Librar= y implementation" + +#string STR_MODULE_DESCRIPTION #language en-US "This module provi= des GoogleTest Library implementation." diff --git a/UnitTestFrameworkPkg/Library/GoogleTestLib/googletest b/UnitTe= stFrameworkPkg/Library/GoogleTestLib/googletest new file mode 160000 index 000000000000..86add13493e5 --- /dev/null +++ b/UnitTestFrameworkPkg/Library/GoogleTestLib/googletest @@ -0,0 +1 @@ +Subproject commit 86add13493e5c881d7e4ba77fb91c1f57752b3a4 diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md index e696412cb3cf..9ce04b7f3eb6 100644 --- a/UnitTestFrameworkPkg/ReadMe.md +++ b/UnitTestFrameworkPkg/ReadMe.md @@ -2,12 +2,67 @@ =20 ## About =20 -This package adds a unit test framework capable of building tests for mult= iple contexts including +This package provides unit test frameworks capable of building tests for m= ultiple contexts including the UEFI shell environment and host-based environments. It allows for unit= test development to focus -on the tests and leave error logging, result formatting, context persistan= ce, and test running to the framework. +on the tests and leave error logging, result formatting, context persisten= ce, and test running to the framework. The unit test framework works well for low level unit tests as well as sys= tem level tests and fits easily in automation frameworks. =20 +### Framework + +The first unit test framework is called **Framework** and is implemented a= s a set of EDK II libraries. +The Framework supports both host-based unit tests and target-based unit te= sts that share the same +source style, macros, and APIs. In some scenarios, the same unit test case= sources can be built +for both host-based unit test execution and target-based unit test executi= on. Host-based unit tests +that require mocked interfaces can use the mocking infrastructure provided= by +[cmocka](https://api.cmocka.org/) that is included in the UnitTestFramewor= kPkg as a submodule. + +### GoogleTest + +The second unit test framework supported by the UnitTestFrameworkPkg is +[GoogleTest](http://google.github.io/googletest/) that can be used to impl= ement host-based unit tests. +Use of GoogleTest for target-based unit tests of EDK II components is not = supported. If a +host-based unit test requires mocked interfaces, then the Framework with c= mocka support should be +used instead. Enabling support for mocked interfaces with GoogleTest is be= ing actively investigated. +[GoogleTest on GitHub](https://github.com/google/googletest) is included i= n the UnitTestFrameworkPkg +as a submodule. + +GoogleTest requires less overhead to register test suites and test cases c= ompared to the Framework. +There are also a number of tools that layer on top of GoogleTest that impr= ove developer productivity. +One example is the VS Code extension +[C++ TestMate](https://marketplace.visualstudio.com/items?itemName=3Dmatep= ek.vscode-catch2-test-adapter) +that may be used to implement, run, and debug unit tests implemented using= GoogleTest. + +If a component can be tested with host-based unit tests without support fo= r mocked interfaces, +then GoogleTest is recommended. The MdePkg contains a port of the BaseSafe= IntLib unit tests in +the GoogleTest style so the differences between GoogleTest and Framework u= nit tests can be reviewed. +The paths to the BaseSafeIntLib unit tests are: + +* MdePkg\Test\UnitTest\Library\BaseSafeIntLib +* MdePkg\Test\GoogleTest\Library\BaseSafeIntLib + +## Framework and GoogleTest Feature Comparison + +| Feature | Framework | GoogleTest | +|:----------------------------|:---------:|:----------:| +| Host Based Unit Tests | YES | YES | +| Target Based Unit Tests | YES | NO | +| Unit Test Source Language | C | C++ | +| Register Test Suite | YES | Auto | +| Register Test Case | YES | Auto | +| Death/Expected Assert Tests | YES | YES | +| Setup/Teardown Hooks | YES | YES | +| Value-Parameterized Tests | NO | YES | +| Typed Tests | NO | YES | +| Type-Parameterized Tests | NO | YES | +| Timeout Support | NO | YES | +| Mocking Support | Cmocka | NO | +| JUNIT XML Reports | YES | YES | +| Execute subset of tests | NO | YES | +| VS Code Extensions | NO | YES | + +## Framework Libraries + ### UnitTestLib =20 The main "framework" library. The core of the framework is the Framework o= bject, which can have any number @@ -31,10 +86,10 @@ in supporting a system reboot in the middle of a test r= un. =20 Library provides function to run at the end of a framework test run and ha= ndles formatting the report. This is a common customization point and allows the unit test framework to= fit its output reports into -other test infrastructure. In this package a simple library instances has = been supplied to output test +other test infrastructure. In this package simple library instances have b= een supplied to output test results to the console as plain text. =20 -## Samples +## Framework Samples =20 There is a sample unit test provided as both an example of how to write a = unit test and leverage many of the features of the framework. This sample can be found in the `Te= st/UnitTest/Sample/SampleUnitTest` @@ -43,7 +98,7 @@ directory. The sample is provided in PEI, SMM, DXE, and UEFI App flavors. It also has= a flavor for the HOST_APPLICATION build type, which can be run on a host system without needing a target. =20 -## Usage +## Framework Usage =20 This section is built a lot like a "Getting Started". We'll go through som= e of the components that are needed when constructing a unit test and some of the decisions that are made by t= he test writer. We'll also describe @@ -51,7 +106,7 @@ how to check for expected conditions in test cases and a= bit of the logging char =20 Most of these examples will refer to the SampleUnitTestUefiShell app found= in this package. =20 -### Requirements - INF +### Framework Requirements - INF =20 In our INF file, we'll need to bring in the `UnitTestLib` library. Conveni= ently, the interface header for the `UnitTestLib` is located in `MdePkg`, so you shouldn't need= to depend on any other @@ -80,7 +135,7 @@ to make sure that the module `BASE_NAME` contains the wo= rd `Test`... BASE_NAME =3D SampleUnitTestUefiShell ``` =20 -### Requirements - Code +### Framework Requirements - Code =20 Not to state the obvious, but let's make sure we have the following includ= e before getting too far along... =20 @@ -90,9 +145,9 @@ Not to state the obvious, but let's make sure we have th= e following include befo =20 Now that we've got that squared away, let's look at our 'Main()'' routine = (or DriverEntryPoint() or whatever). =20 -### Configuring the Framework +### Framework Configuration =20 -Everything in the UnitTestPkg framework is built around an object called -= - conveniently -- the Framework. +Everything in the UnitTestFrameworkPkg framework is built around an object= called -- conveniently -- the Framework. This Framework object will contain all the information about our test, the= test suites and test cases associated with it, the current location within the test pass, and any results that h= ave been recorded so far. =20 @@ -102,7 +157,7 @@ The long name and version strings are just for user pre= sentation and relatively will be used to name any cache files and/or test results, so should be a n= ame that makes sense in that context. These strings are copied internally to the Framework, so using stack-alloc= ated or literal strings is fine. =20 -In the 'SampleUnitTestUefiShell' app, the module name is used as the short= name, so the init looks like this. +In the 'SampleUnitTestUefiShell' app, the module name is used as the short= name, so the initialization looks like this. =20 ```c DEBUG(( DEBUG_INFO, "%a v%a\n", UNIT_TEST_APP_NAME, UNIT_TEST_APP_VERSION = )); @@ -144,11 +199,11 @@ will be used when adding test cases. Great! Now we've finished some of the cruft, red tape, and busy work. We'r= e ready to add some tests. Adding a test to a test suite is accomplished with the -- you guessed it -- `AddTestCase= ` function. It takes in the suite handle; a `CHAR8` string for the description and class name; a function pointer fo= r the test case itself; additional, optional -function pointers for prerequisite check and cleanup routines; and and opt= ional pointer to a context structure. +function pointers for prerequisite check and cleanup routines; and an opti= onal pointer to a context structure. =20 Okay, that's a lot. Let's take it one piece at a time. The description and= class name strings are very similar in usage to the suite title and package name strings in the test suites. The = former is for user presentation and the -latter is for xUnit parsing. The test case function pointer is what is act= ually executed as the "test" and the +latter is for xUnit parsing. The test case function pointer is what is exe= cuted as the "test" and the prototype should be `UNIT_TEST_FUNCTION`. The last three parameters requir= e a little bit more explaining. =20 The prerequisite check function has a prototype of `UNIT_TEST_PREREQUISITE= ` and -- if provided -- will be called @@ -180,7 +235,7 @@ Once all the suites and cases are added, it's time to r= un the Framework. Status =3D RunAllTestSuites( Framework ); ``` =20 -### A Simple Test Case +### Framework - A Simple Test Case =20 We'll take a look at the below test case from 'SampleUnitTestApp'... =20 @@ -217,9 +272,9 @@ _Note_ that this early return can have implications for= memory leakage. =20 At the end, if all test criteria pass, you should return `UNIT_TEST_PASSED= `. =20 -### More Complex Cases +### Framework - More Complex Cases =20 -To write more advanced tests, first take a look at all the Assertion and L= ogging macros provided in the framework. +To write more advanced tests, first look at all the Assertion and Logging = macros provided in the framework. =20 Beyond that, if you're writing host-based tests and want to take a depende= ncy on the UnitTestFrameworkPkg, you can leverage the `cmocka.h` interface and write tests with all the features of= the Cmocka framework. @@ -227,6 +282,125 @@ leverage the `cmocka.h` interface and write tests wit= h all the features of the C Documentation for Cmocka can be found here: https://api.cmocka.org/ =20 +## GoogleTest Samples + +There is a sample unit test provided as both an example of how to write a = unit test and leverage +many of the GoogleTest features. This sample can be found in the `Test/Goo= gleTest/Sample/SampleGoogleTest` +directory. + +The sample is provided for the HOST_APPLICATION build type, which can be r= un on a host system without +needing a target. + +## GoogleTest Usage + +This section is built a lot like a "Getting Started". We'll go through som= e of the components that are needed +when constructing a unit test and some of the decisions that are made by t= he test writer. We'll also describe +how to check for expected conditions in test cases and a bit of the loggin= g characteristics. + +Most of these examples will refer to the SampleGoogleTestHost app found in= this package. + +### GoogleTest Requirements - INF + +In our INF file, we'll need to bring in the `GoogleTest` library. Convenie= ntly, the interface +header for the `GoogleTest` is in `UnitTestFrameworkPkg`, so you shouldn't= need to depend on any other +packages. As long as your DSC file knows where to find the lib implementat= ion that you want to use, +you should be good to go. + +See this example in 'SampleGoogleTestHost.inf'... + +``` +[Packages] + MdePkg/MdePkg.dec + UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec + +[LibraryClasses] + GoogleTestLib + BaseLib + DebugLib +``` + +Also, if you want you test to automatically be picked up by the Test Runne= r plugin, you will need +to make sure that the module `BASE_NAME` contains the word `Test`... + +``` +[Defines] + BASE_NAME =3D SampleGoogleTestHost +``` + +### GoogleTest Requirements - Code + +Not to state the obvious, but let's make sure we have the following includ= e before getting too far along... + +``` +#include +extern "C" { + #include + #include + #include +} +``` + +GoogleTest applications are implemented in C++. The first include brings i= n the +GoogleTest definitions. Other EDK II related include files must be wrapped= in +`extern "C" {}` because they are C include files. Link failures will occur= if +this is not done. + +Now that we've got that squared away, let's look at our 'Main()'' routine = (or DriverEntryPoint() or whatever). + +### GoogleTest Configuration + +Unlike the Framework, GoogleTest does not require test suites or test case= s to +be registered. Instead, the test cases declare the test suite name and test +case name as part of their implementation. The only requirement for Google= Test +is to have a `main()` function that initialize the GoogleTest infrastructu= re and +call the service `RUN_ALL_TESTS()` to run all the unit tests. + +```c +int main(int argc, char* argv[]) { + testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} +``` + +### GoogleTest - A Simple Test Case + +We'll look at the below test case from 'SampleGoogleTestHost'... + +```c +TEST(SimpleMathTests, OnePlusOneShouldEqualTwo) { + UINTN A; + UINTN B; + UINTN C; + + A =3D 1; + B =3D 1; + C =3D A + B; + + ASSERT_EQ (C, 2); +} +``` + +This uses the simplest form of a GoogleTest unit test using `TEST()` that +declares the test suite name and the unit test name within that test suite. +The unit test performs actions and typically makes calls to the code under= test +and contains test assertions to verify that the code under test behaves as +expected for the given inputs. + +In this test case, the `ASSERT_EQ` assertion is being used to establish th= at the business logic has functioned +correctly. There are several assertion macros, and you are encouraged to u= se one that matches as closely to your +intended test criterium as possible, because the logging is specific to th= e macro and more specific macros have more +detailed logs. When in doubt, there are always `ASSERT_TRUE` and `ASSERT_F= ALSE`. Assertion macros that fail their +test criterium will immediately return from the test case with a failed st= atus and log an error string. +_Note_ that this early return can have implications for memory leakage. + +There is no return status from a GooglTest unit test. If no assertions are +triggered then the unit test has a passing status. + +### GoogleTest - More Complex Cases + +To write more advanced tests, take a look at the +[GoogleTest User's Guide](http://google.github.io/googletest/). + ## Development =20 ### Iterating on a Single Test @@ -243,11 +417,11 @@ stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_T= AG=3DVS2017 -p MdePkg -t NOOP =20 ### Hooking BaseLib =20 -Most unit test mocking can be performed by the functions provided in the U= nitTestFramework libraries, but since +Most unit test mocking can be performed by the functions provided in the U= nitTestFrameworkPkg libraries, but since BaseLib is consumed by the Framework itself, it requires different techniq= ues to substitute parts of the functionality. =20 -To solve some of this, the UnitTestFramework consumes a special implementa= tion of BaseLib for host-based tests. +To solve some of this, the UnitTestFrameworkPkg consumes a special impleme= ntation of BaseLib for host-based tests. This implementation contains a [hook table](https://github.com/tianocore/e= dk2/blob/e188ecc8b4aed8fdd26b731d43883861f5e5e7b4/MdePkg/Test/UnitTest/Incl= ude/Library/UnitTestHostBaseLib.h#L507) that can be used to substitute test functionality for any of the BaseLib f= unctions. By default, this implementation will use the underlying BaseLib implementation, so the unit test writer on= ly has to supply minimal code to test a @@ -255,7 +429,7 @@ particular case. =20 ### Debugging the Framework Itself =20 -While most of the tests that are produced by the UnitTestFramework are eas= y to step through in a debugger, the Framework +While most of the tests that are produced by the UnitTestFrameworkPkg are = easy to step through in a debugger, the Framework itself consumes code (mostly Cmocka) that sets its own build flags. These = flags cause parts of the Framework to not export symbols and captures exceptions, and as such are harder to debug. W= e have provided a Stuart parameter to force symbolic debugging to be enabled. @@ -269,15 +443,17 @@ stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_T= AG=3DVS2019 -p MdePkg -t NOOP ## Building and Running Host-Based Tests =20 The EDK2 CI infrastructure provides a convenient way to run all host-based= tests -- in the the entire tree or just -selected packages -- and aggregate all the the reports, including highligh= ting any failures. This functionality is -provided through the Stuart build system (published by EDK2-PyTools) and t= he `NOOPT` build target. +selected packages -- and aggregate all the reports, including highlighting= any failures. This functionality is +provided through the Stuart build system (published by EDK2-PyTools) and t= he `NOOPT` build target. The sections that +follow use Framework examples. Unit tests based on GoogleTest are built an= d run the same way. The text output and +JUNIT XML output format have small differences. =20 ### Building Locally =20 First, to make sure you're working with the latest PyTools, run the follow= ing command: =20 ```bash -# Would recommend to run this in a Python venv, but that's out of scope fo= r this doc. +# Would recommend running this in a Python venv, but that's out of scope f= or this doc. python -m pip install --upgrade -r ./pip-requirements.txt ``` =20 @@ -361,7 +537,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite ``` =20 You can also, if you are so inclined, read the output from the exact insta= nce of the test that was run during -`stuart_ci_build`. The ouput file can be found on a path that looks like: +`stuart_ci_build`. The output file can be found on a path that looks like: =20 `Build//HostTest//...result.= xml` =20 @@ -389,22 +565,30 @@ c:\_uefi\MdePkg\Test\UnitTest\Library\BaseSafeIntLib\= TestBaseSafeIntLib.c:35: er =20 ### XML Reporting Mode =20 -Since these applications are built using the CMocka framework, they can al= so use the following env variables to output -in a structured XML rather than text: +Unit test applications using Framework are built using Cmocka that require= s the +following environment variables to be set to generate structured XML output +rather than text: =20 -```text +``` CMOCKA_MESSAGE_OUTPUT=3Dxml CMOCKA_XML_FILE=3D ``` =20 +Unit test applications using GoogleTest require the following environment +variable to be set to generate structured XML output rather than text: + +``` +GTEST_OUTPUT=3Dxml: +``` + This mode is used by the test running plugin to aggregate the results for = CI test status reporting in the web view. =20 ### Important Note =20 -This works on both Windows and Linux, but is currently limited to x64 arch= itectures. Working on getting others, but we +This works on both Windows and Linux but is currently limited to x64 archi= tectures. Working on getting others, but we also welcome contributions. =20 -## Known Limitations +## Framework Known Limitations =20 ### PEI, DXE, SMM =20 @@ -418,7 +602,7 @@ PEI, DXE, and SMM is forthcoming, but should be conside= red beta/staging for now. The host-based test framework is powered internally by the Cmocka framewor= k. As such, it has abilities that the target-based tests don't (yet). It would be awesome if this meant= that it was a super set of the target-based tests, and it worked just like the target-based tests but= with more features. Unfortunately, -this is not the case. While care has been taken to keep them as close a po= ssible, there are a few known +this is not the case. While care has been taken to keep them as close as p= ossible, there are a few known inconsistencies that we're still ironing out. For example, the logging mes= sages in the target-based tests are cached internally and associated with the running test case. They can = be saved later as part of the reporting lib. This isn't currently possible with host-based. Only the ass= ertion failures are logged. @@ -441,6 +625,9 @@ Non-Host-Based (PEI/DXE/SMM/Shell) Tests for a Function= ality or Feature | Simi ComponentY/ ComponentY.inf ComponentY.c + GoogleTest/ + ComponentYHostGoogleTest.inf # Host-Based Test for Driver Module + ComponentYGoogleTest.cpp UnitTest/ ComponentYHostUnitTest.inf # Host-Based Test for Driver Module ComponentYUnitTest.c @@ -455,11 +642,23 @@ Non-Host-Based (PEI/DXE/SMM/Shell) Tests for a Functi= onality or Feature | Simi SpecificLibDxe/ SpecificLibDxe.c SpecificLibDxe.inf + GoogleTest/ # Host-Based Test for Specific Librar= y Implementation + SpecificLibDxeHostGoogleTest.cpp + SpecificLibDxeHostGoogleTest.inf UnitTest/ # Host-Based Test for Specific Librar= y Implementation SpecificLibDxeHostUnitTest.c SpecificLibDxeHostUnitTest.inf Test/ HostTest.dsc # Host-Based Test Apps + GoogleTest/ + InterfaceX + InterfaceXHostGoogleTest.inf # Host-Based App (should be in Test/= HostTest.dsc) + InterfaceXUnitTest.cpp # Test Logic + + GeneralPurposeLib/ # Host-Based Test for any implementa= tion of GeneralPurposeLib + GeneralPurposeLibTest.cpp + GeneralPurposeLibHostUnitTest.inf + UnitTest/ InterfaceX InterfaceXHostUnitTest.inf # Host-Based App (should be in Test/= HostTest.dsc) diff --git a/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/S= ampleGoogleTest.cpp b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoo= gleTest/SampleGoogleTest.cpp new file mode 100644 index 000000000000..c83e58596a82 --- /dev/null +++ b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGo= ogleTest.cpp @@ -0,0 +1,263 @@ +/** @file + This is a sample to demonstrates the use of GoogleTest that supports host + execution environments. + + Copyright (c) 2022, Intel Corporation. All rights reserved.
+ SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#include +extern "C" { + #include + #include + #include +} + +/** + Sample unit test that verifies the expected result of an unsigned integer + addition operation. +**/ +TEST(SimpleMathTests, OnePlusOneShouldEqualTwo) { + UINTN A; + UINTN B; + UINTN C; + + A =3D 1; + B =3D 1; + C =3D A + B; + + ASSERT_EQ (C, (UINTN)2); +} + +/** + Sample unit test that verifies that a global BOOLEAN is updatable. +**/ +class GlobalBooleanVarTests : public ::testing::Test { + public: + BOOLEAN SampleGlobalTestBoolean =3D FALSE; +}; + +TEST_F(GlobalBooleanVarTests, GlobalBooleanShouldBeChangeable) { + SampleGlobalTestBoolean =3D TRUE; + ASSERT_TRUE (SampleGlobalTestBoolean); + + SampleGlobalTestBoolean =3D FALSE; + ASSERT_FALSE (SampleGlobalTestBoolean); +} + +/** + Sample unit test that logs a warning message and verifies that a global + pointer is updatable. +**/ +class GlobalVarTests : public ::testing::Test { + public: + VOID *SampleGlobalTestPointer =3D NULL; + + protected: + void SetUp() override { + ASSERT_EQ ((UINTN)SampleGlobalTestPointer, (UINTN)NULL); + } + void TearDown() { + SampleGlobalTestPointer =3D NULL; + } +}; + +TEST_F(GlobalVarTests, GlobalPointerShouldBeChangeable) { + SampleGlobalTestPointer =3D (VOID *)-1; + ASSERT_EQ ((UINTN)SampleGlobalTestPointer, (UINTN)((VOID *)-1)); +} + + +/** + Set PcdDebugPropertyMask for each MacroTestsAssertsEnabledDisabled test +**/ +class MacroTestsAssertsEnabledDisabled : public testing::TestWithParam { + void SetUp() { + PatchPcdSet8 (PcdDebugPropertyMask, GetParam()); + } +}; + +/** + Sample unit test using the ASSERT_TRUE() macro. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertTrue) { + UINT64 Result; + + // + // This test passes because expression always evaluated to TRUE. + // + ASSERT_TRUE (TRUE); + + // + // This test passes because expression always evaluates to TRUE. + // + Result =3D LShiftU64 (BIT0, 1); + ASSERT_TRUE (Result =3D=3D BIT1); +} + +/** + Sample unit test using the ASSERT_FALSE() macro. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertFalse) { + UINT64 Result; + + // + // This test passes because expression always evaluated to FALSE. + // + ASSERT_FALSE (FALSE); + + // + // This test passes because expression always evaluates to FALSE. + // + Result =3D LShiftU64 (BIT0, 1); + ASSERT_FALSE (Result =3D=3D BIT0); +} + +/** + Sample unit test using the ASSERT_EQ() macro. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertEqual) { + UINT64 Result; + + // + // This test passes because both values are always equal. + // + ASSERT_EQ (1, 1); + + // + // This test passes because both values are always equal. + // + Result =3D LShiftU64 (BIT0, 1); + ASSERT_EQ (Result, (UINT64)BIT1); +} + +/** + Sample unit test using the ASSERT_STREQ() macro. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertMemEqual) { + CHAR8 *String1; + CHAR8 *String2; + + // + // This test passes because String1 and String2 are the same. + // + String1 =3D (CHAR8 *)"Hello"; + String2 =3D (CHAR8 *)"Hello"; + ASSERT_STREQ (String1, String2); +} + +/** + Sample unit test using the ASSERT_NE() macro. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertNotEqual) { + UINT64 Result; + + // + // This test passes because both values are never equal. + // + ASSERT_NE (0, 1); + + // + // This test passes because both values are never equal. + // + Result =3D LShiftU64 (BIT0, 1); + ASSERT_NE (Result, (UINT64)BIT0); +} + +/** + Sample unit test using the ASSERT_TRUE() and ASSERT(FALSE) + and EFI_EFFOR() macros to check status +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertNotEfiError) { + // + // This test passes because the status is not an EFI error. + // + ASSERT_FALSE (EFI_ERROR (EFI_SUCCESS)); + + // + // This test passes because the status is not an EFI error. + // + ASSERT_FALSE (EFI_ERROR (EFI_WARN_BUFFER_TOO_SMALL)); +} + +/** + Sample unit test using the ASSERT_EQ() macro to compare EFI_STATUS value= s. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertStatusEqual) { + // + // This test passes because the status value are always equal. + // + ASSERT_EQ (EFI_SUCCESS, EFI_SUCCESS); +} + +/** + Sample unit test using ASSERT_NE() macro to make sure a pointer is not N= ULL. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroAssertNotNull) { + UINT64 Result; + + // + // This test passes because the pointer is never NULL. + // + ASSERT_NE (&Result, (UINT64 *)NULL); +} + +/** + Sample unit test using that should not generate any ASSERTs() +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroExpectNoAssertFailure) { + // + // This test passes because it never triggers an ASSERT(). + // + ASSERT (TRUE); + + // + // This test passes because DecimalToBcd() does not ASSERT() if the + // value passed in is <=3D 99. + // + DecimalToBcd8 (99); +} + +/** + Sample unit test using the ASSERT_DEATH() macro to test expected ASSERT(= )s. +**/ +TEST_P(MacroTestsAssertsEnabledDisabled, MacroExpectAssertFailure) { + // + // Skip tests that verify an ASSERT() is triggered if ASSERT()s are disa= bled. + // + if ((PcdGet8 (PcdDebugPropertyMask) & BIT0) =3D=3D 0x00) { + return; + } + + // + // This test passes because it directly triggers an ASSERT(). + // + ASSERT_DEATH (ASSERT (FALSE), ""); + + // + // This test passes because DecimalToBcd() generates an ASSERT() if the + // value passed in is >=3D 100. The expected ASSERT() is caught by the = unit + // test framework and ASSERT_DEATH() returns without an error. + // + ASSERT_DEATH (DecimalToBcd8 (101), ""); +} + +INSTANTIATE_TEST_SUITE_P(ValidInput, + MacroTestsAssertsEnabledDisabled, + ::testing::Values(PcdGet8 (PcdDebugPropertyMask) = | BIT0, PcdGet8 (PcdDebugPropertyMask) & (~BIT0))); + +/** + Sample unit test using the SCOPED_TRACE() macro for trace messages. +**/ +TEST(MacroTestsMessages, MacroTraceMessage) { + // + // Example of logging. + // + SCOPED_TRACE ("SCOPED_TRACE message\n"); +} + +int main(int argc, char* argv[]) { + testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/S= ampleGoogleTestHost.inf b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/Sampl= eGoogleTest/SampleGoogleTestHost.inf new file mode 100644 index 000000000000..37e7c86910ed --- /dev/null +++ b/UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGo= ogleTestHost.inf @@ -0,0 +1,35 @@ +## @file +# This is a sample to demonstrates the use of GoogleTest that supports host +# execution environments. +# +# Copyright (c) 2022, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent +## + +[Defines] + INF_VERSION =3D 0x00010005 + BASE_NAME =3D SampleGoogleTestHost + FILE_GUID =3D 7D8BBFBB-7977-4AEE-A59F-257BF5C2F87C + MODULE_TYPE =3D HOST_APPLICATION + VERSION_STRING =3D 1.0 + +# +# The following information is for reference only and not required by the = build tools. +# +# VALID_ARCHITECTURES =3D IA32 X64 +# + +[Sources] + SampleGoogleTest.cpp + +[Packages] + MdePkg/MdePkg.dec + UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec + +[LibraryClasses] + GoogleTestLib + BaseLib + DebugLib + +[Pcd] + gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask diff --git a/UnitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc b/U= nitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc index 184fdec87acf..708ef7f9ab35 100644 --- a/UnitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc +++ b/UnitTestFrameworkPkg/Test/UnitTestFrameworkPkgHostTest.dsc @@ -23,14 +23,16 @@ [PcdsPatchableInModule] =20 [Components] # - # Build HOST_APPLICATION that tests the SampleUnitTest + # Build HOST_APPLICATIONs that test the SampleUnitTest # UnitTestFrameworkPkg/Test/UnitTest/Sample/SampleUnitTest/SampleUnitTestH= ost.inf + UnitTestFrameworkPkg/Test/GoogleTest/Sample/SampleGoogleTest/SampleGoogl= eTestHost.inf =20 # # Build HOST_APPLICATION Libraries # UnitTestFrameworkPkg/Library/CmockaLib/CmockaLib.inf + UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.inf UnitTestFrameworkPkg/Library/Posix/DebugLibPosix/DebugLibPosix.inf UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAlloca= tionLibPosix.inf UnitTestFrameworkPkg/Library/UnitTestLib/UnitTestLibCmocka.inf diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFr= ameworkPkg/UnitTestFrameworkPkg.ci.yaml index 77d51e13484c..072df6208c92 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml @@ -78,7 +78,8 @@ "SpellCheck": { "AuditOnly": False, # Fails test but run in AuditOnly mo= de to collect log "IgnoreFiles": [ # use gitignore syntax to ignore erro= rs in matching files - "Library/CmockaLib/cmocka/**/*.*" # not going to spell check = a submodule + "Library/CmockaLib/cmocka/**/*.*", # not going to spell check= a submodule + "Library/GoogleTestLib/googletest/**/*.*" # not going to spel= l check a submodule ], "ExtendWords": [ # words to extend to the dictionary f= or this package "testcase", @@ -91,6 +92,7 @@ "NOFAILURE", "cmockery", "DHAVE", # build flag for cmocka in the INF + "gtest", # file name in GoogleTestLib.inf "corthon", # Contact GitHub account in Readme "mdkinney", # Contact GitHub account in Readme "spbrogan" # Contact GitHub account in Readme diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec b/UnitTestFramew= orkPkg/UnitTestFrameworkPkg.dec index 069289f00969..ed12f32009d8 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec @@ -16,11 +16,15 @@ [Defines] PACKAGE_VERSION =3D 1.00 =20 [Includes] + Include Library/CmockaLib/cmocka/include + Library/GoogleTestLib/googletest/googletest/include + Library/GoogleTestLib/googletest/googlemock/include =20 [Includes.Common.Private] PrivateInclude Library/CmockaLib/cmocka/include/cmockery + Library/GoogleTestLib/googletest/googletest =20 [LibraryClasses.Common.Private] ## @libraryclass Allows save and restore unit test internal state @@ -35,6 +39,10 @@ [LibraryClasses.Common.Private] # UnitTestBootLib|PrivateInclude/Library/UnitTestBootLib.h =20 + ## @libraryclass GoogleTest infrastructure + # + GoogleTestLib|Include/Library/GoogleTestLib.h + [Guids] gUnitTestFrameworkPkgTokenSpaceGuid =3D { 0x833d3aba, 0x39b4, 0x43a2, { = 0xb9, 0x30, 0x7a, 0x34, 0x53, 0x39, 0x31, 0xb3 } } =20 diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTe= stFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc index 9beeaef1ba5e..8009337552cc 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc @@ -14,6 +14,7 @@ [LibraryClasses.common.HOST_APPLICATION] CpuLib|MdePkg/Library/BaseCpuLibNull/BaseCpuLibNull.inf CacheMaintenanceLib|MdePkg/Library/BaseCacheMaintenanceLibNull/BaseCache= MaintenanceLibNull.inf CmockaLib|UnitTestFrameworkPkg/Library/CmockaLib/CmockaLib.inf + GoogleTestLib|UnitTestFrameworkPkg/Library/GoogleTestLib/GoogleTestLib.i= nf UnitTestLib|UnitTestFrameworkPkg/Library/UnitTestLib/UnitTestLibCmocka.i= nf DebugLib|UnitTestFrameworkPkg/Library/Posix/DebugLibPosix/DebugLibPosix.= inf MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationL= ibPosix/MemoryAllocationLibPosix.inf @@ -31,6 +32,7 @@ [BuildOptions.common.EDKII.HOST_APPLICATION] # # MSFT # + MSFT:*_*_*_CC_FLAGS =3D /EHsc MSFT:*_*_*_DLINK_FLAGS =3D=3D /out:"$(BIN_DIR)\$(MODULE_NAME_= GUID).exe" /pdb:"$(BIN_DIR)\$(MODULE_NAME_GUID).pdb" /IGNORE:4001 /NOLOGO /= SUBSYSTEM:CONSOLE /DEBUG /STACK:0x40000,0x40000 /NODEFAULTLIB:libcmt.lib li= bcmtd.lib MSFT:*_*_IA32_DLINK_FLAGS =3D /MACHINE:I386 MSFT:*_*_X64_DLINK_FLAGS =3D /MACHINE:AMD64 @@ -50,7 +52,7 @@ [BuildOptions.common.EDKII.HOST_APPLICATION] # GCC:*_*_IA32_DLINK_FLAGS =3D=3D -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m32 -= no-pie GCC:*_*_X64_DLINK_FLAGS =3D=3D -o $(BIN_DIR)/$(MODULE_NAME_GUID) -m64 -= no-pie - GCC:*_*_*_DLINK2_FLAGS =3D=3D -lgcov + GCC:*_*_*_DLINK2_FLAGS =3D=3D -lgcov -lpthread -lstdc++ -lm =20 # # Need to do this link via gcc and not ld as the pathing to libraries ch= anges from OS version to OS version --=20 2.37.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#96116): https://edk2.groups.io/g/devel/message/96116 Mute This Topic: https://groups.io/mt/94902392/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-