From nobody Wed May 15 09:57:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+94547+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+94547+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1664502834; cv=none; d=zohomail.com; s=zohoarc; b=aoR+C6yQm3gk1NntfeN/Gz6aaJlbkr+FiNBxexuIyOUWPDlr8nopXOP1kAzjQIOnEJ7mcfUby68o5RVvQtjSHjMejqsK2NlI1Y6tJoWRcAsI9uiuiV8IiSkTvNoXo8oUWS6ga4d6NTBS6YX1QqgSH3lz4SZJ0sLnkiuVMx+5thM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1664502834; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=JZmblxvukzsVDiJ9lzYIctjTnxtYNw0Y3GwjMSsstoo=; b=QXlTym+hD1DTxvRGf2e5QJkNeTUOsFrim4xtIuIIXGspSpWnH8twrV3Fvrffa2GUvsqiJDvsMOw4s6ZuIJyzEeYRVKbZ7eli3RtGIxsA1rVVDFz7z2+DKmIsBsBNF53YSMs2jyetPYcux56Xtef63ZIHVsM0vMOjvcDYA3/LuXI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+94547+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 16645028340934.903687833826439; Thu, 29 Sep 2022 18:53:54 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id sXu2YY1788612xVmsyE9GA0D; Thu, 29 Sep 2022 18:53:53 -0700 X-Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web10.1110.1664502832031588201 for ; Thu, 29 Sep 2022 18:53:53 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="302991064" X-IronPort-AV: E=Sophos;i="5.93,357,1654585200"; d="scan'208";a="302991064" X-Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2022 18:53:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="685098063" X-IronPort-AV: E=Sophos;i="5.93,357,1654585200"; d="scan'208";a="685098063" X-Received: from gguo-desk.gar.corp.intel.com ([10.5.215.23]) by fmsmga008.fm.intel.com with ESMTP; 29 Sep 2022 18:53:51 -0700 From: "Guo, Gua" To: devel@edk2.groups.io Cc: Gua Guo , Michael D Kinney , Sean Brogan , Bret Barkelew , Michael Kubacki Subject: [edk2-devel] [PATCH 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC Date: Fri, 30 Sep 2022 09:53:46 +0800 Message-Id: <3e761ca731604e3eb08664309a2c16a56c5e472f.1664502701.git.gua.guo@intel.com> In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,gua.guo@intel.com X-Gm-Message-State: V6dCedXlscw4n9L1zvHT607Qx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1664502833; bh=LndAkSfm/ql9SsiMYHFP0rsnGI1s93NQHbsmCw6n+Zs=; h=Cc:Date:From:Reply-To:Subject:To; b=DIVv0Erqxkp98Fty6EfFZiI95HePhsO2YYpORz2dLW1j4AaVlyes47Nvznid0RICwbx r1hUnlbXahtaqZlC4u26gzNG3JCGRKXszILA5ypqE2nUjyH3pql2iMY/FgUGcaDFgi2Ap 0bq62xkm/wfUfFaZkI6s0ZLJL2lJuyvMqKw= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1664502834406100003 Content-Type: text/plain; charset="utf-8" From: Gua Guo In order to collect code coverage after running executable file, generate *.gcda and *.gcno file that require by lcov tool to generate code coverage report. Cc: Michael D Kinney Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael Kubacki Signed-off-by: Gua Guo Reviewed-by: Michael Kubacki --- UnitTestFrameworkPkg/ReadMe.md | 37 +++++++++++++++++-- .../UnitTestFrameworkPkg.ci.yaml | 2 + .../UnitTestFrameworkPkgHost.dsc.inc | 3 +- 3 files changed, 38 insertions(+), 4 deletions(-) diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md index e696412cb3..c00e2411f8 100644 --- a/UnitTestFrameworkPkg/ReadMe.md +++ b/UnitTestFrameworkPkg/ReadMe.md @@ -60,7 +60,7 @@ you should be good to go. =20 See this example in 'SampleUnitTestUefiShell.inf'... =20 -``` +```inf [Packages] MdePkg/MdePkg.dec =20 @@ -75,7 +75,7 @@ See this example in 'SampleUnitTestUefiShell.inf'... Also, if you want you test to automatically be picked up by the Test Runne= r plugin, you will need to make sure that the module `BASE_NAME` contains the word `Test`... =20 -``` +```inf [Defines] BASE_NAME =3D SampleUnitTestUefiShell ``` @@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite ``` =20 You can also, if you are so inclined, read the output from the exact insta= nce of the test that was run during -`stuart_ci_build`. The ouput file can be found on a path that looks like: +`stuart_ci_build`. The output file can be found on a path that looks like: =20 `Build//HostTest//...result.= xml` =20 @@ -399,6 +399,37 @@ CMOCKA_XML_FILE=3D =20 This mode is used by the test running plugin to aggregate the results for = CI test status reporting in the web view. =20 +### Code Coverage + +Host based Unit Tests will automatically enable coverage data. + +For Windows, This is primarily leverage for pipeline builds, but this can = be leveraged locally using the +OpenCppCoverage windows tool, and pycobertura is used to covert this cover= age data to a human readable HTML +file. These tools must be installed to parse code coverage. + +- Windows Prerequisite + ```bash + Download and install https://github.com/OpenCppCoverage/OpenCppCoverage/= tags + python -m pip install --upgrade -r ./pip-requirements.txt + stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=3DVS20= 19 -p MdeModulePkg + Open Build/coverage.html + ``` + + +For Linux, This is primarily leveraged for pipeline builds, but this can b= e leveraged locally using the +lcov linux tool, and parsed using the lcov_cobertura python tool. pycobert= ura is used to +covert this coverage data to a human readable HTML file. These tools must = be installed +to parse code coverage. + +- Linux Prerequisite + ```bash + sudo apt-get install -y lcov + python -m pip install --upgrade -r ./pip-requirements.txt + stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=3DGCC5= -p MdeModulePkg + Open Build/coverage.html + ``` + + ### Important Note =20 This works on both Windows and Linux, but is currently limited to x64 arch= itectures. Working on getting others, but we diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFr= ameworkPkg/UnitTestFrameworkPkg.ci.yaml index 77d51e1348..f68882e1cf 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml @@ -90,6 +90,8 @@ "pytools", "NOFAILURE", "cmockery", + "cobertura", + "pycobertura", "DHAVE", # build flag for cmocka in the INF "corthon", # Contact GitHub account in Readme "mdkinney", # Contact GitHub account in Readme diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTe= stFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc index 4dd8d4ac67..ca46b8513a 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc @@ -25,7 +25,8 @@ GCC:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1 XCODE:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1 !endif - + GCC:*_GCC5_*_CC_FLAGS =3D --coverage + GCC:*_GCC5_*_DLINK_FLAGS =3D --coverage [BuildOptions.common.EDKII.HOST_APPLICATION] # # MSFT --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#94547): https://edk2.groups.io/g/devel/message/94547 Mute This Topic: https://groups.io/mt/94008685/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Wed May 15 09:57:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+94548+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+94548+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1664502835; cv=none; d=zohomail.com; s=zohoarc; b=jIbGQkCZeZ5DICF90x5fuRoI+oiNrdKqh1ZmI7ROtJDNuJ5OxiuNBNAhDwGmCcYuJalOdBKTcYCEedaaHVFMNwQl6iPaBNoMRQTyi21QqrFfJpZBlZBJBifCNIDu9PCvyFQxV+BDi/TqrWyC+7lbB6oozlCXpWUVhtY39wlp9SM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1664502835; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=uDPWy323J+34sROoTfyrwpTs9Y/qOiGDES1azyRSdro=; b=KxIUUejkYTHn3jY1eeHH+nwLX4GOViPvBhKJzzVUsGCaMeSZUWMPYPXEmtGlYpEbzoipqVkvg437tgtlg6MSKzgQrUVYJVxGLtLy4nejW3N1UeGSwD7AOFfDQSxmn1VB3gGzLAn3Psiz+p5fb5SzuY1p1Cxtka128rSw4+DD8LQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+94548+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 166450283570159.97752192007374; Thu, 29 Sep 2022 18:53:55 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id 2H8tYY1788612xCniXprqNe5; Thu, 29 Sep 2022 18:53:55 -0700 X-Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web10.1110.1664502832031588201 for ; Thu, 29 Sep 2022 18:53:54 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="302991079" X-IronPort-AV: E=Sophos;i="5.93,357,1654585200"; d="scan'208";a="302991079" X-Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2022 18:53:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="685098071" X-IronPort-AV: E=Sophos;i="5.93,357,1654585200"; d="scan'208";a="685098071" X-Received: from gguo-desk.gar.corp.intel.com ([10.5.215.23]) by fmsmga008.fm.intel.com with ESMTP; 29 Sep 2022 18:53:52 -0700 From: "Guo, Gua" To: devel@edk2.groups.io Cc: Gua Guo , Bob Feng , Bret Barkelew , Liming Gao , Michael D Kinney , Sean Brogan Subject: [edk2-devel] [PATCH 2/3] BaseTools/Plugin: Add coverage support for Unit Test Date: Fri, 30 Sep 2022 09:53:47 +0800 Message-Id: <487d272749228714039d4da8b4f48d66533383ec.1664502701.git.gua.guo@intel.com> In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,gua.guo@intel.com X-Gm-Message-State: 1wQuflLFKUQJEuD0uWn1TnVbx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1664502835; bh=CboH2VSmX46Ku2zJ3ENTmrqVze7XoMeBStFu0D+Kpj4=; h=Cc:Date:From:Reply-To:Subject:To; b=JcOa3Gy9mfrCrrUgPGG1M3os/IbhinRLUnnRKFHt+0RjXRsUn/t10POe1jzb1BbQDaR LhIZBg4buzYtrSo3Wz8qFuVscFNEGrqKOXcfaI3XtMHe7z+JfmgNr8FmMOVimi/oUk0eL CPK61iiT9aIfGdsXHHC4MJ34vvLh+QGBLJs= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1664502836460100010 Content-Type: text/plain; charset="utf-8" From: Gua Guo For GCC, use lcov to generate Unit Test code coverage report For VS2019, use OpenCppCoverage to generate code coverage report Cc: Bob Feng Cc: Bret Barkelew Cc: Liming Gao Cc: Michael D Kinney Cc: Sean Brogan Signed-off-by: Gua Guo Reviewed-by: Michael Kubacki --- .../HostBasedUnitTestRunner.py | 119 ++++++++++++++++++ 1 file changed, 119 insertions(+) diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunn= er.py b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py index c1eeaf2625..d92de236dc 100644 --- a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py +++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py @@ -112,4 +112,123 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin): " %s - %s" % (case.attrib['na= me'], result.text)) failure_count +=3D 1 =20 + if thebuilder.env.GetValue("TOOL_CHAIN_TAG") =3D=3D "GCC5": + self.gen_code_coverage_gcc(thebuilder) + elif thebuilder.env.GetValue("TOOL_CHAIN_TAG") =3D=3D "VS2019": + self.gen_code_coverage_msvc(thebuilder) + else: + logging.info("Skipping code coverage. Only supported on GC= C.") + return failure_count + + def gen_code_coverage_gcc(self, thebuilder): + logging.info("Generating UnitTest code coverage") + + buildOutputBase =3D thebuilder.env.GetValue("BUILD_OUTPUT_BASE") + workspace =3D thebuilder.env.GetValue("WORKSPACE") + + # Generate base code coverage for all source files + ret =3D RunCmd("lcov", f"--no-external --capture --initial --direc= tory {buildOutputBase} --output-file {buildOutputBase}/cov-base.info --rc l= cov_branch_coverage=3D1") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to build initial cove= rage data.") + return 1 + + # Coverage data for tested files only + ret =3D RunCmd("lcov", f"--capture --directory {buildOutputBase}/ = --output-file {buildOutputBase}/coverage-test.info --rc lcov_branch_coverag= e=3D1") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to build coverage dat= a for tested files.") + return 1 + + # Aggregate all coverage data + ret =3D RunCmd("lcov", f"--add-tracefile {buildOutputBase}/cov-bas= e.info --add-tracefile {buildOutputBase}/coverage-test.info --output-file {= buildOutputBase}/total-coverage.info --rc lcov_branch_coverage=3D1") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to aggregate coverage= data.") + return 1 + + # Generate coverage XML + ret =3D RunCmd("lcov_cobertura",f"{buildOutputBase}/total-coverage= .info -o {buildOutputBase}/compare.xml") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to generate coverage = XML.") + return 1 + + # Filter out auto-generated and test code + ret =3D RunCmd("lcov_cobertura",f"{buildOutputBase}/total-coverage= .info --excludes ^.*UnitTest\|^.*MU\|^.*Mock\|^.*DEBUG -o {buildOutputBase}= /coverage.xml") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed generate filtered cov= erage XML.") + return 1 + + # Generate all coverage file + testCoverageList =3D glob.glob (f"{workspace}/Build/**/total-cover= age.info", recursive=3DTrue) + + coverageFile =3D "" + for testCoverage in testCoverageList: + coverageFile +=3D " --add-tracefile " + testCoverage + ret =3D RunCmd("lcov", f"{coverageFile} --output-file {workspace}/= Build/all-coverage.info --rc lcov_branch_coverage=3D1") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed generate all coverage= file.") + return 1 + + # Generate and HTML file if requested.by each package + ret =3D RunCmd("pycobertura", f"show --format html --output {build= OutputBase}/coverage.html {buildOutputBase}/coverage.xml --source {workspac= e}") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to generate HTML in s= ingle package..") + + # Generate and HTML file if requested.for all package + if os.path.isfile(f"{workspace}/Build/coverage.xml"): + os.remove(f"{workspace}/Build/coverage.xml") + ret =3D RunCmd("lcov_cobertura",f"{workspace}/Build/all-coverage.i= nfo --excludes ^.*UnitTest\|^.*MU\|^.*Mock\|^.*DEBUG -o {workspace}/Build/c= overage.xml") + + if os.path.isfile(f"{workspace}/Build/coverage.html"): + os.remove(f"{workspace}/Build/coverage.html") + ret =3D RunCmd("pycobertura", f"show --format html --output {works= pace}/Build/coverage.html {workspace}/Build/coverage.xml --source {workspac= e}") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to generate HTML.") + + return 0 + + + def gen_code_coverage_msvc(self, thebuilder): + logging.info("Generating UnitTest code coverage") + + + buildOutputBase =3D thebuilder.env.GetValue("BUILD_OUTPUT_BASE") + testList =3D glob.glob(os.path.join(buildOutputBase, "**","*Test*.= exe"), recursive=3DTrue) + workspace =3D thebuilder.env.GetValue("WORKSPACE") + + # Generate coverage file + coverageFile =3D "" + for testFile in testList: + ret =3D RunCmd("OpenCppCoverage", f"--source {workspace} --exp= ort_type binary:{testFile}.cov -- {testFile}") + coverageFile +=3D " --input_coverage=3D" + testFile + ".cov" + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to collect covera= ge data.") + return 1 + + # Generate and HTML file if requested.by each package + ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{build= OutputBase}/coverage.xml --working_dir=3D{workspace}/Build {coverageFile}") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to generate cobertura= format xml in single package.") + return 1 + + ret =3D RunCmd("pycobertura", f"show --format html --output {build= OutputBase}/cverage.html {buildOutputBase}/coverage.xml --source {workspace= }") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to generate HTML in s= ingle package.") + return 1 + + # Generate total report HTML file for all package + testCoverageList =3D glob.glob(os.path.join(workspace, "Build", "*= *","*Test*.exe.cov"), recursive=3DTrue) + coverageFile =3D "" + for testCoverage in testCoverageList: + coverageFile +=3D " --input_coverage=3D" + testCoverage + + ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{works= pace}/Build/coverage.xml --working_dir=3D{workspace}/Build {coverageFile}") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to generate cobertura= format xml.") + return 1 + + ret =3D RunCmd("pycobertura", f"show --format html --output {works= pace}/Build/coverage.html {workspace}/Build/coverage.xml --source {workspac= e}") + if(ret !=3D 0): + logging.error("UnitTest Coverage: Failed to generate HTML.") + return 1 + + return 0 --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#94548): https://edk2.groups.io/g/devel/message/94548 Mute This Topic: https://groups.io/mt/94008686/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Wed May 15 09:57:41 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+94549+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+94549+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1664502836; cv=none; d=zohomail.com; s=zohoarc; b=Cm8SuTkS1mHnwyrW+Vdav5p6K+Q1Qv2dCjHvfQ+uwYfek8P0JJTAxGQ6Kt16W8HEo6qtiNAlwXNjTmAbVoXnBmCbDcyBzDo4D5jEomy+Moq6rQM391uVefxreVCwPUwdM269r0a1H4QrcLd+p1ZjIS9fuDkEWSCJ+I/iYQBLfDY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1664502836; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=RRlBkGiD1F0h1JxH11DVaaIfAke8AQlNuGhCseuWWA0=; b=aWcvcDJRzW80z9vgER++E37TBhIsiaVU6OAj4IvmDngX4VWba9JM8sLEhu+N8OmJWkBShSabjDd4jFlFW6YHMCpPZu9xCHLY/ynXS4O01BWigowyHIBX1YgzrM65ytYg1axm2QfVJR5dd/g5IB1UENmKXCOEdRwKJ0XDHdRt/Fw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+94549+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1664502836924359.1178648709732; Thu, 29 Sep 2022 18:53:56 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id PWdkYY1788612xxfNp36HsDN; Thu, 29 Sep 2022 18:53:56 -0700 X-Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web10.1110.1664502832031588201 for ; Thu, 29 Sep 2022 18:53:56 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="302991080" X-IronPort-AV: E=Sophos;i="5.93,357,1654585200"; d="scan'208";a="302991080" X-Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2022 18:53:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="685098075" X-IronPort-AV: E=Sophos;i="5.93,357,1654585200"; d="scan'208";a="685098075" X-Received: from gguo-desk.gar.corp.intel.com ([10.5.215.23]) by fmsmga008.fm.intel.com with ESMTP; 29 Sep 2022 18:53:54 -0700 From: "Guo, Gua" To: devel@edk2.groups.io Cc: Gua Guo , Sean Brogan , Bret Barkelew , Michael D Kinney , Liming Gao Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Install code coverage tool Date: Fri, 30 Sep 2022 09:53:48 +0800 Message-Id: <4640dcd72927df92d5304411c9a872acd97bf76c.1664502701.git.gua.guo@intel.com> In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,gua.guo@intel.com X-Gm-Message-State: 2vdovMVd4dTdVzEL6urcyiVrx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1664502836; bh=xmUp6ca5b4JZ+0quICDW1YgXqbOmFO+9k06ctOKcuAE=; h=Cc:Date:From:Reply-To:Subject:To; b=dpR6VNk339f8uxWOo/AfbEfbdYVkRm7aLY9sStDOtnvwMMYeWNE/GrLQu9Bdd6+y8wn VFMr9MSd6LSHgGgHw0hikZqppGmEt/p7x1oB2ObjYxscqx+RgjteOWQhcfg+HoKfvUP51 MOh+0uEnnVGsjufTmc9vYedhaJcOIeZYYNo= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1664502838461100014 Content-Type: text/plain; charset="utf-8" From: Gua Guo For Windows add below tool for code coverage 1. OpenCppCoverage: parsing pdb file to generate coverage data 2. pycobertura: show up html format data for coverage data For Linux add below tool for code coverage 1. lcov: parsing gcda gcno file to generate coverage data 2. lcov-cobertura: convert coverage data to cobertura format 3. pycobertura: show up html format data for coverage data Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael D Kinney Cc: Liming Gao Signed-off-by: Gua Guo --- .azurepipelines/Ubuntu-GCC5.yml | 5 ++++- .azurepipelines/Windows-VS2019.yml | 5 +++++ .azurepipelines/templates/pr-gate-build-job.yml | 2 ++ .azurepipelines/templates/pr-gate-steps.yml | 5 +++++ .../HostBasedUnitTestRunner.py | 12 ++++++------ pip-requirements.txt | 3 +++ 6 files changed, 25 insertions(+), 7 deletions(-) diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.= yml index 3760c6efe1..9d53528063 100644 --- a/.azurepipelines/Ubuntu-GCC5.yml +++ b/.azurepipelines/Ubuntu-GCC5.yml @@ -18,4 +18,7 @@ jobs: tool_chain_tag: 'GCC5' vm_image: 'ubuntu-latest' arch_list: "IA32,X64,ARM,AARCH64,RISCV64" - + extra_install_step: + - bash: sudo apt-get install -y lcov + displayName: Install Code Coverage Tools + condition: and(gt(variables.pkg_count, 0), succeeded()) diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-V= S2019.yml index e4bd4b1d22..c07e5bb434 100644 --- a/.azurepipelines/Windows-VS2019.yml +++ b/.azurepipelines/Windows-VS2019.yml @@ -18,3 +18,8 @@ jobs: tool_chain_tag: 'VS2019' vm_image: 'windows-2019' arch_list: "IA32,X64" + extra_install_step: + - powershell: choco install opencppcoverage; Write-Host "##vso[task.pr= ependpath]C:\Program Files\OpenCppCoverage" + displayName: Install Code Coverage Tool + condition: and(gt(variables.pkg_count, 0), succeeded()) + diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipeli= nes/templates/pr-gate-build-job.yml index 0e4ad019bf..0162ea97cb 100644 --- a/.azurepipelines/templates/pr-gate-build-job.yml +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -12,6 +12,7 @@ parameters: tool_chain_tag: '' vm_image: '' arch_list: '' + extra_install_step: [] =20 # Build step jobs: @@ -70,3 +71,4 @@ jobs: build_pkgs: $(Build.Pkgs) build_targets: $(Build.Targets) build_archs: ${{ parameters.arch_list }} + extra_install_step: ${{ parameters.extra_install_step }} diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/= templates/pr-gate-steps.yml index cb431e53fc..0568941399 100644 --- a/.azurepipelines/templates/pr-gate-steps.yml +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -12,6 +12,7 @@ parameters: build_pkgs: '' build_targets: '' build_archs: '' + extra_install_step: [] =20 steps: - checkout: self @@ -37,6 +38,8 @@ steps: displayName: fetch target branch condition: eq(variables['Build.Reason'], 'PullRequest') =20 +- ${{ parameters.extra_install_step }} + # trim the package list if this is a PR - task: CmdLine@1 displayName: Check if ${{ parameters.build_pkgs }} need testing @@ -125,6 +128,8 @@ steps: TestSuites.xml **/BUILD_TOOLS_REPORT.html **/OVERRIDELOG.TXT + coverage.xml + coverage.html flattenFolders: true condition: succeededOrFailed() =20 diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunn= er.py b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py index d92de236dc..c2821cde7d 100644 --- a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py +++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py @@ -194,7 +194,7 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin): buildOutputBase =3D thebuilder.env.GetValue("BUILD_OUTPUT_BASE") testList =3D glob.glob(os.path.join(buildOutputBase, "**","*Test*.= exe"), recursive=3DTrue) workspace =3D thebuilder.env.GetValue("WORKSPACE") - + workspace =3D (workspace + os.sep) if workspace[-1] !=3D os.sep el= se workspace # Generate coverage file coverageFile =3D "" for testFile in testList: @@ -204,13 +204,14 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin): logging.error("UnitTest Coverage: Failed to collect covera= ge data.") return 1 =20 + DiskName =3D workspace[:workspace.find (":\\") + 2] # Generate and HTML file if requested.by each package - ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{build= OutputBase}/coverage.xml --working_dir=3D{workspace}/Build {coverageFile}") + ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{build= OutputBase}/coverage.xml --working_dir=3D{workspace}Build {coverageFile}") if(ret !=3D 0): logging.error("UnitTest Coverage: Failed to generate cobertura= format xml in single package.") return 1 =20 - ret =3D RunCmd("pycobertura", f"show --format html --output {build= OutputBase}/cverage.html {buildOutputBase}/coverage.xml --source {workspace= }") + ret =3D RunCmd("pycobertura", f"show --format html --output {build= OutputBase}/coverage.html {buildOutputBase}/coverage.xml --source {DiskName= }") if(ret !=3D 0): logging.error("UnitTest Coverage: Failed to generate HTML in s= ingle package.") return 1 @@ -221,12 +222,11 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin): for testCoverage in testCoverageList: coverageFile +=3D " --input_coverage=3D" + testCoverage =20 - ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{works= pace}/Build/coverage.xml --working_dir=3D{workspace}/Build {coverageFile}") + ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{works= pace}Build/coverage.xml --working_dir=3D{workspace}Build {coverageFile}") if(ret !=3D 0): logging.error("UnitTest Coverage: Failed to generate cobertura= format xml.") return 1 - - ret =3D RunCmd("pycobertura", f"show --format html --output {works= pace}/Build/coverage.html {workspace}/Build/coverage.xml --source {workspac= e}") + ret =3D RunCmd("pycobertura", f"show --format html --output {works= pace}Build/coverage.html {workspace}Build/coverage.xml --source {DiskName}") if(ret !=3D 0): logging.error("UnitTest Coverage: Failed to generate HTML.") return 1 diff --git a/pip-requirements.txt b/pip-requirements.txt index 967da7cb37..18f5afb9c5 100644 --- a/pip-requirements.txt +++ b/pip-requirements.txt @@ -16,3 +16,6 @@ edk2-pytool-library=3D=3D0.11.2 edk2-pytool-extensions~=3D0.16.0 edk2-basetools=3D=3D0.1.29 antlr4-python3-runtime=3D=3D4.7.1 +pycobertura=3D=3D2.1.0 +lcov-cobertura=3D=3D2.0.2 + --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#94549): https://edk2.groups.io/g/devel/message/94549 Mute This Topic: https://groups.io/mt/94008687/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-