From nobody Sun May 19 00:47:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+97832+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+97832+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1672658289; cv=none; d=zohomail.com; s=zohoarc; b=W/rz3sdSwWaGMWfIScKyh/G6VzS4ku0qCvnG8I5awMG1VFiI3frHvJX+x56HoyTyajX1MCcl5ZXJGKKMVjyC1fCvx7sARz2SqBohTYKyRGYAND7cdAh3r+0zgK6ZjeCHxxYYHpW/gwzgSYE7EkMf9dcLqeyv9A6nvljeROYa8Z4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1672658289; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=0ieu4Cjz4U1Qh+TlC83zCX5+UuYoBhj1hVDW5zXKnYE=; b=GpMQmmZbFox5tqpyoau8ls9jLpO2Eyrhr1oXtOky2mhjof0hzo/a20FvDiRLtF7Ka3MFqsldulQXCowG7A8L/9lUwyaW7i0Eg1v0MWXLgcW8YjfI8VgLL+AEy8Gb1VJGuahD/tdMQmBbYwhpV/P2Gb5hKUfg+aAweoSOaXzDaqo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+97832+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1672658289776870.2277279259367; Mon, 2 Jan 2023 03:18:09 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id m1WHYY1788612xRCCyVmUKsE; Mon, 02 Jan 2023 03:18:09 -0800 X-Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mx.groups.io with SMTP id smtpd.web11.33235.1672658287716569998 for ; Mon, 02 Jan 2023 03:18:09 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10577"; a="407727630" X-IronPort-AV: E=Sophos;i="5.96,293,1665471600"; d="scan'208";a="407727630" X-Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2023 03:18:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10577"; a="654471748" X-IronPort-AV: E=Sophos;i="5.96,293,1665471600"; d="scan'208";a="654471748" X-Received: from gguo-desk.gar.corp.intel.com ([10.5.215.23]) by orsmga002.jf.intel.com with ESMTP; 02 Jan 2023 03:18:07 -0800 From: "Guo, Gua" To: devel@edk2.groups.io Cc: Gua Guo , Michael D Kinney , Sean Brogan , Bret Barkelew , Michael Kubacki Subject: [edk2-devel] [PATCH v7 1/3] UnitTestFrameworkPkg: Add code coverage support for GCC Date: Mon, 2 Jan 2023 19:18:00 +0800 Message-Id: In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,gua.guo@intel.com X-Gm-Message-State: Z9idloDJQQuauYKAkEOMWenix1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1672658289; bh=NMPY0ELe6VeZFAgdNKxXrkaDumeM8k8Rm2huoO6UPIQ=; h=Cc:Date:From:Reply-To:Subject:To; b=ZrQBQqn9P7hlNERJL4FpA5Y9/I2RpgcoKACPky+rtCJQBeXVBGGrfRTWRLV1iKDBzP7 xdV14jhTpdXfm6EG2eubwkblsP+6IgjxdjLIxLbXVtc5blKQrY/41cCJIo0Eka4rQXItR nBKxwjjYM7tz25PEk8fEl6GMsU2f0dCVZhE= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1672658290625100002 Content-Type: text/plain; charset="utf-8" From: Gua Guo In order to collect code coverage after running executable file, generate *.gcda and *.gcno file that require by lcov tool to generate code coverage report. Cc: Michael D Kinney Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael Kubacki Signed-off-by: Gua Guo --- UnitTestFrameworkPkg/ReadMe.md | 41 +++++++++++++++++++ .../UnitTestFrameworkPkg.ci.yaml | 1 + .../UnitTestFrameworkPkgHost.dsc.inc | 3 +- 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md index 9ce04b7f3e..3fa4e1910f 100644 --- a/UnitTestFrameworkPkg/ReadMe.md +++ b/UnitTestFrameworkPkg/ReadMe.md @@ -583,6 +583,47 @@ GTEST_OUTPUT=3Dxml: =20 This mode is used by the test running plugin to aggregate the results for = CI test status reporting in the web view. =20 +### Code Coverage + +Host based Unit Tests will automatically enable coverage data. + +For Windows, This is primarily leverage for pipeline builds, but this can = be leveraged locally using the +OpenCppCoverage windows tool to parse coverage data to cobertura xml forma= t. + +- Windows Prerequisite + ```bash + Download and install https://github.com/OpenCppCoverage/OpenCppCoverage/= releases + python -m pip install --upgrade -r ./pip-requirements.txt + stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=3DVS20= 19 -p MdeModulePkg + Open Build/coverage.xml + ``` + + - How to see code coverage data on IDE Visual Studio + ``` + Open Visual Studio VS2019 or above version + Click "Tools" -> "OpenCppCoverage Settings" + Fill your execute file into "Program to run:" + Click "Tools" -> "Run OpenCppCoverage" + ``` + + +For Linux, This is primarily leveraged for pipeline builds, but this can b= e leveraged locally using the +lcov linux tool, and parsed using the lcov_cobertura python tool to parse = it to cobertura xml format. + +- Linux Prerequisite + ```bash + sudo apt-get install -y lcov + python -m pip install --upgrade -r ./pip-requirements.txt + stuart_ci_build -c .pytool/CISettings.py -t NOOPT TOOL_CHAIN_TAG=3DGCC5= -p MdeModulePkg + Open Build/coverage.xml + ``` + - How to see code coverage data on IDE Visual Studio Code + ``` + Download plugin "Coverage Gutters" + Press Hot Key "Ctrl + Shift + P" and click option "Coverage Gutters: D= isplay Coverage" + ``` + + ### Important Note =20 This works on both Windows and Linux but is currently limited to x64 archi= tectures. Working on getting others, but we diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFr= ameworkPkg/UnitTestFrameworkPkg.ci.yaml index 072df6208c..d8f8e024c4 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml @@ -91,6 +91,7 @@ "pytools", "NOFAILURE", "cmockery", + "cobertura", "DHAVE", # build flag for cmocka in the INF "gtest", # file name in GoogleTestLib.inf "corthon", # Contact GitHub account in Readme diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTe= stFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc index 8009337552..7f5dfa30ed 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc @@ -27,7 +27,8 @@ GCC:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1 XCODE:*_*_*_CC_FLAGS =3D -D UNIT_TESTING_DEBUG=3D1 !endif - + GCC:*_GCC5_*_CC_FLAGS =3D --coverage + GCC:*_GCC5_*_DLINK_FLAGS =3D --coverage [BuildOptions.common.EDKII.HOST_APPLICATION] # # MSFT --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#97832): https://edk2.groups.io/g/devel/message/97832 Mute This Topic: https://groups.io/mt/96004771/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun May 19 00:47:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+97833+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+97833+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1672658291; cv=none; d=zohomail.com; s=zohoarc; b=Eree3HlHYXM08/UPaOV8ZkAflgOeFC3NRhv7SlLmQ9TPUYZdcT2+clExuFbwT3BZmIs/jR/8vonVehnuUK5nQ8AjFW+0ETdA/A+9uO1t7zAQmMvZaqWKeHvBrG1hKYjIBqsUuDs5v77i57vClmCPa5MZZBjPuef8NcyIWjX78Qg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1672658291; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=yZ3y5PGb3bn15NuWEXBwJrANjCcifTV6iDI9G7vfKh4=; b=dfwYxHajt7Ofg8xCXv6Fc/k+4ApNrVCvFeSxbp0owaK6WKq4/hy31tCfafcInRbeSEneGvSvBjv0nU1oYj6TNHRkpERgOf5J0XFLNNmlUFjAI8iN/s9TQFzM4KjwSZxbJ+Cd6pDzqJXRQWIXuYS0lfIPnHzIeeJM1F1qgGaW9ZI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+97833+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 167265829191872.9268977994501; Mon, 2 Jan 2023 03:18:11 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id yXtiYY1788612xDYKY1in2a1; Mon, 02 Jan 2023 03:18:11 -0800 X-Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mx.groups.io with SMTP id smtpd.web11.33235.1672658287716569998 for ; Mon, 02 Jan 2023 03:18:11 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10577"; a="407727657" X-IronPort-AV: E=Sophos;i="5.96,293,1665471600"; d="scan'208";a="407727657" X-Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2023 03:18:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10577"; a="654471751" X-IronPort-AV: E=Sophos;i="5.96,293,1665471600"; d="scan'208";a="654471751" X-Received: from gguo-desk.gar.corp.intel.com ([10.5.215.23]) by orsmga002.jf.intel.com with ESMTP; 02 Jan 2023 03:18:08 -0800 From: "Guo, Gua" To: devel@edk2.groups.io Cc: Gua Guo , Bob Feng , Bret Barkelew , Liming Gao , Michael D Kinney , Sean Brogan Subject: [edk2-devel] [PATCH v7 2/3] BaseTools/Plugin: Add coverage support for Unit Test Date: Mon, 2 Jan 2023 19:18:01 +0800 Message-Id: <8c33c1252be5f9b6b733b47a434a955a68ec9d0f.1672657976.git.gua.guo@intel.com> In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,gua.guo@intel.com X-Gm-Message-State: B4lE5FBK00qT5zAAE3XPbCkox1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1672658291; bh=pkEmOwRGa5gcxNesC4fxnifrTkFDkYiIgUBCWTNBkBw=; h=Cc:Date:From:Reply-To:Subject:To; b=WHnKGZBwTh6O6RXcDlPwy48GK0quNUyLCHTbHhxsOaaNr4I+K6W3LgygjHe7PzT9LHc AK5LXg7cDua4+0Dn8834z/qDS96mNQeUGXCc9CEXHxqfgKidNegGapGazxTLhgOTstmUa XIJGatIrx8l2cAWVztnRtLJY8HHDCzREuqw= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1672658292623100009 Content-Type: text/plain; charset="utf-8" From: Gua Guo For GCC, use lcov to generate Unit Test code coverage report For VS2019, use OpenCppCoverage to generate code coverage report Cc: Bob Feng Cc: Bret Barkelew Cc: Liming Gao Cc: Michael D Kinney Cc: Sean Brogan Signed-off-by: Gua Guo --- .../HostBasedUnitTestRunner.py | 101 +++++++++++++++++- 1 file changed, 100 insertions(+), 1 deletion(-) diff --git a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunn= er.py b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py index a8220aacd3..0e013c5f1a 100644 --- a/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py +++ b/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py @@ -94,7 +94,7 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin): =20 # Run the test. ret =3D RunCmd('"' + test + '"', "", workingdir=3Dcp) - if(ret !=3D 0): + if ret !=3D 0: logging.error("UnitTest Execution Error: " + os.path.basename(test)) else: @@ -115,4 +115,103 @@ class HostBasedUnitTestRunner(IUefiBuildPlugin): " %s - %s" % (case.attrib['na= me'], result.text)) failure_count +=3D 1 =20 + if thebuilder.env.GetValue("CODE_COVERAGE") !=3D "FALSE": + if thebuilder.env.GetValue("TOOL_CHAIN_TAG") =3D=3D "GCC5": + self.gen_code_coverage_gcc(thebuilder) + elif thebuilder.env.GetValue("TOOL_CHAIN_TAG").startswith = ("VS"): + self.gen_code_coverage_msvc(thebuilder) + else: + logging.info("Skipping code coverage. Currently, suppo= rt GCC and MSVC compiler.") + return failure_count + + def gen_code_coverage_gcc(self, thebuilder): + logging.info("Generating UnitTest code coverage") + + buildOutputBase =3D thebuilder.env.GetValue("BUILD_OUTPUT_BASE") + workspace =3D thebuilder.env.GetValue("WORKSPACE") + + # Generate base code coverage for all source files + ret =3D RunCmd("lcov", f"--no-external --capture --initial --direc= tory {buildOutputBase} --output-file {buildOutputBase}/cov-base.info --rc l= cov_branch_coverage=3D1") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed to build initial cove= rage data.") + return 1 + + # Coverage data for tested files only + ret =3D RunCmd("lcov", f"--capture --directory {buildOutputBase}/ = --output-file {buildOutputBase}/coverage-test.info --rc lcov_branch_coverag= e=3D1") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed to build coverage dat= a for tested files.") + return 1 + + # Aggregate all coverage data + ret =3D RunCmd("lcov", f"--add-tracefile {buildOutputBase}/cov-bas= e.info --add-tracefile {buildOutputBase}/coverage-test.info --output-file {= buildOutputBase}/total-coverage.info --rc lcov_branch_coverage=3D1") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed to aggregate coverage= data.") + return 1 + + # Generate coverage XML + ret =3D RunCmd("lcov_cobertura",f"{buildOutputBase}/total-coverage= .info -o {buildOutputBase}/compare.xml") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed to generate coverage = XML.") + return 1 + + # Filter out auto-generated and test code + ret =3D RunCmd("lcov_cobertura",f"{buildOutputBase}/total-coverage= .info --excludes ^.*UnitTest\|^.*MU\|^.*Mock\|^.*DEBUG -o {buildOutputBase}= /coverage.xml") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed generate filtered cov= erage XML.") + return 1 + + # Generate all coverage file + testCoverageList =3D glob.glob (f"{workspace}/Build/**/total-cover= age.info", recursive=3DTrue) + + coverageFile =3D "" + for testCoverage in testCoverageList: + coverageFile +=3D " --add-tracefile " + testCoverage + ret =3D RunCmd("lcov", f"{coverageFile} --output-file {workspace}/= Build/all-coverage.info --rc lcov_branch_coverage=3D1") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed generate all coverage= file.") + return 1 + + # Generate and XML file if requested.for all package + if os.path.isfile(f"{workspace}/Build/coverage.xml"): + os.remove(f"{workspace}/Build/coverage.xml") + ret =3D RunCmd("lcov_cobertura",f"{workspace}/Build/all-coverage.i= nfo --excludes ^.*UnitTest\|^.*MU\|^.*Mock\|^.*DEBUG -o {workspace}/Build/c= overage.xml") + + return 0 + + + def gen_code_coverage_msvc(self, thebuilder): + logging.info("Generating UnitTest code coverage") + + + buildOutputBase =3D thebuilder.env.GetValue("BUILD_OUTPUT_BASE") + testList =3D glob.glob(os.path.join(buildOutputBase, "**","*Test*.= exe"), recursive=3DTrue) + workspace =3D thebuilder.env.GetValue("WORKSPACE") + workspace =3D (workspace + os.sep) if workspace[-1] !=3D os.sep el= se workspace + # Generate coverage file + coverageFile =3D "" + for testFile in testList: + ret =3D RunCmd("OpenCppCoverage", f"--source {workspace} --exp= ort_type binary:{testFile}.cov -- {testFile}") + coverageFile +=3D " --input_coverage=3D" + testFile + ".cov" + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed to collect covera= ge data.") + return 1 + + # Generate and XML file if requested.by each package + ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{os.pa= th.join(buildOutputBase, 'coverage.xml')} --working_dir=3D{workspace}Build = {coverageFile}") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed to generate cobertura= format xml in single package.") + return 1 + + # Generate total report XML file for all package + testCoverageList =3D glob.glob(os.path.join(workspace, "Build", "*= *","*Test*.exe.cov"), recursive=3DTrue) + coverageFile =3D "" + for testCoverage in testCoverageList: + coverageFile +=3D " --input_coverage=3D" + testCoverage + + ret =3D RunCmd("OpenCppCoverage", f"--export_type cobertura:{works= pace}Build/coverage.xml --working_dir=3D{workspace}Build {coverageFile}") + if ret !=3D 0: + logging.error("UnitTest Coverage: Failed to generate cobertura= format xml.") + return 1 + + return 0 --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#97833): https://edk2.groups.io/g/devel/message/97833 Mute This Topic: https://groups.io/mt/96004773/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- From nobody Sun May 19 00:47:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) client-ip=66.175.222.108; envelope-from=bounce+27952+97834+1787277+3901457@groups.io; helo=mail02.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+97834+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=intel.com ARC-Seal: i=1; a=rsa-sha256; t=1672658293; cv=none; d=zohomail.com; s=zohoarc; b=cn0syhO2o+IkrLL9042h0YGmqmDiDaBbs1lYkfWRI/+idwTL8JJWjNGSJ9pIWoBRpjpVSKODhFCrxQqDFdX4d9ecqzSzEruYNGgvEVU5sWMLXCVlp74NB/Pmd1kcPpLVmXqjQKsHt2e0Btl6G8qqVjWUGsASdNjxj09jULdAmLs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1672658293; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:References:Sender:Subject:To; bh=1A/jeDQkYtIItCVu1o6PAAjW4imI9PMnbONGnvPntJ4=; b=fYLGhn9GZm4lrFfa2vYwlLLwGh81ZymxzHzHt5kQx1flZtvhvyS6MhJwuLrGKuBIWVY8aent6+SisPxPowKAoBQYsMErG9C10VNm8rddPVDwQKVLHBIznu+mGIcUy1A/6MqZyCVl8mCxRq+6vvu1fo1hoI2Wh+BotPdrh8s1dus= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce+27952+97834+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by mx.zohomail.com with SMTPS id 1672658293540812.1975117770081; Mon, 2 Jan 2023 03:18:13 -0800 (PST) Return-Path: X-Received: by 127.0.0.2 with SMTP id XKgtYY1788612xFiAmTuoVwi; Mon, 02 Jan 2023 03:18:13 -0800 X-Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mx.groups.io with SMTP id smtpd.web11.33235.1672658287716569998 for ; Mon, 02 Jan 2023 03:18:12 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10577"; a="407727676" X-IronPort-AV: E=Sophos;i="5.96,293,1665471600"; d="scan'208";a="407727676" X-Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2023 03:18:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10577"; a="654471755" X-IronPort-AV: E=Sophos;i="5.96,293,1665471600"; d="scan'208";a="654471755" X-Received: from gguo-desk.gar.corp.intel.com ([10.5.215.23]) by orsmga002.jf.intel.com with ESMTP; 02 Jan 2023 03:18:10 -0800 From: "Guo, Gua" To: devel@edk2.groups.io Cc: Gua Guo , Sean Brogan , Bret Barkelew , Michael D Kinney , Liming Gao Subject: [edk2-devel] [PATCH v7 3/3] .azurepipelines: Install code coverage tool Date: Mon, 2 Jan 2023 19:18:02 +0800 Message-Id: <5318d30db2bb837ebd3718f29f00893e064f1104.1672657976.git.gua.guo@intel.com> In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk List-Unsubscribe: List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,gua.guo@intel.com X-Gm-Message-State: pHPVg12sJ3y8yKcJdLf1NMurx1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1672658293; bh=uGhsEBIZhmHtjTlMrILVQdXN1l/Y216UFb3FzPsUnMc=; h=Cc:Date:From:Reply-To:Subject:To; b=IhWY4/AFX75695DwOGrA9H3C8Pq1rdGLcjMqNJiZrujg5C2XmydlAr5BsUhxaM7WSwz DDsut6YpDzEIsjSJIoHvzkn/YcozaflDR6iQsl9HwGF4KIr2Ps0ZFPYagh7wP665e4yDW Vec+WXkhycgJJqblC6yGI5OwlZz+98Qyxd0= X-ZohoMail-DKIM: pass (identity @groups.io) X-ZM-MESSAGEID: 1672658294645100013 Content-Type: text/plain; charset="utf-8" From: Gua Guo For Windows add below tool for code coverage 1. OpenCppCoverage: parsing pdb file to generate coverage data 2. pycobertura: show up html format data for coverage data For Linux add below tool for code coverage 1. lcov: parsing gcda gcno file to generate coverage data 2. lcov-cobertura: convert coverage data to cobertura format 3. pycobertura: show up html format data for coverage data Cc: Sean Brogan Cc: Bret Barkelew Cc: Michael D Kinney Cc: Liming Gao Signed-off-by: Gua Guo --- .azurepipelines/Ubuntu-GCC5.yml | 5 ++- .azurepipelines/Windows-VS2019.yml | 5 +++ .../templates/pr-gate-build-job.yml | 36 +++++++++++++++++++ .azurepipelines/templates/pr-gate-steps.yml | 4 +++ pip-requirements.txt | 2 ++ 5 files changed, 51 insertions(+), 1 deletion(-) diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.= yml index 1acd8d2a46..f83951eeaf 100644 --- a/.azurepipelines/Ubuntu-GCC5.yml +++ b/.azurepipelines/Ubuntu-GCC5.yml @@ -19,4 +19,7 @@ jobs: tool_chain_tag: 'GCC5' vm_image: 'ubuntu-latest' arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64" - + extra_install_step: + - bash: sudo apt-get install -y lcov + displayName: Install Code Coverage Tools + condition: and(gt(variables.pkg_count, 0), succeeded()) diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-V= S2019.yml index e4bd4b1d22..c07e5bb434 100644 --- a/.azurepipelines/Windows-VS2019.yml +++ b/.azurepipelines/Windows-VS2019.yml @@ -18,3 +18,8 @@ jobs: tool_chain_tag: 'VS2019' vm_image: 'windows-2019' arch_list: "IA32,X64" + extra_install_step: + - powershell: choco install opencppcoverage; Write-Host "##vso[task.pr= ependpath]C:\Program Files\OpenCppCoverage" + displayName: Install Code Coverage Tool + condition: and(gt(variables.pkg_count, 0), succeeded()) + diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipeli= nes/templates/pr-gate-build-job.yml index 7f88b41dc8..f148ec23e2 100644 --- a/.azurepipelines/templates/pr-gate-build-job.yml +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -12,6 +12,7 @@ parameters: tool_chain_tag: '' vm_image: '' arch_list: '' + extra_install_step: [] =20 # Build step jobs: @@ -77,3 +78,38 @@ jobs: build_pkgs: $(Build.Pkgs) build_targets: $(Build.Targets) build_archs: ${{ parameters.arch_list }} + extra_install_step: ${{ parameters.extra_install_step }} + +- job: Build_${{ parameters.tool_chain_tag }}_TARGET_CODE_COVERAGE + dependsOn: Build_${{ parameters.tool_chain_tag }} + workspace: + clean: all + + pool: + vmImage: 'windows-2019' + + steps: + - checkout: self + clean: true + fetchDepth: 1 + submodules: true + + - task: DownloadPipelineArtifact@2 + displayName: 'Download Build Artifacts' + inputs: + buildType: 'current' + targetPath: '$(Build.ArtifactStagingDirectory)' + + - task: CmdLine@2 + displayName: Create code coverage report + inputs: + script: | + dotnet tool install -g dotnet-reportgenerator-globaltool + reportgenerator -reports:$(Build.ArtifactStagingDirectory)/**/co= verage.xml -targetdir:$(Build.ArtifactStagingDirectory)/Coverage -reporttyp= es:Cobertura -filefilters:-*Build*;-*UnitTest*;-*Mock*;*usr/* + + - task: PublishCodeCoverageResults@1 + displayName: 'Publish code coverage' + inputs: + codeCoverageTool: Cobertura + summaryFileLocation: '$(Build.ArtifactStagingDirectory)/Coverage/C= obertura.xml' + diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/= templates/pr-gate-steps.yml index cb431e53fc..080f60aea6 100644 --- a/.azurepipelines/templates/pr-gate-steps.yml +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -12,6 +12,7 @@ parameters: build_pkgs: '' build_targets: '' build_archs: '' + extra_install_step: [] =20 steps: - checkout: self @@ -37,6 +38,8 @@ steps: displayName: fetch target branch condition: eq(variables['Build.Reason'], 'PullRequest') =20 +- ${{ parameters.extra_install_step }} + # trim the package list if this is a PR - task: CmdLine@1 displayName: Check if ${{ parameters.build_pkgs }} need testing @@ -125,6 +128,7 @@ steps: TestSuites.xml **/BUILD_TOOLS_REPORT.html **/OVERRIDELOG.TXT + coverage.xml flattenFolders: true condition: succeededOrFailed() =20 diff --git a/pip-requirements.txt b/pip-requirements.txt index be8c7a1c37..4ffcadddd8 100644 --- a/pip-requirements.txt +++ b/pip-requirements.txt @@ -16,3 +16,5 @@ edk2-pytool-library=3D=3D0.12.1 edk2-pytool-extensions~=3D0.20.0 edk2-basetools=3D=3D0.1.39 antlr4-python3-runtime=3D=3D4.7.1 +lcov-cobertura=3D=3D2.0.2 + --=20 2.31.1.windows.1 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#97834): https://edk2.groups.io/g/devel/message/97834 Mute This Topic: https://groups.io/mt/96004774/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-