[v6] Add a plugin to check Ecc issues for edk2 on open ci

[edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

Posted by Zhang, Shenglei 3 years, 10 months ago

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2606
As planed we will enable Ecc check for edk2 on open ci. And they are
ready now. I appreciate receiving feedback and comments if someone
find errors or false positive issues.

I created a pipline of EccCheck for my forked edk2. Welcome everyone to
create pull request to test the quality of this plugin.
My forked tree: https://github.com/shenglei10/edk2

And I also created some test cases for ECC plugin. Below are test cases.
https://github.com/shenglei10/edk2/tree/ECC
Results can be view in below azure server.
https://dev.azure.com/shengleizhang/shengleizhang/_build?definitionId=12&_a=summary

Patches
1/16: It's a lib necessary for py3 to run Ecc on azure servers.

2/16: EccCheck.py is a plugin to report Ecc issues for commits. It can be run
     on azure servers for open ci, or a local virtual environment.

3/16~16/16: We consider some cases that will report out Ecc issues but they won't
     be fixed, like submodule and industry standard related things. So we
     add two configuration fields "Exception" and "IgnoreFiles" for people
     to use. These patches add configuration in yaml files for Ecc check.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>

v2: Update 1/17, fix the bug that the script can't hanlde multiple commits.

v3: Update 1/17, set the only workalbe workspace is edk2 root directory.
    Update 2/17, designate the version of antlr4 is 4.7.1.
    Add 4/17~17/17.

v4. Update 1/17, remove the function EdksetupRebuild(), instead add
    function SetupEnvironment(). Update variables' format and type hints
    to pass flake8 and mypy.

v5. Conver the former method to plugin solution, to align with
    other check points on open ci.

v6. The 1/16 patch is missed in v5 series. Now add it in v6.

Shenglei Zhang (16):
  pip-requirements.txt: Add Ecc required lib
  .pytool/Plugin: Add a plugin EccCheck
  MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for Ecc check
  ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check
  CryptoPkg/CryptoPkg.ci.yaml: Add configuration for Ecc check
  EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check
  FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check
  FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for Ecc check
  MdePkg/MdePkg.ci.yaml: Add configuration for Ecc check
  NetworkPkg/NetworkPkg.ci.yaml: Add configuration for Ecc check
  OvmfPkg/OvmfPkg.ci.yaml: Add configuration for Ecc check
  PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check
  SecurityPkg/SecurityPkg.ci.yaml: Add configuration for Ecc check
  ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check
  UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check
  UnitTestFrameworkPkg: Add configuration for Ecc check in yaml file

 .pytool/Plugin/EccCheck/EccCheck.py           | 268 ++++++++++++++++++
 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml |  11 +
 .pytool/Plugin/EccCheck/Readme.md             |  15 +
 ArmVirtPkg/ArmVirtPkg.ci.yaml                 |  11 +
 CryptoPkg/CryptoPkg.ci.yaml                   |  11 +
 EmulatorPkg/EmulatorPkg.ci.yaml               |  11 +
 FatPkg/FatPkg.ci.yaml                         |  11 +
 FmpDevicePkg/FmpDevicePkg.ci.yaml             |  11 +
 MdeModulePkg/MdeModulePkg.ci.yaml             |  11 +
 MdePkg/MdePkg.ci.yaml                         |  11 +
 NetworkPkg/NetworkPkg.ci.yaml                 |  11 +
 OvmfPkg/OvmfPkg.ci.yaml                       |  11 +
 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml         |  11 +
 SecurityPkg/SecurityPkg.ci.yaml               |  11 +
 ShellPkg/ShellPkg.ci.yaml                     |  11 +
 UefiCpuPkg/UefiCpuPkg.ci.yaml                 |  11 +
 .../UnitTestFrameworkPkg.ci.yaml              |  10 +
 pip-requirements.txt                          |   1 +
 18 files changed, 448 insertions(+)
 create mode 100644 .pytool/Plugin/EccCheck/EccCheck.py
 create mode 100644 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/EccCheck/Readme.md

-- 
2.18.0.windows.1


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#61879): https://edk2.groups.io/g/devel/message/61879
Mute This Topic: https://groups.io/mt/75227224/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

Posted by Liming Gao 3 years, 9 months ago

Include more people and collect the comments. 

ECC is the source file coding style checker. Here is its wiki page https://github.com/tianocore/tianocore.github.io/wiki/ECC-tool.
If the changed code doesn't follow edk2 coding style, ECC will report the error. 

This patch set enables ECC checker in open CI for each patch coming into edk2 repo. That means new changes need to follow edk2 coding style. 
Otherwise, new changes can't be merged. 

If you have some comments for ECC checker in open CI, please reply this mail. 

Thanks
Liming
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Zhang, Shenglei
> Sent: Wednesday, July 1, 2020 9:55 AM
> To: devel@edk2.groups.io
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> Subject: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci
> 
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2606
> As planed we will enable Ecc check for edk2 on open ci. And they are
> ready now. I appreciate receiving feedback and comments if someone
> find errors or false positive issues.
> 
> I created a pipline of EccCheck for my forked edk2. Welcome everyone to
> create pull request to test the quality of this plugin.
> My forked tree: https://github.com/shenglei10/edk2
> 
> And I also created some test cases for ECC plugin. Below are test cases.
> https://github.com/shenglei10/edk2/tree/ECC
> Results can be view in below azure server.
> https://dev.azure.com/shengleizhang/shengleizhang/_build?definitionId=12&_a=summary
> 
> Patches
> 1/16: It's a lib necessary for py3 to run Ecc on azure servers.
> 
> 2/16: EccCheck.py is a plugin to report Ecc issues for commits. It can be run
>      on azure servers for open ci, or a local virtual environment.
> 
> 3/16~16/16: We consider some cases that will report out Ecc issues but they won't
>      be fixed, like submodule and industry standard related things. So we
>      add two configuration fields "Exception" and "IgnoreFiles" for people
>      to use. These patches add configuration in yaml files for Ecc check.
> 
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> 
> v2: Update 1/17, fix the bug that the script can't hanlde multiple commits.
> 
> v3: Update 1/17, set the only workalbe workspace is edk2 root directory.
>     Update 2/17, designate the version of antlr4 is 4.7.1.
>     Add 4/17~17/17.
> 
> v4. Update 1/17, remove the function EdksetupRebuild(), instead add
>     function SetupEnvironment(). Update variables' format and type hints
>     to pass flake8 and mypy.
> 
> v5. Conver the former method to plugin solution, to align with
>     other check points on open ci.
> 
> v6. The 1/16 patch is missed in v5 series. Now add it in v6.
> 
> Shenglei Zhang (16):
>   pip-requirements.txt: Add Ecc required lib
>   .pytool/Plugin: Add a plugin EccCheck
>   MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for Ecc check
>   ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check
>   CryptoPkg/CryptoPkg.ci.yaml: Add configuration for Ecc check
>   EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check
>   FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check
>   FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for Ecc check
>   MdePkg/MdePkg.ci.yaml: Add configuration for Ecc check
>   NetworkPkg/NetworkPkg.ci.yaml: Add configuration for Ecc check
>   OvmfPkg/OvmfPkg.ci.yaml: Add configuration for Ecc check
>   PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check
>   SecurityPkg/SecurityPkg.ci.yaml: Add configuration for Ecc check
>   ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check
>   UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check
>   UnitTestFrameworkPkg: Add configuration for Ecc check in yaml file
> 
>  .pytool/Plugin/EccCheck/EccCheck.py           | 268 ++++++++++++++++++
>  .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml |  11 +
>  .pytool/Plugin/EccCheck/Readme.md             |  15 +
>  ArmVirtPkg/ArmVirtPkg.ci.yaml                 |  11 +
>  CryptoPkg/CryptoPkg.ci.yaml                   |  11 +
>  EmulatorPkg/EmulatorPkg.ci.yaml               |  11 +
>  FatPkg/FatPkg.ci.yaml                         |  11 +
>  FmpDevicePkg/FmpDevicePkg.ci.yaml             |  11 +
>  MdeModulePkg/MdeModulePkg.ci.yaml             |  11 +
>  MdePkg/MdePkg.ci.yaml                         |  11 +
>  NetworkPkg/NetworkPkg.ci.yaml                 |  11 +
>  OvmfPkg/OvmfPkg.ci.yaml                       |  11 +
>  PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml         |  11 +
>  SecurityPkg/SecurityPkg.ci.yaml               |  11 +
>  ShellPkg/ShellPkg.ci.yaml                     |  11 +
>  UefiCpuPkg/UefiCpuPkg.ci.yaml                 |  11 +
>  .../UnitTestFrameworkPkg.ci.yaml              |  10 +
>  pip-requirements.txt                          |   1 +
>  18 files changed, 448 insertions(+)
>  create mode 100644 .pytool/Plugin/EccCheck/EccCheck.py
>  create mode 100644 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
>  create mode 100644 .pytool/Plugin/EccCheck/Readme.md
> 
> --
> 2.18.0.windows.1
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#62044): https://edk2.groups.io/g/devel/message/62044
Mute This Topic: https://groups.io/mt/75227224/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

Posted by Laszlo Ersek 3 years, 9 months ago

Hi Liming,

On 07/03/20 17:13, Gao, Liming wrote:
> Include more people and collect the comments. 
> 
> ECC is the source file coding style checker. Here is its wiki page https://github.com/tianocore/tianocore.github.io/wiki/ECC-tool.
> If the changed code doesn't follow edk2 coding style, ECC will report the error. 
> 
> This patch set enables ECC checker in open CI for each patch coming into edk2 repo. That means new changes need to follow edk2 coding style. 
> Otherwise, new changes can't be merged. 
> 
> If you have some comments for ECC checker in open CI, please reply this mail. 


(1) The ArmVirtPkg (v6 04/16) and OvmfPkg (v6 11/16) patches already
carry my ACKs; from here:

  https://edk2.groups.io/g/devel/message/61154
  https://edk2.groups.io/g/devel/message/61155


(2) The UefiCpuPkg patch (v6 15/16) *should* also carry my ACK, from here:

  https://edk2.groups.io/g/devel/message/61156

Shenglei picked up my ACK for v4 and v5:

  https://edk2.groups.io/g/devel/message/61283
  https://edk2.groups.io/g/devel/message/61852

but then dropped it for v6:

  https://edk2.groups.io/g/devel/message/61894

Shenglei, can you please explain why you dropped my ACK from the
UefiCpuPkg patch, in v6?


(3) The initial discussion between Shenglei and myself are under the v2
posting:

  https://edk2.groups.io/g/devel/message/60665
  https://edk2.groups.io/g/devel/message/60711
  https://edk2.groups.io/g/devel/message/60961

I'm happy with this work because it lets package maintainers tailor ECC
as they see appropriate.

Thanks,
Laszlo


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#62047): https://edk2.groups.io/g/devel/message/62047
Mute This Topic: https://groups.io/mt/75227224/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

Posted by Zhang, Shenglei 3 years, 9 months ago

Hi Lazslo,

Looks like I missed your ACK. I  will add it in latter versions.

Thanks,
Shenglei

> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Saturday, July 4, 2020 12:02 AM
> To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io; Zhang,
> Shenglei <shenglei.zhang@intel.com>; Leif Lindholm <leif@nuviainc.com>;
> afish@apple.com
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> Subject: Re: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues
> for edk2 on open ci
> 
> Hi Liming,
> 
> On 07/03/20 17:13, Gao, Liming wrote:
> > Include more people and collect the comments.
> >
> > ECC is the source file coding style checker. Here is its wiki page
> https://github.com/tianocore/tianocore.github.io/wiki/ECC-tool.
> > If the changed code doesn't follow edk2 coding style, ECC will report the
> error.
> >
> > This patch set enables ECC checker in open CI for each patch coming into
> edk2 repo. That means new changes need to follow edk2 coding style.
> > Otherwise, new changes can't be merged.
> >
> > If you have some comments for ECC checker in open CI, please reply this
> mail.
> 
> 
> (1) The ArmVirtPkg (v6 04/16) and OvmfPkg (v6 11/16) patches already
> carry my ACKs; from here:
> 
>   https://edk2.groups.io/g/devel/message/61154
>   https://edk2.groups.io/g/devel/message/61155
> 
> 
> (2) The UefiCpuPkg patch (v6 15/16) *should* also carry my ACK, from here:
> 
>   https://edk2.groups.io/g/devel/message/61156
> 
> Shenglei picked up my ACK for v4 and v5:
> 
>   https://edk2.groups.io/g/devel/message/61283
>   https://edk2.groups.io/g/devel/message/61852
> 
> but then dropped it for v6:
> 
>   https://edk2.groups.io/g/devel/message/61894
> 
> Shenglei, can you please explain why you dropped my ACK from the
> UefiCpuPkg patch, in v6?
> 
> 
> (3) The initial discussion between Shenglei and myself are under the v2
> posting:
> 
>   https://edk2.groups.io/g/devel/message/60665
>   https://edk2.groups.io/g/devel/message/60711
>   https://edk2.groups.io/g/devel/message/60961
> 
> I'm happy with this work because it lets package maintainers tailor ECC
> as they see appropriate.
> 
> Thanks,
> Laszlo


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#62058): https://edk2.groups.io/g/devel/message/62058
Mute This Topic: https://groups.io/mt/75227224/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

Posted by Liming Gao 3 years, 9 months ago

Shenglei:
  Thanks for your work to enable ECC checker in open CI. Have you the step to run this checker in local environment? The developer may want to reproduce the issue and fix it. 

Thanks
Liming
-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Zhang, Shenglei
Sent: 2020年7月1日 9:55
To: devel@edk2.groups.io
Cc: Feng, Bob C <bob.c.feng@intel.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
Subject: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2606
As planed we will enable Ecc check for edk2 on open ci. And they are ready now. I appreciate receiving feedback and comments if someone find errors or false positive issues.

I created a pipline of EccCheck for my forked edk2. Welcome everyone to create pull request to test the quality of this plugin.
My forked tree: https://github.com/shenglei10/edk2

And I also created some test cases for ECC plugin. Below are test cases.
https://github.com/shenglei10/edk2/tree/ECC
Results can be view in below azure server.
https://dev.azure.com/shengleizhang/shengleizhang/_build?definitionId=12&_a=summary

Patches
1/16: It's a lib necessary for py3 to run Ecc on azure servers.

2/16: EccCheck.py is a plugin to report Ecc issues for commits. It can be run
     on azure servers for open ci, or a local virtual environment.

3/16~16/16: We consider some cases that will report out Ecc issues but they won't
     be fixed, like submodule and industry standard related things. So we
     add two configuration fields "Exception" and "IgnoreFiles" for people
     to use. These patches add configuration in yaml files for Ecc check.

Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>

v2: Update 1/17, fix the bug that the script can't hanlde multiple commits.

v3: Update 1/17, set the only workalbe workspace is edk2 root directory.
    Update 2/17, designate the version of antlr4 is 4.7.1.
    Add 4/17~17/17.

v4. Update 1/17, remove the function EdksetupRebuild(), instead add
    function SetupEnvironment(). Update variables' format and type hints
    to pass flake8 and mypy.

v5. Conver the former method to plugin solution, to align with
    other check points on open ci.

v6. The 1/16 patch is missed in v5 series. Now add it in v6.

Shenglei Zhang (16):
  pip-requirements.txt: Add Ecc required lib
  .pytool/Plugin: Add a plugin EccCheck
  MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for Ecc check
  ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check
  CryptoPkg/CryptoPkg.ci.yaml: Add configuration for Ecc check
  EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check
  FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check
  FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for Ecc check
  MdePkg/MdePkg.ci.yaml: Add configuration for Ecc check
  NetworkPkg/NetworkPkg.ci.yaml: Add configuration for Ecc check
  OvmfPkg/OvmfPkg.ci.yaml: Add configuration for Ecc check
  PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check
  SecurityPkg/SecurityPkg.ci.yaml: Add configuration for Ecc check
  ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check
  UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check
  UnitTestFrameworkPkg: Add configuration for Ecc check in yaml file

 .pytool/Plugin/EccCheck/EccCheck.py           | 268 ++++++++++++++++++
 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml |  11 +
 .pytool/Plugin/EccCheck/Readme.md             |  15 +
 ArmVirtPkg/ArmVirtPkg.ci.yaml                 |  11 +
 CryptoPkg/CryptoPkg.ci.yaml                   |  11 +
 EmulatorPkg/EmulatorPkg.ci.yaml               |  11 +
 FatPkg/FatPkg.ci.yaml                         |  11 +
 FmpDevicePkg/FmpDevicePkg.ci.yaml             |  11 +
 MdeModulePkg/MdeModulePkg.ci.yaml             |  11 +
 MdePkg/MdePkg.ci.yaml                         |  11 +
 NetworkPkg/NetworkPkg.ci.yaml                 |  11 +
 OvmfPkg/OvmfPkg.ci.yaml                       |  11 +
 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml         |  11 +
 SecurityPkg/SecurityPkg.ci.yaml               |  11 +
 ShellPkg/ShellPkg.ci.yaml                     |  11 +
 UefiCpuPkg/UefiCpuPkg.ci.yaml                 |  11 +
 .../UnitTestFrameworkPkg.ci.yaml              |  10 +
 pip-requirements.txt                          |   1 +
 18 files changed, 448 insertions(+)
 create mode 100644 .pytool/Plugin/EccCheck/EccCheck.py
 create mode 100644 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
 create mode 100644 .pytool/Plugin/EccCheck/Readme.md

--
2.18.0.windows.1





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#61937): https://edk2.groups.io/g/devel/message/61937
Mute This Topic: https://groups.io/mt/75227224/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

Posted by Zhang, Shenglei 3 years, 9 months ago

Liming,

Yes we have instructions to run plugins locally. Since they are not good enough and not easy to find.
I provide the following steps.

Pre-Requisites
* Windows OS
* Make sure you have python 3.7.x or newer available on path
* Make sure you have git installed and available on path

1. Navigate to edk2
2. run `python -m venv pytool-ext-venv`
3. run `pytool-ext-venv\Scripts\activate.bat`
4. run `pip install --upgrade -r pip-requirements.txt`
5. run `stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=NO-TARGET`

It's strongly to remove any anther plugin when developers want to reproduce Ecc issues.

Thanks,
Shenglei

> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Thursday, July 2, 2020 10:31 AM
> To: devel@edk2.groups.io; Zhang, Shenglei <shenglei.zhang@intel.com>
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Sean Brogan <sean.brogan@microsoft.com>
> Subject: RE: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues
> for edk2 on open ci
> 
> Shenglei:
>   Thanks for your work to enable ECC checker in open CI. Have you the step
> to run this checker in local environment? The developer may want to
> reproduce the issue and fix it.
> 
> Thanks
> Liming
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Zhang,
> Shenglei
> Sent: 2020年7月1日 9:55
> To: devel@edk2.groups.io
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>; Sean
> Brogan <sean.brogan@microsoft.com>
> Subject: [edk2-devel] [PATCH v6 00/16] Add a plugin to check Ecc issues for
> edk2 on open ci
> 
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2606
> As planed we will enable Ecc check for edk2 on open ci. And they are ready
> now. I appreciate receiving feedback and comments if someone find errors
> or false positive issues.
> 
> I created a pipline of EccCheck for my forked edk2. Welcome everyone to
> create pull request to test the quality of this plugin.
> My forked tree: https://github.com/shenglei10/edk2
> 
> And I also created some test cases for ECC plugin. Below are test cases.
> https://github.com/shenglei10/edk2/tree/ECC
> Results can be view in below azure server.
> https://dev.azure.com/shengleizhang/shengleizhang/_build?definitionId=12
> &_a=summary
> 
> Patches
> 1/16: It's a lib necessary for py3 to run Ecc on azure servers.
> 
> 2/16: EccCheck.py is a plugin to report Ecc issues for commits. It can be run
>      on azure servers for open ci, or a local virtual environment.
> 
> 3/16~16/16: We consider some cases that will report out Ecc issues but they
> won't
>      be fixed, like submodule and industry standard related things. So we
>      add two configuration fields "Exception" and "IgnoreFiles" for people
>      to use. These patches add configuration in yaml files for Ecc check.
> 
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> 
> v2: Update 1/17, fix the bug that the script can't hanlde multiple commits.
> 
> v3: Update 1/17, set the only workalbe workspace is edk2 root directory.
>     Update 2/17, designate the version of antlr4 is 4.7.1.
>     Add 4/17~17/17.
> 
> v4. Update 1/17, remove the function EdksetupRebuild(), instead add
>     function SetupEnvironment(). Update variables' format and type hints
>     to pass flake8 and mypy.
> 
> v5. Conver the former method to plugin solution, to align with
>     other check points on open ci.
> 
> v6. The 1/16 patch is missed in v5 series. Now add it in v6.
> 
> Shenglei Zhang (16):
>   pip-requirements.txt: Add Ecc required lib
>   .pytool/Plugin: Add a plugin EccCheck
>   MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for Ecc check
>   ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check
>   CryptoPkg/CryptoPkg.ci.yaml: Add configuration for Ecc check
>   EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check
>   FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check
>   FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for Ecc check
>   MdePkg/MdePkg.ci.yaml: Add configuration for Ecc check
>   NetworkPkg/NetworkPkg.ci.yaml: Add configuration for Ecc check
>   OvmfPkg/OvmfPkg.ci.yaml: Add configuration for Ecc check
>   PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check
>   SecurityPkg/SecurityPkg.ci.yaml: Add configuration for Ecc check
>   ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check
>   UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check
>   UnitTestFrameworkPkg: Add configuration for Ecc check in yaml file
> 
>  .pytool/Plugin/EccCheck/EccCheck.py           | 268 ++++++++++++++++++
>  .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml |  11 +
>  .pytool/Plugin/EccCheck/Readme.md             |  15 +
>  ArmVirtPkg/ArmVirtPkg.ci.yaml                 |  11 +
>  CryptoPkg/CryptoPkg.ci.yaml                   |  11 +
>  EmulatorPkg/EmulatorPkg.ci.yaml               |  11 +
>  FatPkg/FatPkg.ci.yaml                         |  11 +
>  FmpDevicePkg/FmpDevicePkg.ci.yaml             |  11 +
>  MdeModulePkg/MdeModulePkg.ci.yaml             |  11 +
>  MdePkg/MdePkg.ci.yaml                         |  11 +
>  NetworkPkg/NetworkPkg.ci.yaml                 |  11 +
>  OvmfPkg/OvmfPkg.ci.yaml                       |  11 +
>  PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml         |  11 +
>  SecurityPkg/SecurityPkg.ci.yaml               |  11 +
>  ShellPkg/ShellPkg.ci.yaml                     |  11 +
>  UefiCpuPkg/UefiCpuPkg.ci.yaml                 |  11 +
>  .../UnitTestFrameworkPkg.ci.yaml              |  10 +
>  pip-requirements.txt                          |   1 +
>  18 files changed, 448 insertions(+)
>  create mode 100644 .pytool/Plugin/EccCheck/EccCheck.py
>  create mode 100644 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
>  create mode 100644 .pytool/Plugin/EccCheck/Readme.md
> 
> --
> 2.18.0.windows.1
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#61966): https://edk2.groups.io/g/devel/message/61966
Mute This Topic: https://groups.io/mt/75227224/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub  [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-