[PATCH v7 0/4] Rewrite jevents program in python

Ian Rogers posted 4 patches 3 years, 9 months ago
tools/perf/Makefile.config               |   27 +-
tools/perf/Makefile.perf                 |   16 +-
tools/perf/pmu-events/Build              |   17 +-
tools/perf/pmu-events/empty-pmu-events.c |  158 +++
tools/perf/pmu-events/jevents.c          | 1342 ----------------------
tools/perf/pmu-events/jevents.py         |  409 +++++++
tools/perf/pmu-events/jsmn.c             |  352 ------
tools/perf/pmu-events/jsmn.h             |   68 --
tools/perf/pmu-events/json.c             |  162 ---
tools/perf/pmu-events/json.h             |   39 -
10 files changed, 604 insertions(+), 1986 deletions(-)
create mode 100644 tools/perf/pmu-events/empty-pmu-events.c
delete mode 100644 tools/perf/pmu-events/jevents.c
create mode 100755 tools/perf/pmu-events/jevents.py
delete mode 100644 tools/perf/pmu-events/jsmn.c
delete mode 100644 tools/perf/pmu-events/jsmn.h
delete mode 100644 tools/perf/pmu-events/json.c
delete mode 100644 tools/perf/pmu-events/json.h
[PATCH v7 0/4] Rewrite jevents program in python
Posted by Ian Rogers 3 years, 9 months ago
New architectures bring new complexity, such as Intel's hybrid
models. jevents provides an alternative to specifying events in the
kernel and exposing them through sysfs, however, it is difficult to
work with. For example, an error in the json input would yield an
error message but no json file or location. It is also a challenge to
update jsmn.c given its forked nature.

The changes here switch from jevents.c to a rewrite in python called
jevents.py. This means there is a build time dependency on python, but
such a dependency already exists for asciidoc (used to generate perf's
man pages). If the build detects that python isn't present or is older
than version 3.6 (released Dec. 2016) then an empty file is
substituted for the generated one.

A challenge with this code is in avoiding regressions. For this reason
the jevents.py produces identical output to jevents.c, validated with a
test script and build target.

v7. Adds Tested-by from John Garry <john.garry@huawei.com>.
v6. Adds Tested-by from Zhengjun Xing <zhengjun.xing@linux.intel.com>
    and Thomas Richter <tmricht@linux.ibm.com>. Fixes issues spotted
    by Jiri Olsa <jolsa@kernel.org>, jsmn.c wasn't deleted and the
    empty pmu-events.c didn't pass the pmu-events test. It also adds a
    missing mkdir which is necessary before creating pmu-events.c and
    tweaks the quiet display code in the empty case to display a gen.
v5. Adds a 2>/dev/null as suggested by David Laight
    <David.Laight@aculab.com>.
v4. Fixes the build for systems with python2 installed by adding a
    patch that makes python3 the preferred python (Reported-by: John
    Garry <john.garry@huawei.com>). It also fixes a bash-ism in the
    jevents-test.sh and fixes the handling of an empty string for a
    metric BriefDescription as one was added for sapphirerapids in the
    metric Execute.
v3. Updates the patches for merged changes (on
    acme/tmp.perf/core). Re-runs all comparisons to make sure the
    generated pmu-events.c isn't altered at all by this change. Adds
    the jevents.c ExtSel fix in:
    https://lore.kernel.org/lkml/20220525140410.1706851-1-zhengjun.xing@linux.intel.com/
    Bumps the python version from 3.5 to 3.6, as f-strings weren't
    introduced until 3.6.
v2. Fixes the build for architectures that don't have pmu-events json
    (Suggested-by: John Garry <john.garry@huawei.com>) and fixes the
    build for python not being present or too old (Suggested-by: Peter
    Zijlstra <peterz@infradead.org>/John Garry <john.garry@huawei.com>).

Ian Rogers (4):
  perf python: Prefer python3
  perf jevents: Add python converter script
  perf jevents: Switch build to use jevents.py
  perf jevents: Remove jevents.c

 tools/perf/Makefile.config               |   27 +-
 tools/perf/Makefile.perf                 |   16 +-
 tools/perf/pmu-events/Build              |   17 +-
 tools/perf/pmu-events/empty-pmu-events.c |  158 +++
 tools/perf/pmu-events/jevents.c          | 1342 ----------------------
 tools/perf/pmu-events/jevents.py         |  409 +++++++
 tools/perf/pmu-events/jsmn.c             |  352 ------
 tools/perf/pmu-events/jsmn.h             |   68 --
 tools/perf/pmu-events/json.c             |  162 ---
 tools/perf/pmu-events/json.h             |   39 -
 10 files changed, 604 insertions(+), 1986 deletions(-)
 create mode 100644 tools/perf/pmu-events/empty-pmu-events.c
 delete mode 100644 tools/perf/pmu-events/jevents.c
 create mode 100755 tools/perf/pmu-events/jevents.py
 delete mode 100644 tools/perf/pmu-events/jsmn.c
 delete mode 100644 tools/perf/pmu-events/jsmn.h
 delete mode 100644 tools/perf/pmu-events/json.c
 delete mode 100644 tools/perf/pmu-events/json.h

-- 
2.37.0.rc0.161.g10f37bed90-goog
Re: [PATCH v7 0/4] Rewrite jevents program in python
Posted by kajoljain 3 years, 9 months ago

On 6/29/22 23:55, Ian Rogers wrote:
> New architectures bring new complexity, such as Intel's hybrid
> models. jevents provides an alternative to specifying events in the
> kernel and exposing them through sysfs, however, it is difficult to
> work with. For example, an error in the json input would yield an
> error message but no json file or location. It is also a challenge to
> update jsmn.c given its forked nature.
> 
> The changes here switch from jevents.c to a rewrite in python called
> jevents.py. This means there is a build time dependency on python, but
> such a dependency already exists for asciidoc (used to generate perf's
> man pages). If the build detects that python isn't present or is older
> than version 3.6 (released Dec. 2016) then an empty file is
> substituted for the generated one.
> 
> A challenge with this code is in avoiding regressions. For this reason
> the jevents.py produces identical output to jevents.c, validated with a
> test script and build target.
> 

Patchset looks good to me.

Reviewed-by: Kajol Jain <kjain@linux.ibm.com>
Tested-by: Kajol Jain <kjain@linux.ibm.com>

Thanks,
Kajol Jain

> v7. Adds Tested-by from John Garry <john.garry@huawei.com>.
> v6. Adds Tested-by from Zhengjun Xing <zhengjun.xing@linux.intel.com>
>     and Thomas Richter <tmricht@linux.ibm.com>. Fixes issues spotted
>     by Jiri Olsa <jolsa@kernel.org>, jsmn.c wasn't deleted and the
>     empty pmu-events.c didn't pass the pmu-events test. It also adds a
>     missing mkdir which is necessary before creating pmu-events.c and
>     tweaks the quiet display code in the empty case to display a gen.
> v5. Adds a 2>/dev/null as suggested by David Laight
>     <David.Laight@aculab.com>.
> v4. Fixes the build for systems with python2 installed by adding a
>     patch that makes python3 the preferred python (Reported-by: John
>     Garry <john.garry@huawei.com>). It also fixes a bash-ism in the
>     jevents-test.sh and fixes the handling of an empty string for a
>     metric BriefDescription as one was added for sapphirerapids in the
>     metric Execute.
> v3. Updates the patches for merged changes (on
>     acme/tmp.perf/core). Re-runs all comparisons to make sure the
>     generated pmu-events.c isn't altered at all by this change. Adds
>     the jevents.c ExtSel fix in:
>     https://lore.kernel.org/lkml/20220525140410.1706851-1-zhengjun.xing@linux.intel.com/
>     Bumps the python version from 3.5 to 3.6, as f-strings weren't
>     introduced until 3.6.
> v2. Fixes the build for architectures that don't have pmu-events json
>     (Suggested-by: John Garry <john.garry@huawei.com>) and fixes the
>     build for python not being present or too old (Suggested-by: Peter
>     Zijlstra <peterz@infradead.org>/John Garry <john.garry@huawei.com>).
> 
> Ian Rogers (4):
>   perf python: Prefer python3
>   perf jevents: Add python converter script
>   perf jevents: Switch build to use jevents.py
>   perf jevents: Remove jevents.c
> 
>  tools/perf/Makefile.config               |   27 +-
>  tools/perf/Makefile.perf                 |   16 +-
>  tools/perf/pmu-events/Build              |   17 +-
>  tools/perf/pmu-events/empty-pmu-events.c |  158 +++
>  tools/perf/pmu-events/jevents.c          | 1342 ----------------------
>  tools/perf/pmu-events/jevents.py         |  409 +++++++
>  tools/perf/pmu-events/jsmn.c             |  352 ------
>  tools/perf/pmu-events/jsmn.h             |   68 --
>  tools/perf/pmu-events/json.c             |  162 ---
>  tools/perf/pmu-events/json.h             |   39 -
>  10 files changed, 604 insertions(+), 1986 deletions(-)
>  create mode 100644 tools/perf/pmu-events/empty-pmu-events.c
>  delete mode 100644 tools/perf/pmu-events/jevents.c
>  create mode 100755 tools/perf/pmu-events/jevents.py
>  delete mode 100644 tools/perf/pmu-events/jsmn.c
>  delete mode 100644 tools/perf/pmu-events/jsmn.h
>  delete mode 100644 tools/perf/pmu-events/json.c
>  delete mode 100644 tools/perf/pmu-events/json.h
>