[PATCH v4 00/17] Compress the pmu_event tables

Ian Rogers posted 17 patches 3 years, 8 months ago
There is a newer version of this series
tools/perf/arch/arm64/util/pmu.c              |   4 +-
tools/perf/pmu-events/Build                   |   6 +-
.../arch/test/test_soc/cpu/metrics.json       |  64 +++
tools/perf/pmu-events/empty-pmu-events.c      | 204 +++++++-
tools/perf/pmu-events/jevents.py              | 495 ++++++++++++++----
tools/perf/pmu-events/pmu-events.h            |  40 +-
tools/perf/tests/expand-cgroup.c              |  25 +-
tools/perf/tests/parse-metric.c               |  77 +--
tools/perf/tests/pmu-events.c                 | 466 +++++++----------
tools/perf/util/metricgroup.c                 | 275 ++++++----
tools/perf/util/metricgroup.h                 |   5 +-
tools/perf/util/pmu.c                         | 139 ++---
tools/perf/util/pmu.h                         |   8 +-
tools/perf/util/s390-sample-raw.c             |  50 +-
14 files changed, 1140 insertions(+), 718 deletions(-)
create mode 100644 tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.json
[PATCH v4 00/17] Compress the pmu_event tables
Posted by Ian Rogers 3 years, 8 months ago
jevents.py creates a number of large arrays from the json events. The
arrays contain pointers to strings that need relocating. The
relocations have file size, run time and memory costs. These changes
refactor the pmu_events API so that the storage of the pmu_event
struct isn't exposed. The format is then changed to an offset within a
combined big string, with adjacent pmu_event struct variables being
next to each other in the string separated by \0 - meaning only the
first variable of the struct needs its offset recording.

Some related fixes are contained with the patches. The architecture
jevents.py creates tables for can now be set by the JEVENTS_ARCH make
variable, with a new 'all' that generates the events and metrics for
all architectures.

An example of the improvement to the file size on x86 is:
no jevents - the same 19,788,464bytes
x86 jevents - ~16.7% file size saving 23,744,288bytes vs 28,502,632bytes
all jevents - ~19.5% file size saving 24,469,056bytes vs 30,379,920bytes
default build options plus NO_LIBBFD=1.

I originally suggested fixing this problem in:
https://lore.kernel.org/linux-perf-users/CAP-5=fVB8G4bdb9T=FncRTh9oBVKCS=+=eowAO+YSgAhab+Dtg@mail.gmail.com/

v4. Fixed an issue with the empty-pmu-events.c spotted by John Garry
    <john.garry@huawei.com>.
v3. Fix an ARM build issue with a missed weak symbol. Perform some
    pytype clean up.
v2. Split the substring folding optimization to its own patch and
    comment tweaks as suggested by Namhyung Kim
    <namhyung@kernel.org>. Recompute the file size savings with the
    latest json events and metrics.

Ian Rogers (17):
  perf jevents: Clean up pytype warnings
  perf jevents: Simplify generation of C-string
  perf jevents: Add JEVENTS_ARCH make option
  perf jevent: Add an 'all' architecture argument
  perf jevents: Remove the type/version variables
  perf jevents: Provide path to json file on error
  perf jevents: Sort json files entries
  perf pmu-events: Hide pmu_sys_event_tables
  perf pmu-events: Avoid passing pmu_events_map
  perf pmu-events: Hide pmu_events_map
  perf test: Use full metric resolution
  perf pmu-events: Move test events/metrics to json
  perf pmu-events: Don't assume pmu_event is an array
  perf pmu-events: Hide the pmu_events
  perf metrics: Copy entire pmu_event in find metric
  perf jevents: Compress the pmu_events_table
  perf jevents: Fold strings optimization

 tools/perf/arch/arm64/util/pmu.c              |   4 +-
 tools/perf/pmu-events/Build                   |   6 +-
 .../arch/test/test_soc/cpu/metrics.json       |  64 +++
 tools/perf/pmu-events/empty-pmu-events.c      | 204 +++++++-
 tools/perf/pmu-events/jevents.py              | 495 ++++++++++++++----
 tools/perf/pmu-events/pmu-events.h            |  40 +-
 tools/perf/tests/expand-cgroup.c              |  25 +-
 tools/perf/tests/parse-metric.c               |  77 +--
 tools/perf/tests/pmu-events.c                 | 466 +++++++----------
 tools/perf/util/metricgroup.c                 | 275 ++++++----
 tools/perf/util/metricgroup.h                 |   5 +-
 tools/perf/util/pmu.c                         | 139 ++---
 tools/perf/util/pmu.h                         |   8 +-
 tools/perf/util/s390-sample-raw.c             |  50 +-
 14 files changed, 1140 insertions(+), 718 deletions(-)
 create mode 100644 tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.json

-- 
2.37.1.559.g78731f0fdb-goog
Re: [PATCH v4 00/17] Compress the pmu_event tables
Posted by Arnaldo Carvalho de Melo 3 years, 8 months ago
Em Thu, Aug 04, 2022 at 03:17:59PM -0700, Ian Rogers escreveu:
> jevents.py creates a number of large arrays from the json events. The
> arrays contain pointers to strings that need relocating. The
> relocations have file size, run time and memory costs. These changes
> refactor the pmu_events API so that the storage of the pmu_event
> struct isn't exposed. The format is then changed to an offset within a
> combined big string, with adjacent pmu_event struct variables being
> next to each other in the string separated by \0 - meaning only the
> first variable of the struct needs its offset recording.
> 
> Some related fixes are contained with the patches. The architecture
> jevents.py creates tables for can now be set by the JEVENTS_ARCH make
> variable, with a new 'all' that generates the events and metrics for
> all architectures.
> 
> An example of the improvement to the file size on x86 is:
> no jevents - the same 19,788,464bytes
> x86 jevents - ~16.7% file size saving 23,744,288bytes vs 28,502,632bytes
> all jevents - ~19.5% file size saving 24,469,056bytes vs 30,379,920bytes
> default build options plus NO_LIBBFD=1.

Applied the first four patches, waiting for the review comments to be
discussed.

- Arnaldo
 
> I originally suggested fixing this problem in:
> https://lore.kernel.org/linux-perf-users/CAP-5=fVB8G4bdb9T=FncRTh9oBVKCS=+=eowAO+YSgAhab+Dtg@mail.gmail.com/
> 
> v4. Fixed an issue with the empty-pmu-events.c spotted by John Garry
>     <john.garry@huawei.com>.
> v3. Fix an ARM build issue with a missed weak symbol. Perform some
>     pytype clean up.
> v2. Split the substring folding optimization to its own patch and
>     comment tweaks as suggested by Namhyung Kim
>     <namhyung@kernel.org>. Recompute the file size savings with the
>     latest json events and metrics.
> 
> Ian Rogers (17):
>   perf jevents: Clean up pytype warnings
>   perf jevents: Simplify generation of C-string
>   perf jevents: Add JEVENTS_ARCH make option
>   perf jevent: Add an 'all' architecture argument
>   perf jevents: Remove the type/version variables
>   perf jevents: Provide path to json file on error
>   perf jevents: Sort json files entries
>   perf pmu-events: Hide pmu_sys_event_tables
>   perf pmu-events: Avoid passing pmu_events_map
>   perf pmu-events: Hide pmu_events_map
>   perf test: Use full metric resolution
>   perf pmu-events: Move test events/metrics to json
>   perf pmu-events: Don't assume pmu_event is an array
>   perf pmu-events: Hide the pmu_events
>   perf metrics: Copy entire pmu_event in find metric
>   perf jevents: Compress the pmu_events_table
>   perf jevents: Fold strings optimization
> 
>  tools/perf/arch/arm64/util/pmu.c              |   4 +-
>  tools/perf/pmu-events/Build                   |   6 +-
>  .../arch/test/test_soc/cpu/metrics.json       |  64 +++
>  tools/perf/pmu-events/empty-pmu-events.c      | 204 +++++++-
>  tools/perf/pmu-events/jevents.py              | 495 ++++++++++++++----
>  tools/perf/pmu-events/pmu-events.h            |  40 +-
>  tools/perf/tests/expand-cgroup.c              |  25 +-
>  tools/perf/tests/parse-metric.c               |  77 +--
>  tools/perf/tests/pmu-events.c                 | 466 +++++++----------
>  tools/perf/util/metricgroup.c                 | 275 ++++++----
>  tools/perf/util/metricgroup.h                 |   5 +-
>  tools/perf/util/pmu.c                         | 139 ++---
>  tools/perf/util/pmu.h                         |   8 +-
>  tools/perf/util/s390-sample-raw.c             |  50 +-
>  14 files changed, 1140 insertions(+), 718 deletions(-)
>  create mode 100644 tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.json
> 
> -- 
> 2.37.1.559.g78731f0fdb-goog

-- 

- Arnaldo
Re: [PATCH v4 00/17] Compress the pmu_event tables
Posted by Arnaldo Carvalho de Melo 3 years, 8 months ago
Em Tue, Aug 09, 2022 at 04:40:43PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Aug 04, 2022 at 03:17:59PM -0700, Ian Rogers escreveu:
> > jevents.py creates a number of large arrays from the json events. The
> > arrays contain pointers to strings that need relocating. The
> > relocations have file size, run time and memory costs. These changes
> > refactor the pmu_events API so that the storage of the pmu_event
> > struct isn't exposed. The format is then changed to an offset within a
> > combined big string, with adjacent pmu_event struct variables being
> > next to each other in the string separated by \0 - meaning only the
> > first variable of the struct needs its offset recording.
> > 
> > Some related fixes are contained with the patches. The architecture
> > jevents.py creates tables for can now be set by the JEVENTS_ARCH make
> > variable, with a new 'all' that generates the events and metrics for
> > all architectures.
> > 
> > An example of the improvement to the file size on x86 is:
> > no jevents - the same 19,788,464bytes
> > x86 jevents - ~16.7% file size saving 23,744,288bytes vs 28,502,632bytes
> > all jevents - ~19.5% file size saving 24,469,056bytes vs 30,379,920bytes
> > default build options plus NO_LIBBFD=1.
> 
> Applied the first four patches, waiting for the review comments to be

Sorry, three.

> discussed.
> 
> - Arnaldo
>  
> > I originally suggested fixing this problem in:
> > https://lore.kernel.org/linux-perf-users/CAP-5=fVB8G4bdb9T=FncRTh9oBVKCS=+=eowAO+YSgAhab+Dtg@mail.gmail.com/
> > 
> > v4. Fixed an issue with the empty-pmu-events.c spotted by John Garry
> >     <john.garry@huawei.com>.
> > v3. Fix an ARM build issue with a missed weak symbol. Perform some
> >     pytype clean up.
> > v2. Split the substring folding optimization to its own patch and
> >     comment tweaks as suggested by Namhyung Kim
> >     <namhyung@kernel.org>. Recompute the file size savings with the
> >     latest json events and metrics.
> > 
> > Ian Rogers (17):
> >   perf jevents: Clean up pytype warnings
> >   perf jevents: Simplify generation of C-string
> >   perf jevents: Add JEVENTS_ARCH make option
> >   perf jevent: Add an 'all' architecture argument
> >   perf jevents: Remove the type/version variables
> >   perf jevents: Provide path to json file on error
> >   perf jevents: Sort json files entries
> >   perf pmu-events: Hide pmu_sys_event_tables
> >   perf pmu-events: Avoid passing pmu_events_map
> >   perf pmu-events: Hide pmu_events_map
> >   perf test: Use full metric resolution
> >   perf pmu-events: Move test events/metrics to json
> >   perf pmu-events: Don't assume pmu_event is an array
> >   perf pmu-events: Hide the pmu_events
> >   perf metrics: Copy entire pmu_event in find metric
> >   perf jevents: Compress the pmu_events_table
> >   perf jevents: Fold strings optimization
> > 
> >  tools/perf/arch/arm64/util/pmu.c              |   4 +-
> >  tools/perf/pmu-events/Build                   |   6 +-
> >  .../arch/test/test_soc/cpu/metrics.json       |  64 +++
> >  tools/perf/pmu-events/empty-pmu-events.c      | 204 +++++++-
> >  tools/perf/pmu-events/jevents.py              | 495 ++++++++++++++----
> >  tools/perf/pmu-events/pmu-events.h            |  40 +-
> >  tools/perf/tests/expand-cgroup.c              |  25 +-
> >  tools/perf/tests/parse-metric.c               |  77 +--
> >  tools/perf/tests/pmu-events.c                 | 466 +++++++----------
> >  tools/perf/util/metricgroup.c                 | 275 ++++++----
> >  tools/perf/util/metricgroup.h                 |   5 +-
> >  tools/perf/util/pmu.c                         | 139 ++---
> >  tools/perf/util/pmu.h                         |   8 +-
> >  tools/perf/util/s390-sample-raw.c             |  50 +-
> >  14 files changed, 1140 insertions(+), 718 deletions(-)
> >  create mode 100644 tools/perf/pmu-events/arch/test/test_soc/cpu/metrics.json
> > 
> > -- 
> > 2.37.1.559.g78731f0fdb-goog
> 
> -- 
> 
> - Arnaldo

-- 

- Arnaldo