[PATCH v5 00/21] Python module cleanup

Ian Rogers posted 21 patches 3 weeks, 3 days ago
There is a newer version of this series
tools/perf/Makefile.perf                      |   7 +-
tools/perf/arch/x86/util/Build                |   2 +-
tools/perf/bench/inject-buildid.c             |  13 +-
tools/perf/builtin-kvm.c                      |  61 ----
tools/perf/builtin-kwork.c                    |   3 +-
tools/perf/builtin-lock.c                     | 137 +------
tools/perf/builtin-script.c                   | 304 +---------------
tools/perf/builtin-stat.c                     |  27 --
tools/perf/builtin-trace.c                    |   1 -
tools/perf/builtin.h                          |   6 -
.../scripts/python/Perf-Trace-Util/Context.c  |  20 +-
tools/perf/tests/stat.c                       |  16 +-
tools/perf/trace/beauty/arch_errno_names.sh   |   3 +-
tools/perf/ui/browsers/scripts.c              | 177 ++++++++-
tools/perf/util/Build                         |   4 +-
tools/perf/util/bpf_kwork.c                   |   2 +-
tools/perf/util/bpf_kwork_top.c               |   2 +-
tools/perf/util/bpf_lock_contention.c         |   2 +-
tools/perf/util/cgroup.c                      |   2 +-
tools/perf/util/config.c                      |  27 ++
tools/perf/util/dlfilter.c                    |   3 +-
tools/perf/util/env.c                         |   4 +
tools/perf/util/env.h                         |   2 -
tools/perf/util/evsel.c                       |  19 +-
tools/perf/util/evsel.h                       |   2 +-
tools/perf/util/intel-pt-decoder/Build        |  18 +-
.../intel-pt-decoder/intel-pt-insn-decoder.c  |   3 -
tools/perf/util/kvm-stat.c                    |  70 ++++
tools/perf/util/kvm-stat.h                    |   3 +
tools/perf/util/kwork.h                       |   6 +-
tools/perf/util/lock-contention.c             | 170 +++++++++
tools/perf/util/lock-contention.h             |  37 +-
tools/perf/util/path.c                        |  10 +
tools/perf/util/path.h                        |   1 +
tools/perf/util/python.c                      | 338 ++++++++----------
.../scripting-engines/trace-event-python.c    |  63 +---
tools/perf/util/stat.h                        |   3 +-
tools/perf/util/trace-event-scripting.c       | 177 +++++++++
tools/perf/util/trace-event.h                 |   5 +-
39 files changed, 872 insertions(+), 878 deletions(-)
create mode 100644 tools/perf/util/kvm-stat.c
create mode 100644 tools/perf/util/lock-contention.c
[PATCH v5 00/21] Python module cleanup
Posted by Ian Rogers 3 weeks, 3 days ago
This patch:
 - removes workarounds for Python 2 module support due to long
   deprecation and challenges in developing new code;
 - constifies variables and parameters to functions;
 - removes python.c stub code which existed due to missing functions
   that are defined in the builtin-* files, in general the builtin-*
   code is moved into util;
 - remove bench and test perf C code from the python module;
 - adds parse_events to the python perf module.
 - improves upon some of the existing python perf module functins.

v5. Rebase. Fix NO_LIBBPF and NO_AUXTRACE related build failures
    caught by Arnaldo and the build-test. Fix NO_AUXTRACE x86
    arch_fetch_insn in the process, which was incorrectly using an
    empty weak symbol stub.
v4. Rebase. Fix the parse events evsel to be embedded in a
    pyrf_evsel. Add __str__/__repr__ functions to evlist and
    evsel. Throw an exception for a bad evlist index.
v3. Move is_directory_at to patch 6 rather than patch 7, respond to
    review feedback on the list from Namhyung.
v2. Add the bottom 4 bullet points - 13 more patches.

Ian Rogers (21):
  perf python: Remove python 2 scripting support
  perf python: Constify variables and parameters
  perf python: Remove unused #include
  perf script: Move scripting_max_stack out of builtin
  perf kvm: Move functions used in util out of builtin
  perf script: Move find_scripts to browser/scripts.c
  perf stat: Move stat_config into config.c
  perf script: Move script_spec code to trace-event-scripting.c
  perf script: Move script_fetch_insn to trace-event-scripting.c
  perf script: Move perf_sample__sprintf_flags to
    trace-event-scripting.c
  perf x86: Define arch_fetch_insn in NO_AUXTRACE builds
  perf intel-pt: Remove stale build comment
  perf env: Move arch errno function to only use in env
  perf lock: Move common lock contention code to new file
  perf bench: Remove reference to cmd_inject
  perf kwork: Make perf_kwork_add_work a callback
  perf build: Remove test library from python shared object
  perf python: Add parse_events function
  perf python: Add __str__ and __repr__ functions to evlist
  perf python: Add __str__ and __repr__ functions to evsel
  perf python: Correctly throw IndexError

 tools/perf/Makefile.perf                      |   7 +-
 tools/perf/arch/x86/util/Build                |   2 +-
 tools/perf/bench/inject-buildid.c             |  13 +-
 tools/perf/builtin-kvm.c                      |  61 ----
 tools/perf/builtin-kwork.c                    |   3 +-
 tools/perf/builtin-lock.c                     | 137 +------
 tools/perf/builtin-script.c                   | 304 +---------------
 tools/perf/builtin-stat.c                     |  27 --
 tools/perf/builtin-trace.c                    |   1 -
 tools/perf/builtin.h                          |   6 -
 .../scripts/python/Perf-Trace-Util/Context.c  |  20 +-
 tools/perf/tests/stat.c                       |  16 +-
 tools/perf/trace/beauty/arch_errno_names.sh   |   3 +-
 tools/perf/ui/browsers/scripts.c              | 177 ++++++++-
 tools/perf/util/Build                         |   4 +-
 tools/perf/util/bpf_kwork.c                   |   2 +-
 tools/perf/util/bpf_kwork_top.c               |   2 +-
 tools/perf/util/bpf_lock_contention.c         |   2 +-
 tools/perf/util/cgroup.c                      |   2 +-
 tools/perf/util/config.c                      |  27 ++
 tools/perf/util/dlfilter.c                    |   3 +-
 tools/perf/util/env.c                         |   4 +
 tools/perf/util/env.h                         |   2 -
 tools/perf/util/evsel.c                       |  19 +-
 tools/perf/util/evsel.h                       |   2 +-
 tools/perf/util/intel-pt-decoder/Build        |  18 +-
 .../intel-pt-decoder/intel-pt-insn-decoder.c  |   3 -
 tools/perf/util/kvm-stat.c                    |  70 ++++
 tools/perf/util/kvm-stat.h                    |   3 +
 tools/perf/util/kwork.h                       |   6 +-
 tools/perf/util/lock-contention.c             | 170 +++++++++
 tools/perf/util/lock-contention.h             |  37 +-
 tools/perf/util/path.c                        |  10 +
 tools/perf/util/path.h                        |   1 +
 tools/perf/util/python.c                      | 338 ++++++++----------
 .../scripting-engines/trace-event-python.c    |  63 +---
 tools/perf/util/stat.h                        |   3 +-
 tools/perf/util/trace-event-scripting.c       | 177 +++++++++
 tools/perf/util/trace-event.h                 |   5 +-
 39 files changed, 872 insertions(+), 878 deletions(-)
 create mode 100644 tools/perf/util/kvm-stat.c
 create mode 100644 tools/perf/util/lock-contention.c

-- 
2.47.0.163.g1226f6d8fa-goog
Re: [PATCH v5 00/21] Python module cleanup
Posted by Arnaldo Carvalho de Melo 3 weeks, 3 days ago
On Wed, Oct 30, 2024 at 06:42:31PM -0700, Ian Rogers wrote:
> This patch:
>  - removes workarounds for Python 2 module support due to long
>    deprecation and challenges in developing new code;
>  - constifies variables and parameters to functions;
>  - removes python.c stub code which existed due to missing functions
>    that are defined in the builtin-* files, in general the builtin-*
>    code is moved into util;
>  - remove bench and test perf C code from the python module;
>  - adds parse_events to the python perf module.
>  - improves upon some of the existing python perf module functins.
> 
> v5. Rebase. Fix NO_LIBBPF and NO_AUXTRACE related build failures
>     caught by Arnaldo and the build-test. Fix NO_AUXTRACE x86
>     arch_fetch_insn in the process, which was incorrectly using an
>     empty weak symbol stub.


I created a 'perf test' shell entry to test the python binding and ran
it after each relevant patch in this series, as well as doing 'perf
script -' to list available scripts, all seem to work.

Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>

I'll send the perf test entry now.

- Arnaldo


> v4. Rebase. Fix the parse events evsel to be embedded in a
>     pyrf_evsel. Add __str__/__repr__ functions to evlist and
>     evsel. Throw an exception for a bad evlist index.
> v3. Move is_directory_at to patch 6 rather than patch 7, respond to
>     review feedback on the list from Namhyung.
> v2. Add the bottom 4 bullet points - 13 more patches.
> 
> Ian Rogers (21):
>   perf python: Remove python 2 scripting support
>   perf python: Constify variables and parameters
>   perf python: Remove unused #include
>   perf script: Move scripting_max_stack out of builtin
>   perf kvm: Move functions used in util out of builtin
>   perf script: Move find_scripts to browser/scripts.c
>   perf stat: Move stat_config into config.c
>   perf script: Move script_spec code to trace-event-scripting.c
>   perf script: Move script_fetch_insn to trace-event-scripting.c
>   perf script: Move perf_sample__sprintf_flags to
>     trace-event-scripting.c
>   perf x86: Define arch_fetch_insn in NO_AUXTRACE builds
>   perf intel-pt: Remove stale build comment
>   perf env: Move arch errno function to only use in env
>   perf lock: Move common lock contention code to new file
>   perf bench: Remove reference to cmd_inject
>   perf kwork: Make perf_kwork_add_work a callback
>   perf build: Remove test library from python shared object
>   perf python: Add parse_events function
>   perf python: Add __str__ and __repr__ functions to evlist
>   perf python: Add __str__ and __repr__ functions to evsel
>   perf python: Correctly throw IndexError
> 
>  tools/perf/Makefile.perf                      |   7 +-
>  tools/perf/arch/x86/util/Build                |   2 +-
>  tools/perf/bench/inject-buildid.c             |  13 +-
>  tools/perf/builtin-kvm.c                      |  61 ----
>  tools/perf/builtin-kwork.c                    |   3 +-
>  tools/perf/builtin-lock.c                     | 137 +------
>  tools/perf/builtin-script.c                   | 304 +---------------
>  tools/perf/builtin-stat.c                     |  27 --
>  tools/perf/builtin-trace.c                    |   1 -
>  tools/perf/builtin.h                          |   6 -
>  .../scripts/python/Perf-Trace-Util/Context.c  |  20 +-
>  tools/perf/tests/stat.c                       |  16 +-
>  tools/perf/trace/beauty/arch_errno_names.sh   |   3 +-
>  tools/perf/ui/browsers/scripts.c              | 177 ++++++++-
>  tools/perf/util/Build                         |   4 +-
>  tools/perf/util/bpf_kwork.c                   |   2 +-
>  tools/perf/util/bpf_kwork_top.c               |   2 +-
>  tools/perf/util/bpf_lock_contention.c         |   2 +-
>  tools/perf/util/cgroup.c                      |   2 +-
>  tools/perf/util/config.c                      |  27 ++
>  tools/perf/util/dlfilter.c                    |   3 +-
>  tools/perf/util/env.c                         |   4 +
>  tools/perf/util/env.h                         |   2 -
>  tools/perf/util/evsel.c                       |  19 +-
>  tools/perf/util/evsel.h                       |   2 +-
>  tools/perf/util/intel-pt-decoder/Build        |  18 +-
>  .../intel-pt-decoder/intel-pt-insn-decoder.c  |   3 -
>  tools/perf/util/kvm-stat.c                    |  70 ++++
>  tools/perf/util/kvm-stat.h                    |   3 +
>  tools/perf/util/kwork.h                       |   6 +-
>  tools/perf/util/lock-contention.c             | 170 +++++++++
>  tools/perf/util/lock-contention.h             |  37 +-
>  tools/perf/util/path.c                        |  10 +
>  tools/perf/util/path.h                        |   1 +
>  tools/perf/util/python.c                      | 338 ++++++++----------
>  .../scripting-engines/trace-event-python.c    |  63 +---
>  tools/perf/util/stat.h                        |   3 +-
>  tools/perf/util/trace-event-scripting.c       | 177 +++++++++
>  tools/perf/util/trace-event.h                 |   5 +-
>  39 files changed, 872 insertions(+), 878 deletions(-)
>  create mode 100644 tools/perf/util/kvm-stat.c
>  create mode 100644 tools/perf/util/lock-contention.c
> 
> -- 
> 2.47.0.163.g1226f6d8fa-goog
>
Re: [PATCH v5 00/21] Python module cleanup
Posted by Ian Rogers 3 weeks, 3 days ago
On Thu, Oct 31, 2024 at 12:40 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> On Wed, Oct 30, 2024 at 06:42:31PM -0700, Ian Rogers wrote:
> > This patch:
> >  - removes workarounds for Python 2 module support due to long
> >    deprecation and challenges in developing new code;
> >  - constifies variables and parameters to functions;
> >  - removes python.c stub code which existed due to missing functions
> >    that are defined in the builtin-* files, in general the builtin-*
> >    code is moved into util;
> >  - remove bench and test perf C code from the python module;
> >  - adds parse_events to the python perf module.
> >  - improves upon some of the existing python perf module functins.
> >
> > v5. Rebase. Fix NO_LIBBPF and NO_AUXTRACE related build failures
> >     caught by Arnaldo and the build-test. Fix NO_AUXTRACE x86
> >     arch_fetch_insn in the process, which was incorrectly using an
> >     empty weak symbol stub.
>
>
> I created a 'perf test' shell entry to test the python binding and ran
> it after each relevant patch in this series, as well as doing 'perf
> script -' to list available scripts, all seem to work.
>
> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>

There is coverage of just "import perf" in:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/python-use.c?h=perf-tools-next
Which I'd been using throughout creating the series.

Thanks,
Ian

> I'll send the perf test entry now.
>
> - Arnaldo
>
>
> > v4. Rebase. Fix the parse events evsel to be embedded in a
> >     pyrf_evsel. Add __str__/__repr__ functions to evlist and
> >     evsel. Throw an exception for a bad evlist index.
> > v3. Move is_directory_at to patch 6 rather than patch 7, respond to
> >     review feedback on the list from Namhyung.
> > v2. Add the bottom 4 bullet points - 13 more patches.
> >
> > Ian Rogers (21):
> >   perf python: Remove python 2 scripting support
> >   perf python: Constify variables and parameters
> >   perf python: Remove unused #include
> >   perf script: Move scripting_max_stack out of builtin
> >   perf kvm: Move functions used in util out of builtin
> >   perf script: Move find_scripts to browser/scripts.c
> >   perf stat: Move stat_config into config.c
> >   perf script: Move script_spec code to trace-event-scripting.c
> >   perf script: Move script_fetch_insn to trace-event-scripting.c
> >   perf script: Move perf_sample__sprintf_flags to
> >     trace-event-scripting.c
> >   perf x86: Define arch_fetch_insn in NO_AUXTRACE builds
> >   perf intel-pt: Remove stale build comment
> >   perf env: Move arch errno function to only use in env
> >   perf lock: Move common lock contention code to new file
> >   perf bench: Remove reference to cmd_inject
> >   perf kwork: Make perf_kwork_add_work a callback
> >   perf build: Remove test library from python shared object
> >   perf python: Add parse_events function
> >   perf python: Add __str__ and __repr__ functions to evlist
> >   perf python: Add __str__ and __repr__ functions to evsel
> >   perf python: Correctly throw IndexError
> >
> >  tools/perf/Makefile.perf                      |   7 +-
> >  tools/perf/arch/x86/util/Build                |   2 +-
> >  tools/perf/bench/inject-buildid.c             |  13 +-
> >  tools/perf/builtin-kvm.c                      |  61 ----
> >  tools/perf/builtin-kwork.c                    |   3 +-
> >  tools/perf/builtin-lock.c                     | 137 +------
> >  tools/perf/builtin-script.c                   | 304 +---------------
> >  tools/perf/builtin-stat.c                     |  27 --
> >  tools/perf/builtin-trace.c                    |   1 -
> >  tools/perf/builtin.h                          |   6 -
> >  .../scripts/python/Perf-Trace-Util/Context.c  |  20 +-
> >  tools/perf/tests/stat.c                       |  16 +-
> >  tools/perf/trace/beauty/arch_errno_names.sh   |   3 +-
> >  tools/perf/ui/browsers/scripts.c              | 177 ++++++++-
> >  tools/perf/util/Build                         |   4 +-
> >  tools/perf/util/bpf_kwork.c                   |   2 +-
> >  tools/perf/util/bpf_kwork_top.c               |   2 +-
> >  tools/perf/util/bpf_lock_contention.c         |   2 +-
> >  tools/perf/util/cgroup.c                      |   2 +-
> >  tools/perf/util/config.c                      |  27 ++
> >  tools/perf/util/dlfilter.c                    |   3 +-
> >  tools/perf/util/env.c                         |   4 +
> >  tools/perf/util/env.h                         |   2 -
> >  tools/perf/util/evsel.c                       |  19 +-
> >  tools/perf/util/evsel.h                       |   2 +-
> >  tools/perf/util/intel-pt-decoder/Build        |  18 +-
> >  .../intel-pt-decoder/intel-pt-insn-decoder.c  |   3 -
> >  tools/perf/util/kvm-stat.c                    |  70 ++++
> >  tools/perf/util/kvm-stat.h                    |   3 +
> >  tools/perf/util/kwork.h                       |   6 +-
> >  tools/perf/util/lock-contention.c             | 170 +++++++++
> >  tools/perf/util/lock-contention.h             |  37 +-
> >  tools/perf/util/path.c                        |  10 +
> >  tools/perf/util/path.h                        |   1 +
> >  tools/perf/util/python.c                      | 338 ++++++++----------
> >  .../scripting-engines/trace-event-python.c    |  63 +---
> >  tools/perf/util/stat.h                        |   3 +-
> >  tools/perf/util/trace-event-scripting.c       | 177 +++++++++
> >  tools/perf/util/trace-event.h                 |   5 +-
> >  39 files changed, 872 insertions(+), 878 deletions(-)
> >  create mode 100644 tools/perf/util/kvm-stat.c
> >  create mode 100644 tools/perf/util/lock-contention.c
> >
> > --
> > 2.47.0.163.g1226f6d8fa-goog
> >