[PATCHSET 0/7] perf lock: New lock contention tracepoints support (v4)

Namhyung Kim posted 7 patches 3 years, 10 months ago
tools/perf/Documentation/perf-lock.txt   |   7 +
tools/perf/Documentation/perf-record.txt |   5 +
tools/perf/builtin-lock.c                | 426 ++++++++++++++++++++++-
tools/perf/util/callchain.c              |  18 +-
4 files changed, 434 insertions(+), 22 deletions(-)
[PATCHSET 0/7] perf lock: New lock contention tracepoints support (v4)
Posted by Namhyung Kim 3 years, 10 months ago
Hello,

Kernel v5.19 will have a new set of tracepoints to track lock
contentions for various lock types.  Unlike tracepoints in LOCKDEP and
LOCK_STAT, it's hit only for contended locks and lock names are not
available.  So it needs to collect stack traces and display the caller
function instead.

Changes in v4)
 * add Acked-by from Ian
 * more comments on trace_lock_handler
 * don't create stats in the contention_end handler
 
Changes in v3)
 * fix build error
 * support data from different kernels/machines
 * skip bad stat unless there's actual bad ones
 
Changes in v2)
 * add Acked-by from Ian
 * print time with a unit for compact output
 * add some comments  (Ian)
 * remove already applied patch
 
This patchset merely adds support for the new tracepoints to the
existing perf lock commands.  So there's no change to the user.  Later
I'll add new a sub-command dedicated to the tracepoints to make use of
the additional information.

Example output:

  $ sudo perf lock record -a sleep 3

  $ perf lock report -F acquired,contended,avg_wait,wait_total

                  Name   acquired  contended     avg wait    total wait

   update_blocked_a...         40         40      3.61 us     144.45 us
   kernfs_fop_open+...          5          5      3.64 us      18.18 us
    _nohz_idle_balance          3          3      2.65 us       7.95 us
   tick_do_update_j...          1          1      6.04 us       6.04 us
    ep_scan_ready_list          1          1      3.93 us       3.93 us
  ...

You can find the code in the 'perf/lock-contention-v4' branch at

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung


Namhyung Kim (7):
  perf lock: Print wait times with unit
  perf lock: Allow to use different kernel symbols
  perf lock: Skip print_bad_events() if nothing bad
  perf lock: Add lock contention tracepoints record support
  perf lock: Handle lock contention tracepoints
  perf record: Allow to specify max stack depth of fp callchain
  perf lock: Look up callchain for the contended locks

 tools/perf/Documentation/perf-lock.txt   |   7 +
 tools/perf/Documentation/perf-record.txt |   5 +
 tools/perf/builtin-lock.c                | 426 ++++++++++++++++++++++-
 tools/perf/util/callchain.c              |  18 +-
 4 files changed, 434 insertions(+), 22 deletions(-)


base-commit: 9886142c7a2226439c1e3f7d9b69f9c7094c3ef6
-- 
2.36.1.476.g0c4daa206d-goog
Re: [PATCHSET 0/7] perf lock: New lock contention tracepoints support (v4)
Posted by Arnaldo Carvalho de Melo 3 years, 9 months ago
Em Wed, Jun 15, 2022 at 09:32:15AM -0700, Namhyung Kim escreveu:
> Hello,
> 
> Kernel v5.19 will have a new set of tracepoints to track lock
> contentions for various lock types.  Unlike tracepoints in LOCKDEP and
> LOCK_STAT, it's hit only for contended locks and lock names are not
> available.  So it needs to collect stack traces and display the caller
> function instead.

Applied to tmp.perf/core, performing some further tests and then will
push to perf/core.

Thanks for you work on this!

- Arnaldo
 
> Changes in v4)
>  * add Acked-by from Ian
>  * more comments on trace_lock_handler
>  * don't create stats in the contention_end handler
>  
> Changes in v3)
>  * fix build error
>  * support data from different kernels/machines
>  * skip bad stat unless there's actual bad ones
>  
> Changes in v2)
>  * add Acked-by from Ian
>  * print time with a unit for compact output
>  * add some comments  (Ian)
>  * remove already applied patch
>  
> This patchset merely adds support for the new tracepoints to the
> existing perf lock commands.  So there's no change to the user.  Later
> I'll add new a sub-command dedicated to the tracepoints to make use of
> the additional information.
> 
> Example output:
> 
>   $ sudo perf lock record -a sleep 3
> 
>   $ perf lock report -F acquired,contended,avg_wait,wait_total
> 
>                   Name   acquired  contended     avg wait    total wait
> 
>    update_blocked_a...         40         40      3.61 us     144.45 us
>    kernfs_fop_open+...          5          5      3.64 us      18.18 us
>     _nohz_idle_balance          3          3      2.65 us       7.95 us
>    tick_do_update_j...          1          1      6.04 us       6.04 us
>     ep_scan_ready_list          1          1      3.93 us       3.93 us
>   ...
> 
> You can find the code in the 'perf/lock-contention-v4' branch at
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
> 
> Thanks,
> Namhyung
> 
> 
> Namhyung Kim (7):
>   perf lock: Print wait times with unit
>   perf lock: Allow to use different kernel symbols
>   perf lock: Skip print_bad_events() if nothing bad
>   perf lock: Add lock contention tracepoints record support
>   perf lock: Handle lock contention tracepoints
>   perf record: Allow to specify max stack depth of fp callchain
>   perf lock: Look up callchain for the contended locks
> 
>  tools/perf/Documentation/perf-lock.txt   |   7 +
>  tools/perf/Documentation/perf-record.txt |   5 +
>  tools/perf/builtin-lock.c                | 426 ++++++++++++++++++++++-
>  tools/perf/util/callchain.c              |  18 +-
>  4 files changed, 434 insertions(+), 22 deletions(-)
> 
> 
> base-commit: 9886142c7a2226439c1e3f7d9b69f9c7094c3ef6
> -- 
> 2.36.1.476.g0c4daa206d-goog

-- 

- Arnaldo
Re: [PATCHSET 0/7] perf lock: New lock contention tracepoints support (v4)
Posted by Namhyung Kim 3 years, 10 months ago
Ping!  Any comments?


On Wed, Jun 15, 2022 at 9:32 AM Namhyung Kim <namhyung@kernel.org> wrote:
>
> Hello,
>
> Kernel v5.19 will have a new set of tracepoints to track lock
> contentions for various lock types.  Unlike tracepoints in LOCKDEP and
> LOCK_STAT, it's hit only for contended locks and lock names are not
> available.  So it needs to collect stack traces and display the caller
> function instead.
>
> Changes in v4)
>  * add Acked-by from Ian
>  * more comments on trace_lock_handler
>  * don't create stats in the contention_end handler
>
> Changes in v3)
>  * fix build error
>  * support data from different kernels/machines
>  * skip bad stat unless there's actual bad ones
>
> Changes in v2)
>  * add Acked-by from Ian
>  * print time with a unit for compact output
>  * add some comments  (Ian)
>  * remove already applied patch
>
> This patchset merely adds support for the new tracepoints to the
> existing perf lock commands.  So there's no change to the user.  Later
> I'll add new a sub-command dedicated to the tracepoints to make use of
> the additional information.
>
> Example output:
>
>   $ sudo perf lock record -a sleep 3
>
>   $ perf lock report -F acquired,contended,avg_wait,wait_total
>
>                   Name   acquired  contended     avg wait    total wait
>
>    update_blocked_a...         40         40      3.61 us     144.45 us
>    kernfs_fop_open+...          5          5      3.64 us      18.18 us
>     _nohz_idle_balance          3          3      2.65 us       7.95 us
>    tick_do_update_j...          1          1      6.04 us       6.04 us
>     ep_scan_ready_list          1          1      3.93 us       3.93 us
>   ...
>
> You can find the code in the 'perf/lock-contention-v4' branch at
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
>
> Thanks,
> Namhyung
>
>
> Namhyung Kim (7):
>   perf lock: Print wait times with unit
>   perf lock: Allow to use different kernel symbols
>   perf lock: Skip print_bad_events() if nothing bad
>   perf lock: Add lock contention tracepoints record support
>   perf lock: Handle lock contention tracepoints
>   perf record: Allow to specify max stack depth of fp callchain
>   perf lock: Look up callchain for the contended locks
>
>  tools/perf/Documentation/perf-lock.txt   |   7 +
>  tools/perf/Documentation/perf-record.txt |   5 +
>  tools/perf/builtin-lock.c                | 426 ++++++++++++++++++++++-
>  tools/perf/util/callchain.c              |  18 +-
>  4 files changed, 434 insertions(+), 22 deletions(-)
>
>
> base-commit: 9886142c7a2226439c1e3f7d9b69f9c7094c3ef6
> --
> 2.36.1.476.g0c4daa206d-goog
>
Re: [PATCHSET 0/7] perf lock: New lock contention tracepoints support (v4)
Posted by Arnaldo Carvalho de Melo 3 years, 10 months ago
Em Fri, Jun 24, 2022 at 09:40:05AM -0700, Namhyung Kim escreveu:
>> Ping!  Any comments?

 
I'll take a look and test it soon.

- Arnaldo

> 
> On Wed, Jun 15, 2022 at 9:32 AM Namhyung Kim <namhyung@kernel.org> wrote:
> >
> > Hello,
> >
> > Kernel v5.19 will have a new set of tracepoints to track lock
> > contentions for various lock types.  Unlike tracepoints in LOCKDEP and
> > LOCK_STAT, it's hit only for contended locks and lock names are not
> > available.  So it needs to collect stack traces and display the caller
> > function instead.
> >
> > Changes in v4)
> >  * add Acked-by from Ian
> >  * more comments on trace_lock_handler
> >  * don't create stats in the contention_end handler
> >
> > Changes in v3)
> >  * fix build error
> >  * support data from different kernels/machines
> >  * skip bad stat unless there's actual bad ones
> >
> > Changes in v2)
> >  * add Acked-by from Ian
> >  * print time with a unit for compact output
> >  * add some comments  (Ian)
> >  * remove already applied patch
> >
> > This patchset merely adds support for the new tracepoints to the
> > existing perf lock commands.  So there's no change to the user.  Later
> > I'll add new a sub-command dedicated to the tracepoints to make use of
> > the additional information.
> >
> > Example output:
> >
> >   $ sudo perf lock record -a sleep 3
> >
> >   $ perf lock report -F acquired,contended,avg_wait,wait_total
> >
> >                   Name   acquired  contended     avg wait    total wait
> >
> >    update_blocked_a...         40         40      3.61 us     144.45 us
> >    kernfs_fop_open+...          5          5      3.64 us      18.18 us
> >     _nohz_idle_balance          3          3      2.65 us       7.95 us
> >    tick_do_update_j...          1          1      6.04 us       6.04 us
> >     ep_scan_ready_list          1          1      3.93 us       3.93 us
> >   ...
> >
> > You can find the code in the 'perf/lock-contention-v4' branch at
> >
> >   git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
> >
> > Thanks,
> > Namhyung
> >
> >
> > Namhyung Kim (7):
> >   perf lock: Print wait times with unit
> >   perf lock: Allow to use different kernel symbols
> >   perf lock: Skip print_bad_events() if nothing bad
> >   perf lock: Add lock contention tracepoints record support
> >   perf lock: Handle lock contention tracepoints
> >   perf record: Allow to specify max stack depth of fp callchain
> >   perf lock: Look up callchain for the contended locks
> >
> >  tools/perf/Documentation/perf-lock.txt   |   7 +
> >  tools/perf/Documentation/perf-record.txt |   5 +
> >  tools/perf/builtin-lock.c                | 426 ++++++++++++++++++++++-
> >  tools/perf/util/callchain.c              |  18 +-
> >  4 files changed, 434 insertions(+), 22 deletions(-)
> >
> >
> > base-commit: 9886142c7a2226439c1e3f7d9b69f9c7094c3ef6
> > --
> > 2.36.1.476.g0c4daa206d-goog
> >

-- 

- Arnaldo
Re: [PATCHSET 0/7] perf lock: New lock contention tracepoints support (v4)
Posted by Namhyung Kim 3 years, 10 months ago
Hi Arnaldo,

On Fri, Jun 24, 2022 at 11:59 AM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Fri, Jun 24, 2022 at 09:40:05AM -0700, Namhyung Kim escreveu:
> >> Ping!  Any comments?
>
>
> I'll take a look and test it soon.

Thanks for doing that!
Namhyung
Re: [PATCHSET 0/7] perf lock: New lock contention tracepoints support (v4)
Posted by Namhyung Kim 3 years, 9 months ago
Genple ping!

On Fri, Jun 24, 2022 at 2:38 PM Namhyung Kim <namhyung@kernel.org> wrote:
>
> Hi Arnaldo,
>
> On Fri, Jun 24, 2022 at 11:59 AM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > Em Fri, Jun 24, 2022 at 09:40:05AM -0700, Namhyung Kim escreveu:
> > >> Ping!  Any comments?
> >
> >
> > I'll take a look and test it soon.
>
> Thanks for doing that!
> Namhyung