Hello,
I've implemented 'latency' subcommand in the perf ftrace command to
show a histogram of function latency.
v2 changes)
* fix build errors (Arnaldo)
* add Arnaldo's test results in the commit message
* show warning when -b is used and BPF support is not enabled (Athira)
* use skel->links to save bpf links (Song)
To handle new subcommands, the existing functionality is moved to
'trace' subcommand while preserving backward compatibility of not
having a subcommand at all (defaults to 'trace').
The latency subcommand accepts a target (kernel, for now) function
with -T option and shows a histogram like below:
$ sudo ./perf ftrace latency -a -T mutex_lock sleep 1
# DURATION | COUNT | GRAPH |
0 - 1 us | 2686 | ###################### |
1 - 2 us | 976 | ######## |
2 - 4 us | 879 | ####### |
4 - 8 us | 481 | #### |
8 - 16 us | 445 | ### |
16 - 32 us | 1 | |
32 - 64 us | 0 | |
64 - 128 us | 0 | |
128 - 256 us | 0 | |
256 - 512 us | 0 | |
512 - 1024 us | 0 | |
1 - 2 ms | 0 | |
2 - 4 ms | 0 | |
4 - 8 ms | 0 | |
8 - 16 ms | 0 | |
16 - 32 ms | 0 | |
32 - 64 ms | 0 | |
64 - 128 ms | 0 | |
128 - 256 ms | 0 | |
256 - 512 ms | 0 | |
512 - 1024 ms | 0 | |
1 - ... s | 0 | |
It basically use the function graph tracer to extract the duration of
the function. But with -b/--use-bpf option, it can use BPF to save
the histogram in the kernel. For the same function, it gets:
$ sudo ./perf ftrace latency -a -b -T mutex_lock sleep 1
# DURATION | COUNT | GRAPH |
0 - 1 us | 4682 | ############################################# |
1 - 2 us | 11 | |
2 - 4 us | 0 | |
4 - 8 us | 0 | |
8 - 16 us | 7 | |
16 - 32 us | 6 | |
32 - 64 us | 0 | |
64 - 128 us | 0 | |
128 - 256 us | 0 | |
256 - 512 us | 0 | |
512 - 1024 us | 0 | |
1 - 2 ms | 0 | |
2 - 4 ms | 0 | |
4 - 8 ms | 0 | |
8 - 16 ms | 0 | |
16 - 32 ms | 0 | |
32 - 64 ms | 0 | |
64 - 128 ms | 0 | |
128 - 256 ms | 0 | |
256 - 512 ms | 0 | |
512 - 1024 ms | 0 | |
1 - ... s | 0 | |
You can get the patches at 'perf/ftrace-latency-v2' branch on
https://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Thanks,
Namhyung
Namhyung Kim (5):
perf ftrace: Add 'trace' subcommand
perf ftrace: Move out common code from __cmd_ftrace
perf ftrace: Add 'latency' subcommand
perf ftrace: Add -b/--use-bpf option for latency subcommand
perf ftrace: Implement cpu and task filters in BPF
tools/perf/Makefile.perf | 2 +-
tools/perf/builtin-ftrace.c | 445 +++++++++++++++++---
tools/perf/util/Build | 1 +
tools/perf/util/bpf_ftrace.c | 152 +++++++
tools/perf/util/bpf_skel/func_latency.bpf.c | 114 +++++
tools/perf/util/ftrace.h | 81 ++++
6 files changed, 725 insertions(+), 70 deletions(-)
create mode 100644 tools/perf/util/bpf_ftrace.c
create mode 100644 tools/perf/util/bpf_skel/func_latency.bpf.c
create mode 100644 tools/perf/util/ftrace.h
base-commit: 8ff4f20f3eb55dea0dbbe5e32043ab6b7427882f
--
2.34.1.173.g76aa8bc2d0-goog