Track processes properly for perf record --off-cpu (v2)

[PATCH 0/4] Track processes properly for perf record --off-cpu (v2)

Posted by Namhyung Kim 3 years, 8 months ago

Hello,

This patch series implements inheritance of offcpu events for the
child processes.  Unlike perf events, BPF cannot know which task it
should track except for ones set in a BPF map at the beginning.  Add
another BPF program to the fork path and add the process id to the map
if the parent is tracked.

Changes in v2)
 * drop already merged fixes
 * fix the shell test to omit noises
 
With this change, it can get the correct off-cpu events for child
processes.  I've tested it with perf bench sched messaging which
creates a lot of processes.

  $ sudo perf record -e dummy --off-cpu -- perf bench sched messaging
  # Running 'sched/messaging' benchmark:
  # 20 sender and receiver processes per group
  # 10 groups == 400 processes run

       Total time: 0.196 [sec]
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.178 MB perf.data (851 samples) ]


  $ sudo perf report --stat | grep -A1 offcpu
  offcpu-time stats:
            SAMPLE events:        851

The benchmark passes messages by read/write and it creates off-cpu
events.  With 400 processes, we can see more than 800 events.

The child process tracking is also enabled when -p option is given.
But -t option does NOT as it only cares about the specific threads.
It may be different what perf_event does now, but I think it makes
more sense.

You can get it from 'perf/offcpu-child-v2' branch in my tree

  https://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung


Namhyung Kim (4):
  perf offcpu: Check process id for the given workload
  perf offcpu: Parse process id separately
  perf offcpu: Track child processes
  perf offcpu: Update offcpu test for child process

 tools/perf/tests/shell/record_offcpu.sh | 57 ++++++++++++++++++++++---
 tools/perf/util/bpf_off_cpu.c           | 53 ++++++++++++++++++++++-
 tools/perf/util/bpf_skel/off_cpu.bpf.c  | 38 ++++++++++++++++-
 3 files changed, 138 insertions(+), 10 deletions(-)


base-commit: b39c9e1b101d2992de9981673919ae55a088792c
-- 
2.37.1.595.g718a3a8f04-goog

Re: [PATCH 0/4] Track processes properly for perf record --off-cpu (v2)

Posted by Arnaldo Carvalho de Melo 3 years, 8 months ago

Em Thu, Aug 11, 2022 at 11:54:52AM -0700, Namhyung Kim escreveu:
> Hello,
> 
> This patch series implements inheritance of offcpu events for the
> child processes.  Unlike perf events, BPF cannot know which task it
> should track except for ones set in a BPF map at the beginning.  Add
> another BPF program to the fork path and add the process id to the map
> if the parent is tracked.

Thanks for resubmitting, applied!

Will be up in perf/core as soon as tests finish.

- Arnaldo
 
> Changes in v2)
>  * drop already merged fixes
>  * fix the shell test to omit noises
>  
> With this change, it can get the correct off-cpu events for child
> processes.  I've tested it with perf bench sched messaging which
> creates a lot of processes.
> 
>   $ sudo perf record -e dummy --off-cpu -- perf bench sched messaging
>   # Running 'sched/messaging' benchmark:
>   # 20 sender and receiver processes per group
>   # 10 groups == 400 processes run
> 
>        Total time: 0.196 [sec]
>   [ perf record: Woken up 1 times to write data ]
>   [ perf record: Captured and wrote 0.178 MB perf.data (851 samples) ]
> 
> 
>   $ sudo perf report --stat | grep -A1 offcpu
>   offcpu-time stats:
>             SAMPLE events:        851
> 
> The benchmark passes messages by read/write and it creates off-cpu
> events.  With 400 processes, we can see more than 800 events.
> 
> The child process tracking is also enabled when -p option is given.
> But -t option does NOT as it only cares about the specific threads.
> It may be different what perf_event does now, but I think it makes
> more sense.
> 
> You can get it from 'perf/offcpu-child-v2' branch in my tree
> 
>   https://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
> 
> Thanks,
> Namhyung
> 
> 
> Namhyung Kim (4):
>   perf offcpu: Check process id for the given workload
>   perf offcpu: Parse process id separately
>   perf offcpu: Track child processes
>   perf offcpu: Update offcpu test for child process
> 
>  tools/perf/tests/shell/record_offcpu.sh | 57 ++++++++++++++++++++++---
>  tools/perf/util/bpf_off_cpu.c           | 53 ++++++++++++++++++++++-
>  tools/perf/util/bpf_skel/off_cpu.bpf.c  | 38 ++++++++++++++++-
>  3 files changed, 138 insertions(+), 10 deletions(-)
> 
> 
> base-commit: b39c9e1b101d2992de9981673919ae55a088792c
> -- 
> 2.37.1.595.g718a3a8f04-goog

-- 

- Arnaldo