From nobody Mon Oct 6 05:01:22 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C78EC2EA729; Fri, 25 Jul 2025 10:08:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753438127; cv=none; b=r16gftUjgXWKB4A4N1+2v2osomGFEVuLWPW/JaAFp2Y5nRJ275Gp2ELn9VU1Tc5M7o2HyZmYxObBLMscCXjPQj6hwOXMfaSpHEKNbRC6NYqqXkJEPuRI2GD63V+SaOndZkwAlbEkKyhj7nq1ADMz/cFhwaBywLqQsDkzAVzS8L0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753438127; c=relaxed/simple; bh=OwmCar/KYMErQOlBy3AccdX0oNiUo/QTaV+xvmFAxKw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=VV4HjT44smPM+3x/Pz+px0OnyiGhobIrhGGiWYJymB7ddryHhfhSFPDU2S61YUFM8TxpyW2B1VTInzskpc9zyAi8oN6IhZ6bbpq7k01G+/8QUDWXwZP7OzqzWFjSpq7DK/TU7MQrEBImnaLK7+bfpzPgd85lMAQd6QYZ0VVhkJo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B15E31A00; Fri, 25 Jul 2025 03:08:37 -0700 (PDT) Received: from e132581.arm.com (e132581.arm.com [10.1.196.87]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 590B73F5A1; Fri, 25 Jul 2025 03:08:40 -0700 (PDT) From: Leo Yan Date: Fri, 25 Jul 2025 11:08:14 +0100 Subject: [PATCH RESEND v3 4/6] perf: auxtrace: Add BPF userspace program for AUX pause and resume Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250725-perf_aux_pause_resume_bpf_rebase-v3-4-ae21deb49d1a@arm.com> References: <20250725-perf_aux_pause_resume_bpf_rebase-v3-0-ae21deb49d1a@arm.com> In-Reply-To: <20250725-perf_aux_pause_resume_bpf_rebase-v3-0-ae21deb49d1a@arm.com> To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Jiri Olsa , Ian Rogers , Adrian Hunter , KP Singh , Matt Bobrowski , Song Liu , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Yonghong Song , John Fastabend , Stanislav Fomichev , Hao Luo , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , James Clark , Suzuki K Poulose , Mike Leach Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Leo Yan X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1753438103; l=13437; i=leo.yan@arm.com; s=20250604; h=from:subject:message-id; bh=OwmCar/KYMErQOlBy3AccdX0oNiUo/QTaV+xvmFAxKw=; b=jGJ4P6jo8OV4kkKh90zidmGGLSq1iL5WRc92wzR9lwWTEbWSBIwC3qfkU72b4FLHWkpFN0gex UOj3tnBXbk0AOmRmf1NcpcnTmyvylhs9pNU+fqlTcC3kHWl92HUM7hu X-Developer-Key: i=leo.yan@arm.com; a=ed25519; pk=k4BaDbvkCXzBFA7Nw184KHGP5thju8lKqJYIrOWxDhI= This commit adds support for the BPF userspace program for AUX pause and resume. A list is maintained to track trigger points; each trigger point attaches to BPF programs when a session is opened and detaches when the session is closed. auxtrace__update_bpf_map() updates the AUX perf event pointer in the BPF map. The BPF kernel program then retrieves the event handler from the map to control AUX tracing. The auxtrace__set_bpf_filter() function updates the CPU and task filters for the BPF kernel program. Signed-off-by: Leo Yan --- tools/perf/util/Build | 4 + tools/perf/util/auxtrace.h | 43 ++++ tools/perf/util/bpf_auxtrace_pause.c | 408 +++++++++++++++++++++++++++++++= ++++ 3 files changed, 455 insertions(+) diff --git a/tools/perf/util/Build b/tools/perf/util/Build index 7910d908c814feec5e5e008f3a8b45384d796432..8ab29136344c3d37178f94aa1bd= 4b70ab54a7ab4 100644 --- a/tools/perf/util/Build +++ b/tools/perf/util/Build @@ -186,6 +186,10 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-util-$(CONFIG_PERF_BPF_SKEL) +=3D bpf_kwork_top.o endif =20 +ifeq ($(CONFIG_AUXTRACE),y) + perf-util-$(CONFIG_PERF_BPF_SKEL) +=3D bpf_auxtrace_pause.o +endif + perf-util-$(CONFIG_LIBELF) +=3D symbol-elf.o perf-util-$(CONFIG_LIBELF) +=3D probe-file.o perf-util-$(CONFIG_LIBELF) +=3D probe-event.o diff --git a/tools/perf/util/auxtrace.h b/tools/perf/util/auxtrace.h index b0db84d27b255dc2f1aff446012598b045bbd5d3..52831e501dea1ebe476aed103a9= 20b77d400e5f7 100644 --- a/tools/perf/util/auxtrace.h +++ b/tools/perf/util/auxtrace.h @@ -907,4 +907,47 @@ void itrace_synth_opts__clear_time_range(struct itrace= _synth_opts *opts =20 #endif =20 +#if defined(HAVE_AUXTRACE_SUPPORT) && defined(HAVE_BPF_SKEL) + +int auxtrace__prepare_bpf(struct auxtrace_record *itr, const char *str); +int auxtrace__set_bpf_filter(struct evlist *evlist, struct record_opts *op= ts); +int auxtrace__enable_bpf(void); +int auxtrace__cleanup_bpf(void); +int auxtrace__update_bpf_map(struct evsel *evsel, int cpu_map_idx, int fd); + +#else /* HAVE_AUXTRACE_SUPPORT && HAVE_BPF_SKEL */ + +static inline int auxtrace__prepare_bpf(struct auxtrace_record *itr + __maybe_unused, + const char *str __maybe_unused) +{ + return -EINVAL; +} + +static inline int auxtrace__set_bpf_filter(struct evlist *evlist __maybe_u= nused, + struct record_opts *opts + __maybe_unused) +{ + return -EINVAL; +} + +static inline int auxtrace__enable_bpf(void) +{ + return -EINVAL; +} + +static inline int auxtrace__cleanup_bpf(void) +{ + return -EINVAL; +} + +static int auxtrace__update_bpf_map(struct evsel *evsel __maybe_unused, + int cpu_map_idx __maybe_unused, + int fd __maybe_unused) +{ + return -EINVAL; +} + +#endif + #endif diff --git a/tools/perf/util/bpf_auxtrace_pause.c b/tools/perf/util/bpf_aux= trace_pause.c new file mode 100644 index 0000000000000000000000000000000000000000..ed77b1e19dcf9da65cacf98def3= 49c0ce9f83d46 --- /dev/null +++ b/tools/perf/util/bpf_auxtrace_pause.c @@ -0,0 +1,408 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* Copyright 2024 Arm Limited */ + +#include +#include +#include +#include + +#include + +#include "util/auxtrace.h" +#include "util/cpumap.h" +#include "util/thread_map.h" +#include "util/debug.h" +#include "util/evlist.h" +#include "util/bpf_counter.h" +#include "util/record.h" +#include "util/target.h" + +#include "util/bpf_skel/auxtrace_pause.skel.h" + +/* The valid controlling type is "p" (pause) and "r" (resume) */ +#define is_attach_kprobe(str) \ + (!strcmp((str), "kprobe") || !strcmp((str), "kretprobe")) +#define is_attach_uprobe(str) \ + (!strcmp((str), "uprobe") || !strcmp((str), "uretprobe")) +#define is_attach_tracepoint(str) \ + (!strcmp((str), "tp") || !strcmp((str), "tracepoint")) + +/* The valid controlling type is "p" (pause) and "r" (resume) */ +#define is_valid_ctrl_type(str) \ + (!strcmp((str), "p") || !strcmp((str), "r")) + +static struct auxtrace_pause_bpf *skel; + +struct trigger_entry { + struct list_head list; + char *arg0; + char *arg1; + char *arg2; + char *arg3; +}; + +static int trigger_entry_num; +static LIST_HEAD(trigger_list); +static struct bpf_link **trigger_links; + +static void auxtrace__free_bpf_trigger_list(void) +{ + struct trigger_entry *entry, *next; + + list_for_each_entry_safe(entry, next, &trigger_list, list) { + free(entry->arg0); + free(entry->arg1); + free(entry->arg2); + free(entry->arg3); + free(entry); + } + + trigger_entry_num =3D 0; +} + +static int auxtrace__alloc_bpf_trigger_list(const char *str) +{ + char *cmd_str; + char *substr, *saveptr1; + struct trigger_entry *entry; + int ret =3D 0; + + if (!str) + return -EINVAL; + + cmd_str =3D strdup(str); + if (!cmd_str) + return -ENOMEM; + + substr =3D strtok_r(cmd_str, ",", &saveptr1); + for ( ; substr !=3D NULL; substr =3D strtok_r(NULL, ",", &saveptr1)) { + char *fmt1_str, *fmt2_str, *fmt3_str, *fmt4_str, *fmt; + + entry =3D zalloc(sizeof(*entry)); + if (!entry) { + ret =3D -ENOMEM; + goto out; + } + + /* + * A trigger is expressed with several fields with separator ":". + * The first field is specified for attach types, it can be one + * of types listed below: + * kprobe / kretprobe + * uprobe / uretprobe + * tp / tracepoint + * + * The kprobe and kretprobe trigger format is: + * {kprobe|kretprobe}:{p|r}:function_name + * + * The uprobe and uretprobe trigger format is: + * {uprobe|uretprobe}:{p|r}:executable:function_name + * + * Tracepoint trigger format is: + * {tp|tracepoint}:{p|r}:category:tracepint_name + * + * The last field is used to express the controlling type: "p" + * means aux pause and "r" is for aux resume. + */ + fmt1_str =3D strtok_r(substr, ":", &fmt); + fmt2_str =3D strtok_r(NULL, ":", &fmt); + fmt3_str =3D strtok_r(NULL, ":", &fmt); + if (!fmt1_str || !fmt2_str || !fmt3_str) { + pr_err("Failed to parse bpf aux pause string: %s\n", + substr); + ret =3D -EINVAL; + goto out; + } + + entry->arg0 =3D strdup(fmt1_str); + entry->arg1 =3D strdup(fmt2_str); + entry->arg2 =3D strdup(fmt3_str); + if (!entry->arg0 || !entry->arg1 || !entry->arg2) { + ret =3D -ENOMEM; + goto out; + } + + if (!is_attach_kprobe(entry->arg0) && + !is_attach_uprobe(entry->arg0) && + !is_attach_tracepoint(entry->arg0)) { + pr_err("Failed to support bpf aux pause attach: %s\n", + entry->arg0); + ret =3D -EINVAL; + goto out; + } + + if (!is_valid_ctrl_type(entry->arg1)) { + pr_err("Failed to support bpf aux pause ctrl: %s\n", + entry->arg1); + ret =3D -EINVAL; + goto out; + } + + if (!is_attach_kprobe(entry->arg0)) { + fmt4_str =3D strtok_r(NULL, ":", &fmt); + if (!fmt4_str) { + ret =3D -ENOMEM; + goto out; + } + + entry->arg3 =3D strdup(fmt4_str); + if (!entry->arg3) { + ret =3D -ENOMEM; + goto out; + } + } + + if (ret) + goto out; + + list_add(&entry->list, &trigger_list); + trigger_entry_num++; + } + + free(cmd_str); + return 0; + +out: + free(cmd_str); + if (entry) { + free(entry->arg0); + free(entry->arg1); + free(entry->arg2); + free(entry->arg3); + free(entry); + } + auxtrace__free_bpf_trigger_list(); + return ret; +} + +int auxtrace__prepare_bpf(struct auxtrace_record *itr, const char *str) +{ + int ret; + + if (!itr || !str) + return 0; + + skel =3D auxtrace_pause_bpf__open(); + if (!skel) { + pr_err("Failed to open func latency skeleton\n"); + return -1; + } + + ret =3D auxtrace__alloc_bpf_trigger_list(str); + if (ret) { + auxtrace_pause_bpf__destroy(skel); + skel =3D NULL; + return ret; + } + + return 0; +} + +static struct bpf_link *auxtrace__attach_bpf_prog(struct trigger_entry *en= try) +{ + struct bpf_link *link =3D NULL; + LIBBPF_OPTS(bpf_uprobe_opts, uprobe_opts); + + if (!strcmp(entry->arg0, "kprobe")) { + if (!strcmp(entry->arg1, "p")) { + link =3D bpf_program__attach_kprobe( + skel->progs.kprobe_event_pause, + false, entry->arg2); + } else if (!strcmp(entry->arg1, "r")) { + link =3D bpf_program__attach_kprobe( + skel->progs.kprobe_event_resume, + false, entry->arg2); + } + } else if (!strcmp(entry->arg0, "kretprobe")) { + if (!strcmp(entry->arg1, "p")) { + link =3D bpf_program__attach_kprobe( + skel->progs.kretprobe_event_pause, + true, entry->arg2); + } else if (!strcmp(entry->arg1, "r")) { + link =3D bpf_program__attach_kprobe( + skel->progs.kretprobe_event_resume, + true, entry->arg2); + } + } else if (!strcmp(entry->arg0, "uprobe")) { + uprobe_opts.func_name =3D entry->arg3; + uprobe_opts.retprobe =3D false; + if (!strcmp(entry->arg1, "p")) { + link =3D bpf_program__attach_uprobe_opts( + skel->progs.uprobe_event_pause, + -1, entry->arg2, 0, &uprobe_opts); + } else if (!strcmp(entry->arg1, "r")) { + link =3D bpf_program__attach_uprobe_opts( + skel->progs.uprobe_event_resume, + -1, entry->arg2, 0, &uprobe_opts); + } + } else if (!strcmp(entry->arg0, "uretprobe")) { + uprobe_opts.func_name =3D entry->arg3; + uprobe_opts.retprobe =3D true; + if (!strcmp(entry->arg1, "p")) { + link =3D bpf_program__attach_uprobe_opts( + skel->progs.uretprobe_event_pause, + -1, entry->arg2, 0, &uprobe_opts); + } else if (!strcmp(entry->arg1, "r")) { + link =3D bpf_program__attach_uprobe_opts( + skel->progs.uretprobe_event_resume, + -1, entry->arg2, 0, &uprobe_opts); + } + + } else if (is_attach_tracepoint(entry->arg0)) { + if (!strcmp(entry->arg1, "p")) { + link =3D bpf_program__attach_tracepoint( + skel->progs.tp_event_pause, + entry->arg2, entry->arg3); + } else if (!strcmp(entry->arg1, "r")) { + link =3D bpf_program__attach_tracepoint( + skel->progs.tp_event_resume, + entry->arg2, entry->arg3); + } + } + + return link; +} + +int auxtrace__set_bpf_filter(struct evlist *evlist, struct record_opts *op= ts) +{ + int fd, err; + int i, ncpus =3D 1, ntasks =3D 1; + struct trigger_entry *trigger_entry; + struct target *target; + + if (!skel) + return 0; + + if (!opts) + return -EINVAL; + + target =3D &opts->target; + + if (target__has_cpu(target)) { + ncpus =3D perf_cpu_map__nr(evlist->core.user_requested_cpus); + bpf_map__set_max_entries(skel->maps.cpu_filter, ncpus); + skel->rodata->has_cpu =3D 1; + } + + if (target__has_task(target) || target__none(target)) { + ntasks =3D perf_thread_map__nr(evlist->core.threads); + bpf_map__set_max_entries(skel->maps.task_filter, ntasks); + skel->rodata->has_task =3D 1; + } + + if (target->per_thread) + skel->rodata->per_thread =3D 1; + + bpf_map__set_max_entries(skel->maps.events, libbpf_num_possible_cpus()); + + err =3D auxtrace_pause_bpf__load(skel); + if (err) { + pr_err("Failed to load func latency skeleton: %d\n", err); + goto out; + } + + if (target__has_cpu(target)) { + u32 cpu; + u8 val =3D 1; + + fd =3D bpf_map__fd(skel->maps.cpu_filter); + + for (i =3D 0; i < ncpus; i++) { + cpu =3D perf_cpu_map__cpu(evlist->core.user_requested_cpus, i).cpu; + bpf_map_update_elem(fd, &cpu, &val, BPF_ANY); + } + } + + if (target__has_task(target) || target__none(target)) { + u32 pid; + u8 val =3D 1; + + fd =3D bpf_map__fd(skel->maps.task_filter); + + for (i =3D 0; i < ntasks; i++) { + pid =3D perf_thread_map__pid(evlist->core.threads, i); + bpf_map_update_elem(fd, &pid, &val, BPF_ANY); + } + } + + trigger_links =3D zalloc(sizeof(*trigger_links) * trigger_entry_num); + if (!trigger_links) + return -ENOMEM; + + i =3D 0; + list_for_each_entry(trigger_entry, &trigger_list, list) { + trigger_links[i] =3D auxtrace__attach_bpf_prog(trigger_entry); + err =3D libbpf_get_error(trigger_links[i]); + if (err) { + pr_err("Failed to attach bpf program to aux pause entry\n"); + pr_err(" arg0=3D%s arg1=3D%s arg2=3D%s arg3=3D%s\n", + trigger_entry->arg0, trigger_entry->arg1, + trigger_entry->arg2, trigger_entry->arg3); + trigger_links[i] =3D NULL; + goto out; + } + i++; + } + + return 0; + +out: + for (i =3D 0; i < trigger_entry_num; i++) { + if (!trigger_links[i]) + continue; + bpf_link__destroy(trigger_links[i]); + } + + return err; +} + +int auxtrace__enable_bpf(void) +{ + if (!skel) + return 0; + + skel->bss->enabled =3D 1; + return 0; +} + +int auxtrace__cleanup_bpf(void) +{ + int i; + + if (!skel) + return 0; + + for (i =3D 0; i < trigger_entry_num; i++) { + if (!trigger_links[i]) + continue; + bpf_link__destroy(trigger_links[i]); + } + + auxtrace__free_bpf_trigger_list(); + auxtrace_pause_bpf__destroy(skel); + return 0; +} + +int auxtrace__update_bpf_map(struct evsel *evsel, int cpu_map_idx, int fd) +{ + int ret; + + if (!skel) + return 0; + + if (!evsel->needs_auxtrace_mmap) + return 0; + + ret =3D bpf_map_update_elem(bpf_map__fd(skel->maps.events), + &cpu_map_idx, &fd, BPF_ANY); + if (ret) { + pr_err("Failed to update BPF map for auxtrace: %s.\n", + strerror(errno)); + if (errno =3D=3D EOPNOTSUPP) + pr_err(" Try to disable inherit mode with option '-i'.\n"); + return ret; + } + + return 0; +} --=20 2.34.1