From nobody Sat Nov 30 05:31:48 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBA771B013F; Thu, 12 Sep 2024 15:10:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726153804; cv=none; b=cyX9qmhInxOuAfXNhW8FpN4iJKDlv9UdlFMUdD/eaEDQqcUVb15O/sDuxJFhuyPMDn285e1s/i4D/wyUxcpyfYK/HSdA6wScnuDfsIZOAyCwdFqLeuOUrLDZOR0qzyD3yxPQou7xzUW/9CXPa095qf6Y2SfuclwE0w1/9gCfmMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726153804; c=relaxed/simple; bh=NwNnbT1qjWgUxt7WAVjsHV2ZFS9s9UwVIBFdP6/hdDs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=GFSHTELPTbziW8OMCY+FwgioXvvM20jdDwSPbt9R8Ljmjqsk8gnNzDCM4xqIcvSFtu7qpg2gdR6jKwkUR8izW+KRaQT/zgzGjYccDznxTwP+imuQzy+2QVaCL+LFm71zG3z4IgGS338IDZifEdv7bkFQ62uyest8i2N9c/ILy8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NIubXr3d; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NIubXr3d" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2C9BCC4CEC3; Thu, 12 Sep 2024 15:10:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1726153803; bh=NwNnbT1qjWgUxt7WAVjsHV2ZFS9s9UwVIBFdP6/hdDs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NIubXr3d8fw88yNDqMqCGxmkiYzsok9jPoGGKTTnWRO6ley82NKdd+2vi6l6aJ9VJ a92OuneZj6GprZ6IX66LOoGs/PKeN6MAifBIQqdYcuTKFCNsegQ7A/f9Nc3dp9hTe7 vvN7BFxehQw1w9k9xCBFD+Z8+QnDYe/ooIKpHB+Yoo0H0uSlBWuNWuz3/ILShrlSRv 10oX6C8PjseYGHzuQ7KxtOvKRHOkzLhVmhLyfgoSyIYpANWP5FfbIY330jo46RL4Sl cWXT+hzLURyo0TY+75+LHdURRbQa+OyFyD6nbUirALAx6FtOmGfdcQSXnmNlHjRcSu 7jBMQyaIm+Cdw== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner , Guo Ren , linux-arch@vger.kernel.org Subject: [PATCH v14 10/19] tracing/fprobe: Enable fprobe events with CONFIG_DYNAMIC_FTRACE_WITH_ARGS Date: Fri, 13 Sep 2024 00:09:58 +0900 Message-Id: <172615379798.133222.11361355578012444592.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <172615368656.133222.2336770908714920670.stgit@devnote2> References: <172615368656.133222.2336770908714920670.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Allow fprobe events to be enabled with CONFIG_DYNAMIC_FTRACE_WITH_ARGS. With this change, fprobe events mostly use ftrace_regs instead of pt_regs. Note that if the arch doesn't enable HAVE_PT_REGS_COMPAT_FTRACE_REGS, fprobe events will not be able to be used from perf. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v9: - Copy store_trace_entry_data() as store_fprobe_entry_data() for fprobe. Chagnes in v3: - Use ftrace_regs_get_return_value(). Changes in v2: - Define ftrace_regs_get_kernel_stack_nth() for !CONFIG_HAVE_REGS_AND_STACK_ACCESS_API. Changes from previous series: Update against the new series. --- include/linux/ftrace.h | 17 ++++++ kernel/trace/Kconfig | 1=20 kernel/trace/trace_fprobe.c | 107 +++++++++++++++++++++++++----------= ---- kernel/trace/trace_probe_tmpl.h | 2 - 4 files changed, 86 insertions(+), 41 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 3edf2427ae73..63fb91088a23 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -255,6 +255,23 @@ static __always_inline bool ftrace_regs_has_args(struc= t ftrace_regs *fregs) frame_pointer(&(fregs)->regs) #endif =20 +#ifdef CONFIG_HAVE_REGS_AND_STACK_ACCESS_API +static __always_inline unsigned long +ftrace_regs_get_kernel_stack_nth(struct ftrace_regs *fregs, unsigned int n= th) +{ + unsigned long *stackp; + + stackp =3D (unsigned long *)ftrace_regs_get_stack_pointer(fregs); + if (((unsigned long)(stackp + nth) & ~(THREAD_SIZE - 1)) =3D=3D + ((unsigned long)stackp & ~(THREAD_SIZE - 1))) + return *(stackp + nth); + + return 0; +} +#else /* !CONFIG_HAVE_REGS_AND_STACK_ACCESS_API */ +#define ftrace_regs_get_kernel_stack_nth(fregs, nth) (0L) +#endif /* CONFIG_HAVE_REGS_AND_STACK_ACCESS_API */ + typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *op, struct ftrace_regs *fregs); =20 diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 2e2b39699542..0fc4c3129c19 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -680,7 +680,6 @@ config FPROBE_EVENTS select TRACING select PROBE_EVENTS select DYNAMIC_EVENTS - depends on DYNAMIC_FTRACE_WITH_REGS default y help This allows user to add tracing events on the function entry and diff --git a/kernel/trace/trace_fprobe.c b/kernel/trace/trace_fprobe.c index 273cdf3cf70c..86cd6a8c806a 100644 --- a/kernel/trace/trace_fprobe.c +++ b/kernel/trace/trace_fprobe.c @@ -133,7 +133,7 @@ static int process_fetch_insn(struct fetch_insn *code, void *rec, void *edata, void *dest, void *base) { - struct pt_regs *regs =3D rec; + struct ftrace_regs *fregs =3D rec; unsigned long val; int ret; =20 @@ -141,17 +141,17 @@ process_fetch_insn(struct fetch_insn *code, void *rec= , void *edata, /* 1st stage: get value from context */ switch (code->op) { case FETCH_OP_STACK: - val =3D regs_get_kernel_stack_nth(regs, code->param); + val =3D ftrace_regs_get_kernel_stack_nth(fregs, code->param); break; case FETCH_OP_STACKP: - val =3D kernel_stack_pointer(regs); + val =3D ftrace_regs_get_stack_pointer(fregs); break; case FETCH_OP_RETVAL: - val =3D regs_return_value(regs); + val =3D ftrace_regs_get_return_value(fregs); break; #ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API case FETCH_OP_ARG: - val =3D regs_get_kernel_argument(regs, code->param); + val =3D ftrace_regs_get_argument(fregs, code->param); break; case FETCH_OP_EDATA: val =3D *(unsigned long *)((unsigned long)edata + code->offset); @@ -174,7 +174,7 @@ NOKPROBE_SYMBOL(process_fetch_insn) /* function entry handler */ static nokprobe_inline void __fentry_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - struct pt_regs *regs, + struct ftrace_regs *fregs, struct trace_event_file *trace_file) { struct fentry_trace_entry_head *entry; @@ -188,41 +188,71 @@ __fentry_trace_func(struct trace_fprobe *tf, unsigned= long entry_ip, if (trace_trigger_soft_disabled(trace_file)) return; =20 - dsize =3D __get_data_size(&tf->tp, regs, NULL); + dsize =3D __get_data_size(&tf->tp, fregs, NULL); =20 entry =3D trace_event_buffer_reserve(&fbuffer, trace_file, sizeof(*entry) + tf->tp.size + dsize); if (!entry) return; =20 - fbuffer.regs =3D regs; + fbuffer.regs =3D ftrace_get_regs(fregs); entry =3D fbuffer.entry =3D ring_buffer_event_data(fbuffer.event); entry->ip =3D entry_ip; - store_trace_args(&entry[1], &tf->tp, regs, NULL, sizeof(*entry), dsize); + store_trace_args(&entry[1], &tf->tp, fregs, NULL, sizeof(*entry), dsize); =20 trace_event_buffer_commit(&fbuffer); } =20 static void fentry_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - struct pt_regs *regs) + struct ftrace_regs *fregs) { struct event_file_link *link; =20 trace_probe_for_each_link_rcu(link, &tf->tp) - __fentry_trace_func(tf, entry_ip, regs, link->file); + __fentry_trace_func(tf, entry_ip, fregs, link->file); } NOKPROBE_SYMBOL(fentry_trace_func); =20 +static nokprobe_inline +void store_fprobe_entry_data(void *edata, struct trace_probe *tp, struct f= trace_regs *fregs) +{ + struct probe_entry_arg *earg =3D tp->entry_arg; + unsigned long val =3D 0; + int i; + + if (!earg) + return; + + for (i =3D 0; i < earg->size; i++) { + struct fetch_insn *code =3D &earg->code[i]; + + switch (code->op) { + case FETCH_OP_ARG: + val =3D ftrace_regs_get_argument(fregs, code->param); + break; + case FETCH_OP_ST_EDATA: + *(unsigned long *)((unsigned long)edata + code->offset) =3D val; + break; + case FETCH_OP_END: + goto end; + default: + break; + } + } +end: + return; +} + /* function exit handler */ static int trace_fprobe_entry_handler(struct fprobe *fp, unsigned long ent= ry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data) { struct trace_fprobe *tf =3D container_of(fp, struct trace_fprobe, fp); =20 if (tf->tp.entry_arg) - store_trace_entry_data(entry_data, &tf->tp, regs); + store_fprobe_entry_data(entry_data, &tf->tp, fregs); =20 return 0; } @@ -230,7 +260,7 @@ NOKPROBE_SYMBOL(trace_fprobe_entry_handler) =20 static nokprobe_inline void __fexit_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data, struct trace_event_file *trace_file) { struct fexit_trace_entry_head *entry; @@ -244,60 +274,63 @@ __fexit_trace_func(struct trace_fprobe *tf, unsigned = long entry_ip, if (trace_trigger_soft_disabled(trace_file)) return; =20 - dsize =3D __get_data_size(&tf->tp, regs, entry_data); + dsize =3D __get_data_size(&tf->tp, fregs, entry_data); =20 entry =3D trace_event_buffer_reserve(&fbuffer, trace_file, sizeof(*entry) + tf->tp.size + dsize); if (!entry) return; =20 - fbuffer.regs =3D regs; + fbuffer.regs =3D ftrace_get_regs(fregs); entry =3D fbuffer.entry =3D ring_buffer_event_data(fbuffer.event); entry->func =3D entry_ip; entry->ret_ip =3D ret_ip; - store_trace_args(&entry[1], &tf->tp, regs, entry_data, sizeof(*entry), ds= ize); + store_trace_args(&entry[1], &tf->tp, fregs, entry_data, sizeof(*entry), d= size); =20 trace_event_buffer_commit(&fbuffer); } =20 static void fexit_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, void *entry_data) + unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data) { struct event_file_link *link; =20 trace_probe_for_each_link_rcu(link, &tf->tp) - __fexit_trace_func(tf, entry_ip, ret_ip, regs, entry_data, link->file); + __fexit_trace_func(tf, entry_ip, ret_ip, fregs, entry_data, link->file); } NOKPROBE_SYMBOL(fexit_trace_func); =20 #ifdef CONFIG_PERF_EVENTS =20 static int fentry_perf_func(struct trace_fprobe *tf, unsigned long entry_i= p, - struct pt_regs *regs) + struct ftrace_regs *fregs) { struct trace_event_call *call =3D trace_probe_event_call(&tf->tp); struct fentry_trace_entry_head *entry; struct hlist_head *head; int size, __size, dsize; + struct pt_regs *regs; int rctx; =20 head =3D this_cpu_ptr(call->perf_events); if (hlist_empty(head)) return 0; =20 - dsize =3D __get_data_size(&tf->tp, regs, NULL); + dsize =3D __get_data_size(&tf->tp, fregs, NULL); __size =3D sizeof(*entry) + tf->tp.size + dsize; size =3D ALIGN(__size + sizeof(u32), sizeof(u64)); size -=3D sizeof(u32); =20 - entry =3D perf_trace_buf_alloc(size, NULL, &rctx); + entry =3D perf_trace_buf_alloc(size, ®s, &rctx); if (!entry) return 0; =20 + regs =3D ftrace_fill_perf_regs(fregs, regs); + entry->ip =3D entry_ip; memset(&entry[1], 0, dsize); - store_trace_args(&entry[1], &tf->tp, regs, NULL, sizeof(*entry), dsize); + store_trace_args(&entry[1], &tf->tp, fregs, NULL, sizeof(*entry), dsize); perf_trace_buf_submit(entry, size, rctx, call->event.type, 1, regs, head, NULL); return 0; @@ -306,31 +339,34 @@ NOKPROBE_SYMBOL(fentry_perf_func); =20 static void fexit_perf_func(struct trace_fprobe *tf, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data) { struct trace_event_call *call =3D trace_probe_event_call(&tf->tp); struct fexit_trace_entry_head *entry; struct hlist_head *head; int size, __size, dsize; + struct pt_regs *regs; int rctx; =20 head =3D this_cpu_ptr(call->perf_events); if (hlist_empty(head)) return; =20 - dsize =3D __get_data_size(&tf->tp, regs, entry_data); + dsize =3D __get_data_size(&tf->tp, fregs, entry_data); __size =3D sizeof(*entry) + tf->tp.size + dsize; size =3D ALIGN(__size + sizeof(u32), sizeof(u64)); size -=3D sizeof(u32); =20 - entry =3D perf_trace_buf_alloc(size, NULL, &rctx); + entry =3D perf_trace_buf_alloc(size, ®s, &rctx); if (!entry) return; =20 + regs =3D ftrace_fill_perf_regs(fregs, regs); + entry->func =3D entry_ip; entry->ret_ip =3D ret_ip; - store_trace_args(&entry[1], &tf->tp, regs, entry_data, sizeof(*entry), ds= ize); + store_trace_args(&entry[1], &tf->tp, fregs, entry_data, sizeof(*entry), d= size); perf_trace_buf_submit(entry, size, rctx, call->event.type, 1, regs, head, NULL); } @@ -342,17 +378,14 @@ static int fentry_dispatcher(struct fprobe *fp, unsig= ned long entry_ip, void *entry_data) { struct trace_fprobe *tf =3D container_of(fp, struct trace_fprobe, fp); - struct pt_regs *regs =3D ftrace_get_regs(fregs); int ret =3D 0; =20 - if (!regs) - return 0; - if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) - fentry_trace_func(tf, entry_ip, regs); + fentry_trace_func(tf, entry_ip, fregs); + #ifdef CONFIG_PERF_EVENTS if (trace_probe_test_flag(&tf->tp, TP_FLAG_PROFILE)) - ret =3D fentry_perf_func(tf, entry_ip, regs); + ret =3D fentry_perf_func(tf, entry_ip, fregs); #endif return ret; } @@ -363,16 +396,12 @@ static void fexit_dispatcher(struct fprobe *fp, unsig= ned long entry_ip, void *entry_data) { struct trace_fprobe *tf =3D container_of(fp, struct trace_fprobe, fp); - struct pt_regs *regs =3D ftrace_get_regs(fregs); - - if (!regs) - return; =20 if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) - fexit_trace_func(tf, entry_ip, ret_ip, regs, entry_data); + fexit_trace_func(tf, entry_ip, ret_ip, fregs, entry_data); #ifdef CONFIG_PERF_EVENTS if (trace_probe_test_flag(&tf->tp, TP_FLAG_PROFILE)) - fexit_perf_func(tf, entry_ip, ret_ip, regs, entry_data); + fexit_perf_func(tf, entry_ip, ret_ip, fregs, entry_data); #endif } NOKPROBE_SYMBOL(fexit_dispatcher); diff --git a/kernel/trace/trace_probe_tmpl.h b/kernel/trace/trace_probe_tmp= l.h index 2caf0d2afb32..f39b37fcdb3b 100644 --- a/kernel/trace/trace_probe_tmpl.h +++ b/kernel/trace/trace_probe_tmpl.h @@ -232,7 +232,7 @@ process_fetch_insn_bottom(struct fetch_insn *code, unsi= gned long val, =20 /* Sum up total data length for dynamic arrays (strings) */ static nokprobe_inline int -__get_data_size(struct trace_probe *tp, struct pt_regs *regs, void *edata) +__get_data_size(struct trace_probe *tp, void *regs, void *edata) { struct probe_arg *arg; int i, len, ret =3D 0;