From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EDF93B19BA; Tue, 22 Sep 2026 04:25:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051103; cv=none; b=t/ZdNzJIjEDo/Ll7x1Wfx3ae9BNfOcYAYmtduSYWx6T7f4cQEVhZWszrA2NFbkryLyGwtmbtWG8FIyy8pIqnJWGVtFwRu/6j0dUBDH7K/fExPgUqgeGE5MZ9lX2O4CmbXVQ8C+0ELNKBYS0fqsBAfVu0l+x0iXnC3FTnnB2C3dI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051103; c=relaxed/simple; bh=++emXcP9SPk1N6KlszECK+hTyKvu/kLTGVZdg7HV0Vo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=JHJH2Yxnr1PUpx+G/zTOUI18Us9FDFNW0IXEJJt+lmh1imqe2zpqzSm9WPOT7qKzmOV0XflXzq+/qcLDJLq0cAy5l5ytWyUmVoyIXfAr6B8E+wINUm9P6Dta2T8sFDMg3Y/Nn9aTJ80V6oB6D4rRi22OnYRAjbX1n3FZGi0qX/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Q9vsazbS; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q9vsazbS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 004BB1F000FF; Tue, 22 Sep 2026 04:24:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051102; bh=ECQ2YrO3c6fTy3LNvcZ/qkWRkP34JaQc5nHEXsbIZYs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Q9vsazbSFWTo5U6CPnSMDUrbuQR2IpclOPrYwrj6DA1iF1EC4SEKlyc6aqW07l9pt sxU2MKo2Qcm30AuaRSCdmGle9dG4uCUBqmr9nw8T+lFv3VKgpVnbHFt9ZlfKJYP2wJ sVGa1NN/40op0Gcsk6z83+Yq8WLZCqwLcY87gW+vDxBcmMb8Kd8iZ4a3glIA4QoeJv g5llcML+kVX/KEEgZI8kdrdfTX3CkH7vUv9VM5x9ifsK1P2QuLqcPu1AqtTgcWRdUN ELykU/8lLuXyWRRzd0vRoQ0Y7XcOCmzuJa9r/HPWF8lnn+7v6dvjeUVfdvl/Axg6OG NZcsLl1NFl9bQ== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 01/13] x86/mce: Fix hardware debug register corruption on task migration Date: Tue, 22 Sep 2026 13:24:55 +0900 Message-ID: <179005109564.388919.3937970081044095776.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) In exc_machine_check_user(), local_db_save() and local_db_restore() are invoked in the outer entry stubs (DEFINE_IDTENTRY_MCE_USER, DEFINE_FREDENTRY_MCE, and DEFINE_IDTENTRY_RAW), surrounding exc_machine_check_user(). However, exc_machine_check_user() calls irqentry_exit_to_user_mode(), which handles pending thread work and may schedule() if TIF_NEED_RESCHED is set. If the task migrates to another CPU during schedule(), local_db_restore() runs on the new CPU with the dr7 state saved from the old CPU. This corrupts the new CPU's DR7 hardware debug register and leaves the old CPU's DR7 disabled. In short, local_db_save() and local_db_restore() pair must be run on the same CPU. To fix this, move local_db_save() and local_db_restore() inside exc_machine_check_user() and exc_machine_check_kernel(). In exc_machine_check_user(), DR7 is saved and restored strictly around do_machine_check() to avoid schedule() during migration. In exc_machine_check_kernel(), local_db_save() is called at the entry point to prevent early memory accesses from triggering nested #DB exceptions, and restored on all exits. Fixes: cd840e424f27 ("x86/entry, mce: Disallow #DB during #MC") Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) Acked-by: Peter Zijlstra (Intel) --- Changes in v15: - Move local_db_save() to the entry of exc_machine_check_kernel() and restore it on early return from mce_check_crashing_cpu() to prevent nested #DB on watched variables. Changes in v14: - Newly added. --- arch/x86/kernel/cpu/mce/core.c | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index ab469605fc89..39f238952e14 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -2108,6 +2108,9 @@ bool filter_mce(struct mce *m) static __always_inline void exc_machine_check_kernel(struct pt_regs *regs) { irqentry_state_t irq_state; + unsigned long dr7; + + dr7 =3D local_db_save(); =20 WARN_ON_ONCE(user_mode(regs)); =20 @@ -2116,20 +2119,26 @@ static __always_inline void exc_machine_check_kerne= l(struct pt_regs *regs) * mce_check_crashing_cpu() for details. */ if (mca_cfg.initialized && mce_check_crashing_cpu()) - return; + goto out; =20 irq_state =3D irqentry_nmi_enter(regs); =20 do_machine_check(regs); =20 irqentry_nmi_exit(regs, irq_state); +out: + local_db_restore(dr7); } =20 static __always_inline void exc_machine_check_user(struct pt_regs *regs) { + unsigned long dr7; + irqentry_enter_from_user_mode(regs); =20 + dr7 =3D local_db_save(); do_machine_check(regs); + local_db_restore(dr7); =20 irqentry_exit_to_user_mode(regs); } @@ -2138,21 +2147,13 @@ static __always_inline void exc_machine_check_user(= struct pt_regs *regs) /* MCE hit kernel mode */ DEFINE_IDTENTRY_MCE(exc_machine_check) { - unsigned long dr7; - - dr7 =3D local_db_save(); exc_machine_check_kernel(regs); - local_db_restore(dr7); } =20 /* The user mode variant. */ DEFINE_IDTENTRY_MCE_USER(exc_machine_check) { - unsigned long dr7; - - dr7 =3D local_db_save(); exc_machine_check_user(regs); - local_db_restore(dr7); } =20 #ifdef CONFIG_X86_FRED @@ -2169,28 +2170,20 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check) */ DEFINE_FREDENTRY_MCE(exc_machine_check) { - unsigned long dr7; - - dr7 =3D local_db_save(); if (user_mode(regs)) exc_machine_check_user(regs); else exc_machine_check_kernel(regs); - local_db_restore(dr7); } #endif #else /* 32bit unified entry point */ DEFINE_IDTENTRY_RAW(exc_machine_check) { - unsigned long dr7; - - dr7 =3D local_db_save(); if (user_mode(regs)) exc_machine_check_user(regs); else exc_machine_check_kernel(regs); - local_db_restore(dr7); } #endif From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E27A3B19BA; Tue, 22 Sep 2026 04:25:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051115; cv=none; b=TT6bR866dEPx2SoxhvLYuQY10d30TTplSQe/KyX6l+HUNjtnGbwB8em/34/ipocMDMLq3si+j4Lz7A2L9Ei20K7DAgPAUEJFii5nyINVuFxzwnHKeaxlkAI81FUolMHCooZtqz/qOfbw46L7LHSrRHJp6JdXd9sxaNx2X1GEHE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051115; c=relaxed/simple; bh=ea9qxXx9kg89UWY0dCkt4q7OHjwdGM8gcXLohsiMyBI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MDos36bG5QlQ/i+8xoPyGCp28Ccujzu0yCqs1c28MyelxmoO65oEbe6KjpeP1Y7oEVmOR60D8x5188c0iJnujv8+0oHoXz1oQT2V9RWTx6AZGa6nCjw6XeXovXHLU/0szkP3pVKrbygYqCRhaeyLs5RAsRsjMfCdHAAtl4I2sag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TkV2NN5i; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TkV2NN5i" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE4251F000FF; Tue, 22 Sep 2026 04:25:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051114; bh=dsQx+QviCP8DSIGNuzmfZuXg5PLuR8qG1Ne4KObCFng=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=TkV2NN5ivtgtiw5IZBPl4swOC1Jc4gMRkcQZEuBmjd5PK+DdEnRdfudQi8DPMzj52 YfE2eScGGtJMD/z580IaumAmyBR0ElA1BxkALJ8k95F1GW+Vw5V0mZblJWdPQzcddA 3iUVkGy7klIoLWslBgUC228Vuhny6P/Tj0E+7ULgOb2/yHJQYOWpg8j33cArW12qIb FYnrHml9SwXjWGo7+eS8hnsy3WUd3fvi/Ua/KRaIsOZh0u3RKjehqkY7HKCIdw1IfN uudUKELd46HHwn4VTthZiJ3tNVvDo6SEOcu/ifEQDwcxBnSA0yXwhwT7RWfjN6oS+1 8K4IRudzRPTUA== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 02/13] perf/x86, KVM: Prevent host debug register leak into guest OS on NMI Date: Tue, 22 Sep 2026 13:25:07 +0900 Message-ID: <179005110742.388919.1509641807739909065.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) When KVM enters a guest OS, host hardware breakpoints are disabled before running the guest. However, an NMI can occur while executing in guest mode or during guest transition, where arch_install_hw_breakpoint() can be invoked from NMI context. If arch_install_hw_breakpoint() is executed while the CPU is in guest mode or during the VM entry transition window, hardware debug registers can be modified with host breakpoint settings, leaking host breakpoints into the guest OS or clobbering the guest's debug registers. To prevent this without introducing ad-hoc per-CPU flags, introduce an .in_guest callback to struct perf_guest_info_callbacks and an inline helper perf_guest_in_guest(). In KVM, implement .in_guest by checking whether the current vCPU is in IN_GUEST_MODE or EXITING_GUEST_MODE. In arch_install_hw_breakpoint(), check perf_guest_in_guest() and return -EBUSY if the CPU is running or transitioning to/from a guest. Fixes: f85d40160691 ("KVM: X86: Disable hardware breakpoints unconditionall= y before kvm_x86->run()") Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) --- Changes in v17: - Drop changes to local_db_save() and local_db_restore(). - Drop cpu_dr_in_guest per-CPU flag. - Introduce .in_guest callback in perf_guest_info_callbacks and perf_guest_in_guest() helper to query vcpu->mode. (avoid breaking perf_guest_state() users) - In arch_install_hw_breakpoint(), check perf_guest_in_guest() Changes in v16: - Newly added. --- arch/x86/kernel/hw_breakpoint.c | 3 +++ include/linux/perf_event.h | 9 ++++++++- kernel/events/core.c | 5 +++++ virt/kvm/kvm_main.c | 12 ++++++++++++ 4 files changed, 28 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoin= t.c index f846c15f21ca..0473a5c95856 100644 --- a/arch/x86/kernel/hw_breakpoint.c +++ b/arch/x86/kernel/hw_breakpoint.c @@ -102,6 +102,9 @@ int arch_install_hw_breakpoint(struct perf_event *bp) =20 lockdep_assert_irqs_disabled(); =20 + if (perf_guest_in_guest()) + return -EBUSY; + for (i =3D 0; i < HBP_NUM; i++) { struct perf_event **slot =3D this_cpu_ptr(&bp_per_reg[i]); =20 diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 5842552294c1..194e69270460 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1677,8 +1677,8 @@ struct perf_guest_info_callbacks { unsigned int (*state)(void); unsigned long (*get_ip)(void); unsigned int (*handle_intel_pt_intr)(void); - void (*handle_mediated_pmi)(void); + bool (*in_guest)(void); }; =20 #ifdef CONFIG_GUEST_PERF_EVENTS @@ -1689,6 +1689,7 @@ DECLARE_STATIC_CALL(__perf_guest_state, *perf_guest_c= bs->state); DECLARE_STATIC_CALL(__perf_guest_get_ip, *perf_guest_cbs->get_ip); DECLARE_STATIC_CALL(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->ha= ndle_intel_pt_intr); DECLARE_STATIC_CALL(__perf_guest_handle_mediated_pmi, *perf_guest_cbs->han= dle_mediated_pmi); +DECLARE_STATIC_CALL(__perf_guest_in_guest, *perf_guest_cbs->in_guest); =20 static inline unsigned int perf_guest_state(void) { @@ -1710,6 +1711,11 @@ static inline void perf_guest_handle_mediated_pmi(vo= id) static_call(__perf_guest_handle_mediated_pmi)(); } =20 +static inline bool perf_guest_in_guest(void) +{ + return static_call(__perf_guest_in_guest)(); +} + extern void perf_register_guest_info_callbacks(struct perf_guest_info_call= backs *cbs); extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_ca= llbacks *cbs); =20 @@ -1718,6 +1724,7 @@ extern void perf_unregister_guest_info_callbacks(stru= ct perf_guest_info_callback static inline unsigned int perf_guest_state(void) { return 0; } static inline unsigned long perf_guest_get_ip(void) { return 0; } static inline unsigned int perf_guest_handle_intel_pt_intr(void) { return = 0; } +static inline bool perf_guest_in_guest(void) { return false; } =20 #endif /* !CONFIG_GUEST_PERF_EVENTS */ =20 diff --git a/kernel/events/core.c b/kernel/events/core.c index a6c8e38a3110..c5dead7c5f71 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -7749,6 +7749,7 @@ DEFINE_STATIC_CALL_RET0(__perf_guest_state, *perf_gue= st_cbs->state); DEFINE_STATIC_CALL_RET0(__perf_guest_get_ip, *perf_guest_cbs->get_ip); DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs= ->handle_intel_pt_intr); DEFINE_STATIC_CALL_RET0(__perf_guest_handle_mediated_pmi, *perf_guest_cbs-= >handle_mediated_pmi); +DEFINE_STATIC_CALL_RET0(__perf_guest_in_guest, *perf_guest_cbs->in_guest); =20 void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *= cbs) { @@ -7767,6 +7768,9 @@ void perf_register_guest_info_callbacks(struct perf_g= uest_info_callbacks *cbs) if (cbs->handle_mediated_pmi) static_call_update(__perf_guest_handle_mediated_pmi, cbs->handle_mediated_pmi); + + if (cbs->in_guest) + static_call_update(__perf_guest_in_guest, cbs->in_guest); } EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks); =20 @@ -7780,6 +7784,7 @@ void perf_unregister_guest_info_callbacks(struct perf= _guest_info_callbacks *cbs) static_call_update(__perf_guest_get_ip, (void *)&__static_call_return0); static_call_update(__perf_guest_handle_intel_pt_intr, (void *)&__static_c= all_return0); static_call_update(__perf_guest_handle_mediated_pmi, (void *)&__static_ca= ll_return0); + static_call_update(__perf_guest_in_guest, (void *)&__static_call_return0); synchronize_rcu(); } EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 65eb26a0520d..7779d500fe77 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -6476,11 +6476,23 @@ static unsigned long kvm_guest_get_ip(void) return kvm_arch_vcpu_get_ip(vcpu); } =20 +static bool kvm_guest_in_guest(void) +{ + struct kvm_vcpu *vcpu =3D kvm_get_running_vcpu(); + + if (!vcpu) + return false; + + return READ_ONCE(vcpu->mode) =3D=3D IN_GUEST_MODE || + READ_ONCE(vcpu->mode) =3D=3D EXITING_GUEST_MODE; +} + static struct perf_guest_info_callbacks kvm_guest_cbs =3D { .state =3D kvm_guest_state, .get_ip =3D kvm_guest_get_ip, .handle_intel_pt_intr =3D NULL, .handle_mediated_pmi =3D NULL, + .in_guest =3D kvm_guest_in_guest, }; =20 void __kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void), From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F6253B840F; Tue, 22 Sep 2026 04:25:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051127; cv=none; b=LGZDaa/timMO7TeAz8MHoJjyCY7bJVRHxgP62ZigISZ7/UIanHb588JSKsA8gA/RkxMpjqmaAWl1gb+uWJKSRRNwBYu5JdVw+rc0YuVDN2W94pas5mJw0Szdq8iWEqGioarRnunvd/GPSvpFYXpwWVDlfkFeOnUKVPsdrpTBrHI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051127; c=relaxed/simple; bh=9JzVLEdLEEgWBEjCDGr5PSYNOat8277b7qjrVAt6sOs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bArRPxeH7CdIJrvm/34kg3S5N4vQtXlq+MIy/Z+L4FGAE0CJD45kl0TJeJQzvp45clX5QIQMLVWCBYQ+uc4BqYdmg2Rpeu2XsX7k47EapUZzpLfQRein5CAF9lB/qwr6IK8Oej6pPG4G1WOPJ48BxnsFKwtXHN+IxIkp59O/Nqo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XAnWuUKb; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XAnWuUKb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C94281F000FF; Tue, 22 Sep 2026 04:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051125; bh=46fcdwTGcPcabI+Q1n5enE0f4J6WoNtorEV47gXLU54=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=XAnWuUKbU+iEtDu5kFLbii6kyT+iQcdII9KuIMfMwDh2ZwakHm3I+MUtaklmJWLPE Dd92zN8vt10KWvJ2aRulGp+j0Igpr6E//cMp+WuTglhzV48lEuKlXlsS5LZ/Vq77Yt pXtBTGRq2kGd8gotwlgIO8jcV3OdTS1srmASBt5Ib6hve57O1kvu1FvZYoykFoq9wh 3XhpC87AKmUx3BQKEJY/VCFbvcqnKaPLt2zUMpxbtwvA9m529kV714gG7Gnrw9hVnA +132vGF+sxumYNZ0XiJdkqsZH1RVo52AyzbISvDdJIMD6+zeyP+6Rri/0wIm+loJDB 4qVDYyIpLnVQA== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 03/13] x86/hw_breakpoints: Make DR7 updates NMI safe Date: Tue, 22 Sep 2026 13:25:19 +0900 Message-ID: <179005111951.388919.3131603399440492416.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Jinchao Wang Hardware breakpoint installation and removal run with IRQs disabled, but an NMI can still enter the same code through KGDB. The interrupted operation and the NMI can consequently claim the same slot or overwrite each other's DR7 state. Claim and release per-CPU slots with cmpxchg. Update cpu_dr7 with single-instruction per-CPU operations, and preserve hardware-first disable and hardware-last enable ordering. Add a per-CPU sequence number so interrupted DR7 writers and restore paths detect an NMI update and retry from the latest shadow state. Link: https://lore.kernel.org/all/4ee0a2efc9e8387af83286b8495b7d490247e165.= 1785067572.git.wangjinchao600@gmail.com/ Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Jinchao Wang Signed-off-by: Masami Hiramatsu (Google) --- Changes in v17: - Drop cpu_dr_in_guest checks in local_db_save(), local_db_restore(), and within_cpu_entry(). Changes in v16: - In local_db_save(), remove the hypervisor check so all paths go through the sequence retry loop, preventing an intervening NMI from bypassing the sequence check on return. - In local_db_save(), check cpu_dr7_seq before clearing hardware DR7 and load from shadow cpu_dr7 on sequence retry to prevent destructively clobbering hardware DR7 state installed by an NMI. - In local_db_restore(), restore from shadow cpu_dr7 whenever cpu_dr7_seq changed instead of evaluating dr7 ? cpu_dr7 : 0, ensuring breakpoints installed by an NMI are preserved when dr7 was initially 0. Changes in v15: - Always preserve DR7_FIXED_1 in arch_install_hw_breakpoint(), arch_uninstall_hw_breakpoint(), and hw_breakpoint_restore() to comply with the x86 architectural specification. - Wrap local_db_save() in a cpu_dr7_seq loop to detect NMI breakpoint updates and ensure DR7 is disabled before entry. - In local_db_restore(), if dr7 is 0 and an NMI modified breakpoints, restore DR7_FIXED_1 to undo the NMI's overwrite and preserve KVM guest isolation. - Always write "val | DR7_FIXED_1" to hardware DR7 in local_db_restore() to ensure architectural compliance. - Avoid modifying the saved dr7 argument in-place across retry loops. Changes in v14: - In local_db_restore(), return immediately if saved dr7 is 0 to avoid overwriting intentionally cleared hardware DR7 state (e.g. KVM guest entry) with cpu_dr7 when an NMI increments cpu_dr7_seq. Changes in v13: - Add compiler barriers (barrier()) before checking cpu_dr7_seq to prevent the compiler from hoisting the sequence check before debug register updates or shadow state modifications. Changes in v12: - Use stack variables dr7 and dr7_seq directly in exc_nmi() and remove unnecessary per-CPU nmi_dr7 and nmi_dr7_seq variables. --- arch/x86/include/asm/debugreg.h | 68 +++++++++++++++++++-------- arch/x86/kernel/cpu/mce/core.c | 10 ++-- arch/x86/kernel/hw_breakpoint.c | 100 ++++++++++++++++++++---------------= ---- arch/x86/kernel/nmi.c | 7 ++- arch/x86/kernel/traps.c | 10 +++- 5 files changed, 116 insertions(+), 79 deletions(-) diff --git a/arch/x86/include/asm/debugreg.h b/arch/x86/include/asm/debugre= g.h index 854d82b88ff4..515d2d313d0c 100644 --- a/arch/x86/include/asm/debugreg.h +++ b/arch/x86/include/asm/debugreg.h @@ -18,6 +18,7 @@ #define DR7_FIXED_1 0x00000400 =20 DECLARE_PER_CPU(unsigned long, cpu_dr7); +DECLARE_PER_CPU(unsigned int, cpu_dr7_seq); =20 #ifndef CONFIG_PARAVIRT_XXL /* @@ -125,40 +126,69 @@ static __always_inline bool hw_breakpoint_active(void) =20 extern void hw_breakpoint_restore(void); =20 -static __always_inline unsigned long local_db_save(void) +static __always_inline void local_db_save(unsigned long *dr7, + unsigned int *dr7_seq) { - unsigned long dr7; + bool retrying =3D false; =20 - if (cpu_feature_enabled(X86_FEATURE_HYPERVISOR) && !hw_breakpoint_active(= )) - return 0; + do { + /* Inner loop: get DR7 with checking cpu_dr7_seq. */ + *dr7_seq =3D this_cpu_read(cpu_dr7_seq); =20 - get_debugreg(dr7, 7); + if (unlikely(retrying)) { + *dr7 =3D this_cpu_read(cpu_dr7); + } else { + get_debugreg(*dr7, 7); =20 - /* Architecturally set bit */ - dr7 &=3D ~DR7_FIXED_1; - if (dr7) - set_debugreg(DR7_FIXED_1, 7); + /* Architecturally set bit */ + *dr7 &=3D ~DR7_FIXED_1; + } =20 - /* - * Ensure the compiler doesn't lower the above statements into - * the critical section; disabling breakpoints late would not - * be good. - */ - barrier(); + barrier(); + if (unlikely(*dr7_seq !=3D this_cpu_read(cpu_dr7_seq))) { + retrying =3D true; + continue; + } + + /* Outer loop: clear DR7 and retry if sequence number is updated. */ + if (*dr7) + set_debugreg(DR7_FIXED_1, 7); =20 - return dr7; + /* + * Ensure the compiler doesn't lower the above statements into + * the critical section; disabling breakpoints late would not + * be good. + */ + barrier(); + retrying =3D true; + } while (unlikely(*dr7_seq !=3D this_cpu_read(cpu_dr7_seq))); } =20 -static __always_inline void local_db_restore(unsigned long dr7) +static __always_inline void local_db_restore(unsigned long dr7, + unsigned int dr7_seq) { + unsigned long val; + unsigned int seq; + /* * Ensure the compiler doesn't raise this statement into * the critical section; enabling breakpoints early would * not be good. */ barrier(); - if (dr7) - set_debugreg(dr7, 7); + do { + seq =3D this_cpu_read(cpu_dr7_seq); + if (seq =3D=3D dr7_seq) { + if (!dr7) + return; + val =3D dr7; + } else { + val =3D this_cpu_read(cpu_dr7); + } + + set_debugreg(val | DR7_FIXED_1, 7); + barrier(); + } while (unlikely(seq !=3D this_cpu_read(cpu_dr7_seq))); } =20 #ifdef CONFIG_CPU_SUP_AMD diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 39f238952e14..b3669d6f88d4 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -2109,8 +2109,9 @@ static __always_inline void exc_machine_check_kernel(= struct pt_regs *regs) { irqentry_state_t irq_state; unsigned long dr7; + unsigned int dr7_seq; =20 - dr7 =3D local_db_save(); + local_db_save(&dr7, &dr7_seq); =20 WARN_ON_ONCE(user_mode(regs)); =20 @@ -2127,18 +2128,19 @@ static __always_inline void exc_machine_check_kerne= l(struct pt_regs *regs) =20 irqentry_nmi_exit(regs, irq_state); out: - local_db_restore(dr7); + local_db_restore(dr7, dr7_seq); } =20 static __always_inline void exc_machine_check_user(struct pt_regs *regs) { unsigned long dr7; + unsigned int dr7_seq; =20 irqentry_enter_from_user_mode(regs); =20 - dr7 =3D local_db_save(); + local_db_save(&dr7, &dr7_seq); do_machine_check(regs); - local_db_restore(dr7); + local_db_restore(dr7, dr7_seq); =20 irqentry_exit_to_user_mode(regs); } diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoin= t.c index 0473a5c95856..901323ae7d6a 100644 --- a/arch/x86/kernel/hw_breakpoint.c +++ b/arch/x86/kernel/hw_breakpoint.c @@ -40,6 +40,8 @@ DEFINE_PER_CPU(unsigned long, cpu_dr7); EXPORT_PER_CPU_SYMBOL(cpu_dr7); =20 +/* Sequence number of the per-CPU DR7 state. */ +DEFINE_PER_CPU(unsigned int, cpu_dr7_seq); /* Per cpu debug address registers values */ static DEFINE_PER_CPU(unsigned long, cpu_debugreg[HBP_NUM]); =20 @@ -97,7 +99,7 @@ int decode_dr7(unsigned long dr7, int bpnum, unsigned *le= n, unsigned *type) int arch_install_hw_breakpoint(struct perf_event *bp) { struct arch_hw_breakpoint *info =3D counter_arch_bp(bp); - unsigned long *dr7; + unsigned int seq; int i; =20 lockdep_assert_irqs_disabled(); @@ -106,32 +108,25 @@ int arch_install_hw_breakpoint(struct perf_event *bp) return -EBUSY; =20 for (i =3D 0; i < HBP_NUM; i++) { - struct perf_event **slot =3D this_cpu_ptr(&bp_per_reg[i]); - - if (!*slot) { - *slot =3D bp; + if (!this_cpu_cmpxchg(bp_per_reg[i], NULL, bp)) break; - } } =20 if (WARN_ONCE(i =3D=3D HBP_NUM, "Can't find any breakpoint slot")) return -EBUSY; =20 - set_debugreg(info->address, i); - __this_cpu_write(cpu_debugreg[i], info->address); - - dr7 =3D this_cpu_ptr(&cpu_dr7); - *dr7 |=3D encode_dr7(i, info->len, info->type); - - /* - * Ensure we first write cpu_dr7 before we set the DR7 register. - * This ensures an NMI never see cpu_dr7 0 when DR7 is not. - */ - barrier(); - - set_debugreg(*dr7, 7); - if (info->mask) - amd_set_dr_addr_mask(info->mask, i); + do { + seq =3D this_cpu_inc_return(cpu_dr7_seq); + this_cpu_write(cpu_debugreg[i], info->address); + barrier(); + set_debugreg(info->address, i); + if (info->mask) + amd_set_dr_addr_mask(info->mask, i); + this_cpu_or(cpu_dr7, encode_dr7(i, info->len, info->type)); + barrier(); + set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7); + barrier(); + } while (seq !=3D this_cpu_read(cpu_dr7_seq)); =20 return 0; } @@ -149,36 +144,34 @@ void arch_uninstall_hw_breakpoint(struct perf_event *= bp) { struct arch_hw_breakpoint *info =3D counter_arch_bp(bp); unsigned long dr7; + unsigned int seq; int i; =20 lockdep_assert_irqs_disabled(); =20 for (i =3D 0; i < HBP_NUM; i++) { - struct perf_event **slot =3D this_cpu_ptr(&bp_per_reg[i]); - - if (*slot =3D=3D bp) { - *slot =3D NULL; + if (this_cpu_read(bp_per_reg[i]) =3D=3D bp) break; - } } =20 if (WARN_ONCE(i =3D=3D HBP_NUM, "Can't find any breakpoint slot")) return; =20 - dr7 =3D this_cpu_read(cpu_dr7); - dr7 &=3D ~__encode_dr7(i, info->len, info->type); - - set_debugreg(dr7, 7); - if (info->mask) - amd_set_dr_addr_mask(0, i); - - /* - * Ensure the write to cpu_dr7 is after we've set the DR7 register. - * This ensures an NMI never see cpu_dr7 0 when DR7 is not. - */ - barrier(); - - this_cpu_write(cpu_dr7, dr7); + do { + seq =3D this_cpu_inc_return(cpu_dr7_seq); + dr7 =3D this_cpu_read(cpu_dr7); + dr7 &=3D ~__encode_dr7(i, info->len, info->type); + set_debugreg(dr7 | DR7_FIXED_1, 7); + if (info->mask) + amd_set_dr_addr_mask(0, i); + barrier(); + this_cpu_and(cpu_dr7, + ~__encode_dr7(i, info->len, info->type)); + barrier(); + } while (seq !=3D this_cpu_read(cpu_dr7_seq)); + + WARN_ONCE(this_cpu_cmpxchg(bp_per_reg[i], bp, NULL) !=3D bp, + "Can't release breakpoint slot"); } =20 static int arch_bp_generic_len(int x86_len) @@ -312,13 +305,14 @@ static inline bool within_cpu_entry(unsigned long add= r, unsigned long end) sizeof(struct tlb_state))) return true; =20 - /* - * When in guest (X86_FEATURE_HYPERVISOR), local_db_save() - * will read per-cpu cpu_dr7 before clear dr7 register. - */ + /* local_db_save() reads this state before clearing DR7. */ if (within_area(addr, end, (unsigned long)&per_cpu(cpu_dr7, cpu), sizeof(cpu_dr7))) return true; + if (within_area(addr, end, + (unsigned long)&per_cpu(cpu_dr7_seq, cpu), + sizeof(cpu_dr7_seq))) + return true; } =20 return false; @@ -486,12 +480,18 @@ void flush_ptrace_hw_breakpoint(struct task_struct *t= sk) =20 void hw_breakpoint_restore(void) { - set_debugreg(__this_cpu_read(cpu_debugreg[0]), 0); - set_debugreg(__this_cpu_read(cpu_debugreg[1]), 1); - set_debugreg(__this_cpu_read(cpu_debugreg[2]), 2); - set_debugreg(__this_cpu_read(cpu_debugreg[3]), 3); - set_debugreg(DR6_RESERVED, 6); - set_debugreg(__this_cpu_read(cpu_dr7), 7); + unsigned int seq; + + do { + seq =3D this_cpu_inc_return(cpu_dr7_seq); + set_debugreg(this_cpu_read(cpu_debugreg[0]), 0); + set_debugreg(this_cpu_read(cpu_debugreg[1]), 1); + set_debugreg(this_cpu_read(cpu_debugreg[2]), 2); + set_debugreg(this_cpu_read(cpu_debugreg[3]), 3); + set_debugreg(DR6_RESERVED, 6); + set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7); + barrier(); + } while (seq !=3D this_cpu_read(cpu_dr7_seq)); } EXPORT_SYMBOL_FOR_KVM(hw_breakpoint_restore); =20 diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 3c9f60d6ca5a..c3806f7bd3e8 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -531,11 +531,12 @@ enum nmi_states { }; static DEFINE_PER_CPU(enum nmi_states, nmi_state); static DEFINE_PER_CPU(unsigned long, nmi_cr2); -static DEFINE_PER_CPU(unsigned long, nmi_dr7); =20 DEFINE_IDTENTRY_RAW(exc_nmi) { irqentry_state_t irq_state; + unsigned long dr7; + unsigned int dr7_seq; struct nmi_stats *nsp =3D this_cpu_ptr(&nmi_stats); =20 /* @@ -572,7 +573,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi) */ sev_es_ist_enter(regs); =20 - this_cpu_write(nmi_dr7, local_db_save()); + local_db_save(&dr7, &dr7_seq); =20 irq_state =3D irqentry_nmi_enter(regs); =20 @@ -594,7 +595,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi) =20 irqentry_nmi_exit(regs, irq_state); =20 - local_db_restore(this_cpu_read(nmi_dr7)); + local_db_restore(dr7, dr7_seq); =20 sev_es_ist_exit(); =20 diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 30aa8369957e..018abe736285 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1231,8 +1231,12 @@ static noinstr void exc_debug_kernel(struct pt_regs = *regs, unsigned long dr6) * it results in an endless recursion and stack overflow. Thus we stay * with the IDT approach, i.e., save DR7 and disable #DB. */ - unsigned long dr7 =3D local_db_save(); - irqentry_state_t irq_state =3D irqentry_nmi_enter(regs); + unsigned long dr7; + unsigned int dr7_seq; + irqentry_state_t irq_state; + + local_db_save(&dr7, &dr7_seq); + irq_state =3D irqentry_nmi_enter(regs); instrumentation_begin(); =20 /* @@ -1289,7 +1293,7 @@ static noinstr void exc_debug_kernel(struct pt_regs *= regs, unsigned long dr6) instrumentation_end(); irqentry_nmi_exit(regs, irq_state); =20 - local_db_restore(dr7); + local_db_restore(dr7, dr7_seq); } =20 static noinstr void exc_debug_user(struct pt_regs *regs, unsigned long dr6) From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1F9D23EAB4; Tue, 22 Sep 2026 04:25:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051139; cv=none; b=c62IGmeACx1VQw4LtJrNbMkNpyMAPnf7I4cRG/2kntw3WxN35C3jW0mAD63AMB8ehUJ2K7jLtCFE7kKf1kpcjrdhEBPHeq9sIbVomHn/B/yvVPxX4A3uuMngoEGT2RUQedeAQVp5yEW6pVpWkn5UuNT0cRBakDO8PktD4gPiNKo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051139; c=relaxed/simple; bh=FP1ZO8k6dm2mq9IXVy0kJFnWXyVvZf0o1cKIp7XICdA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=spViM5nuy/2V7sk7TAV27BPv6g8x94Oep+aCKsww/W7mTj2upsJc0nN/8/KRyow7E2I+Ce29Azc2AUkPnVWePI7I9AO1F3sfVaqqTh3uywdFresxDlKwxeci5T2xILcUuZdSAuUDErpD5WBBhZqTtuDCmXZWjIkKEBlgcCFrhlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OXrdoqC0; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OXrdoqC0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BE731F000FF; Tue, 22 Sep 2026 04:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051137; bh=fqbG6vas082OHQpmqS/hQzgdwEej8KUz2Ruuf0Wo7c4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=OXrdoqC0MMT7oPXXdRi1yokdI4tdtcqALoC9SjolHKWBXe9P4EP3/vx0H4QydEca6 sJ8bAwnPBt4upWW6uENAGnGOEHOL6aFM8IbeA7e7Q5tPQzqJbegf2Sc0u1XSFjY1Ri poigNhCS67wOYJT9N6NQ68unoWLON6aawvUdrU05ZqNOsnuW1OzUSoKE5eqSESW628 vhdMAtAlp9s3oUiu8Iq61M2BKSvr9rqTMwtguEhg3bCWU2l9Xnesl7E0lSbQd6EYTZ aESgCzsl2v0Co1fFTZhSgYlYtLKz9qjk1KFhULqMcNVPnXuVNm+nAW1Lo2agKuDtqE HYSsHq8+305Zw== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 04/13] x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API Date: Tue, 22 Sep 2026 13:25:31 +0900 Message-ID: <179005113108.388919.2257490475086393913.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Jinchao Wang Wprobe needs to move an active per-CPU watchpoint without releasing and reserving its hardware slot. Add arch_modify_local_hw_breakpoint_addr() as the architecture backend for modify_local_hw_breakpoint_addr(). It validates the new address, finds the installed local slot, and updates the hardware debug register and per-CPU shadow register. The core layer (modify_local_hw_breakpoint_addr()) handles updating bp->attr.bp_addr and counter_arch_bp(bp)->address. The caller must provide an installed local event and a valid address. Slot ownership, breakpoint type, length, mask and DR7 remain unchanged. Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.= 1785067572.git.wangjinchao600@gmail.com/ Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Jinchao Wang Signed-off-by: Masami Hiramatsu (Google) --- Changes in v17: - In arch_modify_local_hw_breakpoint_addr(), check perf_guest_in_guest() instead of cpu_dr_in_guest. Changes in v16: - Clear breakpoint in shadow cpu_dr7 before setting hardware DR7 so an intervening NMI cannot re-enable the breakpoint via local_db_restore() while the address register is being updated, preventing spurious #DB exceptions. - Check cpu_dr_in_guest and return -EBUSY to reject modifying local breakpoints while executing a KVM guest. Changes in v15: - Temporarily clear the breakpoint enable bit in DR7 before updating the debug address register, and re-enable it afterward, to comply with the x86 hardware specification and avoid spurious #DB exceptions. Changes in v13: - Add compiler barrier (barrier()) before checking cpu_dr7_seq to prevent hoisting the sequence check before set_debugreg(). - Clarify in commit log and code comment that the core layer manages updating bp->attr.bp_addr and counter_arch_bp(bp)->address. Changes in v11: - Return int error code instead of void. - Validate the new address using hw_breakpoint_arch_parse() before updating registers. - Increment cpu_dr7_seq in a sequence loop for NMI protection. - Return -ENOENT if the breakpoint slot is not found on the local CPU. --- arch/x86/include/asm/hw_breakpoint.h | 2 + arch/x86/kernel/hw_breakpoint.c | 61 ++++++++++++++++++++++++++++++= ++++ 2 files changed, 63 insertions(+) diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw= _breakpoint.h index 0bc931cd0698..f35ec491f6dd 100644 --- a/arch/x86/include/asm/hw_breakpoint.h +++ b/arch/x86/include/asm/hw_breakpoint.h @@ -59,6 +59,8 @@ extern int hw_breakpoint_exceptions_notify(struct notifie= r_block *unused, =20 =20 int arch_install_hw_breakpoint(struct perf_event *bp); +int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp, + unsigned long addr); void arch_uninstall_hw_breakpoint(struct perf_event *bp); void hw_breakpoint_pmu_read(struct perf_event *bp); void hw_breakpoint_pmu_unthrottle(struct perf_event *bp); diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoin= t.c index 901323ae7d6a..b413adc2bbbf 100644 --- a/arch/x86/kernel/hw_breakpoint.c +++ b/arch/x86/kernel/hw_breakpoint.c @@ -131,6 +131,67 @@ int arch_install_hw_breakpoint(struct perf_event *bp) return 0; } =20 +/* + * Modify the address of an installed perf counter breakpoint on local CPU. + * + * This updates the debug address register and per-CPU shadow state without + * changing slot allocation or DR7. The core layer (modify_local_hw_breakp= oint_addr()) + * manages updating bp->attr.bp_addr and counter_arch_bp(bp)->address. + * + * Atomic: called with IRQs disabled and handles variables and registers + * local to this CPU. + */ +int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp, + unsigned long addr) +{ + struct arch_hw_breakpoint *info =3D counter_arch_bp(bp); + struct arch_hw_breakpoint hw; + struct perf_event_attr attr =3D bp->attr; + unsigned int seq; + int i, ret; + + lockdep_assert_irqs_disabled(); + + if (perf_guest_in_guest()) + return -EBUSY; + + attr.bp_addr =3D addr; + ret =3D hw_breakpoint_arch_parse(bp, &attr, &hw); + if (ret) + return ret; + + for (i =3D 0; i < HBP_NUM; i++) { + if (this_cpu_read(bp_per_reg[i]) =3D=3D bp) + break; + } + + if (WARN_ONCE(i =3D=3D HBP_NUM, "Can't find any breakpoint slot")) + return -ENOENT; + + do { + seq =3D this_cpu_inc_return(cpu_dr7_seq); + + /* Disable breakpoint in shadow DR7 and hardware DR7 */ + this_cpu_and(cpu_dr7, ~__encode_dr7(i, info->len, info->type)); + barrier(); + set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7); + barrier(); + + this_cpu_write(cpu_debugreg[i], addr); + barrier(); + set_debugreg(addr, i); + barrier(); + + /* Re-enable breakpoint in shadow DR7 and hardware DR7 */ + this_cpu_or(cpu_dr7, encode_dr7(i, info->len, info->type)); + barrier(); + set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7); + barrier(); + } while (seq !=3D this_cpu_read(cpu_dr7_seq)); + + return 0; +} + /* * Uninstall the breakpoint contained in the given counter. * From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A6473E40E4; Tue, 22 Sep 2026 04:25:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051151; cv=none; b=K17hXk0SdBKOQ99LCNBRFRz6rGh5zai/bUAcKpqZ7Cmcu3thW1kMwqzPy7IkRsxPyz1lbzJ4l+yemph2A2aiLZNww3lFZYmY3kZxxiRdoVOkmsdhOCSx/NN9fvlf0xbBx/8yet3bRkNpew1WSQ3EP2wjaXAoyqsWW6ESLyW4ee4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051151; c=relaxed/simple; bh=WSSazbM+fHfaUAOgnSMHO0iUtKbGi34yU9nXSmgXbfI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Q479Myr5xB+l35DXk6+slXtM/jV0sWUqhYwdzrNNDywjFc9/62e9gXsQh+0NWKoCN7Cy+tsA6S9lBCJzBSjnzhkmjWfA0qJal5LuoiX0dZCApTFQZjeBO4G83UmZvt2vb1Ua4/+p5F/MpeBV4GfOM1NNqGq4iewIo4empRaR2co= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RB67amyj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RB67amyj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 670731F000FF; Tue, 22 Sep 2026 04:25:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051149; bh=jQqBZgRQel9T6CKiVbYZiYgcese2NqaCIfPTKivOYNM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=RB67amyjtHCus3SRNlqbThEZpF4QHLy4p+djBZdc8YngAHOY8aEUcP0PEVjUZjacQ GyOZBIYXXhX6c64c0/svxaKAjgclo1FL0x1diVE07NqTCpL6Qy7mz8MAry5jtBtKuA 2mJJvwu9fyh3n4Rx/O/6fdofDEVElJf0+hVE/lXfgaM7wfvIcAoOQhUZZhpWyEwTYl NJW/JHfB7RI2Z+0sSdDvjkjLsO2j5XBGCQO2IMHYX6C9j48IrX5c415nPfhWHJc1P4 W6DiFtNij8lC62aRgtqVmJARep0yHjLdjFokeSfOAxga6YVVvEGqjvtW7RxG17Vmsi SLZIEdx5xhm5Q== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 05/13] HWBP: Add modify_local_hw_breakpoint_addr() API Date: Tue, 22 Sep 2026 13:25:43 +0900 Message-ID: <179005114295.388919.7866464354733590848.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add modify_local_hw_breakpoint_addr() to update only the watched address of an installed hardware breakpoint on the local CPU without releasing and reserving its hardware slot. This is available when the architecture selects CONFIG_HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR. The caller must provide an installed local event and a valid address, and update other CPUs separately. Link: https://lore.kernel.org/all/f9c49dfa49bdc57ba8c0574bc9981c1e581acf92.= 1785067572.git.wangjinchao600@gmail.com/ Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) Signed-off-by: Jinchao Wang --- Changes in v15: - Use WRITE_ONCE() for counter_arch_bp(bp)->address and bp->attr.bp_addr in both update and rollback paths to prevent data races with concurrent readers (e.g. NMI handlers). Changes in v13: - Add comment to modify_local_hw_breakpoint_addr() clarifying the early update behavior of bp->attr.bp_addr during the NMI race window. Changes in v12: - Update bp->attr.bp_addr and counter_arch_bp(bp)->address before calling arch_modify_local_hw_breakpoint_addr() to prevent race conditions during interrupt handling. - Restore counter_arch_bp(bp)->address and bp->attr.bp_addr if arch_modify_local_hw_breakpoint_addr() fails. Changes in v11: - Check return value of arch_modify_local_hw_breakpoint_addr() and return error on failure. - Update bp->attr.bp_addr and counter_arch_bp(bp)->address only after arch modification succeeds. --- arch/Kconfig | 8 +++++++ arch/x86/Kconfig | 1 + include/linux/hw_breakpoint.h | 6 +++++ kernel/events/hw_breakpoint.c | 48 +++++++++++++++++++++++++++++++++++++= ++++ 4 files changed, 63 insertions(+) diff --git a/arch/Kconfig b/arch/Kconfig index 45c657772362..d021bc31ef14 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -474,6 +474,14 @@ config HAVE_MIXED_BREAKPOINTS_REGS Select this option if your arch implements breakpoints under the latter fashion. =20 +config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR + bool + depends on HAVE_HW_BREAKPOINT + help + Select this if the architecture can modify the address of an + installed hardware breakpoint on the local CPU without releasing + and reserving its hardware slot. + config HAVE_USER_RETURN_NOTIFIER bool =20 diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 15fd9ec5ecac..38e29e465022 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -249,6 +249,7 @@ config X86 select HAVE_FUNCTION_TRACER select HAVE_GCC_PLUGINS select HAVE_HW_BREAKPOINT + select HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR select HAVE_IOREMAP_PROT select HAVE_IRQ_EXIT_ON_IRQ_STACK if X86_64 select HAVE_IRQ_TIME_ACCOUNTING diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h index db199d653dd1..bf65c7fffd99 100644 --- a/include/linux/hw_breakpoint.h +++ b/include/linux/hw_breakpoint.h @@ -81,6 +81,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr, perf_overflow_handler_t triggered, void *context); =20 +int modify_local_hw_breakpoint_addr(struct perf_event *bp, + unsigned long addr); + extern int register_perf_hw_breakpoint(struct perf_event *bp); extern void unregister_hw_breakpoint(struct perf_event *bp); extern void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cp= u_events); @@ -124,6 +127,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *att= r, perf_overflow_handler_t triggered, void *context) { return NULL; } static inline int +modify_local_hw_breakpoint_addr(struct perf_event *bp, + unsigned long addr) { return -EOPNOTSUPP; } +static inline int register_perf_hw_breakpoint(struct perf_event *bp) { return -ENOSYS; } static inline void unregister_hw_breakpoint(struct perf_event *bp) { } static inline void diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c index 789add0c185a..69bfb17b87c4 100644 --- a/kernel/events/hw_breakpoint.c +++ b/kernel/events/hw_breakpoint.c @@ -888,6 +888,54 @@ void unregister_wide_hw_breakpoint(struct perf_event *= __percpu *cpu_events) } EXPORT_SYMBOL_GPL(unregister_wide_hw_breakpoint); =20 +/** + * modify_local_hw_breakpoint_addr - update a local breakpoint address + * @bp: the hwbp perf event for this CPU + * @addr: the new address for @bp + * + * Update only the address of an installed breakpoint on the local CPU wit= hout + * releasing and reserving its hardware slot. The caller must update other= CPUs. + * Return 0, or -EOPNOTSUPP if the architecture does not support this oper= ation. + * + * Note: bp->attr.bp_addr and counter_arch_bp(bp)->address are updated bef= ore + * modifying hardware registers so that interrupt/NMI handlers and CPU ent= ry + * paths observe the new address during the transition. If an NMI hits the= old + * breakpoint address before the hardware register update finishes, the ev= ent + * handler will see bp->attr.bp_addr already pointing to @addr. If the + * architecture update fails, both are rolled back to the previous address. + * Callers must be prepared for this behavior. + */ +#ifdef CONFIG_HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR +int modify_local_hw_breakpoint_addr(struct perf_event *bp, + unsigned long addr) +{ + unsigned long old_addr; + int ret; + + lockdep_assert_irqs_disabled(); + + old_addr =3D bp->attr.bp_addr; + WRITE_ONCE(counter_arch_bp(bp)->address, addr); + WRITE_ONCE(bp->attr.bp_addr, addr); + + ret =3D arch_modify_local_hw_breakpoint_addr(bp, addr); + if (ret) { + WRITE_ONCE(counter_arch_bp(bp)->address, old_addr); + WRITE_ONCE(bp->attr.bp_addr, old_addr); + return ret; + } + + return 0; +} +#else +int modify_local_hw_breakpoint_addr(struct perf_event *bp, + unsigned long addr) +{ + return -EOPNOTSUPP; +} +#endif +EXPORT_SYMBOL_GPL(modify_local_hw_breakpoint_addr); + /** * hw_breakpoint_is_used - check if breakpoints are currently used * From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B6B03EB818; Tue, 22 Sep 2026 04:26:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051165; cv=none; b=kfYFZueRDbkHjfvI+3OQ+P+x8aTMKsli4crOPWl/MV4/7J4TkKsrENypKepP7FPLg5Sp+DQcw4AVu6qLOmsQVxcP6o/RLDyXmYUYupr+otDhjSoI2sXi+jUkijTUGuwtCX4y/2bUK2nGavlTm0LSu+cLRSz6OahqJ5ti4i1lzmo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051165; c=relaxed/simple; bh=GFSXf293+6Esl395fU7sIC7K8jg8E/x4qZ5liEtpVHM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uj0vIxIjAoHOy/qKWuij4wC+psTXSIaPHsM/9TGq/2Ddbwmj+9ASbdvuUt9ysHrK/WldZL83lsMp/W4pHlw/424nz11+aUkZa7qI+T+izyTAHsTfnadDDRETZxcNT2qmUy+XJJv30bhtPn9E2zwG2fRGrtsboguMxLXouR/gvd0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TVrVBbna; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TVrVBbna" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ADB591F00893; Tue, 22 Sep 2026 04:25:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051162; bh=kds3CzsOz/3eozclfBKWbVGx+2Amu9pD3oHRoGo0V7U=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=TVrVBbna6gW/dYhBLPeMrzbdASjr6ZUZFRhi9OIR6U0tpESrJbQReG422HXABopd8 nwI+PMb+LZTw+5nM8kaTozE4pR30J8zQ14/aGQLqjlVYUavUdM1hpRbk2b37DItvCc EblbUOxLbRd1TWjuGpfoI2NR3NfFVJGkXn2l4pe1d6Amn+huai0a2Fj7FK/TtYPvKI LEyJYoIkEKf7nmviDSik+pxUFWfLhKKwJkT6FjhmxBvpdeMGfHeRwLzQVdeVandw2A t5aTzXslRahqQ15/+dgUiBYTcZ6nonyk+5pL5iilG2aLAez9hO8BxTPkf3XCqRERbX xUEmVAJ0n8OXg== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 06/13] tracing/wprobe: Add wprobe (watchpoint probe) trace event support Date: Tue, 22 Sep 2026 13:25:55 +0900 Message-ID: <179005115525.388919.3173173873626389076.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add hardware-breakpoint-based dynamic trace event support (wprobe). Wprobe creates a dynamic event on data read/write accesses using hardware breakpoints and logs the access context and fetchargs. Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.= 1785067572.git.wangjinchao600@gmail.com/ Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) --- Changes in v16: - Remove non-existent wprobe_events file description from readme_msg in kernel/trace/trace.c as wprobe events are created via dynamic_events. Changes in v15: - Introduce trace_wprobe_is_valid_addr() to validate target address and natural alignment for wprobe length during event creation. - Move local variable declarations to function entry in trace_wprobe_show() to comply with kernel C coding standards. - Fix indentation in kernel/trace/trace.c to use tabs instead of spaces. Changes in v14: - In parse_address_spec(), reject address 0 and user-space addresses by changing the check to '_addr < TASK_SIZE'. - In trace_wprobe_show(), format symbol and offset explicitly instead of using '%pS' to ensure dynamic event restoration works properly. - Add offset field to struct trace_wprobe and pass it to alloc_trace_wprobe(). Changes in v13: - Fix documentation notation to SYMBOL[+|-OFFS] as same as others. - Fix documentation indents and add default value. - Fix README file to add []. - Add a note about module unloading. Changes in v12: - Fix syntax comment typo ('b' to 'w') and remove dead 'ret =3D 0' initialization in __trace_wprobe_create(). - Fix documentation notation to SYMBOL[[+|-]OFFS]. Changes in v11: - Add trace_wprobe_is_busy() to prevent releasing busy events. Changes in v9: - Use kzalloc_flex for alloc_trace_wprobe. --- Documentation/trace/index.rst | 1=20 Documentation/trace/wprobetrace.rst | 78 ++++ include/linux/trace_events.h | 2=20 kernel/trace/Kconfig | 13 + kernel/trace/Makefile | 1=20 kernel/trace/trace.c | 5=20 kernel/trace/trace.h | 5=20 kernel/trace/trace_probe.c | 21 + kernel/trace/trace_probe.h | 10=20 kernel/trace/trace_wprobe.c | 775 +++++++++++++++++++++++++++++++= ++++ 10 files changed, 907 insertions(+), 4 deletions(-) create mode 100644 Documentation/trace/wprobetrace.rst create mode 100644 kernel/trace/trace_wprobe.c diff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst index f4058e8e92e3..4f3fa60e4ecd 100644 --- a/Documentation/trace/index.rst +++ b/Documentation/trace/index.rst @@ -36,6 +36,7 @@ the Linux kernel. kprobes kprobetrace fprobetrace + wprobetrace eprobetrace fprobe ring-buffer-design diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wpro= betrace.rst new file mode 100644 index 000000000000..31cc68b464e9 --- /dev/null +++ b/Documentation/trace/wprobetrace.rst @@ -0,0 +1,78 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Watchpoint probe (wprobe) Event Tracing +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. Author: Masami Hiramatsu + +Overview +-------- + +Wprobe event is a dynamic event based on the hardware breakpoint, which is +similar to other probe events, but it is for watching data access. It allo= ws +you to trace which code accesses a specified data. + +As same as other dynamic events, wprobe events are defined via +`dynamic_events` interface file on tracefs. + +Synopsis of wprobe-events +------------------------- +:: + + w:[GRP/][EVENT] SPEC [FETCHARGS] : Probe on data a= ccess + + GRP : Group name for wprobe. If omitted, use "wprobes" for it. + EVENT : Event name for wprobe. If omitted, an event name is + generated based on the address or symbol. + SPEC : Breakpoint specification. + [r|w|rw]@[:LENGTH] + + r|w|rw : Access type, r for read, w for write, and rw for both. + Default is rw if omitted. + ADDRESS : Address to watch (hexadecimal). MUST be in kernel space. + SYMBOL[+|-OFFS] : Symbol name to watch. (Optional positive/negative off= set) + LENGTH : Length of the data to watch in bytes. (1, 2, 4, or 8) + Default is 4. + + FETCHARGS : Arguments. Each probe can have up to 128 args. + $addr : Fetch the accessing address. + $value : Fetch the memory value at the accessing address (same a= s +0($addr)). + @ADDR : Fetch memory at ADDR (ADDR should be in kernel) + @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symb= ol) + +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*1)(= \*2) + \IMM : Store an immediate value to the argument. + NAME=3DFETCHARG : Set NAME as the argument name of FETCHARG. + FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types + (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types + (x8/x16/x32/x64), "char", "string", "ustring", "symbol",= "symstr" + and bitfield are supported. + + (\*1) This is useful for fetching a field of data structures. + (\*2) "u" means user-space dereference. + +For the details of TYPE, see :ref:`kprobetrace documentation `. + +Usage examples +-------------- +Here is an example to add a wprobe event on a variable `jiffies`. +:: + + # echo 'w:my_jiffies w@jiffies' >> dynamic_events + # cat dynamic_events + w:wprobes/my_jiffies w@jiffies + # echo 1 > events/wprobes/enable + # cat trace | head + # TASK-PID CPU# ||||| TIMESTAMP FUNCTION + # | | | ||||| | | + -0 [000] d.Z1. 717.026259: my_jiffies: (tick_do_up= date_jiffies64+0xbe/0x130) + -0 [000] d.Z1. 717.026373: my_jiffies: (tick_do_up= date_jiffies64+0xbe/0x130) + +You can see the code which writes to `jiffies` is `tick_do_update_jiffies6= 4()`. + +Notes +----- +Wprobe event does not disable itself even if the module is unloaded. +For example, if you add a wprobe event on a module variable, and then +unload the module, the wprobe event will still be enabled. This is for +watching the address is used unexpectedly after the module is unloaded. diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h index 5cbd09c8be8d..43ffd9a76d88 100644 --- a/include/linux/trace_events.h +++ b/include/linux/trace_events.h @@ -337,6 +337,7 @@ enum { TRACE_EVENT_FL_UPROBE_BIT, TRACE_EVENT_FL_EPROBE_BIT, TRACE_EVENT_FL_FPROBE_BIT, + TRACE_EVENT_FL_WPROBE_BIT, TRACE_EVENT_FL_CUSTOM_BIT, TRACE_EVENT_FL_TEST_STR_BIT, }; @@ -367,6 +368,7 @@ enum { TRACE_EVENT_FL_UPROBE =3D (1 << TRACE_EVENT_FL_UPROBE_BIT), TRACE_EVENT_FL_EPROBE =3D (1 << TRACE_EVENT_FL_EPROBE_BIT), TRACE_EVENT_FL_FPROBE =3D (1 << TRACE_EVENT_FL_FPROBE_BIT), + TRACE_EVENT_FL_WPROBE =3D (1 << TRACE_EVENT_FL_WPROBE_BIT), TRACE_EVENT_FL_CUSTOM =3D (1 << TRACE_EVENT_FL_CUSTOM_BIT), TRACE_EVENT_FL_TEST_STR =3D (1 << TRACE_EVENT_FL_TEST_STR_BIT), }; diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 0ab5916575a9..b58c2565024f 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -862,6 +862,19 @@ config EPROBE_EVENTS convert the type of an event field. For example, turn an address into a string. =20 +config WPROBE_EVENTS + bool "Enable wprobe-based dynamic events" + depends on TRACING + depends on HAVE_HW_BREAKPOINT + select PROBE_EVENTS + select DYNAMIC_EVENTS + help + This allows the user to add watchpoint tracing events based on + hardware breakpoints on the fly via the ftrace interface. + + Those events can be inserted wherever hardware breakpoints can be + set, and record accessed memory address and values. + config BPF_EVENTS depends on BPF_SYSCALL depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile index f934ff586bd4..141c8323de20 100644 --- a/kernel/trace/Makefile +++ b/kernel/trace/Makefile @@ -126,6 +126,7 @@ obj-$(CONFIG_FTRACE_RECORD_RECURSION) +=3D trace_recurs= ion_record.o obj-$(CONFIG_FPROBE) +=3D fprobe.o obj-$(CONFIG_RETHOOK) +=3D rethook.o obj-$(CONFIG_FPROBE_EVENTS) +=3D trace_fprobe.o +obj-$(CONFIG_WPROBE_EVENTS) +=3D trace_wprobe.o =20 obj-$(CONFIG_TRACEPOINT_BENCHMARK) +=3D trace_benchmark.o obj-$(CONFIG_RV) +=3D rv/ diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index a946e0183fd1..da85e2568c41 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -4295,7 +4295,7 @@ static const char readme_msg[] =3D "\t\t\t Write into this file to define/undefine new trace events.\n" #endif #if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS) || \ - defined(CONFIG_FPROBE_EVENTS) + defined(CONFIG_FPROBE_EVENTS) || defined(CONFIG_WPROBE_EVENTS) "\t accepts: event-definitions (one definition per line)\n" #if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS) "\t Format: p[:[/][]] []\n" @@ -4305,6 +4305,9 @@ static const char readme_msg[] =3D "\t f[:[/][]] [%return] []\n" "\t t[:[/][]] []\n" #endif +#ifdef CONFIG_WPROBE_EVENTS + "\t w[:[/][]] [r|w|rw]@[:] []\n" +#endif #ifdef CONFIG_HIST_TRIGGERS "\t s:[synthetic/] []\n" #endif diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 74a7a50d1e78..18588c550a98 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -179,6 +179,11 @@ struct fexit_trace_entry_head { unsigned long ret_ip; }; =20 +struct wprobe_trace_entry_head { + struct trace_entry ent; + unsigned long ip; +}; + #define TRACE_BUF_SIZE 1024 =20 struct trace_array; diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c index 804442b2f7d2..64e7e363e8e2 100644 --- a/kernel/trace/trace_probe.c +++ b/kernel/trace/trace_probe.c @@ -1403,6 +1403,23 @@ static int parse_probe_vars(char *orig_arg, const st= ruct fetch_type *t, return 0; } =20 + /* wprobe only support "$addr" and "$value" variable */ + if (ctx->flags & TPARG_FL_WPROBE) { + if (!strcmp(arg, "addr")) { + code->op =3D FETCH_OP_BADDR; + return 0; + } + if (!strcmp(arg, "value")) { + code->op =3D FETCH_OP_BADDR; + code++; + code->op =3D FETCH_OP_DEREF; + code->offset =3D 0; + *pcode =3D code; + return 0; + } + goto inval; + } + if (strcmp(arg, "comm") =3D=3D 0 || strcmp(arg, "COMM") =3D=3D 0) { code->op =3D FETCH_OP_COMM; return 0; @@ -1462,8 +1479,8 @@ static int parse_probe_arg_register(char *arg, struct= fetch_insn *code, { int ret; =20 - if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE)) { - /* eprobe and fprobe do not handle registers */ + if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE | TPARG_FL_WPROBE)) { + /* eprobe, fprobe and wprobe do not handle registers */ trace_probe_log_err(ctx->offset, BAD_VAR); return -EINVAL; } diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h index d1fb3520700f..f20cafb87cc6 100644 --- a/kernel/trace/trace_probe.h +++ b/kernel/trace/trace_probe.h @@ -90,6 +90,7 @@ typedef int (*print_type_func_t)(struct trace_seq *, void= *, void *); FETCH_OP(STACK, param), /* Stack: .param =3D index */ \ FETCH_OP(STACKP, none), /* Stack pointer */ \ FETCH_OP(RETVAL, none), /* Return value */ \ + FETCH_OP(BADDR, none), /* Break address */ \ FETCH_OP(IMM, imm), /* Immediate: .immediate */ \ FETCH_OP(COMM, none), /* Current comm */ \ FETCH_OP(CURRENT, none), /* Current task_struct address */\ @@ -420,6 +421,7 @@ static inline int traceprobe_get_entry_data_size(struct= trace_probe *tp) #define TPARG_FL_USER BIT(4) #define TPARG_FL_FPROBE BIT(5) #define TPARG_FL_TPOINT BIT(6) +#define TPARG_FL_WPROBE BIT(7) #define TPARG_FL_LOC_MASK GENMASK(4, 0) =20 static inline bool tparg_is_function_entry(unsigned int flags) @@ -546,6 +548,10 @@ extern int traceprobe_define_arg_fields(struct trace_e= vent_call *event_call, C(ARG_TOO_LONG, "Argument expression is too long"), \ C(ARRAY_NO_CLOSE, "Array is not closed"), \ C(ARRAY_TOO_BIG, "Array number is too big"), \ + C(BAD_ACCESS_ADDR, "Invalid access memory address"), \ + C(BAD_ACCESS_FMT, "Access memory address requires @"), \ + C(BAD_ACCESS_LEN, "This memory access length is not supported"), \ + C(BAD_ACCESS_TYPE, "Bad memory access type"), \ C(BAD_ADDR_SUFFIX, "Invalid probed address suffix"), \ C(BAD_ARG_NAME, "Argument name must follow the same rules as C identifie= rs"), \ C(BAD_ARG_NUM, "Invalid argument number"), \ @@ -628,7 +634,9 @@ extern int traceprobe_define_arg_fields(struct trace_ev= ent_call *event_call, C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), \ C(TYPECAST_REQ_FIELD, "Typecast requires a field access"), \ C(TYPECAST_SYM_OFFSET, "@SYM+/-OFFSET with typecast needs parentheses"), \ - C(USED_ARG_NAME, "This argument name is already used"), + C(USED_ARG_NAME, "This argument name is already used"), \ + C(WPROBE_NO_MAXACT, "Watchpoint probe does not support maxactive"), \ + C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"), =20 #undef C #define C(a, b) TP_ERR_##a diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c new file mode 100644 index 000000000000..9c6100b63e8d --- /dev/null +++ b/kernel/trace/trace_wprobe.c @@ -0,0 +1,775 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Hardware-breakpoint-based tracing events + * + * Copyright (C) 2023, Masami Hiramatsu + */ +#define pr_fmt(fmt) "trace_wprobe: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "trace_dynevent.h" +#include "trace_probe.h" +#include "trace_probe_kernel.h" +#include "trace_probe_tmpl.h" +#include "trace_output.h" + +#define WPROBE_EVENT_SYSTEM "wprobes" + +static int trace_wprobe_create(const char *raw_command); +static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev); +static int trace_wprobe_release(struct dyn_event *ev); +static bool trace_wprobe_is_busy(struct dyn_event *ev); +static bool trace_wprobe_match(const char *system, const char *event, + int argc, const char **argv, struct dyn_event *ev); + +static struct dyn_event_operations trace_wprobe_ops =3D { + .create =3D trace_wprobe_create, + .show =3D trace_wprobe_show, + .is_busy =3D trace_wprobe_is_busy, + .free =3D trace_wprobe_release, + .match =3D trace_wprobe_match, +}; + +struct trace_wprobe { + struct dyn_event devent; + struct perf_event * __percpu *bp_event; + unsigned long addr; + int offset; + int len; + int type; + const char *symbol; + struct trace_probe tp; +}; + +static bool is_trace_wprobe(struct dyn_event *ev) +{ + return ev->ops =3D=3D &trace_wprobe_ops; +} + +static struct trace_wprobe *to_trace_wprobe(struct dyn_event *ev) +{ + return container_of(ev, struct trace_wprobe, devent); +} + +#define for_each_trace_wprobe(pos, dpos) \ + for_each_dyn_event(dpos) \ + if (is_trace_wprobe(dpos) && (pos =3D to_trace_wprobe(dpos))) + +static bool trace_wprobe_is_busy(struct dyn_event *ev) +{ + struct trace_wprobe *tw =3D to_trace_wprobe(ev); + + return trace_probe_is_enabled(&tw->tp); +} + +static bool trace_wprobe_match(const char *system, const char *event, + int argc, const char **argv, struct dyn_event *ev) +{ + struct trace_wprobe *tw =3D to_trace_wprobe(ev); + + if (event[0] !=3D '\0' && strcmp(trace_probe_name(&tw->tp), event)) + return false; + + if (system && strcmp(trace_probe_group_name(&tw->tp), system)) + return false; + + return trace_probe_match_command_args(&tw->tp, argc, argv); +} + +/* + * Note that we don't verify the fetch_insn code, since it does not come + * from user space. + */ +static int +process_fetch_insn(struct fetch_insn *code, void *rec, void *edata, + void *dest, void *base) +{ + void *baddr =3D rec; + unsigned long val; + int ret; + +retry: + /* 1st stage: get value from context */ + switch (code->op) { + case FETCH_OP_BADDR: + val =3D (unsigned long)baddr; + break; + case FETCH_NOP_SYMBOL: /* Ignore a place holder */ + code++; + goto retry; + default: + ret =3D process_common_fetch_insn(code, &val); + if (ret < 0) + return ret; + } + code++; + + return process_fetch_insn_bottom(code, val, dest, base); +} +NOKPROBE_SYMBOL(process_fetch_insn) + +static void wprobe_trace_handler(struct trace_wprobe *tw, + unsigned long addr, + struct pt_regs *regs, + struct trace_event_file *trace_file) +{ + struct wprobe_trace_entry_head *entry; + struct trace_event_call *call =3D trace_probe_event_call(&tw->tp); + struct trace_event_buffer fbuffer; + int dsize; + + if (WARN_ON_ONCE(call !=3D trace_file->event_call)) + return; + + if (trace_trigger_soft_disabled(trace_file)) + return; + + if (READ_ONCE(tw->addr) !=3D addr) + return; + + dsize =3D __get_data_size(&tw->tp, (void *)addr, NULL); + + entry =3D trace_event_buffer_reserve(&fbuffer, trace_file, + sizeof(*entry) + tw->tp.size + dsize); + if (!entry) + return; + + entry->ip =3D instruction_pointer(regs); + store_trace_args(&entry[1], &tw->tp, (void *)addr, NULL, sizeof(*entry), = dsize); + + fbuffer.regs =3D regs; + trace_event_buffer_commit(&fbuffer); +} + +static void wprobe_perf_handler(struct perf_event *bp, + struct perf_sample_data *data, + struct pt_regs *regs) +{ + struct trace_wprobe *tw =3D bp->overflow_handler_context; + struct event_file_link *link; + unsigned long addr =3D bp->attr.bp_addr; + + trace_probe_for_each_link_rcu(link, &tw->tp) + wprobe_trace_handler(tw, addr, regs, link->file); +} + +static int __register_trace_wprobe(struct trace_wprobe *tw) +{ + struct perf_event_attr attr; + int i, ret; + + if (tw->bp_event) + return -EINVAL; + + for (i =3D 0; i < tw->tp.nr_args; i++) { + ret =3D traceprobe_update_arg(&tw->tp.args[i]); + if (ret) + return ret; + } + + hw_breakpoint_init(&attr); + attr.bp_addr =3D tw->addr; + attr.bp_len =3D tw->len; + attr.bp_type =3D tw->type; + + tw->bp_event =3D register_wide_hw_breakpoint(&attr, wprobe_perf_handler, = tw); + if (IS_ERR_PCPU(tw->bp_event)) { + int ret =3D PTR_ERR_PCPU(tw->bp_event); + + tw->bp_event =3D NULL; + return ret; + } + + return 0; +} + +static void __unregister_trace_wprobe(struct trace_wprobe *tw) +{ + if (tw->bp_event) { + unregister_wide_hw_breakpoint(tw->bp_event); + tw->bp_event =3D NULL; + } +} + +static void free_trace_wprobe(struct trace_wprobe *tw) +{ + if (tw) { + trace_probe_cleanup(&tw->tp); + kfree(tw->symbol); + kfree(tw); + } +} +DEFINE_FREE(free_trace_wprobe, struct trace_wprobe *, + if (!IS_ERR_OR_NULL(_T)) + free_trace_wprobe(_T)) + + +static struct trace_wprobe *alloc_trace_wprobe(const char *group, + const char *event, + const char *symbol, + int offset, + unsigned long addr, + int len, int type, int nargs) +{ + struct trace_wprobe *tw __free(free_trace_wprobe) =3D NULL; + int ret; + + tw =3D kzalloc_flex(*tw, tp.args, nargs); + if (!tw) + return ERR_PTR(-ENOMEM); + + if (symbol) { + tw->symbol =3D kstrdup(symbol, GFP_KERNEL); + if (!tw->symbol) + return ERR_PTR(-ENOMEM); + } + tw->offset =3D offset; + tw->addr =3D addr; + tw->len =3D len; + tw->type =3D type; + + ret =3D trace_probe_init(&tw->tp, event, group, false, nargs); + if (ret < 0) + return ERR_PTR(ret); + + dyn_event_init(&tw->devent, &trace_wprobe_ops); + return_ptr(tw); +} + +static struct trace_wprobe *find_trace_wprobe(const char *event, + const char *group) +{ + struct dyn_event *pos; + struct trace_wprobe *tw; + + for_each_trace_wprobe(tw, pos) + if (strcmp(trace_probe_name(&tw->tp), event) =3D=3D 0 && + strcmp(trace_probe_group_name(&tw->tp), group) =3D=3D 0) + return tw; + return NULL; +} + +static enum print_line_t +print_wprobe_event(struct trace_iterator *iter, int flags, + struct trace_event *event) +{ + struct wprobe_trace_entry_head *field; + struct trace_seq *s =3D &iter->seq; + struct trace_probe *tp; + + field =3D (struct wprobe_trace_entry_head *)iter->ent; + tp =3D trace_probe_primary_from_call( + container_of(event, struct trace_event_call, event)); + if (WARN_ON_ONCE(!tp)) + goto out; + + trace_seq_printf(s, "%s: (", trace_probe_name(tp)); + + if (!seq_print_ip_sym_offset(s, field->ip, flags)) + goto out; + + trace_seq_putc(s, ')'); + + if (trace_probe_print_args(s, tp->args, tp->nr_args, + (u8 *)&field[1], field) < 0) + goto out; + + trace_seq_putc(s, '\n'); +out: + return trace_handle_return(s); +} + +static int wprobe_event_define_fields(struct trace_event_call *event_call) +{ + int ret; + struct wprobe_trace_entry_head field; + struct trace_probe *tp; + + tp =3D trace_probe_primary_from_call(event_call); + if (WARN_ON_ONCE(!tp)) + return -ENOENT; + + DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0); + + return traceprobe_define_arg_fields(event_call, sizeof(field), tp); +} + +static struct trace_event_functions wprobe_funcs =3D { + .trace =3D print_wprobe_event +}; + +static struct trace_event_fields wprobe_fields_array[] =3D { + { .type =3D TRACE_FUNCTION_TYPE, + .define_fields =3D wprobe_event_define_fields }, + {} +}; + +static int wprobe_register(struct trace_event_call *event, + enum trace_reg type, void *data); + +static inline void init_trace_event_call(struct trace_wprobe *tw) +{ + struct trace_event_call *call =3D trace_probe_event_call(&tw->tp); + + call->event.funcs =3D &wprobe_funcs; + call->class->fields_array =3D wprobe_fields_array; + call->flags =3D TRACE_EVENT_FL_WPROBE; + call->class->reg =3D wprobe_register; +} + +static int register_wprobe_event(struct trace_wprobe *tw) +{ + init_trace_event_call(tw); + return trace_probe_register_event_call(&tw->tp); +} + +static int register_trace_wprobe_event(struct trace_wprobe *tw) +{ + struct trace_wprobe *old_tw; + int ret; + + guard(mutex)(&event_mutex); + + old_tw =3D find_trace_wprobe(trace_probe_name(&tw->tp), + trace_probe_group_name(&tw->tp)); + if (old_tw) { + /* + * Wprobe does not support sibling probes because the event + * trigger (set_wprobe/clear_wprobe) identifies the target + * wprobe by its event name. Having multiple wprobes sharing + * the same event name would make the target ambiguous. + */ + trace_probe_log_set_index(0); + trace_probe_log_err(0, WPROBE_NO_SIBLING); + return -EBUSY; + } + + ret =3D register_wprobe_event(tw); + if (ret) { + trace_probe_log_set_index(0); + if (ret =3D=3D -EEXIST) + trace_probe_log_err(0, EVENT_EXIST); + else if (ret !=3D -ENOMEM) + trace_probe_log_err(0, FAIL_REG_PROBE); + return ret; + } + + dyn_event_add(&tw->devent, trace_probe_event_call(&tw->tp)); + return 0; +} +static int unregister_wprobe_event(struct trace_wprobe *tw) +{ + return trace_probe_unregister_event_call(&tw->tp); +} + +static int unregister_trace_wprobe(struct trace_wprobe *tw) +{ + if (trace_probe_has_sibling(&tw->tp)) + goto unreg; + + if (trace_probe_is_enabled(&tw->tp)) + return -EBUSY; + + if (trace_event_dyn_busy(trace_probe_event_call(&tw->tp))) + return -EBUSY; + + if (unregister_wprobe_event(tw)) + return -EBUSY; + +unreg: + __unregister_trace_wprobe(tw); + dyn_event_remove(&tw->devent); + trace_probe_unlink(&tw->tp); + + return 0; +} + +static int enable_trace_wprobe(struct trace_event_call *call, + struct trace_event_file *file) +{ + struct trace_probe *tp; + struct trace_wprobe *tw; + bool enabled; + int ret =3D 0; + + tp =3D trace_probe_primary_from_call(call); + if (WARN_ON_ONCE(!tp)) + return -ENODEV; + enabled =3D trace_probe_is_enabled(tp); + + if (file) { + ret =3D trace_probe_add_file(tp, file); + if (ret) + return ret; + } else { + trace_probe_set_flag(tp, TP_FLAG_PROFILE); + } + + if (!enabled) { + list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) { + ret =3D __register_trace_wprobe(tw); + if (ret < 0) { + struct trace_wprobe *tmp; + + list_for_each_entry(tmp, trace_probe_probe_list(tp), tp.list) { + if (tmp =3D=3D tw) + break; + __unregister_trace_wprobe(tmp); + } + if (file) + trace_probe_remove_file(tp, file); + else + trace_probe_clear_flag(tp, TP_FLAG_PROFILE); + return ret; + } + } + } + + return 0; +} + +static int disable_trace_wprobe(struct trace_event_call *call, + struct trace_event_file *file) +{ + struct trace_wprobe *tw; + struct trace_probe *tp; + + tp =3D trace_probe_primary_from_call(call); + if (WARN_ON_ONCE(!tp)) + return -ENODEV; + + if (file) { + if (!trace_probe_get_file_link(tp, file)) + return -ENOENT; + if (!trace_probe_has_single_file(tp)) + goto out; + trace_probe_clear_flag(tp, TP_FLAG_TRACE); + } else { + trace_probe_clear_flag(tp, TP_FLAG_PROFILE); + } + + if (!trace_probe_is_enabled(tp)) { + list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) { + __unregister_trace_wprobe(tw); + } + } + +out: + if (file) + trace_probe_remove_file(tp, file); + + return 0; +} + +static int wprobe_register(struct trace_event_call *event, + enum trace_reg type, void *data) +{ + struct trace_event_file *file =3D data; + + switch (type) { + case TRACE_REG_REGISTER: + return enable_trace_wprobe(event, file); + case TRACE_REG_UNREGISTER: + return disable_trace_wprobe(event, file); + +#ifdef CONFIG_PERF_EVENTS + case TRACE_REG_PERF_REGISTER: + case TRACE_REG_PERF_UNREGISTER: + case TRACE_REG_PERF_OPEN: + case TRACE_REG_PERF_CLOSE: + case TRACE_REG_PERF_ADD: + case TRACE_REG_PERF_DEL: + return -EOPNOTSUPP; +#endif + } + return 0; +} + +static bool trace_wprobe_is_valid_addr(unsigned long addr, int len) +{ + if (addr < TASK_SIZE) + return false; + + if (addr & (len - 1)) + return false; + + return true; +} + +static int parse_address_spec(const char *spec, unsigned long *addr, int *= type, + int *len, char **symbol, int *offset) +{ + char *_spec __free(kfree) =3D NULL; + int _len =3D HW_BREAKPOINT_LEN_4; + int _type =3D HW_BREAKPOINT_RW; + unsigned long _addr =3D 0; + int _offset =3D 0; + char *at, *col; + + _spec =3D kstrdup(spec, GFP_KERNEL); + if (!_spec) + return -ENOMEM; + + at =3D strchr(_spec, '@'); + col =3D strchr(_spec, ':'); + + if (!at) { + trace_probe_log_err(0, BAD_ACCESS_FMT); + return -EINVAL; + } + + if (at !=3D _spec) { + *at =3D '\0'; + + if (strcmp(_spec, "r") =3D=3D 0) + _type =3D HW_BREAKPOINT_R; + else if (strcmp(_spec, "w") =3D=3D 0) + _type =3D HW_BREAKPOINT_W; + else if (strcmp(_spec, "rw") =3D=3D 0) + _type =3D HW_BREAKPOINT_RW; + else { + trace_probe_log_err(0, BAD_ACCESS_TYPE); + return -EINVAL; + } + } + + if (col) { + *col =3D '\0'; + if (kstrtoint(col + 1, 0, &_len)) { + trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN); + return -EINVAL; + } + + switch (_len) { + case 1: + _len =3D HW_BREAKPOINT_LEN_1; + break; + case 2: + _len =3D HW_BREAKPOINT_LEN_2; + break; + case 4: + _len =3D HW_BREAKPOINT_LEN_4; + break; + case 8: + _len =3D HW_BREAKPOINT_LEN_8; + break; + default: + trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN); + return -EINVAL; + } + } + + if (kstrtoul(at + 1, 0, &_addr) !=3D 0) { + char *off_str =3D strpbrk(at + 1, "+-"); + + if (off_str) { + if (kstrtoint(off_str, 0, &_offset) !=3D 0) { + trace_probe_log_err(off_str - _spec, BAD_PROBE_ADDR); + return -EINVAL; + } + *off_str =3D '\0'; + } + _addr =3D kallsyms_lookup_name(at + 1); + if (!_addr) { + trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR); + return -ENOENT; + } + _addr +=3D _offset; + *symbol =3D kstrdup(at + 1, GFP_KERNEL); + if (!*symbol) + return -ENOMEM; + } + + if (!trace_wprobe_is_valid_addr(_addr, _len)) { + trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR); + return -EINVAL; + } + + *addr =3D _addr; + *offset =3D _offset; + *type =3D _type; + *len =3D _len; + return 0; +} + +static int __trace_wprobe_create(int argc, const char *argv[]) +{ + /* + * Argument syntax: + * w[:[GRP/][EVENT]] SPEC + * + * SPEC: + * [r|w|rw]@[ADDR|SYMBOL[+OFFS]][:LEN] + */ + struct traceprobe_parse_context *ctx __free(traceprobe_parse_context) =3D= NULL; + struct trace_wprobe *tw __free(free_trace_wprobe) =3D NULL; + const char *event =3D NULL, *group =3D WPROBE_EVENT_SYSTEM; + const char *tplog __free(trace_probe_log_clear) =3D NULL; + char *symbol __free(kfree) =3D NULL; + char *gbuf __free(kfree) =3D NULL; + char *ebuf __free(kfree) =3D NULL; + unsigned long addr; + int len, type, offset, i; + int ret; + + if (argv[0][0] !=3D 'w') + return -ECANCELED; + + tplog =3D trace_probe_log_init("wprobe", argc, argv); + + if (argc < 2) { + trace_probe_log_set_index(0); + trace_probe_log_err(0, NO_ARG_BODY); + return -EINVAL; + } + + if (argv[0][1] !=3D '\0') { + if (argv[0][1] !=3D ':') { + trace_probe_log_set_index(0); + trace_probe_log_err(1, WPROBE_NO_MAXACT); + return -EINVAL; + } + event =3D &argv[0][2]; + } + + trace_probe_log_set_index(1); + ret =3D parse_address_spec(argv[1], &addr, &type, &len, &symbol, &offset); + if (ret < 0) + return ret; + + trace_probe_log_set_index(0); + if (event) { + gbuf =3D kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL); + if (!gbuf) + return -ENOMEM; + ret =3D traceprobe_parse_event_name(&event, &group, gbuf, + event - argv[0]); + if (ret) + return ret; + } + + if (!event) { + /* Make a new event name */ + ebuf =3D kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL); + if (!ebuf) + return -ENOMEM; + if (symbol) + snprintf(ebuf, MAX_EVENT_NAME_LEN, "%s", symbol); + else + snprintf(ebuf, MAX_EVENT_NAME_LEN, "w_0x%lx", addr); + sanitize_event_name(ebuf); + event =3D ebuf; + } + + argc -=3D 2; argv +=3D 2; + if (argc > MAX_TRACE_ARGS) { + trace_probe_log_set_index(2); + trace_probe_log_err(0, TOO_MANY_ARGS); + return -E2BIG; + } + tw =3D alloc_trace_wprobe(group, event, symbol, offset, addr, len, type, = argc); + if (IS_ERR(tw)) + return PTR_ERR(tw); + + ctx =3D kzalloc_obj(*ctx); + if (!ctx) + return -ENOMEM; + + ctx->flags =3D TPARG_FL_KERNEL | TPARG_FL_WPROBE; + + /* parse arguments */ + for (i =3D 0; i < argc; i++) { + trace_probe_log_set_index(i + 2); + ctx->offset =3D 0; + ret =3D traceprobe_parse_probe_arg(&tw->tp, i, argv[i], ctx); + if (ret) + return ret; /* This can be -ENOMEM */ + } + + ret =3D traceprobe_set_print_fmt(&tw->tp, PROBE_PRINT_NORMAL); + if (ret < 0) + return ret; + + ret =3D register_trace_wprobe_event(tw); + if (!ret) + tw =3D NULL; /* To avoid free */ + + return ret; +} + +static int trace_wprobe_create(const char *raw_command) +{ + return trace_probe_create(raw_command, __trace_wprobe_create); +} + +static int trace_wprobe_release(struct dyn_event *ev) +{ + struct trace_wprobe *tw =3D to_trace_wprobe(ev); + int ret =3D unregister_trace_wprobe(tw); + + if (!ret) + free_trace_wprobe(tw); + return ret; +} + +static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev) +{ + struct trace_wprobe *tw =3D to_trace_wprobe(ev); + const char *type_str; + int i, len; + + seq_printf(m, "w:%s/%s", trace_probe_group_name(&tw->tp), + trace_probe_name(&tw->tp)); + + if (tw->type =3D=3D HW_BREAKPOINT_R) + type_str =3D "r"; + else if (tw->type =3D=3D HW_BREAKPOINT_W) + type_str =3D "w"; + else + type_str =3D "rw"; + + if (tw->len =3D=3D HW_BREAKPOINT_LEN_1) + len =3D 1; + else if (tw->len =3D=3D HW_BREAKPOINT_LEN_2) + len =3D 2; + else if (tw->len =3D=3D HW_BREAKPOINT_LEN_4) + len =3D 4; + else + len =3D 8; + + if (tw->symbol) { + if (tw->offset) + seq_printf(m, " %s@%s%+d:%d", type_str, tw->symbol, + tw->offset, len); + else + seq_printf(m, " %s@%s:%d", type_str, tw->symbol, len); + } else { + seq_printf(m, " %s@0x%lx:%d", type_str, tw->addr, len); + } + + for (i =3D 0; i < tw->tp.nr_args; i++) + seq_printf(m, " %s=3D%s", tw->tp.args[i].name, tw->tp.args[i].comm); + seq_putc(m, '\n'); + + return 0; +} + +static __init int init_wprobe_trace(void) +{ + return dyn_event_register(&trace_wprobe_ops); +} +fs_initcall(init_wprobe_trace); + From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63A7F3E44E7; Tue, 22 Sep 2026 04:26:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051175; cv=none; b=b8C2Pz7puYB5oi5OHE9HJQYOZ3G2MoJpkRDY4GA1btXPjPF4fRNVLCZei6euz4OYvDn4LFuLzXNLoYpZLdjvklss3UKAbjuvZLhv50FFnnCrCW5ffwrucgnCQjowRrDWisV0aiJmrk0KIqsW3VKPQB+exMxth1K02CIezeRBvE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051175; c=relaxed/simple; bh=vFJelAs6RkUEfep9+zWnllKBieB5TLqkKRYzQduYJEE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ejNoFETKe7kAsY5bP8K63Z0evTTge32qRXl7gdk/wbbTYellC7/fgzOSflZkQdKZJECt/xcc3NTSW1IxD9myGdT9mWc10g4fzkOChNdmh1AhZOXyEncRNoY5E1dA6byRyZP4eMrzO2pjz1s9apYN7zsymJfDHFRTliN9al5jaTg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mhAgZO8M; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mhAgZO8M" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DB171F000FF; Tue, 22 Sep 2026 04:26:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051173; bh=DVcGk6Bv8eEUx8YBEhCPRwunKrlLmLX0y57ylhq+Cto=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=mhAgZO8MPq8kW7LF4IWpMyjwzM7ORHCA7N8ePzFGO7fTTPUuAWKOQ2aALR5O8QhxZ q33Bh3oYcEWXfucMwow5mlmBegu7ZZzK8pf8Lu98mKtjcBNht0hKzpWe1qpKuiGG7s 5lrjX9jD8d3oDp/+674Nox7us0PvkjAOHQl8vcJv23tybnG7acIpZNzSVAkbeVj8wj zyNKZ66Pz7C8w80N5G95TyKpKQc0QzFyK4/X0ZWXkqx9qGrWyoMlp/tuqbtGQxmCTG owg2VN/3aO/PqJVQy8hO0roo9ERMQTGe+sDmgQX6C3R2uI9vPTfuqifhBWN9JAzDTb MbF+fwMjUpKww== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 07/13] x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires Date: Tue, 22 Sep 2026 13:26:07 +0900 Message-ID: <179005116728.388919.1381770935956884195.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add CONFIG_HAVE_POST_BREAKPOINT_HOOK which indicates the hw_breakpoint on that architecture fires after the target memory has been modified. This is currently x86 only behavior. Signed-off-by: Masami Hiramatsu (Google) Reviewed-by: Jinchao Wang --- Changes in v11: - Move select to alphabetically sorted place. --- arch/Kconfig | 10 ++++++++++ arch/x86/Kconfig | 1 + kernel/trace/Kconfig | 1 + 3 files changed, 12 insertions(+) diff --git a/arch/Kconfig b/arch/Kconfig index d021bc31ef14..5292bd05d5d4 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -474,6 +474,16 @@ config HAVE_MIXED_BREAKPOINTS_REGS Select this option if your arch implements breakpoints under the latter fashion. =20 +config HAVE_POST_BREAKPOINT_HOOK + bool + depends on HAVE_HW_BREAKPOINT + help + Depending on the arch implementation of hardware breakpoints, + some of them provide breakpoint hook after the target memory + is modified. + Select this option if your arch implements breakpoints overflow + handler hooks after the target memory is modified. + config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR bool depends on HAVE_HW_BREAKPOINT diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 38e29e465022..e4eaf0b607d3 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -289,6 +289,7 @@ config X86 select MMU_GATHER_RCU_TABLE_FREE select MMU_GATHER_MERGE_VMAS select HAVE_POSIX_CPU_TIMERS_TASK_WORK + select HAVE_POST_BREAKPOINT_HOOK select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE if UNWINDER_ORC || STACK_VALIDATION select HAVE_FUNCTION_ARG_ACCESS_API diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index b58c2565024f..d9b6fa5c35d9 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -866,6 +866,7 @@ config WPROBE_EVENTS bool "Enable wprobe-based dynamic events" depends on TRACING depends on HAVE_HW_BREAKPOINT + depends on HAVE_POST_BREAKPOINT_HOOK select PROBE_EVENTS select DYNAMIC_EVENTS help From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 969673EB0FB; Tue, 22 Sep 2026 04:26:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051187; cv=none; b=hQ2I6xfDidzGtS1hMDqEPn3xPA+j3WKbH9PuV4ppwD+eWitUa3r97SLOi6YEVV5prISnnag12kY1Ls+dHeFjgL0hCHP7dzD/itZYpugpnlSoWGaQNy9eBbPPEpDvEWv27lyNWbLsoa4ivP1FBrITtOdmsJ6TYF2wwEg6ujJB3Xs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051187; c=relaxed/simple; bh=bjSx1jVjckK9liIZZkUj2MwoFMgx8zJqitZsCqi9xdY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DEjz1D40Me1aLcmaattfsi7Tr7Z9im7gGzBDC95G35pZ1V5ISCyCHxOS4bnwALUke2xeurj2Hq9lvUqYVjSjQoTIQKoi2p61hzKqtYtENT4dtxbsiseZ5D+PKNLpOvzVOGZH7o6m/RQNRBR1J4yGv3p//UwY1mx8yErk7GuITcU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=b7a5xTOi; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="b7a5xTOi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1B9D1F00893; Tue, 22 Sep 2026 04:26:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051185; bh=WneT79VUrBBDprz0zU1MRNHOGHbJ8XBndDJPjozOKt8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=b7a5xTOiRm27C/0oZkG2kyL8Px/zKMHPtQhHP4RQALCKlCpADwkFDBzCYtiKzwIOH 2T0ICsCciRISCy1pFqSg1KsziFg5UXxU1sgL6G1jiM6C7ZXKa2LWY1pBHgsKv8S8Sh QlXV7a7rUm56eV1Bf+Hs5c1TBdR4VHyrv1qQU9E/vwFGHuYdLmfkH+1kpFA1MB6NYg Z3PsKxggBVCJ35/pOVVK2jnRBxiK/tnJNR5RWXlQfZ6QP4ywF9qRyEqQb81uFObs75 Go6G7tgb8q9HIZPiZATGS+jPz/jUyTSTa6DAhWzarjm8PEQcMlhjMr/6Me9GJYpDNq fU+tcLlyVYZRA== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 08/13] selftests: tracing: Add a basic testcase for wprobe Date: Tue, 22 Sep 2026 13:26:19 +0900 Message-ID: <179005117931.388919.6236117656921986202.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add 'add_remove_wprobe.tc' testcase for testing wprobe event that tests adding and removing operations of the wprobe event. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v17: - Catch failures when enabling wprobe and call exit_unresolved in case hardware watchpoints are exhausted. Changes in v9: - Fix command check logic to prevent early exit under 'set -e' (errexit) when grep or test fails. - Simplify enable/disable status checks by removing cat pipes. Changes in v8: - Fixed silently test failure path. --- tools/testing/selftests/ftrace/config | 1=20 .../ftrace/test.d/dynevent/add_remove_wprobe.tc | 66 ++++++++++++++++= ++++ 2 files changed, 67 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remo= ve_wprobe.tc diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftest= s/ftrace/config index 544de0db5f58..d2f503722020 100644 --- a/tools/testing/selftests/ftrace/config +++ b/tools/testing/selftests/ftrace/config @@ -27,3 +27,4 @@ CONFIG_STACK_TRACER=3Dy CONFIG_TRACER_SNAPSHOT=3Dy CONFIG_UPROBES=3Dy CONFIG_UPROBE_EVENTS=3Dy +CONFIG_WPROBE_EVENTS=3Dy diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wpro= be.tc b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc new file mode 100644 index 000000000000..3c622fb07e7f --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc @@ -0,0 +1,66 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Generic dynamic event - add/remove wprobe events +# requires: dynamic_events "w[:[/][]] [r|w|rw]@[:= ]":README + +echo 0 > events/enable +echo > dynamic_events + +# Use jiffies as a variable that is frequently written to. +TARGET=3Djiffies + +echo "w:my_wprobe w@$TARGET" >> dynamic_events + +if ! grep -q my_wprobe dynamic_events; then + echo "Failed to create wprobe event" + exit_fail +fi + +if [ ! -d events/wprobes/my_wprobe ]; then + echo "Failed to create wprobe event directory" + exit_fail +fi + +if ! echo 1 > events/wprobes/my_wprobe/enable; then + echo "Failed to enable wprobe (hardware watchpoints may be exhausted)" + exit_unresolved +fi + +# Check if the event is enabled +if ! grep -q 1 events/wprobes/my_wprobe/enable; then + echo "Failed to enable wprobe event" + exit_fail +fi + +# Let some time pass to trigger the breakpoint +sleep 1 + +# Check if we got any trace output +if ! grep -q my_wprobe trace; then + echo "wprobe event was not triggered" + exit_fail +fi + +echo 0 > events/wprobes/my_wprobe/enable + +# Check if the event is disabled +if ! grep -q 0 events/wprobes/my_wprobe/enable; then + echo "Failed to disable wprobe event" + exit_fail +fi + +echo "-:my_wprobe" >> dynamic_events + +if grep -q my_wprobe dynamic_events; then + echo "Failed to remove wprobe event" + exit_fail +fi + +if [ -d events/wprobes/my_wprobe ]; then + echo "Failed to remove wprobe event directory" + exit_fail +fi + +clear_trace + +exit 0 From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80D4F3EB0FF; Tue, 22 Sep 2026 04:26:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051199; cv=none; b=Avx+vSpG5ePUw2fqu5hWWIuXCSzZXaceaUw2rOqkBf6qcoFMFI2V6/9djG1KhxyTxV4qavBHzcH6KTjKnJ24jVarZkD3ZRzzADwEeJMT/8En4KycNf2IxKWxgJw7HpM3pXAhy6ZP7mhuwTewIS7j9Dj0WKcwZfKb+ijuILh2iMA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051199; c=relaxed/simple; bh=w/fxHVNtA+8suk/aVIjfFU697JVRJgCgDhdycK2gkzs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DtuhVr/jW/iD2Crz25Hu0iIAGH30XXdKLH0K50jY1fK43UA298sXtE4WOC9ewDT6cYI5aHS/llKzD3u0wL2LYCwn5mYeZ0AF+Ep8Oi7hLbGzOGYlBJxwRdZtFlkEo5+kGt030K9rSrhyJ6mGctuHP8KUpwKY1jcL1cWlAyoitXY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lU4X4hxk; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lU4X4hxk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20A091F000FF; Tue, 22 Sep 2026 04:26:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051197; bh=o0qMmOfhKF4hfQTENoYGXCxXEOkOcEeucQ2rUqmquDU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=lU4X4hxkOH1axShMDRRGb5Re7i+gAOO24wBquUgmdCckJ/HBIoOixc4OIyPgiwPlV O+/MgiVQGQ6wNBBAUwUKecGXt5pmRdIvN7HywaK61hDgP4mcaAyyCg+C249KCB93FS 8EDD/IqJfMZIJFlVxOQIyJrpdwsP9yAflH3osOPd8f3WUARqAgXwrN/moaZfjJSZ9a k+6uKimiiGS4pGis25FZnGQKEcVuQLkIOm1ARy6AcpqAlmXVf5su8wZlOs622qymJn d2kMedEFYoRn0T6bkOAf8VZUuBuHwusVLrVNV+hAGVTAf21jRXJfx4pkTUFtUMRZmt 5FCZ2r/bZO37A== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 09/13] selftests: tracing: Add syntax testcase for wprobe Date: Tue, 22 Sep 2026 13:26:30 +0900 Message-ID: <179005119078.388919.13826248487433752738.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add "wprobe_syntax_errors.tc" testcase for testing syntax errors of the watch probe events. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v14: - Update comment for invalid offset syntax check to BAD_PROBE_ADDR. - Add testcase for invalid address 0 error check (BAD_ACCESS_ADDR). Changes in v12: - Add NO_ARG_BODY error case. Changes in v11: - Add WPROBE_NO_MAXACT syntax error case. --- .../test.d/dynevent/wprobes_syntax_errors.tc | 23 ++++++++++++++++= ++++ 1 file changed, 23 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/wprobes_= syntax_errors.tc diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_= errors.tc b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_e= rrors.tc new file mode 100644 index 000000000000..e0df58fe7aa3 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.= tc @@ -0,0 +1,23 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Watch probe event parser error log check +# requires: dynamic_events "w[:[/][]] [r|w|rw]@[:= ]":README + +check_error() { # command-with-error-pos-by-^ + ftrace_errlog_check 'wprobe' "$1" 'dynamic_events' +} + +check_error '^w' # NO_ARG_BODY +check_error 'w^10 w@jiffies' # WPROBE_NO_MAXACT +check_error 'w ^symbol' # BAD_ACCESS_FMT +check_error 'w ^a@symbol' # BAD_ACCESS_TYPE +check_error 'w w@^symbol' # BAD_ACCESS_ADDR +check_error 'w w@^0:4' # BAD_ACCESS_ADDR +check_error 'w w@jiffies^+offset' # BAD_PROBE_ADDR +check_error 'w w@jiffies:^100' # BAD_ACCESS_LEN +check_error 'w w@jiffies ^$arg1' # BAD_VAR +check_error 'w w@jiffies ^$retval' # BAD_VAR +check_error 'w w@jiffies ^$stack' # BAD_VAR +check_error 'w w@jiffies ^%ax' # BAD_VAR + +exit 0 From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2A4E3E5A0A; Tue, 22 Sep 2026 04:26:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051212; cv=none; b=B5dpX0+05hwqtR+kQddsVMOjs6RI14qXNaYYSQWlo/BNAb/uYG01hFhQ4qeKDwS+CYUCyPYf6pT7HjmN7a2qmtWWp2WAuogbFrfU4fMHW1Xk1hv3UGCCuO96OmtIARbFagPuDS5/AotWKsrRaPx5BQ4HJE6FVtTeq88nz3FUyLI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051212; c=relaxed/simple; bh=uwUEmsKdKmjWp6ULcr8hcXWLuBHggdXkbTkg8vc7Vdk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DvOY/zuPQn1EXo8vLEWPMuV4g4q80jAUO8/Cyi542nZQhT4oI5YjOevEAio2KLj+rs+eKz2T0rWuxWXQvQaSnzqwtwNZ0eQQYTrPEp48pBdh5FlVL9LkvNA7I20uDP/Y5X+2ufFi2pTA2dgsEpC8VQLirRgVaV4ukUy76eyvGmw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YRFvq95B; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YRFvq95B" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BAB01F000FF; Tue, 22 Sep 2026 04:26:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051209; bh=qdpiECcCuIjHjcnnH7B1K+pnBO5/BkygGG6M6VK5Q3Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=YRFvq95BwCYXP6abkRXCkGdvZUCunJxNsQvBF/aRJGlvEtPhwPagW3sHPn857FhhM UXPU/X7jW+gCuJYtll+gUD8KEwA9ohQbJWR7MF0E39vCEUqD6h19fB7xaOOWn+kCiU /wjJMH2tvR5M7oBVcgE83FEN46jrcYIaXnJ9OpkXORMyydQWATrHD/SFNQRba+WEWL yvP4WvZSt45aCJ/RrbmAx2jMnDj60ALK8tlSK8m0qafm1xqr6uaA63leOw8YTbuuul vdzg0W99wTLahtEbvamwTPzebpoMa3vH5PiyamqSs7hTDwNIDrwxk3NllnRUNfkWrf QjB2hbtXUoQKQ== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 10/13] tracing/wprobe: Add set_wprobe and clear_wprobe event triggers Date: Tue, 22 Sep 2026 13:26:42 +0900 Message-ID: <179005120276.388919.1645992887114424511.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add set_wprobe and clear_wprobe event triggers to dynamically attach and detach hardware breakpoint address monitoring based on event field contents. Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.= 1785067572.git.wangjinchao600@gmail.com/ Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) --- Changes in v17: - Add dynamic CPU hotplug callbacks to handle CPU offline/online. - Restore EVENT_FILE_FL_SOFT_DISABLED_BIT in wprobe_trigger_free(). - Check whether trigger event has "unlimited" field before skipping "unlimited" in clear_wprobe trigger parsing. - Call tracepoint_synchronize_unregister() before freeing trigger data via data->cmd_ops->free() on registration failure. - Implement __wprobe_count_func()/wprobe_count_func() helpers to consolidate precondition checks and use it for count_func() and inside wprobe_trigger() function. Changes in v16: - Immediately update the hardware breakpoint on the local CPU via trace_wprobe_update_local() in wprobe_trigger() to avoid workqueue latency, and only kick irq_work for other CPUs. - Skip redundant updates in trace_wprobe_update_local() if bp->attr.bp_addr already matches the target address. - Use strcmp() instead of strncmp() for "unlimited" to avoid misinterpreting field names starting with "unlimited" as count. - Normalize "unlimited" and "count=3D" parameters before creating trigger data so trigger parsing handles count keywords correctly. Changes in v15: - Use trace_wprobe_is_valid_addr() in wprobe_trigger() to validate address and natural alignment for wprobe length. - Record SMP local update failures in tw->missed counter. - Expose missed event count as a trailing comment in trigger file. - Add set_wprobe and clear_wprobe triggers to readme_msg in trace.c. - Fix line length exceeding 100 columns in wprobe_trigger_log_err macro. - Initialize irq_work and work structs immediately after allocation in alloc_trace_wprobe() and check tw->work.func in free_trace_wprobe() to prevent WARN_ON(!work->func) on early allocation failure. Changes in v14: - Allow '-1' as a dummy address in parse_address_spec() to set initial address to WPROBE_DEFAULT_CLEAR_ADDRESS for trigger-only wprobes. - In trace_wprobe_show(), format WPROBE_DEFAULT_CLEAR_ADDRESS as '-1'. - Update documentation and examples to use '-1' instead of '0'. - Skip field parsing in clear_wprobe if parameter is a count keyword. - Set tw->addr to WPROBE_DEFAULT_CLEAR_ADDRESS before soft-enabling event and roll back on failure. - Fix memory and refcount leak of trigger_data/wprobe_data on error path. Changes in v13: - Align WPROBE_DEFAULT_CLEAR_ADDRESS to 8 bytes using dedicated u64. - Remove manual count decrement to rely on core event_trigger_count(). - Parse optional FIELD argument in clear_wprobe command. - Fix trigger reference count leak and error cleanup in command parsing. - Update document to add [:COUNT] to trigger syntax and fix adjust forms. - Add notes about the exclusive trigger setting behavior. - Remove unused dyn_event_ops_mutex. Changes in v12: - Decrement trigger data->count only when watchpoint state is actually changed. - Remove dyn_event_ops_mutex to fix lockdep circular dependency deadlock. - Add event_trigger_init() call to prevent premature freeing of trigger_data. --- Documentation/trace/wprobetrace.rst | 107 +++++ include/linux/trace_events.h | 1=20 kernel/trace/Kconfig | 10=20 kernel/trace/trace.c | 4=20 kernel/trace/trace.h | 1=20 kernel/trace/trace_events_trigger.c | 2=20 kernel/trace/trace_probe.c | 2=20 kernel/trace/trace_probe.h | 9=20 kernel/trace/trace_wprobe.c | 757 +++++++++++++++++++++++++++++++= ++++ 9 files changed, 889 insertions(+), 4 deletions(-) diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wpro= betrace.rst index 31cc68b464e9..df8985283312 100644 --- a/Documentation/trace/wprobetrace.rst +++ b/Documentation/trace/wprobetrace.rst @@ -76,3 +76,110 @@ Wprobe event does not disable itself even if the module= is unloaded. For example, if you add a wprobe event on a module variable, and then unload the module, the wprobe event will still be enabled. This is for watching the address is used unexpectedly after the module is unloaded. + +Combination with trigger action +------------------------------- +The event trigger action can extend the utilization of this wprobe. + +- set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT] +- clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]] + +Set these triggers to the target event, then the WPROBE event will be +setup to trace the memory access at FIELD[+|-ADJUST] address. +When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is +forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only +if its watching address is the same as the FIELD[+|-ADJUST] value. +If COUNT is specified, it will set/clear WPEVENT only if it hits COUNT +times. + +Notes: +- set_wprobe only works on the wprobe which is NOT set a valid address yet, + and it must be enabled after the trigger is set. +- clear_wprobe only works on the wprobe which is set a valid address, and = it + will be soft-disabled after the trigger is cleared. +- Therefore, if a trigger sets/clears a wprobe, other/same trigger events + will not work (on the same event) while the wprobe is set. + +The set_wprobe trigger does not change the type and length, these +must be set when creating a new wprobe. + +The WPROBE event must be disabled when setting the new trigger +and it will be busy afterwards. Recommended usage is to add a new +wprobe at invalid dummy address (-1) and keep disabled. + +Wprobe triggers only support target addresses in kernel memory. If a +set_wprobe trigger evaluates to a user-space memory address or NULL +pointer, the trigger action ignores the update and skips setting the +watchpoint. + +Wprobe triggers are not supported on kprobe_events, because kprobes +themselves can use software breakpoints which conflicts with wprobe +operation. + + +For example, trace the first 8 bytes of the dentry data structure passed +to do_truncate() until it is deleted by dentry_kill(). +(Note: all tracefs setup uses '>>' so that it does not kick do_truncate()) +:: + + # echo 'w:watch rw@-1:8 address=3D$addr value=3D+0($addr)' >> dynamic_ev= ents + # echo 'f:truncate do_truncate dentry=3D$arg2' >> dynamic_events + # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger + # echo 'f:dentry_kill dentry_kill dentry=3D$arg1' >> dynamic_events + # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger + # echo 1 >> events/fprobes/truncate/enable + # echo 1 >> events/fprobes/dentry_kill/enable + + # echo aaa > /tmp/hoge + # echo bbb > /tmp/hoge + # echo ccc > /tmp/hoge + # rm /tmp/hoge + +Then, the trace data will show:: + + # tracer: nop + # + # entries-in-buffer/entries-written: 32/32 #P:8 + # + # _-----=3D> irqs-off/BH-disabled + # / _----=3D> need-resched + # | / _---=3D> hardirq/softirq + # || / _--=3D> preempt-depth + # ||| / _-=3D> migrate-disable + # |||| / delay + # TASK-PID CPU# ||||| TIMESTAMP FUNCTION + # | | | ||||| | | + sh-107 [004] ...1. 9.990418: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004ad6618 + sh-107 [004] ...1. 9.990914: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004b3de78 + sh-107 [004] ...1. 9.993175: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff8880049ddd40 + sh-107 [004] ..... 9.995198: truncate: (do_truncate= +0x4/0x120) dentry=3D0xffff8880048083a8 + sh-107 [004] ...1. 9.995389: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff8880049db998 + sh-107 [004] ..Zff 9.997503: watch: (lookup_fast+0x= aa/0x150) address=3D0xffff8880048083a8 value=3D0x8200080 + sh-107 [004] ..Zff 9.997509: watch: (path_openat+0x= 211/0xda0) address=3D0xffff8880048083a8 value=3D0x8200080 + sh-107 [004] ..Zff 9.997514: watch: (path_openat+0x= a56/0xda0) address=3D0xffff8880048083a8 value=3D0x8200080 + sh-107 [004] ..Zff 9.997518: watch: (path_openat+0x= ae2/0xda0) address=3D0xffff8880048083a8 value=3D0x8200080 + sh-107 [004] ..... 9.997521: truncate: (do_truncate= +0x4/0x120) dentry=3D0xffff8880048083a8 + sh-107 [004] ...1. 9.997582: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004808270 + sh-107 [004] ...1. 9.999365: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff8880049db728 + sh-107 [004] ...1. 9.999388: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004b1c000 + rm-113 [005] ..Zff 10.000965: watch: (lookup_fast+0x= aa/0x150) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.000971: watch: (path_lookupat+= 0x97/0x1e0) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.000984: watch: (lookup_fast+0x= aa/0x150) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.000988: watch: (path_lookupat+= 0x97/0x1e0) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.001010: watch: (lookup_one_qst= r_excl+0x28/0x140) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.001014: watch: (lookup_one_qst= r_excl+0xd1/0x140) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.001018: watch: (may_delete_den= try+0x1c/0x200) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.001021: watch: (may_delete_den= try+0x195/0x200) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] ..Zff 10.001031: watch: (vfs_unlink+0x5= e/0x260) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] d.Z.. 10.001067: watch: (d_make_discard= able+0x1b/0x40) address=3D0xffff8880048083a8 value=3D0x8200080 + rm-113 [005] d.Z.. 10.001071: watch: (d_make_discard= able+0x29/0x40) address=3D0xffff8880048083a8 value=3D0x200080 + rm-113 [005] ...1. 10.001072: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff8880048083a8 + rm-113 [005] ...1. 10.001218: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff8880048083a8 + sh-107 [004] ...1. 10.001416: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff8880049db110 + sh-107 [004] ...1. 10.001444: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff8880049db248 + sh-107 [004] ...1. 10.001500: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004ad6618 + sh-107 [004] ...1. 10.002067: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004b41e78 + sh-107 [004] ...1. 10.904920: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004b41e78 + sh-107 [004] ...1. 10.905129: dentry_kill: (dentry_k= ill+0x0/0x2c0) dentry=3D0xffff888004ad6618 + +You can see the watch event is correctly configured on the dentry. diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h index 43ffd9a76d88..f81a5308c116 100644 --- a/include/linux/trace_events.h +++ b/include/linux/trace_events.h @@ -738,6 +738,7 @@ enum event_trigger_type { ETT_EVENT_HIST =3D (1 << 4), ETT_HIST_ENABLE =3D (1 << 5), ETT_EVENT_EPROBE =3D (1 << 6), + ETT_EVENT_WPROBE =3D (1 << 7), }; =20 extern int filter_match_preds(struct event_filter *filter, void *rec); diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index d9b6fa5c35d9..5fd8ed63c516 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -876,6 +876,16 @@ config WPROBE_EVENTS Those events can be inserted wherever hardware breakpoints can be set, and record accessed memory address and values. =20 +config WPROBE_TRIGGERS + depends on WPROBE_EVENTS + depends on HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR + bool + default y + help + This adds an event trigger which will set the wprobe on a specific + field of an event. This allows user to trace the memory access of + an address pointed by the event field. + config BPF_EVENTS depends on BPF_SYSCALL depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index da85e2568c41..61d7d8bfb157 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -4362,6 +4362,10 @@ static const char readme_msg[] =3D "\t trigger: traceon, traceoff\n" "\t enable_event::\n" "\t disable_event::\n" +#ifdef CONFIG_WPROBE_TRIGGERS + "\t set_wprobe::[+|-]\n" + "\t clear_wprobe:[:[+|-]]\n" +#endif #ifdef CONFIG_HIST_TRIGGERS "\t enable_hist::\n" "\t disable_hist::\n" diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 18588c550a98..a61f840b18bd 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -1983,6 +1983,7 @@ trigger_data_alloc(struct event_command *cmd_ops, cha= r *cmd, char *param, void *private_data); extern void trigger_data_free(struct event_trigger_data *data); extern int event_trigger_init(struct event_trigger_data *data); +extern void event_trigger_free(struct event_trigger_data *data); extern int trace_event_trigger_enable_disable(struct trace_event_file *fil= e, int trigger_enable); extern void update_cond_flag(struct trace_event_file *file); diff --git a/kernel/trace/trace_events_trigger.c b/kernel/trace/trace_event= s_trigger.c index 149300cc5e8a..120e6ec634b7 100644 --- a/kernel/trace/trace_events_trigger.c +++ b/kernel/trace/trace_events_trigger.c @@ -589,7 +589,7 @@ int event_trigger_init(struct event_trigger_data *data) * Usually used directly as the @free method in event trigger * implementations. */ -static void +void event_trigger_free(struct event_trigger_data *data) { if (WARN_ON_ONCE(data->ref <=3D 0)) diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c index 64e7e363e8e2..77266e042418 100644 --- a/kernel/trace/trace_probe.c +++ b/kernel/trace/trace_probe.c @@ -20,7 +20,7 @@ #undef C #define C(a, b) b =20 -static const char *trace_probe_err_text[] =3D { ERRORS }; +const char *trace_probe_err_text[] =3D { ERRORS }; =20 static const char *reserved_field_names[] =3D { "common_type", diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h index f20cafb87cc6..1e11077ead67 100644 --- a/kernel/trace/trace_probe.h +++ b/kernel/trace/trace_probe.h @@ -636,7 +636,12 @@ extern int traceprobe_define_arg_fields(struct trace_e= vent_call *event_call, C(TYPECAST_SYM_OFFSET, "@SYM+/-OFFSET with typecast needs parentheses"), \ C(USED_ARG_NAME, "This argument name is already used"), \ C(WPROBE_NO_MAXACT, "Watchpoint probe does not support maxactive"), \ - C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"), + C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"),= \ + C(WPROBE_ON_KPROBE, "Wprobe trigger is not supported on kprobe event"), \ + C(WPROBE_NOT_FOUND, "Target wprobe event is not found"), \ + C(WPROBE_BUSY, "Target wprobe event is already enabled"), \ + C(WPROBE_NEED_FIELD, "Wprobe trigger requires a target field"), \ + C(WPROBE_BAD_FIELD, "Target field must be pointer size"), =20 #undef C #define C(a, b) TP_ERR_##a @@ -660,6 +665,8 @@ void __trace_probe_log_err(int offset, int err); =20 DEFINE_FREE(trace_probe_log_clear, const char *, if (_T) trace_probe_log_c= lear()) =20 +extern const char *trace_probe_err_text[]; + #define trace_probe_log_err(offs, err) \ __trace_probe_log_err(offs, TP_ERR_##err) =20 diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c index 9c6100b63e8d..f1d6aa320f21 100644 --- a/kernel/trace/trace_wprobe.c +++ b/kernel/trace/trace_wprobe.c @@ -6,7 +6,11 @@ */ #define pr_fmt(fmt) "trace_wprobe: " fmt =20 +#include #include +#include +#include +#include #include #include #include @@ -15,11 +19,16 @@ #include #include #include +#include #include #include +#include +#include +#include =20 #include =20 +#include "trace.h" #include "trace_dynevent.h" #include "trace_probe.h" #include "trace_probe_kernel.h" @@ -45,12 +54,24 @@ static struct dyn_event_operations trace_wprobe_ops =3D= { =20 struct trace_wprobe { struct dyn_event devent; + struct list_head active_list; struct perf_event * __percpu *bp_event; unsigned long addr; int offset; int len; int type; const char *symbol; + raw_spinlock_t lock; + struct irq_work irq_work; + struct work_struct work; + atomic_t missed; + /* + * work_pending is set to 1 before irq_work_queue() and cleared to 0 + * after wprobe_work_func() finishes on_each_cpu(). This prevents a + * new trigger from overwriting tw->addr while the work is propagating + * the old address to per-CPU debug registers via IPI. + */ + atomic_t work_pending; struct trace_probe tp; }; =20 @@ -64,6 +85,9 @@ static struct trace_wprobe *to_trace_wprobe(struct dyn_ev= ent *ev) return container_of(ev, struct trace_wprobe, devent); } =20 +static DEFINE_MUTEX(wprobe_mutex); +static LIST_HEAD(active_wprobes); + #define for_each_trace_wprobe(pos, dpos) \ for_each_dyn_event(dpos) \ if (is_trace_wprobe(dpos) && (pos =3D to_trace_wprobe(dpos))) @@ -193,20 +217,130 @@ static int __register_trace_wprobe(struct trace_wpro= be *tw) return ret; } =20 + mutex_lock(&wprobe_mutex); + list_add(&tw->active_list, &active_wprobes); + mutex_unlock(&wprobe_mutex); + return 0; } =20 static void __unregister_trace_wprobe(struct trace_wprobe *tw) { if (tw->bp_event) { + mutex_lock(&wprobe_mutex); + list_del_init(&tw->active_list); + mutex_unlock(&wprobe_mutex); + + irq_work_sync(&tw->irq_work); + cancel_work_sync(&tw->work); unregister_wide_hw_breakpoint(tw->bp_event); tw->bp_event =3D NULL; } } =20 +static int wprobe_cpu_online(unsigned int cpu) +{ + struct trace_wprobe *tw; + + mutex_lock(&wprobe_mutex); + list_for_each_entry(tw, &active_wprobes, active_list) { + if (tw->bp_event && !per_cpu(*tw->bp_event, cpu)) { + struct perf_event_attr attr; + struct perf_event *bp; + + hw_breakpoint_init(&attr); + attr.bp_addr =3D READ_ONCE(tw->addr); + attr.bp_len =3D tw->len; + attr.bp_type =3D tw->type; + + bp =3D perf_event_create_kernel_counter(&attr, cpu, NULL, + wprobe_perf_handler, tw); + if (!IS_ERR(bp)) + per_cpu(*tw->bp_event, cpu) =3D bp; + } + } + mutex_unlock(&wprobe_mutex); + + return 0; +} + +static int wprobe_cpu_offline(unsigned int cpu) +{ + struct trace_wprobe *tw; + + mutex_lock(&wprobe_mutex); + list_for_each_entry(tw, &active_wprobes, active_list) { + if (tw->bp_event) { + struct perf_event *bp =3D per_cpu(*tw->bp_event, cpu); + + if (bp) { + per_cpu(*tw->bp_event, cpu) =3D NULL; + unregister_hw_breakpoint(bp); + } + } + } + mutex_unlock(&wprobe_mutex); + + return 0; +} + +static int trace_wprobe_update_local(struct trace_wprobe *tw, unsigned lon= g addr) +{ + struct perf_event * __percpu *pevent; + struct perf_event *bp; + + pevent =3D tw->bp_event; + if (!pevent) + return -EINVAL; + + bp =3D *this_cpu_ptr(pevent); + if (!bp) + return -EINVAL; + + if (bp->attr.bp_addr =3D=3D addr) + return 0; + + return modify_local_hw_breakpoint_addr(bp, addr); +} + +static void wprobe_smp_update_func(void *info) +{ + struct trace_wprobe *tw =3D info; + unsigned long addr =3D READ_ONCE(tw->addr); + + if (trace_wprobe_update_local(tw, addr)) + atomic_inc(&tw->missed); +} + +static void wprobe_work_func(struct work_struct *work) +{ + struct trace_wprobe *tw =3D container_of(work, struct trace_wprobe, work); + + cpus_read_lock(); + on_each_cpu(wprobe_smp_update_func, tw, true); + cpus_read_unlock(); + /* + * Clear work_pending after all CPUs have updated their local debug + * registers. A new trigger may now update tw->addr and queue a new + * irq_work. + */ + atomic_set(&tw->work_pending, 0); +} + +static void wprobe_irq_work_func(struct irq_work *irq_work) +{ + struct trace_wprobe *tw =3D container_of(irq_work, struct trace_wprobe, i= rq_work); + + schedule_work(&tw->work); +} + static void free_trace_wprobe(struct trace_wprobe *tw) { if (tw) { + if (tw->work.func) { + irq_work_sync(&tw->irq_work); + cancel_work_sync(&tw->work); + } trace_probe_cleanup(&tw->tp); kfree(tw->symbol); kfree(tw); @@ -231,6 +365,13 @@ static struct trace_wprobe *alloc_trace_wprobe(const c= har *group, if (!tw) return ERR_PTR(-ENOMEM); =20 + raw_spin_lock_init(&tw->lock); + INIT_LIST_HEAD(&tw->active_list); + init_irq_work(&tw->irq_work, wprobe_irq_work_func); + INIT_WORK(&tw->work, wprobe_work_func); + atomic_set(&tw->missed, 0); + atomic_set(&tw->work_pending, 0); + if (symbol) { tw->symbol =3D kstrdup(symbol, GFP_KERNEL); if (!tw->symbol) @@ -509,6 +650,11 @@ static bool trace_wprobe_is_valid_addr(unsigned long a= ddr, int len) return true; } =20 +#ifdef CONFIG_WPROBE_TRIGGERS +static u64 wprobe_trigger_clear_target __aligned(8); +#define WPROBE_DEFAULT_CLEAR_ADDRESS ((unsigned long)&wprobe_trigger_clear= _target) +#endif + static int parse_address_spec(const char *spec, unsigned long *addr, int *= type, int *len, char **symbol, int *offset) { @@ -572,7 +718,13 @@ static int parse_address_spec(const char *spec, unsign= ed long *addr, int *type, } } =20 +#ifdef CONFIG_WPROBE_TRIGGERS + if (strcmp(at + 1, "-1") =3D=3D 0) { + _addr =3D WPROBE_DEFAULT_CLEAR_ADDRESS; + } else if (kstrtoul(at + 1, 0, &_addr) !=3D 0) { +#else if (kstrtoul(at + 1, 0, &_addr) !=3D 0) { +#endif char *off_str =3D strpbrk(at + 1, "+-"); =20 if (off_str) { @@ -756,6 +908,10 @@ static int trace_wprobe_show(struct seq_file *m, struc= t dyn_event *ev) tw->offset, len); else seq_printf(m, " %s@%s:%d", type_str, tw->symbol, len); +#ifdef CONFIG_WPROBE_TRIGGERS + } else if (tw->addr =3D=3D WPROBE_DEFAULT_CLEAR_ADDRESS) { + seq_printf(m, " %s@-1:%d", type_str, len); +#endif } else { seq_printf(m, " %s@0x%lx:%d", type_str, tw->addr, len); } @@ -767,9 +923,608 @@ static int trace_wprobe_show(struct seq_file *m, stru= ct dyn_event *ev) return 0; } =20 +static enum cpuhp_state wprobe_hp_state __ro_after_init; + static __init int init_wprobe_trace(void) { - return dyn_event_register(&trace_wprobe_ops); + int ret; + + ret =3D cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "trace/wprobe:online", + wprobe_cpu_online, wprobe_cpu_offline); + if (ret < 0) + return ret; + + wprobe_hp_state =3D ret; + + ret =3D dyn_event_register(&trace_wprobe_ops); + if (ret) + cpuhp_remove_state(wprobe_hp_state); + + return ret; } fs_initcall(init_wprobe_trace); =20 +#ifdef CONFIG_WPROBE_TRIGGERS + +#define SET_WPROBE_STR "set_wprobe" +#define CLEAR_WPROBE_STR "clear_wprobe" +#define wprobe_trigger_log_err(file, glob, offs, err) \ + tracing_log_err((file)->tr, "wprobe_trigger", glob, \ + trace_probe_err_text, TP_ERR_##err, offs) + +struct wprobe_trigger_data { + struct rcu_head rcu; + struct trace_event_file *file; + struct trace_wprobe *tw; + int offset; + long adjust; + const char *field; + bool clear; +}; + +static bool __wprobe_count_func(struct event_trigger_data *data, + unsigned long addr) +{ + struct wprobe_trigger_data *wprobe_data =3D data->private_data; + struct trace_wprobe *tw =3D wprobe_data->tw; + unsigned long cur_addr; + + if (in_nmi()) + return false; + + if (atomic_read(&tw->work_pending)) + return false; + + cur_addr =3D READ_ONCE(tw->addr); + + if (!wprobe_data->clear) { + if (cur_addr !=3D WPROBE_DEFAULT_CLEAR_ADDRESS) + return false; + if (!trace_wprobe_is_valid_addr(addr, tw->len)) + return false; + } else { + if (cur_addr =3D=3D WPROBE_DEFAULT_CLEAR_ADDRESS) + return false; + if (wprobe_data->field && cur_addr !=3D addr) + return false; + } + + return true; +} + +static bool wprobe_count_func(struct event_trigger_data *data, + struct trace_buffer *buffer, void *rec, + struct ring_buffer_event *event) +{ + struct wprobe_trigger_data *wprobe_data =3D data->private_data; + struct trace_wprobe *tw =3D wprobe_data->tw; + unsigned long addr =3D 0; + + if (!data->count) + return false; + + if (wprobe_data->field) { + if (!rec) + return false; + addr =3D *(unsigned long *)((char *)rec + wprobe_data->offset); + addr +=3D wprobe_data->adjust; + } + + if (!__wprobe_count_func(data, addr)) { + atomic_inc(&tw->missed); + return false; + } + + if (data->count !=3D -1) + (data->count)--; + + return true; +} + +static void wprobe_trigger(struct event_trigger_data *data, + struct trace_buffer *buffer, void *rec, + struct ring_buffer_event *event) +{ + struct wprobe_trigger_data *wprobe_data =3D data->private_data; + struct trace_wprobe *tw =3D wprobe_data->tw; + unsigned long target_addr, addr =3D 0, flags; + + if (wprobe_data->field) { + if (!rec) + return; + addr =3D *(unsigned long *)((char *)rec + wprobe_data->offset); + addr +=3D wprobe_data->adjust; + } + + raw_spin_lock_irqsave(&tw->lock, flags); + + if (!(data->flags & EVENT_TRIGGER_FL_COUNT)) { + if (!__wprobe_count_func(data, addr)) { + atomic_inc(&tw->missed); + goto out; + } + } + + target_addr =3D wprobe_data->clear ? WPROBE_DEFAULT_CLEAR_ADDRESS : addr; + + /* + * Try updating the hardware breakpoint on the local CPU first. + * If this fails, skip updating tw->addr and do not queue irq_work. + * Note that this wprobe will not be traced until tw_addr is updated. + * (e.g. NMI happens on this CPU.) + */ + if (trace_wprobe_update_local(tw, target_addr)) { + atomic_inc(&tw->missed); + goto out; + } + + WRITE_ONCE(tw->addr, target_addr); + if (!wprobe_data->clear) + clear_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags); + else + set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags); + + /* + * For other CPUs, kick irq_work to asynchronously propagate + * the new address. + */ + if (num_online_cpus() > 1) { + atomic_set(&tw->work_pending, 1); + irq_work_queue(&tw->irq_work); + } + +out: + raw_spin_unlock_irqrestore(&tw->lock, flags); +} + +static void free_wprobe_trigger_data(struct wprobe_trigger_data *wprobe_da= ta) +{ + if (wprobe_data) { + kfree(wprobe_data->field); + kfree(wprobe_data); + } +} +DEFINE_FREE(free_wprobe_trigger_data, struct wprobe_trigger_data *, free_w= probe_trigger_data(_T)); + +static void free_private_wprobe_trigger_data(struct event_trigger_data *da= ta) +{ + free_wprobe_trigger_data(data->private_data); +} + +static int wprobe_trigger_print(struct seq_file *m, + struct event_trigger_data *data) +{ + struct wprobe_trigger_data *wprobe_data =3D data->private_data; + int missed =3D atomic_read(&wprobe_data->tw->missed); + + if (wprobe_data->clear) { + seq_printf(m, "%s:%s", CLEAR_WPROBE_STR, + trace_event_name(wprobe_data->file->event_call)); + if (wprobe_data->field) { + seq_printf(m, ":%s%+ld", + wprobe_data->field, wprobe_data->adjust); + } + } else { + seq_printf(m, "%s:%s:%s%+ld", SET_WPROBE_STR, + trace_event_name(wprobe_data->file->event_call), + wprobe_data->field, wprobe_data->adjust); + } + + if (data->count =3D=3D -1) + seq_puts(m, ":unlimited"); + else + seq_printf(m, ":count=3D%ld", data->count); + + if (data->filter_str) + seq_printf(m, " if %s", data->filter_str); + + if (missed) + seq_printf(m, " # missed: %d", missed); + + seq_putc(m, '\n'); + + return 0; +} + +static struct wprobe_trigger_data * +wprobe_trigger_alloc(struct trace_wprobe *tw, struct trace_event_file *fil= e, + bool clear) +{ + struct wprobe_trigger_data *wprobe_data; + + wprobe_data =3D kzalloc_obj(*wprobe_data); + if (!wprobe_data) + return NULL; + + wprobe_data->tw =3D tw; + wprobe_data->clear =3D clear; + wprobe_data->file =3D file; + + return wprobe_data; +} + +static void wprobe_trigger_free(struct event_trigger_data *data) +{ + struct wprobe_trigger_data *wprobe_data =3D data->private_data; + + if (WARN_ON_ONCE(data->ref <=3D 0)) + return; + + data->ref--; + if (!data->ref) { + /* + * Ensure SOFT_DISABLED flag is restored before clearing soft + * mode so that if the user did not explicitly enable this + * event, __ftrace_event_enable_disable() unregisters it. + */ + set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags); + /* Remove the SOFT_MODE flag */ + trace_event_enable_disable(wprobe_data->file, 0, 1); + trace_event_put_ref(wprobe_data->file->event_call); + trigger_data_free(data); + } +} + +static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops, + struct trace_event_file *file, + char *glob, char *cmd, + char *param_and_filter) +{ + /* + * set_wprobe:EVENT:FIELD[+OFFS] + * clear_wprobe:EVENT[:FIELD[+OFFS]] + */ + struct wprobe_trigger_data *wprobe_data =3D NULL; + struct event_trigger_data *trigger_data =3D NULL; + struct trace_event_file *wprobe_file; + struct trace_array *tr =3D file->tr; + char *event_str, *field_str, *comment; + struct ftrace_event_field *field; + struct trace_event_call *event; + bool remove, clear =3D false; + unsigned long orig_addr =3D 0; + struct trace_wprobe *tw; + char *param, *filter; + int ret; + + remove =3D event_trigger_check_remove(glob); + + if (!strcmp(cmd, CLEAR_WPROBE_STR)) + clear =3D true; + + if (param_and_filter) { + /* Recover original trigger string to show the error log correctly. */ + if (*(param_and_filter - 1) =3D=3D '\0') + *(param_and_filter - 1) =3D ':'; + comment =3D strchr(param_and_filter, '#'); + if (comment) + *comment =3D '\0'; + } + + if (event_trigger_empty_param(param_and_filter)) { + wprobe_trigger_log_err(file, glob, strlen(cmd) + 1, WPROBE_NOT_FOUND); + return -EINVAL; + } + + ret =3D event_trigger_separate_filter(param_and_filter, ¶m, &filter, = true); + if (ret) + return ret; + + if (file->event_call->flags & TRACE_EVENT_FL_KPROBE) { + wprobe_trigger_log_err(file, glob, 0, WPROBE_ON_KPROBE); + return -EOPNOTSUPP; + } + + event_str =3D strsep(¶m, ":"); + + /* Find target wprobe */ + tw =3D find_trace_wprobe(event_str, WPROBE_EVENT_SYSTEM); + if (!tw) { + wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND); + return -ENOENT; + } + /* The target wprobe must not be used (unless clear) */ + if (!remove && !clear && trace_probe_is_enabled(&tw->tp)) { + wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_BUSY); + return -EBUSY; + } + + wprobe_file =3D find_event_file(tr, WPROBE_EVENT_SYSTEM, event_str); + if (!wprobe_file) { + wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND); + return -EINVAL; + } + + wprobe_data =3D wprobe_trigger_alloc(tw, wprobe_file, clear); + if (!wprobe_data) + return -ENOMEM; + + /* clear_wprobe does not need field, but can have optional field. */ + if (!clear) { + char *offs; + + /* Find target field, which must be equivalent to "void *" */ + field_str =3D strsep(¶m, ":"); + if (!field_str) { + wprobe_trigger_log_err(file, glob, strlen(glob), WPROBE_NEED_FIELD); + ret =3D -EINVAL; + goto out_free; + } + + offs =3D strpbrk(field_str, "+-"); + if (offs) { + long val; + + if (kstrtol(offs, 0, &val) < 0) { + wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS); + ret =3D -EINVAL; + goto out_free; + } + wprobe_data->adjust =3D val; + *offs =3D '\0'; + } + + event =3D file->event_call; + field =3D trace_find_event_field(event, field_str); + if (!field) { + wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD); + ret =3D -ENOENT; + goto out_free; + } + + if (field->size !=3D sizeof(void *)) { + wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD); + ret =3D -ENOEXEC; + goto out_free; + } + wprobe_data->offset =3D field->offset; + wprobe_data->field =3D kstrdup(field_str, GFP_KERNEL); + if (!wprobe_data->field) { + ret =3D -ENOMEM; + goto out_free; + } + } else if (param && (isalpha(param[0]) || param[0] =3D=3D '_')) { + if (strncmp(param, "count=3D", 6) !=3D 0 && + (strcmp(param, "unlimited") !=3D 0 || + trace_find_event_field(file->event_call, "unlimited"))) { + char *offs; + + field_str =3D strsep(¶m, ":"); + offs =3D strpbrk(field_str, "+-"); + if (offs) { + long val; + + if (kstrtol(offs, 0, &val) < 0) { + wprobe_trigger_log_err(file, glob, + offs - glob, + BAD_DEREF_OFFS); + ret =3D -EINVAL; + goto out_free; + } + wprobe_data->adjust =3D val; + *offs =3D '\0'; + } + + event =3D file->event_call; + field =3D trace_find_event_field(event, field_str); + if (!field) { + wprobe_trigger_log_err(file, glob, + field_str - glob, + NO_EVENT_FIELD); + ret =3D -ENOENT; + goto out_free; + } + + if (field->size !=3D sizeof(void *)) { + wprobe_trigger_log_err(file, glob, + field_str - glob, + WPROBE_BAD_FIELD); + ret =3D -ENOEXEC; + goto out_free; + } + wprobe_data->offset =3D field->offset; + wprobe_data->field =3D kstrdup(field_str, GFP_KERNEL); + if (!wprobe_data->field) { + ret =3D -ENOMEM; + goto out_free; + } + } + } + + if (param) { + if (!strcmp(param, "unlimited")) + param =3D NULL; + else if (!strncmp(param, "count=3D", 6)) + param +=3D 6; + } + + trigger_data =3D trigger_data_alloc(cmd_ops, cmd, param, wprobe_data); + if (!trigger_data) { + ret =3D -ENOMEM; + goto out_free; + } + + trigger_data->private_data_free =3D free_private_wprobe_trigger_data; + + if (remove) { + event_trigger_unregister(cmd_ops, file, glob+1, trigger_data); + trigger_data_free(trigger_data); + return 0; + } + + /* Up the trigger_data count to make sure nothing frees it on failure */ + event_trigger_init(trigger_data); + + ret =3D event_trigger_parse_num(param, trigger_data); + if (ret) { + wprobe_trigger_log_err(file, glob, param - glob, BAD_IMM); + goto out_free_trigger; + } + + ret =3D event_trigger_set_filter(cmd_ops, file, filter, trigger_data); + if (ret < 0) + goto out_free_trigger; + + /* Soft-enable (register) wprobe event on WPROBE_DEFAULT_CLEAR_ADDRESS */ + if (!trace_event_try_get_ref(wprobe_file->event_call)) { + ret =3D -ENODEV; + goto out_free_trigger; + } + + if (!clear) { + orig_addr =3D tw->addr; + WRITE_ONCE(tw->addr, WPROBE_DEFAULT_CLEAR_ADDRESS); + } + + ret =3D trace_event_enable_disable(wprobe_file, 1, 1); + if (ret < 0) { + if (!clear) + WRITE_ONCE(tw->addr, orig_addr); + goto out_put; + } + + ret =3D event_trigger_register(cmd_ops, file, glob, trigger_data); + if (ret) + goto out_disable; + + /* Balance the event_trigger_init() at registration start */ + event_trigger_free(trigger_data); + return 0; + +out_disable: + if (!clear) + WRITE_ONCE(tw->addr, orig_addr); + trace_event_enable_disable(wprobe_file, 0, 1); +out_put: + trace_event_put_ref(wprobe_file->event_call); +out_free_trigger: + event_trigger_reset_filter(cmd_ops, trigger_data); + trigger_data_free(trigger_data); + return ret; + +out_free: + free_wprobe_trigger_data(wprobe_data); + return ret; +} + +/* Return event_trigger_data if there is a trigger which points the same w= probe */ +static struct event_trigger_data * +wprobe_trigger_find_same(struct event_trigger_data *test, + struct trace_event_file *file) +{ + struct wprobe_trigger_data *test_wprobe_data =3D test->private_data; + struct wprobe_trigger_data *wprobe_data; + struct event_trigger_data *iter; + + list_for_each_entry(iter, &file->triggers, list) { + wprobe_data =3D iter->private_data; + if (!wprobe_data || + iter->cmd_ops->trigger_type !=3D + test->cmd_ops->trigger_type) + continue; + if (wprobe_data->tw =3D=3D test_wprobe_data->tw && + wprobe_data->clear =3D=3D test_wprobe_data->clear) + return iter; + } + return NULL; +} + +static int wprobe_register_trigger(char *glob, + struct event_trigger_data *data, + struct trace_event_file *file) +{ + int ret =3D 0; + + lockdep_assert_held(&event_mutex); + + /* The same wprobe is not accept on the same file (event) */ + if (wprobe_trigger_find_same(data, file)) + return -EEXIST; + + if (data->cmd_ops->init) { + ret =3D data->cmd_ops->init(data); + if (ret < 0) + return ret; + } + + list_add_rcu(&data->list, &file->triggers); + + update_cond_flag(file); + ret =3D trace_event_trigger_enable_disable(file, 1); + if (ret < 0) { + list_del_rcu(&data->list); + update_cond_flag(file); + tracepoint_synchronize_unregister(); + if (data->cmd_ops->free) + data->cmd_ops->free(data); + } + return ret; +} + +static void wprobe_unregister_trigger(char *glob, + struct event_trigger_data *test, + struct trace_event_file *file) +{ + struct event_trigger_data *data; + + lockdep_assert_held(&event_mutex); + + data =3D wprobe_trigger_find_same(test, file); + if (!data) + return; + + list_del_rcu(&data->list); + trace_event_trigger_enable_disable(file, 0); + update_cond_flag(file); + tracepoint_synchronize_unregister(); + if (data->cmd_ops->free) + data->cmd_ops->free(data); +} + +static struct event_command trigger_wprobe_set_cmd =3D { + .name =3D SET_WPROBE_STR, + .trigger_type =3D ETT_EVENT_WPROBE, + /* This triggers after when the event is recorded. */ + .flags =3D EVENT_CMD_FL_NEEDS_REC, + .parse =3D wprobe_trigger_cmd_parse, + .reg =3D wprobe_register_trigger, + .unreg =3D wprobe_unregister_trigger, + .set_filter =3D set_trigger_filter, + .trigger =3D wprobe_trigger, + .count_func =3D wprobe_count_func, + .print =3D wprobe_trigger_print, + .init =3D event_trigger_init, + .free =3D wprobe_trigger_free, +}; + +static struct event_command trigger_wprobe_clear_cmd =3D { + .name =3D CLEAR_WPROBE_STR, + .trigger_type =3D ETT_EVENT_WPROBE, + /* This triggers after when the event is recorded. */ + .flags =3D EVENT_CMD_FL_NEEDS_REC, + .parse =3D wprobe_trigger_cmd_parse, + .reg =3D wprobe_register_trigger, + .unreg =3D wprobe_unregister_trigger, + .set_filter =3D set_trigger_filter, + .trigger =3D wprobe_trigger, + .count_func =3D wprobe_count_func, + .print =3D wprobe_trigger_print, + .init =3D event_trigger_init, + .free =3D wprobe_trigger_free, +}; + +static __init int init_trigger_wprobe_cmds(void) +{ + int ret; + + ret =3D register_event_command(&trigger_wprobe_set_cmd); + if (WARN_ON(ret < 0)) + return ret; + ret =3D register_event_command(&trigger_wprobe_clear_cmd); + if (WARN_ON(ret < 0)) + unregister_event_command(&trigger_wprobe_set_cmd); + + return ret; +} +fs_initcall(init_trigger_wprobe_cmds); +#endif /* CONFIG_WPROBE_TRIGGERS */ From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D37843E5A17; Tue, 22 Sep 2026 04:27:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051224; cv=none; b=tgCB7pe2dJwBO4pgonjSwcRfeXjkVOhOoubeHMZ1yukGYGkezmhrhuBBfSo22SNzwNgOg4ThY+UrDqBXvRvJmNLYNZXuS9OPmEFWIu/hMox3/A0oAj+egmv6Vx+M7jjHLKUdnrxkNbXVLdsW0OZGZJWJRaWakZpnEgaBXmj3ZK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051224; c=relaxed/simple; bh=c0/qR1SMZxF5cWG19yIP+5RRKpXPEcs5L4hNccV29MU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lFYXL6w5mydhOJOScmj46FEtvBsdPMm3G1lZhOgAVGhTp3FQWtxCswP4FyrcTSy+SCz1rCAR/aCBoweP5ffkOm42a97xs3mj/NpsHcy97z0GL6HpndTmDVtph87RHAsZxELF0lP3TyL/DW41VBPHS/ccEUbKfL70mgc3/myMGJY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BQWnGIYz; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BQWnGIYz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 297BF1F000FF; Tue, 22 Sep 2026 04:26:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051222; bh=EJatQcf7temTjSJDgO2Xx1XJlLt/jjbTKlNt6gcWfKk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=BQWnGIYz5Z78TltXkCBFo4RyI/cjNraWJNITfDra+8rsq5GSYOj7f5jXaJ6MA6cTE w+oPOmW/zPZooPKDenPcBVjKI1czDQNmfbpKdxPck8DSiJk8lBiAGFn2udZM4q9EGB XuBfl6tZutIdSmfPppbaCrQuB7NKHvs9GfAEwenW7222+/+eVWYrbzVuZVhK1Gudln F8DukcLO6JaL667wyQHc0VjAZosBA/9a6pGSwgDmBU85tvrY4IusMbKhDX3taUC+fn 7/l8yFhcJjvhtV8G/CvT34pEJfwEeFC21rd2hue0nWDqrFAUuqu1Rl0ZNcYY7pGXUF RJtTuc1WyplNw== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 11/13] selftests: tracing: Add wprobe trigger testcases Date: Tue, 22 Sep 2026 13:26:54 +0900 Message-ID: <179005121474.388919.14749093387340836147.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Add testcases for checking wprobe triggers. This sets set_wprobe and clear_wprobe triggers on fprobe event and static trace event to monitor memory accesses within the trace-events-sample kernel module. Also add a testcase for verifying wprobe triggers across CPU hotplug, and a testcase for verifying wprobe trigger syntax error logging. Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.= 1785067572.git.wangjinchao600@gmail.com/ Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) --- Changes in v17: - Add trigger-wprobe-hotplug.tc to verify wprobe triggers across CPU offline and online transitions. Changes in v15: - Add explicit failure checks with fail helper in trigger-wprobe.tc. - Declare fprobe README requirement in trigger-wprobe.tc. - Add trigger file check to trigger-wprobe-syntax-errors.tc requires line. - Fix subject tag to tracing. Changes in v14: - Update dummy wprobe event definition to use '-1' instead of '0'. - Add BTF argument dependency requirement in trigger-wprobe.tc. - Add delay before clear_trace in trigger-wprobe.tc to avoid race with clear_wprobe trigger execution. Changes in v13: - Look up target function dynamically in trigger-wprobe-syntax-errors.tc. - Disable set_wprobe trigger before verifying clear_wprobe in trigger-wprobe.tc to avoid race condition with sample_timer_cb. Changes in v12: - Add trigger-wprobe-syntax-errors.tc for verifying wprobe trigger syntax error logging. - Fix requires line in trigger-wprobe-syntax-errors.tc so test is not evaluated as unsupported prior to execution. - Define dfd=3D$arg1 fetcharg on testevent for trigger syntax checks. Changes in v11: - Update testcase to use trace-events-sample kernel module instead of VFS file operations. --- tools/testing/selftests/ftrace/config | 1=20 .../test.d/trigger/trigger-wprobe-hotplug.tc | 150 ++++++++++++++++= ++++ .../test.d/trigger/trigger-wprobe-syntax-errors.tc | 37 +++++ .../ftrace/test.d/trigger/trigger-wprobe.tc | 107 ++++++++++++++ 4 files changed, 295 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-w= probe-hotplug.tc create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-w= probe-syntax-errors.tc create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-w= probe.tc diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftest= s/ftrace/config index d2f503722020..ecdee77f360f 100644 --- a/tools/testing/selftests/ftrace/config +++ b/tools/testing/selftests/ftrace/config @@ -28,3 +28,4 @@ CONFIG_TRACER_SNAPSHOT=3Dy CONFIG_UPROBES=3Dy CONFIG_UPROBE_EVENTS=3Dy CONFIG_WPROBE_EVENTS=3Dy +CONFIG_WPROBE_TRIGGERS=3Dy diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-h= otplug.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-ho= tplug.tc new file mode 100644 index 000000000000..f82b13d813db --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.= tc @@ -0,0 +1,150 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event trigger - test wprobe trigger across cpu hotplug +# requires: dynamic_events "w[:[/][]] [r|w|rw]@[:= ]":README "f[:[/][]] [%return] []":README ev= ents/sched/sched_process_fork/trigger "[(structname[,field])][->fi= eld[->field|.field...]]":README + +fail() { #msg + echo "$1" + exit_fail +} + +if ! which nproc >/dev/null 2>&1 ; then + nproc() { + ls -d /sys/devices/system/cpu/cpu[0-9]* | wc -l + } +fi + +NP=3D`nproc` +if [ $NP -le 1 ] ; then + echo "We cannot test cpu hotplug in UP environment" + exit_unresolved +fi + +# Find an online CPU that can be offlined +for i in /sys/devices/system/cpu/cpu[1-9]*; do + if [ -f $i/online ] && [ "$(cat $i/online)" =3D "1" ]; then + cpu=3D$i + break + fi +done + +if [ -z "$cpu" ]; then + echo "We cannot test cpu hotplug without a hotpluggable online cpu" + exit_unresolved +fi + +rmmod trace-events-sample ||: +if ! modprobe trace-events-sample ; then + echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EV= ENTS=3Dm" + exit_unresolved +fi + +cleanup_wprobe_triggers() { + [ -n "$cpu" ] && [ -f "$cpu/online" ] && echo 1 > $cpu/online || true + if [ -f events/fprobes/testevent/trigger ]; then + reset_trigger_file events/fprobes/testevent/trigger || true + fi + if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then + reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true + fi + echo 0 > events/enable 2>/dev/null || true + echo > dynamic_events 2>/dev/null || true + sleep 1 + rmmod trace-events-sample 2>/dev/null || true + return 0 +} + +trap cleanup_wprobe_triggers EXIT + +echo 0 > tracing_on + +# we will skip this test if fprobe is not supported. +if ! grep -Fq "f[:[/][]] [%return] []" READ= ME; then + echo "UNRESOLVED: fprobe is not supported" + exit_unresolved +fi + +# we will skip this test if the target function does not exist. +if ! grep -wq "sample_timer_cb" /proc/kallsyms; then + echo "UNRESOLVED: sample_timer_cb not found" + exit_unresolved +fi + +:;: "Add a wprobe event used by trigger" ;: +echo 'w:watch rw@-1:8 address=3D$addr value=3D$value' > dynamic_events + +:;: "Add events for triggering wprobe" ;: +echo 'f:fprobes/testevent sample_timer_cb timer=3Dt' >> dynamic_events + +:;: "Enable all events before setting triggers" ;: +echo 1 > tracing_on +echo 1 >> events/fprobes/testevent/enable +echo 1 >> events/sample-trace/foo_bar_with_fn/enable + +:;: "Set set_wprobe trigger on testevent" ;: +echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger +if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to set set_wprobe trigger" +fi + +:;: "Take target CPU offline" ;: +echo 0 > $cpu/online +sleep 1 + +# Wait for sample_timer_cb to fire and set_wprobe trigger to activate whil= e CPU is offline +sleep 2 + +:;: "Check set_wprobe trigger activated the watchpoint" ;: +if ! grep -q watch trace; then + fail "Failed to trigger watchpoint while CPU is offline" +fi + +:;: "Bring target CPU back online" ;: +echo 1 > $cpu/online +sleep 1 + +# Clear trace and verify that watchpoint triggers after CPU is onlined +clear_trace +sleep 2 + +:;: "Verify watchpoint triggers after CPU came back online" ;: +if ! grep -q watch trace; then + fail "Failed to trigger watchpoint after CPU was onlined" +fi + +:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;: +echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger +if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; th= en + fail "Failed to set clear_wprobe trigger" +fi + +# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpo= int +echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger + +# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate = watchpoint +sleep 2 + +# Clear trace and wait to ensure no new watchpoint events are generated +clear_trace +sleep 1 + +:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;: +if grep -q watch trace; then + fail "Failed to clear watchpoint" +fi + +:;: "Remove wprobe triggers" ;: +echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger +if grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to remove set_wprobe trigger" +fi +if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then + fail "Failed to remove clear_wprobe trigger" +fi + +:;: "Disable events and remove dynamic events" ;: +echo 0 > events/enable +echo > dynamic_events +clear_trace + +exit 0 diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-s= yntax-errors.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wpr= obe-syntax-errors.tc new file mode 100644 index 000000000000..dd1ce4baa62b --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-e= rrors.tc @@ -0,0 +1,37 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event trigger - test wprobe trigger syntax errors +# requires: dynamic_events error_log "w[:[/][]] [r|w|rw]@[:]":README "f[:[/][]] [%return] []"= :README events/sched/sched_process_fork/trigger + +check_error() { # command-with-error-pos-by-^ + ftrace_errlog_check "wprobe_trigger" "$1" "events/fprobes/testevent/tr= igger" +} + +TARGET_FUNC=3D$(grep -m 1 -E -w "[tT] (vfs_read|do_sys_openat2|do_sys_open= )" /proc/kallsyms | awk '{print $3}') +if [ -z "$TARGET_FUNC" ]; then + echo "UNRESOLVED: target function not found" + exit_unresolved +fi + +# Add a dummy fprobe event to attach triggers to +echo "f:fprobes/testevent $TARGET_FUNC dfd=3D\$arg1" > dynamic_events + +# Add a target wprobe event +echo 'w:watch rw@-1:8' >> dynamic_events + +# Test errors on trigger syntax +check_error 'set_wprobe:^non_exist_wprobe:dfd' # WPROBE_NOT_FOUND +check_error 'set_wprobe:^' # WPROBE_NOT_FOUND +check_error 'set_wprobe:watch^' # WPROBE_NEED_FIELD +check_error 'set_wprobe:watch:^non_exist_field' # NO_EVENT_FIELD + +# Enable target wprobe event and test WPROBE_BUSY error +echo 1 > events/wprobes/watch/enable +check_error 'set_wprobe:^watch:dfd' # WPROBE_BUSY +echo 0 > events/wprobes/watch/enable + +# Cleanup +echo '-:watch' >> dynamic_events +echo '-:testevent' >> dynamic_events + +exit 0 diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.t= c b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc new file mode 100644 index 000000000000..043e1c915ccf --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc @@ -0,0 +1,107 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event trigger - test wprobe trigger +# requires: dynamic_events "w[:[/][]] [r|w|rw]@[:= ]":README "f[:[/][]] [%return] []":README ev= ents/sched/sched_process_fork/trigger "[(structname[,field])][->fi= eld[->field|.field...]]":README + +fail() { #msg + echo "$1" + exit_fail +} + +rmmod trace-events-sample ||: +if ! modprobe trace-events-sample ; then + echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EV= ENTS=3Dm" + exit_unresolved +fi + +cleanup_wprobe_triggers() { + if [ -f events/fprobes/testevent/trigger ]; then + reset_trigger_file events/fprobes/testevent/trigger || true + fi + if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then + reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true + fi + echo 0 > events/enable 2>/dev/null || true + echo > dynamic_events 2>/dev/null || true + sleep 1 + rmmod trace-events-sample 2>/dev/null || true + return 0 +} + +trap cleanup_wprobe_triggers EXIT + +echo 0 > tracing_on + +# we will skip this test if fprobe is not supported. +if ! grep -Fq "f[:[/][]] [%return] []" READ= ME; then + echo "UNRESOLVED: fprobe is not supported" + exit_unresolved +fi + +# we will skip this test if the target function does not exist. +if ! grep -wq "sample_timer_cb" /proc/kallsyms; then + echo "UNRESOLVED: sample_timer_cb not found" + exit_unresolved +fi + +:;: "Add a wprobe event used by trigger" ;: +echo 'w:watch rw@-1:8 address=3D$addr value=3D$value' > dynamic_events + +:;: "Add events for triggering wprobe" ;: +echo 'f:fprobes/testevent sample_timer_cb timer=3Dt' >> dynamic_events + +:;: "Enable all events before setting triggers" ;: +echo 1 > tracing_on +echo 1 >> events/fprobes/testevent/enable +echo 1 >> events/sample-trace/foo_bar_with_fn/enable + +:;: "Set set_wprobe trigger on testevent" ;: +echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger +if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to set set_wprobe trigger" +fi + +# Wait for sample_timer_cb to fire and set_wprobe trigger to activate +sleep 2 + +:;: "Check set_wprobe trigger activated the watchpoint" ;: +if ! grep -q watch trace; then + fail "Failed to trigger watchpoint" +fi + +:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;: +echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger +if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; th= en + fail "Failed to set clear_wprobe trigger" +fi + +# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpo= int +echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger + +# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate = watchpoint +sleep 2 + +# Clear trace and wait to ensure no new watchpoint events are generated +clear_trace +sleep 1 + +:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;: +if grep -q watch trace; then + fail "Failed to clear watchpoint" +fi + +:;: "Remove wprobe triggers" ;: +echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger +if grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to remove set_wprobe trigger" +fi +if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then + fail "Failed to remove clear_wprobe trigger" +fi + +:;: "Disable events and remove dynamic events" ;: +echo 0 > events/enable +echo > dynamic_events +clear_trace + +exit 0 From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 712703EB7F0; Tue, 22 Sep 2026 04:27:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051236; cv=none; b=WmkOKZf9OY2YzAiaCoNEm9MTNV8FL+xttDy28hu4wr7p55F+2ssnK8XchHLcaGnMa+ae+L011JmiHknAHzZ5QdORtHuSqS2JyJ22K7NH9mSdpjLBjKZKA8oS4F1/4Yf0v2NYItBqwE6T3NZt6R4dArCz64MXf+zL9wNSxP/S8GM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051236; c=relaxed/simple; bh=e/C78XjGWa1OTBKL/h44DVrP+RllBzkzi98oGrqxUJs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DEodJzcnK5+gBXwEgXYhU0zDSZG6GdJg0i04oR8bens5U1zD2T61UhiAzwBZRY6Z1GCd5beYoOoYXy3IEWKjyG3xZ6F2bWcfDo8YOpsYP+6WHAxc6D3uqsSOi8c3Y9O4/dTSnvxlD5leAP/l8l8jJQrICGXdsPyrpQuVyfzk+k0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ooSgyV5K; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ooSgyV5K" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B4501F000FF; Tue, 22 Sep 2026 04:27:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051234; bh=TrESpmidR65VjoimvstZtAP8oQ1Bl/ZOOD8urWC4AJ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ooSgyV5KQMCqqlIt79EsZ/D2PpScQX2Y4JrVmcVfKMtNOZbPg5cpvKrxQRwjRg/Q5 xQ/6+TXGyWlJFBclO1mGVuK0iAOWQg7s1B5TJDsv/IAOlKZGqIlxkCQq7KKGADo+3C qGGfE79EeX46psmctpUUQKY+gjpxvRTsXZ3ZxL6dVfQ0uwfAp1E7+/puefEeLnBKPi DL+fbOU59yQWUZjH6zbWAKMb/9U94HDSigLVyfAVGAJtg7ar79jqjBrztQlHF7W/Tm baeIGYSxTXdFwKPiq+z2/ZTyte0Di2MBUzkPFqMrQkC8ywOApq0wFDc6fyma9wAh6V 28u1c2yAHDHdQ== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 12/13] tracing/wprobe: Support BTF typecast in fetchargs Date: Tue, 22 Sep 2026 13:27:07 +0900 Message-ID: <179005122765.388919.3824091331444647185.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Allow BTF typecast syntax (STRUCT)FETCHARG->MEMBER in wprobe event fetchargs. Previously, handle_typecast() rejected any probe context that was not a function entry/return or tracepoint event probe. Wprobe events use $addr (the accessed address) and $value (the value at that address). By enabling BTF typecast, users can now cast these to a concrete struct type and access its fields directly. For example: echo 'w:watch rw@-1:8 dflag=3D(dentry)$addr->d_flags' >> dynamic_events With a set_wprobe trigger pointing the watchpoint at a dentry address, the resulting trace shows d_flags being accessed at that location. Note that $addr and $value are restricted to kernel-space memory, which is consistent with the existing TPARG_FL_KERNEL flag used when parsing wprobe fetchargs. Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) --- Changes in v15: - Document BTF typecast syntax for wprobe fetchargs in wprobetrace.rst. - Add explicit failure checks with fail helper in trigger-wprobe-btf-typecast.tc. - Declare fprobe README requirement in trigger-wprobe-btf-typecast.tc. Changes in v14: - Update dummy wprobe event definition to use '-1' instead of '0'. Changes in v11: - Update trigger-wprobe-btf-typecast.tc to use trace-events-sample kernel module. - Fix commit comment. Changes in v9: - Newly added. --- Documentation/trace/wprobetrace.rst | 8 ++ kernel/trace/trace_probe.c | 13 +++ kernel/trace/trace_probe.h | 5 + tools/testing/selftests/ftrace/config | 1=20 .../test.d/trigger/trigger-wprobe-btf-typecast.tc | 85 ++++++++++++++++= ++++ 5 files changed, 111 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-w= probe-btf-typecast.tc diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wpro= betrace.rst index df8985283312..20a11443c6db 100644 --- a/Documentation/trace/wprobetrace.rst +++ b/Documentation/trace/wprobetrace.rst @@ -47,6 +47,14 @@ Synopsis of wprobe-events (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types (x8/x16/x32/x64), "char", "string", "ustring", "symbol",= "symstr" and bitfield are supported. + (STRUCT[,ASGN])FETCHARG->MEMBER[->MEMBER] : If BTF is supported, typeca= st + FETCHARG to a pointer to STRUCT and then dereference the + pointer defined by ->MEMBER. ASGN can be specified optio= nally. + If ASGN is specified, FETCHARG will be cast to the same = offset + position as the ASGN member, rather than to the beginnin= g of + the STRUCT. + (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the= above + can also be used with another FETCHARG. =20 (\*1) This is useful for fetching a field of data structures. (\*2) "u" means user-space dereference. diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c index 77266e042418..ccf489635ad7 100644 --- a/kernel/trace/trace_probe.c +++ b/kernel/trace/trace_probe.c @@ -890,6 +890,16 @@ static int query_btf_struct(const char *sname, struct = traceprobe_parse_context * ctx->struct_btf =3D NULL; } =20 + if (ctx->btf) { + id =3D btf_find_by_name_kind(ctx->btf, sname, BTF_KIND_STRUCT); + if (id > 0) { + btf_get(ctx->btf); + ctx->struct_btf =3D ctx->btf; + ctx->last_struct =3D btf_type_by_id(ctx->struct_btf, id); + return 0; + } + } + id =3D bpf_find_btf_id(sname, BTF_KIND_STRUCT, &btf); if (id < 0) return id; @@ -965,7 +975,8 @@ static int handle_typecast(char *arg, struct traceprobe= _parse_context *ctx) =20 if (!(tparg_is_event_probe(ctx->flags) || tparg_is_function_entry(ctx->flags) || - tparg_is_function_return(ctx->flags))) { + tparg_is_function_return(ctx->flags) || + tparg_is_wprobe(ctx->flags))) { trace_probe_log_err(ctx->offset, NOSUP_BTFARG); return -EOPNOTSUPP; } diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h index 1e11077ead67..c54c554b1949 100644 --- a/kernel/trace/trace_probe.h +++ b/kernel/trace/trace_probe.h @@ -439,6 +439,11 @@ static inline bool tparg_is_event_probe(unsigned int f= lags) return !!(flags & TPARG_FL_TEVENT); } =20 +static inline bool tparg_is_wprobe(unsigned int flags) +{ + return !!(flags & TPARG_FL_WPROBE); +} + /* Each typecast consumes nested level. So the max number of typecast is 8= . */ #define TRACEPROBE_MAX_NESTED_LEVEL 8 =20 diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftest= s/ftrace/config index ecdee77f360f..f067874902ed 100644 --- a/tools/testing/selftests/ftrace/config +++ b/tools/testing/selftests/ftrace/config @@ -1,5 +1,6 @@ CONFIG_BPF_SYSCALL=3Dy CONFIG_DEBUG_INFO_BTF=3Dy +CONFIG_DEBUG_INFO_BTF_MODULES=3Dy CONFIG_DEBUG_INFO_DWARF4=3Dy CONFIG_EPROBE_EVENTS=3Dy CONFIG_FPROBE=3Dy diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-b= tf-typecast.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wpro= be-btf-typecast.tc new file mode 100644 index 000000000000..3d4c9fd6b8e2 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-type= cast.tc @@ -0,0 +1,85 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event trigger - test wprobe trigger with BTF typecast fetch= args +# requires: dynamic_events "w[:[/][]] [r|w|rw]@[:= ]":README "f[:[/][]] [%return] []":README ev= ents/sched/sched_process_fork/trigger "[(structname[,field])][->fi= eld[->field|.field...]]":README + +fail() { #msg + echo "$1" + exit_fail +} + +rmmod trace-events-sample ||: +if ! modprobe trace-events-sample ; then + echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EV= ENTS=3Dm" + exit_unresolved +fi + +cleanup_wprobe_triggers() { + if [ -f events/fprobes/testevent/trigger ]; then + reset_trigger_file events/fprobes/testevent/trigger || true + fi + echo 0 > events/enable 2>/dev/null || true + echo > dynamic_events 2>/dev/null || true + sleep 1 + rmmod trace-events-sample 2>/dev/null || true + return 0 +} + +trap cleanup_wprobe_triggers EXIT + +echo 0 > tracing_on + +# we will skip this test if fprobe is not supported. +if ! grep -Fq "f[:[/][]] [%return] []" READ= ME; then + echo "UNRESOLVED: fprobe is not supported" + exit_unresolved +fi + +# we will skip this test if the target function does not exist. +if ! grep -wq "sample_timer_cb" /proc/kallsyms; then + echo "UNRESOLVED: sample_timer_cb not found" + exit_unresolved +fi + +:;: "Add a wprobe event with BTF typecast fetchargs" ;: +# (foo_timer_data,timer)$addr->counter reads counter from struct foo_timer= _data via BTF typecast +echo 'w:watch rw@-1:8 address=3D$addr counter=3D(foo_timer_data,timer)$add= r->counter' >> dynamic_events + +:;: "Check the wprobe event is registered with counter field" ;: +if ! grep -q "counter" dynamic_events; then + fail "Failed to register counter field in dynamic_events" +fi + +:;: "Add fprobe event for sample_timer_cb" ;: +echo 'f:fprobes/testevent sample_timer_cb timer=3Dt' >> dynamic_events + +:;: "Enable all events before setting triggers" ;: +echo 1 > tracing_on +echo 1 >> events/fprobes/testevent/enable + +:;: "Set set_wprobe trigger on testevent" ;: +echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger +if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to set set_wprobe trigger" +fi + +# Wait for sample_timer_cb to fire and set_wprobe trigger to activate +sleep 3 + +:;: "Check set_wprobe trigger activated the watchpoint" ;: +if ! grep -q watch trace; then + fail "Failed to trigger watchpoint" +fi + +:;: "Remove wprobe triggers" ;: +echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger +if grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to remove set_wprobe trigger" +fi + +:;: "Disable events and remove dynamic events" ;: +echo 0 > events/enable +echo > dynamic_events +clear_trace + +exit 0 From nobody Thu Sep 24 17:02:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5C333EBF1A; Tue, 22 Sep 2026 04:27:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051248; cv=none; b=XxkkZsikFZuiBbHrq0MCl4S9KolZnkYbEdnpvEsEm7MCXGJ2PhTyDXAlBC4cCtu1ZVju4y3zJNOB5sWuWtcIIDpslHqD3uaa+OMJS3Mb1TRo/E7jrowATQLTwfWDRe68jJ+KXbF8BPChLLr/vE1QNriZfg08W8y2ZFZtj7rvljM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790051248; c=relaxed/simple; bh=fXb2gWtFrlvb9gL3c/NEqJtb08ga4jsJ3BQT3fybBWw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lOPq3W1ZQnPwXJMXx/RQM8vPmGxSWO9I3sGp32CnWloCNIPdoYZ6wqSsMlJKS6cFuVr+ZR2PGCZMu3dk4hdo4+02AY12qSNglsY4keka+q6CBg54uhDa+0dFTXw+iTZn8FVt+yhWUVNbWfjcIaPpycHo6k5Hc35qoMLfJMtD7+g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jUv4MxO2; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jUv4MxO2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A562B1F000FF; Tue, 22 Sep 2026 04:27:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790051246; bh=pbQTOP5WHCVeqto9PyVnRfft151zMF5eOa8D7roMGlg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jUv4MxO2C4LR6w8BBoSOf6ZyPGTVxyDQ/oEaMRHuWN4l5pipAoAKie5KCUV9BceCl 1ssUmjWKaZ1fG4Or3qfpHtlo9u+5tGbBlU+Q+n0P7TBnYkcuiS4GwkEUXWhaMnDft2 bXBbAtSL9FalDV0QP+pxE+i0nVWNIMQmD8/m6qTPcS8PUaKeZIcsKmiSuWhi7AecFJ vzFxlMaOoxiIoIAI5NVuouVUDgraLhkK3LJe7wDBd9tNAlpXDkAZdRlYGHZlKCWT8T OAF2wFzfklzI6yr+LC39GfBponSsdrlR2FgNXply8fsDBgxYzaI+ruKohK49y2Ua5q D3Vt9CJ1eCGMg== From: "Masami Hiramatsu (Google)" To: Steven Rostedt , Peter Zijlstra , Ingo Molnar , Sean Christopherson Cc: Jinchao Wang , Mathieu Desnoyers , Masami Hiramatsu , Thomas Gleixner , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , Alexander Shishkin , Ian Rogers , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org, x86@kernel.org, Paolo Bonzini , kvm@vger.kernel.org Subject: [PATCH v17 13/13] tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger Date: Tue, 22 Sep 2026 13:27:19 +0900 Message-ID: <179005123942.388919.17930857651144049898.stgit@devnote2> X-Mailer: git-send-email 2.43.0 In-Reply-To: <179005108298.388919.4535333252892590932.stgit@devnote2> References: <179005108298.388919.4535333252892590932.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Masami Hiramatsu (Google) Allow set_wprobe trigger to use BTF struct offset resolution to specify the target address field. Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.= 1785067572.git.wangjinchao600@gmail.com/ Assisted-by: Antigravity:gemini-3.8-flash Signed-off-by: Masami Hiramatsu (Google) --- Changes in v15: - Duplicate field string to avoid modifying glob in place in wprobe_trigger_typecast_parse() so tracing_log_err() prints pristine command string. - Remove leftover offset and adjust debug print in wprobe_trigger_print(). - Document BTF struct offset resolution syntax in wprobetrace.rst. - Add explicit failure checks with fail helper in trigger-wprobe-btf-offset.tc. - Declare fprobe README requirement in trigger-wprobe-btf-offset.tc. Changes in v14: - Update dummy wprobe event definition to use '-1' instead of '0'. - Pass member_type to btf_find_struct_member() and check its kflag in get_offset_of_field(). Changes in v13: - Check for field token before calling wprobe_trigger_field_parse() in clear_wprobe to avoid spurious error log entries on numeric counts. Changes in v12: - Refactor field parsing logic into wprobe_trigger_field_parse(). - Remove unused variable count_str in wprobe_trigger_cmd_parse(). --- Documentation/trace/wprobetrace.rst | 7=20 kernel/trace/trace_wprobe.c | 294 +++++++++++++++-= ---- .../test.d/trigger/trigger-wprobe-btf-offset.tc | 85 ++++++ 3 files changed, 308 insertions(+), 78 deletions(-) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-w= probe-btf-offset.tc diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wpro= betrace.rst index 20a11443c6db..1d9b32d02d53 100644 --- a/Documentation/trace/wprobetrace.rst +++ b/Documentation/trace/wprobetrace.rst @@ -90,10 +90,17 @@ Combination with trigger action The event trigger action can extend the utilization of this wprobe. =20 - set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT] +- set_wprobe:WPEVENT:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT] - clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]] +- clear_wprobe:WPEVENT[:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COU= NT]] =20 Set these triggers to the target event, then the WPROBE event will be setup to trace the memory access at FIELD[+|-ADJUST] address. +If BTF is supported, the target address can also be resolved using BTF +struct offset resolution: `(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST]`. +Here, EVENT_FIELD is an event field containing a pointer to STRUCT (or to +the ASGN member if ASGN is specified), and MEMBER is the struct member who= se +offset is resolved automatically via BTF. When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only if its watching address is the same as the FIELD[+|-ADJUST] value. diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c index f1d6aa320f21..121310b72522 100644 --- a/kernel/trace/trace_wprobe.c +++ b/kernel/trace/trace_wprobe.c @@ -29,6 +29,7 @@ #include =20 #include "trace.h" +#include "trace_btf.h" #include "trace_dynevent.h" #include "trace_probe.h" #include "trace_probe_kernel.h" @@ -1165,6 +1166,213 @@ static void wprobe_trigger_free(struct event_trigge= r_data *data) } } =20 +#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS + +static int get_offset_of_field(struct btf *btf, const struct btf_type *typ= e, char *field_name) +{ + const struct btf_member *field; + const struct btf_type *mtype; + int bitoffs =3D 0; + u32 anon_offs; + char *next; + + do { + next =3D strchr(field_name, '.'); + if (next) + *next++ =3D '\0'; + + field =3D btf_find_struct_member(btf, type, field_name, &anon_offs, &mty= pe); + if (IS_ERR_OR_NULL(field)) + return -ENOENT; + + if (btf_type_kflag(mtype)) { + /* Reject bitfield member access */ + if (BTF_MEMBER_BITFIELD_SIZE(field->offset)) + return -EINVAL; + bitoffs +=3D anon_offs + BTF_MEMBER_BIT_OFFSET(field->offset); + } else { + bitoffs +=3D anon_offs + field->offset; + } + + field_name =3D next; + if (next) { + type =3D btf_type_skip_modifiers(btf, field->type, NULL); + if (!type) + return -ENOENT; + } + } while (next); + return bitoffs / BITS_PER_BYTE; +} + +/* btf_put(NULL) is acceptable. */ +DEFINE_FREE(btf_put, struct btf *, btf_put(_T)) + +/* parse typecast: (TYPE[,ASGN])EVENT_FIELD->FIELD[.SUBFIELD...][+-OFFS] a= nd set adjust. */ +static int wprobe_trigger_typecast_parse(char *field_str, + struct trace_event_file *file, + struct wprobe_trigger_data *wprobe_data, + const char *glob) +{ + struct btf *btf __free(btf_put) =3D NULL; + char *buf __free(kfree) =3D NULL; + struct ftrace_event_field *field; + const struct btf_type *type; + char *assign_field; + char *event_field; + char *type_field; + char *type_name; + char *offs; + long val =3D 0; + int base_offset =3D field_str - glob; + int event_field_offset; + int id, adjust; + + buf =3D kstrdup(field_str, GFP_KERNEL); + if (!buf) + return -ENOMEM; + + type_name =3D buf + 1; + event_field =3D strchr(type_name, ')'); + if (!event_field) { + wprobe_trigger_log_err(file, glob, + base_offset + (type_name - buf), + DEREF_OPEN_BRACE); + return -EINVAL; + } + *event_field++ =3D '\0'; + + /* Check the optional assign field. */ + assign_field =3D strchr(type_name, ','); + if (assign_field) + *assign_field++ =3D '\0'; + + /* Get the type field name. */ + type_field =3D strstr(event_field, "->"); + if (!type_field) { + wprobe_trigger_log_err(file, glob, + base_offset + (event_field - buf), + TYPECAST_REQ_FIELD); + return -EINVAL; + } + *type_field =3D '\0'; + type_field +=3D 2; + + offs =3D strpbrk(type_field, "+-"); + if (offs) { + if (kstrtol(offs, 0, &val) < 0) { + wprobe_trigger_log_err(file, glob, + base_offset + (offs - buf), + BAD_DEREF_OFFS); + return -EINVAL; + } + *offs =3D '\0'; + } + + /* find type from BTF */ + id =3D bpf_find_btf_id(type_name, BTF_KIND_STRUCT, &btf); + if (id < 0) { + wprobe_trigger_log_err(file, glob, + base_offset + (type_name - buf), + BAD_BTF_TID); + return id; + } + + type =3D btf_type_by_id(btf, id); + if (!type) { + wprobe_trigger_log_err(file, glob, + base_offset + (type_name - buf), + BAD_BTF_TID); + return -EINVAL; + } + + adjust =3D get_offset_of_field(btf, type, type_field); + if (adjust < 0) { + wprobe_trigger_log_err(file, glob, + base_offset + (type_field - buf), + NO_BTF_FIELD); + return adjust; + } + wprobe_data->adjust =3D adjust + val; + + if (assign_field) { + /* assign_field should be a struct field */ + adjust =3D get_offset_of_field(btf, type, assign_field); + if (adjust < 0) { + wprobe_trigger_log_err(file, glob, + base_offset + (assign_field - buf), + NO_BTF_FIELD); + return adjust; + } + wprobe_data->adjust -=3D adjust; + } + + event_field_offset =3D base_offset + (event_field - buf); + field =3D trace_find_event_field(file->event_call, event_field); + if (!field) { + wprobe_trigger_log_err(file, glob, event_field_offset, NO_EVENT_FIELD); + return -ENOENT; + } + if (field->size !=3D sizeof(void *)) { + wprobe_trigger_log_err(file, glob, event_field_offset, WPROBE_BAD_FIELD); + return -ENOEXEC; + } + wprobe_data->offset =3D field->offset; + wprobe_data->field =3D kstrdup(event_field, GFP_KERNEL); + if (!wprobe_data->field) + return -ENOMEM; + + return 0; +} +#else +static int wprobe_trigger_typecast_parse(char *field_str, + struct trace_event_file *file, + struct wprobe_trigger_data *wprobe_data, + const char *glob) +{ + wprobe_trigger_log_err(file, glob, field_str - glob, NOSUP_BTFARG); + return -EOPNOTSUPP; +} +#endif /* CONFIG_PROBE_EVENTS_BTF_ARGS */ + +static int wprobe_trigger_field_parse(char *field_str, struct trace_event_= file *file, + struct wprobe_trigger_data *wprobe_data, + const char *glob) +{ + struct ftrace_event_field *field; + char *offs; + + if (field_str[0] =3D=3D '(') + return wprobe_trigger_typecast_parse(field_str, file, wprobe_data, glob); + + offs =3D strpbrk(field_str, "+-"); + if (offs) { + long val; + + if (kstrtol(offs, 0, &val) < 0) { + wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS); + return -EINVAL; + } + wprobe_data->adjust =3D val; + *offs =3D '\0'; + } + + field =3D trace_find_event_field(file->event_call, field_str); + if (!field) { + wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD); + return -ENOENT; + } + if (field->size !=3D sizeof(void *)) { + wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD); + return -ENOEXEC; + } + wprobe_data->offset =3D field->offset; + wprobe_data->field =3D kstrdup(field_str, GFP_KERNEL); + if (!wprobe_data->field) + return -ENOMEM; + + return 0; +} + static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops, struct trace_event_file *file, char *glob, char *cmd, @@ -1177,10 +1385,8 @@ static int wprobe_trigger_cmd_parse(struct event_com= mand *cmd_ops, struct wprobe_trigger_data *wprobe_data =3D NULL; struct event_trigger_data *trigger_data =3D NULL; struct trace_event_file *wprobe_file; + char *event_str, *comment; struct trace_array *tr =3D file->tr; - char *event_str, *field_str, *comment; - struct ftrace_event_field *field; - struct trace_event_call *event; bool remove, clear =3D false; unsigned long orig_addr =3D 0; struct trace_wprobe *tw; @@ -1241,93 +1447,25 @@ static int wprobe_trigger_cmd_parse(struct event_co= mmand *cmd_ops, =20 /* clear_wprobe does not need field, but can have optional field. */ if (!clear) { - char *offs; + char *field_str =3D strsep(¶m, ":"); =20 - /* Find target field, which must be equivalent to "void *" */ - field_str =3D strsep(¶m, ":"); if (!field_str) { wprobe_trigger_log_err(file, glob, strlen(glob), WPROBE_NEED_FIELD); ret =3D -EINVAL; goto out_free; } - - offs =3D strpbrk(field_str, "+-"); - if (offs) { - long val; - - if (kstrtol(offs, 0, &val) < 0) { - wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS); - ret =3D -EINVAL; - goto out_free; - } - wprobe_data->adjust =3D val; - *offs =3D '\0'; - } - - event =3D file->event_call; - field =3D trace_find_event_field(event, field_str); - if (!field) { - wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD); - ret =3D -ENOENT; - goto out_free; - } - - if (field->size !=3D sizeof(void *)) { - wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD); - ret =3D -ENOEXEC; - goto out_free; - } - wprobe_data->offset =3D field->offset; - wprobe_data->field =3D kstrdup(field_str, GFP_KERNEL); - if (!wprobe_data->field) { - ret =3D -ENOMEM; + ret =3D wprobe_trigger_field_parse(field_str, file, wprobe_data, glob); + if (ret < 0) goto out_free; - } - } else if (param && (isalpha(param[0]) || param[0] =3D=3D '_')) { + } else if (param && (isalpha(param[0]) || param[0] =3D=3D '_' || param[0]= =3D=3D '(')) { if (strncmp(param, "count=3D", 6) !=3D 0 && (strcmp(param, "unlimited") !=3D 0 || trace_find_event_field(file->event_call, "unlimited"))) { - char *offs; - - field_str =3D strsep(¶m, ":"); - offs =3D strpbrk(field_str, "+-"); - if (offs) { - long val; - - if (kstrtol(offs, 0, &val) < 0) { - wprobe_trigger_log_err(file, glob, - offs - glob, - BAD_DEREF_OFFS); - ret =3D -EINVAL; - goto out_free; - } - wprobe_data->adjust =3D val; - *offs =3D '\0'; - } - - event =3D file->event_call; - field =3D trace_find_event_field(event, field_str); - if (!field) { - wprobe_trigger_log_err(file, glob, - field_str - glob, - NO_EVENT_FIELD); - ret =3D -ENOENT; - goto out_free; - } + char *field_str =3D strsep(¶m, ":"); =20 - if (field->size !=3D sizeof(void *)) { - wprobe_trigger_log_err(file, glob, - field_str - glob, - WPROBE_BAD_FIELD); - ret =3D -ENOEXEC; + ret =3D wprobe_trigger_field_parse(field_str, file, wprobe_data, glob); + if (ret < 0) goto out_free; - } - wprobe_data->offset =3D field->offset; - wprobe_data->field =3D kstrdup(field_str, GFP_KERNEL); - if (!wprobe_data->field) { - ret =3D -ENOMEM; - goto out_free; - } } } =20 diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-b= tf-offset.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe= -btf-offset.tc new file mode 100644 index 000000000000..8bc8ca02a712 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offs= et.tc @@ -0,0 +1,85 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event trigger - test set_wprobe trigger with BTF struct off= set +# requires: dynamic_events "w[:[/][]] [r|w|rw]@[:= ]":README "f[:[/][]] [%return] []":README ev= ents/sched/sched_process_fork/trigger "[(structname[,field])][->fi= eld[->field|.field...]]":README + +fail() { #msg + echo "$1" + exit_fail +} + +rmmod trace-events-sample ||: +if ! modprobe trace-events-sample ; then + echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EV= ENTS=3Dm" + exit_unresolved +fi + +cleanup_wprobe_triggers() { + if [ -f events/fprobes/testevent/trigger ]; then + reset_trigger_file events/fprobes/testevent/trigger || true + fi + echo 0 > events/enable 2>/dev/null || true + echo > dynamic_events 2>/dev/null || true + sleep 1 + rmmod trace-events-sample 2>/dev/null || true + return 0 +} + +trap cleanup_wprobe_triggers EXIT + +echo 0 > tracing_on + +# we will skip this test if fprobe is not supported. +if ! grep -Fq "f[:[/][]] [%return] []" READ= ME; then + echo "UNRESOLVED: fprobe is not supported" + exit_unresolved +fi + +# we will skip this test if the target function does not exist. +if ! grep -wq "sample_timer_cb" /proc/kallsyms; then + echo "UNRESOLVED: sample_timer_cb not found" + exit_unresolved +fi + +:;: "Add a wprobe event watching 8 bytes" ;: +echo 'w:watch rw@-1:8 address=3D$addr value=3D$value' >> dynamic_events + +:;: "Add fprobe event for sample_timer_cb" ;: +# sample_timer_cb(struct timer_list *t) +# container_of(t, struct foo_timer_data, timer) +echo 'f:fprobes/testevent sample_timer_cb timer=3Dt' >> dynamic_events + +:;: "Enable all events before setting triggers" ;: +echo 1 > tracing_on +echo 1 >> events/fprobes/testevent/enable + +:;: "Set set_wprobe trigger using BTF struct offset resolution" ;: +# Syntax: set_wprobe:WPEVENT:(STRUCT,FIELD)EVENT_FIELD->MEMBER +# (foo_timer_data,timer) is the BTF struct type and field name +# timer->expires is the struct member whose offset is resolved automatical= ly via BTF +echo 'set_wprobe:watch:(foo_timer_data,timer)timer->timer.expires' >> even= ts/fprobes/testevent/trigger +if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to set set_wprobe trigger" +fi + +# Wait for sample_timer_cb to fire and set_wprobe trigger to activate +sleep 3 + +:;: "Check set_wprobe trigger activated the watchpoint" ;: +if ! grep -q watch trace; then + fail "Failed to trigger watchpoint" +fi + +:;: "Remove wprobe triggers" ;: +# Since we don't know actual offset of timer->expires in foo_timer_data, w= e use reset_trigger_file +reset_trigger_file events/fprobes/testevent/trigger +if grep -q ^set_wprobe events/fprobes/testevent/trigger; then + fail "Failed to remove set_wprobe trigger" +fi + +:;: "Disable events and remove dynamic events" ;: +echo 0 > events/enable +echo > dynamic_events +clear_trace + +exit 0