[PATCH v15 00/12] tracing: wprobe: x86: Add wprobe for watchpoint

Masami Hiramatsu (Google) posted 12 patches 2 weeks, 4 days ago
There is a newer version of this series
Documentation/trace/index.rst                      |    1
Documentation/trace/wprobetrace.rst                |  200 +++
arch/Kconfig                                       |   18
arch/x86/Kconfig                                   |    2
arch/x86/include/asm/debugreg.h                    |   67 +
arch/x86/include/asm/hw_breakpoint.h               |    2
arch/x86/kernel/cpu/mce/core.c                     |   29
arch/x86/kernel/hw_breakpoint.c                    |  151 +-
arch/x86/kernel/nmi.c                              |    7
arch/x86/kernel/traps.c                            |   10
include/linux/hw_breakpoint.h                      |    6
include/linux/trace_events.h                       |    3
kernel/events/hw_breakpoint.c                      |   48 +
kernel/trace/Kconfig                               |   24
kernel/trace/Makefile                              |    1
kernel/trace/trace.c                               |   13
kernel/trace/trace.h                               |    6
kernel/trace/trace_events_trigger.c                |    2
kernel/trace/trace_probe.c                         |   36
kernel/trace/trace_probe.h                         |   22
kernel/trace/trace_wprobe.c                        | 1522 ++++++++++++++++++++
tools/testing/selftests/ftrace/config              |    3
.../ftrace/test.d/dynevent/add_remove_wprobe.tc    |   63 +
.../test.d/dynevent/wprobes_syntax_errors.tc       |   23
.../test.d/trigger/trigger-wprobe-btf-offset.tc    |   85 +
.../test.d/trigger/trigger-wprobe-btf-typecast.tc  |   85 +
.../test.d/trigger/trigger-wprobe-syntax-errors.tc |   37
.../ftrace/test.d/trigger/trigger-wprobe.tc        |  107 +
28 files changed, 2476 insertions(+), 97 deletions(-)
create mode 100644 Documentation/trace/wprobetrace.rst
create mode 100644 kernel/trace/trace_wprobe.c
create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
[PATCH v15 00/12] tracing: wprobe: x86: Add wprobe for watchpoint
Posted by Masami Hiramatsu (Google) 2 weeks, 4 days ago
Hi,

Here is the 15th version of the series for adding new wprobe (watch probe)
which provides memory access tracing event. Moreover, this can be used
via event trigger. Thus it can trace memory access on dynamically
allocated objects too.
The previous version is here:

  https://lore.kernel.org/all/178810001186.64882.2161016469449127450.stgit@devnote2/

In this version, previous bugfix patches for BTF kflag resolution and
RCU-protected kprobe blacklist were sent to Linus via probe/fixes, reducing
the series to 12 patches. This version addresses feedbacks from Sashiko on
v14[1], and more local fixes such as x86 debug register manipulation, validates
watched address and natural alignment, tracks missed events on SMP update
failures(*), prevents in-place command string modification during trigger
parsing, adds explicit failure checks in selftests, and documents BTF
typecast and struct offset syntax. I also added Assited-by: tags for this
update.

[1] https://sashiko.dev/#/patchset/178810001186.64882.2161016469449127450.stgit%40devnote2
(*) currently it just increase a missed counter.

Detailed changes are here.

Changes in v15:
 - x86/mce: Move local_db_save() to entry of exc_machine_check_kernel()
   and restore it on early return from mce_check_crashing_cpu() to
   prevent nested #DB on watched variables.
 - x86/hw_breakpoints: Always preserve DR7_FIXED_1 in
   arch_install_hw_breakpoint(), arch_uninstall_hw_breakpoint(), and
   hw_breakpoint_restore() for architectural compliance.
 - x86/hw_breakpoints: Wrap local_db_save() in a cpu_dr7_seq loop to
   detect NMI breakpoint updates and ensure DR7 is disabled before entry.
 - x86/hw_breakpoints: In local_db_restore(), restore DR7_FIXED_1 if
   saved dr7 is 0 and an NMI updated breakpoints, preserving KVM guest
   isolation.
 - x86/hw_breakpoints: Always write "val | DR7_FIXED_1" to hardware DR7
   in local_db_restore() for architectural compliance.
 - x86/hw_breakpoints: Avoid modifying saved dr7 in-place across retry
   loops.
 - x86/hw_breakpoints: Temporarily clear breakpoint enable bit in DR7
   before updating debug address register and re-enable it afterward to
   comply with x86 specification and avoid spurious #DBs.
 - HWBP: Annotate counter_arch_bp(bp)->address and bp->attr.bp_addr with
   WRITE_ONCE() in both update and rollback paths to prevent data races
   with concurrent readers (e.g. NMI handlers).
 - tracing/wprobe: Introduce trace_wprobe_is_valid_addr() to validate
   target address and natural alignment for wprobe length during event
   creation and trigger setup.
 - tracing/wprobe: Initialize irq_work and work immediately after
   allocation in alloc_trace_wprobe() and check tw->work.func in
   free_trace_wprobe() to prevent WARN_ON(!work->func) on early failures.
 - tracing/wprobe: Record SMP local update failures in tw->missed counter
   and expose missed event count as a trailing comment in trigger file.
 - tracing/wprobe: Duplicate field string in wprobe_trigger_typecast_parse()
   to avoid modifying glob in place so tracing_log_err() logs the pristine
   command string.
 - tracing/wprobe: Remove leftover offset and adjust debug print in
   wprobe_trigger_print().
 - tracing/wprobe: Add set_wprobe and clear_wprobe triggers to readme_msg
   in trace.c.
 - tracing/wprobe: Move variable declarations to function entry in
   trace_wprobe_show(), fix tab indentation in trace.c, and wrap lines
   exceeding 100 columns.
 - Documentation: Document BTF typecast in fetchargs and BTF struct
   offset resolution in set_wprobe trigger in wprobetrace.rst.
 - selftests/ftrace: Add explicit failure checks with fail helper across
   trigger-wprobe.tc, trigger-wprobe-btf-typecast.tc, and
   trigger-wprobe-btf-offset.tc.
 - selftests/ftrace: Declare fprobe README requirement in trigger tests
   and trigger file check in trigger-wprobe-syntax-errors.tc requires line.


Public branch
-------------
I will push this branch as topic/wprobe-v2 to my tree so that it
can be easily tested.

https://git.kernel.org/pub/scm/linux/kernel/git/mhiramat/linux.git/log/?h=topic/wprobe-v2

This is based on linux-trace tree's linux-trace/probes/fixes.

Usage
-----

The basic usage of this wprobe is similar to other probes;

  w:[GRP/][EVENT] [r|w|rw]@<ADDRESS|SYMBOL[+|-OFFS]>[:LEN] [FETCHARGS]

This defines a new wprobe event. For example, to trace jiffies update,
you can do;

 echo 'w:my_jiffies w@jiffies:8 value=+0($addr)' >> dynamic_events
 echo 1 > events/wprobes/my_jiffies/enable

Moreover, this can be combined with event trigger to trace the memory
access on slab objects. The trigger syntax is;

  set_wprobe:WPROBE_EVENT:FIELD[+|-OFFSET][:COUNT] [if FILTER]
  clear_wprobe:WPROBE_EVENT[:FIELD[+|-OFFSET][:COUNT]] [if FILTER]

set_wprobe sets WPROBE_EVENT's watch address on FIELD[+|-OFFSET].
clear_wprobe clears WPROBE_EVENT's watch address if it is set to
FIELD[+|-OFFSET]. If FIELD is omitted, forcibly clear the watch address
when trigger event is hit.

For example, trace the first 8 byte of the dentry data structure passed
to do_truncate() until it is deleted by dentry_kill().
(Note: all tracefs setup uses '>>' so that it does not kick do_truncate())

  # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' > dynamic_events

  # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
  # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger

  # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
  # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger

  # echo 1 >> events/fprobes/truncate/enable
  # echo 1 >> events/fprobes/dentry_kill/enable

  # echo aaa > /tmp/hoge
  # echo bbb > /tmp/hoge
  # echo ccc > /tmp/hoge
  # rm /tmp/hoge

Then, the trace data will show;

 # tracer: nop
 #
 # entries-in-buffer/entries-written: 32/32   #P:8
 #
 #                                _-----=> irqs-off/BH-disabled
 #                               / _----=> need-resched
 #                              | / _---=> hardirq/softirq
 #                              || / _--=> preempt-depth
 #                              ||| / _-=> migrate-disable
 #                              |||| /     delay
 #           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
 #              | |         |   |||||     |         |
               sh-107     [004] ...1.     9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
               sh-107     [004] ...1.     9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
               sh-107     [004] ...1.     9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
               sh-107     [004] .....     9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
               sh-107     [004] ...1.     9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
               sh-107     [004] ..Zff     9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] ..Zff     9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] ..Zff     9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] ..Zff     9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] .....     9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
               sh-107     [004] ...1.     9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
               sh-107     [004] ...1.     9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
               sh-107     [004] ...1.     9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
               rm-113     [005] ..Zff    10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] d.Z..    10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] d.Z..    10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
               rm-113     [005] ...1.    10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
               rm-113     [005] ...1.    10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
               sh-107     [004] ...1.    10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
               sh-107     [004] ...1.    10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
               sh-107     [004] ...1.    10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
               sh-107     [004] ...1.    10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
               sh-107     [004] ...1.    10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
               sh-107     [004] ...1.    10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618


Thank you,

---
base-commit: 0c4256196b3a105307e2235fbfd85e768bbcdd0f

Jinchao Wang (2):
      x86/hw_breakpoints: Make DR7 updates NMI safe
      x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API

Masami Hiramatsu (Google) (10):
      x86/mce: Fix hardware debug register corruption on task migration
      HWBP: Add modify_local_hw_breakpoint_addr() API
      tracing/wprobe: Add wprobe (watchpoint probe) trace event support
      x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires
      selftests: tracing: Add a basic testcase for wprobe
      selftests: tracing: Add syntax testcase for wprobe
      tracing/wprobe: Add set_wprobe and clear_wprobe event triggers
      selftests: tracing: Add wprobe trigger testcase
      tracing/wprobe: Support BTF typecast in fetchargs
      tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger


 Documentation/trace/index.rst                      |    1 
 Documentation/trace/wprobetrace.rst                |  200 +++
 arch/Kconfig                                       |   18 
 arch/x86/Kconfig                                   |    2 
 arch/x86/include/asm/debugreg.h                    |   67 +
 arch/x86/include/asm/hw_breakpoint.h               |    2 
 arch/x86/kernel/cpu/mce/core.c                     |   29 
 arch/x86/kernel/hw_breakpoint.c                    |  151 +-
 arch/x86/kernel/nmi.c                              |    7 
 arch/x86/kernel/traps.c                            |   10 
 include/linux/hw_breakpoint.h                      |    6 
 include/linux/trace_events.h                       |    3 
 kernel/events/hw_breakpoint.c                      |   48 +
 kernel/trace/Kconfig                               |   24 
 kernel/trace/Makefile                              |    1 
 kernel/trace/trace.c                               |   13 
 kernel/trace/trace.h                               |    6 
 kernel/trace/trace_events_trigger.c                |    2 
 kernel/trace/trace_probe.c                         |   36 
 kernel/trace/trace_probe.h                         |   22 
 kernel/trace/trace_wprobe.c                        | 1522 ++++++++++++++++++++
 tools/testing/selftests/ftrace/config              |    3 
 .../ftrace/test.d/dynevent/add_remove_wprobe.tc    |   63 +
 .../test.d/dynevent/wprobes_syntax_errors.tc       |   23 
 .../test.d/trigger/trigger-wprobe-btf-offset.tc    |   85 +
 .../test.d/trigger/trigger-wprobe-btf-typecast.tc  |   85 +
 .../test.d/trigger/trigger-wprobe-syntax-errors.tc |   37 
 .../ftrace/test.d/trigger/trigger-wprobe.tc        |  107 +
 28 files changed, 2476 insertions(+), 97 deletions(-)
 create mode 100644 Documentation/trace/wprobetrace.rst
 create mode 100644 kernel/trace/trace_wprobe.c
 create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc

--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
Re: [PATCH v15 00/12] tracing: wprobe: x86: Add wprobe for watchpoint
Posted by Jinchao Wang 2 weeks ago
On 9/7/2026 11:46 AM, Masami Hiramatsu (Google) wrote:
> Hi,
> 
> Here is the 15th version of the series for adding new wprobe (watch probe)
> which provides memory access tracing event. Moreover, this can be used
> via event trigger. Thus it can trace memory access on dynamically
> allocated objects too.
> The previous version is here:
> 
>   https://lore.kernel.org/all/178810001186.64882.2161016469449127450.stgit@devnote2/
> 
> In this version, previous bugfix patches for BTF kflag resolution and
> RCU-protected kprobe blacklist were sent to Linus via probe/fixes, reducing
> the series to 12 patches. This version addresses feedbacks from Sashiko on
> v14[1], and more local fixes such as x86 debug register manipulation, validates
> watched address and natural alignment, tracks missed events on SMP update
> failures(*), prevents in-place command string modification during trigger
> parsing, adds explicit failure checks in selftests, and documents BTF
> typecast and struct offset syntax. I also added Assited-by: tags for this
> update.
> 
> [1] https://sashiko.dev/#/patchset/178810001186.64882.2161016469449127450.stgit%40devnote2
> (*) currently it just increase a missed counter.
> 
> Detailed changes are here.
> 
> Changes in v15:
>  - x86/mce: Move local_db_save() to entry of exc_machine_check_kernel()
>    and restore it on early return from mce_check_crashing_cpu() to
>    prevent nested #DB on watched variables.
>  - x86/hw_breakpoints: Always preserve DR7_FIXED_1 in
>    arch_install_hw_breakpoint(), arch_uninstall_hw_breakpoint(), and
>    hw_breakpoint_restore() for architectural compliance.
>  - x86/hw_breakpoints: Wrap local_db_save() in a cpu_dr7_seq loop to
>    detect NMI breakpoint updates and ensure DR7 is disabled before entry.
>  - x86/hw_breakpoints: In local_db_restore(), restore DR7_FIXED_1 if
>    saved dr7 is 0 and an NMI updated breakpoints, preserving KVM guest
>    isolation.
>  - x86/hw_breakpoints: Always write "val | DR7_FIXED_1" to hardware DR7
>    in local_db_restore() for architectural compliance.
>  - x86/hw_breakpoints: Avoid modifying saved dr7 in-place across retry
>    loops.
>  - x86/hw_breakpoints: Temporarily clear breakpoint enable bit in DR7
>    before updating debug address register and re-enable it afterward to
>    comply with x86 specification and avoid spurious #DBs.
>  - HWBP: Annotate counter_arch_bp(bp)->address and bp->attr.bp_addr with
>    WRITE_ONCE() in both update and rollback paths to prevent data races
>    with concurrent readers (e.g. NMI handlers).
>  - tracing/wprobe: Introduce trace_wprobe_is_valid_addr() to validate
>    target address and natural alignment for wprobe length during event
>    creation and trigger setup.
>  - tracing/wprobe: Initialize irq_work and work immediately after
>    allocation in alloc_trace_wprobe() and check tw->work.func in
>    free_trace_wprobe() to prevent WARN_ON(!work->func) on early failures.
>  - tracing/wprobe: Record SMP local update failures in tw->missed counter
>    and expose missed event count as a trailing comment in trigger file.
>  - tracing/wprobe: Duplicate field string in wprobe_trigger_typecast_parse()
>    to avoid modifying glob in place so tracing_log_err() logs the pristine
>    command string.
>  - tracing/wprobe: Remove leftover offset and adjust debug print in
>    wprobe_trigger_print().
>  - tracing/wprobe: Add set_wprobe and clear_wprobe triggers to readme_msg
>    in trace.c.
>  - tracing/wprobe: Move variable declarations to function entry in
>    trace_wprobe_show(), fix tab indentation in trace.c, and wrap lines
>    exceeding 100 columns.
>  - Documentation: Document BTF typecast in fetchargs and BTF struct
>    offset resolution in set_wprobe trigger in wprobetrace.rst.
>  - selftests/ftrace: Add explicit failure checks with fail helper across
>    trigger-wprobe.tc, trigger-wprobe-btf-typecast.tc, and
>    trigger-wprobe-btf-offset.tc.
>  - selftests/ftrace: Declare fprobe README requirement in trigger tests
>    and trigger file check in trigger-wprobe-syntax-errors.tc requires line.
> 
> 
> Public branch
> -------------
> I will push this branch as topic/wprobe-v2 to my tree so that it
> can be easily tested.
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/mhiramat/linux.git/log/?h=topic/wprobe-v2
> 
> This is based on linux-trace tree's linux-trace/probes/fixes.
> 
> Usage
> -----
> 
> The basic usage of this wprobe is similar to other probes;
> 
>   w:[GRP/][EVENT] [r|w|rw]@<ADDRESS|SYMBOL[+|-OFFS]>[:LEN] [FETCHARGS]
> 
> This defines a new wprobe event. For example, to trace jiffies update,
> you can do;
> 
>  echo 'w:my_jiffies w@jiffies:8 value=+0($addr)' >> dynamic_events
>  echo 1 > events/wprobes/my_jiffies/enable
> 
> Moreover, this can be combined with event trigger to trace the memory
> access on slab objects. The trigger syntax is;
> 
>   set_wprobe:WPROBE_EVENT:FIELD[+|-OFFSET][:COUNT] [if FILTER]
>   clear_wprobe:WPROBE_EVENT[:FIELD[+|-OFFSET][:COUNT]] [if FILTER]
> 
> set_wprobe sets WPROBE_EVENT's watch address on FIELD[+|-OFFSET].
> clear_wprobe clears WPROBE_EVENT's watch address if it is set to
> FIELD[+|-OFFSET]. If FIELD is omitted, forcibly clear the watch address
> when trigger event is hit.
> 
> For example, trace the first 8 byte of the dentry data structure passed
> to do_truncate() until it is deleted by dentry_kill().
> (Note: all tracefs setup uses '>>' so that it does not kick do_truncate())
> 
>   # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' > dynamic_events
> 
>   # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
>   # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
> 
>   # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
>   # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger
> 
>   # echo 1 >> events/fprobes/truncate/enable
>   # echo 1 >> events/fprobes/dentry_kill/enable
> 
>   # echo aaa > /tmp/hoge
>   # echo bbb > /tmp/hoge
>   # echo ccc > /tmp/hoge
>   # rm /tmp/hoge
> 
> Then, the trace data will show;
> 
>  # tracer: nop
>  #
>  # entries-in-buffer/entries-written: 32/32   #P:8
>  #
>  #                                _-----=> irqs-off/BH-disabled
>  #                               / _----=> need-resched
>  #                              | / _---=> hardirq/softirq
>  #                              || / _--=> preempt-depth
>  #                              ||| / _-=> migrate-disable
>  #                              |||| /     delay
>  #           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
>  #              | |         |   |||||     |         |
>                sh-107     [004] ...1.     9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
>                sh-107     [004] ...1.     9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
>                sh-107     [004] ...1.     9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
>                sh-107     [004] .....     9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
>                sh-107     [004] ...1.     9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
>                sh-107     [004] ..Zff     9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
>                sh-107     [004] ..Zff     9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
>                sh-107     [004] ..Zff     9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
>                sh-107     [004] ..Zff     9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
>                sh-107     [004] .....     9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
>                sh-107     [004] ...1.     9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
>                sh-107     [004] ...1.     9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
>                sh-107     [004] ...1.     9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
>                rm-113     [005] ..Zff    10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] ..Zff    10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] d.Z..    10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
>                rm-113     [005] d.Z..    10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
>                rm-113     [005] ...1.    10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
>                rm-113     [005] ...1.    10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
>                sh-107     [004] ...1.    10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
>                sh-107     [004] ...1.    10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
>                sh-107     [004] ...1.    10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
>                sh-107     [004] ...1.    10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
>                sh-107     [004] ...1.    10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
>                sh-107     [004] ...1.    10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
> 
> 
> Thank you,
> 
> ---
> base-commit: 0c4256196b3a105307e2235fbfd85e768bbcdd0f
> 
> Jinchao Wang (2):
>       x86/hw_breakpoints: Make DR7 updates NMI safe
>       x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API
> 
> Masami Hiramatsu (Google) (10):
>       x86/mce: Fix hardware debug register corruption on task migration
>       HWBP: Add modify_local_hw_breakpoint_addr() API
>       tracing/wprobe: Add wprobe (watchpoint probe) trace event support
>       x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires
>       selftests: tracing: Add a basic testcase for wprobe
>       selftests: tracing: Add syntax testcase for wprobe
>       tracing/wprobe: Add set_wprobe and clear_wprobe event triggers
>       selftests: tracing: Add wprobe trigger testcase
>       tracing/wprobe: Support BTF typecast in fetchargs
>       tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger
> 
> 
>  Documentation/trace/index.rst                      |    1 
>  Documentation/trace/wprobetrace.rst                |  200 +++
>  arch/Kconfig                                       |   18 
>  arch/x86/Kconfig                                   |    2 
>  arch/x86/include/asm/debugreg.h                    |   67 +
>  arch/x86/include/asm/hw_breakpoint.h               |    2 
>  arch/x86/kernel/cpu/mce/core.c                     |   29 
>  arch/x86/kernel/hw_breakpoint.c                    |  151 +-
>  arch/x86/kernel/nmi.c                              |    7 
>  arch/x86/kernel/traps.c                            |   10 
>  include/linux/hw_breakpoint.h                      |    6 
>  include/linux/trace_events.h                       |    3 
>  kernel/events/hw_breakpoint.c                      |   48 +
>  kernel/trace/Kconfig                               |   24 
>  kernel/trace/Makefile                              |    1 
>  kernel/trace/trace.c                               |   13 
>  kernel/trace/trace.h                               |    6 
>  kernel/trace/trace_events_trigger.c                |    2 
>  kernel/trace/trace_probe.c                         |   36 
>  kernel/trace/trace_probe.h                         |   22 
>  kernel/trace/trace_wprobe.c                        | 1522 ++++++++++++++++++++
>  tools/testing/selftests/ftrace/config              |    3 
>  .../ftrace/test.d/dynevent/add_remove_wprobe.tc    |   63 +
>  .../test.d/dynevent/wprobes_syntax_errors.tc       |   23 
>  .../test.d/trigger/trigger-wprobe-btf-offset.tc    |   85 +
>  .../test.d/trigger/trigger-wprobe-btf-typecast.tc  |   85 +
>  .../test.d/trigger/trigger-wprobe-syntax-errors.tc |   37 
>  .../ftrace/test.d/trigger/trigger-wprobe.tc        |  107 +
>  28 files changed, 2476 insertions(+), 97 deletions(-)
>  create mode 100644 Documentation/trace/wprobetrace.rst
>  create mode 100644 kernel/trace/trace_wprobe.c
>  create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
>  create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
>  create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
>  create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
>  create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
>  create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
> 
> --
> Masami Hiramatsu (Google) <mhiramat@kernel.org>
Hi Masami,

I see the watch address update currently goes irq_work -> workqueue ->
on_each_cpu().  I would like to discuss that path.

The race window wprobe may be used to watch can be very narrow --
sometimes it lives entirely inside a single function -- and the trigger
does not always run in in_task() context.  The current path defers the
update to a work item, so by the time the debug registers are updated the
window may already be gone.

Now that we have modify_local_hw_breakpoint_addr(), the trigger could arm
the local CPU directly and use smp_call_function_single_async() for the
remote ones.  That should improve the overlap between the watch and the window.

What do you think?

Thanks,
Jinchao
Re: [PATCH v15 00/12] tracing: wprobe: x86: Add wprobe for watchpoint
Posted by Masami Hiramatsu (Google) 1 week, 6 days ago
On Fri, 11 Sep 2026 15:27:41 +0800
Jinchao Wang <wangjinchao600@gmail.com> wrote:

> Hi Masami,
> 
> I see the watch address update currently goes irq_work -> workqueue ->
> on_each_cpu().  I would like to discuss that path.
> 
> The race window wprobe may be used to watch can be very narrow --
> sometimes it lives entirely inside a single function -- and the trigger
> does not always run in in_task() context.  The current path defers the
> update to a work item, so by the time the debug registers are updated the
> window may already be gone.
> 
> Now that we have modify_local_hw_breakpoint_addr(), the trigger could arm
> the local CPU directly and use smp_call_function_single_async() for the
> remote ones.  That should improve the overlap between the watch and the window.

Hm, that is a good idea. Maybe we can check the current context and if
possible call modify_local_hw_breakcpoint_addr() directly for current
CPU core, and kick irq_work for other cores.
Does this make sense for you?

Thanks,

> 
> What do you think?
> 
> Thanks,
> Jinchao


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>
Re: [PATCH v15 00/12] tracing: wprobe: x86: Add wprobe for watchpoint
Posted by Jinchao Wang 1 week, 4 days ago
On 9/12/2026 9:36 PM, Masami Hiramatsu (Google) wrote:
> On Fri, 11 Sep 2026 15:27:41 +0800
> Jinchao Wang <wangjinchao600@gmail.com> wrote:
> 
>> Hi Masami,
>>
>> I see the watch address update currently goes irq_work -> workqueue ->
>> on_each_cpu().  I would like to discuss that path.
>>
>> The race window wprobe may be used to watch can be very narrow --
>> sometimes it lives entirely inside a single function -- and the trigger
>> does not always run in in_task() context.  The current path defers the
>> update to a work item, so by the time the debug registers are updated the
>> window may already be gone.
>>
>> Now that we have modify_local_hw_breakpoint_addr(), the trigger could arm
>> the local CPU directly and use smp_call_function_single_async() for the
>> remote ones.  That should improve the overlap between the watch and the window.
> 
> Hm, that is a good idea. Maybe we can check the current context and if
> possible call modify_local_hw_breakcpoint_addr() directly for current
> CPU core, and kick irq_work for other cores.
> Does this make sense for you?

Yes, that makes sense.

> 
> Thanks,
> 
>>
>> What do you think?
>>
>> Thanks,
>> Jinchao
> 
>