From nobody Tue Apr 7 22:01:46 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0228B40DFD5 for ; Wed, 11 Mar 2026 13:02:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234151; cv=none; b=VAngj/Nut9Ta30VmPInIyvv7pZfGy7pt7i6vyfn/fcroZzmvnp0fhAUlHbEIB2UwJ3neXNX9978fFFB8C8LHWU1gOymY2ieR1EaiAEDoP422UMcYbxWP2jaKZDUnNHdOMTRje49dM1bQd8zLavoN5notk/p44qku7d5/hKkfE1g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234151; c=relaxed/simple; bh=alxhS5zNt5yhZDlpWUot0Dy3jmoiPvwWrfIvY63/tcE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SUd6oAsTXSRG3ooKevUBFb7x6EnE8zHwEftC79P02xZHqUEWqCAOikixBvvwcu6y22nqadJ2/yDGbWHcOQlyUZYDiGByIOvjA16avnRCI7QyKF+CHpujdfU8EiM/jP9fD95sK6otGU2Nj4P8434HVauRGkO4KlCDtLApOmD+muc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Tt2zLps+; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Tt2zLps+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773234149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wDwbdAOWQrkyyH+jPzV483o27gJ9IQVrrz9O2kAMCZY=; b=Tt2zLps+R9ZZhFEfQhnnq22hI7gsMqHD617VThf0HUtrg7/dT3zjA8x2CfPHkkr7rt2ii/ 9kLfVrllwOPmXsq0KLLoMcrBpUaSIT4LJEOkhtOTy1mdYVn6zAn/5nEVreATdba+FZH2Dc ZXn0sd88LpHz5cn+GqTWiiA8rLKfsWc= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-663-pe716fNZPJqrBdtiFVtqJQ-1; Wed, 11 Mar 2026 09:02:23 -0400 X-MC-Unique: pe716fNZPJqrBdtiFVtqJQ-1 X-Mimecast-MFC-AGG-ID: pe716fNZPJqrBdtiFVtqJQ_1773234140 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F26DA19560B5; Wed, 11 Mar 2026 13:02:18 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.22.80.206]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 63EFE3000223; Wed, 11 Mar 2026 13:02:12 +0000 (UTC) From: Wander Lairson Costa To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Masami Hiramatsu , Mathieu Desnoyers , Andrew Morton , linux-kernel@vger.kernel.org (open list:SCHEDULER), linux-trace-kernel@vger.kernel.org (open list:TRACING) Cc: acme@kernel.org, williams@redhat.com, gmonaco@redhat.com, Wander Lairson Costa Subject: [PATCH v3 1/4] tracing/preemptirq: Optimize preempt_disable/enable() tracepoint overhead Date: Wed, 11 Mar 2026 09:50:15 -0300 Message-ID: <20260311125021.197638-2-wander@redhat.com> In-Reply-To: <20260311125021.197638-1-wander@redhat.com> References: <20260311125021.197638-1-wander@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" When CONFIG_TRACE_PREEMPT_TOGGLE is enabled, preempt_count_add() and preempt_count_sub() become external function calls (defined in kernel/sched/core.c) rather than inlined operations. These functions also perform preempt_count() checks and call trace_preempt_on/off() unconditionally, even when no tracing consumer is active. Reduce this overhead by splitting the #if logic in preempt.h into three cases. When CONFIG_DEBUG_PREEMPT or CONFIG_PREEMPT_TRACER is set, keep external function calls because DEBUG_PREEMPT needs runtime validation checks, and PREEMPT_TRACER needs the preemptoff latency tracer hooks (tracer_preempt_on/off, called via trace_preempt_on/off). When CONFIG_TRACE_PREEMPT_TOGGLE alone is set, provide new inline versions of preempt_count_add/sub() that check the tracepoint static key via the __preempt_trace_enabled() macro before calling into the tracing path. The macro evaluates to true when the preempt_enable or preempt_disable tracepoint has subscribers AND the preempt count equals val (indicating the first preempt disable or last preempt enable), preserving the original preempt_latency_start/stop semantics. When none of the above are set, use pure inline macros with no tracing overhead. The preempt_count_dec_and_test() macro is refactored out of the three-way #if into a separate block shared by the first two cases, since both need it to call the (potentially inline) preempt_count_sub() before checking should_resched(). The inline path calls thin __trace_preempt_on/off() wrappers (added in trace_preemptirq.c) that invoke trace_preempt_on/off(), keeping the full tracepoint machinery out of the inline code. The #include is placed inside the CONFIG_TRACE_PREEMPT_TOGGLE block rather than at the top of the file to avoid a circular include dependency on architectures where asm/irqflags.h includes linux/preempt.h (e.g. m68k): preempt.h -> tracepoint-defs.h -> static_key.h -> jump_label.h -> atomic.h -> irqflags.h -> asm/irqflags.h -> preempt.h (guarded) If the include were at the top, this chain would be traversed before hardirq_count() is defined (at line 110), causing a build failure on m68k. Placing it inside the #elif block ensures it is only pulled in when CONFIG_TRACE_PREEMPT_TOGGLE is enabled and avoids the cycle for configurations that do not select it. In core.c, narrow the compilation guard for the external preempt_count_add/sub() from CONFIG_DEBUG_PREEMPT || CONFIG_TRACE_PREEMPT_TOGGLE to CONFIG_DEBUG_PREEMPT || CONFIG_PREEMPT_TRACER, since CONFIG_TRACE_PREEMPT_TOGGLE is now handled inline. Signed-off-by: Wander Lairson Costa Suggested-by: Steven Rostedt --- include/linux/preempt.h | 49 +++++++++++++++++++++++++++++++-- kernel/sched/core.c | 2 +- kernel/trace/trace_preemptirq.c | 19 +++++++++++++ 3 files changed, 66 insertions(+), 4 deletions(-) diff --git a/include/linux/preempt.h b/include/linux/preempt.h index d964f965c8ffc..f59a92f930d81 100644 --- a/include/linux/preempt.h +++ b/include/linux/preempt.h @@ -189,17 +189,60 @@ static __always_inline unsigned char interrupt_contex= t_level(void) */ #define in_atomic_preempt_off() (preempt_count() !=3D PREEMPT_DISABLE_OFFS= ET) =20 -#if defined(CONFIG_DEBUG_PREEMPT) || defined(CONFIG_TRACE_PREEMPT_TOGGLE) +#if defined(CONFIG_DEBUG_PREEMPT) || defined(CONFIG_PREEMPT_TRACER) extern void preempt_count_add(int val); extern void preempt_count_sub(int val); -#define preempt_count_dec_and_test() \ - ({ preempt_count_sub(1); should_resched(0); }) +#elif defined(CONFIG_TRACE_PREEMPT_TOGGLE) +/* + * Avoid the circular dependency on architectures where asm/irqflags.h + * includes linux/preempt.h (e.g. m68k): + * + * preempt.h <--------------------+ + * tracepoint-defs.h | + * static_key.h | + * jump_label.h | + * atomic.h | + * irqflags.h | + * asm/irqflags.h | + * preempt.h --------------+ + */ +#include + +extern void __trace_preempt_on(void); +extern void __trace_preempt_off(void); + +DECLARE_TRACEPOINT(preempt_enable); +DECLARE_TRACEPOINT(preempt_disable); + +#define __preempt_trace_enabled(type, val) \ + (tracepoint_enabled(preempt_##type) && preempt_count() =3D=3D (val)) + +static __always_inline void preempt_count_add(int val) +{ + __preempt_count_add(val); + + if (__preempt_trace_enabled(disable, val)) + __trace_preempt_off(); +} + +static __always_inline void preempt_count_sub(int val) +{ + if (__preempt_trace_enabled(enable, val)) + __trace_preempt_on(); + + __preempt_count_sub(val); +} #else #define preempt_count_add(val) __preempt_count_add(val) #define preempt_count_sub(val) __preempt_count_sub(val) #define preempt_count_dec_and_test() __preempt_count_dec_and_test() #endif =20 +#if defined(CONFIG_DEBUG_PREEMPT) || defined(CONFIG_TRACE_PREEMPT_TOGGLE) +#define preempt_count_dec_and_test() \ + ({ preempt_count_sub(1); should_resched(0); }) +#endif + #define __preempt_count_inc() __preempt_count_add(1) #define __preempt_count_dec() __preempt_count_sub(1) =20 diff --git a/kernel/sched/core.c b/kernel/sched/core.c index b7f77c165a6e0..125e5d71d1bd3 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5733,7 +5733,7 @@ static inline void sched_tick_stop(int cpu) { } #endif /* !CONFIG_NO_HZ_FULL */ =20 #if defined(CONFIG_PREEMPTION) && (defined(CONFIG_DEBUG_PREEMPT) || \ - defined(CONFIG_TRACE_PREEMPT_TOGGLE)) + defined(CONFIG_PREEMPT_TRACER)) /* * If the value passed in is equal to the current preempt count * then we just disabled preemption. Start timing the latency. diff --git a/kernel/trace/trace_preemptirq.c b/kernel/trace/trace_preemptir= q.c index 0c42b15c38004..9f098fcb28012 100644 --- a/kernel/trace/trace_preemptirq.c +++ b/kernel/trace/trace_preemptirq.c @@ -115,6 +115,25 @@ NOKPROBE_SYMBOL(trace_hardirqs_off); =20 #ifdef CONFIG_TRACE_PREEMPT_TOGGLE =20 +#if !defined(CONFIG_DEBUG_PREEMPT) && !defined(CONFIG_PREEMPT_TRACER) +EXPORT_TRACEPOINT_SYMBOL(preempt_disable); +EXPORT_TRACEPOINT_SYMBOL(preempt_enable); + +void __trace_preempt_on(void) +{ + trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip()); +} +EXPORT_SYMBOL(__trace_preempt_on); +NOKPROBE_SYMBOL(__trace_preempt_on); + +void __trace_preempt_off(void) +{ + trace_preempt_off(CALLER_ADDR0, get_lock_parent_ip()); +} +EXPORT_SYMBOL(__trace_preempt_off); +NOKPROBE_SYMBOL(__trace_preempt_off); +#endif /* !CONFIG_DEBUG_PREEMPT */ + void trace_preempt_on(unsigned long a0, unsigned long a1) { trace(preempt_enable, TP_ARGS(a0, a1)); --=20 2.53.0 From nobody Tue Apr 7 22:01:46 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F37FC2D8771 for ; Wed, 11 Mar 2026 13:02:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234155; cv=none; b=uwNfWNAC1BW2TmiivdtiAgA4MPLzmPnOT+B3SuH91bkeQqySDpwfKyHO7Ny2SGlbTCGdby3XaZ/MWIeJ3Ilxn4eYyLK6E52/7dpZc8scrpO0fQ/aOclZ/6QOHp2fslBbrIKLb0L28xzvDDO/ZrAqiDHhuyMsU9rE5s6y5fZsDWo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234155; c=relaxed/simple; bh=UrUp6nfMg0UsSBxqAU/KgQgWt9eB0jo6UHftMfqrkBU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AnHWO8+5nUOcuQIIsxLlD3O1k8lsMVjpqPHy70Upgkbl/IJK3+Mk04LOuHAuYc2AfvZFm8D5Jvl0OnCzHt/DO7Z/hWd8Pxz4c/rwKlZ8MzGSV6LT4iW1a0EBGKFbYEekMVwK18A2j7NWduzHPhdSiKT0k/VOjgm5ztPhdgaQyWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YyhWuFU0; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YyhWuFU0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773234152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L0p430872sznMzJxH8mT2+q8E3XDrCfM5KcPZiV1a8U=; b=YyhWuFU0KNmlT+NPn/NN+WEoNyhyeKkSp0wGSkv13Qt9ws2BbM0f+is/PiCMAaau0ypq+/ SzjJmUbHNSMcme5pXyt1CD2Mz2NJeyrDqDlWx3eNe9j3U2tslqcN9vZ9mdHW40cXZqls34 CeyBg+mIV9mw6g3UuE2lVUCIW3HNvnA= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-578-lUpYh_5pNlOX0qZ1thti8Q-1; Wed, 11 Mar 2026 09:02:28 -0400 X-MC-Unique: lUpYh_5pNlOX0qZ1thti8Q-1 X-Mimecast-MFC-AGG-ID: lUpYh_5pNlOX0qZ1thti8Q_1773234146 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 132EC19560B5; Wed, 11 Mar 2026 13:02:26 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.22.80.206]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D3C2130002D2; Wed, 11 Mar 2026 13:02:19 +0000 (UTC) From: Wander Lairson Costa To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Masami Hiramatsu , Mathieu Desnoyers , Andrew Morton , linux-kernel@vger.kernel.org (open list:SCHEDULER), linux-trace-kernel@vger.kernel.org (open list:TRACING) Cc: acme@kernel.org, williams@redhat.com, gmonaco@redhat.com, Wander Lairson Costa Subject: [PATCH v3 2/4] trace/preemptirq: make TRACE_PREEMPT_TOGGLE user-selectable Date: Wed, 11 Mar 2026 09:50:16 -0300 Message-ID: <20260311125021.197638-3-wander@redhat.com> In-Reply-To: <20260311125021.197638-1-wander@redhat.com> References: <20260311125021.197638-1-wander@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Make TRACE_PREEMPT_TOGGLE directly selectable so that preempt_enable/preempt_disable tracepoints can be enabled in production kernels without requiring the preemptoff latency tracer overhead. Signed-off-by: Wander Lairson Costa Suggested-by: Steven Rostedt --- kernel/trace/Kconfig | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 49de13cae4288..e007459ecf361 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -413,10 +413,10 @@ config STACK_TRACER Say N if unsure. =20 config TRACE_PREEMPT_TOGGLE - bool + bool "Preempt disable/enable tracing hooks" help - Enables hooks which will be called when preemption is first disabled, - and last enabled. + Enables hooks into preemption disable and enable paths for + tracing or latency measurement. =20 config IRQSOFF_TRACER bool "Interrupts-off Latency Tracer" --=20 2.53.0 From nobody Tue Apr 7 22:01:46 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 681D9285CA2 for ; Wed, 11 Mar 2026 13:02:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234164; cv=none; b=MVDaIXvYZrRv3FW8OFtxqd+bBwJ/dawXPIC0DSVtwFvA9VH529SZgG96r2weq8s0P3TEM0WqFFDeES/gsmHOIucWpvHCEpjPtfTNBXOX9fXTh/iNS5Wxd+xBw4aq/GpLH7IY2Ht8J3IV1Y6JKiLpNTWETdrFwIC5dgPo6AsgWDU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234164; c=relaxed/simple; bh=cOgWE7xPYdBb7ngV1nOTKdOodYtpLLNf1JLEVdngPx0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HtrDVxQ1gJtfZ38z7gXS/HFJB0upWvQZJ4rBNyFFGSXkmJaz0+qhzWUoMz6o5Bb/BcMksaUa/A5JHiReriL37uEZV34pMpOg/XEHqujR87O57KR98KbvSAPAEDuPvb5r1dhKX5yWJOMOkBRtWr6Gf97Ww3P3pctYYvEPUr8czcc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=dEYebbl1; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dEYebbl1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773234162; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zr3KKQRC6yYka+uWB2bxIN0vAy+Wc13bRkpoGNmLdig=; b=dEYebbl15zArNFTpyPii74eYXh81Baiajp4uUydbBwGm6Tnz0DPNJGtnOudcc82f0Jgq4D hsz7uBHzj5krcCiMC4rWq3VDlT0HOu+HPOaHSInyj9MT2T+29Jq28isilpXcEip4F0ylr0 jooQOPB86PqZPfKHB5T1qlg2NKVwZE0= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-686-QnUWSdx2O_ObujOxbUlKvg-1; Wed, 11 Mar 2026 09:02:35 -0400 X-MC-Unique: QnUWSdx2O_ObujOxbUlKvg-1 X-Mimecast-MFC-AGG-ID: QnUWSdx2O_ObujOxbUlKvg_1773234154 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AC901180044D; Wed, 11 Mar 2026 13:02:33 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.22.80.206]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id ECA293000223; Wed, 11 Mar 2026 13:02:26 +0000 (UTC) From: Wander Lairson Costa To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Masami Hiramatsu , Mathieu Desnoyers , Andrew Morton , linux-kernel@vger.kernel.org (open list:SCHEDULER), linux-trace-kernel@vger.kernel.org (open list:TRACING) Cc: acme@kernel.org, williams@redhat.com, gmonaco@redhat.com, Wander Lairson Costa Subject: [PATCH v3 3/4] trace/preemptirq: add TRACE_IRQFLAGS_TOGGLE Date: Wed, 11 Mar 2026 09:50:17 -0300 Message-ID: <20260311125021.197638-4-wander@redhat.com> In-Reply-To: <20260311125021.197638-1-wander@redhat.com> References: <20260311125021.197638-1-wander@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" The IRQ disable/enable tracepoints are currently gated behind TRACE_IRQFLAGS, a hidden config that cannot be selected directly by users. It is only enabled when selected by PROVE_LOCKING or IRQSOFF_TRACER, both of which carry overhead beyond what is needed for just the tracepoints. Introduce TRACE_IRQFLAGS_TOGGLE, a user-selectable config that enables the irq_disable and irq_enable tracepoints independently. It is mutually exclusive with TRACE_IRQFLAGS and mirrors how TRACE_PREEMPT_TOGGLE works for preemption tracepoints. Make this option depend on CONFIG_JUMP_LABEL to avoid a circular header dependency. Without TRACE_IRQFLAGS, irqflags.h must check the static key before invoking the tracepoint. Using tracepoint_enabled() for this check pulls in tracepoint_defs.h, which eventually includes atomic.h and cmpxchg.h, circling back to irqflags.h. Enforcing CONFIG_JUMP_LABEL allows the use of static_key_false() directly, avoiding the inclusion of the full tracepoint header chain and preventing the cycle. Signed-off-by: Wander Lairson Costa Suggested-by: Steven Rostedt --- kernel/trace/Kconfig | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index e007459ecf361..8bea77b5f1200 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -162,7 +162,7 @@ config RING_BUFFER_ALLOW_SWAP =20 config PREEMPTIRQ_TRACEPOINTS bool - depends on TRACE_PREEMPT_TOGGLE || TRACE_IRQFLAGS + depends on TRACE_PREEMPT_TOGGLE || TRACE_IRQFLAGS || TRACE_IRQFLAGS_TOGGLE select TRACING default y help @@ -418,6 +418,17 @@ config TRACE_PREEMPT_TOGGLE Enables hooks into preemption disable and enable paths for tracing or latency measurement. =20 +config TRACE_IRQFLAGS_TOGGLE + bool "IRQ disable/enable tracing hooks" + default n + depends on TRACE_IRQFLAGS_SUPPORT && JUMP_LABEL && !TRACE_IRQFLAGS + help + Enables hooks into IRQ disable and enable paths for tracing. + + This provides the irq_disable and irq_enable tracepoints + without pulling in the full TRACE_IRQFLAGS infrastructure + used by lockdep and the irqsoff latency tracer. + config IRQSOFF_TRACER bool "Interrupts-off Latency Tracer" default n --=20 2.53.0 From nobody Tue Apr 7 22:01:46 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C935640DFD5 for ; Wed, 11 Mar 2026 13:02:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234172; cv=none; b=cWFzKJrOJDBsAu+rPB//AQIadEpTpEoLprVHGjDaFOLpI8voIQ7qeBQgLdC9UqnI4pFoi3ZlmMyJGRC6NfnQqc955hOZ1eU6dvHXF5AuifQXGrdy3/oaZsCwt+yQfQ2Iw5CzycuFxaWNRxrhQwk5HeonLwlnC+5Rna0d6y9Svr0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773234172; c=relaxed/simple; bh=teOn68X+rakWXclwQ/xGoKsS+finLnzl7W/8Tq2j3iI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=o6lFlPT4Xgy9YG1q8goSicY8rWUDX0T9Wc0Cyeq1MaqKG4UfgKz9A1VBdrI/kGEX4vW28HdjrT/r0mpVFh2PEKWi3wQ2KZCy6VXlVRdvERLcIHzBr61HyR3CMp/KVgcDE2PsqlD1fKtRyQAdxKU8hQdqa/G7PNmQgUYQ6eX5+Tk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VX1zHi6E; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VX1zHi6E" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1773234170; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PG7jO0eAaiDEl9f2zxR5XHew37Lwbqb6vu7OXKvAPhs=; b=VX1zHi6EFRgsePpkOV/d2qp8utVZnjZzjuzLECvUx5aX52XsSwpToJ/B1SJmwMjRw+grgN Te0XiVNNsLzybdsPI9fPpK8ogtqPqFR0yX+nH4WBxF4cq976N5NKyh3c5clUqKPsUPExEj 1jrggpz+xB6UMW/3ZefGfAMnFpsP9XM= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-515-6iQr92ouPRuJFhOaAojBHg-1; Wed, 11 Mar 2026 09:02:45 -0400 X-MC-Unique: 6iQr92ouPRuJFhOaAojBHg-1 X-Mimecast-MFC-AGG-ID: 6iQr92ouPRuJFhOaAojBHg_1773234162 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 57930180035C; Wed, 11 Mar 2026 13:02:42 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.22.80.206]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 286E83000223; Wed, 11 Mar 2026 13:02:35 +0000 (UTC) From: Wander Lairson Costa To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Masami Hiramatsu , Mathieu Desnoyers , Andrew Morton , Wander Lairson Costa , linux-kernel@vger.kernel.org (open list), linux-trace-kernel@vger.kernel.org (open list:TRACING) Cc: acme@kernel.org, williams@redhat.com, gmonaco@redhat.com Subject: [PATCH v3 4/4] trace/preemptirq: Implement trace_irqflags hooks Date: Wed, 11 Mar 2026 09:50:18 -0300 Message-ID: <20260311125021.197638-5-wander@redhat.com> In-Reply-To: <20260311125021.197638-1-wander@redhat.com> References: <20260311125021.197638-1-wander@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" The previous commit introduced the CONFIG_TRACE_IRQFLAGS_TOGGLE symbol. This patch implements the actual infrastructure to allow tracing irq_disable and irq_enable events without pulling in the heavy CONFIG_TRACE_IRQFLAGS dependencies like lockdep or the irqsoff tracer. The implementation hooks into the local_irq_* macros in irqflags.h. Instead of using the heavy trace_hardirqs_on/off calls, it uses lightweight tracepoint_enabled() checks. If the tracepoint is enabled, it calls into specific wrapper functions in trace_preemptirq.c. These wrappers check the raw hardware state via raw_irqs_disabled() to filter out redundant events, such as disabling interrupts when they are already disabled. This approach is simpler than the full TRACE_IRQFLAGS method which requires maintaining a per-cpu software state variable. To support this, the tracepoint definitions are exposed under the new configuration. Additionally, a circular header dependency involving irqflags.h, tracepoint-defs.h, and atomic.h is resolved by moving the atomic.h inclusion to tracepoint.h, allowing irqflags.h to include tracepoint-defs.h safely. Signed-off-by: Wander Lairson Costa Suggested-by: Steven Rostedt --- include/linux/irqflags.h | 62 ++++++++++++++++++++++++++++++- include/linux/tracepoint-defs.h | 1 - include/linux/tracepoint.h | 1 + include/trace/events/preemptirq.h | 2 +- kernel/trace/trace_preemptirq.c | 49 ++++++++++++++++++++++++ 5 files changed, 112 insertions(+), 3 deletions(-) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index 57b074e0cfbbb..f40557bebd325 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -18,6 +18,19 @@ #include #include =20 +/* + * Avoid the circular dependency + * irqflags.h <-----------------+ + * tracepoint_defs.h | + * static_key.h | + * jump_label.h | + * atomic.h | + * cmpxchg.h ---------+ + */ +#ifdef CONFIG_TRACE_IRQFLAGS_TOGGLE +#include +#endif + struct task_struct; =20 /* Currently lockdep_softirqs_on/off is used only by lockdep */ @@ -232,7 +245,54 @@ extern void warn_bogus_irq_restore(void); } while (0) =20 =20 -#else /* !CONFIG_TRACE_IRQFLAGS */ +#elif defined(CONFIG_TRACE_IRQFLAGS_TOGGLE) /* !CONFIG_TRACE_IRQFLAGS */ + +DECLARE_TRACEPOINT(irq_enable); +DECLARE_TRACEPOINT(irq_disable); + +void trace_local_irq_enable(void); +void trace_local_irq_disable(void); +void trace_local_irq_save(unsigned long flags); +void trace_local_irq_restore(unsigned long flags); +void trace_safe_halt(void); + +#define local_irq_enable() \ + do { \ + if (tracepoint_enabled(irq_enable)) \ + trace_local_irq_enable(); \ + raw_local_irq_enable(); \ + } while (0) + +#define local_irq_disable() \ + do { \ + if (tracepoint_enabled(irq_disable)) \ + trace_local_irq_disable(); \ + else \ + raw_local_irq_disable(); \ + } while (0) + +#define local_irq_save(flags) \ + do { \ + raw_local_irq_save(flags); \ + if (tracepoint_enabled(irq_disable)) \ + trace_local_irq_save(flags); \ + } while (0) + +#define local_irq_restore(flags) \ + do { \ + if (tracepoint_enabled(irq_enable)) \ + trace_local_irq_restore(flags); \ + raw_local_irq_restore(flags); \ + } while (0) + +#define safe_halt() \ + do { \ + if (tracepoint_enabled(irq_enable)) \ + trace_safe_halt(); \ + raw_safe_halt(); \ + } while (0) + +#else /* !CONFIG_TRACE_IRQFLAGS_TOGGLE */ =20 #define local_irq_enable() do { raw_local_irq_enable(); } while (0) #define local_irq_disable() do { raw_local_irq_disable(); } while (0) diff --git a/include/linux/tracepoint-defs.h b/include/linux/tracepoint-def= s.h index aebf0571c736e..cb1f15a4e43f0 100644 --- a/include/linux/tracepoint-defs.h +++ b/include/linux/tracepoint-defs.h @@ -8,7 +8,6 @@ * trace_print_flags{_u64}. Otherwise linux/tracepoint.h should be used. */ =20 -#include #include =20 struct static_call_key; diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h index 22ca1c8b54f32..e7d8c5ca00c79 100644 --- a/include/linux/tracepoint.h +++ b/include/linux/tracepoint.h @@ -20,6 +20,7 @@ #include #include #include +#include =20 struct module; struct tracepoint; diff --git a/include/trace/events/preemptirq.h b/include/trace/events/preem= ptirq.h index f99562d2b496b..a607a6f4e29ca 100644 --- a/include/trace/events/preemptirq.h +++ b/include/trace/events/preemptirq.h @@ -32,7 +32,7 @@ DECLARE_EVENT_CLASS(preemptirq_template, (void *)((unsigned long)(_stext) + __entry->parent_offs)) ); =20 -#ifdef CONFIG_TRACE_IRQFLAGS +#if defined(CONFIG_TRACE_IRQFLAGS) || defined(CONFIG_TRACE_IRQFLAGS_TOGGLE) DEFINE_EVENT(preemptirq_template, irq_disable, TP_PROTO(unsigned long ip, unsigned long parent_ip), TP_ARGS(ip, parent_ip)); diff --git a/kernel/trace/trace_preemptirq.c b/kernel/trace/trace_preemptir= q.c index 9f098fcb28012..0f32da96d2f01 100644 --- a/kernel/trace/trace_preemptirq.c +++ b/kernel/trace/trace_preemptirq.c @@ -111,8 +111,57 @@ void trace_hardirqs_off(void) } EXPORT_SYMBOL(trace_hardirqs_off); NOKPROBE_SYMBOL(trace_hardirqs_off); + #endif /* CONFIG_TRACE_IRQFLAGS */ =20 +#ifdef CONFIG_TRACE_IRQFLAGS_TOGGLE +EXPORT_TRACEPOINT_SYMBOL(irq_disable); +EXPORT_TRACEPOINT_SYMBOL(irq_enable); + +void trace_local_irq_enable(void) +{ + if (raw_irqs_disabled()) + trace(irq_enable, TP_ARGS(CALLER_ADDR0, CALLER_ADDR1)); +} +EXPORT_SYMBOL(trace_local_irq_enable); +NOKPROBE_SYMBOL(trace_local_irq_enable); + +void trace_local_irq_disable(void) +{ + const bool was_disabled =3D raw_irqs_disabled(); + + raw_local_irq_disable(); + if (!was_disabled) + trace(irq_disable, TP_ARGS(CALLER_ADDR0, CALLER_ADDR1)); +} +EXPORT_SYMBOL(trace_local_irq_disable); +NOKPROBE_SYMBOL(trace_local_irq_disable); + +void trace_local_irq_save(unsigned long flags) +{ + if (!raw_irqs_disabled_flags(flags)) + trace(irq_disable, TP_ARGS(CALLER_ADDR0, CALLER_ADDR1)); +} +EXPORT_SYMBOL(trace_local_irq_save); +NOKPROBE_SYMBOL(trace_local_irq_save); + +void trace_local_irq_restore(unsigned long flags) +{ + if (!raw_irqs_disabled_flags(flags) && raw_irqs_disabled()) + trace(irq_enable, TP_ARGS(CALLER_ADDR0, CALLER_ADDR1)); +} +EXPORT_SYMBOL(trace_local_irq_restore); +NOKPROBE_SYMBOL(trace_local_irq_restore); + +void trace_safe_halt(void) +{ + if (raw_irqs_disabled()) + trace(irq_enable, TP_ARGS(CALLER_ADDR0, CALLER_ADDR1)); +} +EXPORT_SYMBOL(trace_safe_halt); +NOKPROBE_SYMBOL(trace_safe_halt); +#endif /* CONFIG_TRACE_IRQFLAGS_TOGGLE */ + #ifdef CONFIG_TRACE_PREEMPT_TOGGLE =20 #if !defined(CONFIG_DEBUG_PREEMPT) && !defined(CONFIG_PREEMPT_TRACER) --=20 2.53.0