From nobody Tue Sep 29 13:20:17 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A87EE41DDF3; Fri, 7 Aug 2026 16:00:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786118408; cv=none; b=LK6Faxhm8XhkQHZTcwhFe9c2ZXKOfxsVhDdWWhtWyRFxJNSxpE7PmRkxAl/GyOOmnDpbe2lV9K3gcHlhhHzIKkoGmH5A/qclJcJi0PBlXFsb1MtOWj71BL1BNQv+DKPAI+wqPuYwPMAUt5POREYvZ7lpt3yOK7CyjqPmn0gph7Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786118408; c=relaxed/simple; bh=94dUDfn6XKySJrH+1Fq2kXR05A4fe8v69Iav+QD0rHk=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=eKFmYmDjQ7iCSBTWdlDfw/uJEQ8bUPQWI8egoMo4RSscNkN1zo6Pv5+6MC0NKne9DfeKXfF9DFI/wa4LQ+Sm2WExAx3mMRnaqt/ViwAV+xscLMMv749lWqWij9AA8qpYVcWUgyOvIFyWZRpZN0fVS1Qf0NoQQK5FmN8jHg5euFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=YJCGSdVy; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=psC7b7W9; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="YJCGSdVy"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="psC7b7W9" Date: Fri, 07 Aug 2026 16:00:03 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1786118405; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DMJRylM9hNlEFvhYyb6jamM22NHnq3S+Zcm+xRLTy+8=; b=YJCGSdVy6TAv6qFJ2kYgmryzMQ5eveGcndXqm6ne12PFwpWQQy2SYFS68OJxTv8iS8Ln4y yY+ePYzij0BL1PXB4zth8OtEZQB5s1cz1FJ3BMxWuUBfIjlafIcemQ2XRbruYsSUcVpT7l vP6PtCyP+w9g2GkJFaraXLHj273AuOzAJiwhn75L1++kuctvElXOfxTQzsj+Yo20MQizV9 kUBFcXvnHYv1sZDlJ0LDcdy9ESH0lwJrzoUU6vKhi0bqFgYi0wRobgyldVDzWsS5Soh8im s26DdAJWKB1TsxEiehBxqkZ9O8cPrkGsQQ+Ev/eWcwNTrv9zhoDfNLXyyASxEg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1786118405; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DMJRylM9hNlEFvhYyb6jamM22NHnq3S+Zcm+xRLTy+8=; b=psC7b7W9hVhOmZQKxrdFAmXJCBODZ9KcHmXwm0zTETj5j0JolZFPOMh0e2NWwe4EZEUsue LMsf6ZSgR+cJ1wDQ== From: "tip-bot2 for Dmitry Ilvokhin" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: locking/core] locking/qspinlock: Add contended_release tracepoint Cc: Dmitry Ilvokhin , "Peter Zijlstra (Intel)" , Juergen Gross , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <0d998e22a0c595f670cfc6725bb683323aced5cb.1785778551.git.d@ilvokhin.com> References: <0d998e22a0c595f670cfc6725bb683323aced5cb.1785778551.git.d@ilvokhin.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178611840305.708.11230361838398685213.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the locking/core branch of tip: Commit-ID: f7e2cb6d495aa1a88368650970a230b45870859b Gitweb: https://git.kernel.org/tip/f7e2cb6d495aa1a88368650970a230b45= 870859b Author: Dmitry Ilvokhin AuthorDate: Tue, 04 Aug 2026 07:15:43=20 Committer: Peter Zijlstra CommitterDate: Fri, 07 Aug 2026 17:58:10 +02:00 locking/qspinlock: Add contended_release tracepoint Unlike mutex and rw_semaphore, qspinlock has no owner field, so "perf lock contention --lock-owner" cannot attribute a contended spinlock to its holder. The waiter-side contention_begin event records that a spinlock is contended, but not by whom. Firing contended_release in the holder's context at unlock is the only way to capture the holder of a contended spinlock. Combine the contention check, trace call and release in an out-of-line queued_spin_release_traced() so the compiler need not preserve the lock pointer in a callee-saved register across the call. The check in queued_spin_unlock() is paid on every unlock, even while the tracepoint is disabled: a static-branch NOP on x86_64, and a few more instructions to manage a stack frame elsewhere. Gate it behind CONFIG_QUEUED_SPINLOCKS_TRACE_CONTENDED_RELEASE (default n) so nobody pays for a tracepoint they do not use. Sleeping locks fire contended_release regardless. On x86 this generic path is used only with PARAVIRT_SPINLOCKS=3Dn (e.g. defconfig). PARAVIRT_SPINLOCKS=3Dy kernels keep the paravirt static_call unlock and are wired up separately. All below are with the QUEUED_SPINLOCKS_TRACE_CONTENDED_RELEASE option enabled. _raw_spin_unlock(), x86_64 defconfig, GCC 11, tracepoint compiled in but disabled. The unlock is the single 'movb'. The only instruction added to the executed path is the 2-byte static-branch NOP. The CALL to the traced helper and the JMP back are emitted out of line and are reached only once the static branch is patched on: endbr64 ; 4 bytes xchg %ax,%ax ; 2 static-branch NOP ; (added) movb $0x0,(%rdi) ; 3 unlock (single store) A: decl %gs:__preempt_count ; 7 je B ; 2 jmp __x86_return_thunk ; 5 call queued_spin_release_traced ; 5 out of line, reached ; only when the ; tracepoint is on jmp A ; 2 (added) B: call __SCT__preempt_schedule ; 5 jmp __x86_return_thunk ; 5 Baseline is the same stream without the NOP and the out-of-line CALL/JMP: 31 bytes vs 40 (+9 bytes). Binary size impact on x86_64, defconfig: +680 bytes (+0.00%), since all standard configs out-of-line unlock. Architectures with inlined unlock (s390 (always), csky and loongarch (both when !PREEMPTION)) will see a bigger increase in binary size. On the same path (x86_64, PARAVIRT_SPINLOCKS=3Dn) with the tracepoint disabled, a _raw_spin_unlock()-heavy nginx workload [1] shows no measurable difference between baseline and patched kernels in throughput, latency, cycles, instructions, IPC, or L1 instruction-cache misses (kernel and total): all deltas stay within run-to-run noise. Unlike x86, on arm64 the frame setup code (STP, MOV and LDP) lands on the executed path in addition to static-branch NOP. Binary size impact on arm64, defconfig: +932 bytes (+0.00%). The _raw_spin_unlock()-heavy nginx workload reflects the larger hot path: L1 instruction-cache misses rise ~1.4% (kernel and total) and instruction count ~0.4%, consistent with the per-unlock frame. cpu_cycles, throughput and latency show no measurable change and are within run-to-run noise. Architectures with fully custom qspinlock implementations (e.g. PowerPC) are not covered by this change. [1]: https://lore.kernel.org/all/aiphFXe_TPNPxZ_n@shell.ilvokhin.com/ Signed-off-by: Dmitry Ilvokhin Signed-off-by: Peter Zijlstra (Intel) Acked-by: Juergen Gross Link: https://patch.msgid.link/0d998e22a0c595f670cfc6725bb683323aced5cb.178= 5778551.git.d@ilvokhin.com --- include/asm-generic/qspinlock.h | 21 +++++++++++++++++++++ kernel/Kconfig.locks | 20 ++++++++++++++++++++ kernel/locking/qspinlock.c | 22 ++++++++++++++++++++++ 3 files changed, 63 insertions(+) diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinloc= k.h index ae45289..2ca94e4 100644 --- a/include/asm-generic/qspinlock.h +++ b/include/asm-generic/qspinlock.h @@ -41,6 +41,7 @@ =20 #include #include +#include =20 #ifndef queued_spin_is_locked /** @@ -130,12 +131,32 @@ static __always_inline void queued_spin_release(struc= t qspinlock *lock) #endif =20 #ifndef queued_spin_unlock + +DECLARE_TRACEPOINT(contended_release); + +extern void queued_spin_release_traced(struct qspinlock *lock); + /** * queued_spin_unlock - unlock a queued spinlock * @lock : Pointer to queued spinlock structure + * + * Generic tracing wrapper around the arch-overridable + * queued_spin_release(). */ static __always_inline void queued_spin_unlock(struct qspinlock *lock) { + /* + * Trace and release are combined in queued_spin_release_traced() so + * the compiler does not need to preserve the lock pointer across the + * function call, avoiding callee-saved register save/restore on the + * hot path. queued_spin_release() is therefore called both here and in + * queued_spin_release_traced(). Keep the two in sync. + */ + if (IS_ENABLED(CONFIG_QUEUED_SPINLOCKS_TRACE_CONTENDED_RELEASE) && + tracepoint_enabled(contended_release)) { + queued_spin_release_traced(lock); + return; + } queued_spin_release(lock); } #endif diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index 4198f02..1c6423a 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -243,6 +243,26 @@ config QUEUED_SPINLOCKS def_bool y if ARCH_USE_QUEUED_SPINLOCKS depends on SMP =20 +config QUEUED_SPINLOCKS_TRACE_CONTENDED_RELEASE + bool "Trace contended_release on queued spinlocks" + depends on QUEUED_SPINLOCKS && TRACEPOINTS + help + Fire the lock:contended_release tracepoint when a contended queued + spinlock is released, so it is possible to attribute a contended + spinlock to its holder. + + Architectures that can patch the unlock site do this at no cost and + do not need this option. + + Everywhere else the check is compiled into queued_spin_unlock() and + a small cost is paid on every unlock even when the tracepoint is + disabled: a static-branch NOP and possibly a few more instructions + to manage a stack frame. + + Sleeping locks fire lock:contended_release regardless of this option. + + If unsure, say N. + config BPF_ARCH_SPINLOCK bool =20 diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c index af8d122..33fe6d4 100644 --- a/kernel/locking/qspinlock.c +++ b/kernel/locking/qspinlock.c @@ -104,6 +104,28 @@ static __always_inline u32 __pv_wait_head_or_lock(str= uct qspinlock *lock, #define queued_spin_lock_slowpath native_queued_spin_lock_slowpath #endif =20 +#if !defined(queued_spin_unlock) && \ + IS_ENABLED(CONFIG_QUEUED_SPINLOCKS_TRACE_CONTENDED_RELEASE) +/* + * Out-of-line trace-and-release path for queued_spin_unlock(), used when + * the contended_release tracepoint is enabled. + * + * queued_spin_release() is duplicated here on purpose: doing the release + * in this function (rather than tracing here and releasing in the caller) + * lets queued_spin_unlock() return right after the call, so the + * tracepoint-disabled hot path never has to keep lock live across a call + * in a callee-saved register. Keep this release in sync with the one in + * queued_spin_unlock(). + */ +void __lockfunc queued_spin_release_traced(struct qspinlock *lock) +{ + if (queued_spin_is_contended(lock)) + trace_call__contended_release(lock); + queued_spin_release(lock); +} +EXPORT_SYMBOL(queued_spin_release_traced); +#endif + #endif /* _GEN_PV_LOCK_SLOWPATH */ =20 /**