From nobody Wed Apr 15 16:42:59 2026 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C52138642F for ; Wed, 4 Mar 2026 07:47:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772610453; cv=none; b=h9VK28V+WVYXnqcayj3Cs3IZaVJwPDw4fadosPZRtVx6IrFFs+ekjIjYU6hpnYVWnIUefSbR2oySVe3dFq0XGB5m6k+WZMDhDVOUG8acMfLzDKu/LwZPSwlafdCtrHzJYb+GwGOhELUcBBBmG6WV+W8om1KR251EopqpWxkCI0s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772610453; c=relaxed/simple; bh=6RIg3+xlc56a9TiATGgbmkLKt/1FmZfcHuoKSyt/chw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=briFYsqylzPfLy2ivKDJuYJbewwM8tEypB2eMHOnM2/KjNXNEG8hdGUbMmHmgtlZTaLHIvXYJWKl5w0PZtDgJhFKN4rzsFCk7vuiYWfHuZrvfKj4+xtqbMldOYxFAZc4ZFebeFeA/ZScoUWGTlA6rbTQTjuiBtragxWMvFSjW8c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nNP3pDQl; arc=none smtp.client-ip=209.85.216.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nNP3pDQl" Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-3597c40a838so1447843a91.1 for ; Tue, 03 Mar 2026 23:47:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772610446; x=1773215246; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7ugG+Q6CRrXfDVHr7sdXkPMCc63GqTD3eiVmBzfzheY=; b=nNP3pDQlHa0AD2Z3QzLIf7NljbzABm10nlytc7vp9yxz53wgzeWTCV7GMyc0rfko9T xk7Yk6wmIIP9qgDh7FRZASMGD+j979z8KxMtZS5JnOEIjwQDssr+ws5flZV+jkzUSWSf t+Yiyf5FYZEwG7XtPxP9/YgBsEMxa9kypdMhM1N650hGKlQ/pFK1FUqjAINQIA1fa7rq yN3evzG9uW0tdL5a18zNQx/VVKTwLHQlMlF6jv2vaKwo4CoQ/GA1ByXtNUUB8vJKYYhB PZvdjkV6WMV/s7cD98bnkPKTpj+UP97jE9wY4vdmMlsVKYT9F0O00yE3fWzrKQ6pWXPC 8s5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772610446; x=1773215246; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=7ugG+Q6CRrXfDVHr7sdXkPMCc63GqTD3eiVmBzfzheY=; b=nRw2mKiXMPkexYbq6EzvH1JMyJJnhtd5UoJo/Dw4Zkax+S4T+fS9NdUv4+nw6Thv/0 ZEe14ag/ZOKA5HRd76kuzT9LqRavXTlkGnCrdp1/7+SgCXHPzklu1g/5RlQI5yNkpJIe SLroA1pEU02K+LCd3OGjc1kQJ7Td3vEkd+5kUwS+S5k5ylp0GDKA8LSD9Tsu2UzpVhN/ Gpdkq5K2tYeoS/1/KEn/BCMr7LcJ+cfGv3mGWauoRsTnbT1nXUXp9iqknf19LNQuk1D8 TFfrgE8KRzW2BQZ5f6ZuT4ulXnhyQ+CTE2ar3NHFx6xRccatyiYDtVyp+dGQ+Lb/IuFc 4gnA== X-Gm-Message-State: AOJu0Yy5y6WWhYulW4MNJlsFXkmgJlCVPVOhFeAmvO5OVHRDHT317ffJ h//vJXBYD+7fj9biNk04y7SYl9Bstdx9DSwG3tXkzVTTXqJV0Dxscpm5 X-Gm-Gg: ATEYQzyzncsYUrnqu104h7+NGk5ybyP5F2OIcQRom6BjMp/iCWEwGTSKTGEGXsoU/ZA E8PjXcDHP6Q8ZUcSbV/5EI/L/ZK4DiTAYcFsN0HU2okX1XDejBWYQsvcgjOf6dUr4+e8fnMW7+U l3hb9p026hWE3BYHcK1lQKds/heJEKGJfE47LHwDH4ah+nPhFWbQ59o6qpLXuB5FSLPLy5ac5kT SOYj0ugED0f4TUUEA10ppBljFGbHkdkHQT6GLnRpO0nqd0N+6Q9emlynfKpuS++/n2re7JfZmil nvj87FvPYW6cKJopbpvp/MUh+Ofh1XP5lzR+jvCM/r3SGO7BF7miPnMQZy/SxcaZXbsIXhd01mN c4Si0yewTNi2LhyhEDuzk9+EzJYy9+7/23kKQL2j+qPTRa7vvWTa3dyw3wZrvGkiXv7vXaDVxda CyDg61G2oyywwiIOtZmqmXe9K7/2HvQrZBS7cspAa0JAu43Yz5/8a+d0ylH7kSaQ== X-Received: by 2002:a17:90b:1a92:b0:356:7b41:d348 with SMTP id 98e67ed59e1d1-359a6a3d157mr1010057a91.20.1772610446311; Tue, 03 Mar 2026 23:47:26 -0800 (PST) Received: from localhost.localdomain ([2409:891f:1ba6:a29e:d73:6202:f71f:51b0]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3599c39def7sm4352078a91.12.2026.03.03.23.47.22 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 03 Mar 2026 23:47:25 -0800 (PST) From: Yafang Shao To: peterz@infradead.org, mingo@redhat.com, will@kernel.org, boqun@kernel.org, longman@redhat.com, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, mathieu.desnoyers@efficios.com Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [RFC PATCH 1/2] locking: add mutex_lock_nospin() Date: Wed, 4 Mar 2026 15:46:49 +0800 Message-ID: <20260304074650.58165-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20260304074650.58165-1-laoar.shao@gmail.com> References: <20260304074650.58165-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce mutex_lock_nospin(), a helper that disables optimistic spinning on the owner for specific heavy locks. This prevents long spinning times that can lead to latency spikes for other tasks on the same runqueue. Signed-off-by: Yafang Shao --- include/linux/mutex.h | 3 +++ kernel/locking/mutex.c | 39 ++++++++++++++++++++++++++++++++------- 2 files changed, 35 insertions(+), 7 deletions(-) diff --git a/include/linux/mutex.h b/include/linux/mutex.h index ecaa0440f6ec..1e488bb24b57 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -189,11 +189,13 @@ extern int __must_check mutex_lock_interruptible_nest= ed(struct mutex *lock, extern int __must_check _mutex_lock_killable(struct mutex *lock, unsigned int subclass, struct lockdep_map *nest_lock) __cond_acquires(0,= lock); extern void mutex_lock_io_nested(struct mutex *lock, unsigned int subclass= ) __acquires(lock); +extern void mutex_lock_nospin_nested(struct mutex *lock, unsigned int subc= lass); =20 #define mutex_lock(lock) mutex_lock_nested(lock, 0) #define mutex_lock_interruptible(lock) mutex_lock_interruptible_nested(loc= k, 0) #define mutex_lock_killable(lock) _mutex_lock_killable(lock, 0, NULL) #define mutex_lock_io(lock) mutex_lock_io_nested(lock, 0) +#define mutex_lock_nospin(lock) mutex_lock_nospin_nested(lock, 0) =20 #define mutex_lock_nest_lock(lock, nest_lock) \ do { \ @@ -215,6 +217,7 @@ extern void mutex_lock(struct mutex *lock) __acquires(l= ock); extern int __must_check mutex_lock_interruptible(struct mutex *lock) __con= d_acquires(0, lock); extern int __must_check mutex_lock_killable(struct mutex *lock) __cond_acq= uires(0, lock); extern void mutex_lock_io(struct mutex *lock) __acquires(lock); +extern void mutex_lock_nospin(struct mutex *lock) __acquires(lock); =20 # define mutex_lock_nested(lock, subclass) mutex_lock(lock) # define mutex_lock_interruptible_nested(lock, subclass) mutex_lock_interr= uptible(lock) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index 2a1d165b3167..03d3b0749882 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -290,6 +290,14 @@ void __sched mutex_lock(struct mutex *lock) __mutex_lock_slowpath(lock); } EXPORT_SYMBOL(mutex_lock); + +void __sched mutex_lock_nospin(struct mutex *lock) +{ + might_sleep(); + + if (!__mutex_trylock_fast(lock)) + __mutex_lock_nospin(lock); +} #endif =20 #include "ww_mutex.h" @@ -443,8 +451,11 @@ static inline int mutex_can_spin_on_owner(struct mutex= *lock) */ static __always_inline bool mutex_optimistic_spin(struct mutex *lock, struct ww_acquire_ctx *ww_ctx, - struct mutex_waiter *waiter) + struct mutex_waiter *waiter, const bool nospin) { + if (nospin) + return false; + if (!waiter) { /* * The purpose of the mutex_can_spin_on_owner() function is @@ -577,7 +588,8 @@ EXPORT_SYMBOL(ww_mutex_unlock); static __always_inline int __sched __mutex_lock_common(struct mutex *lock, unsigned int state, unsigned int s= ubclass, struct lockdep_map *nest_lock, unsigned long ip, - struct ww_acquire_ctx *ww_ctx, const bool use_ww_ctx) + struct ww_acquire_ctx *ww_ctx, const bool use_ww_ctx, + const bool nospin) { DEFINE_WAKE_Q(wake_q); struct mutex_waiter waiter; @@ -615,7 +627,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int st= ate, unsigned int subclas =20 trace_contention_begin(lock, LCB_F_MUTEX | LCB_F_SPIN); if (__mutex_trylock(lock) || - mutex_optimistic_spin(lock, ww_ctx, NULL)) { + mutex_optimistic_spin(lock, ww_ctx, NULL, nospin)) { /* got the lock, yay! */ lock_acquired(&lock->dep_map, ip); if (ww_ctx) @@ -716,7 +728,7 @@ __mutex_lock_common(struct mutex *lock, unsigned int st= ate, unsigned int subclas * to run. */ clear_task_blocked_on(current, lock); - if (mutex_optimistic_spin(lock, ww_ctx, &waiter)) + if (mutex_optimistic_spin(lock, ww_ctx, &waiter, nospin)) break; set_task_blocked_on(current, lock); trace_contention_begin(lock, LCB_F_MUTEX); @@ -773,14 +785,21 @@ static int __sched __mutex_lock(struct mutex *lock, unsigned int state, unsigned int subclass, struct lockdep_map *nest_lock, unsigned long ip) { - return __mutex_lock_common(lock, state, subclass, nest_lock, ip, NULL, fa= lse); + return __mutex_lock_common(lock, state, subclass, nest_lock, ip, NULL, fa= lse, false); +} + +static int __sched +__mutex_lock_nospin(struct mutex *lock, unsigned int state, unsigned int s= ubclass, + struct lockdep_map *nest_lock, unsigned long ip) +{ + return __mutex_lock_common(lock, state, subclass, nest_lock, ip, NULL, fa= lse, true); } =20 static int __sched __ww_mutex_lock(struct mutex *lock, unsigned int state, unsigned int subcl= ass, unsigned long ip, struct ww_acquire_ctx *ww_ctx) { - return __mutex_lock_common(lock, state, subclass, NULL, ip, ww_ctx, true); + return __mutex_lock_common(lock, state, subclass, NULL, ip, ww_ctx, true,= false); } =20 /** @@ -861,11 +880,17 @@ mutex_lock_io_nested(struct mutex *lock, unsigned int= subclass) =20 token =3D io_schedule_prepare(); __mutex_lock_common(lock, TASK_UNINTERRUPTIBLE, - subclass, NULL, _RET_IP_, NULL, 0); + subclass, NULL, _RET_IP_, NULL, 0, false); io_schedule_finish(token); } EXPORT_SYMBOL_GPL(mutex_lock_io_nested); =20 +void __sched +mutex_lock_nospin_nested(struct mutex *lock, unsigned int subclass) +{ + __mutex_lock_nospin(lock, TASK_UNINTERRUPTIBLE, subclass, NULL, _RET_IP_); +} + static inline int ww_mutex_deadlock_injection(struct ww_mutex *lock, struct ww_acquire_ctx *= ctx) { --=20 2.47.3