From nobody Fri Jul 24 22:54:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3DF1D381EAB; Wed, 22 Jul 2026 18:58:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784746703; cv=none; b=bPRGYaDWQ7rpJcMGrihz8zP6shXhBZgxx1yWTYaQIGmAD5vGnaOzfEu8y+p/Tdwbj1+5GqtmLiEAheMqPbROvgd5b1T8S+AodZE4sDkD4+GQnvmaR9UmhBIjXoO2x159vZBP6SOH/rZ13c9GhVxUJEN0/UYBp8+00CW8928gDAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784746703; c=relaxed/simple; bh=0ZnBhELDzFe7u/mVw8S5BA/0x+Cw7fjgFGNtgoHMxwA=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=nxGP9N+hePr4XTPNKnKDy7shx33mqp1qIWyFnA9H8Z24A7ilXioKZOn3EZqHbKMZay0wujcvvzUI/Z2fCoAz33D2mM0Ch4Pw3ZWzNbBJyrMZ6BqsJ/05bwLxO7eQaLvqiOsy4HxmRzf8YttCxduG8JMt2o5fu5f/ulxiJZNmQMg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=JttEzfw9; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=D+2TiM/j; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="JttEzfw9"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="D+2TiM/j" Date: Wed, 22 Jul 2026 18:58:13 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1784746695; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SDw5h2iHsO0SlVAnoE6+ESSUR9a4m5neKHQP/9VRO0E=; b=JttEzfw9/i4+gsCfRdHvSPhTMFdy5NOAXFcoLy9SANmPhPgplPqYUCe713bSzIOFvyDYvg u2baVIjlz2jZdSGweO8aAgaQ02lcCcrUFsuglp0lggoGODvF8RleMNBNuCb5CeH+9t8ghs 1Nn4em2RMxEtwo6Je75PzbJX4W0NcPBKpsVOQCzf0re2+TBqNopz3hYxXXl+TYBym4t5E0 fWOXUue0k/b+5AASsR5qpWGBG/Cq9q4VrdGhSc69EwJ6f31SD9b3Ffr2yevlKzmg4WI4lQ XCvuOlTlwxLnYBbzxP6myepcnT2jB/9nXqNc/Pwr+HXUVwPJkwSiTdh+3UIeHw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1784746695; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SDw5h2iHsO0SlVAnoE6+ESSUR9a4m5neKHQP/9VRO0E=; b=D+2TiM/jS2tUfZtcKSyY1OejT+5YKL+AyB4ff5r5dZ65LjuCNHWepxdsWeQtOEqVZKMUAK xRd9lSNMKx+DcJDA== From: "tip-bot2 for Chuyi Zhou" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: smp/urgent] smp: Make CSD lock acquisition atomic for debug mode Cc: Chuyi Zhou , "Paul E. McKenney" , Thomas Gleixner , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260716004539.13983-2-paulmck@kernel.org> References: <20260716004539.13983-2-paulmck@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178474669317.2943223.4347840134862103129.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the smp/urgent branch of tip: Commit-ID: 35551efb155e3b83445a6c3f66cb498d5efc182c Gitweb: https://git.kernel.org/tip/35551efb155e3b83445a6c3f66cb498d5= efc182c Author: Chuyi Zhou AuthorDate: Wed, 15 Jul 2026 17:45:39 -07:00 Committer: Thomas Gleixner CommitterDate: Wed, 22 Jul 2026 20:57:08 +02:00 smp: Make CSD lock acquisition atomic for debug mode Commit b0473dcd4b1d ("smp: Improve smp_call_function_single() CSD-lock diagnostics") changed smp_call_function_single() so that, when CSD lock debugging is enabled, async !wait calls use the destination CPU csd_data. That improves diagnostics, but it also removes the single-writer property that made the old csd_lock() safe: multiple CPUs can now prepare the same destination CPU CSD concurrently. csd_lock() currently waits for CSD_FLAG_LOCK to clear and then sets the bit with a non-atomic read-modify-write. Two senders can both see an unlocked CSD, set the bit, overwrite the callback fields, and enqueue the same llist node. Re-adding a node that is already the queue head can make node->next point to itself, leaving the target CPU stuck walking call_single_queue. Later synchronous work, such as a TLB shootdown, can then remain queued and trigger soft-lockup warnings or panics. Keep the single csd_lock() implementation, but when CSD lock debugging is enabled, acquire CSD_FLAG_LOCK with try_cmpxchg_acquire(). This makes the destination CPU CSD a real atomic lock in the only configuration where it can be shared by multiple remote senders, while preserving the existing non-debug fast path. Fixes: b0473dcd4b1d ("smp: Improve smp_call_function_single() CSD-lock diag= nostics") Signed-off-by: Chuyi Zhou Signed-off-by: Paul E. McKenney Signed-off-by: Thomas Gleixner Link: https://patch.msgid.link/20260716004539.13983-2-paulmck@kernel.org --- kernel/smp.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/kernel/smp.c b/kernel/smp.c index dc6582b..52dffc8 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -137,10 +137,10 @@ csd_do_func(smp_call_func_t func, void *info, call_si= ngle_data_t *csd) trace_csd_function_exit(func, csd); } =20 -#ifdef CONFIG_CSD_LOCK_WAIT_DEBUG - static DEFINE_STATIC_KEY_MAYBE(CONFIG_CSD_LOCK_WAIT_DEBUG_DEFAULT, csdlock= _debug_enabled); =20 +#ifdef CONFIG_CSD_LOCK_WAIT_DEBUG + /* * Parse the csdlock_debug=3D kernel boot parameter. * @@ -342,6 +342,10 @@ static __always_inline void csd_lock_wait(call_single_= data_t *csd) smp_cond_load_acquire(&csd->node.u_flags, !(VAL & CSD_FLAG_LOCK)); } #else +static __always_inline void __csd_lock_wait(call_single_data_t *csd) +{ +} + static void csd_lock_record(call_single_data_t *csd) { } @@ -354,8 +358,23 @@ static __always_inline void csd_lock_wait(call_single_= data_t *csd) =20 static __always_inline void csd_lock(call_single_data_t *csd) { - csd_lock_wait(csd); - csd->node.u_flags |=3D CSD_FLAG_LOCK; + if (IS_ENABLED(CONFIG_CSD_LOCK_WAIT_DEBUG) && + static_branch_unlikely(&csdlock_debug_enabled)) { + + for (;;) { + unsigned int flags; + + __csd_lock_wait(csd); + flags =3D READ_ONCE(csd->node.u_flags); + + if (!(flags & CSD_FLAG_LOCK) && + try_cmpxchg_acquire(&csd->node.u_flags, &flags, flags | CSD_FLAG_LO= CK)) + break; + } + } else { + csd_lock_wait(csd); + csd->node.u_flags |=3D CSD_FLAG_LOCK; + } =20 /* * prevent CPU from reordering the above assignment