From nobody Fri Jul 24 21:53:42 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B55D3382F07; Thu, 23 Jul 2026 10:21:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784802084; cv=none; b=ZDDkSKsaIfq6A2XgGI9ciR6Th8U+wH7LOCBiewuMp3At4QEv6JWQmhmvahyj5yDg8MJKo9vTLJOJvVbmRF9W29bGP1CyUerjDuRz6IaLyzuv4wYiyqvD6Dlh0IAzZQIMRhoxAkecDl2rnp1lraASFSiKaugLkH4JLw+mItK3EPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784802084; c=relaxed/simple; bh=p9LLPQFh6Fjg6qgBtRX/h3CmRMSS+W4/6pJCE9Ate5k=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=RWKUV7aXW2GkJNsjl8GVB74WmM/RVj0f2VHFcbkwacljNRfMlrDuTmJHYa/amIbpp8Ys4lyZ9ft7Bzcf8GoJwY9AzbG6NbvYvo1Re827NiOpn1bFQ2beU0B8U/OaNhNvwiLFBvhKXbxem8FLUbEUFmuTKdv0jP7GfD10+qsAA84= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=dLFWRmfx; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=gFSKAklU; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="dLFWRmfx"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="gFSKAklU" Date: Thu, 23 Jul 2026 10:21:19 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1784802081; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XW6D6vL3YG7SSAiFZfpA0iQO75itDI74ahJS6m4td2U=; b=dLFWRmfxs9PpqJWsCL9/xDBYD/LfBn6IHHAzM1daw1cdfAKCaj10muTYXHX0YB3BNE9Lng 5E6Q0x6Sj4t/1O15kWRTNpsqItQsAsDgOqCualWjAmCwmFpFH8YszblVAz2E/j0NKCh6DO DJdmEk/fsrx9eO3kkG6CVM7VWSodP9uL4Wu+s8I5fdYnkMWcafOIBLuv26fGgmVrhNLhdI HFmYmaMG6Z+Ik3BKcYy6pKJ9XHO6k5+B23dtK0jHiv+E9ClmxYx2QtTCM8s9kCNdX8738L DyRhiOgpNBWKXiccoV5efxEFTkbpKwrLn3iqew21AJ8nws27wtZBzZyx4DaEOw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1784802081; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XW6D6vL3YG7SSAiFZfpA0iQO75itDI74ahJS6m4td2U=; b=gFSKAklUqrbva9s0sLBrom6e8NKDfU/Fh7o7PFLwVACsefn75UoGEIJtX6PN3CIMMKodNS 0hjg+cWUnen2aNBQ== From: "tip-bot2 for Chuyi Zhou" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: smp/core] smp: Enable preemption early in smp_call_function_many_cond() Cc: Chuyi Zhou , Thomas Gleixner , "Paul E. McKenney" , Sebastian Andrzej Siewior , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260709122933.4021501-7-zhouchuyi@bytedance.com> References: <20260709122933.4021501-7-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178480207922.2943223.10816725434232573668.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the smp/core branch of tip: Commit-ID: 8df8a6028309a549ba1b83cf4f01e8e0f7f23f3f Gitweb: https://git.kernel.org/tip/8df8a6028309a549ba1b83cf4f01e8e0f= 7f23f3f Author: Chuyi Zhou AuthorDate: Thu, 09 Jul 2026 20:29:25 +08:00 Committer: Thomas Gleixner CommitterDate: Thu, 16 Jul 2026 09:24:55 +02:00 smp: Enable preemption early in smp_call_function_many_cond() smp_call_function_many_cond() still has to keep the caller pinned to the current CPU while the remote IPI request is built and dispatched. This protects the queueing state and CPU-hotplug boundary that are required before the synchronous wait starts: - It protects the current CPU's per-CPU scratch cpumask, cfd->cpumask_ipi. Another task running on the same CPU could otherwise enter smp_call_function_many_cond() and reuse that scratch cpumask before the current caller has finished building and sending the IPI request. - It provides the CPU-hotplug exclusion required by the CSD queueing side. New CSDs must not be queued after smpcfd_dying_cpu() has flushed the outgoing CPU's callback queue. Keeping preemption disabled until all required CSDs have been queued and the corresponding IPIs have been sent prevents CPU offline from crossing that boundary in the middle of the queueing operation. The CSD acquisition side also relies on that caller-side CPU pinning. csd_lock() waits for CSD_FLAG_LOCK to clear and then marks the CSD busy with a regular store, so another task on the same CPU must not be allowed to acquire and reinitialize the same per-CPU CSD concurrently. After the callbacks have been queued and the IPIs have been sent, the caller only performs the final csd_lock_wait() completion wait. If it is preempted there, another task running on the original CPU may enter smp_call_function_many_cond(), but any attempt to reuse the same per-CPU CSD will block in csd_lock() until the previous callback clears CSD_FLAG_LOCK. The final csd_lock_wait() does not acquire or reinitialize the CSD, so it does not need the same caller-side preemption-disabled protection. The wait mask is task-local, so it cannot be overwritten by another task on the original CPU. The per-CPU CSD storage also remains allocated across CPU offline, so csd_lock_wait() can safely dereference it even if the target CPU is offlined after the caller is unpinned. With those requirements satisfied, enable preemption before the synchronous csd_lock_wait() loop. This makes the potentially long wait preemptible and migratable while keeping the CPU-pinned section around the remote CPU selection and IPI dispatch. Signed-off-by: Chuyi Zhou Signed-off-by: Thomas Gleixner Tested-by: Paul E. McKenney Reviewed-by: Sebastian Andrzej Siewior Link: https://patch.msgid.link/20260709122933.4021501-7-zhouchuyi@bytedance= .com --- kernel/smp.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/kernel/smp.c b/kernel/smp.c index e6c1d83..f6f8529 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -871,15 +871,14 @@ static void smp_call_function_many_cond(const struct = cpumask *mask, unsigned int scf_flags, smp_cond_func_t cond_func) { - int cpu, last_cpu, this_cpu =3D smp_processor_id(); struct cpumask *cpumask, *task_mask; bool wait =3D scf_flags & SCF_WAIT; struct call_function_data *cfd; + int cpu, last_cpu, this_cpu; bool run_remote =3D false; int nr_cpus =3D 0; =20 - lockdep_assert_preemption_disabled(); - + this_cpu =3D get_cpu(); cfd =3D this_cpu_ptr(&cfd_data); task_mask =3D smp_task_ipi_mask(current); if (task_mask) @@ -965,6 +964,16 @@ static void smp_call_function_many_cond(const struct c= pumask *mask, local_irq_restore(flags); } =20 + /* + * The IPI work has been queued and dispatched. On PREEMPT kernels, + * tasks created through dup_task_struct() have task-local wait masks. + * The boot init_task can fall back to cfd->cpumask when the mask is + * not inlined, but other tasks still use task-local masks and cannot + * overwrite it. On !PREEMPT kernels, preempt_enable() cannot schedule + * another task, so the per-CPU mask remains protected. + */ + put_cpu(); + if (run_remote && wait) { for_each_cpu(cpu, cpumask) { call_single_data_t *csd; @@ -977,15 +986,14 @@ static void smp_call_function_many_cond(const struct = cpumask *mask, =20 /** * smp_call_function_many() - Run a function on a set of CPUs. - * @mask: The set of cpus to run on (only runs on online subset). - * @func: The function to run. This must be fast and non-blocking. - * @info: An arbitrary pointer to pass to the function. - * @wait: If true, wait (atomically) until function has completed - * on other CPUs. + * @mask: The set of cpus to run on (only runs on online subset). + * @func: The function to run. This must be fast and non-blocking. + * @info: An arbitrary pointer to pass to the function. + * @wait: If true, wait (atomically) until function has completed + * on other CPUs. * * You must not call this function with disabled interrupts or from a - * hardware interrupt handler or from a bottom half handler. Preemption - * must be disabled when calling this function. + * hardware interrupt handler or from a bottom half handler. * * @func is not called on the local CPU even if @mask contains it. Consid= er * using on_each_cpu_cond_mask() instead if this is not desirable.