From nobody Fri Sep 25 13:55:14 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6248D3603F6 for ; Fri, 11 Sep 2026 14:38:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789137507; cv=none; b=YpSOwoFfjxJz1nyXGxaiJnqliFPJg1BjHIxCIPJtE1u/qyG6FomYEPW/lJExlCLeALJUxvQWOd1YyMiK1bEdXKIq+dzesPaYMFIXRDduTzklJgpkTLWYN3ENft0muJLvqtsXiDJWsH7JeeyhlE60/VbLiEeCBiRcCzk1cdMLBlA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789137507; c=relaxed/simple; bh=GJvCmx2MfMYm9equO33kvsTJuKSOu+ge/w2GnIbcBK4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j3veO/HkUPPFmzhDU4j7K/nUE95wbXHYnMNbBmzzOGC74wv0Bh+Q+Gkw4n5+p/h2njqy53GTDtMDJ4gKD4VlUcIYBdfO5QvmxAcpF3B/qZ/1yeAwYRCwrMJrmH0EFpzwqAoCnb6COnHeXmfwgXywtAvhDeZop+WJOvE5hZ6UvZU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=ITZF1ajq; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=pBsiYd67; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="ITZF1ajq"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="pBsiYd67" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1789137501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/fwlCvAZvaN3yiglrF68ZaeLaN7r8+E8m5jAkiJbzdY=; b=ITZF1ajqz1hVyoggygO19afb8l4EWH8LsDpSIXrNKSdvXEeNesFIc1HR+b07dbxkoUb+xG 6CSQ62DR7TdsiOP0ALMSSZGSiF1E/RP02jg4WUFzvfkstllbrAWC0GaBUJRcM3ZAblRrQU c3gPgyCg7ngGtvyXSQRPjkhmqr3uHyH9bBMxcrkwt6QVQJ5Vank10Zy3WAsA6+/Rg3IAQ0 FYRH7v/w51eefMLu1nPFGGCs/EAB1yzThA8fhr9AkXHnxrs01B4Jw5Lfod/AUKEUMQBOil RHis+iNt8HZ6en68bYbLinYrEp2DFxyCq6qAww8Mj5b3rKdV2IzZ28LrelR0Jg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1789137501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/fwlCvAZvaN3yiglrF68ZaeLaN7r8+E8m5jAkiJbzdY=; b=pBsiYd67/gpRFTA8bkw6Cn9XAiR8IGY7BGlw37sxePpgMI/7RSeWbJXD+XIygfCFGDokEY KP0jkzcgpBazp+DQ== To: linux-kernel@vger.kernel.org, linux-rt-devel@lists.linux.dev Cc: Thomas Gleixner , Peter Zijlstra , Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt Subject: [PATCH 1/3] irq_work: Update a comment regarding CPU hotplug invocation Date: Fri, 11 Sep 2026 16:38:12 +0200 Message-ID: <20260911143815.997254-2-bigeasy@linutronix.de> In-Reply-To: <20260911143815.997254-1-bigeasy@linutronix.de> References: <20260911143815.997254-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" hotplug_cfd() no longer exists and the flow changed a bit since the rework of CPU hotplug as of commit 31487f8328f20 ("smp/cfd: Convert core to hotplug state machine"). Update the comment to reflect reality as of today. Signed-off-by: Sebastian Andrzej Siewior --- kernel/irq_work.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/kernel/irq_work.c b/kernel/irq_work.c index f7e2dc2c30c62..9f19c05c9962c 100644 --- a/kernel/irq_work.c +++ b/kernel/irq_work.c @@ -252,10 +252,7 @@ static void irq_work_run_list(struct llist_head *list) irq_work_single(work); } =20 -/* - * hotplug calls this through: - * hotplug_cfd() -> flush_smp_call_function_queue() - */ +/* CPU hotplug calls this through smpcfd_dying_cpu() */ void irq_work_run(void) { irq_work_run_list(this_cpu_ptr(&raised_list)); --=20 2.55.0 From nobody Fri Sep 25 13:55:14 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDF41345736 for ; Fri, 11 Sep 2026 14:38:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789137508; cv=none; b=aFdfgXMAXIL0pPbLhevC4SvCDwEfpwie15VyVDSdWOL0zz2dKhBj3ULJ49Xc7wx5LbsyW7jDf9lBdIJy2jvLwY8b94yk+CKHhE32sKvM1ieQz/uGrwHbwHaY2BqagsaWzXLvXdS/Hj5c0vuoj/i10qt1JSd7JVL+S/DZ1Jr6Zdc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789137508; c=relaxed/simple; bh=R7R18QEPttI5tJ0Mnid1UU0N5zuu9+//SNy8MOd6bO4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p5+pOyPwxiKyI+WQMvN6USF6+9GQvmK/iNUkOe71CR5a7ENuQl3R4Erqk6cwRqPjJUuBK4jpIVX3kCY10zKPcPB+tOt9krjrpp1ZBqW6rMLaffi2hWCYHFdbvD0V9bAC8j/0UFKXGx+n+kNTEX4AjLkKpomFCTY044eiL+m8hRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=qGPwVphw; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=19WPZ2GN; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="qGPwVphw"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="19WPZ2GN" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1789137501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D9IFnuoISe8DVZ4o8aEwdjhC6POqjZl+keV1u44bYjE=; b=qGPwVphwNXxowtP7Vv3qfHiY2O1+kE0YLhyRRIe0Xv7c05S/z2aR2xIj6Pr6/vS25TI8rE /Cbus7YMLfxbPFyf5mLr2W3H/RQFk3U7bKjpxvvqmNlYQHFbNmFCeOLt6oRj6SQt/h4PLU 7KT5LS28ykAtghs4B7r+c46cpF6dQ5hiuiHAytbxqP9OYPnkhn/jT3KqDXmRJH3piNsiVa 94do4qidRHRnYb9KSZuNE459smv5oe2KgzraLQJdRbjzXETVuclXZkjxrIFpCrCHkdrfHR OjxzkxdUUuPx17i04LdBjGd1KA69/MDeN2yFSHDSjB9L2zSxp+aYyl3MOxZnXA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1789137501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D9IFnuoISe8DVZ4o8aEwdjhC6POqjZl+keV1u44bYjE=; b=19WPZ2GN5M/wPzdSaCQB5gsPYjpVRUlsrF5K2uWPXcGcWR27MzW3i4PKedvrVoJ9kHEDk1 ecyIx09iKYf8IfDA== To: linux-kernel@vger.kernel.org, linux-rt-devel@lists.linux.dev Cc: Thomas Gleixner , Peter Zijlstra , Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt Subject: [PATCH 2/3] irq_work: Flush lazy work CPU down on PREEMPT_RT Date: Fri, 11 Sep 2026 16:38:13 +0200 Message-ID: <20260911143815.997254-3-bigeasy@linutronix.de> In-Reply-To: <20260911143815.997254-1-bigeasy@linutronix.de> References: <20260911143815.997254-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" PREEMPT_RT invokes IRQ_WORK_LAZY callbacks and callbacks which are not explicitly marked IRQ_WORK_HARD_IRQ from thread context. If a CPU gets shutdown, the irq_work is flushed during smpcfd_dying_cpu() which is invoked on the target CPU with disabled interrupts. The callbacks enqueued on lazy_list on PREEMPT_RT are not flushed because they require thread context. The callbacks remain and get processed once the CPU gets back online. per-CPU irq_work items continue to work for enqueues on their local-CPU, only the work on the "offline" CPU get stuck. "Single" irq_work are worse because they remain "claimed" and can not be used again. One "symptom" is the late printk() in CPU down path which enqueues the irq_work for printing but it never gets scheduled. Further prints don't print to the console because the irq_work is "pending" of the offline CPU. Another problem is the canceling/ flushing of the irq_work which is stuck. Since the irq_work will not continue, the task waiting for its completion will block waiting. While looking through the users of irq_work, it does not matter if the callback is enqueued on another CPU. This makes it possible to invoke the callbacks from a remote CPU by smpcfd_dead_cpu(). This is invoked from the control CPU with enabled interrupts shortly after the hotplugged CPU is shutdown so the llist can be accessed safely. Add irq_work_run_cpu() which invokes the irq_work callbacks of the specified dead CPU. Invoke it from smpcfd_dead_cpu(). Signed-off-by: Sebastian Andrzej Siewior --- include/linux/irq_work.h | 2 ++ kernel/irq_work.c | 11 +++++++++++ kernel/smp.c | 1 + 3 files changed, 14 insertions(+) diff --git a/include/linux/irq_work.h b/include/linux/irq_work.h index c5afd053ae32c..7184e12739653 100644 --- a/include/linux/irq_work.h +++ b/include/linux/irq_work.h @@ -59,6 +59,7 @@ void irq_work_sync(struct irq_work *work); #include =20 void irq_work_run(void); +void irq_work_run_cpu(unsigned int cpu); bool irq_work_needs_cpu(void); void irq_work_single(void *arg); =20 @@ -67,6 +68,7 @@ void arch_irq_work_raise(void); #else static inline bool irq_work_needs_cpu(void) { return false; } static inline void irq_work_run(void) { } +static inline void irq_work_run_cpu(unsigned int cpu) { } static inline void irq_work_single(void *arg) { } #endif =20 diff --git a/kernel/irq_work.c b/kernel/irq_work.c index 9f19c05c9962c..73eabcbdcd50c 100644 --- a/kernel/irq_work.c +++ b/kernel/irq_work.c @@ -263,6 +263,17 @@ void irq_work_run(void) } EXPORT_SYMBOL_GPL(irq_work_run); =20 +void irq_work_run_cpu(unsigned int cpu) +{ + if (WARN_ON_ONCE(!cpumask_test_cpu(cpu, cpu_dying_mask))) + return; + + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) + return; + + irq_work_run_list(per_cpu_ptr(&lazy_list, cpu)); +} + void irq_work_tick(void) { struct llist_head *raised =3D this_cpu_ptr(&raised_list); diff --git a/kernel/smp.c b/kernel/smp.c index b696bcc60c08f..8f530a094c304 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -87,6 +87,7 @@ int smpcfd_dead_cpu(unsigned int cpu) =20 free_cpumask_var(cfd->cpumask); free_cpumask_var(cfd->cpumask_ipi); + irq_work_run_cpu(cpu); return 0; } =20 --=20 2.55.0 From nobody Fri Sep 25 13:55:14 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A138F36194E for ; Fri, 11 Sep 2026 14:38:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789137507; cv=none; b=dhLI1Olm8HdpamJ52RWLM3f0EnPXVptB86dKYuKk7HvDNC8K0xBqAjbZkmIkTO65vs4dVpoP3fSH97NMCtBbv1wDQLYSQom/MNkOnT8RgUxFAwNiAaL7INgj5S0lWz0OghFqMd/iNBAyYLTrN80iLy/c3T913EwxN3aQNaCWBOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789137507; c=relaxed/simple; bh=G949rd2BksPJPpHrTKqHHeH5/djt9kP0MvWFp7GAx7I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=g0OnIBWuG8YfEQml+fx21M14JYDS8DWRzzwONba1PCzt03nFISGXR3km2KFKQC3CDxKuk15sUONRJXFn1q3M81XAQHBcxzntIfvwbp2ms3WTIsb3RiuVw80LUnJ/U9cGweO0EkUw0iPjk96+4ekdTVfjsXvcqz+WdfpGxHdugoE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=AlydMHuj; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Y5LkwNFn; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="AlydMHuj"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Y5LkwNFn" From: Sebastian Andrzej Siewior DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1789137502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7fnhCFqimC/U/Uavu1hoREyaaaFUjMv/5s5k7FC9gQA=; b=AlydMHujNCfNMpCWYFVDLLwXRO9q9caQxsvHmhIMz/2xx3fUuRjlA9JswlzZpa853WlzEF emkRXQnJuRXCiES5757NMqIJHDHYm50Yq4XCjGpaOzOG/aTjUPIYxlr2bAPZRKgRIVHzkg dlZZwMHtVWunF9MRofvC/R7S3+GzDmXGfvtZdMvsvk7Aeos6BlP8Ee3UMvnYNGJ2ZZa80r x3Hk0UM12glMsu/DNJSMbd8MLl6H2jXRGDcbpdluDqP5q3sSXUbSIgQdxjIYiXGB5e8nTI C/0NURe962tExwSKCteXag8wgvnkkBWkhL2Usmu67vY37Xy7Hhv+kJ9Naum23w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1789137502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7fnhCFqimC/U/Uavu1hoREyaaaFUjMv/5s5k7FC9gQA=; b=Y5LkwNFnKn7PUVSkOB9tggzIn1BJFBkw3id2mk4JXcYQ5pQTIUSRje+uoReR8E3hmZA0Pz COUuYtqrcdcLK6BA== To: linux-kernel@vger.kernel.org, linux-rt-devel@lists.linux.dev Cc: Thomas Gleixner , Peter Zijlstra , Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt Subject: [PATCH 3/3] smpboot: Don't park the thread if work is pending Date: Fri, 11 Sep 2026 16:38:14 +0200 Message-ID: <20260911143815.997254-4-bigeasy@linutronix.de> In-Reply-To: <20260911143815.997254-1-bigeasy@linutronix.de> References: <20260911143815.997254-1-bigeasy@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Once a smpboot thread gets work assigned it receives a wake up. If the thread had no chance to run and receives a parking requesst during a CPU-hotplug event then it is parked without running the callback first. As a result the enqueued work remains stuck until the CPU gets back online. There is nothing wrong with invoking the thread function first and parking in the following iteration. This would ensure that the callbacks are processed before the shutdown. Honor the park request once thread function does not need to run. Signed-off-by: Sebastian Andrzej Siewior --- kernel/smpboot.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/smpboot.c b/kernel/smpboot.c index 4503b60ce9bd2..3f60e8c6dd301 100644 --- a/kernel/smpboot.c +++ b/kernel/smpboot.c @@ -103,6 +103,7 @@ static int smpboot_thread_fn(void *data) { struct smpboot_thread_data *td =3D data; struct smp_hotplug_thread *ht =3D td->ht; + bool should_run; =20 while (1) { set_current_state(TASK_INTERRUPTIBLE); @@ -117,7 +118,8 @@ static int smpboot_thread_fn(void *data) return 0; } =20 - if (kthread_should_park()) { + should_run =3D td->status =3D=3D HP_THREAD_ACTIVE && ht->thread_should_r= un(td->cpu); + if (kthread_should_park() && !should_run) { __set_current_state(TASK_RUNNING); preempt_enable(); if (ht->park && td->status =3D=3D HP_THREAD_ACTIVE) { @@ -151,7 +153,7 @@ static int smpboot_thread_fn(void *data) continue; } =20 - if (!ht->thread_should_run(td->cpu)) { + if (!should_run) { preempt_enable_no_resched(); schedule(); } else { --=20 2.55.0