kernel/sched/rt.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
CPU0 becomes overloaded when hosting a CPU-bound RT task, a non-CPU-bound
RT task, and a CFS task stuck in kernel space. When other CPUs switch from
RT to non-RT tasks, RT load balancing (LB) is triggered; with
HAVE_RT_PUSH_IPI enabled, they send IPIs to CPU0 to drive the execution
of rto_push_irq_work_func. During push_rt_task on CPU0,
if next_task->prio < rq->donor->prio, resched_curr() sets NEED_RESCHED
and after the push operation completes, CPU0 calls rto_next_cpu().
Since only CPU0 is overloaded in this scenario, rto_next_cpu() should
ideally return -1 (no further IPI needed).
However, multiple CPUs invoking tell_cpu_to_push() during LB increments
rd->rto_loop_next. Even when rd->rto_cpu is set to -1, the mismatch between
rd->rto_loop and rd->rto_loop_next forces rto_next_cpu() to restart its
search from -1. With CPU0 remaining overloaded (satisfying rt_nr_migratory
&& rt_nr_total > 1), it gets reselected, causing CPU0 to queue irq_work to
itself and send self-IPIs repeatedly. As long as CPU0 stays overloaded and
other CPUs run pull_rt_tasks(), it falls into an infinite self-IPI loop,
which triggers a CPU hardlockup due to continuous self-interrupts.
The trigging scenario is as follows:
cpu0 cpu1 cpu2
pull_rt_task
tell_cpu_to_push
<------------irq_work_queue_on
rto_push_irq_work_func
push_rt_task
resched_curr(rq) pull_rt_task
rto_next_cpu tell_cpu_to_push
<-------------------------- atomic_inc(rto_loop_next)
rd->rto_loop != next
rto_next_cpu
irq_work_queue_on
rto_push_irq_work_func
Fix redundant self-IPI by filtering the initiating CPU in rto_next_cpu().
This solution has been verified to effectively eliminate spurious self-IPIs
and prevent CPU hardlockup scenarios.
Fixes: 4bdced5c9a29 ("sched/rt: Simplify the IPI based RT balancing logic")
Suggested-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Chen Jinghuang <chenjinghuang2@huawei.com>
Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
Changes since v2: https://lore.kernel.org/all/20251125083649.1814558-1-chenjinghuang2@huawei.com/
- Replace the original "check NEED_RESCHED on target CPU"
logic with "skip the currently executing CPU"
- This modification eliminates self-IPIS
Changes since v1: https://lore.kernel.org/all/20251121014004.564508-1-chenjinghuang2@huawei.com/
- Remove unneeded extra whitespace
- Add Reviewed-by tag from Steven Rostedt
---
kernel/sched/rt.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index f1867fe8e5c5..ec10ed7bb75d 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -2100,6 +2100,7 @@ static void push_rt_tasks(struct rq *rq)
*/
static int rto_next_cpu(struct root_domain *rd)
{
+ int this_cpu = smp_processor_id();
int next;
int cpu;
@@ -2118,10 +2119,13 @@ static int rto_next_cpu(struct root_domain *rd)
*/
for (;;) {
- /* When rto_cpu is -1 this acts like cpumask_first() */
- cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
+ do {
+ /* When rto_cpu is -1 this acts like cpumask_first() */
+ cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
- rd->rto_cpu = cpu;
+ rd->rto_cpu = cpu;
+ /* Do not send IPI to self */
+ } while (cpu == this_cpu);
if (cpu < nr_cpu_ids)
return cpu;
--
2.34.1Hello Chen, Steve,
On 1/5/2026 9:10 AM, Chen Jinghuang wrote:
> @@ -2118,10 +2119,13 @@ static int rto_next_cpu(struct root_domain *rd)
> */
> for (;;) {
>
> - /* When rto_cpu is -1 this acts like cpumask_first() */
> - cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
> + do {
> + /* When rto_cpu is -1 this acts like cpumask_first() */
> + cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
>
> - rd->rto_cpu = cpu;
> + rd->rto_cpu = cpu;
> + /* Do not send IPI to self */
> + } while (cpu == this_cpu);
nit.
Since we are already within an infinite for-loop, can't we simply do:
/* Do not send IPI to self */
if (cpu == this_cpu)
continue;
here and go evaluate cpumask_next() again instead of adding another
do-while? Was the nested loop intentional to highlight these bits
explicitly?
>
> if (cpu < nr_cpu_ids)
> return cpu;
--
Thanks and Regards,
Prateek
On Tue, 6 Jan 2026 14:11:57 +0530
K Prateek Nayak <kprateek.nayak@amd.com> wrote:
> Hello Chen, Steve,
>
> On 1/5/2026 9:10 AM, Chen Jinghuang wrote:
> > @@ -2118,10 +2119,13 @@ static int rto_next_cpu(struct root_domain *rd)
> > */
> > for (;;) {
> >
> > - /* When rto_cpu is -1 this acts like cpumask_first() */
> > - cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
> > + do {
> > + /* When rto_cpu is -1 this acts like cpumask_first() */
> > + cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
> >
> > - rd->rto_cpu = cpu;
> > + rd->rto_cpu = cpu;
> > + /* Do not send IPI to self */
> > + } while (cpu == this_cpu);
>
> nit.
>
> Since we are already within an infinite for-loop, can't we simply do:
>
> /* Do not send IPI to self */
> if (cpu == this_cpu)
> continue;
>
> here and go evaluate cpumask_next() again instead of adding another
> do-while? Was the nested loop intentional to highlight these bits
> explicitly?
Hmm, yeah, I think I added the loop to express this issue. But I think your
suggestion could work too. I originally had something like that, but for
some reason I thought it added an extra branch (over the added do { } while).
>
> >
> > if (cpu < nr_cpu_ids)
> > return cpu;
>
for (;;) {
/* When rto_cpu is -1 this acts like cpumask_first() */
cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
rd->rto_cpu = cpu;
+ /* Do not send IPI to self */
+ if (cpu == this_cpu)
+ continue;
+
if (cpu < nr_cpu_ids)
return cpu;
rd->rto_cpu = -1;
/*
* ACQUIRE ensures we see the @rto_mask changes
* made prior to the @next value observed.
*
* Matches WMB in rt_set_overload().
*/
next = atomic_read_acquire(&rd->rto_loop_next);
if (rd->rto_loop == next)
break;
rd->rto_loop = next;
}
Looks to be equivalent.
Chen, care to send a new version?
-- Steve
Peter,
Are you OK with this change? If so, care to take it?
-- Steve
On Mon, 5 Jan 2026 03:40:12 +0000
Chen Jinghuang <chenjinghuang2@huawei.com> wrote:
> CPU0 becomes overloaded when hosting a CPU-bound RT task, a non-CPU-bound
> RT task, and a CFS task stuck in kernel space. When other CPUs switch from
> RT to non-RT tasks, RT load balancing (LB) is triggered; with
> HAVE_RT_PUSH_IPI enabled, they send IPIs to CPU0 to drive the execution
> of rto_push_irq_work_func. During push_rt_task on CPU0,
> if next_task->prio < rq->donor->prio, resched_curr() sets NEED_RESCHED
> and after the push operation completes, CPU0 calls rto_next_cpu().
> Since only CPU0 is overloaded in this scenario, rto_next_cpu() should
> ideally return -1 (no further IPI needed).
>
> However, multiple CPUs invoking tell_cpu_to_push() during LB increments
> rd->rto_loop_next. Even when rd->rto_cpu is set to -1, the mismatch between
> rd->rto_loop and rd->rto_loop_next forces rto_next_cpu() to restart its
> search from -1. With CPU0 remaining overloaded (satisfying rt_nr_migratory
> && rt_nr_total > 1), it gets reselected, causing CPU0 to queue irq_work to
> itself and send self-IPIs repeatedly. As long as CPU0 stays overloaded and
> other CPUs run pull_rt_tasks(), it falls into an infinite self-IPI loop,
> which triggers a CPU hardlockup due to continuous self-interrupts.
>
> The trigging scenario is as follows:
>
> cpu0 cpu1 cpu2
> pull_rt_task
> tell_cpu_to_push
> <------------irq_work_queue_on
> rto_push_irq_work_func
> push_rt_task
> resched_curr(rq) pull_rt_task
> rto_next_cpu tell_cpu_to_push
> <-------------------------- atomic_inc(rto_loop_next)
> rd->rto_loop != next
> rto_next_cpu
> irq_work_queue_on
> rto_push_irq_work_func
>
> Fix redundant self-IPI by filtering the initiating CPU in rto_next_cpu().
> This solution has been verified to effectively eliminate spurious self-IPIs
> and prevent CPU hardlockup scenarios.
>
> Fixes: 4bdced5c9a29 ("sched/rt: Simplify the IPI based RT balancing logic")
> Suggested-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> Signed-off-by: Chen Jinghuang <chenjinghuang2@huawei.com>
> Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
>
> ---
> Changes since v2: https://lore.kernel.org/all/20251125083649.1814558-1-chenjinghuang2@huawei.com/
> - Replace the original "check NEED_RESCHED on target CPU"
> logic with "skip the currently executing CPU"
> - This modification eliminates self-IPIS
>
> Changes since v1: https://lore.kernel.org/all/20251121014004.564508-1-chenjinghuang2@huawei.com/
> - Remove unneeded extra whitespace
> - Add Reviewed-by tag from Steven Rostedt
> ---
> kernel/sched/rt.c | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
> index f1867fe8e5c5..ec10ed7bb75d 100644
> --- a/kernel/sched/rt.c
> +++ b/kernel/sched/rt.c
> @@ -2100,6 +2100,7 @@ static void push_rt_tasks(struct rq *rq)
> */
> static int rto_next_cpu(struct root_domain *rd)
> {
> + int this_cpu = smp_processor_id();
> int next;
> int cpu;
>
> @@ -2118,10 +2119,13 @@ static int rto_next_cpu(struct root_domain *rd)
> */
> for (;;) {
>
> - /* When rto_cpu is -1 this acts like cpumask_first() */
> - cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
> + do {
> + /* When rto_cpu is -1 this acts like cpumask_first() */
> + cpu = cpumask_next(rd->rto_cpu, rd->rto_mask);
>
> - rd->rto_cpu = cpu;
> + rd->rto_cpu = cpu;
> + /* Do not send IPI to self */
> + } while (cpu == this_cpu);
>
> if (cpu < nr_cpu_ids)
> return cpu;
© 2016 - 2026 Red Hat, Inc.