kernel/sched/core.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
From: Peilin He <he.peilin@zte.com.cn>
Background
==========
When repeated migrate_disable() calls are made with missing the
corresponding migrate_enable() calls, there is a risk of
'migration_disabled' going upper overflow because
'migration_disabled' is a type of unsigned short whose max value is
65535.
In PREEMPT_RT kernel, if 'migration_disabled' goes upper overflow, it may
make the migrate_disable() ineffective within local_lock_irqsave(). This
is because, during the scheduling procedure, the value of
'migration_disabled' will be checked, which can trigger CPU migration.
Consequently, the count of 'rcu_read_lock_nesting' may leak due to
local_lock_irqsave() and local_unlock_irqrestore() occurring on different
CPUs.
Usecase
========
For example, When I developed a driver, I encountered a warning like
"WARNING: CPU: 4 PID: 260 at kernel/rcu/tree_plugin.h:315
rcu_note_context_switch+0xa8/0x4e8" warning. It took me half a month
to locate this issue. Ultimately, I discovered that the lack of upper
overflow detection mechanism in migrate_disable() was the root cause,
leading to a significant amount of time spent on problem localization.
If the upper overflow detection mechanism was added to migrate_disable(),
the root cause could be very quickly and easily identified.
Effect
======
Using WARN_ON_ONCE() to check if 'migration_disabled' is upper overflow
can help developers identify the issue quickly.
Signed-off-by: Peilin He<he.peilin@zte.com.cn>
Signed-off-by: xu xin <xu.xin16@zte.com.cn>
Reviewed-by: Yunkai Zhang <zhang.yunkai@zte.com.cn>
Reviewed-by: Qiang Tu <tu.qiang35@zte.com.cn>
Reviewed-by: Kun Jiang <jiang.kun2@zte.com.cn>
Reviewed-by: Fan Yu <fan.yu9@zte.com.cn>
Cc: Yang Yang <yang.yang29@zte.com.cn>
Cc: Liu Chun <liu.chun2@zte.com.cn>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
---
v2->v3:
Some fixes according to:
https://lore.kernel.org/all/20240704134716.GU11386@noisy.programming.kicks-ass.net/
1.Convert p->migration_disabled to a signed type and check earlier.
2.Add overflow check for p->migration_disabled in migrate.enable().
3.Check for overflow on debug builds.
v1->v2:
Some fixes according to:
https://lore.kernel.org/all/20240702124334.762dbd5a@rorschach.local.home/
1.Merge if conditions into WARN().
2.Remove the newline character '\n'. Right, we don't need the redundant \n.
kernel/sched/core.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 8cc4975d6b2b..1992f2848732 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2259,6 +2259,12 @@ void migrate_disable(void)
struct task_struct *p = current;
if (p->migration_disabled) {
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ *Warn about overflow half-way through the range.
+ */
+ WARN_ON_ONCE((s16)p->migration_disabled < 0);
+#endif
p->migration_disabled++;
return;
}
@@ -2277,14 +2283,20 @@ void migrate_enable(void)
.flags = SCA_MIGRATE_ENABLE,
};
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ * Check both overflow from migrate_disable() and superfluous
+ * migrate_enable().
+ */
+ if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
+ return;
+#endif
+
if (p->migration_disabled > 1) {
p->migration_disabled--;
return;
}
- if (WARN_ON_ONCE(!p->migration_disabled))
- return;
-
/*
* Ensure stop_task runs either before or after this, and that
* __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
--
2.17.1On Tue, Jul 16, 2024 at 10:42:44AM +0800, xu.xin16@zte.com.cn wrote: > From: Peilin He <he.peilin@zte.com.cn> > > Background > ========== > When repeated migrate_disable() calls are made with missing the > corresponding migrate_enable() calls, there is a risk of > 'migration_disabled' going upper overflow because > 'migration_disabled' is a type of unsigned short whose max value is > 65535. > > In PREEMPT_RT kernel, if 'migration_disabled' goes upper overflow, it may > make the migrate_disable() ineffective within local_lock_irqsave(). This > is because, during the scheduling procedure, the value of > 'migration_disabled' will be checked, which can trigger CPU migration. > Consequently, the count of 'rcu_read_lock_nesting' may leak due to > local_lock_irqsave() and local_unlock_irqrestore() occurring on different > CPUs. > > Usecase > ======== > For example, When I developed a driver, I encountered a warning like > "WARNING: CPU: 4 PID: 260 at kernel/rcu/tree_plugin.h:315 > rcu_note_context_switch+0xa8/0x4e8" warning. It took me half a month > to locate this issue. Ultimately, I discovered that the lack of upper > overflow detection mechanism in migrate_disable() was the root cause, > leading to a significant amount of time spent on problem localization. > > If the upper overflow detection mechanism was added to migrate_disable(), > the root cause could be very quickly and easily identified. > > Effect > ====== > Using WARN_ON_ONCE() to check if 'migration_disabled' is upper overflow > can help developers identify the issue quickly. > > Signed-off-by: Peilin He<he.peilin@zte.com.cn> > Signed-off-by: xu xin <xu.xin16@zte.com.cn> > Reviewed-by: Yunkai Zhang <zhang.yunkai@zte.com.cn> > Reviewed-by: Qiang Tu <tu.qiang35@zte.com.cn> > Reviewed-by: Kun Jiang <jiang.kun2@zte.com.cn> > Reviewed-by: Fan Yu <fan.yu9@zte.com.cn> > Cc: Yang Yang <yang.yang29@zte.com.cn> > Cc: Liu Chun <liu.chun2@zte.com.cn> > Suggested-by: Peter Zijlstra <peterz@infradead.org> Thanks, I'll queue this for sched/urgent once -rc1 rolls around.
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 0ec8d5aed4d14055aab4e2746def33f8b0d409c3
Gitweb: https://git.kernel.org/tip/0ec8d5aed4d14055aab4e2746def33f8b0d409c3
Author: Peilin He <he.peilin@zte.com.cn>
AuthorDate: Tue, 16 Jul 2024 10:42:44 +08:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Mon, 29 Jul 2024 12:22:34 +02:00
sched/core: Add WARN_ON_ONCE() to check overflow for migrate_disable()
Background
==========
When repeated migrate_disable() calls are made with missing the
corresponding migrate_enable() calls, there is a risk of
'migration_disabled' going upper overflow because
'migration_disabled' is a type of unsigned short whose max value is
65535.
In PREEMPT_RT kernel, if 'migration_disabled' goes upper overflow, it may
make the migrate_disable() ineffective within local_lock_irqsave(). This
is because, during the scheduling procedure, the value of
'migration_disabled' will be checked, which can trigger CPU migration.
Consequently, the count of 'rcu_read_lock_nesting' may leak due to
local_lock_irqsave() and local_unlock_irqrestore() occurring on different
CPUs.
Usecase
========
For example, When I developed a driver, I encountered a warning like
"WARNING: CPU: 4 PID: 260 at kernel/rcu/tree_plugin.h:315
rcu_note_context_switch+0xa8/0x4e8" warning. It took me half a month
to locate this issue. Ultimately, I discovered that the lack of upper
overflow detection mechanism in migrate_disable() was the root cause,
leading to a significant amount of time spent on problem localization.
If the upper overflow detection mechanism was added to migrate_disable(),
the root cause could be very quickly and easily identified.
Effect
======
Using WARN_ON_ONCE() to check if 'migration_disabled' is upper overflow
can help developers identify the issue quickly.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Peilin He<he.peilin@zte.com.cn>
Signed-off-by: xu xin <xu.xin16@zte.com.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Yunkai Zhang <zhang.yunkai@zte.com.cn>
Reviewed-by: Qiang Tu <tu.qiang35@zte.com.cn>
Reviewed-by: Kun Jiang <jiang.kun2@zte.com.cn>
Reviewed-by: Fan Yu <fan.yu9@zte.com.cn>
Link: https://lkml.kernel.org/r/20240716104244764N2jD8gnBpnsLjCDnQGQ8c@zte.com.cn
---
kernel/sched/core.c | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2c61b4f..db5823f 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2233,6 +2233,12 @@ void migrate_disable(void)
struct task_struct *p = current;
if (p->migration_disabled) {
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ *Warn about overflow half-way through the range.
+ */
+ WARN_ON_ONCE((s16)p->migration_disabled < 0);
+#endif
p->migration_disabled++;
return;
}
@@ -2251,14 +2257,20 @@ void migrate_enable(void)
.flags = SCA_MIGRATE_ENABLE,
};
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ * Check both overflow from migrate_disable() and superfluous
+ * migrate_enable().
+ */
+ if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
+ return;
+#endif
+
if (p->migration_disabled > 1) {
p->migration_disabled--;
return;
}
- if (WARN_ON_ONCE(!p->migration_disabled))
- return;
-
/*
* Ensure stop_task runs either before or after this, and that
* __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().© 2016 - 2025 Red Hat, Inc.