[v1] sched_ext: initialize built-in idle state before ops.init()

[PATCH sched_ext/for-6.15] sched_ext: initialize built-in idle state before ops.init()

Posted by Andrea Righi 8 months, 3 weeks ago

A BPF scheduler may want to use the built-in idle cpumasks in ops.init()
before the scheduler is fully initialized, either directly or through a
BPF timer for example.

However, this would result in an error, since the idle state has not
been properly initialized yet.

This can be easily verified by modifying scx_simple to call
scx_bpf_get_idle_cpumask() in ops.init():

$ sudo scx_simple

DEBUG DUMP
===========================================================================

scx_simple[121] triggered exit kind 1024:
  runtime error (built-in idle tracking is disabled)
...

Fix this by properly initializing the idle state before ops.init() is
called. With this change applied:

$ sudo scx_simple
local=2 global=0
local=19 global=11
local=23 global=11
...

Fixes: d73249f88743d ("sched_ext: idle: Make idle static keys private")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 kernel/sched/ext.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 06561d6717c9a..1ba02755ae8ad 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -5361,6 +5361,8 @@ static int scx_ops_enable(struct sched_ext_ops *ops, struct bpf_link *link)
 	 */
 	cpus_read_lock();
 
+	scx_idle_enable(ops);
+
 	if (scx_ops.init) {
 		ret = SCX_CALL_OP_RET(SCX_KF_UNLOCKED, init);
 		if (ret) {
@@ -5427,8 +5429,6 @@ static int scx_ops_enable(struct sched_ext_ops *ops, struct bpf_link *link)
 	if (scx_ops.cpu_acquire || scx_ops.cpu_release)
 		static_branch_enable(&scx_ops_cpu_preempt);
 
-	scx_idle_enable(ops);
-
 	/*
 	 * Lock out forks, cgroup on/offlining and moves before opening the
 	 * floodgate so that they don't wander into the operations prematurely.
-- 
2.49.0

Re: [PATCH sched_ext/for-6.15] sched_ext: initialize built-in idle state before ops.init()

Posted by Andrea Righi 8 months, 3 weeks ago

On Mon, Mar 24, 2025 at 09:57:53AM +0100, Andrea Righi wrote:
...
> diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> index 06561d6717c9a..1ba02755ae8ad 100644
> --- a/kernel/sched/ext.c
> +++ b/kernel/sched/ext.c
> @@ -5361,6 +5361,8 @@ static int scx_ops_enable(struct sched_ext_ops *ops, struct bpf_link *link)
>  	 */
>  	cpus_read_lock();
>  
> +	scx_idle_enable(ops);
> +

Actually, I just noticed a problem: if we call scx_idle_enable() under
cpus_read_lock() we may re-acquire cpu_hotplug_lock because of the
static_branch_enable/disable() calls, that are trying to re-acquire the
lock, which is not correct.

So, we either need to use static_branch_enable/disable_cpuslocked() or
place scx_idle_enable() outside of cpus_read_lock().

I just notice this from a lockdep splat on an arm64 machine (not sure why
lockdep was happy when I was testing this in vng):

[   65.974439] WARNING: possible recursive locking detected
...
[   65.983540] --------------------------------------------
[   65.989039] scx_bpfland/3883 is trying to acquire lock:
[   65.994447] ffffb80a490991d8 (cpu_hotplug_lock){++++}-{0:0}, at: cpus_read_lock+0x18/0x30
[   66.002941]
               but task is already holding lock:
[   66.008978] ffffb80a490991d8 (cpu_hotplug_lock){++++}-{0:0}, at: cpus_read_lock+0x18/0x30
[   66.017455]
               other info that might help us debug this:
[   66.024212]  Possible unsafe locking scenario:

[   66.030338]        CPU0
[   66.032855]        ----
[   66.035372]   lock(cpu_hotplug_lock);
[   66.039154]   lock(cpu_hotplug_lock);
[   66.042935]
                *** DEADLOCK ***

Anyway, please ignore this patch, I'll send a new one soon.

-Andrea

Re: [PATCH sched_ext/for-6.15] sched_ext: initialize built-in idle state before ops.init()

Posted by Changwoo Min 8 months, 3 weeks ago

Hi Andrea,

This is a nice catch! Looks good to me.

Regards,
Changwoo Min

On 2025-03-24 17:57, Andrea Righi wrote:
> A BPF scheduler may want to use the built-in idle cpumasks in ops.init()
> before the scheduler is fully initialized, either directly or through a
> BPF timer for example.
> 
> However, this would result in an error, since the idle state has not
> been properly initialized yet.
> 
> This can be easily verified by modifying scx_simple to call
> scx_bpf_get_idle_cpumask() in ops.init():
> 
> $ sudo scx_simple
> 
> DEBUG DUMP
> ===========================================================================
> 
> scx_simple[121] triggered exit kind 1024:
>   runtime error (built-in idle tracking is disabled)
> ...
> 
> Fix this by properly initializing the idle state before ops.init() is
> called. With this change applied:
> 
> $ sudo scx_simple
> local=2 global=0
> local=19 global=11
> local=23 global=11
> ...
> 
> Fixes: d73249f88743d ("sched_ext: idle: Make idle static keys private")
> Signed-off-by: Andrea Righi <arighi@nvidia.com>
> ---
>  kernel/sched/ext.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> index 06561d6717c9a..1ba02755ae8ad 100644
> --- a/kernel/sched/ext.c
> +++ b/kernel/sched/ext.c
> @@ -5361,6 +5361,8 @@ static int scx_ops_enable(struct sched_ext_ops *ops, struct bpf_link *link)
>  	 */
>  	cpus_read_lock();
>  
> +	scx_idle_enable(ops);
> +
>  	if (scx_ops.init) {
>  		ret = SCX_CALL_OP_RET(SCX_KF_UNLOCKED, init);
>  		if (ret) {
> @@ -5427,8 +5429,6 @@ static int scx_ops_enable(struct sched_ext_ops *ops, struct bpf_link *link)
>  	if (scx_ops.cpu_acquire || scx_ops.cpu_release)
>  		static_branch_enable(&scx_ops_cpu_preempt);
>  
> -	scx_idle_enable(ops);
> -
>  	/*
>  	 * Lock out forks, cgroup on/offlining and moves before opening the
>  	 * floodgate so that they don't wander into the operations prematurely.