1. Patchew
  2. linux
  3. View series

[PATCH] scx: Fix maximal BPF selftest prog

David Vernet posted 1 patch 1 year ago 
tools/testing/selftests/sched_ext/maximal.bpf.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
[PATCH] scx: Fix maximal BPF selftest prog
Posted by David Vernet 1 year ago 
maximal.bpf.c is still dispatching to and consuming from SCX_DSQ_GLOBAL.
Let's have it use its own DSQ to avoid any runtime errors.

Signed-off-by: David Vernet <void@manifault.com>
---
 tools/testing/selftests/sched_ext/maximal.bpf.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/sched_ext/maximal.bpf.c b/tools/testing/selftests/sched_ext/maximal.bpf.c
index 4c005fa71810..430f5e13bf55 100644
--- a/tools/testing/selftests/sched_ext/maximal.bpf.c
+++ b/tools/testing/selftests/sched_ext/maximal.bpf.c
@@ -12,6 +12,8 @@
 
 char _license[] SEC("license") = "GPL";
 
+#define DSQ_ID 0
+
 s32 BPF_STRUCT_OPS(maximal_select_cpu, struct task_struct *p, s32 prev_cpu,
 		   u64 wake_flags)
 {
@@ -20,7 +22,7 @@ s32 BPF_STRUCT_OPS(maximal_select_cpu, struct task_struct *p, s32 prev_cpu,
 
 void BPF_STRUCT_OPS(maximal_enqueue, struct task_struct *p, u64 enq_flags)
 {
-	scx_bpf_dsq_insert(p, SCX_DSQ_GLOBAL, SCX_SLICE_DFL, enq_flags);
+	scx_bpf_dsq_insert(p, DSQ_ID, SCX_SLICE_DFL, enq_flags);
 }
 
 void BPF_STRUCT_OPS(maximal_dequeue, struct task_struct *p, u64 deq_flags)
@@ -28,7 +30,7 @@ void BPF_STRUCT_OPS(maximal_dequeue, struct task_struct *p, u64 deq_flags)
 
 void BPF_STRUCT_OPS(maximal_dispatch, s32 cpu, struct task_struct *prev)
 {
-	scx_bpf_dsq_move_to_local(SCX_DSQ_GLOBAL);
+	scx_bpf_dsq_move_to_local(DSQ_ID);
 }
 
 void BPF_STRUCT_OPS(maximal_runnable, struct task_struct *p, u64 enq_flags)
@@ -123,7 +125,7 @@ void BPF_STRUCT_OPS(maximal_cgroup_set_weight, struct cgroup *cgrp, u32 weight)
 
 s32 BPF_STRUCT_OPS_SLEEPABLE(maximal_init)
 {
-	return 0;
+	return scx_bpf_create_dsq(DSQ_ID, -1);
 }
 
 void BPF_STRUCT_OPS(maximal_exit, struct scx_exit_info *info)
-- 
2.46.1
Re: [PATCH] scx: Fix maximal BPF selftest prog
Posted by Tejun Heo 1 year ago 
On Mon, Dec 09, 2024 at 09:29:24AM -0600, David Vernet wrote:
> maximal.bpf.c is still dispatching to and consuming from SCX_DSQ_GLOBAL.
> Let's have it use its own DSQ to avoid any runtime errors.
> 
> Signed-off-by: David Vernet <void@manifault.com>

Applied to sched_ext/for-6.13-fixes.

Thanks.

-- 
tejun
Re: [PATCH] scx: Fix maximal BPF selftest prog
Posted by Ihor Solodrai 1 year ago 
On Monday, December 9th, 2024 at 7:29 AM, David Vernet <void@manifault.com> wrote:

> 
> 
> maximal.bpf.c is still dispatching to and consuming from SCX_DSQ_GLOBAL.
> Let's have it use its own DSQ to avoid any runtime errors.
> 
> Signed-off-by: David Vernet void@manifault.com
> 
> ---
> tools/testing/selftests/sched_ext/maximal.bpf.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)

Hi David, thanks for looking into the test failures.

I ran BPF CI on kernel with this change on top of bpf-next (6e8ba494d87d) + tests build fix [1].

I ran the workflow twice. A second run on aarch64 failed:
https://github.com/kernel-patches/vmtest/actions/runs/12263276418/job/34217056880

Failed tests are "dsp_local_on" and "exit".

The difference in comparison to a successful run is a deadlock, which
happens on attempt to attach a scheduler, assuming I read the log
correctly. See a paste below.

I'd like to note that the selftests/sched_ext/runner log is very
verbose (~23M), although I can't tell how necessary this is for
debugging.

[1] https://lore.kernel.org/bpf/20241121214014.3346203-1-ihor.solodrai@pm.me/

2024-12-10T20:20:55.6921607Z ##[group]selftests/sched_ext - Executing selftests/sched_ext/runner
2024-12-10T20:20:55.7556144Z 
2024-12-10T20:20:55.7558626Z ================================
2024-12-10T20:20:55.7560796Z WARNING: inconsistent lock state
2024-12-10T20:20:55.7564132Z 6.13.0-rc2-gb23d77932777-dirty #1 Not tainted
2024-12-10T20:20:55.7566359Z --------------------------------
2024-12-10T20:20:55.7570239Z inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
2024-12-10T20:20:55.7573094Z runner/96 [HC0[0]:SC0[0]:HE1:SE1] takes:
2024-12-10T20:20:55.7579001Z ffff0000fb5f4858 (&rq->__lock){?.-.}-{2:2}, at: raw_spin_rq_lock_nested+0x2c/0x50
2024-12-10T20:20:55.7581586Z {IN-HARDIRQ-W} state was registered at:
2024-12-10T20:20:55.7583722Z   __lock_acquire+0x3c0/0xb18
2024-12-10T20:20:55.7586124Z   lock_acquire.part.0+0xe0/0x250
2024-12-10T20:20:55.7588061Z   lock_acquire+0x88/0x160
2024-12-10T20:20:55.7590605Z   _raw_spin_lock_nested+0x5c/0xb0
2024-12-10T20:20:55.7592640Z   try_to_wake_up+0x244/0x560
2024-12-10T20:20:55.7594719Z   wake_up_process+0x20/0x38
2024-12-10T20:20:55.7596673Z   hrtimer_wakeup+0x28/0x48
2024-12-10T20:20:55.7598757Z   __run_hrtimer+0x32c/0x3a0
2024-12-10T20:20:55.7601330Z   __hrtimer_run_queues+0xb4/0x140
2024-12-10T20:20:55.7603618Z   hrtimer_run_queues+0xe4/0x1d8
2024-12-10T20:20:55.7606009Z   update_process_times+0x3c/0x160
2024-12-10T20:20:55.7607957Z   tick_periodic+0x44/0x140
2024-12-10T20:20:55.7610565Z   tick_handle_periodic+0x34/0xa8
2024-12-10T20:20:55.7613159Z   arch_timer_handler_virt+0x34/0x58
2024-12-10T20:20:55.7615811Z   handle_percpu_devid_irq+0xb0/0x2a0
2024-12-10T20:20:55.7618562Z   generic_handle_domain_irq+0x34/0x58
2024-12-10T20:20:55.7620462Z   gic_handle_irq+0x5c/0xd8
2024-12-10T20:20:55.7622667Z   call_on_irq_stack+0x24/0x58
2024-12-10T20:20:55.7625060Z   do_interrupt_handler+0x88/0x98
2024-12-10T20:20:55.7626970Z   el1_interrupt+0x34/0x50
2024-12-10T20:20:55.7629327Z   el1h_64_irq_handler+0x18/0x28
2024-12-10T20:20:55.7631094Z   el1h_64_irq+0x6c/0x70
2024-12-10T20:20:55.7633416Z   default_idle_call+0xac/0x254
2024-12-10T20:20:55.7635665Z   default_idle_call+0xa8/0x254
2024-12-10T20:20:55.7637184Z   do_idle+0xc4/0x118
2024-12-10T20:20:55.7639486Z   cpu_startup_entry+0x3c/0x50
2024-12-10T20:20:55.7641200Z   rest_init+0x118/0x1a8
2024-12-10T20:20:55.7643382Z   start_kernel+0x59c/0x678
2024-12-10T20:20:55.7645786Z   __primary_switched+0x88/0x98
2024-12-10T20:20:55.7647582Z irq event stamp: 33739
2024-12-10T20:20:55.7653885Z hardirqs last  enabled at (33739): [<ffff8000800b699c>] scx_ops_bypass+0x174/0x3b8
2024-12-10T20:20:55.7660311Z hardirqs last disabled at (33738): [<ffff800080d48ad4>] _raw_spin_lock_irqsave+0xb4/0xd8
2024-12-10T20:20:55.7666393Z softirqs last  enabled at (32824): [<ffff800080056a34>] handle_softirqs+0x46c/0x490
2024-12-10T20:20:55.7672087Z softirqs last disabled at (32815): [<ffff8000800101bc>] __do_softirq+0x1c/0x28
2024-12-10T20:20:55.7672575Z 
2024-12-10T20:20:55.7675191Z other info that might help us debug this:
2024-12-10T20:20:55.7678564Z  Possible unsafe locking scenario:
2024-12-10T20:20:55.7678867Z 
2024-12-10T20:20:55.7680124Z        CPU0
2024-12-10T20:20:55.7680709Z        ----
2024-12-10T20:20:55.7682424Z   lock(&rq->__lock);
2024-12-10T20:20:55.7683722Z   <Interrupt>
2024-12-10T20:20:55.7685546Z     lock(&rq->__lock);
2024-12-10T20:20:55.7685788Z 
2024-12-10T20:20:55.7687156Z  *** DEADLOCK ***
2024-12-10T20:20:55.7687380Z 
2024-12-10T20:20:55.7689485Z 3 locks held by runner/96:
2024-12-10T20:20:55.7696730Z  #0: ffff8000821226f8 (update_mutex){+.+.}-{4:4}, at: bpf_struct_ops_link_create+0x13c/0x1b0
2024-12-10T20:20:55.7703773Z  #1: ffff80008207e7f0 (scx_ops_enable_mutex){+.+.}-{4:4}, at: scx_ops_enable.isra.0+0x88/0xaa8
2024-12-10T20:20:55.7709594Z  #2: ffff80008207e060 (bypass_lock){+.+.}-{2:2}, at: scx_ops_bypass+0x50/0x3b8
2024-12-10T20:20:55.7710078Z 
2024-12-10T20:20:55.7710895Z stack backtrace:
2024-12-10T20:20:55.7717184Z CPU: 1 UID: 0 PID: 96 Comm: runner Not tainted 6.13.0-rc2-gb23d77932777-dirty #1
2024-12-10T20:20:55.7719782Z Hardware name: linux,dummy-virt (DT)
2024-12-10T20:20:55.7722429Z Sched_ext: create_dsq (enabling)
2024-12-10T20:20:55.7723286Z Call trace:
2024-12-10T20:20:55.7725440Z  show_stack+0x20/0x38 (C)
2024-12-10T20:20:55.7727531Z  dump_stack_lvl+0xa0/0xf0
2024-12-10T20:20:55.7729283Z  dump_stack+0x18/0x28
2024-12-10T20:20:55.7732218Z  print_usage_bug.part.0+0x270/0x320
2024-12-10T20:20:55.7734277Z  mark_lock_irq+0x3e0/0x538
2024-12-10T20:20:55.7736126Z  mark_lock+0x1c0/0x290
2024-12-10T20:20:55.7738113Z  mark_usage+0x108/0x170
2024-12-10T20:20:55.7740306Z  __lock_acquire+0x3c0/0xb18
2024-12-10T20:20:55.7742923Z  lock_acquire.part.0+0xe0/0x250
2024-12-10T20:20:55.7744782Z  lock_acquire+0x88/0x160
2024-12-10T20:20:55.7747510Z  _raw_spin_lock_nested+0x5c/0xb0
2024-12-10T20:20:55.7750224Z  raw_spin_rq_lock_nested+0x2c/0x50
2024-12-10T20:20:55.7752314Z  scx_ops_bypass+0xfc/0x3b8
2024-12-10T20:20:55.7755093Z  scx_ops_enable.isra.0+0x2c8/0xaa8
2024-12-10T20:20:55.7756819Z  bpf_scx_reg+0x18/0x30
2024-12-10T20:20:55.7760072Z  bpf_struct_ops_link_create+0x154/0x1b0
2024-12-10T20:20:55.7762017Z  link_create+0x14c/0x350
2024-12-10T20:20:55.7763821Z  __sys_bpf+0x3fc/0xb90
2024-12-10T20:20:55.7765976Z  __arm64_sys_bpf+0x2c/0x48
2024-12-10T20:20:55.7768134Z  invoke_syscall+0x50/0x120
2024-12-10T20:20:55.7771191Z  el0_svc_common.constprop.0+0x48/0xf0
2024-12-10T20:20:55.7772796Z  do_el0_svc+0x24/0x38
2024-12-10T20:20:55.7774452Z  el0_svc+0x48/0x110
2024-12-10T20:20:55.7777321Z  el0t_64_sync_handler+0x10c/0x138
2024-12-10T20:20:55.7779253Z  el0t_64_sync+0x198/0x1a0
2024-12-10T20:20:55.7782673Z ------------[ cut here ]------------
2024-12-10T20:20:55.7786247Z raw_local_irq_restore() called with IRQs enabled
2024-12-10T20:20:55.7792987Z WARNING: CPU: 1 PID: 96 at kernel/locking/irqflag-debug.c:10 warn_bogus_irq_restore+0x30/0x40
2024-12-10T20:20:55.7794104Z Modules linked in:
2024-12-10T20:20:55.7799949Z CPU: 1 UID: 0 PID: 96 Comm: runner Not tainted 6.13.0-rc2-gb23d77932777-dirty #1
2024-12-10T20:20:55.7802378Z Hardware name: linux,dummy-virt (DT)
2024-12-10T20:20:55.7804767Z Sched_ext: create_dsq (enabling)
2024-12-10T20:20:55.7809450Z pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
2024-12-10T20:20:55.7812011Z pc : warn_bogus_irq_restore+0x30/0x40
2024-12-10T20:20:55.7814814Z lr : warn_bogus_irq_restore+0x30/0x40
2024-12-10T20:20:55.7816303Z sp : ffff800084d53a50
2024-12-10T20:20:55.7821165Z x29: ffff800084d53a50 x28: ffff0000c0960000 x27: ffff80008205d078
2024-12-10T20:20:55.7825770Z x26: ffff800081cf6840 x25: ffff800082059350 x24: ffff800082256958
2024-12-10T20:20:55.7830346Z x23: 0000000000000000 x22: ffff0000fb5f4858 x21: ffff80008207d6b8
2024-12-10T20:20:55.7834929Z x20: 0000000000000001 x19: ffff0000fb5f4840 x18: 00000000fffffffd
2024-12-10T20:20:55.7839534Z x17: 3764333262672d32 x16: 63722d302e33312e x15: ffff800084d52c90
2024-12-10T20:20:55.7844210Z x14: 0000000000000000 x13: 64656c62616e6520 x12: 7351524920687469
2024-12-10T20:20:55.7849054Z x11: 772064656c6c6163 x10: ffff8000820da8b0 x9 : ffff800080101b00
2024-12-10T20:20:55.7854102Z x8 : 00000000ffffefff x7 : ffff8000820da8b0 x6 : 80000000fffff000
2024-12-10T20:20:55.7858669Z x5 : 000000000000017f x4 : 0000000000000000 x3 : 0000000000000000
2024-12-10T20:20:55.7863487Z x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff0000c0960000
2024-12-10T20:20:55.7864284Z Call trace:
2024-12-10T20:20:55.7867341Z  warn_bogus_irq_restore+0x30/0x40 (P)
2024-12-10T20:20:55.7870193Z  warn_bogus_irq_restore+0x30/0x40 (L)
2024-12-10T20:20:55.7872308Z  scx_ops_bypass+0x224/0x3b8
2024-12-10T20:20:55.7875067Z  scx_ops_enable.isra.0+0x2c8/0xaa8
2024-12-10T20:20:55.7876915Z  bpf_scx_reg+0x18/0x30
2024-12-10T20:20:55.7880088Z  bpf_struct_ops_link_create+0x154/0x1b0
2024-12-10T20:20:55.7881875Z  link_create+0x14c/0x350
2024-12-10T20:20:55.7883755Z  __sys_bpf+0x3fc/0xb90
2024-12-10T20:20:55.7885864Z  __arm64_sys_bpf+0x2c/0x48
2024-12-10T20:20:55.7887978Z  invoke_syscall+0x50/0x120
2024-12-10T20:20:55.7890961Z  el0_svc_common.constprop.0+0x48/0xf0
2024-12-10T20:20:55.7892546Z  do_el0_svc+0x24/0x38
2024-12-10T20:20:55.7894224Z  el0_svc+0x48/0x110
2024-12-10T20:20:55.7896940Z  el0t_64_sync_handler+0x10c/0x138
2024-12-10T20:20:55.7898846Z  el0t_64_sync+0x198/0x1a0
2024-12-10T20:20:55.7900627Z irq event stamp: 33739
2024-12-10T20:20:55.7906972Z hardirqs last  enabled at (33739): [<ffff8000800b699c>] scx_ops_bypass+0x174/0x3b8
2024-12-10T20:20:55.7913438Z hardirqs last disabled at (33738): [<ffff800080d48ad4>] _raw_spin_lock_irqsave+0xb4/0xd8
2024-12-10T20:20:55.7919488Z softirqs last  enabled at (32824): [<ffff800080056a34>] handle_softirqs+0x46c/0x490
2024-12-10T20:20:55.7925161Z softirqs last disabled at (32815): [<ffff8000800101bc>] __do_softirq+0x1c/0x28
2024-12-10T20:20:55.7927698Z ---[ end trace 0000000000000000 ]---
2024-12-10T20:20:55.8646931Z sched_ext: BPF scheduler "create_dsq" enabled
2024-12-10T20:20:55.9047921Z sched_ext: BPF scheduler "create_dsq" disabled (unregistered from user space)
2024-12-10T20:20:55.9647423Z sched_ext: BPF scheduler "enq_last_no_enq_fails" disabled (runtime error)
2024-12-10T20:20:55.9653845Z sched_ext: enq_last_no_enq_fails: SCX_OPS_ENQ_LAST requires ops.enqueue() to be implemented
2024-12-10T20:20:55.9656238Z    scx_ops_enable.isra.0+0xa98/0xaa8
2024-12-10T20:20:55.9658038Z    bpf_scx_reg+0x18/0x30
2024-12-10T20:20:55.9661235Z    bpf_struct_ops_link_create+0x154/0x1b0
2024-12-10T20:20:55.9663113Z    link_create+0x14c/0x350
2024-12-10T20:20:55.9664999Z    __sys_bpf+0x3fc/0xb90
2024-12-10T20:20:55.9667216Z    __arm64_sys_bpf+0x2c/0x48
2024-12-10T20:20:55.9669376Z    invoke_syscall+0x50/0x120
2024-12-10T20:20:55.9672393Z    el0_svc_common.constprop.0+0x48/0xf0
2024-12-10T20:20:55.9674125Z    do_el0_svc+0x24/0x38
2024-12-10T20:20:55.9675781Z    el0_svc+0x48/0x110
2024-12-10T20:20:55.9678659Z    el0t_64_sync_handler+0x10c/0x138
2024-12-10T20:20:55.9680560Z    el0t_64_sync+0x198/0x1a0
2024-12-10T20:20:55.9864103Z sched_ext: BPF scheduler "enq_select_cpu_fails" enabled
2024-12-10T20:20:56.0347550Z sched_ext: BPF scheduler "enq_select_cpu_fails" disabled (runtime error)
2024-12-10T20:20:56.0354154Z sched_ext: enq_select_cpu_fails: kfunc with mask 0x8 called from an operation only allowing 0x4
2024-12-10T20:20:56.0356414Z    scx_bpf_select_cpu_dfl+0x8c/0x98
2024-12-10T20:20:56.0361395Z    bpf_prog_b68dda6ca71e9089_enq_select_cpu_fails_enqueue+0x54/0xb0
2024-12-10T20:20:56.0364138Z    bpf__sched_ext_ops_enqueue+0x50/0x74
2024-12-10T20:20:56.0366225Z    do_enqueue_task+0xf8/0x240
2024-12-10T20:20:56.0368567Z    enqueue_task_scx+0x190/0x288
2024-12-10T20:20:56.0370532Z    enqueue_task+0x44/0xe8
2024-12-10T20:20:56.0372845Z    ttwu_do_activate+0x88/0x298
2024-12-10T20:20:56.0375059Z    try_to_wake_up+0x2b8/0x560
2024-12-10T20:20:56.0377269Z    wake_up_process+0x20/0x38
2024-12-10T20:20:56.0379030Z    kick_pool+0xa4/0x190
2024-12-10T20:20:56.0381198Z    __queue_work+0x540/0x600
2024-12-10T20:20:56.0383276Z    queue_work_on+0xb8/0x100
2024-12-10T20:20:56.0385314Z    bpf_prog_free+0x90/0xa8
2024-12-10T20:20:56.0387681Z    __bpf_prog_put_rcu+0x44/0x60
2024-12-10T20:20:56.0389757Z    rcu_do_batch+0x1e8/0xa90
2024-12-10T20:20:56.0391541Z    rcu_core+0x174/0x378
2024-12-10T20:20:56.0393744Z    rcu_core_si+0x18/0x30
2024-12-10T20:20:56.0395820Z    handle_softirqs+0x12c/0x490
2024-12-10T20:20:56.0397644Z    __do_softirq+0x1c/0x28
2024-12-10T20:20:56.0399798Z    ____do_softirq+0x18/0x30
2024-12-10T20:20:56.0402104Z    call_on_irq_stack+0x24/0x58
2024-12-10T20:20:56.0404662Z    do_softirq_own_stack+0x24/0x38
2024-12-10T20:20:56.0406780Z    __irq_exit_rcu+0x140/0x180
2024-12-10T20:20:56.0408766Z    irq_exit_rcu+0x18/0x48
2024-12-10T20:20:56.0410801Z    el1_interrupt+0x38/0x50
2024-12-10T20:20:56.0413210Z    el1h_64_irq_handler+0x18/0x28
2024-12-10T20:20:56.0415052Z    el1h_64_irq+0x6c/0x70
2024-12-10T20:20:56.0417472Z    default_idle_call+0xac/0x254
2024-12-10T20:20:56.0419830Z    default_idle_call+0xa8/0x254
2024-12-10T20:20:56.0421507Z    do_idle+0xc4/0x118
2024-12-10T20:20:56.0423811Z    cpu_startup_entry+0x3c/0x50
2024-12-10T20:20:56.0426578Z    secondary_start_kernel+0xdc/0x108
2024-12-10T20:20:56.0428987Z    __secondary_switched+0xc0/0xc8
2024-12-10T20:20:57.1047514Z sched_ext: BPF scheduler "ddsp_bogus_dsq_fail" enabled
2024-12-10T20:20:57.1647261Z sched_ext: BPF scheduler "ddsp_bogus_dsq_fail" disabled (runtime error)
2024-12-10T20:20:57.1652666Z sched_ext: ddsp_bogus_dsq_fail: non-existent DSQ 0xcafef00d for rcu_sched[16]
2024-12-10T20:20:57.1655003Z    find_dsq_for_dispatch+0xf4/0x158
2024-12-10T20:20:57.1657196Z    direct_dispatch+0x58/0x228
2024-12-10T20:20:57.1659464Z    do_enqueue_task+0x17c/0x240
2024-12-10T20:20:57.1661801Z    enqueue_task_scx+0x190/0x288
2024-12-10T20:20:57.1663937Z    enqueue_task+0x44/0xe8
2024-12-10T20:20:57.1666068Z    ttwu_do_activate+0x88/0x298
2024-12-10T20:20:57.1668231Z    try_to_wake_up+0x2b8/0x560
2024-12-10T20:20:57.1670117Z    swake_up_one+0x4c/0x98
2024-12-10T20:20:57.1672701Z    swake_up_one_online+0x54/0xc0
2024-12-10T20:20:57.1674986Z    rcu_gp_kthread_wake+0x68/0x98
2024-12-10T20:20:57.1678136Z    rcu_accelerate_cbs_unlocked+0xf0/0x128
2024-12-10T20:20:57.1679813Z    rcu_core+0x350/0x378
2024-12-10T20:20:57.1681646Z    rcu_core_si+0x18/0x30
2024-12-10T20:20:57.1683953Z    handle_softirqs+0x12c/0x490
2024-12-10T20:20:57.1685841Z    __do_softirq+0x1c/0x28
2024-12-10T20:20:57.1687901Z    ____do_softirq+0x18/0x30
2024-12-10T20:20:57.1690172Z    call_on_irq_stack+0x24/0x58
2024-12-10T20:20:57.1692602Z    do_softirq_own_stack+0x24/0x38
2024-12-10T20:20:57.1694778Z    __irq_exit_rcu+0x140/0x180
2024-12-10T20:20:57.1696640Z    irq_exit_rcu+0x18/0x48
2024-12-10T20:20:57.1698672Z    el1_interrupt+0x38/0x50
2024-12-10T20:20:57.1701082Z    el1h_64_irq_handler+0x18/0x28
2024-12-10T20:20:57.1702858Z    el1h_64_irq+0x6c/0x70
2024-12-10T20:20:57.1705280Z    default_idle_call+0xac/0x254
2024-12-10T20:20:57.1707600Z    default_idle_call+0xa8/0x254
2024-12-10T20:20:57.1709171Z    do_idle+0xc4/0x118
2024-12-10T20:20:57.1711481Z    cpu_startup_entry+0x40/0x50
2024-12-10T20:20:57.1714251Z    secondary_start_kernel+0xdc/0x108
2024-12-10T20:20:57.1716568Z    __secondary_switched+0xc0/0xc8
2024-12-10T20:20:58.2547951Z sched_ext: BPF scheduler "ddsp_vtimelocal_fail" enabled
2024-12-10T20:20:58.3147251Z sched_ext: BPF scheduler "ddsp_vtimelocal_fail" disabled (runtime error)
2024-12-10T20:20:58.3152469Z sched_ext: ddsp_vtimelocal_fail: cannot use vtime ordering for built-in DSQs
2024-12-10T20:20:58.3154552Z    dispatch_enqueue+0x350/0x360
2024-12-10T20:20:58.3156792Z    direct_dispatch+0x144/0x228
2024-12-10T20:20:58.3159057Z    do_enqueue_task+0x17c/0x240
2024-12-10T20:20:58.3161402Z    enqueue_task_scx+0x190/0x288
2024-12-10T20:20:58.3163484Z    enqueue_task+0x44/0xe8
2024-12-10T20:20:58.3165655Z    ttwu_do_activate+0x88/0x298
2024-12-10T20:20:58.3167841Z    try_to_wake_up+0x2b8/0x560
2024-12-10T20:20:58.3169741Z    swake_up_one+0x4c/0x98
2024-12-10T20:20:58.3172304Z    swake_up_one_online+0x54/0xc0
2024-12-10T20:20:58.3174591Z    rcu_gp_kthread_wake+0x68/0x98
2024-12-10T20:20:58.3177003Z    rcu_report_qs_rsp+0xc0/0x110
2024-12-10T20:20:58.3179438Z    rcu_report_qs_rnp+0x324/0x380
2024-12-10T20:20:58.3181831Z    rcu_report_qs_rdp+0x1ac/0x1c8
2024-12-10T20:20:58.3183880Z    rcu_core+0x264/0x378
2024-12-10T20:20:58.3185468Z    rcu_core_si+0x18/0x30
2024-12-10T20:20:58.3187895Z    handle_softirqs+0x12c/0x490
2024-12-10T20:20:58.3189738Z    __do_softirq+0x1c/0x28
2024-12-10T20:20:58.3191821Z    ____do_softirq+0x18/0x30
2024-12-10T20:20:58.3194130Z    call_on_irq_stack+0x24/0x58
2024-12-10T20:20:58.3196588Z    do_softirq_own_stack+0x24/0x38
2024-12-10T20:20:58.3198745Z    __irq_exit_rcu+0x140/0x180
2024-12-10T20:20:58.3200627Z    irq_exit_rcu+0x18/0x48
2024-12-10T20:20:58.3202698Z    el1_interrupt+0x38/0x50
2024-12-10T20:20:58.3205112Z    el1h_64_irq_handler+0x18/0x28
2024-12-10T20:20:58.3206944Z    el1h_64_irq+0x6c/0x70
2024-12-10T20:20:58.3209342Z    default_idle_call+0xac/0x254
2024-12-10T20:20:58.3211702Z    default_idle_call+0xa8/0x254
2024-12-10T20:20:58.3213312Z    do_idle+0xc4/0x118
2024-12-10T20:20:58.3215643Z    cpu_startup_entry+0x40/0x50
2024-12-10T20:20:58.3218478Z    secondary_start_kernel+0xdc/0x108
2024-12-10T20:20:58.3220846Z    __secondary_switched+0xc0/0xc8
2024-12-10T20:20:59.3646877Z sched_ext: BPF scheduler "dsp_local_on" enabled
2024-12-10T20:21:00.3650453Z ERR: dsp_local_on.c:37
2024-12-10T20:21:00.3652199Z Expected skel->data->uei.kind == EXIT_KIND(SCX_EXIT_ERROR) (0 == 1024)
2024-12-10T20:21:00.3873220Z ERR: exit.c:30
2024-12-10T20:21:00.3873570Z Failed to attach scheduler
2024-12-10T20:21:00.4746585Z psci: CPU1 killed (polled 0 ms)
2024-12-10T20:21:00.4946799Z sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
2024-12-10T20:21:00.4953445Z sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 0 not allowed for ksoftirqd/1[23]
2024-12-10T20:21:00.4955934Z    task_can_run_on_remote_rq+0xdc/0x108
2024-12-10T20:21:00.4958580Z    dispatch_to_local_dsq+0x74/0x1d0
2024-12-10T20:21:00.4960911Z    flush_dispatch_buf+0x18c/0x1d8
2024-12-10T20:21:00.4962903Z    balance_one+0x154/0x2f8
2024-12-10T20:21:00.4964749Z    balance_scx+0x58/0x98
2024-12-10T20:21:00.4966663Z    __schedule+0x390/0x838
2024-12-10T20:21:00.4968348Z    schedule+0x54/0x138
2024-12-10T20:21:00.4970453Z    worker_thread+0xec/0x360
2024-12-10T20:21:00.4972076Z    kthread+0x100/0x110
2024-12-10T20:21:00.4974136Z    ret_from_fork+0x10/0x20
2024-12-10T20:21:00.5545933Z sched_ext: BPF scheduler "hotplug_cbs" enabled
2024-12-10T20:21:00.6145106Z Detected PIPT I-cache on CPU1
2024-12-10T20:21:00.6149673Z GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000
2024-12-10T20:21:00.6154461Z CPU1: Booted secondary processor 0x0000000001 [0x413fd0c1]
2024-12-10T20:21:00.8046976Z sched_ext: BPF scheduler "hotplug_cbs" disabled (unregistered from BPF)
2024-12-10T20:21:00.8047755Z EXIT: unregistered from BPF (hotplug event detected (1 going online))
2024-12-10T20:21:00.9246328Z sched_ext: BPF scheduler "hotplug_cbs" enabled
2024-12-10T20:21:01.0246233Z psci: CPU1 killed (polled 0 ms)
2024-12-10T20:21:01.0946840Z sched_ext: BPF scheduler "hotplug_cbs" disabled (unregistered from BPF)
2024-12-10T20:21:01.0947621Z EXIT: unregistered from BPF (hotplug event detected (1 going offline))
2024-12-10T20:21:01.1545076Z Detected PIPT I-cache on CPU1
2024-12-10T20:21:01.1549600Z GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000
2024-12-10T20:21:01.1554406Z CPU1: Booted secondary processor 0x0000000001 [0x413fd0c1]
2024-12-10T20:21:01.2245966Z psci: CPU1 killed (polled 0 ms)


> 
> diff --git a/tools/testing/selftests/sched_ext/maximal.bpf.c b/tools/testing/selftests/sched_ext/maximal.bpf.c
> index 4c005fa71810..430f5e13bf55 100644
> --- a/tools/testing/selftests/sched_ext/maximal.bpf.c
> +++ b/tools/testing/selftests/sched_ext/maximal.bpf.c
> @@ -12,6 +12,8 @@
> 
> char _license[] SEC("license") = "GPL";
> 
> +#define DSQ_ID 0
> +
> s32 BPF_STRUCT_OPS(maximal_select_cpu, struct task_struct *p, s32 prev_cpu,
> u64 wake_flags)
> {
> @@ -20,7 +22,7 @@ s32 BPF_STRUCT_OPS(maximal_select_cpu, struct task_struct *p, s32 prev_cpu,
> 
> void BPF_STRUCT_OPS(maximal_enqueue, struct task_struct *p, u64 enq_flags)
> {
> - scx_bpf_dsq_insert(p, SCX_DSQ_GLOBAL, SCX_SLICE_DFL, enq_flags);
> + scx_bpf_dsq_insert(p, DSQ_ID, SCX_SLICE_DFL, enq_flags);
> }
> 
> void BPF_STRUCT_OPS(maximal_dequeue, struct task_struct *p, u64 deq_flags)
> @@ -28,7 +30,7 @@ void BPF_STRUCT_OPS(maximal_dequeue, struct task_struct *p, u64 deq_flags)
> 
> void BPF_STRUCT_OPS(maximal_dispatch, s32 cpu, struct task_struct *prev)
> {
> - scx_bpf_dsq_move_to_local(SCX_DSQ_GLOBAL);
> + scx_bpf_dsq_move_to_local(DSQ_ID);
> }
> 
> void BPF_STRUCT_OPS(maximal_runnable, struct task_struct *p, u64 enq_flags)
> @@ -123,7 +125,7 @@ void BPF_STRUCT_OPS(maximal_cgroup_set_weight, struct cgroup *cgrp, u32 weight)
> 
> s32 BPF_STRUCT_OPS_SLEEPABLE(maximal_init)
> {
> - return 0;
> + return scx_bpf_create_dsq(DSQ_ID, -1);
> }
> 
> void BPF_STRUCT_OPS(maximal_exit, struct scx_exit_info *info)
> --
> 2.46.1
[PATCH sched_ext/for-6.13-fixes] sched_ext: Fix invalid irq restore in scx_ops_bypass()
Posted by Tejun Heo 1 year ago 
While adding outer irqsave/restore locking, 0e7ffff1b811 ("scx: Fix raciness
in scx_ops_bypass()") forgot to convert an inner rq_unlock_irqrestore() to
rq_unlock() which could re-enable IRQ prematurely leading to the following
warning:

  raw_local_irq_restore() called with IRQs enabled
  WARNING: CPU: 1 PID: 96 at kernel/locking/irqflag-debug.c:10 warn_bogus_irq_restore+0x30/0x40
  ...
  Sched_ext: create_dsq (enabling)
  pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
  pc : warn_bogus_irq_restore+0x30/0x40
  lr : warn_bogus_irq_restore+0x30/0x40
  ...
  Call trace:
   warn_bogus_irq_restore+0x30/0x40 (P)
   warn_bogus_irq_restore+0x30/0x40 (L)
   scx_ops_bypass+0x224/0x3b8
   scx_ops_enable.isra.0+0x2c8/0xaa8
   bpf_scx_reg+0x18/0x30
  ...
  irq event stamp: 33739
  hardirqs last  enabled at (33739): [<ffff8000800b699c>] scx_ops_bypass+0x174/0x3b8
  hardirqs last disabled at (33738): [<ffff800080d48ad4>] _raw_spin_lock_irqsave+0xb4/0xd8

Drop the stray _irqrestore().

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Ihor Solodrai <ihor.solodrai@pm.me>
Link: http://lkml.kernel.org/r/qC39k3UsonrBYD_SmuxHnZIQLsuuccoCrkiqb_BT7DvH945A1_LZwE4g-5Pu9FcCtqZt4lY1HhIPi0homRuNWxkgo1rgP3bkxa0donw8kV4=@pm.me
Fixes: 0e7ffff1b811 ("scx: Fix raciness in scx_ops_bypass()")
Cc: stable@vger.kernel.org # v6.12
---
 kernel/sched/ext.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 7fff1d045477..98519e6d0dcd 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -4763,7 +4763,7 @@ static void scx_ops_bypass(bool bypass)
 		 * sees scx_rq_bypassing() before moving tasks to SCX.
 		 */
 		if (!scx_enabled()) {
-			rq_unlock_irqrestore(rq, &rf);
+			rq_unlock(rq, &rf);
 			continue;
 		}
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix invalid irq restore in scx_ops_bypass()
Posted by Ihor Solodrai 1 year ago 
On Wednesday, December 11th, 2024 at 1:01 PM, Tejun Heo <tj@kernel.org> wrote:

> 
> 
> While adding outer irqsave/restore locking, 0e7ffff1b811 ("scx: Fix raciness
> in scx_ops_bypass()") forgot to convert an inner rq_unlock_irqrestore() to
> rq_unlock() which could re-enable IRQ prematurely leading to the following
> warning:
> 
> raw_local_irq_restore() called with IRQs enabled
> WARNING: CPU: 1 PID: 96 at kernel/locking/irqflag-debug.c:10 warn_bogus_irq_restore+0x30/0x40
> ...
> Sched_ext: create_dsq (enabling)
> pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> pc : warn_bogus_irq_restore+0x30/0x40
> lr : warn_bogus_irq_restore+0x30/0x40
> ...
> Call trace:
> warn_bogus_irq_restore+0x30/0x40 (P)
> warn_bogus_irq_restore+0x30/0x40 (L)
> scx_ops_bypass+0x224/0x3b8
> scx_ops_enable.isra.0+0x2c8/0xaa8
> bpf_scx_reg+0x18/0x30
> ...
> irq event stamp: 33739
> hardirqs last enabled at (33739): [<ffff8000800b699c>] scx_ops_bypass+0x174/0x3b8
> 
> hardirqs last disabled at (33738): [<ffff800080d48ad4>] _raw_spin_lock_irqsave+0xb4/0xd8
> 
> 
> Drop the stray _irqrestore().
> 
> Signed-off-by: Tejun Heo tj@kernel.org
> 
> Reported-by: Ihor Solodrai ihor.solodrai@pm.me
> 
> Link: http://lkml.kernel.org/r/qC39k3UsonrBYD_SmuxHnZIQLsuuccoCrkiqb_BT7DvH945A1_LZwE4g-5Pu9FcCtqZt4lY1HhIPi0homRuNWxkgo1rgP3bkxa0donw8kV4=@pm.me
> Fixes: 0e7ffff1b811 ("scx: Fix raciness in scx_ops_bypass()")
> Cc: stable@vger.kernel.org # v6.12
> ---
> kernel/sched/ext.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> index 7fff1d045477..98519e6d0dcd 100644
> --- a/kernel/sched/ext.c
> +++ b/kernel/sched/ext.c
> @@ -4763,7 +4763,7 @@ static void scx_ops_bypass(bool bypass)
> * sees scx_rq_bypassing() before moving tasks to SCX.
> */
> if (!scx_enabled()) {
> - rq_unlock_irqrestore(rq, &rf);
> + rq_unlock(rq, &rf);
> continue;
> }

Hi Tejun,


I tried this patch on BPF CI: the pipeline ran 3 times
successfully. That's 12 selftests/sched_ext runs in total.

https://github.com/kernel-patches/vmtest/actions/runs/12301284063

Tested-by: Ihor Solodrai ihor.solodrai@pm.me

Thanks for the fix!
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix invalid irq restore in scx_ops_bypass()
Posted by Ihor Solodrai 12 months ago 
Hi Tejun,

I re-enabled selftests/sched_ext on BPF CI today. The kernel on CI
includes this patch. Sometimes there is a failure on attempt to attach
a dsp_local_on scheduler.

Examples of failed jobs:

  * https://github.com/kernel-patches/bpf/actions/runs/12379720791/job/34555104994
  * https://github.com/kernel-patches/bpf/actions/runs/12382862660/job/34564648924
  * https://github.com/kernel-patches/bpf/actions/runs/12381361846/job/34560047798

Here is a piece of log that is present in failed run, but not in
a successful run:

2024-12-17T19:30:12.9010943Z [    5.285022] sched_ext: BPF scheduler "dsp_local_on" enabled
2024-12-17T19:30:13.9022892Z ERR: dsp_local_on.c:37
2024-12-17T19:30:13.9025841Z Expected skel->data->uei.kind == EXIT_KIND(SCX_EXIT_ERROR) (0 == 1024)
2024-12-17T19:30:13.9256108Z ERR: exit.c:30
2024-12-17T19:30:13.9256641Z Failed to attach scheduler
2024-12-17T19:30:13.9611443Z [    6.345087] smpboot: CPU 1 is now offline

Could you please investigate?

Thanks.
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix invalid irq restore in scx_ops_bypass()
Posted by Tejun Heo 12 months ago 
Hello,

On Tue, Dec 17, 2024 at 11:44:08PM +0000, Ihor Solodrai wrote:
> I re-enabled selftests/sched_ext on BPF CI today. The kernel on CI
> includes this patch. Sometimes there is a failure on attempt to attach
> a dsp_local_on scheduler.
> 
> Examples of failed jobs:
> 
>   * https://github.com/kernel-patches/bpf/actions/runs/12379720791/job/34555104994
>   * https://github.com/kernel-patches/bpf/actions/runs/12382862660/job/34564648924
>   * https://github.com/kernel-patches/bpf/actions/runs/12381361846/job/34560047798
> 
> Here is a piece of log that is present in failed run, but not in
> a successful run:
> 
> 2024-12-17T19:30:12.9010943Z [    5.285022] sched_ext: BPF scheduler "dsp_local_on" enabled
> 2024-12-17T19:30:13.9022892Z ERR: dsp_local_on.c:37
> 2024-12-17T19:30:13.9025841Z Expected skel->data->uei.kind == EXIT_KIND(SCX_EXIT_ERROR) (0 == 1024)
> 2024-12-17T19:30:13.9256108Z ERR: exit.c:30
> 2024-12-17T19:30:13.9256641Z Failed to attach scheduler
> 2024-12-17T19:30:13.9611443Z [    6.345087] smpboot: CPU 1 is now offline
> 
> Could you please investigate?

The test prog is wrong in assuming all possible CPUs to be consecutive and
online but I'm not sure whether that's what's making the test flaky. Do you
have dmesg from a failed run?

Thanks.

-- 
tejun
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix invalid irq restore in scx_ops_bypass()
Posted by Ihor Solodrai 12 months ago 
On Wednesday, December 18th, 2024 at 10:34 AM, Tejun Heo <tj@kernel.org> wrote:

> 
> 
> Hello,
> 
> On Tue, Dec 17, 2024 at 11:44:08PM +0000, Ihor Solodrai wrote:
> 
> > I re-enabled selftests/sched_ext on BPF CI today. The kernel on CI
> > includes this patch. Sometimes there is a failure on attempt to attach
> > a dsp_local_on scheduler.
> > 
> > Examples of failed jobs:
> > 
> > * https://github.com/kernel-patches/bpf/actions/runs/12379720791/job/34555104994
> > * https://github.com/kernel-patches/bpf/actions/runs/12382862660/job/34564648924
> > * https://github.com/kernel-patches/bpf/actions/runs/12381361846/job/34560047798
> > 
> > Here is a piece of log that is present in failed run, but not in
> > a successful run:
> > 
> > 2024-12-17T19:30:12.9010943Z [ 5.285022] sched_ext: BPF scheduler "dsp_local_on" enabled
> > 2024-12-17T19:30:13.9022892Z ERR: dsp_local_on.c:37
> > 2024-12-17T19:30:13.9025841Z Expected skel->data->uei.kind == EXIT_KIND(SCX_EXIT_ERROR) (0 == 1024)
> > 2024-12-17T19:30:13.9256108Z ERR: exit.c:30
> > 2024-12-17T19:30:13.9256641Z Failed to attach scheduler
> > 2024-12-17T19:30:13.9611443Z [ 6.345087] smpboot: CPU 1 is now offline
> > 
> > Could you please investigate?
> 
> 
> The test prog is wrong in assuming all possible CPUs to be consecutive and
> online but I'm not sure whether that's what's making the test flaky. Do you
> have dmesg from a failed run?

Tejun, can you elaborate on what you're looking for in the logs?
My understanding is that QEMU prints some of the dmesg messages.
QEMU output is available in raw logs.

Here is a link (you have to login to github to open):

https://productionresultssa1.blob.core.windows.net/actions-results/99cd995e-679f-4180-872b-d31e1f564837/workflow-job-run-7216a7c9-5129-5959-a45a-28d6f9b737e2/logs/job/job-logs.txt?rsct=text%2Fplain&se=2024-12-19T22%3A57%3A01Z&sig=z%2B%2FUtIIhli4VG%2FCCVxawBnubNwfIIsl9Q2FlTVvM8q0%3D&ske=2024-12-20T07%3A00%3A35Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-12-19T19%3A00%3A35Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2024-11-04&sp=r&spr=https&sr=b&st=2024-12-19T22%3A46%3A56Z&sv=2024-11-04

Generally, you can access raw logs by going to the job, and 
clicking the gear on the topright -> "View raw logs".

> 
> Thanks.
> 
> --
> tejun
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix invalid irq restore in scx_ops_bypass()
Posted by Ihor Solodrai 12 months ago 
On Thursday, December 19th, 2024 at 2:51 PM, Ihor Solodrai <ihor.solodrai@pm.me> wrote:

> [...]
> 
> Tejun, can you elaborate on what you're looking for in the logs?
> My understanding is that QEMU prints some of the dmesg messages.
> QEMU output is available in raw logs.

I made changes to the CI scripts to explicitly dump dmesg in case of a failure.
It looks like most of that log was already printed.

Job: https://github.com/kernel-patches/bpf/actions/runs/12436924307/job/34726070343

Raw log: https://productionresultssa11.blob.core.windows.net/actions-results/a10f57cb-19e3-487a-9fb0-69742cfbef1b/workflow-job-run-4c580b44-6466-54d8-b922-6f707064e5ca/logs/job/job-logs.txt?rsct=text%2Fplain&se=2024-12-20T19%3A34%3A55Z&sig=kQ09k9r01VtP4p%2FgYvvCmm2FUuOHfsLjU3ARzks4xmE%3D&ske=2024-12-21T07%3A00%3A50Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-12-20T19%3A00%3A50Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2024-11-04&sp=r&spr=https&sr=b&st=2024-12-20T19%3A24%3A50Z&sv=2024-11-04 

Search for "dmesg start".

> [...]
[PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Tejun Heo 11 months, 3 weeks ago 
The dsp_local_on selftest expects the scheduler to fail by trying to
schedule an e.g. CPU-affine task to the wrong CPU. However, this isn't
guaranteed to happen in the 1 second window that the test is running.
Besides, it's odd to have this particular exception path tested when there
are no other tests that verify that the interface is working at all - e.g.
the test would pass if dsp_local_on interface is completely broken and fails
on any attempt.

Flip the test so that it verifies that the feature works. While at it, fix a
typo in the info message.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Ihor Solodrai <ihor.solodrai@pm.me>
Link: http://lkml.kernel.org/r/Z1n9v7Z6iNJ-wKmq@slm.duckdns.org
---
 tools/testing/selftests/sched_ext/dsp_local_on.bpf.c |    5 ++++-
 tools/testing/selftests/sched_ext/dsp_local_on.c     |    5 +++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c b/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
index 6325bf76f47e..fbda6bf54671 100644
--- a/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
+++ b/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
@@ -43,7 +43,10 @@ void BPF_STRUCT_OPS(dsp_local_on_dispatch, s32 cpu, struct task_struct *prev)
 	if (!p)
 		return;
 
-	target = bpf_get_prandom_u32() % nr_cpus;
+	if (p->nr_cpus_allowed == nr_cpus)
+		target = bpf_get_prandom_u32() % nr_cpus;
+	else
+		target = scx_bpf_task_cpu(p);
 
 	scx_bpf_dsq_insert(p, SCX_DSQ_LOCAL_ON | target, SCX_SLICE_DFL, 0);
 	bpf_task_release(p);
diff --git a/tools/testing/selftests/sched_ext/dsp_local_on.c b/tools/testing/selftests/sched_ext/dsp_local_on.c
index 472851b56854..0ff27e57fe43 100644
--- a/tools/testing/selftests/sched_ext/dsp_local_on.c
+++ b/tools/testing/selftests/sched_ext/dsp_local_on.c
@@ -34,9 +34,10 @@ static enum scx_test_status run(void *ctx)
 	/* Just sleeping is fine, plenty of scheduling events happening */
 	sleep(1);
 
-	SCX_EQ(skel->data->uei.kind, EXIT_KIND(SCX_EXIT_ERROR));
 	bpf_link__destroy(link);
 
+	SCX_EQ(skel->data->uei.kind, EXIT_KIND(SCX_EXIT_UNREG));
+
 	return SCX_TEST_PASS;
 }
 
@@ -50,7 +51,7 @@ static void cleanup(void *ctx)
 struct scx_test dsp_local_on = {
 	.name = "dsp_local_on",
 	.description = "Verify we can directly dispatch tasks to a local DSQs "
-		       "from osp.dispatch()",
+		       "from ops.dispatch()",
 	.setup = setup,
 	.run = run,
 	.cleanup = cleanup,
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Ihor Solodrai 11 months ago 
On Tuesday, December 24th, 2024 at 4:09 PM, Tejun Heo <tj@kernel.org> wrote:

> 
> 
> The dsp_local_on selftest expects the scheduler to fail by trying to
> schedule an e.g. CPU-affine task to the wrong CPU. However, this isn't
> guaranteed to happen in the 1 second window that the test is running.
> Besides, it's odd to have this particular exception path tested when there
> are no other tests that verify that the interface is working at all - e.g.
> the test would pass if dsp_local_on interface is completely broken and fails
> on any attempt.
> 
> Flip the test so that it verifies that the feature works. While at it, fix a
> typo in the info message.
> 
> Signed-off-by: Tejun Heo tj@kernel.org
> 
> Reported-by: Ihor Solodrai ihor.solodrai@pm.me
> 
> Link: http://lkml.kernel.org/r/Z1n9v7Z6iNJ-wKmq@slm.duckdns.org
> ---
> tools/testing/selftests/sched_ext/dsp_local_on.bpf.c | 5 ++++-
> tools/testing/selftests/sched_ext/dsp_local_on.c | 5 +++--
> 2 files changed, 7 insertions(+), 3 deletions(-)

Hi Tejun.

I've tried running sched_ext selftests on BPF CI today, applying a set
of patches from sched_ext/for-6.13-fixes, including this one.

You can see the list of patches I added here:
https://github.com/kernel-patches/vmtest/pull/332/files

With that, dsq_local_on has failed on x86_64 (llvm-18), although it
passed with other configurations:
https://github.com/kernel-patches/vmtest/actions/runs/12798804552/job/35683769806

Here is a piece of log that appears to be relevant:

    2025-01-15T23:28:55.8238375Z [    5.334631] sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
    2025-01-15T23:28:55.8243034Z [    5.335420] sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 1 not allowed for kworker/u8:1[33]
    2025-01-15T23:28:55.8246187Z [    5.336139]    dispatch_to_local_dsq+0x13e/0x1f0
    2025-01-15T23:28:55.8249296Z [    5.336474]    flush_dispatch_buf+0x13d/0x170
    2025-01-15T23:28:55.8252083Z [    5.336793]    balance_scx+0x225/0x3e0
    2025-01-15T23:28:55.8254695Z [    5.337065]    __schedule+0x406/0xe80
    2025-01-15T23:28:55.8257121Z [    5.337330]    schedule+0x41/0xb0
    2025-01-15T23:28:55.8260146Z [    5.337574]    schedule_timeout+0xe5/0x160
    2025-01-15T23:28:55.8263080Z [    5.337875]    rcu_tasks_kthread+0xb1/0xc0
    2025-01-15T23:28:55.8265477Z [    5.338169]    kthread+0xfa/0x120
    2025-01-15T23:28:55.8268202Z [    5.338410]    ret_from_fork+0x37/0x50
    2025-01-15T23:28:55.8271272Z [    5.338690]    ret_from_fork_asm+0x1a/0x30
    2025-01-15T23:28:56.7349562Z ERR: dsp_local_on.c:39
    2025-01-15T23:28:56.7350182Z Expected skel->data->uei.kind == EXIT_KIND(SCX_EXIT_UNREG) (1024 == 64)

Could you please take a look?

Thank you.

> 
> [...]
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Tejun Heo 10 months, 4 weeks ago 
Hello, sorry about the delay.

On Wed, Jan 15, 2025 at 11:50:37PM +0000, Ihor Solodrai wrote:
...
>     2025-01-15T23:28:55.8238375Z [    5.334631] sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
>     2025-01-15T23:28:55.8243034Z [    5.335420] sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 1 not allowed for kworker/u8:1[33]

That's a head scratcher. It's a single node 2 cpu instance and all unbound
kworkers should be allowed on all CPUs. I'll update the test to test the
actual cpumask but can you see whether this failure is consistent or flaky?

Thanks.

-- 
tejun
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Ihor Solodrai 10 months, 4 weeks ago 
On Tuesday, January 21st, 2025 at 5:40 PM, Tejun Heo <tj@kernel.org> wrote:

> 
> 
> Hello, sorry about the delay.
> 
> On Wed, Jan 15, 2025 at 11:50:37PM +0000, Ihor Solodrai wrote:
> ...
> 
> > 2025-01-15T23:28:55.8238375Z [ 5.334631] sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
> > 2025-01-15T23:28:55.8243034Z [ 5.335420] sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 1 not allowed for kworker/u8:1[33]
> 
> 
> That's a head scratcher. It's a single node 2 cpu instance and all unbound
> kworkers should be allowed on all CPUs. I'll update the test to test the
> actual cpumask but can you see whether this failure is consistent or flaky?

I re-ran all the jobs, and all sched_ext jobs have failed (3/3).
Previous time only 1 of 3 runs failed.

https://github.com/kernel-patches/vmtest/actions/runs/12798804552/job/36016405680

> 
> Thanks.
> 
> --
> tejun
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Andrea Righi 10 months, 4 weeks ago 
On Wed, Jan 22, 2025 at 07:10:00PM +0000, Ihor Solodrai wrote:
> 
> On Tuesday, January 21st, 2025 at 5:40 PM, Tejun Heo <tj@kernel.org> wrote:
> 
> > 
> > 
> > Hello, sorry about the delay.
> > 
> > On Wed, Jan 15, 2025 at 11:50:37PM +0000, Ihor Solodrai wrote:
> > ...
> > 
> > > 2025-01-15T23:28:55.8238375Z [ 5.334631] sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
> > > 2025-01-15T23:28:55.8243034Z [ 5.335420] sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 1 not allowed for kworker/u8:1[33]
> > 
> > 
> > That's a head scratcher. It's a single node 2 cpu instance and all unbound
> > kworkers should be allowed on all CPUs. I'll update the test to test the
> > actual cpumask but can you see whether this failure is consistent or flaky?
> 
> I re-ran all the jobs, and all sched_ext jobs have failed (3/3).
> Previous time only 1 of 3 runs failed.
> 
> https://github.com/kernel-patches/vmtest/actions/runs/12798804552/job/36016405680

Oh I see what happens, SCX_DSQ_LOCAL_ON is (incorrectly) resolved to 0.

More exactly, none of the enum values are being resolved correctly, likely
due to the CO:RE enum refactoring. There’s probably something broken in
tools/testing/selftests/sched_ext/Makefile, I’ll take a look.

Thanks,
-Andrea
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Tejun Heo 10 months, 3 weeks ago 
On Thu, Jan 23, 2025 at 10:40:52AM +0100, Andrea Righi wrote:
> On Wed, Jan 22, 2025 at 07:10:00PM +0000, Ihor Solodrai wrote:
> > 
> > On Tuesday, January 21st, 2025 at 5:40 PM, Tejun Heo <tj@kernel.org> wrote:
> > 
> > > 
> > > 
> > > Hello, sorry about the delay.
> > > 
> > > On Wed, Jan 15, 2025 at 11:50:37PM +0000, Ihor Solodrai wrote:
> > > ...
> > > 
> > > > 2025-01-15T23:28:55.8238375Z [ 5.334631] sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
> > > > 2025-01-15T23:28:55.8243034Z [ 5.335420] sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 1 not allowed for kworker/u8:1[33]
> > > 
> > > 
> > > That's a head scratcher. It's a single node 2 cpu instance and all unbound
> > > kworkers should be allowed on all CPUs. I'll update the test to test the
> > > actual cpumask but can you see whether this failure is consistent or flaky?
> > 
> > I re-ran all the jobs, and all sched_ext jobs have failed (3/3).
> > Previous time only 1 of 3 runs failed.
> > 
> > https://github.com/kernel-patches/vmtest/actions/runs/12798804552/job/36016405680
> 
> Oh I see what happens, SCX_DSQ_LOCAL_ON is (incorrectly) resolved to 0.
> 
> More exactly, none of the enum values are being resolved correctly, likely
> due to the CO:RE enum refactoring. There’s probably something broken in
> tools/testing/selftests/sched_ext/Makefile, I’ll take a look.

Yeah, we need to add SCX_ENUM_INIT() to each test. Will do that once the
pending pull request is merged. The original report is a separate issue tho.
I'm still a bit baffled by it.

Thanks.

-- 
tejun
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Andrea Righi 10 months, 3 weeks ago 
On Thu, Jan 23, 2025 at 06:57:58AM -1000, Tejun Heo wrote:
> On Thu, Jan 23, 2025 at 10:40:52AM +0100, Andrea Righi wrote:
> > On Wed, Jan 22, 2025 at 07:10:00PM +0000, Ihor Solodrai wrote:
> > > 
> > > On Tuesday, January 21st, 2025 at 5:40 PM, Tejun Heo <tj@kernel.org> wrote:
> > > 
> > > > 
> > > > 
> > > > Hello, sorry about the delay.
> > > > 
> > > > On Wed, Jan 15, 2025 at 11:50:37PM +0000, Ihor Solodrai wrote:
> > > > ...
> > > > 
> > > > > 2025-01-15T23:28:55.8238375Z [ 5.334631] sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
> > > > > 2025-01-15T23:28:55.8243034Z [ 5.335420] sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 1 not allowed for kworker/u8:1[33]
> > > > 
> > > > 
> > > > That's a head scratcher. It's a single node 2 cpu instance and all unbound
> > > > kworkers should be allowed on all CPUs. I'll update the test to test the
> > > > actual cpumask but can you see whether this failure is consistent or flaky?
> > > 
> > > I re-ran all the jobs, and all sched_ext jobs have failed (3/3).
> > > Previous time only 1 of 3 runs failed.
> > > 
> > > https://github.com/kernel-patches/vmtest/actions/runs/12798804552/job/36016405680
> > 
> > Oh I see what happens, SCX_DSQ_LOCAL_ON is (incorrectly) resolved to 0.
> > 
> > More exactly, none of the enum values are being resolved correctly, likely
> > due to the CO:RE enum refactoring. There’s probably something broken in
> > tools/testing/selftests/sched_ext/Makefile, I’ll take a look.
> 
> Yeah, we need to add SCX_ENUM_INIT() to each test. Will do that once the
> pending pull request is merged. The original report is a separate issue tho.
> I'm still a bit baffled by it.

For the enum part: https://lore.kernel.org/all/20250123124606.242115-1-arighi@nvidia.com/

And yeah, I missed that the original bug report was about the unbound
kworker not allowed to be dispatched on cpu 1. Weird... I'm wondering if we
need to do the cpumask_cnt / scx_bpf_dsq_cancel() game, like we did with
scx_rustland to handle concurrent affinity changes, but in this case the
kworker shouldn't have its affinity changed...

-Andrea
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Andrea Righi 10 months, 3 weeks ago 
On Thu, Jan 23, 2025 at 07:45:08PM +0100, Andrea Righi wrote:
> On Thu, Jan 23, 2025 at 06:57:58AM -1000, Tejun Heo wrote:
> > On Thu, Jan 23, 2025 at 10:40:52AM +0100, Andrea Righi wrote:
> > > On Wed, Jan 22, 2025 at 07:10:00PM +0000, Ihor Solodrai wrote:
> > > > 
> > > > On Tuesday, January 21st, 2025 at 5:40 PM, Tejun Heo <tj@kernel.org> wrote:
> > > > 
> > > > > 
> > > > > 
> > > > > Hello, sorry about the delay.
> > > > > 
> > > > > On Wed, Jan 15, 2025 at 11:50:37PM +0000, Ihor Solodrai wrote:
> > > > > ...
> > > > > 
> > > > > > 2025-01-15T23:28:55.8238375Z [ 5.334631] sched_ext: BPF scheduler "dsp_local_on" disabled (runtime error)
> > > > > > 2025-01-15T23:28:55.8243034Z [ 5.335420] sched_ext: dsp_local_on: SCX_DSQ_LOCAL[_ON] verdict target cpu 1 not allowed for kworker/u8:1[33]
> > > > > 
> > > > > 
> > > > > That's a head scratcher. It's a single node 2 cpu instance and all unbound
> > > > > kworkers should be allowed on all CPUs. I'll update the test to test the
> > > > > actual cpumask but can you see whether this failure is consistent or flaky?
> > > > 
> > > > I re-ran all the jobs, and all sched_ext jobs have failed (3/3).
> > > > Previous time only 1 of 3 runs failed.
> > > > 
> > > > https://github.com/kernel-patches/vmtest/actions/runs/12798804552/job/36016405680
> > > 
> > > Oh I see what happens, SCX_DSQ_LOCAL_ON is (incorrectly) resolved to 0.
> > > 
> > > More exactly, none of the enum values are being resolved correctly, likely
> > > due to the CO:RE enum refactoring. There’s probably something broken in
> > > tools/testing/selftests/sched_ext/Makefile, I’ll take a look.
> > 
> > Yeah, we need to add SCX_ENUM_INIT() to each test. Will do that once the
> > pending pull request is merged. The original report is a separate issue tho.
> > I'm still a bit baffled by it.
> 
> For the enum part: https://lore.kernel.org/all/20250123124606.242115-1-arighi@nvidia.com/
> 
> And yeah, I missed that the original bug report was about the unbound
> kworker not allowed to be dispatched on cpu 1. Weird... I'm wondering if we
> need to do the cpumask_cnt / scx_bpf_dsq_cancel() game, like we did with
> scx_rustland to handle concurrent affinity changes, but in this case the
> kworker shouldn't have its affinity changed...

Thinking more about this, scx_bpf_task_cpu(p) returns the last known CPU
where the task p was running, but it doesn't necessarily give a CPU where
the task can run at any time. In general it's probably a safer choice to
rely on p->cpus_ptr, maybe doing bpf_cpumask_any_distribute(p->cpus_ptr)
for this test case.

However, I still don't see why the unbound kworker couldn't be dispatched
on cpu 1 in this particular case...

-Andrea
[PATCH sched_ext/for-6.14-fixes] sched_ext: selftests/dsp_local_on: Fix sporadic failures
Posted by Tejun Heo 10 months, 3 weeks ago 
From e9fe182772dcb2630964724fd93e9c90b68ea0fd Mon Sep 17 00:00:00 2001
From: Tejun Heo <tj@kernel.org>
Date: Fri, 24 Jan 2025 10:48:25 -1000

dsp_local_on has several incorrect assumptions, one of which is that
p->nr_cpus_allowed always tracks p->cpus_ptr. This is not true when a task
is scheduled out while migration is disabled - p->cpus_ptr is temporarily
overridden to the previous CPU while p->nr_cpus_allowed remains unchanged.

This led to sporadic test faliures when dsp_local_on_dispatch() tries to put
a migration disabled task to a different CPU. Fix it by keeping the previous
CPU when migration is disabled.

There are SCX schedulers that make use of p->nr_cpus_allowed. They should
also implement explicit handling for p->migration_disabled.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Ihor Solodrai <ihor.solodrai@pm.me>
Cc: Andrea Righi <arighi@nvidia.com>
Cc: Changwoo Min <changwoo@igalia.com>
---
Applying to sched_ext/for-6.14-fixes. Thanks.

 tools/testing/selftests/sched_ext/dsp_local_on.bpf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c b/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
index fbda6bf54671..758b479bd1ee 100644
--- a/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
+++ b/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
@@ -43,7 +43,7 @@ void BPF_STRUCT_OPS(dsp_local_on_dispatch, s32 cpu, struct task_struct *prev)
 	if (!p)
 		return;
 
-	if (p->nr_cpus_allowed == nr_cpus)
+	if (p->nr_cpus_allowed == nr_cpus && !p->migration_disabled)
 		target = bpf_get_prandom_u32() % nr_cpus;
 	else
 		target = scx_bpf_task_cpu(p);
-- 
2.48.1
Re: [PATCH sched_ext/for-6.14-fixes] sched_ext: selftests/dsp_local_on: Fix sporadic failures
Posted by Andrea Righi 10 months, 3 weeks ago 
On Fri, Jan 24, 2025 at 12:00:38PM -1000, Tejun Heo wrote:
> From e9fe182772dcb2630964724fd93e9c90b68ea0fd Mon Sep 17 00:00:00 2001
> From: Tejun Heo <tj@kernel.org>
> Date: Fri, 24 Jan 2025 10:48:25 -1000
> 
> dsp_local_on has several incorrect assumptions, one of which is that
> p->nr_cpus_allowed always tracks p->cpus_ptr. This is not true when a task
> is scheduled out while migration is disabled - p->cpus_ptr is temporarily
> overridden to the previous CPU while p->nr_cpus_allowed remains unchanged.
> 
> This led to sporadic test faliures when dsp_local_on_dispatch() tries to put
> a migration disabled task to a different CPU. Fix it by keeping the previous
> CPU when migration is disabled.
> 
> There are SCX schedulers that make use of p->nr_cpus_allowed. They should
> also implement explicit handling for p->migration_disabled.
> 
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Reported-by: Ihor Solodrai <ihor.solodrai@pm.me>
> Cc: Andrea Righi <arighi@nvidia.com>
> Cc: Changwoo Min <changwoo@igalia.com>
> ---
> Applying to sched_ext/for-6.14-fixes. Thanks.
> 
>  tools/testing/selftests/sched_ext/dsp_local_on.bpf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c b/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
> index fbda6bf54671..758b479bd1ee 100644
> --- a/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
> +++ b/tools/testing/selftests/sched_ext/dsp_local_on.bpf.c
> @@ -43,7 +43,7 @@ void BPF_STRUCT_OPS(dsp_local_on_dispatch, s32 cpu, struct task_struct *prev)
>  	if (!p)
>  		return;
>  
> -	if (p->nr_cpus_allowed == nr_cpus)
> +	if (p->nr_cpus_allowed == nr_cpus && !p->migration_disabled)

This doesn't work with !CONFIG_SMP, maybe we can introduce a helper like:

static bool is_migration_disabled(const struct task_struct *p)
{
	if (bpf_core_field_exists(p->migration_disabled))
		return p->migration_disabled;
	return false;
}

>  		target = bpf_get_prandom_u32() % nr_cpus;
>  	else
>  		target = scx_bpf_task_cpu(p);
> -- 
> 2.48.1
> 

Thanks,
-Andrea
Re: [PATCH sched_ext/for-6.14-fixes] sched_ext: selftests/dsp_local_on: Fix sporadic failures
Posted by Tejun Heo 10 months, 3 weeks ago 
On Sat, Jan 25, 2025 at 05:54:23AM +0100, Andrea Righi wrote:
...
> > -	if (p->nr_cpus_allowed == nr_cpus)
> > +	if (p->nr_cpus_allowed == nr_cpus && !p->migration_disabled)
> 
> This doesn't work with !CONFIG_SMP, maybe we can introduce a helper like:
> 
> static bool is_migration_disabled(const struct task_struct *p)
> {
> 	if (bpf_core_field_exists(p->migration_disabled))
> 		return p->migration_disabled;
> 	return false;

Ah, right. Would you care to send the patch?

Thanks.

-- 
tejun
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix dsq_local_on selftest
Posted by Tejun Heo 11 months, 3 weeks ago 
On Tue, Dec 24, 2024 at 02:09:15PM -1000, Tejun Heo wrote:
> The dsp_local_on selftest expects the scheduler to fail by trying to
> schedule an e.g. CPU-affine task to the wrong CPU. However, this isn't
> guaranteed to happen in the 1 second window that the test is running.
> Besides, it's odd to have this particular exception path tested when there
> are no other tests that verify that the interface is working at all - e.g.
> the test would pass if dsp_local_on interface is completely broken and fails
> on any attempt.
> 
> Flip the test so that it verifies that the feature works. While at it, fix a
> typo in the info message.
> 
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Reported-by: Ihor Solodrai <ihor.solodrai@pm.me>
> Link: http://lkml.kernel.org/r/Z1n9v7Z6iNJ-wKmq@slm.duckdns.org

Applied to sched_ext/for-6.13-fixes.

Thanks.

-- 
tejun
Re: [PATCH sched_ext/for-6.13-fixes] sched_ext: Fix invalid irq restore in scx_ops_bypass()
Posted by Tejun Heo 1 year ago 
On Wed, Dec 11, 2024 at 11:01:51AM -1000, Tejun Heo wrote:
> While adding outer irqsave/restore locking, 0e7ffff1b811 ("scx: Fix raciness
> in scx_ops_bypass()") forgot to convert an inner rq_unlock_irqrestore() to
> rq_unlock() which could re-enable IRQ prematurely leading to the following
> warning:
> 
>   raw_local_irq_restore() called with IRQs enabled
>   WARNING: CPU: 1 PID: 96 at kernel/locking/irqflag-debug.c:10 warn_bogus_irq_restore+0x30/0x40
>   ...
>   Sched_ext: create_dsq (enabling)
>   pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
>   pc : warn_bogus_irq_restore+0x30/0x40
>   lr : warn_bogus_irq_restore+0x30/0x40
>   ...
>   Call trace:
>    warn_bogus_irq_restore+0x30/0x40 (P)
>    warn_bogus_irq_restore+0x30/0x40 (L)
>    scx_ops_bypass+0x224/0x3b8
>    scx_ops_enable.isra.0+0x2c8/0xaa8
>    bpf_scx_reg+0x18/0x30
>   ...
>   irq event stamp: 33739
>   hardirqs last  enabled at (33739): [<ffff8000800b699c>] scx_ops_bypass+0x174/0x3b8
>   hardirqs last disabled at (33738): [<ffff800080d48ad4>] _raw_spin_lock_irqsave+0xb4/0xd8
> 
> Drop the stray _irqrestore().
> 
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Reported-by: Ihor Solodrai <ihor.solodrai@pm.me>
> Link: http://lkml.kernel.org/r/qC39k3UsonrBYD_SmuxHnZIQLsuuccoCrkiqb_BT7DvH945A1_LZwE4g-5Pu9FcCtqZt4lY1HhIPi0homRuNWxkgo1rgP3bkxa0donw8kV4=@pm.me
> Fixes: 0e7ffff1b811 ("scx: Fix raciness in scx_ops_bypass()")
> Cc: stable@vger.kernel.org # v6.12

Applying to sched_ext/for-6.13-fixes.

Thanks.

-- 
tejun
Re: [PATCH] scx: Fix maximal BPF selftest prog
Posted by Andrea Righi 1 year ago 
On Mon, Dec 09, 2024 at 09:29:24AM -0600, David Vernet wrote:
> maximal.bpf.c is still dispatching to and consuming from SCX_DSQ_GLOBAL.
> Let's have it use its own DSQ to avoid any runtime errors.
> 
> Signed-off-by: David Vernet <void@manifault.com>

Looks good to me.

Tested-by: Andrea Righi <arighi@nvidia.com>

> ---
>  tools/testing/selftests/sched_ext/maximal.bpf.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/testing/selftests/sched_ext/maximal.bpf.c b/tools/testing/selftests/sched_ext/maximal.bpf.c
> index 4c005fa71810..430f5e13bf55 100644
> --- a/tools/testing/selftests/sched_ext/maximal.bpf.c
> +++ b/tools/testing/selftests/sched_ext/maximal.bpf.c
> @@ -12,6 +12,8 @@
>  
>  char _license[] SEC("license") = "GPL";
>  
> +#define DSQ_ID 0
> +
>  s32 BPF_STRUCT_OPS(maximal_select_cpu, struct task_struct *p, s32 prev_cpu,
>  		   u64 wake_flags)
>  {
> @@ -20,7 +22,7 @@ s32 BPF_STRUCT_OPS(maximal_select_cpu, struct task_struct *p, s32 prev_cpu,
>  
>  void BPF_STRUCT_OPS(maximal_enqueue, struct task_struct *p, u64 enq_flags)
>  {
> -	scx_bpf_dsq_insert(p, SCX_DSQ_GLOBAL, SCX_SLICE_DFL, enq_flags);
> +	scx_bpf_dsq_insert(p, DSQ_ID, SCX_SLICE_DFL, enq_flags);
>  }
>  
>  void BPF_STRUCT_OPS(maximal_dequeue, struct task_struct *p, u64 deq_flags)
> @@ -28,7 +30,7 @@ void BPF_STRUCT_OPS(maximal_dequeue, struct task_struct *p, u64 deq_flags)
>  
>  void BPF_STRUCT_OPS(maximal_dispatch, s32 cpu, struct task_struct *prev)
>  {
> -	scx_bpf_dsq_move_to_local(SCX_DSQ_GLOBAL);
> +	scx_bpf_dsq_move_to_local(DSQ_ID);
>  }
>  
>  void BPF_STRUCT_OPS(maximal_runnable, struct task_struct *p, u64 enq_flags)
> @@ -123,7 +125,7 @@ void BPF_STRUCT_OPS(maximal_cgroup_set_weight, struct cgroup *cgrp, u32 weight)
>  
>  s32 BPF_STRUCT_OPS_SLEEPABLE(maximal_init)
>  {
> -	return 0;
> +	return scx_bpf_create_dsq(DSQ_ID, -1);
>  }
>  
>  void BPF_STRUCT_OPS(maximal_exit, struct scx_exit_info *info)
> -- 
> 2.46.1
> 

-Andrea

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.