mips/smp: Call rcu_cpu_starting() earlier

[PATCH] mips/smp: Call rcu_cpu_starting() earlier

Posted by Stefan Wiehler 1 year, 11 months ago

rcu_cpu_starting() must be called before clockevents_register_device() to avoid
the following lockdep splat triggered by calling list_add() when
CONFIG_PROVE_RCU_LIST=y:

  WARNING: suspicious RCU usage
  ...
  -----------------------------
  kernel/locking/lockdep.c:3680 RCU-list traversed in non-reader section!!

  other info that might help us debug this:

  RCU used illegally from offline CPU!
  rcu_scheduler_active = 1, debug_locks = 1
  no locks held by swapper/1/0.
  ...
  Call Trace:
  [<ffffffff8012a434>] show_stack+0x64/0x158
  [<ffffffff80a93d98>] dump_stack_lvl+0x90/0xc4
  [<ffffffff801c9e9c>] __lock_acquire+0x1404/0x2940
  [<ffffffff801cbf3c>] lock_acquire+0x14c/0x448
  [<ffffffff80aa4260>] _raw_spin_lock_irqsave+0x50/0x88
  [<ffffffff8021e0c8>] clockevents_register_device+0x60/0x1e8
  [<ffffffff80130ff0>] r4k_clockevent_init+0x220/0x3a0
  [<ffffffff801339d0>] start_secondary+0x50/0x3b8

raw_smp_processor_id() is required in order to avoid calling into lockdep
before RCU has declared the CPU to be watched for readers.

See also commit 29368e093921 ("x86/smpboot:  Move rcu_cpu_starting() earlier"),
commit de5d9dae150c ("s390/smp: move rcu_cpu_starting() earlier") and commit
99f070b62322 ("powerpc/smp: Call rcu_cpu_starting() earlier").

Signed-off-by: Stefan Wiehler <stefan.wiehler@nokia.com>
Reviewed-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
---
 arch/mips/kernel/smp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 8fbef537fb88..81f6c4f8fbc1 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -351,10 +351,11 @@ early_initcall(mips_smp_ipi_init);
  */
 asmlinkage void start_secondary(void)
 {
-	unsigned int cpu;
+	unsigned int cpu = raw_smp_processor_id();
 
 	cpu_probe();
 	per_cpu_trap_init(false);
+	rcu_cpu_starting(cpu);
 	mips_clockevent_init();
 	mp_ops->init_secondary();
 	cpu_report();
@@ -366,7 +367,6 @@ asmlinkage void start_secondary(void)
 	 */
 
 	calibrate_delay();
-	cpu = smp_processor_id();
 	cpu_data[cpu].udelay_val = loops_per_jiffy;
 
 	set_cpu_sibling_map(cpu);
-- 
2.42.0

Re: [PATCH] mips/smp: Call rcu_cpu_starting() earlier

Posted by Greg Kroah-Hartman 1 year, 11 months ago

On Fri, Jan 19, 2024 at 05:38:11PM +0100, Stefan Wiehler wrote:
> rcu_cpu_starting() must be called before clockevents_register_device() to avoid
> the following lockdep splat triggered by calling list_add() when
> CONFIG_PROVE_RCU_LIST=y:
> 
>   WARNING: suspicious RCU usage
>   ...
>   -----------------------------
>   kernel/locking/lockdep.c:3680 RCU-list traversed in non-reader section!!
> 
>   other info that might help us debug this:
> 
>   RCU used illegally from offline CPU!
>   rcu_scheduler_active = 1, debug_locks = 1
>   no locks held by swapper/1/0.
>   ...
>   Call Trace:
>   [<ffffffff8012a434>] show_stack+0x64/0x158
>   [<ffffffff80a93d98>] dump_stack_lvl+0x90/0xc4
>   [<ffffffff801c9e9c>] __lock_acquire+0x1404/0x2940
>   [<ffffffff801cbf3c>] lock_acquire+0x14c/0x448
>   [<ffffffff80aa4260>] _raw_spin_lock_irqsave+0x50/0x88
>   [<ffffffff8021e0c8>] clockevents_register_device+0x60/0x1e8
>   [<ffffffff80130ff0>] r4k_clockevent_init+0x220/0x3a0
>   [<ffffffff801339d0>] start_secondary+0x50/0x3b8
> 
> raw_smp_processor_id() is required in order to avoid calling into lockdep
> before RCU has declared the CPU to be watched for readers.
> 
> See also commit 29368e093921 ("x86/smpboot:  Move rcu_cpu_starting() earlier"),
> commit de5d9dae150c ("s390/smp: move rcu_cpu_starting() earlier") and commit
> 99f070b62322 ("powerpc/smp: Call rcu_cpu_starting() earlier").
> 
> Signed-off-by: Stefan Wiehler <stefan.wiehler@nokia.com>
> Reviewed-by: Huacai Chen <chenhuacai@loongson.cn>
> Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
> ---
>  arch/mips/kernel/smp.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

What is the git commit id of this change in Linus's tree?  What
kernel(s) should this be applied to?

thanks,

greg k-h

Re: [PATCH] mips/smp: Call rcu_cpu_starting() earlier

Posted by Florian Fainelli 1 year, 11 months ago


On 1/19/2024 10:38 PM, Greg Kroah-Hartman wrote:
> On Fri, Jan 19, 2024 at 05:38:11PM +0100, Stefan Wiehler wrote:
>> rcu_cpu_starting() must be called before clockevents_register_device() to avoid
>> the following lockdep splat triggered by calling list_add() when
>> CONFIG_PROVE_RCU_LIST=y:
>>
>>    WARNING: suspicious RCU usage
>>    ...
>>    -----------------------------
>>    kernel/locking/lockdep.c:3680 RCU-list traversed in non-reader section!!
>>
>>    other info that might help us debug this:
>>
>>    RCU used illegally from offline CPU!
>>    rcu_scheduler_active = 1, debug_locks = 1
>>    no locks held by swapper/1/0.
>>    ...
>>    Call Trace:
>>    [<ffffffff8012a434>] show_stack+0x64/0x158
>>    [<ffffffff80a93d98>] dump_stack_lvl+0x90/0xc4
>>    [<ffffffff801c9e9c>] __lock_acquire+0x1404/0x2940
>>    [<ffffffff801cbf3c>] lock_acquire+0x14c/0x448
>>    [<ffffffff80aa4260>] _raw_spin_lock_irqsave+0x50/0x88
>>    [<ffffffff8021e0c8>] clockevents_register_device+0x60/0x1e8
>>    [<ffffffff80130ff0>] r4k_clockevent_init+0x220/0x3a0
>>    [<ffffffff801339d0>] start_secondary+0x50/0x3b8
>>
>> raw_smp_processor_id() is required in order to avoid calling into lockdep
>> before RCU has declared the CPU to be watched for readers.
>>
>> See also commit 29368e093921 ("x86/smpboot:  Move rcu_cpu_starting() earlier"),
>> commit de5d9dae150c ("s390/smp: move rcu_cpu_starting() earlier") and commit
>> 99f070b62322 ("powerpc/smp: Call rcu_cpu_starting() earlier").
>>
>> Signed-off-by: Stefan Wiehler <stefan.wiehler@nokia.com>
>> Reviewed-by: Huacai Chen <chenhuacai@loongson.cn>
>> Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
>> ---
>>   arch/mips/kernel/smp.c | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> What is the git commit id of this change in Linus's tree?  What
> kernel(s) should this be applied to?

The upstream commit is 55702ec9603ebeffb15e6f7b113623fe1d8872f4 
("mips/smp: Call rcutree_report_cpu_starting() earlier") and this change 
from Stefan should be applied to both the 6.1 stable and 6.6 stable 
branches. Thanks!
-- 
Florian

Re: [PATCH] mips/smp: Call rcu_cpu_starting() earlier

Posted by Greg Kroah-Hartman 1 year, 11 months ago

On Sun, Jan 21, 2024 at 08:52:55AM -0800, Florian Fainelli wrote:
> 
> 
> On 1/19/2024 10:38 PM, Greg Kroah-Hartman wrote:
> > On Fri, Jan 19, 2024 at 05:38:11PM +0100, Stefan Wiehler wrote:
> > > rcu_cpu_starting() must be called before clockevents_register_device() to avoid
> > > the following lockdep splat triggered by calling list_add() when
> > > CONFIG_PROVE_RCU_LIST=y:
> > > 
> > >    WARNING: suspicious RCU usage
> > >    ...
> > >    -----------------------------
> > >    kernel/locking/lockdep.c:3680 RCU-list traversed in non-reader section!!
> > > 
> > >    other info that might help us debug this:
> > > 
> > >    RCU used illegally from offline CPU!
> > >    rcu_scheduler_active = 1, debug_locks = 1
> > >    no locks held by swapper/1/0.
> > >    ...
> > >    Call Trace:
> > >    [<ffffffff8012a434>] show_stack+0x64/0x158
> > >    [<ffffffff80a93d98>] dump_stack_lvl+0x90/0xc4
> > >    [<ffffffff801c9e9c>] __lock_acquire+0x1404/0x2940
> > >    [<ffffffff801cbf3c>] lock_acquire+0x14c/0x448
> > >    [<ffffffff80aa4260>] _raw_spin_lock_irqsave+0x50/0x88
> > >    [<ffffffff8021e0c8>] clockevents_register_device+0x60/0x1e8
> > >    [<ffffffff80130ff0>] r4k_clockevent_init+0x220/0x3a0
> > >    [<ffffffff801339d0>] start_secondary+0x50/0x3b8
> > > 
> > > raw_smp_processor_id() is required in order to avoid calling into lockdep
> > > before RCU has declared the CPU to be watched for readers.
> > > 
> > > See also commit 29368e093921 ("x86/smpboot:  Move rcu_cpu_starting() earlier"),
> > > commit de5d9dae150c ("s390/smp: move rcu_cpu_starting() earlier") and commit
> > > 99f070b62322 ("powerpc/smp: Call rcu_cpu_starting() earlier").
> > > 
> > > Signed-off-by: Stefan Wiehler <stefan.wiehler@nokia.com>
> > > Reviewed-by: Huacai Chen <chenhuacai@loongson.cn>
> > > Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
> > > ---
> > >   arch/mips/kernel/smp.c | 4 ++--
> > >   1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > What is the git commit id of this change in Linus's tree?  What
> > kernel(s) should this be applied to?
> 
> The upstream commit is 55702ec9603ebeffb15e6f7b113623fe1d8872f4 ("mips/smp:
> Call rcutree_report_cpu_starting() earlier") and this change from Stefan
> should be applied to both the 6.1 stable and 6.6 stable branches. Thanks!

Thanks, now queued up.

greg k-h