On Mon, 8 Sept 2025 at 21:35, Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
>
> This is the start of the stable review cycle for the 6.1.151 release.
> There are 101 patches in this series, all will be posted as a response
> to this one. If anyone has any issues with these being applied, please
> let me know.
>
> Responses should be made by Wed, 10 Sep 2025 15:18:27 +0000.
> Anything received after that time might be too late.
>
> The whole patch series can be found in one patch at:
> https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.1.151-rc2.gz
> or in the git tree and branch at:
> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.1.y
> and the diffstat can be found below.
>
> thanks,
>
> greg k-h
While running booting Juno-r2 device with Linux stable-rc 6.1.151-rc2
kernel found this RCU info followed by boot hang.
Regression Analysis:
- Reproducibility? Validation is in progress
Boot regression: stable-rc 6.1.151-rc2 juno-r2 cpuidle_enter_state hang
Reported-by: Linux Kernel Functional Testing <lkft@linaro.org>
### Boot log
[ 975.847953] rcu: INFO: rcu_preempt self-detected stall on CPU
[ 975.853724] rcu: \t3-...!: (5249 ticks this GP)
idle=25dc/1/0x4000000000000002 softirq=1901/1901 fqs=0
[ 975.862973] \t(t=5253 jiffies g=2077 q=8479 ncpus=6)
[ 975.867862] rcu: rcu_preempt kthread timer wakeup didn't happen for
5253 jiffies! g2077 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
[ 975.879192] rcu: \tPossible timer handling issue on cpu=5 timer-softirq=408
[ 975.886079] rcu: rcu_preempt kthread starved for 5259 jiffies!
g2077 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=5
[ 975.896453] rcu: \tUnless rcu_preempt kthread gets sufficient CPU
time, OOM is now expected behavior.
[ 975.905601] rcu: RCU grace-period kthread stack dump:
[ 975.910658] task:rcu_preempt state:I stack:0 pid:16
ppid:2 flags:0x00000008
[ 975.919035] Call trace:
[ 975.921482] __switch_to+0x154/0x1f8
[ 975.925075] __schedule+0x494/0x8a0
[ 975.928577] schedule+0x84/0xe8
[ 975.931729] schedule_timeout+0xac/0x19c
[ 975.935662] rcu_gp_fqs_loop+0x1f4/0x808
[ 975.939598] rcu_gp_kthread+0x70/0x238
[ 975.943359] kthread+0xe8/0x1cc
[ 975.946509] ret_from_fork+0x10/0x20
[ 975.950096] rcu: Stack dump where RCU GP kthread last ran:
[ 975.955589] Task dump for CPU 5:
[ 975.958819] task:swapper/5 state:R running task stack:0
pid:0 ppid:1 flags:0x00000008
[ 975.968764] Call trace:
[ 975.971210] __switch_to+0x154/0x1f8
[ 975.974798] ct_idle_enter+0x10/0x1c
[ 975.978384] 0xffff00097edb0640
[ 975.981541] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.1.151-rc2 #1
[ 975.987912] Hardware name: ARM Juno development board (r2) (DT)
[ 975.993843] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 976.000823] pc : cpuidle_enter_state+0x16c/0x444
[ 976.005456] lr : cpuidle_enter_state+0x160/0x444
[ 976.010086] sp : ffff80000aaa3d70
[ 976.013403] x29: ffff80000aaa3d70 x28: ffff80000a2de000 x27: ffff80000a2a5638
[ 976.020566] x26: 0000000000000001 x25: ffff0008222b4898 x24: 000000e33511dfc8
[ 976.027727] x23: 000000e334d58fb4 x22: 0000000000000001 x21: 0000000000000001
[ 976.034888] x20: ffff0008222b4880 x19: ffff00097ed72640 x18: 0000000000000219
[ 976.042049] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000
[ 976.049210] x14: ffff80000a9020dc x13: ffff80000aaa0000 x12: ffff80000aaa4000
[ 976.056372] x11: 1ada3b6729410000 x10: 0000000000000000 x9 : 0000000000000000
[ 976.063532] x8 : 00000000000000e0 x7 : 00000072b5503510 x6 : 0000000000300000
[ 976.070693] x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffff80000aaa3dd0
[ 976.077854] x2 : ffff80000aaa3d08 x1 : ffff80000905fdec x0 : ffff80000905fa90
[ 976.085015] Call trace:
[ 976.087462] cpuidle_enter_state+0x16c/0x444
[ 976.091746] cpuidle_enter+0x44/0x5c
[ 976.095331] do_idle+0x1f4/0x2c4
[ 976.098571] cpu_startup_entry+0x40/0x44
[ 976.102507] secondary_start_kernel+0x12c/0x150
[ 976.107053] __secondary_switched+0xb0/0xb4
[ 976.111251] Task dump for CPU 4:
[ 976.114482] task:swapper/4 state:R running task stack:0
pid:0 ppid:1 flags:0x00000008
[ 976.124427] Call trace:
[ 976.126873] __switch_to+0x154/0x1f8
[ 976.130462] psci_enter_idle_state+0x5c/0x7c
[ 976.134747] cpuidle_enter_state+0x118/0x444
[ 976.139028] cpuidle_enter+0x44/0x5c
[ 976.142613] do_idle+0x1f4/0x2c4
[ 976.145852] cpu_startup_entry+0x40/0x44
[ 976.149788] secondary_start_kernel+0x12c/0x150
[ 976.154333] __secondary_switched+0xb0/0xb4
[ 976.158530] Task dump for CPU 5:
[ 976.161761] task:rcu_preempt state:R running task stack:0
pid:16 ppid:2 flags:0x00000008
[ 976.171705] Call trace:
[ 976.174152] __switch_to+0x154/0x1f8
[ 976.177739] 0x0
[ 1039.195951] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1039.202056] rcu: \t5-...!: (0 ticks this GP) idle=e028/0/0x0
softirq=1512/1512 fqs=1 (false positive?)
[ 1039.211296] \t(detected by 2, t=21087 jiffies, g=2077, q=8483 ncpus=6)
[ 1039.217746] Task dump for CPU 5:
[ 1039.220974] task:swapper/5 state:R running task stack:0
pid:0 ppid:1 flags:0x00000008
[ 1039.230911] Call trace:
[ 1039.233355] __switch_to+0x154/0x1f8
[ 1039.236938] ct_idle_enter+0x10/0x1c
[ 1039.240518] 0xffff00097edb0640
[ 1039.243661] rcu: rcu_preempt kthread timer wakeup didn't happen for
15764 jiffies! g2077 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
[ 1039.255073] rcu: \tPossible timer handling issue on cpu=5 timer-softirq=408
[ 1039.261957] rcu: rcu_preempt kthread starved for 15770 jiffies!
g2077 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=5
[ 1039.272411] rcu: \tUnless rcu_preempt kthread gets sufficient CPU
time, OOM is now expected behavior.
[ 1039.281555] rcu: RCU grace-period kthread stack dump:
[ 1039.286609] task:rcu_preempt state:I stack:0 pid:16
ppid:2 flags:0x00000008
[ 1039.294977] Call trace:
[ 1039.297420] __switch_to+0x154/0x1f8
[ 1039.301001] __schedule+0x494/0x8a0
[ 1039.304495] schedule+0x84/0xe8
[ 1039.307641] schedule_timeout+0xac/0x19c
[ 1039.311567] rcu_gp_fqs_loop+0x1f4/0x808
[ 1039.315496] rcu_gp_kthread+0x70/0x238
[ 1039.319249] kthread+0xe8/0x1cc
[ 1039.322391] ret_from_fork+0x10/0x20
[ 1039.325971] rcu: Stack dump where RCU GP kthread last ran:
[ 1039.331459] Task dump for CPU 5:
[ 1039.334686] task:swapper/5 state:R running task stack:0
pid:0 ppid:1 flags:0x00000008
[ 1039.344622] Call trace:
[ 1039.347065] __switch_to+0x154/0x1f8
[ 1039.350646] ct_idle_enter+0x10/0x1c
[ 1039.354225] 0xffff00097edb0640
[ 1039.412051] sd 2:0:0:0: [sda] Synchronizing SCSI cache
[ 1039.417488] sd 2:0:0:0: [sda] Synchronize Cache(10) failed: Result:
hostbyte=0x01 driverbyte=DRIVER_OK
- https://qa-reports.linaro.org/api/testruns/29825860/log_file/
- https://lkft.validation.linaro.org/scheduler/job/8439988#L2145
- https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/32QLuLpnnwp9AnZEJVgmvKIFxXT
- https://storage.tuxsuite.com/public/linaro/lkft/builds/32QLrp4K8PORz7gTLeqeiSIGTKL/config
- https://storage.tuxsuite.com/public/linaro/lkft/builds/32QLrp4K8PORz7gTLeqeiSIGTKL/
## Build
* kernel: 6.1.151-rc2
* git: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
* git commit: e60b159208e69c485efd270b6bd1fedd07e1aaad
* git describe: v6.1.149-153-ge60b159208e6
* test details:
https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.1.y/build/v6.1.149-153-ge60b159208e6
## Test Regressions (compared to v6.1.149-51-gcdcdd968ff27)
## Metric Regressions (compared to v6.1.149-51-gcdcdd968ff27)
## Test Fixes (compared to v6.1.149-51-gcdcdd968ff27)
## Metric Fixes (compared to v6.1.149-51-gcdcdd968ff27)
## Test result summary
total: 231778, pass: 215616, fail: 4719, skip: 11160, xfail: 283
## Build Summary
* arc: 5 total, 5 passed, 0 failed
* arm: 133 total, 133 passed, 0 failed
* arm64: 41 total, 41 passed, 0 failed
* i386: 21 total, 21 passed, 0 failed
* mips: 26 total, 25 passed, 1 failed
* parisc: 4 total, 4 passed, 0 failed
* powerpc: 32 total, 31 passed, 1 failed
* riscv: 11 total, 11 passed, 0 failed
* s390: 14 total, 14 passed, 0 failed
* sh: 10 total, 10 passed, 0 failed
* sparc: 7 total, 7 passed, 0 failed
* x86_64: 33 total, 33 passed, 0 failed
## Test suites summary
* boot
* commands
* kselftest-arm64
* kselftest-breakpoints
* kselftest-capabilities
* kselftest-clone3
* kselftest-core
* kselftest-cpu-hotplug
* kselftest-exec
* kselftest-fpu
* kselftest-futex
* kselftest-intel_pstate
* kselftest-kcmp
* kselftest-kvm
* kselftest-livepatch
* kselftest-membarrier
* kselftest-mincore
* kselftest-mqueue
* kselftest-openat2
* kselftest-ptrace
* kselftest-rseq
* kselftest-rtc
* kselftest-sigaltstack
* kselftest-size
* kselftest-timers
* kselftest-tmpfs
* kselftest-tpm2
* kselftest-user_events
* kselftest-vDSO
* kselftest-x86
* kunit
* kvm-unit-tests
* lava
* libgpiod
* libhugetlbfs
* log-parser-boot
* log-parser-build-clang
* log-parser-build-gcc
* log-parser-test
* ltp-capability
* ltp-commands
* ltp-containers
* ltp-controllers
* ltp-cpuhotplug
* ltp-crypto
* ltp-cve
* ltp-dio
* ltp-fcntl-locktests
* ltp-fs
* ltp-fs_bind
* ltp-fs_perms_simple
* ltp-hugetlb
* ltp-math
* ltp-mm
* ltp-nptl
* ltp-pty
* ltp-sched
* ltp-smoke
* ltp-syscalls
* ltp-tracing
* modules
* perf
* rcutorture
--
Linaro LKFT
https://lkft.linaro.org