[PATCH v2] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit

Christophe Leroy posted 1 patch 4 months, 4 weeks ago
arch/powerpc/kernel/entry_32.S  | 18 +++++++++++++++++-
arch/powerpc/kernel/interrupt.c |  2 +-
2 files changed, 18 insertions(+), 2 deletions(-)
[PATCH v2] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
Posted by Christophe Leroy 4 months, 4 weeks ago
Commit 13799748b957 ("powerpc/64: use interrupt restart table to speed
up return from interrupt") removed the inconditional clearing of
MSR[RI] when returning from interrupt into kernel. But powerpc/32
doesn't implement interrupt restart table hence still need MSR[RI]
to be cleared.

It could be added back in interrupt_exit_kernel_prepare() but it is
easier and better to add it back in entry_32.S for following reasons:
- Writing to MSR must be followed by a synchronising instruction
- The smaller the non recoverable section is the better it is

So add a macro called clr_ri and use it in the three places that play
up with SRR0/SRR1. Use it just before another mtspr for synchronisation
to avoid having to add an isync.

Now that's done in entry_32.S, exit_must_hard_disable() can return
false for non book3s/64, taking into account that BOOKE doesn't have
MSR_RI.

Also add back blacklisting syscall_exit_finish for kprobe. This was
initially added by commit 7cdf44013885 ("powerpc/entry32: Blacklist
syscall exit points for kprobe.") then lost with
commit 6f76a01173cc ("powerpc/syscall: implement system call
entry/exit logic in C for PPC32").

Fixes: 6f76a01173cc ("powerpc/syscall: implement system call entry/exit logic in C for PPC32")
Fixes: 13799748b957 ("powerpc/64: use interrupt restart table to speed up return from interrupt")
Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
---
 arch/powerpc/kernel/entry_32.S  | 18 +++++++++++++++++-
 arch/powerpc/kernel/interrupt.c |  2 +-
 2 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S
index c37480176a1c..16f8ee6cb2cd 100644
--- a/arch/powerpc/kernel/entry_32.S
+++ b/arch/powerpc/kernel/entry_32.S
@@ -101,6 +101,17 @@ SYM_FUNC_END(__kuep_unlock)
 .endm
 #endif
 
+.macro	clr_ri trash
+#ifndef CONFIG_BOOKE
+#ifdef CONFIG_PPC_8xx
+	mtspr   SPRN_NRI, \trash
+#else
+	li	\trash, MSR_KERNEL & ~MSR_RI
+	mtmsr	\trash
+#endif
+#endif
+.endm
+
 	.globl	transfer_to_syscall
 transfer_to_syscall:
 	stw	r3, ORIG_GPR3(r1)
@@ -149,6 +160,7 @@ ret_from_syscall:
 	cmpwi	r3,0
 	REST_GPR(3, r1)
 syscall_exit_finish:
+	clr_ri	r4
 	mtspr	SPRN_SRR0,r7
 	mtspr	SPRN_SRR1,r8
 
@@ -168,6 +180,7 @@ syscall_exit_finish:
 	REST_GPR(0, r1)
 	REST_GPRS(3, 12, r1)
 	b	1b
+_ASM_NOKPROBE_SYMBOL(syscall_exit_finish)
 
 #ifdef CONFIG_44x
 .L44x_icache_flush:
@@ -224,10 +237,11 @@ fast_exception_return:
 	/* Clear the exception marker on the stack to avoid confusing stacktrace */
 	li	r10, 0
 	stw	r10, 8(r11)
-	REST_GPR(10, r11)
+	clr_ri	r10
 	mtspr	SPRN_SRR1,r9
 	mtspr	SPRN_SRR0,r12
 	REST_GPR(9, r11)
+	REST_GPR(10, r11)
 	REST_GPR(12, r11)
 	REST_GPR(11, r11)
 	rfi
@@ -256,6 +270,7 @@ interrupt_return:
 .Lfast_user_interrupt_return:
 	lwz	r11,_NIP(r1)
 	lwz	r12,_MSR(r1)
+	clr_ri	r4
 	mtspr	SPRN_SRR0,r11
 	mtspr	SPRN_SRR1,r12
 
@@ -298,6 +313,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_NEED_PAIRED_STWCX)
 	cmpwi	cr1,r3,0
 	lwz	r11,_NIP(r1)
 	lwz	r12,_MSR(r1)
+	clr_ri	r4
 	mtspr	SPRN_SRR0,r11
 	mtspr	SPRN_SRR1,r12
 
diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c
index e0c681d0b076..aea6f7e8e9c6 100644
--- a/arch/powerpc/kernel/interrupt.c
+++ b/arch/powerpc/kernel/interrupt.c
@@ -38,7 +38,7 @@ static inline bool exit_must_hard_disable(void)
 #else
 static inline bool exit_must_hard_disable(void)
 {
-	return true;
+	return false;
 }
 #endif
 
-- 
2.49.0
Re: [PATCH v2] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
Posted by Guenter Roeck 1 month, 3 weeks ago
Hi,

On Thu, Sep 11, 2025 at 02:30:12PM +0200, Christophe Leroy wrote:
> Commit 13799748b957 ("powerpc/64: use interrupt restart table to speed
> up return from interrupt") removed the inconditional clearing of
> MSR[RI] when returning from interrupt into kernel. But powerpc/32
> doesn't implement interrupt restart table hence still need MSR[RI]
> to be cleared.
> 
> It could be added back in interrupt_exit_kernel_prepare() but it is
> easier and better to add it back in entry_32.S for following reasons:
> - Writing to MSR must be followed by a synchronising instruction
> - The smaller the non recoverable section is the better it is
> 
> So add a macro called clr_ri and use it in the three places that play
> up with SRR0/SRR1. Use it just before another mtspr for synchronisation
> to avoid having to add an isync.
> 
> Now that's done in entry_32.S, exit_must_hard_disable() can return
> false for non book3s/64, taking into account that BOOKE doesn't have
> MSR_RI.
> 
> Also add back blacklisting syscall_exit_finish for kprobe. This was
> initially added by commit 7cdf44013885 ("powerpc/entry32: Blacklist
> syscall exit points for kprobe.") then lost with
> commit 6f76a01173cc ("powerpc/syscall: implement system call
> entry/exit logic in C for PPC32").
> 
> Fixes: 6f76a01173cc ("powerpc/syscall: implement system call entry/exit logic in C for PPC32")
> Fixes: 13799748b957 ("powerpc/64: use interrupt restart table to speed up return from interrupt")
> Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>

This patch causes several of my ppc64:ppce500 boot tests with e5500 CPU
to stall.

I have not found a common denominator (the proboem seems independent
of boot media and network interface) , but the problem is repeatable.
Roughtly 50-70% of boot attempts stall.

Reverting this patch fixes the problem.

Bisect log is attached for reference.

Thanks,
Guenter

---
# bad: [8f0b4cce4481fb22653697cced8d0d04027cb1e8] Linux 6.19-rc1
# good: [7d0a66e4bb9081d75c82ec4957c50034cb0ea449] Linux 6.18
git bisect start 'HEAD' 'v6.18'
# good: [6dfafbd0299a60bfb5d5e277fdf100037c7ded07] Merge tag 'drm-next-2025-12-03' of https://gitlab.freedesktop.org/drm/kernel
git bisect good 6dfafbd0299a60bfb5d5e277fdf100037c7ded07
# bad: [09cab48db950b6fb8c114314a20c0fd5a80cf990] Merge tag 'soc-arm-6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect bad 09cab48db950b6fb8c114314a20c0fd5a80cf990
# good: [6044a1ee9dca906a807ba786421dc4254191ffd5] Merge tag 'devicetree-for-6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
git bisect good 6044a1ee9dca906a807ba786421dc4254191ffd5
# good: [ac20755937e037e586b1ca18a6717d31b1cbce93] Merge tag 'sysctl-6.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl
git bisect good ac20755937e037e586b1ca18a6717d31b1cbce93
# good: [4b9d25b4d38035b7b2624afd6852dfe4684f0226] Merge tag 'vfs-6.19-rc1.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
git bisect good 4b9d25b4d38035b7b2624afd6852dfe4684f0226
# good: [63a9b0bc65d5d3ea96a57e7985ea22a8582fbbe5] Merge tag 'kvm-riscv-6.19-1' of https://github.com/kvm-riscv/linux into HEAD
git bisect good 63a9b0bc65d5d3ea96a57e7985ea22a8582fbbe5
# good: [e0c26d47def7382d7dbd9cad58bc653aed75737a] Merge tag 'kvm-s390-next-6.19-1' of https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD
git bisect good e0c26d47def7382d7dbd9cad58bc653aed75737a
# bad: [07025b51c1149951d64804c73014499bb3564dca] Merge tag 'riscv-for-linus-6.19-mw1' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux
git bisect bad 07025b51c1149951d64804c73014499bb3564dca
# bad: [b296fda58d1d095c95c8207b09856b2ceafa1397] powerpc/64s/hash: Update directMap page counters for Hash
git bisect bad b296fda58d1d095c95c8207b09856b2ceafa1397
# bad: [39fe29e7f2fd38b0fee9bf987d180dee976dd2c7] powerpc: 86xx: Rename wdt@ nodes to watchdog@
git bisect bad 39fe29e7f2fd38b0fee9bf987d180dee976dd2c7
# bad: [10e1c77c3636d815db802ceef588522c2d2d947c] powerpc/32: Fix unpaired stwcx. on interrupt exit
git bisect bad 10e1c77c3636d815db802ceef588522c2d2d947c
# good: [7afe2383eff05f76f4ce2cfda658b7889c89f101] powerpc/kdump: Fix size calculation for hot-removed memory ranges
git bisect good 7afe2383eff05f76f4ce2cfda658b7889c89f101
# bad: [2997876c4a1a5864baa13d7393c2b68cf5b51183] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
git bisect bad 2997876c4a1a5864baa13d7393c2b68cf5b51183
# good: [98fa236044ca4f8841107382fb03832101fa7328] powerpc/8xx: Remove specific code from fast_exception_return
git bisect good 98fa236044ca4f8841107382fb03832101fa7328
# first bad commit: [2997876c4a1a5864baa13d7393c2b68cf5b51183] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
Re: [PATCH v2] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
Posted by Christophe Leroy (CS GROUP) 1 month, 3 weeks ago
Hi,

Le 17/12/2025 à 17:03, Guenter Roeck a écrit :
> Hi,
> 
> On Thu, Sep 11, 2025 at 02:30:12PM +0200, Christophe Leroy wrote:
>> Commit 13799748b957 ("powerpc/64: use interrupt restart table to speed
>> up return from interrupt") removed the inconditional clearing of
>> MSR[RI] when returning from interrupt into kernel. But powerpc/32
>> doesn't implement interrupt restart table hence still need MSR[RI]
>> to be cleared.
>>
>> It could be added back in interrupt_exit_kernel_prepare() but it is
>> easier and better to add it back in entry_32.S for following reasons:
>> - Writing to MSR must be followed by a synchronising instruction
>> - The smaller the non recoverable section is the better it is
>>
>> So add a macro called clr_ri and use it in the three places that play
>> up with SRR0/SRR1. Use it just before another mtspr for synchronisation
>> to avoid having to add an isync.
>>
>> Now that's done in entry_32.S, exit_must_hard_disable() can return
>> false for non book3s/64, taking into account that BOOKE doesn't have
>> MSR_RI.
>>
>> Also add back blacklisting syscall_exit_finish for kprobe. This was
>> initially added by commit 7cdf44013885 ("powerpc/entry32: Blacklist
>> syscall exit points for kprobe.") then lost with
>> commit 6f76a01173cc ("powerpc/syscall: implement system call
>> entry/exit logic in C for PPC32").
>>
>> Fixes: 6f76a01173cc ("powerpc/syscall: implement system call entry/exit logic in C for PPC32")
>> Fixes: 13799748b957 ("powerpc/64: use interrupt restart table to speed up return from interrupt")
>> Signed-off-by: Christophe Leroy <christophe.leroy@csgroup.eu>
> 
> This patch causes several of my ppc64:ppce500 boot tests with e5500 CPU
> to stall.
> 
> I have not found a common denominator (the proboem seems independent
> of boot media and network interface) , but the problem is repeatable.
> Roughtly 50-70% of boot attempts stall.

I know the issue, was reported by Erhard.

I have a fix under test, will likely send it tomorrow or on friday.

Thanks
Christophe

> 
> Reverting this patch fixes the problem.
> 
> Bisect log is attached for reference.
> 
> Thanks,
> Guenter
> 
> ---
> # bad: [8f0b4cce4481fb22653697cced8d0d04027cb1e8] Linux 6.19-rc1
> # good: [7d0a66e4bb9081d75c82ec4957c50034cb0ea449] Linux 6.18
> git bisect start 'HEAD' 'v6.18'
> # good: [6dfafbd0299a60bfb5d5e277fdf100037c7ded07] Merge tag 'drm-next-2025-12-03' of https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.freedesktop.org%2Fdrm%2Fkernel&data=05%7C02%7Cchristophe.leroy2%40cs-soprasteria.com%7C1c5d3eacabe74f8fcd5f08de3d92c779%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639015897805615312%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=nEi%2F%2ByYYAhR5G2p23tHcDMlfi11SyRFs%2BOXU5PPCWIQ%3D&reserved=0
> git bisect good 6dfafbd0299a60bfb5d5e277fdf100037c7ded07
> # bad: [09cab48db950b6fb8c114314a20c0fd5a80cf990] Merge tag 'soc-arm-6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
> git bisect bad 09cab48db950b6fb8c114314a20c0fd5a80cf990
> # good: [6044a1ee9dca906a807ba786421dc4254191ffd5] Merge tag 'devicetree-for-6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
> git bisect good 6044a1ee9dca906a807ba786421dc4254191ffd5
> # good: [ac20755937e037e586b1ca18a6717d31b1cbce93] Merge tag 'sysctl-6.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl
> git bisect good ac20755937e037e586b1ca18a6717d31b1cbce93
> # good: [4b9d25b4d38035b7b2624afd6852dfe4684f0226] Merge tag 'vfs-6.19-rc1.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
> git bisect good 4b9d25b4d38035b7b2624afd6852dfe4684f0226
> # good: [63a9b0bc65d5d3ea96a57e7985ea22a8582fbbe5] Merge tag 'kvm-riscv-6.19-1' of https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkvm-riscv%2Flinux&data=05%7C02%7Cchristophe.leroy2%40cs-soprasteria.com%7C1c5d3eacabe74f8fcd5f08de3d92c779%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639015897805638856%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=cMLT0je4tG%2FOlzSrrv5Qtrg9UG0CEq%2BuTtJ9Mqjyi0c%3D&reserved=0 into HEAD
> git bisect good 63a9b0bc65d5d3ea96a57e7985ea22a8582fbbe5
> # good: [e0c26d47def7382d7dbd9cad58bc653aed75737a] Merge tag 'kvm-s390-next-6.19-1' of https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Fkvms390%2Flinux&data=05%7C02%7Cchristophe.leroy2%40cs-soprasteria.com%7C1c5d3eacabe74f8fcd5f08de3d92c779%7C8b87af7d86474dc78df45f69a2011bb5%7C0%7C0%7C639015897805654700%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=%2FG7w4h3HaMV%2FZ%2Bk3sLINdVjnrdS4bjkNX8ORcpl7Fkk%3D&reserved=0 into HEAD
> git bisect good e0c26d47def7382d7dbd9cad58bc653aed75737a
> # bad: [07025b51c1149951d64804c73014499bb3564dca] Merge tag 'riscv-for-linus-6.19-mw1' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux
> git bisect bad 07025b51c1149951d64804c73014499bb3564dca
> # bad: [b296fda58d1d095c95c8207b09856b2ceafa1397] powerpc/64s/hash: Update directMap page counters for Hash
> git bisect bad b296fda58d1d095c95c8207b09856b2ceafa1397
> # bad: [39fe29e7f2fd38b0fee9bf987d180dee976dd2c7] powerpc: 86xx: Rename wdt@ nodes to watchdog@
> git bisect bad 39fe29e7f2fd38b0fee9bf987d180dee976dd2c7
> # bad: [10e1c77c3636d815db802ceef588522c2d2d947c] powerpc/32: Fix unpaired stwcx. on interrupt exit
> git bisect bad 10e1c77c3636d815db802ceef588522c2d2d947c
> # good: [7afe2383eff05f76f4ce2cfda658b7889c89f101] powerpc/kdump: Fix size calculation for hot-removed memory ranges
> git bisect good 7afe2383eff05f76f4ce2cfda658b7889c89f101
> # bad: [2997876c4a1a5864baa13d7393c2b68cf5b51183] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
> git bisect bad 2997876c4a1a5864baa13d7393c2b68cf5b51183
> # good: [98fa236044ca4f8841107382fb03832101fa7328] powerpc/8xx: Remove specific code from fast_exception_return
> git bisect good 98fa236044ca4f8841107382fb03832101fa7328
> # first bad commit: [2997876c4a1a5864baa13d7393c2b68cf5b51183] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
> 

Re: [PATCH v2] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
Posted by Madhavan Srinivasan 2 months, 2 weeks ago
On Thu, 11 Sep 2025 14:30:12 +0200, Christophe Leroy wrote:
> Commit 13799748b957 ("powerpc/64: use interrupt restart table to speed
> up return from interrupt") removed the inconditional clearing of
> MSR[RI] when returning from interrupt into kernel. But powerpc/32
> doesn't implement interrupt restart table hence still need MSR[RI]
> to be cleared.
> 
> It could be added back in interrupt_exit_kernel_prepare() but it is
> easier and better to add it back in entry_32.S for following reasons:
> - Writing to MSR must be followed by a synchronising instruction
> - The smaller the non recoverable section is the better it is
> 
> [...]

Applied to powerpc/next.

[1/1] powerpc/32: Restore clearing of MSR[RI] at interrupt/syscall exit
      https://git.kernel.org/powerpc/c/2997876c4a1a5864baa13d7393c2b68cf5b51183

Thanks