[v3] xen/pv: Fix a boot up hang revealed by int3 self test

[Xen-devel] [PATCH v3] xen/pv: Fix a boot up hang revealed by int3 self test

Posted by Zhenzhong Duan 4 years, 9 months ago

Commit 7457c0da024b ("x86/alternatives: Add int3_emulate_call()
selftest") is used to ensure there is a gap setup in int3 exception stack
which could be used for inserting call return address.

This gap is missed in XEN PV int3 exception entry path, then below panic
triggered:

[    0.772876] general protection fault: 0000 [#1] SMP NOPTI
[    0.772886] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.2.0+ #11
[    0.772893] RIP: e030:int3_magic+0x0/0x7
[    0.772905] RSP: 3507:ffffffff82203e98 EFLAGS: 00000246
[    0.773334] Call Trace:
[    0.773334]  alternative_instructions+0x3d/0x12e
[    0.773334]  check_bugs+0x7c9/0x887
[    0.773334]  ? __get_locked_pte+0x178/0x1f0
[    0.773334]  start_kernel+0x4ff/0x535
[    0.773334]  ? set_init_arg+0x55/0x55
[    0.773334]  xen_start_kernel+0x571/0x57a

For 64bit PV guests, Xen's ABI enters the kernel with using SYSRET, with
%rcx/%r11 on the stack. To convert back to "normal" looking exceptions,
the xen thunks do 'xen_*: pop %rcx; pop %r11; jmp *'.

E.g. Extracting 'xen_pv_trap xenint3' we have:
xen_xenint3:
 pop %rcx;
 pop %r11;
 jmp xenint3

As xenint3 and int3 entry code are same except xenint3 doesn't generate
a gap, we can fix it by using int3 and drop useless xenint3.

Signed-off-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 bootup test pass with PV guest.

 v3: set ist_okay to false for int3 per PeterZ
     add Andrew's comments to patch description

 v2: fix up description.
---
 arch/x86/entry/entry_64.S    | 1 -
 arch/x86/include/asm/traps.h | 2 +-
 arch/x86/xen/enlighten_pv.c  | 2 +-
 arch/x86/xen/xen-asm_64.S    | 1 -
 4 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 0ea4831..35a66fc 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -1176,7 +1176,6 @@ idtentry stack_segment		do_stack_segment	has_error_code=1
 #ifdef CONFIG_XEN_PV
 idtentry xennmi			do_nmi			has_error_code=0
 idtentry xendebug		do_debug		has_error_code=0
-idtentry xenint3		do_int3			has_error_code=0
 #endif
 
 idtentry general_protection	do_general_protection	has_error_code=1
diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h
index 7d6f3f3..f2bd284 100644
--- a/arch/x86/include/asm/traps.h
+++ b/arch/x86/include/asm/traps.h
@@ -40,7 +40,7 @@
 asmlinkage void xen_divide_error(void);
 asmlinkage void xen_xennmi(void);
 asmlinkage void xen_xendebug(void);
-asmlinkage void xen_xenint3(void);
+asmlinkage void xen_int3(void);
 asmlinkage void xen_overflow(void);
 asmlinkage void xen_bounds(void);
 asmlinkage void xen_invalid_op(void);
diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c
index 4722ba2..30c14cb 100644
--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -596,12 +596,12 @@ struct trap_array_entry {
 
 static struct trap_array_entry trap_array[] = {
 	{ debug,                       xen_xendebug,                    true },
-	{ int3,                        xen_xenint3,                     true },
 	{ double_fault,                xen_double_fault,                true },
 #ifdef CONFIG_X86_MCE
 	{ machine_check,               xen_machine_check,               true },
 #endif
 	{ nmi,                         xen_xennmi,                      true },
+	{ int3,                        xen_int3,                        false },
 	{ overflow,                    xen_overflow,                    false },
 #ifdef CONFIG_IA32_EMULATION
 	{ entry_INT80_compat,          xen_entry_INT80_compat,          false },
diff --git a/arch/x86/xen/xen-asm_64.S b/arch/x86/xen/xen-asm_64.S
index 1e9ef0b..ebf610b 100644
--- a/arch/x86/xen/xen-asm_64.S
+++ b/arch/x86/xen/xen-asm_64.S
@@ -32,7 +32,6 @@ xen_pv_trap divide_error
 xen_pv_trap debug
 xen_pv_trap xendebug
 xen_pv_trap int3
-xen_pv_trap xenint3
 xen_pv_trap xennmi
 xen_pv_trap overflow
 xen_pv_trap bounds
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] xen/pv: Fix a boot up hang revealed by int3 self test

Posted by Juergen Gross 4 years, 9 months ago

On 14.07.19 11:15, Zhenzhong Duan wrote:
> Commit 7457c0da024b ("x86/alternatives: Add int3_emulate_call()
> selftest") is used to ensure there is a gap setup in int3 exception stack
> which could be used for inserting call return address.
> 
> This gap is missed in XEN PV int3 exception entry path, then below panic
> triggered:
> 
> [    0.772876] general protection fault: 0000 [#1] SMP NOPTI
> [    0.772886] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.2.0+ #11
> [    0.772893] RIP: e030:int3_magic+0x0/0x7
> [    0.772905] RSP: 3507:ffffffff82203e98 EFLAGS: 00000246
> [    0.773334] Call Trace:
> [    0.773334]  alternative_instructions+0x3d/0x12e
> [    0.773334]  check_bugs+0x7c9/0x887
> [    0.773334]  ? __get_locked_pte+0x178/0x1f0
> [    0.773334]  start_kernel+0x4ff/0x535
> [    0.773334]  ? set_init_arg+0x55/0x55
> [    0.773334]  xen_start_kernel+0x571/0x57a
> 
> For 64bit PV guests, Xen's ABI enters the kernel with using SYSRET, with
> %rcx/%r11 on the stack. To convert back to "normal" looking exceptions,
> the xen thunks do 'xen_*: pop %rcx; pop %r11; jmp *'.
> 
> E.g. Extracting 'xen_pv_trap xenint3' we have:
> xen_xenint3:
>   pop %rcx;
>   pop %r11;
>   jmp xenint3
> 
> As xenint3 and int3 entry code are same except xenint3 doesn't generate
> a gap, we can fix it by using int3 and drop useless xenint3.
> 
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v3] xen/pv: Fix a boot up hang revealed by int3 self test

Posted by Juergen Gross 4 years, 9 months ago

On 14.07.19 11:15, Zhenzhong Duan wrote:
> Commit 7457c0da024b ("x86/alternatives: Add int3_emulate_call()
> selftest") is used to ensure there is a gap setup in int3 exception stack
> which could be used for inserting call return address.
> 
> This gap is missed in XEN PV int3 exception entry path, then below panic
> triggered:
> 
> [    0.772876] general protection fault: 0000 [#1] SMP NOPTI
> [    0.772886] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.2.0+ #11
> [    0.772893] RIP: e030:int3_magic+0x0/0x7
> [    0.772905] RSP: 3507:ffffffff82203e98 EFLAGS: 00000246
> [    0.773334] Call Trace:
> [    0.773334]  alternative_instructions+0x3d/0x12e
> [    0.773334]  check_bugs+0x7c9/0x887
> [    0.773334]  ? __get_locked_pte+0x178/0x1f0
> [    0.773334]  start_kernel+0x4ff/0x535
> [    0.773334]  ? set_init_arg+0x55/0x55
> [    0.773334]  xen_start_kernel+0x571/0x57a
> 
> For 64bit PV guests, Xen's ABI enters the kernel with using SYSRET, with
> %rcx/%r11 on the stack. To convert back to "normal" looking exceptions,
> the xen thunks do 'xen_*: pop %rcx; pop %r11; jmp *'.
> 
> E.g. Extracting 'xen_pv_trap xenint3' we have:
> xen_xenint3:
>   pop %rcx;
>   pop %r11;
>   jmp xenint3
> 
> As xenint3 and int3 entry code are same except xenint3 doesn't generate
> a gap, we can fix it by using int3 and drop useless xenint3.
> 
> Signed-off-by: Zhenzhong Duan <zhenzhong.duan@oracle.com>

Pushed to xen/tip.git for-linus-5.3a


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel