[Xen-devel] [PATCH] x86: Fix boot with CONFIG_XSM enabled following c/s 7177f589ba

Andrew Cooper posted 1 patch 14 weeks ago
Failed in applying to current master (apply log)
xen/arch/x86/setup.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

[Xen-devel] [PATCH] x86: Fix boot with CONFIG_XSM enabled following c/s 7177f589ba

Posted by Andrew Cooper 14 weeks ago
Currently, booting staging fails with:

  (XEN) Using APIC driver default
  (XEN) ----[ Xen-4.13-unstable  x86_64  debug=y   Not tainted ]----
  (XEN) CPU:    0
  (XEN) RIP:    e008:[<ffff82d08038f66e>] __x86_indirect_thunk_rax+0xe/0x10
  (XEN) RFLAGS: 0000000000010016   CONTEXT: hypervisor
  (XEN) rax: c2c2c2c2c2c2c2c2   rbx: ffff83003f4cc000   rcx: 0000000000000000
  <snip>
  (XEN) Xen code around <ffff82d08038f66e> (__x86_indirect_thunk_rax+0xe/0x10):
  (XEN)  ae e8 eb fb 48 89 04 24 <c3> 90 e8 05 00 00 00 0f ae e8 eb fb 48 89 0c 24
  (XEN) Xen stack trace from rsp=ffff82d080827d28:
  (XEN)    c2c2c2c2c2c2c2c2 ffff82d080207588 ffff82d080827d68 0000000000000000
  <snip>
  (XEN) Xen call trace:
  (XEN)    [<ffff82d08038f66e>] __x86_indirect_thunk_rax+0xe/0x10
  (XEN)    [<ffff82d0806078a9>] setup_system_domains+0x18/0xab
  (XEN)    [<ffff82d08062d9c8>] __start_xen+0x1ea9/0x2935
  (XEN)    [<ffff82d0802000f3>] __high_start+0x53/0x55
  (XEN)
  (XEN) ****************************************
  (XEN) Panic on CPU 0:
  (XEN) GENERAL PROTECTION FAULT
  (XEN) [error_code=0000]
  (XEN) ****************************************

UBSAN (which I happened to have active in my build at the time) identifies the
problem explicitly:

  (XEN) Using APIC driver default
  (XEN) ================================================================================
  (XEN) UBSAN: Undefined behaviour in /local/xen.git/xen/include/xsm/xsm.h:309:19
  (XEN) member access within null pointer of type 'struct xsm_operations'
  (XEN) ----[ Xen-4.13-unstable  x86_64  debug=y   Not tainted ]----

"adjust system domain creation (and call it earlier on x86)" didn't account
for the fact that domain_create() depends on XSM already being set up.  There
is nothing very interesting which xsm_multiboot_init() more than allocating
memory, which means it is safe to move earlier during boot.

The resulting boot now looks like:

  (XEN) Using APIC driver default
  (XEN) XSM Framework v1.0.0 initialized
  (XEN) Flask: 128 avtab hash slots, 283 rules.
  (XEN) Flask: 128 avtab hash slots, 283 rules.
  (XEN) Flask:  4 users, 3 roles, 38 types, 2 bools
  (XEN) Flask:  13 classes, 283 rules
  (XEN) Flask:  Starting in enforcing mode.
  (XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]

and

  (XEN) Using APIC driver default
  (XEN) XSM Framework v1.0.0 initialized
  (XEN) Initialising XSM SILO mode
  (XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wl@xen.org>
CC: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/arch/x86/setup.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 6c3a7ed..d201191 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1533,6 +1533,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
     mmio_ro_ranges = rangeset_new(NULL, "r/o mmio ranges",
                                   RANGESETF_prettyprint_hex);
 
+    xsm_multiboot_init(module_map, mbi);
+
     setup_system_domains();
 
     acpi_boot_init();
@@ -1583,8 +1585,6 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 
     init_IRQ();
 
-    xsm_multiboot_init(module_map, mbi);
-
     microcode_grab_module(module_map, mbi);
 
     timer_init();
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86: Fix boot with CONFIG_XSM enabled following c/s 7177f589ba

Posted by Andrew Cooper 14 weeks ago
On 07/06/2019 13:07, Andrew Cooper wrote:
> Currently, booting staging fails with:
>
>   (XEN) Using APIC driver default
>   (XEN) ----[ Xen-4.13-unstable  x86_64  debug=y   Not tainted ]----
>   (XEN) CPU:    0
>   (XEN) RIP:    e008:[<ffff82d08038f66e>] __x86_indirect_thunk_rax+0xe/0x10
>   (XEN) RFLAGS: 0000000000010016   CONTEXT: hypervisor
>   (XEN) rax: c2c2c2c2c2c2c2c2   rbx: ffff83003f4cc000   rcx: 0000000000000000
>   <snip>
>   (XEN) Xen code around <ffff82d08038f66e> (__x86_indirect_thunk_rax+0xe/0x10):
>   (XEN)  ae e8 eb fb 48 89 04 24 <c3> 90 e8 05 00 00 00 0f ae e8 eb fb 48 89 0c 24
>   (XEN) Xen stack trace from rsp=ffff82d080827d28:
>   (XEN)    c2c2c2c2c2c2c2c2 ffff82d080207588 ffff82d080827d68 0000000000000000
>   <snip>
>   (XEN) Xen call trace:
>   (XEN)    [<ffff82d08038f66e>] __x86_indirect_thunk_rax+0xe/0x10
>   (XEN)    [<ffff82d0806078a9>] setup_system_domains+0x18/0xab
>   (XEN)    [<ffff82d08062d9c8>] __start_xen+0x1ea9/0x2935
>   (XEN)    [<ffff82d0802000f3>] __high_start+0x53/0x55
>   (XEN)
>   (XEN) ****************************************
>   (XEN) Panic on CPU 0:
>   (XEN) GENERAL PROTECTION FAULT
>   (XEN) [error_code=0000]
>   (XEN) ****************************************
>
> UBSAN (which I happened to have active in my build at the time) identifies the
> problem explicitly:
>
>   (XEN) Using APIC driver default
>   (XEN) ================================================================================
>   (XEN) UBSAN: Undefined behaviour in /local/xen.git/xen/include/xsm/xsm.h:309:19
>   (XEN) member access within null pointer of type 'struct xsm_operations'
>   (XEN) ----[ Xen-4.13-unstable  x86_64  debug=y   Not tainted ]----
>
> "adjust system domain creation (and call it earlier on x86)" didn't account
> for the fact that domain_create() depends on XSM already being set up.  There
> is nothing very interesting which xsm_multiboot_init() more than allocating
> memory, which means it is safe to move earlier during boot.

Oh - perhaps it is worth pointing out that we end up following the NULL
function pointer xsm_ops->alloc_security_domain() and execute part of
the 16bit IVT until we end up with the retpoline explosion.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86: Fix boot with CONFIG_XSM enabled following c/s 7177f589ba

Posted by Andrew Cooper 14 weeks ago
On 07/06/2019 13:12, Andrew Cooper wrote:
> On 07/06/2019 13:07, Andrew Cooper wrote:
>> Currently, booting staging fails with:
>>
>>   (XEN) Using APIC driver default
>>   (XEN) ----[ Xen-4.13-unstable  x86_64  debug=y   Not tainted ]----
>>   (XEN) CPU:    0
>>   (XEN) RIP:    e008:[<ffff82d08038f66e>] __x86_indirect_thunk_rax+0xe/0x10
>>   (XEN) RFLAGS: 0000000000010016   CONTEXT: hypervisor
>>   (XEN) rax: c2c2c2c2c2c2c2c2   rbx: ffff83003f4cc000   rcx: 0000000000000000
>>   <snip>
>>   (XEN) Xen code around <ffff82d08038f66e> (__x86_indirect_thunk_rax+0xe/0x10):
>>   (XEN)  ae e8 eb fb 48 89 04 24 <c3> 90 e8 05 00 00 00 0f ae e8 eb fb 48 89 0c 24
>>   (XEN) Xen stack trace from rsp=ffff82d080827d28:
>>   (XEN)    c2c2c2c2c2c2c2c2 ffff82d080207588 ffff82d080827d68 0000000000000000
>>   <snip>
>>   (XEN) Xen call trace:
>>   (XEN)    [<ffff82d08038f66e>] __x86_indirect_thunk_rax+0xe/0x10
>>   (XEN)    [<ffff82d0806078a9>] setup_system_domains+0x18/0xab
>>   (XEN)    [<ffff82d08062d9c8>] __start_xen+0x1ea9/0x2935
>>   (XEN)    [<ffff82d0802000f3>] __high_start+0x53/0x55
>>   (XEN)
>>   (XEN) ****************************************
>>   (XEN) Panic on CPU 0:
>>   (XEN) GENERAL PROTECTION FAULT
>>   (XEN) [error_code=0000]
>>   (XEN) ****************************************
>>
>> UBSAN (which I happened to have active in my build at the time) identifies the
>> problem explicitly:
>>
>>   (XEN) Using APIC driver default
>>   (XEN) ================================================================================
>>   (XEN) UBSAN: Undefined behaviour in /local/xen.git/xen/include/xsm/xsm.h:309:19
>>   (XEN) member access within null pointer of type 'struct xsm_operations'
>>   (XEN) ----[ Xen-4.13-unstable  x86_64  debug=y   Not tainted ]----
>>
>> "adjust system domain creation (and call it earlier on x86)" didn't account
>> for the fact that domain_create() depends on XSM already being set up.  There
>> is nothing very interesting which xsm_multiboot_init() more than allocating
>> memory, which means it is safe to move earlier during boot.
> Oh - perhaps it is worth pointing out that we end up following the NULL
> function pointer xsm_ops->alloc_security_domain() and execute part of
> the 16bit IVT until we end up with the retpoline explosion.

I have adjusted this portion of the commit message to read:

---8<---

"adjust system domain creation (and call it earlier on x86)" didn't account
for the fact that domain_create() depends on XSM already being set up.

Therefore, domain_create() follows xsm_ops->alloc_security_domain() which is
offset 0 from a NULL pointer, meaning that we execute the 16bit IVT until
happening to explode in __x86_indirect_thunk_rax().

There is nothing very interesting which xsm_multiboot_init() more than
allocating memory, which means it is safe to move earlier during setup.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86: Fix boot with CONFIG_XSM enabled following c/s 7177f589ba

Posted by Jan Beulich 14 weeks ago
>>> On 07.06.19 at 14:20, <andrew.cooper3@citrix.com> wrote:
> "adjust system domain creation (and call it earlier on x86)" didn't account
> for the fact that domain_create() depends on XSM already being set up.
> 
> Therefore, domain_create() follows xsm_ops->alloc_security_domain() which is
> offset 0 from a NULL pointer, meaning that we execute the 16bit IVT until
> happening to explode in __x86_indirect_thunk_rax().
> 
> There is nothing very interesting which xsm_multiboot_init() more than
> allocating memory, which means it is safe to move earlier during setup.

The last sentence looks somewhat odd to me (the "which" in particular).
Perhaps you could rephrase that?

Reviewed-by: Jan Beulich <jbeulich@suse.com>

As to not considering XSM - I did specifically check that the respective
Flask function is fine. I forgot that there's an indirect call in between.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] x86: Fix boot with CONFIG_XSM enabled following c/s 7177f589ba

Posted by Andrew Cooper 14 weeks ago
On 07/06/2019 14:13, Jan Beulich wrote:
>>>> On 07.06.19 at 14:20, <andrew.cooper3@citrix.com> wrote:
>> "adjust system domain creation (and call it earlier on x86)" didn't account
>> for the fact that domain_create() depends on XSM already being set up.
>>
>> Therefore, domain_create() follows xsm_ops->alloc_security_domain() which is
>> offset 0 from a NULL pointer, meaning that we execute the 16bit IVT until
>> happening to explode in __x86_indirect_thunk_rax().
>>
>> There is nothing very interesting which xsm_multiboot_init() more than
>> allocating memory, which means it is safe to move earlier during setup.
> The last sentence looks somewhat odd to me (the "which" in particular).
> Perhaps you could rephrase that?

Oops - I meant "very interesting that xsm_multiboot_init() does more"

> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel