[PATCH v4 05/14] x86/traps: Move traps_init() earlier on boot

Andrew Cooper posted 14 patches 3 days, 4 hours ago
[PATCH v4 05/14] x86/traps: Move traps_init() earlier on boot
Posted by Andrew Cooper 3 days, 4 hours ago
We wish to make use of opt_fred earlier on boot, which involves moving
traps_init() earlier, but this comes with several ordering complications.

The feature word containing FRED needs collecting in early_cpu_init(), and
legacy_syscall_init() cannot be called that early because it relies on the
stubs being allocated, yet must be called ahead of cpu_init() so the SYSCALL
linkage MSRs are set up before being cached.

Delaying legacy_syscall_init() is easy enough based on a system_state check.
Reuse bsp_traps_reinit() to cause a call to legacy_syscall_init() to occur at
the same point as previously.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>

v4:
 * New

I don't particualrly like this solution, but the layout of these functions
change for FRED.  Any adjustments need to consider the logic at the end of the
series, not at this point.
---
 xen/arch/x86/cpu/common.c  |  4 +++-
 xen/arch/x86/setup.c       |  4 +++-
 xen/arch/x86/traps-setup.c | 12 +++++++++++-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c
index bfa63fcfb721..5d0523a78b52 100644
--- a/xen/arch/x86/cpu/common.c
+++ b/xen/arch/x86/cpu/common.c
@@ -407,7 +407,9 @@ void __init early_cpu_init(bool verbose)
 		}
 
 		if (max_subleaf >= 1)
-			cpuid_count(7, 1, &eax, &ebx, &ecx,
+			cpuid_count(7, 1,
+                                    &c->x86_capability[FEATURESET_7a1],
+                                    &ebx, &ecx,
 				    &c->x86_capability[FEATURESET_7d1]);
 	}
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 675de3a649ea..0816a713e1c8 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1386,6 +1386,8 @@ void asmlinkage __init noreturn __start_xen(void)
     else
         panic("Bootloader provided no memory information\n");
 
+    traps_init();
+
     /* Choose shadow stack early, to set infrastructure up appropriately. */
     if ( !boot_cpu_has(X86_FEATURE_CET_SS) )
         opt_xen_shstk = 0;
@@ -2078,7 +2080,7 @@ void asmlinkage __init noreturn __start_xen(void)
                                            &this_cpu(stubs).mfn);
     BUG_ON(!this_cpu(stubs.addr));
 
-    traps_init(); /* Needs stubs allocated, must be before presmp_initcalls. */
+    bsp_traps_reinit(); /* Needs stubs allocated, must be before presmp_initcalls. */
 
     cpu_init();
 
diff --git a/xen/arch/x86/traps-setup.c b/xen/arch/x86/traps-setup.c
index c5fc71c75bca..b2c161943d1e 100644
--- a/xen/arch/x86/traps-setup.c
+++ b/xen/arch/x86/traps-setup.c
@@ -346,6 +346,10 @@ void __init traps_init(void)
 
 /*
  * Re-initialise all state referencing the early-boot stack.
+ *
+ * This is called twice during boot, first to ensure legacy_syscall_init() has
+ * run (deferred from earlier), and second when the virtual address of the BSP
+ * stack changes.
  */
 void __init bsp_traps_reinit(void)
 {
@@ -359,7 +363,13 @@ void __init bsp_traps_reinit(void)
  */
 void percpu_traps_init(void)
 {
-    legacy_syscall_init();
+    /*
+     * Skip legacy_syscall_init() at early boot.  It requires the stubs being
+     * allocated, limiting the placement of the traps_init() call, and gets
+     * re-done anyway by bsp_traps_reinit().
+     */
+    if ( system_state > SYS_STATE_early_boot )
+        legacy_syscall_init();
 
     if ( cpu_has_xen_lbr )
         wrmsrl(MSR_IA32_DEBUGCTLMSR, IA32_DEBUGCTLMSR_LBR);
-- 
2.39.5


Re: [PATCH v4 05/14] x86/traps: Move traps_init() earlier on boot
Posted by Jan Beulich 15 hours ago
On 28.02.2026 00:16, Andrew Cooper wrote:
> We wish to make use of opt_fred earlier on boot, which involves moving
> traps_init() earlier, but this comes with several ordering complications.
> 
> The feature word containing FRED needs collecting in early_cpu_init(), and
> legacy_syscall_init() cannot be called that early because it relies on the
> stubs being allocated, yet must be called ahead of cpu_init() so the SYSCALL
> linkage MSRs are set up before being cached.
> 
> Delaying legacy_syscall_init() is easy enough based on a system_state check.
> Reuse bsp_traps_reinit() to cause a call to legacy_syscall_init() to occur at
> the same point as previously.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Irrespective ...

> @@ -359,7 +363,13 @@ void __init bsp_traps_reinit(void)
>   */
>  void percpu_traps_init(void)
>  {
> -    legacy_syscall_init();
> +    /*
> +     * Skip legacy_syscall_init() at early boot.  It requires the stubs being
> +     * allocated, limiting the placement of the traps_init() call, and gets
> +     * re-done anyway by bsp_traps_reinit().
> +     */
> +    if ( system_state > SYS_STATE_early_boot )
> +        legacy_syscall_init();

... I wonder if simply pulling this out of this function wouldn't be slightly
neater. To me at least, syscall/sysenter are only a remote from of "trap".

Jan
Re: [PATCH v4 05/14] x86/traps: Move traps_init() earlier on boot
Posted by Andrew Cooper 11 hours ago
On 02/03/2026 11:39 am, Jan Beulich wrote:
> On 28.02.2026 00:16, Andrew Cooper wrote:
>> We wish to make use of opt_fred earlier on boot, which involves moving
>> traps_init() earlier, but this comes with several ordering complications.
>>
>> The feature word containing FRED needs collecting in early_cpu_init(), and
>> legacy_syscall_init() cannot be called that early because it relies on the
>> stubs being allocated, yet must be called ahead of cpu_init() so the SYSCALL
>> linkage MSRs are set up before being cached.
>>
>> Delaying legacy_syscall_init() is easy enough based on a system_state check.
>> Reuse bsp_traps_reinit() to cause a call to legacy_syscall_init() to occur at
>> the same point as previously.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>

Thanks.

>
> Irrespective ...
>
>> @@ -359,7 +363,13 @@ void __init bsp_traps_reinit(void)
>>   */
>>  void percpu_traps_init(void)
>>  {
>> -    legacy_syscall_init();
>> +    /*
>> +     * Skip legacy_syscall_init() at early boot.  It requires the stubs being
>> +     * allocated, limiting the placement of the traps_init() call, and gets
>> +     * re-done anyway by bsp_traps_reinit().
>> +     */
>> +    if ( system_state > SYS_STATE_early_boot )
>> +        legacy_syscall_init();
> ... I wonder if simply pulling this out of this function wouldn't be slightly
> neater. To me at least, syscall/sysenter are only a remote from of "trap".

I'm not a massive fan of how we (well, Linux) uses "traps" when it's
different from x86 term of the same name.

But, setting up the syscall stub has always been part of traps_init(),
and for FRED it's combined.


As noted, this changes again as FRED gets plumbed in, so really you need
to look at patch 8.  I'm not a massive fan of how it's ended up, but I
can't think of anything simpler.

~Andrew