[v1] x86: allow Kconfig control over psABI level

[PATCH 1/3] x86: allow Kconfig control over psABI level

Posted by Jan Beulich 2 years, 7 months ago

Newer hardware offers more efficient and/or flexible and/or capable
instructions, some of which we can make good use of in the hypervisor
as well. Allow a basic way (no alternatives patching) of enabling their
use. Of course this means that hypervisors thus built won't work
anymore on older, less capable hardware.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
TBD: Should we, just like for NX, add an early check in assembly code,
     to have a "clean" failure rather than a random crash?

Whereas the baseline -> v2 step isn't much of a difference (we'll gain
more there by a subsequent patch), v2 -> v3, while presumably (or shall
I say hopefully) faster, yields an overall growth of .text size by (in
my build) about 2k. The primary reason for this appear to be conversions
of SHL-by-immediate to SHLX.

--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -118,6 +118,36 @@ config HVM
 
 	  If unsure, say Y.
 
+choice
+	prompt "base psABI level"
+	default X86_64_BASELINE
+	help
+	  The psABI defines 4 levels of ISA extension sets as a coarse granular
+	  way of identifying advanced functionality that would be uniformly
+	  available in respectively newer hardware.  While v4 is not really of
+	  interest for Xen, the others can be selected here, making the
+	  resulting Xen no longer work on older hardware.  This option won't
+	  have any effect if the toolchain doesn't support the distinction.
+
+	  If unsure, stick to the default.
+
+config X86_64_BASELINE
+	bool "baseline"
+
+config X86_64_V2
+	bool "v2"
+	help
+	  This enables POPCNT and CX16, besides other extensions which are of
+	  no interest here.
+
+config X86_64_V3
+	bool "v3"
+	help
+	  This enables BMI, BMI2, LZCNT, and MOVBE, besides other extensions
+	  which are of no interest here.
+
+endchoice
+
 config XEN_SHSTK
 	bool "Supervisor Shadow Stacks"
 	depends on HAS_AS_CET_SS
--- a/xen/arch/x86/arch.mk
+++ b/xen/arch/x86/arch.mk
@@ -36,6 +36,10 @@ CFLAGS += -mno-red-zone -fpic
 # the SSE setup for variadic function calls.
 CFLAGS += -mno-mmx -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
 
+# Enable the selected baseline ABI, if supported by the compiler.
+CFLAGS-$(CONFIG_X86_64_V2) += $(call cc-option,$(CC),-march=x86-64-v2)
+CFLAGS-$(CONFIG_X86_64_V3) += $(call cc-option,$(CC),-march=x86-64-v3)
+
 ifeq ($(CONFIG_INDIRECT_THUNK),y)
 # Compile with gcc thunk-extern, indirect-branch-register if available.
 CFLAGS-$(CONFIG_CC_IS_GCC) += -mindirect-branch=thunk-extern

Re: [PATCH 1/3] x86: allow Kconfig control over psABI level

Posted by Jan Beulich 2 years, 6 months ago

On 12.07.2023 14:33, Jan Beulich wrote:
> Newer hardware offers more efficient and/or flexible and/or capable
> instructions, some of which we can make good use of in the hypervisor
> as well. Allow a basic way (no alternatives patching) of enabling their
> use. Of course this means that hypervisors thus built won't work
> anymore on older, less capable hardware.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> ---
> TBD: Should we, just like for NX, add an early check in assembly code,
>      to have a "clean" failure rather than a random crash?

TBD: While older compilers (apparently gcc10 and older; not sure about
     clang) won't recognize -march=x86-64-v2 etc, we could fall back to
     passing -mpopcnt and alike explicitly.

Jan

> Whereas the baseline -> v2 step isn't much of a difference (we'll gain
> more there by a subsequent patch), v2 -> v3, while presumably (or shall
> I say hopefully) faster, yields an overall growth of .text size by (in
> my build) about 2k. The primary reason for this appear to be conversions
> of SHL-by-immediate to SHLX.
> 
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -118,6 +118,36 @@ config HVM
>  
>  	  If unsure, say Y.
>  
> +choice
> +	prompt "base psABI level"
> +	default X86_64_BASELINE
> +	help
> +	  The psABI defines 4 levels of ISA extension sets as a coarse granular
> +	  way of identifying advanced functionality that would be uniformly
> +	  available in respectively newer hardware.  While v4 is not really of
> +	  interest for Xen, the others can be selected here, making the
> +	  resulting Xen no longer work on older hardware.  This option won't
> +	  have any effect if the toolchain doesn't support the distinction.
> +
> +	  If unsure, stick to the default.
> +
> +config X86_64_BASELINE
> +	bool "baseline"
> +
> +config X86_64_V2
> +	bool "v2"
> +	help
> +	  This enables POPCNT and CX16, besides other extensions which are of
> +	  no interest here.
> +
> +config X86_64_V3
> +	bool "v3"
> +	help
> +	  This enables BMI, BMI2, LZCNT, and MOVBE, besides other extensions
> +	  which are of no interest here.
> +
> +endchoice
> +
>  config XEN_SHSTK
>  	bool "Supervisor Shadow Stacks"
>  	depends on HAS_AS_CET_SS
> --- a/xen/arch/x86/arch.mk
> +++ b/xen/arch/x86/arch.mk
> @@ -36,6 +36,10 @@ CFLAGS += -mno-red-zone -fpic
>  # the SSE setup for variadic function calls.
>  CFLAGS += -mno-mmx -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
>  
> +# Enable the selected baseline ABI, if supported by the compiler.
> +CFLAGS-$(CONFIG_X86_64_V2) += $(call cc-option,$(CC),-march=x86-64-v2)
> +CFLAGS-$(CONFIG_X86_64_V3) += $(call cc-option,$(CC),-march=x86-64-v3)
> +
>  ifeq ($(CONFIG_INDIRECT_THUNK),y)
>  # Compile with gcc thunk-extern, indirect-branch-register if available.
>  CFLAGS-$(CONFIG_CC_IS_GCC) += -mindirect-branch=thunk-extern
> 
>

Re: [PATCH 1/3] x86: allow Kconfig control over psABI level

Posted by Jason Andryuk 2 years, 6 months ago

On Wed, Jul 12, 2023 at 8:34 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> Newer hardware offers more efficient and/or flexible and/or capable
> instructions, some of which we can make good use of in the hypervisor
> as well. Allow a basic way (no alternatives patching) of enabling their
> use. Of course this means that hypervisors thus built won't work
> anymore on older, less capable hardware.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

With two comments.

> ---
> TBD: Should we, just like for NX, add an early check in assembly code,
>      to have a "clean" failure rather than a random crash?

That is less confusing than a random crash.

> Whereas the baseline -> v2 step isn't much of a difference (we'll gain
> more there by a subsequent patch), v2 -> v3, while presumably (or shall
> I say hopefully) faster, yields an overall growth of .text size by (in
> my build) about 2k. The primary reason for this appear to be conversions
> of SHL-by-immediate to SHLX.
>
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -118,6 +118,36 @@ config HVM
>
>           If unsure, say Y.
>
> +choice
> +       prompt "base psABI level"
> +       default X86_64_BASELINE
> +       help
> +         The psABI defines 4 levels of ISA extension sets as a coarse granular
> +         way of identifying advanced functionality that would be uniformly
> +         available in respectively newer hardware.  While v4 is not really of
> +         interest for Xen, the others can be selected here, making the
> +         resulting Xen no longer work on older hardware.  This option won't
> +         have any effect if the toolchain doesn't support the distinction.
> +
> +         If unsure, stick to the default.
> +
> +config X86_64_BASELINE
> +       bool "baseline"
> +
> +config X86_64_V2
> +       bool "v2"
> +       help
> +         This enables POPCNT and CX16, besides other extensions which are of
> +         no interest here.
> +
> +config X86_64_V3
> +       bool "v3"
> +       help
> +         This enables BMI, BMI2, LZCNT, and MOVBE, besides other extensions
> +         which are of no interest here.

Would it make sense to try and specify models?  According to this:
https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels

x86-64-v2
(circa 2009: Nehalem and Jaguar)
Atom Silvermont (2013)
VIA Nano and Eden "C" (2015)

x86-64-v3
(circa 2015: Haswell and Excavator)
Atom Gracemont (2021)

Regards,
Jason

Re: [PATCH 1/3] x86: allow Kconfig control over psABI level

Posted by Jan Beulich 2 years, 6 months ago

On 17.07.2023 14:23, Jason Andryuk wrote:
> On Wed, Jul 12, 2023 at 8:34 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> Newer hardware offers more efficient and/or flexible and/or capable
>> instructions, some of which we can make good use of in the hypervisor
>> as well. Allow a basic way (no alternatives patching) of enabling their
>> use. Of course this means that hypervisors thus built won't work
>> anymore on older, less capable hardware.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> Reviewed-by: Jason Andryuk <jandryuk@gmail.com>

Thanks.

> With two comments.
> 
>> ---
>> TBD: Should we, just like for NX, add an early check in assembly code,
>>      to have a "clean" failure rather than a random crash?
> 
> That is less confusing than a random crash.

Of course. My thinking behind not doing this right away was that people
aren't supposed to use such a hypervisor on incapable hardware. So we
would likely add code for something which ought to never happen. Still
for v2 I guess I'll add such code (then of course invalidating your R-b
again); in the worst case it should be easy to rip out again from the
patch.

>> --- a/xen/arch/x86/Kconfig
>> +++ b/xen/arch/x86/Kconfig
>> @@ -118,6 +118,36 @@ config HVM
>>
>>           If unsure, say Y.
>>
>> +choice
>> +       prompt "base psABI level"
>> +       default X86_64_BASELINE
>> +       help
>> +         The psABI defines 4 levels of ISA extension sets as a coarse granular
>> +         way of identifying advanced functionality that would be uniformly
>> +         available in respectively newer hardware.  While v4 is not really of
>> +         interest for Xen, the others can be selected here, making the
>> +         resulting Xen no longer work on older hardware.  This option won't
>> +         have any effect if the toolchain doesn't support the distinction.
>> +
>> +         If unsure, stick to the default.
>> +
>> +config X86_64_BASELINE
>> +       bool "baseline"
>> +
>> +config X86_64_V2
>> +       bool "v2"
>> +       help
>> +         This enables POPCNT and CX16, besides other extensions which are of
>> +         no interest here.
>> +
>> +config X86_64_V3
>> +       bool "v3"
>> +       help
>> +         This enables BMI, BMI2, LZCNT, and MOVBE, besides other extensions
>> +         which are of no interest here.
> 
> Would it make sense to try and specify models?  According to this:
> https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels
> 
> x86-64-v2
> (circa 2009: Nehalem and Jaguar)
> Atom Silvermont (2013)
> VIA Nano and Eden "C" (2015)
> 
> x86-64-v3
> (circa 2015: Haswell and Excavator)
> Atom Gracemont (2021)

I'm not sure how useful this would be without supplying a reasonably
complete list of models. I don't think we can expect people to be aware
of the sequence of model names, without which specifying just the first
respective models isn't very useful.

Jan

Re: [PATCH 1/3] x86: allow Kconfig control over psABI level

Posted by Jan Beulich 2 years, 7 months ago

On 12.07.2023 14:33, Jan Beulich wrote:
> --- a/xen/arch/x86/arch.mk
> +++ b/xen/arch/x86/arch.mk
> @@ -36,6 +36,10 @@ CFLAGS += -mno-red-zone -fpic
>  # the SSE setup for variadic function calls.
>  CFLAGS += -mno-mmx -mno-sse $(call cc-option,$(CC),-mskip-rax-setup)
>  
> +# Enable the selected baseline ABI, if supported by the compiler.
> +CFLAGS-$(CONFIG_X86_64_V2) += $(call cc-option,$(CC),-march=x86-64-v2)
> +CFLAGS-$(CONFIG_X86_64_V3) += $(call cc-option,$(CC),-march=x86-64-v3)

Hmm, I should have remembered that this won't work with older gcc,
because of how $(cc-option ...) works. I recall someone else already
fell into this trap a few years ago, but then they weren't following
up on the suggestions on how to make their somewhat fragile workaround
more robust. Now I'm on the hook ...

Jan