[PATCH v2 05/11] x86: add CONFIG_X86_64_NATIVE option

Arnd Bergmann posted 11 patches 1 year ago
There is a newer version of this series
[PATCH v2 05/11] x86: add CONFIG_X86_64_NATIVE option
Posted by Arnd Bergmann 1 year ago
From: Arnd Bergmann <arnd@arndb.de>

As a replacement for the obsole MK8/MPSC/MCORE2 configuration options,
allow building a specialized kernel for the local CPU, which is useful
for users building their own kernels, and does not require maintaining
a list of possible CPU options.

Between -march=native and -mtune=native, I pick the former in order
to give the best performance through the use of extra instructions,
but at the cost of not being able to run on older CPUs at all. This
creates a small risk of running into illegal instruction faults when
the resulting binary ends up being run on a machine other than the
one it was built on.

Link: https://lore.kernel.org/lkml/CAHk-=wji1sV93yKbc==Z7OSSHBiDE=LAdG_d5Y-zPBrnSs0k2A@mail.gmail.com/
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 arch/x86/Kconfig.cpu | 14 ++++++++++++++
 arch/x86/Makefile    |  5 +++++
 2 files changed, 19 insertions(+)

diff --git a/arch/x86/Kconfig.cpu b/arch/x86/Kconfig.cpu
index 8fcb8ccee44b..d634b163e913 100644
--- a/arch/x86/Kconfig.cpu
+++ b/arch/x86/Kconfig.cpu
@@ -257,6 +257,20 @@ config X86_GENERIC
 	  This is really intended for distributors who need more
 	  generic optimizations.
 
+config X86_64_NATIVE
+	bool "Build with -march=native optimization"
+	depends on X86_64
+	help
+	  Make it possible to have a slightly better optimized kernel for
+	  the machine it is built on, by passing -march=native instead
+	  the more generic -march=x86-64 option.  This lets compilers
+	  use extensions to the x86-64 instruction set that were not
+	  present in the original AMD Opteron and Intel Pentium4 CPUs,
+	  and schedule instructions for the pipeline model.
+
+	  Select this option only when building a kernel to run locally,
+	  as it may be incompatible with any other processor.
+
 #
 # Define implied options from the CPU selection here
 config X86_INTERNODE_CACHE_SHIFT
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 8120085b00a4..bf45b84c138f 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -178,8 +178,13 @@ else
 	# Use -mskip-rax-setup if supported.
 	KBUILD_CFLAGS += $(call cc-option,-mskip-rax-setup)
 
+ifdef CONFIG_X86_64_NATIVE
+        KBUILD_CFLAGS += -march=native
+        KBUILD_RUSTFLAGS += -Ctarget-cpu=native
+else
         KBUILD_CFLAGS += -march=x86-64 -mtune=generic
         KBUILD_RUSTFLAGS += -Ctarget-cpu=x86-64 -Ztune-cpu=generic
+endif
 
         KBUILD_CFLAGS += -mno-red-zone
         KBUILD_CFLAGS += -mcmodel=kernel
-- 
2.39.5
Re: [PATCH v2 05/11] x86: add CONFIG_X86_64_NATIVE option
Posted by Josh Triplett 1 year ago
On Tue, Dec 10, 2024 at 03:49:39PM +0100, Arnd Bergmann wrote:
> From: Arnd Bergmann <arnd@arndb.de>
> 
> As a replacement for the obsole MK8/MPSC/MCORE2 configuration options,
> allow building a specialized kernel for the local CPU, which is useful
> for users building their own kernels, and does not require maintaining
> a list of possible CPU options.
> 
> Between -march=native and -mtune=native, I pick the former in order
> to give the best performance through the use of extra instructions,
> but at the cost of not being able to run on older CPUs at all. This
> creates a small risk of running into illegal instruction faults when
> the resulting binary ends up being run on a machine other than the
> one it was built on.

Setting this option makes the kernel build non-reproducible (as it'll
build differently on different systems).

I personally think this is a little questionable, as opposed to having
well-defined optimization levels that are tested and supported and
selecting one of those with a `make nativeconfig`. But if this does get
added:

- Could you please document in
  Documentation/kbuild/reproducible-builds.rst that this option should
  not be set if you want reproducible builds?

- Could you add something similar to the config options LTO uses, to
  make `allyesconfig` and `allmodconfig` turn this back *off*?
Re: [PATCH v2 05/11] x86: add CONFIG_X86_64_NATIVE option
Posted by irecca.kun@gmail.com 1 year ago
Hello.

On 12/10/24 14:49, Arnd Bergmann wrote:
> As a replacement for the obsole MK8/MPSC/MCORE2 configuration options,
> allow building a specialized kernel for the local CPU, which is useful
> for users building their own kernels, and does not require maintaining
> a list of possible CPU options.

That potentially introduces problems. Namely compilers could apply auto-vectorization, which currently breaks the kernel.
We probably need an additional patch like this: https://github.com/zen-kernel/zen-kernel/commit/95b7981ba2e5c86529de0e895c2d9e428aa3f7dc
Re: [PATCH v2 05/11] x86: add CONFIG_X86_64_NATIVE option
Posted by Arnd Bergmann 1 year ago
On Tue, Dec 10, 2024, at 20:05, irecca.kun@gmail.com wrote:
> Hello.
>
> On 12/10/24 14:49, Arnd Bergmann wrote:
>> As a replacement for the obsole MK8/MPSC/MCORE2 configuration options,
>> allow building a specialized kernel for the local CPU, which is useful
>> for users building their own kernels, and does not require maintaining
>> a list of possible CPU options.
>
> That potentially introduces problems. Namely compilers could apply 
> auto-vectorization, which currently breaks the kernel.
> We probably need an additional patch like this: 
> https://github.com/zen-kernel/zen-kernel/commit/95b7981ba2e5c86529de0e895c2d9e428aa3f7dc

I don't see how either  -mno-avx2 or -fno-tree-vectorize would be
needed here: avx2 is already turned off because of -mno-avx,
and -ftree-vectorize is only enabled by default at -O3 level,
which we don't use (and which doesn't add any instructions).
There may be other flags that need to be disabled though.

With the flags we currently pass to the kernel, this is the
difference in the gcc-14 -Q --help=target output between the
x86-64 baseline and the -march=emeraldrapids, which is currently
the most featureful:

--- b	2024-12-10 21:37:27.182448452 +0100
+++ a	2024-12-10 21:41:14.118310513 +0100
@@ -12,24 +12,24 @@
   -mabm                       		[disabled]
   -maccumulate-outgoing-args  		[disabled]
   -maddress-mode=             		long
-  -madx                       		[disabled]
-  -maes                       		[disabled]
+  -madx                       		[enabled]
+  -maes                       		[enabled]
   -malign-data=               		compat
   -malign-double              		[disabled]
   -malign-functions=          		0
   -malign-jumps=              		0
   -malign-loops=              		0
   -malign-stringops           		[enabled]
-  -mamx-bf16                  		[disabled]
+  -mamx-bf16                  		[enabled]
   -mamx-complex               		[disabled]
   -mamx-fp16                  		[disabled]
-  -mamx-int8                  		[disabled]
-  -mamx-tile                  		[disabled]
+  -mamx-int8                  		[enabled]
+  -mamx-tile                  		[enabled]
   -mandroid                   		[disabled]
   -mapx-features=             		none
   -mapx-inline-asm-use-gpr32  		[disabled]
   -mapxf                      		[disabled]
-  -march=                     		x86-64
+  -march=                     		emeraldrapids
   -masm=                      		att
   -mavx                       		[disabled]
   -mavx10.1                   		-mavx10.1-256
@@ -62,27 +62,27 @@
   -mavxvnniint16              		[disabled]
   -mavxvnniint8               		[disabled]
   -mbionic                    		[disabled]
-  -mbmi                       		[disabled]
-  -mbmi2                      		[disabled]
+  -mbmi                       		[enabled]
+  -mbmi2                      		[enabled]
   -mbranch-cost=<0,5>         		3
   -mcall-ms2sysv-xlogues      		[disabled]
   -mcet-switch                		[disabled]
   -mcld                       		[disabled]
-  -mcldemote                  		[disabled]
-  -mclflushopt                		[disabled]
-  -mclwb                      		[disabled]
+  -mcldemote                  		[enabled]
+  -mclflushopt                		[enabled]
+  -mclwb                      		[enabled]
   -mclzero                    		[disabled]
   -mcmodel=                   		kernel
   -mcmpccxadd                 		[disabled]
   -mcpu=                      		
   -mcrc32                     		[disabled]
-  -mcx16                      		[disabled]
+  -mcx16                      		[enabled]
   -mdaz-ftz                   		[disabled]
   -mdirect-extern-access      		[enabled]
   -mdispatch-scheduler        		[disabled]
   -mdump-tune-features        		[disabled]
-  -menqcmd                    		[disabled]
-  -mevex512                   		[disabled]
+  -menqcmd                    		[enabled]
+  -mevex512                   		[enabled]
   -mf16c                      		[disabled]
   -mfancy-math-387            		[disabled]
   -mfentry                    		[disabled]
@@ -94,17 +94,17 @@
   -mforce-indirect-call       		[disabled]
   -mfp-ret-in-387             		[disabled]
   -mfpmath=                   		387
-  -mfsgsbase                  		[disabled]
+  -mfsgsbase                  		[enabled]
   -mfunction-return=          		keep
   -mfused-madd                		-ffp-contract=fast
   -mfxsr                      		[enabled]
   -mgather                    		-mtune-ctrl=use_gather
   -mgeneral-regs-only         		[disabled]
-  -mgfni                      		[disabled]
+  -mgfni                      		[enabled]
   -mglibc                     		[enabled]
   -mhard-float                		[disabled]
   -mharden-sls=               		none
-  -mhle                       		[disabled]
+  -mhle                       		[enabled]
   -mhreset                    		[disabled]
   -miamcu                     		[disabled]
   -mieee-fp                   		[enabled]
@@ -123,15 +123,15 @@
   -mlong-double-64            		[disabled]
   -mlong-double-80            		[enabled]
   -mlwp                       		[disabled]
-  -mlzcnt                     		[disabled]
+  -mlzcnt                     		[enabled]
   -mmanual-endbr              		[disabled]
   -mmemcpy-strategy=          		
   -mmemset-strategy=          		
   -mmitigate-rop              		[disabled]
   -mmmx                       		[disabled]
-  -mmovbe                     		[disabled]
-  -mmovdir64b                 		[disabled]
-  -mmovdiri                   		[disabled]
+  -mmovbe                     		[enabled]
+  -mmovdir64b                 		[enabled]
+  -mmovdiri                   		[enabled]
   -mmove-max=                 		128
   -mmpx                       		[disabled]
   -mms-bitfields              		[disabled]
@@ -152,23 +152,23 @@
   -mpc32                      		[disabled]
   -mpc64                      		[disabled]
   -mpc80                      		[disabled]
-  -mpclmul                    		[disabled]
+  -mpclmul                    		[enabled]
   -mpcommit                   		[disabled]
-  -mpconfig                   		[disabled]
-  -mpku                       		[disabled]
-  -mpopcnt                    		[disabled]
+  -mpconfig                   		[enabled]
+  -mpku                       		[enabled]
+  -mpopcnt                    		[enabled]
   -mprefer-avx128             		-mprefer-vector-width=128
   -mprefer-vector-width=      		none
   -mpreferred-stack-boundary= 		3
   -mprefetchi                 		[disabled]
   -mprefetchwt1               		[disabled]
-  -mprfchw                    		[disabled]
-  -mptwrite                   		[disabled]
+  -mprfchw                    		[enabled]
+  -mptwrite                   		[enabled]
   -mpush-args                 		[enabled]
   -mraoint                    		[disabled]
-  -mrdpid                     		[disabled]
-  -mrdrnd                     		[disabled]
-  -mrdseed                    		[disabled]
+  -mrdpid                     		[enabled]
+  -mrdrnd                     		[enabled]
+  -mrdseed                    		[enabled]
   -mrecip                     		[disabled]
   -mrecip=                    		
   -mrecord-mcount             		[disabled]
@@ -178,11 +178,11 @@
   -mrelax-cmpxchg-loop        		[disabled]
   -mrtd                       		[disabled]
   -mrtm                       		[disabled]
-  -msahf                      		[disabled]
+  -msahf                      		[enabled]
   -mscatter                   		-mtune-ctrl=use_scatter
-  -mserialize                 		[disabled]
-  -msgx                       		[disabled]
-  -msha                       		[disabled]
+  -mserialize                 		[enabled]
+  -msgx                       		[enabled]
+  -msha                       		[enabled]
   -msha512                    		[disabled]
   -mshstk                     		[disabled]
   -mskip-rax-setup            		[enabled]
@@ -212,11 +212,11 @@
   -mtbm                       		[disabled]
   -mtls-dialect=              		gnu
   -mtls-direct-seg-refs       		[enabled]
-  -mtsxldtrk                  		[disabled]
+  -mtsxldtrk                  		[enabled]
   -mtune-ctrl=                		
   -mtune=                     		generic
   -muclibc                    		[disabled]
-  -muintr                     		[disabled]
+  -muintr                     		[enabled]
   -munroll-only-small-loops   		[enabled]
   -musermsr                   		[disabled]
   -mvaes                      		[disabled]
@@ -224,15 +224,15 @@
   -mvect8-ret-in-mem          		[disabled]
   -mvpclmulqdq                		[disabled]
   -mvzeroupper                		[enabled]
-  -mwaitpkg                   		[disabled]
-  -mwbnoinvd                  		[disabled]
+  -mwaitpkg                   		[enabled]
+  -mwbnoinvd                  		[enabled]
   -mwidekl                    		[disabled]
   -mx32                       		[disabled]
   -mxop                       		[disabled]
-  -mxsave                     		[disabled]
-  -mxsavec                    		[disabled]
-  -mxsaveopt                  		[disabled]
-  -mxsaves                    		[disabled]
+  -mxsave                     		[enabled]
+  -mxsavec                    		[enabled]
+  -mxsaveopt                  		[enabled]
+  -mxsaves                    		[enabled]
 
I don't know what most of them do, but the ones I looked
up seem to be mainly integer operations.

     Arnd
Re: [PATCH v2 05/11] x86: add CONFIG_X86_64_NATIVE option
Posted by irecca.kun@gmail.com 1 year ago
On 12/11/24 01:56, Arnd Bergmann wrote:
> -ftree-vectorize is only enabled by default at -O3 level

You maybe missed it, but vectorization is enabled at -O2 level since GCC 12. https://www.phoronix.com/news/GCC-12-Auto-Vec-O2
Same goes for clang, it pretty much always had vectorized on -O2, iirc.

> avx2 is already turned off because of -mno-avx,

Hmm, I'm not sure. But if we look at existing KBUILD_RUSTFLAGS below, it has both -avx and -avx2.