From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 581F024DCF6; Thu, 13 Aug 2026 02:20:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587620; cv=none; b=rFyFRzU1KUaNnaxWH3UZr0JL+EjDoPaSKuXxJDMzf0ZU8htURolVMNDbNZ2KrPa7QMBBNfVnuiO8kLWxYURiKaAg4z+q8F9aogBEunbZikn7z7vUiwRZ30Kf+ZMwuba1jIIEMzEW9oL29PHBFyKDLD40dIu+3vXqRH+JozqR5+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587620; c=relaxed/simple; bh=zH06ZKSgGhEhXr0gPfiGkd8DxI44+kr5wz8zEOUWQZk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fgo7OUMxSEwOo8b5420tSUbi3x2bIwt37Cgb0F4Yu9YYLvXBHsvcJoWRLFjMGHUnNap+cfkZ61LkhA48LrbQv6OBJ77oVEEAF6ZD4shSYCt1zbAQEH1H1UT7TUgHuqesxhtRg3YnSSfeaU0JHe5bHAvB5wtnXpXWbpUU3YHqaiw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=b54acgNZ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="b54acgNZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E79EC1F00A3E; Thu, 13 Aug 2026 02:20:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587619; bh=56BaSgitaCOqK+plEdyRoTXW75Ul7dUQbsnofeonq2Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=b54acgNZyd8vGhYdRRYpbxz0Zc/bz1/1S1fnQak8e2Nl7ogo5RpUmPtlcBNNFArys K1u/fPN5MXNcEDv72S1QDo34/b8WlNoYXzYIln+qlkx+WBLJlXM+Bujz0iO5v63IcN mFm6W+IrR/O1hZjHj29TU+abXdRAQk+cI6osQVNL2CxFwlpj/lckaQYD6JKkCjJM1V +2a5Ebc6Bc4j4CgR1qO15zFBfsFv/XAtnWOsn3aoJfaR0M5rp2jlaetOUB8Gyn2IMq EVDwe4Uxjr+r/qhY9a4myDhzsaJh/lOwEZVcn7QOjANoQDALnf5S+4HYgnebySY17E TuaxZpBtTB8uA== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v3 1/8] x86/fpu: Check for missing AVX and AVX-512 xstate bits Date: Wed, 12 Aug 2026 19:14:59 -0700 Message-ID: <20260813021506.55129-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If the CPU declares AVX or AVX-512 support, verify that the corresponding xstate bits are also set. If not, warn and clear them. This eliminates the perceived need for AVX and AVX-512 optimized code in the kernel to call cpu_has_xfeatures(). That has never been universally done, which strongly suggests that it has never really been needed in practice, but this should remove any remaining doubt. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/x86/kernel/fpu/xstate.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index a7b6524a9dea..97cfd4fb6cc0 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -806,7 +806,7 @@ static u64 __init guest_default_mask(void) void __init fpu__init_system_xstate(unsigned int legacy_size) { unsigned int eax, ebx, ecx, edx; - u64 xfeatures; + u64 xfeatures, mask; int err; int i; =20 @@ -818,6 +818,8 @@ void __init fpu__init_system_xstate(unsigned int legacy= _size) if (!boot_cpu_has(X86_FEATURE_XSAVE)) { pr_info("x86/fpu: x87 FPU will use %s\n", boot_cpu_has(X86_FEATURE_FXSR) ? "FXSAVE" : "FSAVE"); + /* Disable all dependent flags too */ + setup_clear_cpu_cap(X86_FEATURE_XSAVE); return; } =20 @@ -833,7 +835,8 @@ void __init fpu__init_system_xstate(unsigned int legacy= _size) cpuid_count(CPUID_LEAF_XSTATE, 1, &eax, &ebx, &ecx, &edx); fpu_kernel_cfg.max_features |=3D ecx + ((u64)edx << 32); =20 - if ((fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSSE) !=3D XFEATURE_MAS= K_FPSSE) { + mask =3D XFEATURE_MASK_FPSSE; + if ((fpu_kernel_cfg.max_features & mask) !=3D mask) { /* * This indicates that something really unexpected happened * with the enumeration. Disable XSAVE and try to continue @@ -844,6 +847,24 @@ void __init fpu__init_system_xstate(unsigned int legac= y_size) goto out_disable; } =20 + mask |=3D XFEATURE_MASK_YMM; + if (boot_cpu_has(X86_FEATURE_AVX)) { + if ((fpu_kernel_cfg.max_features & mask) !=3D mask) { + pr_err(FW_BUG + "x86/fpu: Disabling AVX support due to missing xstate features\n= "); + setup_clear_cpu_cap(X86_FEATURE_AVX); + } + } + + mask |=3D XFEATURE_MASK_AVX512; + if (boot_cpu_has(X86_FEATURE_AVX512F)) { + if ((fpu_kernel_cfg.max_features & mask) !=3D mask) { + pr_err(FW_BUG + "x86/fpu: Disabling AVX-512 support due to missing xstate featur= es\n"); + setup_clear_cpu_cap(X86_FEATURE_AVX512F); + } + } + if (fpu_kernel_cfg.max_features & XFEATURE_MASK_APX && fpu_kernel_cfg.max_features & (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_= BNDCSR)) { /* --=20 2.55.0 From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F2E52ED141; Thu, 13 Aug 2026 02:20:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587620; cv=none; b=UUvWV/DKCepxyY9tagVfBa0mXkQf+MsbwtqpRN/mzGfbieVdBONWeo5aDo5rkbzpixJ/bt6bXCqqHfzj9mxyGcazEhGrZqVUGG3y20OixsHDrDrXR56cLX8yq0zwMbrHYj1a1ghMSKMS9wQ9rKFvYprzloenyqSCBWxUTIG3+xg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587620; c=relaxed/simple; bh=7Z5v95LGHs2mjLET21NVji2yunu/nW8nT1iilSCIx10=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=abnsaRQqRJDUJ79LKnQpHLN4s3hy5rCveMVu6Sku1dDcvUNkRNu+s5/Kw77aJOIiji1hVPvCrtnQhUFTu2DFbaxIIXYfokrbiwBay2ZiJYK+J4Blzm/fFfIJzmiyOme/szgsEGwt/3Xbox9zWAoQ/twTpnim45bEYqpIT5tiAVs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=axPBaESm; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="axPBaESm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4E9AE1F00A3D; Thu, 13 Aug 2026 02:20:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587619; bh=WCMvK5ohQOFymbdY0OzMDpa7FyF9XhAAUoMQkc808Wk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=axPBaESmVMptnpIxS/8qhUzNmdTdK4y6Ur/oNMoWkpl1HCNLKwK7jaPE/ad4k6RX9 q/wrtaXEQAtYb0LAFhmlEvYEAtgz2hKHj17zXEYcuO7fxQ112BO3t+q6pkEHLKZ6Ef Oq9bZqeMgLm6A+nueM1H2MxChVPAHtCw3zXPa6sqJWuqzp2eeK9vUFz77nppsqAO6K nglfjCLmW5ajm03ybT5gasGo91QRlcyyISv4H74M7SOhHwqyB44xN4A3KdtmH9At6j bVwwFsvuslG0Im+JWXrN00i+v9Q1bKGL4o/MeaGJYdr0f0EIAzO6YZRmQ2oEutG671 sWzGRiXbXW3Lg== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v3 2/8] um: Check for missing AVX and AVX-512 xstate bits Date: Wed, 12 Aug 2026 19:15:00 -0700 Message-ID: <20260813021506.55129-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If the CPU declares AVX or AVX-512 support, verify that all the corresponding bits are set in the XCR0 register. If any are missing, warn and don't set the corresponding X86_FEATURE_* flags. This eliminates the perceived need for UML-supporting AVX and AVX-512 optimized code in the kernel (that is, lib/raid/ currently) to start checking the XCR0 bits in addition to X86_FEATURE_AVX*. This aligns UML with the vast majority of userspace programs, which check the XCR0 bits before considering AVX and AVX-512 to be supported, as per the procedure documented in Intel's CPU manual. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/um/kernel/um_arch.c | 78 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 77 insertions(+), 1 deletion(-) diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c index 2141f5f1f5a2..aafbaef2ae82 100644 --- a/arch/um/kernel/um_arch.c +++ b/arch/um/kernel/um_arch.c @@ -264,12 +264,88 @@ unsigned long brk_start; =20 #define MIN_VMALLOC (32 * 1024 * 1024) =20 +static u64 __init read_xcr0(void) +{ + u32 a, b, c, d; + + asm volatile("cpuid" + : "=3Da"(a), "=3Db"(b), "=3Dc"(c), "=3Dd"(d) + : "a"(0), "c"(0)); + if (a >=3D 1) { /* max_leaf >=3D 1 */ + asm volatile("cpuid" + : "=3Da"(a), "=3Db"(b), "=3Dc"(c), "=3Dd"(d) + : "a"(1), "c"(0)); + if (c & (1 << 27)) { /* XSAVE enabled by OS */ + asm volatile("xgetbv" : "=3Dd"(d), "=3Da"(a) : "c"(0)); + return ((u64)d << 32) | a; + } + } + return 0; +} + +static void __init validate_and_set_cpu_cap(int cap, u64 xcr0) +{ + /* + * Check for missing xstate features right away, so that there's no + * perceived need for all optimized code in the kernel to do so. + */ + switch (cap) { + case X86_FEATURE_AVX: + case X86_FEATURE_AVX2: + case X86_FEATURE_AVX_VNNI: + case X86_FEATURE_FMA: + case X86_FEATURE_VAES: + case X86_FEATURE_VPCLMULQDQ: + if ((xcr0 & 0x7) !=3D 0x7) { + static bool warned; + + if (!warned) { + os_warn("Disabling AVX support due to missing xstate features\n"); + warned =3D true; + } + return; + } + break; + case X86_FEATURE_AVX512F: + case X86_FEATURE_AVX512BW: + case X86_FEATURE_AVX512CD: + case X86_FEATURE_AVX512DQ: + case X86_FEATURE_AVX512ER: + case X86_FEATURE_AVX512IFMA: + case X86_FEATURE_AVX512PF: + case X86_FEATURE_AVX512VBMI: + case X86_FEATURE_AVX512VL: + case X86_FEATURE_AVX512_4FMAPS: + case X86_FEATURE_AVX512_4VNNIW: + case X86_FEATURE_AVX512_BF16: + case X86_FEATURE_AVX512_BITALG: + case X86_FEATURE_AVX512_FP16: + case X86_FEATURE_AVX512_VBMI2: + case X86_FEATURE_AVX512_VNNI: + case X86_FEATURE_AVX512_VP2INTERSECT: + case X86_FEATURE_AVX512_VPOPCNTDQ: + if ((xcr0 & 0xe7) !=3D 0xe7) { + static bool warned; + + if (!warned) { + os_warn("Disabling AVX-512 support due to missing xstate features\n"); + warned =3D true; + } + return; + } + break; + } + set_cpu_cap(&boot_cpu_data, cap); +} + static void __init parse_host_cpu_flags(char *line) { + u64 xcr0 =3D read_xcr0(); int i; + for (i =3D 0; i < 32*NCAPINTS; i++) { if ((x86_cap_flags[i] !=3D NULL) && strstr(line, x86_cap_flags[i])) - set_cpu_cap(&boot_cpu_data, i); + validate_and_set_cpu_cap(i, xcr0); } } =20 --=20 2.55.0 From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 371FE2F4A05; Thu, 13 Aug 2026 02:20:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587621; cv=none; b=BjBTULSETfzC66v5KS2v8KGK12Tec5/mfcbZlFIdhSim+vwpN/+4xulx5wdOXDMOOweeB8YcUk7dsbtoD/EnIbKsDfL5Lis4EHM2v1P+lspTEv4Lx2ja3RA7FkZW8GU2hB/GOnPCR741f8aWRbiZyuaqtgRp0ooj27Fg+Nz6+14= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587621; c=relaxed/simple; bh=j71FQSXxVyY13IVMc20ZI/TX8wDr4Hvb0YpK4t8lc9s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=atW2U9x7XtqJtj70TxdguLynqFjXUQEqoZd8Gk52HXyGQWQtdT7RiaHo7Rl8mGjLaOhcuJLFeYaBkW+u+8e1u4u3Q81/vn0Ny8QDmzyJcws0pZz0w0fTMQw0VDeKy3VoUpCDluaO6hJaACwH7zCug5LcYJ9c0sa6CLaYJVsp+Dg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jbfF0Cr2; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jbfF0Cr2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A9B481F00AC4; Thu, 13 Aug 2026 02:20:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587619; bh=VKmmGWRx3Pfbo+nD8y5eQpPaN0sWZSIp/Ijjeyx4vLE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jbfF0Cr2ucRoUEyaURyJ+GLQEY4g65br8Xp7+8ORRcCV5v8kUOOM5cxSVV7q/eTiF IchirtVC2pYhgKjYzav90m35Pct7JvygB5aS0OH/tqjRuj7QXsHbKqPZCWyIJo1Ruv yd73eybjb9QIv5Kw5lddbk1zvauhRfeKEVe8RW8ex+R5l1UQA1r8Sc1zJDsYQVvrK9 cS4NuD78GQ3XpQiuIT3Sr7kvFcdb9Le4SqID1A9Zd9uAxKcAPoGPmFxSIjD8+7Irtv 1KihH0Glp5ND+U/YSL8iE1kRoErrvQu01fo15qqUbzByYVaciQ5cSHE8a+AALD7uKF 5MSyR3mPaDrtg== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v3 3/8] crypto: x86 - Stop using cpu_has_xfeatures() Date: Wed, 12 Aug 2026 19:15:01 -0700 Message-ID: <20260813021506.55129-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Checking both boot_cpu_has(X86_FEATURE_AVX*) and cpu_has_xfeatures() has never really been needed in practice, and it's never been universally done (e.g., lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML now explicitly clear the AVX and AVX-512 flags if their xfeatures are missing, which should remove any remaining doubts. Thus, remove all the calls to cpu_has_xfeatures(), as well as the related checks of boot_cpu_has(X86_FEATURE_OSXSAVE). In a few cases there was no corresponding boot_cpu_has(X86_FEATURE_AVX*) check, so add the missing ones. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/x86/crypto/aegis128-aesni-glue.c | 3 +-- arch/x86/crypto/aesni-intel_glue.c | 7 ++----- arch/x86/crypto/aria_aesni_avx2_glue.c | 11 +---------- arch/x86/crypto/aria_aesni_avx_glue.c | 11 +---------- arch/x86/crypto/aria_gfni_avx512_glue.c | 11 +---------- arch/x86/crypto/camellia_aesni_avx2_glue.c | 11 +---------- arch/x86/crypto/camellia_aesni_avx_glue.c | 11 +---------- arch/x86/crypto/cast5_avx_glue.c | 7 ++----- arch/x86/crypto/cast6_avx_glue.c | 7 ++----- arch/x86/crypto/serpent_avx2_glue.c | 9 +-------- arch/x86/crypto/serpent_avx_glue.c | 7 ++----- arch/x86/crypto/sm4_aesni_avx2_glue.c | 11 +---------- arch/x86/crypto/sm4_aesni_avx_glue.c | 11 +---------- arch/x86/crypto/twofish_avx_glue.c | 6 ++---- 14 files changed, 19 insertions(+), 104 deletions(-) diff --git a/arch/x86/crypto/aegis128-aesni-glue.c b/arch/x86/crypto/aegis1= 28-aesni-glue.c index f1adfba1a76e..09fc0b15b0e9 100644 --- a/arch/x86/crypto/aegis128-aesni-glue.c +++ b/arch/x86/crypto/aegis128-aesni-glue.c @@ -265,8 +265,7 @@ static struct aead_alg crypto_aegis128_aesni_alg =3D { static int __init crypto_aegis128_aesni_module_init(void) { if (!boot_cpu_has(X86_FEATURE_XMM4_1) || - !boot_cpu_has(X86_FEATURE_AES) || - !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL)) + !boot_cpu_has(X86_FEATURE_AES)) return -ENODEV; =20 return crypto_register_aead(&crypto_aegis128_aesni_alg); diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-int= el_glue.c index f522fff9231e..f6f899db7482 100644 --- a/arch/x86/crypto/aesni-intel_glue.c +++ b/arch/x86/crypto/aesni-intel_glue.c @@ -1548,8 +1548,7 @@ static int __init register_avx_algs(void) if (!boot_cpu_has(X86_FEATURE_AVX2) || !boot_cpu_has(X86_FEATURE_VAES) || !boot_cpu_has(X86_FEATURE_VPCLMULQDQ) || - !boot_cpu_has(X86_FEATURE_PCLMULQDQ) || - !cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + !boot_cpu_has(X86_FEATURE_PCLMULQDQ)) return 0; err =3D crypto_register_skciphers(skcipher_algs_vaes_avx2, ARRAY_SIZE(skcipher_algs_vaes_avx2)); @@ -1562,9 +1561,7 @@ static int __init register_avx_algs(void) =20 if (!boot_cpu_has(X86_FEATURE_AVX512BW) || !boot_cpu_has(X86_FEATURE_AVX512VL) || - !boot_cpu_has(X86_FEATURE_BMI2) || - !cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | - XFEATURE_MASK_AVX512, NULL)) + !boot_cpu_has(X86_FEATURE_BMI2)) return 0; =20 if (boot_cpu_has(X86_FEATURE_PREFER_YMM)) { diff --git a/arch/x86/crypto/aria_aesni_avx2_glue.c b/arch/x86/crypto/aria_= aesni_avx2_glue.c index 1487a49bfbac..371be2fb6469 100644 --- a/arch/x86/crypto/aria_aesni_avx2_glue.c +++ b/arch/x86/crypto/aria_aesni_avx2_glue.c @@ -195,22 +195,13 @@ static struct skcipher_alg aria_algs[] =3D { =20 static int __init aria_avx2_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX2 or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - if (boot_cpu_has(X86_FEATURE_GFNI)) { aria_ops.aria_encrypt_16way =3D aria_aesni_avx_gfni_encrypt_16way; aria_ops.aria_decrypt_16way =3D aria_aesni_avx_gfni_decrypt_16way; diff --git a/arch/x86/crypto/aria_aesni_avx_glue.c b/arch/x86/crypto/aria_a= esni_avx_glue.c index e4e3d78915a5..d23fc91c0ebd 100644 --- a/arch/x86/crypto/aria_aesni_avx_glue.c +++ b/arch/x86/crypto/aria_aesni_avx_glue.c @@ -182,21 +182,12 @@ static struct skcipher_alg aria_algs[] =3D { =20 static int __init aria_avx_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - if (boot_cpu_has(X86_FEATURE_GFNI)) { aria_ops.aria_encrypt_16way =3D aria_aesni_avx_gfni_encrypt_16way; aria_ops.aria_decrypt_16way =3D aria_aesni_avx_gfni_decrypt_16way; diff --git a/arch/x86/crypto/aria_gfni_avx512_glue.c b/arch/x86/crypto/aria= _gfni_avx512_glue.c index 363cbf4399cc..e05bbeb22d4a 100644 --- a/arch/x86/crypto/aria_gfni_avx512_glue.c +++ b/arch/x86/crypto/aria_gfni_avx512_glue.c @@ -196,24 +196,15 @@ static struct skcipher_alg aria_algs[] =3D { =20 static int __init aria_avx512_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || !boot_cpu_has(X86_FEATURE_AVX512F) || !boot_cpu_has(X86_FEATURE_AVX512VL) || - !boot_cpu_has(X86_FEATURE_GFNI) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_GFNI)) { pr_info("AVX512/GFNI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | - XFEATURE_MASK_AVX512, &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - aria_ops.aria_encrypt_16way =3D aria_aesni_avx_gfni_encrypt_16way; aria_ops.aria_decrypt_16way =3D aria_aesni_avx_gfni_decrypt_16way; aria_ops.aria_ctr_crypt_16way =3D aria_aesni_avx_gfni_ctr_crypt_16way; diff --git a/arch/x86/crypto/camellia_aesni_avx2_glue.c b/arch/x86/crypto/c= amellia_aesni_avx2_glue.c index 2d2f4e16537c..073fa3bb8388 100644 --- a/arch/x86/crypto/camellia_aesni_avx2_glue.c +++ b/arch/x86/crypto/camellia_aesni_avx2_glue.c @@ -97,22 +97,13 @@ static struct skcipher_alg camellia_algs[] =3D { =20 static int __init camellia_aesni_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX2 or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(camellia_algs, ARRAY_SIZE(camellia_algs)); } diff --git a/arch/x86/crypto/camellia_aesni_avx_glue.c b/arch/x86/crypto/ca= mellia_aesni_avx_glue.c index 5c321f255eb7..872e5e07220f 100644 --- a/arch/x86/crypto/camellia_aesni_avx_glue.c +++ b/arch/x86/crypto/camellia_aesni_avx_glue.c @@ -98,21 +98,12 @@ static struct skcipher_alg camellia_algs[] =3D { =20 static int __init camellia_aesni_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(camellia_algs, ARRAY_SIZE(camellia_algs)); } diff --git a/arch/x86/crypto/cast5_avx_glue.c b/arch/x86/crypto/cast5_avx_g= lue.c index 3aca04d43b34..5de35e863370 100644 --- a/arch/x86/crypto/cast5_avx_glue.c +++ b/arch/x86/crypto/cast5_avx_glue.c @@ -92,11 +92,8 @@ static struct skcipher_alg cast5_algs[] =3D { =20 static int __init cast5_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 diff --git a/arch/x86/crypto/cast6_avx_glue.c b/arch/x86/crypto/cast6_avx_g= lue.c index c4dd28c30303..3d7ea48007bc 100644 --- a/arch/x86/crypto/cast6_avx_glue.c +++ b/arch/x86/crypto/cast6_avx_glue.c @@ -92,11 +92,8 @@ static struct skcipher_alg cast6_algs[] =3D { =20 static int __init cast6_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 diff --git a/arch/x86/crypto/serpent_avx2_glue.c b/arch/x86/crypto/serpent_= avx2_glue.c index f5f2121b7956..72a9e2b306d6 100644 --- a/arch/x86/crypto/serpent_avx2_glue.c +++ b/arch/x86/crypto/serpent_avx2_glue.c @@ -93,17 +93,10 @@ static struct skcipher_alg serpent_algs[] =3D { =20 static int __init serpent_avx2_init(void) { - const char *feature_name; - - if (!boot_cpu_has(X86_FEATURE_AVX2) || !boot_cpu_has(X86_FEATURE_OSXSAVE)= ) { + if (!boot_cpu_has(X86_FEATURE_AVX2)) { pr_info("AVX2 instructions are not detected.\n"); return -ENODEV; } - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } =20 return crypto_register_skciphers(serpent_algs, ARRAY_SIZE(serpent_algs)); diff --git a/arch/x86/crypto/serpent_avx_glue.c b/arch/x86/crypto/serpent_a= vx_glue.c index 9c8b3a335d5c..42c4e1569674 100644 --- a/arch/x86/crypto/serpent_avx_glue.c +++ b/arch/x86/crypto/serpent_avx_glue.c @@ -100,11 +100,8 @@ static struct skcipher_alg serpent_algs[] =3D { =20 static int __init serpent_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 diff --git a/arch/x86/crypto/sm4_aesni_avx2_glue.c b/arch/x86/crypto/sm4_ae= sni_avx2_glue.c index fec0ab7a63dd..eef73894e777 100644 --- a/arch/x86/crypto/sm4_aesni_avx2_glue.c +++ b/arch/x86/crypto/sm4_aesni_avx2_glue.c @@ -98,22 +98,13 @@ static struct skcipher_alg sm4_aesni_avx2_skciphers[] = =3D { =20 static int __init sm4_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX2 or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(sm4_aesni_avx2_skciphers, ARRAY_SIZE(sm4_aesni_avx2_skciphers)); } diff --git a/arch/x86/crypto/sm4_aesni_avx_glue.c b/arch/x86/crypto/sm4_aes= ni_avx_glue.c index 88caf418a06f..ed383da5ff46 100644 --- a/arch/x86/crypto/sm4_aesni_avx_glue.c +++ b/arch/x86/crypto/sm4_aesni_avx_glue.c @@ -314,21 +314,12 @@ static struct skcipher_alg sm4_aesni_avx_skciphers[] = =3D { =20 static int __init sm4_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(sm4_aesni_avx_skciphers, ARRAY_SIZE(sm4_aesni_avx_skciphers)); } diff --git a/arch/x86/crypto/twofish_avx_glue.c b/arch/x86/crypto/twofish_a= vx_glue.c index 9e20db013750..985bc54a2340 100644 --- a/arch/x86/crypto/twofish_avx_glue.c +++ b/arch/x86/crypto/twofish_avx_glue.c @@ -102,10 +102,8 @@ static struct skcipher_alg twofish_algs[] =3D { =20 static int __init twofish_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, &feature_na= me)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 --=20 2.55.0 From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D5B433438F; Thu, 13 Aug 2026 02:20:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587621; cv=none; b=RsM1S9/pApJHHEbkBo+6wQQVxQIa8k5tgWZ+DCUXIowI3uWHOqQ05O3YKl6mHhnfT2m1nreoxXhvBdJfik2nEHTEkLgYElJxbqCJYO6Y8qjGYoRXhsvpAGAxYTwAS/jgmuUE1ituyPSWRFTOdiDuzB9hFZjlyudlUAjYz9rb4bM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587621; c=relaxed/simple; bh=e0/5EyKvRR38AkedRTxpFtVqo23cOjcvx5sQexInGDU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IpHUW9BbrKDqZpjg6AAfH1ZVxU5F9MqBYXs0xVBnstftspVHx33hwlk8gSdKkmYkQj7M58NeypCgz5LzfNCqva2gyMquxJEgUwKs8dJMvChuMFoh4Y3wjwdN5c13dIn7AnDX8FT/zgUuXQ2T5hNW92IbrJbpMBrLtR3I69Wiy8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SblhIKAv; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SblhIKAv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10BDE1F00ACA; Thu, 13 Aug 2026 02:20:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587620; bh=DdqWq0+mPRqIC9aeO4PAgts2wuJAHghXz/rr2+6DDJI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=SblhIKAvDPFbxThcSToULUlkkW9tgmhISwNyPnS1saeizVRyKp2WK7nQZXcr2gAMC UnvJmir5nvhGBAxmTHqUmwu3SPBg5q0hfVzuKkDKJjLtijoC9kF/4Z2q6WHtuCxnFL kuunPXqJUa/vuruTt/oZUpnzS9WUP4yKR7L9yRzdFZBxSQf+7MwlQMmgEXdHiEXEd9 908xob1KJ27beVI4yjTO3fh6G3WvJv969m3UovxCG1Wtq0hXPIDO3RXZUHgzh5E5K2 zu2WCDsN7A5l42OVntbgho467McZ3gwlss1IkpZi9yeNpDE1lIWgddsVhec6tA4Zsw LcR3Ax2Dq3hzg== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v3 4/8] lib/crypto: x86: Stop using cpu_has_xfeatures() Date: Wed, 12 Aug 2026 19:15:02 -0700 Message-ID: <20260813021506.55129-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Checking both boot_cpu_has() and cpu_has_xfeatures() has never really been needed in practice, and it's never been universally done (e.g., lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML now explicitly clear the AVX and AVX-512 flags if their xfeatures are missing, which should remove any remaining doubts. Thus, remove all the calls to cpu_has_xfeatures(). Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/crypto/x86/blake2s.h | 4 +--- lib/crypto/x86/chacha.h | 3 +-- lib/crypto/x86/nh.h | 4 +--- lib/crypto/x86/poly1305.h | 7 ++----- lib/crypto/x86/sha1.h | 4 +--- lib/crypto/x86/sha256.h | 4 +--- lib/crypto/x86/sha512.h | 3 +-- lib/crypto/x86/sm3.h | 3 +-- 8 files changed, 9 insertions(+), 23 deletions(-) diff --git a/lib/crypto/x86/blake2s.h b/lib/crypto/x86/blake2s.h index f8eed6cb042e..0f7c51f055c8 100644 --- a/lib/crypto/x86/blake2s.h +++ b/lib/crypto/x86/blake2s.h @@ -55,8 +55,6 @@ static void blake2s_mod_init_arch(void) if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX512F) && - boot_cpu_has(X86_FEATURE_AVX512VL) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | - XFEATURE_MASK_AVX512, NULL)) + boot_cpu_has(X86_FEATURE_AVX512VL)) static_branch_enable(&blake2s_use_avx512); } diff --git a/lib/crypto/x86/chacha.h b/lib/crypto/x86/chacha.h index 10cf8f1c569d..c79562aac56b 100644 --- a/lib/crypto/x86/chacha.h +++ b/lib/crypto/x86/chacha.h @@ -165,8 +165,7 @@ static void chacha_mod_init_arch(void) static_branch_enable(&chacha_use_simd); =20 if (boot_cpu_has(X86_FEATURE_AVX) && - boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) { + boot_cpu_has(X86_FEATURE_AVX2)) { static_branch_enable(&chacha_use_avx2); =20 if (boot_cpu_has(X86_FEATURE_AVX512VL) && diff --git a/lib/crypto/x86/nh.h b/lib/crypto/x86/nh.h index 83361c2e9783..342636dcb750 100644 --- a/lib/crypto/x86/nh.h +++ b/lib/crypto/x86/nh.h @@ -37,9 +37,7 @@ static void nh_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_XMM2)) { static_branch_enable(&have_sse2); - if (boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - NULL)) + if (boot_cpu_has(X86_FEATURE_AVX2)) static_branch_enable(&have_avx2); } } diff --git a/lib/crypto/x86/poly1305.h b/lib/crypto/x86/poly1305.h index ee92e3740a78..b061b9926fa5 100644 --- a/lib/crypto/x86/poly1305.h +++ b/lib/crypto/x86/poly1305.h @@ -143,15 +143,12 @@ static void poly1305_emit(const struct poly1305_state= *ctx, #define poly1305_mod_init_arch poly1305_mod_init_arch static void poly1305_mod_init_arch(void) { - if (boot_cpu_has(X86_FEATURE_AVX) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + if (boot_cpu_has(X86_FEATURE_AVX)) static_branch_enable(&poly1305_use_avx); - if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2)) static_branch_enable(&poly1305_use_avx2); if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX512F) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | XFEATURE_MA= SK_AVX512, NULL) && /* Skylake downclocks unacceptably much when using zmm, but later gen= erations are fast. */ boot_cpu_data.x86_vfm !=3D INTEL_SKYLAKE_X) static_branch_enable(&poly1305_use_avx512); diff --git a/lib/crypto/x86/sha1.h b/lib/crypto/x86/sha1.h index c48a0131fd12..6aff433466e7 100644 --- a/lib/crypto/x86/sha1.h +++ b/lib/crypto/x86/sha1.h @@ -59,9 +59,7 @@ static void sha1_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_SHA_NI)) { static_call_update(sha1_blocks_x86, sha1_blocks_ni); - } else if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - NULL) && - boot_cpu_has(X86_FEATURE_AVX)) { + } else if (boot_cpu_has(X86_FEATURE_AVX)) { if (boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_BMI1) && boot_cpu_has(X86_FEATURE_BMI2)) diff --git a/lib/crypto/x86/sha256.h b/lib/crypto/x86/sha256.h index 0ee69d8e39fe..e98ffdaf4b14 100644 --- a/lib/crypto/x86/sha256.h +++ b/lib/crypto/x86/sha256.h @@ -104,9 +104,7 @@ static void sha256_mod_init_arch(void) boot_cpu_has(X86_FEATURE_PHE_EN) && boot_cpu_data.x86 >=3D 0x07) { static_call_update(sha256_blocks_x86, sha256_blocks_phe); - } else if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - NULL) && - boot_cpu_has(X86_FEATURE_AVX)) { + } else if (boot_cpu_has(X86_FEATURE_AVX)) { if (boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_BMI2)) static_call_update(sha256_blocks_x86, diff --git a/lib/crypto/x86/sha512.h b/lib/crypto/x86/sha512.h index 0213c70cedd0..4e177b4606bd 100644 --- a/lib/crypto/x86/sha512.h +++ b/lib/crypto/x86/sha512.h @@ -37,8 +37,7 @@ static void sha512_blocks(struct sha512_block_state *stat= e, #define sha512_mod_init_arch sha512_mod_init_arch static void sha512_mod_init_arch(void) { - if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL) && - boot_cpu_has(X86_FEATURE_AVX)) { + if (boot_cpu_has(X86_FEATURE_AVX)) { if (boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_BMI2)) static_call_update(sha512_blocks_x86, diff --git a/lib/crypto/x86/sm3.h b/lib/crypto/x86/sm3.h index 3834780f2f6a..e06d4a22e4fa 100644 --- a/lib/crypto/x86/sm3.h +++ b/lib/crypto/x86/sm3.h @@ -33,7 +33,6 @@ static void sm3_blocks(struct sm3_block_state *state, #define sm3_mod_init_arch sm3_mod_init_arch static void sm3_mod_init_arch(void) { - if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_BMI2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_BMI2)) static_call_update(sm3_blocks_x86, sm3_blocks_avx); } --=20 2.55.0 From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCDED33B96B; Thu, 13 Aug 2026 02:20:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587621; cv=none; b=kCrW2KnrXhg2TkV0PrOSL4e63Mqaj1PxX/WVBtmRUFjvRmTM8S6RZXNKgA7QqYxF+oNE6QcyimoZp2XOI3jLEsbNu1UprJWY3IiT9By8H6HgQH9NbZ5fGSnriuia/M1uQDE4Zzp9qJMu2ckPEsRD/95x88EB3+Bdl5flX5nczQg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587621; c=relaxed/simple; bh=8EtIjUyWp+Owf958VUqMsdagPce6VSl8WYVhBIP0kNU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oDkT0Y8E2kWcclS8QLM1YHVjBeBN8S9lJ0N064Z4cW1ft089b/QI1KIN5aN60jvO7BUKYDdWl0UpzkdL3WZc1nhaVyxNg1eOHLg9iS9rUpYPjDyt9zQVzEoGwI60zaIP+wZ4rrVJpk3CZljzhbU0gv3ht7Q04839houEDluQF2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IgiWjcmV; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IgiWjcmV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B7461F00A3A; Thu, 13 Aug 2026 02:20:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587620; bh=r4lrJRkT6oIO9uz4byVKnhiX3ckXssiVruBvq7U+FfI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=IgiWjcmVhU1gKXdR+1IvaeAgk+3vKhkKAgCls+SdvY1Tl+fuzFVGuERmu/8/5/3HY bHrmDIWanBHy3f2c8rMsmLqWeUZwgU0coN5CVoWDwvvqvXW7z30hQIwWK12f0DIF2u HhxzF4M1x4q4dhxl2cbfE4LH4QOgE0XDwWuT/CPangnuSbxr6G6YvYobfLgA0lt2zb Xd8uulN7c2B868pVEU2hPJMnJyyiHScfLtFb79BjSIxb8ufNfzCuk/TqvjG/2nCDhZ kD0k8awSpCP+jC6+fvdcUzbeOeh+fz6h93yD1A01DFetYFFPf5s3ChMqEESbT6bpGT s6hav4GzWX3KA== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v3 5/8] lib/crc: x86: Stop using cpu_has_xfeatures() Date: Wed, 12 Aug 2026 19:15:03 -0700 Message-ID: <20260813021506.55129-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Checking both boot_cpu_has() and cpu_has_xfeatures() has never really been needed in practice, and it's never been universally done (e.g., lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML now explicitly clear the AVX and AVX-512 flags if their xfeatures are missing, which should remove any remaining doubts. Thus, remove all the calls to cpu_has_xfeatures(). Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/crc/x86/crc-pclmul-template.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/lib/crc/x86/crc-pclmul-template.h b/lib/crc/x86/crc-pclmul-tem= plate.h index 02744831c6fa..893119bb7c07 100644 --- a/lib/crc/x86/crc-pclmul-template.h +++ b/lib/crc/x86/crc-pclmul-template.h @@ -27,16 +27,14 @@ DEFINE_STATIC_CALL(prefix##_pclmul, prefix##_pclmul_sse) static inline bool have_vpclmul(void) { return boot_cpu_has(X86_FEATURE_VPCLMULQDQ) && - boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_YMM, NULL); + boot_cpu_has(X86_FEATURE_AVX2); } =20 static inline bool have_avx512(void) { return boot_cpu_has(X86_FEATURE_AVX512BW) && boot_cpu_has(X86_FEATURE_AVX512VL) && - !boot_cpu_has(X86_FEATURE_PREFER_YMM) && - cpu_has_xfeatures(XFEATURE_MASK_AVX512, NULL); + !boot_cpu_has(X86_FEATURE_PREFER_YMM); } =20 /* --=20 2.55.0 From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42571341062; Thu, 13 Aug 2026 02:20:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587622; cv=none; b=cbUCYVSF+GcCGIeH58fgqYH6rYmMZ7i/swNS6u1D6tDbrOqnK8gZKUZxA3dYTwp/7jNTCsTjLxzmjgfWdV7+RtJZ9Ghz2B1i9cqlVSuP025fbGFhv7Ck7JKqBCxxrc7Kblu6fUq+mZqM8GNj9uXaci8X5wCItogehAZ+i9W7Occ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587622; c=relaxed/simple; bh=mF5Si+fJ2SqCKuU/60v4aWm6TNcVfLmMHZQC50LpG1A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KijNUpIBADmMhx161+jYFJgLsfxFofe4iweEEbH4K+Z2WgOhjC7qg+aCeBUxTmLhT3KAUYCkao2APA87He9jsxwnlSqhL2AymAPuIVOxAhERo36FYgJ4BEyXc8x5UmevPp2C0tJSBIYVGpIPBeOuc/RHnNFicKPQ1/5SJUYxZkY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=C4Sg8aE0; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="C4Sg8aE0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C653A1F00A3F; Thu, 13 Aug 2026 02:20:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587621; bh=Q2A0VpZ3jEW63u/fB0uqkGhkDFeSh86cNwd3FvE985o=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=C4Sg8aE0DUbh5ju4dG+p+6BhIvIC/yloRPIln0CHHzIDIf2bbAcvc/rN9mK785z0Z ET3+D3+v3rbjhwq3NNJsVpzQjqn0w/bxkuovj2jvDKCNrLrSfn5Win+rMnkAESYmva SL2JZpIGfPsZaGHUCbLrT8Qv3Mdur8U3On5EqrjwyCsMNGjKYS+jVgs/QKE0fxALEO EvKT7oO4LGWRao+6g2P5sFAK0OQF0uUI4kOa0vJtjjsBDm8NhDxD3ZKHU8eI5VonW5 1NB5Gj562PVObvKwBsqgEqRN4/lY9feqf5rq5rkYOLdpZLbTBpAup4lnHaNYhPAKwV 9wAlWaeVaQoSA== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v3 6/8] x86/fpu: Remove cpu_has_xfeatures() Date: Wed, 12 Aug 2026 19:15:04 -0700 Message-ID: <20260813021506.55129-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The only remaining caller of cpu_has_xfeatures() is print_xstate_features(), which uses it only to check and get the name of a single feature. Remove it and just inline the needed code into print_xstate_features(). This also makes the "unknown xstate feature" entry at index XFEATURE_MAX of xfeature_names[] unnecessary, so remove that too. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/x86/include/asm/fpu/api.h | 9 ------- arch/x86/kernel/fpu/xstate.c | 44 +++------------------------------- 2 files changed, 3 insertions(+), 50 deletions(-) diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index 90c63fe19c0f..cfed8b24d64f 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -99,15 +99,6 @@ static inline void fpregs_assert_state_consistent(void) = { } */ extern void switch_fpu_return(void); =20 -/* - * Query the presence of one or more xfeatures. Works on any legacy CPU as= well. - * - * If 'feature_name' is set then put a human-readable description of - * the feature there as well - this can be used to print error (or success) - * messages. - */ -extern int cpu_has_xfeatures(u64 xfeatures_mask, const char **feature_name= ); - /* Trap handling */ extern int fpu__exception_code(struct fpu *fpu, int trap_nr); extern void fpu_sync_fpstate(struct fpu *fpu); diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 97cfd4fb6cc0..0b17452d1788 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -66,8 +66,8 @@ static const char *xfeature_names[] =3D "AMX Tile config", "AMX Tile data", "APX registers", - "unknown xstate feature", }; +static_assert(ARRAY_SIZE(xfeature_names) =3D=3D XFEATURE_MAX); =20 static unsigned short xsave_cpuid_features[] __initdata =3D { [XFEATURE_FP] =3D X86_FEATURE_FPU, @@ -122,44 +122,6 @@ static inline unsigned int next_xfeature_order(unsigne= d int i, u64 mask) #define XSTATE_FLAG_SUPERVISOR BIT(0) #define XSTATE_FLAG_ALIGNED64 BIT(1) =20 -/* - * Return whether the system supports a given xfeature. - * - * Also return the name of the (most advanced) feature that the caller req= uested: - */ -int cpu_has_xfeatures(u64 xfeatures_needed, const char **feature_name) -{ - u64 xfeatures_missing =3D xfeatures_needed & ~fpu_kernel_cfg.max_features; - - if (unlikely(feature_name)) { - long xfeature_idx, max_idx; - u64 xfeatures_print; - /* - * So we use FLS here to be able to print the most advanced - * feature that was requested but is missing. So if a driver - * asks about "XFEATURE_MASK_SSE | XFEATURE_MASK_YMM" we'll print the - * missing AVX feature - this is the most informative message - * to users: - */ - if (xfeatures_missing) - xfeatures_print =3D xfeatures_missing; - else - xfeatures_print =3D xfeatures_needed; - - xfeature_idx =3D fls64(xfeatures_print)-1; - max_idx =3D ARRAY_SIZE(xfeature_names)-1; - xfeature_idx =3D min(xfeature_idx, max_idx); - - *feature_name =3D xfeature_names[xfeature_idx]; - } - - if (xfeatures_missing) - return 0; - - return 1; -} -EXPORT_SYMBOL_GPL(cpu_has_xfeatures); - static bool xfeature_is_aligned64(int xfeature_nr) { return xstate_flags[xfeature_nr] & XSTATE_FLAG_ALIGNED64; @@ -302,9 +264,9 @@ static void __init print_xstate_features(void) =20 for (i =3D 0; i < XFEATURE_MAX; i++) { u64 mask =3D BIT_ULL(i); - const char *name; + const char *name =3D xfeature_names[i]; =20 - if (cpu_has_xfeatures(mask, &name)) + if (fpu_kernel_cfg.max_features & mask) pr_info("x86/fpu: Supporting XSAVE feature 0x%03Lx: '%s'\n", mask, name= ); } } --=20 2.55.0 From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A413343D72; Thu, 13 Aug 2026 02:20:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587622; cv=none; b=WeZ5zdFHAQX2Y7I92+vNryNsYK3QbESDtinr7sjp1s4pzznbnbTU5dvEU7Gyyodi7j353P6eUfEAq65Q2NlBEuMKpG8nbXL2e5XP87peym4T0z1j9vuzMiMTzu8vhWR0KNLzL9Hqa5Klj529enxpcCmgDd5x+eSorlj+BI4wYfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587622; c=relaxed/simple; bh=bURSDeOHD7BeXhOUO95qSmh0YND+CXXzB5NqaBliqI4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ABeLUD6mN5JXtQiTvL65G4wLLWcb2nX40VHrjjSF8oHT7Qx0lFYrY2W+X69TO1RC2IRYwdh9DhaKIpp+vFt7iDdLUEQ9vrfQ1baT4vFn9xBnELHbfwDg7BJRkBja2Myx3JD/NHsBn/sbpZTCdLMg9texR/JDl0RnTNCbsadCT20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KtDbgGp6; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KtDbgGp6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D3151F00A3E; Thu, 13 Aug 2026 02:20:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587621; bh=CGFmBktDb18Ou8jdzee9BRhN2jpwroPuYWGDnSphFl0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=KtDbgGp6IL8GZ8uJj1u1T7P4Uc1PgQXdz1qtwXGQ5moJuYQoOjFBNKqxKWrhENLVs KMg+sJomVyF5ueu4LHaO2EjLI4lylQOCQSQi2Y6+yBU9MVv3i+hYymm6z214Fh+4gv 3tdkqQ9/QekaowLkneojwQCIIdAi+qnhMZy8l274Z6oyVxomgJ58j+dmWcaiSj4wiH Yc9Sz3gpmA0BojOuppbwB6EvwDHAxpN29jtbq2lRDRzQHjeCywZHdt0v3P8lqIFe6P q2VIxg1ZH4hMmzRC4l7gUtom3XA6m0+nY9cnr+svy2rtGr4NQmhdBP7AZaF/uOvFOQ 7DxF3aiAGO3pQ== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v3 7/8] xor: Remove redundant X86_FEATURE_OSXSAVE check Date: Wed, 12 Aug 2026 19:15:05 -0700 Message-ID: <20260813021506.55129-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X86_FEATURE_AVX implies X86_FEATURE_OSXSAVE already. Reviewed-by: Christoph Hellwig Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/raid/xor/x86/xor_arch.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/raid/xor/x86/xor_arch.h b/lib/raid/xor/x86/xor_arch.h index 99fe85a213c6..991abe3f4bbd 100644 --- a/lib/raid/xor/x86/xor_arch.h +++ b/lib/raid/xor/x86/xor_arch.h @@ -18,8 +18,7 @@ extern struct xor_block_template xor_block_avx; */ static __always_inline void __init arch_xor_init(void) { - if (boot_cpu_has(X86_FEATURE_AVX) && - boot_cpu_has(X86_FEATURE_OSXSAVE)) { + if (boot_cpu_has(X86_FEATURE_AVX)) { xor_force(&xor_block_avx); } else if (IS_ENABLED(CONFIG_X86_64) || boot_cpu_has(X86_FEATURE_XMM)) { xor_register(&xor_block_sse); --=20 2.55.0 From nobody Tue Sep 29 04:09:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1F28344DB1; Thu, 13 Aug 2026 02:20:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587623; cv=none; b=c1DkdoLc1fPFuaTpO6PQr2OcZmweilCw1DZzYrDIzbpw3tnvNfl16VjdjXUM8TDC6G+rFoWOeI8f1O4M1Cr8W+SLT/V8RcG7WIfx3t38z2REKD//6R3WIfzlUAsSOpVb4N6Sp7K9/yEXjoRdkSCqdWbdRblfwl4TnRDpvqYvP5E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786587623; c=relaxed/simple; bh=/hH1Oq7lFtBDDRA+Jt8m9GlWTDXAmxkyI3neWGcJ520=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ky73jnfTqX2ia8Rc+KFsTAzgvwDRH/UHGCmMiwigz2Zv5ruC7Y9h6rcz2J2x40e/PWr2ouJWeAxxq3Fv0qXCiewZXyqgSmmJGN0YxkXshPLdJlY9Glr6jMYNF2VNqPZ4RiJWFd54PRXTsU1WDb4DqDToWipAGb6MrGXzQvTCkss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=At/Yj275; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="At/Yj275" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 883FA1F00A3D; Thu, 13 Aug 2026 02:20:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786587621; bh=jI/Z+ELMGffsK7yD9bEbeMnNPGiLXDU1v0SJC2WQaC0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=At/Yj275NZvuxcN/im15cZY/CJQbAe2+9gFIKF1slqeZYU60EytaQVsCWtwoULThJ 5qurZO+RGa4T9efqwriH3sMI+d5jZ6lOjQvAUUUMWeIdDtBQbcsvSsH5r1BZTs+w2f ZltJXeA7x+gm3L7tQyRbcwemitI0gijuDXaKFV2qeqP2qG1Zh3oW/xrkh46ljldlMn QeE1Ln9F4UXusykdT7ZwBS/WRAjVJO7V8HKr1nclU6okKvn5NKaOup7S8E388xDFgY VvAgGvl9e5zxNtgK3+IJ9pQi0E/nuliLTpsL7w2yLcsOEHVLdgZkLC10tc7vUi5f9E yAPhC6Td973nw== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers , David Laight Subject: [PATCH v3 8/8] xor: Add AVX-512 optimized xor_gen() Date: Wed, 12 Aug 2026 19:15:06 -0700 Message-ID: <20260813021506.55129-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260813021506.55129-1-ebiggers@kernel.org> References: <20260813021506.55129-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an implementation of xor_gen() using AVX-512. It uses 512-bit vectors, i.e. ZMM registers. It also uses the vpternlogq instruction to do three-input XORs when applicable. It's enabled on x86_64 CPUs that have AVX512F && !PREFER_YMM. In practice that means: - AMD Zen 4 and later (client and server) - Intel Sapphire Rapids and later (server) - Intel Rocket Lake (client) - Intel Nova Lake and later (client) The !PREFER_YMM condition excludes the older AVX-512 implementations in Intel Skylake Server and Intel Ice Lake. They could run this code, but they're known to have overly-eager downclocking when ZMM registers are used. This is the same policy that the crypto and CRC code uses. Benchmark on AMD Ryzen 9 9950X (Zen 5): src_cnt avx avx512 Improvement =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1 56353 MB/s 75388 MB/s 33% 2 54274 MB/s 68409 MB/s 26% 3 44649 MB/s 64042 MB/s 43% 4 41315 MB/s 55002 MB/s 33% Reviewed-by: David Laight Reviewed-by: Christoph Hellwig Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/raid/xor/Makefile | 2 +- lib/raid/xor/x86/xor-avx512.c | 121 ++++++++++++++++++++++++++++++++++ lib/raid/xor/x86/xor_arch.h | 30 ++++++--- 3 files changed, 142 insertions(+), 11 deletions(-) create mode 100644 lib/raid/xor/x86/xor-avx512.c diff --git a/lib/raid/xor/Makefile b/lib/raid/xor/Makefile index e8ecec3c09f9..4a0e5c6d8298 100644 --- a/lib/raid/xor/Makefile +++ b/lib/raid/xor/Makefile @@ -29,7 +29,7 @@ xor-$(CONFIG_SPARC32) +=3D sparc/xor-sparc32.o xor-$(CONFIG_SPARC64) +=3D sparc/xor-sparc64.o sparc/xor-sparc64-glue.o xor-$(CONFIG_S390) +=3D s390/xor.o xor-$(CONFIG_X86_32) +=3D x86/xor-avx.o x86/xor-sse.o x86/xor-mmx.o -xor-$(CONFIG_X86_64) +=3D x86/xor-avx.o x86/xor-sse.o +xor-$(CONFIG_X86_64) +=3D x86/xor-avx.o x86/xor-sse.o x86/xor-avx512.o obj-y +=3D tests/ =20 CFLAGS_xor-neon.o +=3D $(CC_FLAGS_FPU) -I$(src)/$(SRCARCH) diff --git a/lib/raid/xor/x86/xor-avx512.c b/lib/raid/xor/x86/xor-avx512.c new file mode 100644 index 000000000000..17f57900d827 --- /dev/null +++ b/lib/raid/xor/x86/xor-avx512.c @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * AVX-512 optimized implementation of xor_gen() + * + * Copyright 2026 Google LLC + */ + +#include +#include +#include "xor_impl.h" +#include "xor_arch.h" + +/* + * Implementation notes: + * + * Unrolling by the number of buffers (2-5) is very important. + * + * Unrolling by length is less important, especially when using register-i= ndexed + * addressing with negative indices from the end of the buffers. That app= roach + * results in just two loop control instructions being needed per iteratio= n, + * regardless of the number of buffers. + * + * In fact, benchmarks showed that the 2 and 3 buffer cases require only 2x + * unrolling by length, while the 4 and 5 buffer cases don't require any + * unrolling by length. Benchmarks also showed that the register-indexed + * addressing isn't a bottleneck either; i.e., we can't do any better by + * incrementing the pointers as we go along, even with more unrolling. + */ + +static void xor_avx512_2(long bytes, u8 *p1, const u8 *p2) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 64(%1,%0), %%zmm1\n" + "vpxorq (%2,%0), %%zmm0, %%zmm0\n" + "vpxorq 64(%2,%0), %%zmm1, %%zmm1\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "vmovdqa64 %%zmm1, 64(%1,%0)\n" + "add $128, %0\n" + "jnz 1b\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes) + : "memory", "cc"); +} + +static void xor_avx512_3(long bytes, u8 *p1, const u8 *p2, const u8 *p3) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 64(%1,%0), %%zmm1\n" + "vmovdqa64 (%2,%0), %%zmm2\n" + "vmovdqa64 64(%2,%0), %%zmm3\n" + "vpternlogq $0x96, (%3,%0), %%zmm2, %%zmm0\n" + "vpternlogq $0x96, 64(%3,%0), %%zmm3, %%zmm1\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "vmovdqa64 %%zmm1, 64(%1,%0)\n" + "add $128, %0\n" + "jnz 1b\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes) + : "memory", "cc"); +} + +static void xor_avx512_4(long bytes, u8 *p1, const u8 *p2, const u8 *p3, + const u8 *p4) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 (%2,%0), %%zmm1\n" + "vpxorq (%3,%0), %%zmm0, %%zmm0\n" + "vpternlogq $0x96, (%4,%0), %%zmm1, %%zmm0\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "add $64, %0\n" + "jnz 1b\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes), + "r"(p4 + bytes) + : "memory", "cc"); +} + +static void xor_avx512_5(long bytes, u8 *p1, const u8 *p2, const u8 *p3, + const u8 *p4, const u8 *p5) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 (%2,%0), %%zmm1\n" + "vpternlogq $0x96, (%3,%0), %%zmm1, %%zmm0\n" + "vmovdqa64 (%4,%0), %%zmm1\n" + "vpternlogq $0x96, (%5,%0), %%zmm1, %%zmm0\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "add $64, %0\n" + "jnz 1b\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes), + "r"(p4 + bytes), "r"(p5 + bytes) + : "memory", "cc"); +} + +DO_XOR_BLOCKS(avx512_inner, xor_avx512_2, xor_avx512_3, xor_avx512_4, + xor_avx512_5); + +/* + * Preconditions: bytes is a nonzero multiple of 512, and all buffers are + * 64-byte aligned. + */ +static void xor_gen_avx512(void *dest, void **srcs, unsigned int src_cnt, + unsigned int bytes) +{ + kernel_fpu_begin(); + xor_gen_avx512_inner(dest, srcs, src_cnt, bytes); + kernel_fpu_end(); +} + +struct xor_block_template xor_block_avx512 =3D { + .name =3D "avx512", + .xor_gen =3D xor_gen_avx512, +}; diff --git a/lib/raid/xor/x86/xor_arch.h b/lib/raid/xor/x86/xor_arch.h index 991abe3f4bbd..ed5921d2e2aa 100644 --- a/lib/raid/xor/x86/xor_arch.h +++ b/lib/raid/xor/x86/xor_arch.h @@ -6,21 +6,31 @@ extern struct xor_block_template xor_block_p5_mmx; extern struct xor_block_template xor_block_sse; extern struct xor_block_template xor_block_sse_pf64; extern struct xor_block_template xor_block_avx; +extern struct xor_block_template xor_block_avx512; =20 -/* - * When SSE is available, use it as it can write around L2. We may also b= e able - * to load into the L1 only depending on how the cpu deals with a load to = a line - * that is being prefetched. - * - * When AVX2 is available, force using it as it is better by all measures. - * - * 32-bit without MMX can fall back to the generic routines. - */ static __always_inline void __init arch_xor_init(void) { - if (boot_cpu_has(X86_FEATURE_AVX)) { + if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_AVX512F) && + !boot_cpu_has(X86_FEATURE_PREFER_YMM)) { + /* + * Use the AVX-512 code on CPUs that support AVX-512 without + * overly-eager downclocking. On such CPUs the AVX-512 code + * should always work at least as well as the AVX code, so + * runtime selection is unnecessary. + * + * The AVX-512 code can work on X86_32. However, due to lack of + * use case for that, for now it's built only for X86_64. + */ + xor_force(&xor_block_avx512); + } else if (boot_cpu_has(X86_FEATURE_AVX)) { + /* AVX will be the best; no need to try others. */ xor_force(&xor_block_avx); } else if (IS_ENABLED(CONFIG_X86_64) || boot_cpu_has(X86_FEATURE_XMM)) { + /* + * When SSE is available, use it as it can write around L2. We + * may also be able to load into the L1 only depending on how + * the cpu deals with a load to a line that is being prefetched. + */ xor_register(&xor_block_sse); xor_register(&xor_block_sse_pf64); } else if (boot_cpu_has(X86_FEATURE_MMX)) { --=20 2.55.0