From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7CED427FB2; Sat, 15 Aug 2026 18:12:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817554; cv=none; b=O6wW/bzLsnvIINfc32nCrSedmm0ZOmkKkCbkkm8C86y1nO/Ajo1V77ErkssRMKugumb1ZXyZJu70BxRHPLi+Okrh9zMGsPm2bDmrLWv3p3VDhDxQuFYHyJ5xHmZXIilyhNVH9lHn12CyYe/Y/MuRkkgI1MqdV72ACOIkb9fWRIk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817554; c=relaxed/simple; bh=zH06ZKSgGhEhXr0gPfiGkd8DxI44+kr5wz8zEOUWQZk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UjcfzqtRcCQ8+3pl5XkX6ZurnybvJLaMO68nraio30NGA0Xw4cjg5lXqTBJfQWlO1oINEY8ThNx4fQx+28Z9Tk9GSIMXYXoyB6Axn/y0wyrj8bDO6VS4zHZ83Y8Esk3hXh2tIrNxKxUnP3chZw3oiVtu6egtbS/5QtZARyurL80= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ibtt13p/; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ibtt13p/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7F1E1F00A3D; Sat, 15 Aug 2026 18:12:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817552; bh=56BaSgitaCOqK+plEdyRoTXW75Ul7dUQbsnofeonq2Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ibtt13p/ut3DXj5E1hvkhl1UNTj+CB2aiqwLD1F87/vzPhrcIE59TiioTJtgqrUMq yTuSga0hziN5WGfJgjieQJC2AffTSCOOUIzjYXMBkgV/e7KGKsvWstcibmeiQzHV0F 5ELl2w90P/v5Pbwd2rB/My+efCOG3or5vhM+J0VwrWjTbOPC5+XTKJJ9F3AtsEsnsr 4k2udMVv6Ab9G68lxpgLu75JhoSVS/jgIjpOspTgqWFi9LjJuN9Fqw+2F8gYTN4K8A QcrRhFfI1myb2qsBXx0e+hxRt2WN/l5TO/s2CwEmZaYfdpSsVMjYUYqWI7w6o0SbLJ 7ynQB+TjFHXeg== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v4 1/8] x86/fpu: Check for missing AVX and AVX-512 xstate bits Date: Sat, 15 Aug 2026 11:10:17 -0700 Message-ID: <20260815181024.28924-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If the CPU declares AVX or AVX-512 support, verify that the corresponding xstate bits are also set. If not, warn and clear them. This eliminates the perceived need for AVX and AVX-512 optimized code in the kernel to call cpu_has_xfeatures(). That has never been universally done, which strongly suggests that it has never really been needed in practice, but this should remove any remaining doubt. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/x86/kernel/fpu/xstate.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index a7b6524a9dea..97cfd4fb6cc0 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -806,7 +806,7 @@ static u64 __init guest_default_mask(void) void __init fpu__init_system_xstate(unsigned int legacy_size) { unsigned int eax, ebx, ecx, edx; - u64 xfeatures; + u64 xfeatures, mask; int err; int i; =20 @@ -818,6 +818,8 @@ void __init fpu__init_system_xstate(unsigned int legacy= _size) if (!boot_cpu_has(X86_FEATURE_XSAVE)) { pr_info("x86/fpu: x87 FPU will use %s\n", boot_cpu_has(X86_FEATURE_FXSR) ? "FXSAVE" : "FSAVE"); + /* Disable all dependent flags too */ + setup_clear_cpu_cap(X86_FEATURE_XSAVE); return; } =20 @@ -833,7 +835,8 @@ void __init fpu__init_system_xstate(unsigned int legacy= _size) cpuid_count(CPUID_LEAF_XSTATE, 1, &eax, &ebx, &ecx, &edx); fpu_kernel_cfg.max_features |=3D ecx + ((u64)edx << 32); =20 - if ((fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSSE) !=3D XFEATURE_MAS= K_FPSSE) { + mask =3D XFEATURE_MASK_FPSSE; + if ((fpu_kernel_cfg.max_features & mask) !=3D mask) { /* * This indicates that something really unexpected happened * with the enumeration. Disable XSAVE and try to continue @@ -844,6 +847,24 @@ void __init fpu__init_system_xstate(unsigned int legac= y_size) goto out_disable; } =20 + mask |=3D XFEATURE_MASK_YMM; + if (boot_cpu_has(X86_FEATURE_AVX)) { + if ((fpu_kernel_cfg.max_features & mask) !=3D mask) { + pr_err(FW_BUG + "x86/fpu: Disabling AVX support due to missing xstate features\n= "); + setup_clear_cpu_cap(X86_FEATURE_AVX); + } + } + + mask |=3D XFEATURE_MASK_AVX512; + if (boot_cpu_has(X86_FEATURE_AVX512F)) { + if ((fpu_kernel_cfg.max_features & mask) !=3D mask) { + pr_err(FW_BUG + "x86/fpu: Disabling AVX-512 support due to missing xstate featur= es\n"); + setup_clear_cpu_cap(X86_FEATURE_AVX512F); + } + } + if (fpu_kernel_cfg.max_features & XFEATURE_MASK_APX && fpu_kernel_cfg.max_features & (XFEATURE_MASK_BNDREGS | XFEATURE_MASK_= BNDCSR)) { /* --=20 2.55.0 From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97CE842466C; Sat, 15 Aug 2026 18:12:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817555; cv=none; b=WHlIgOKfPxKeIaQTRMj2lEXFbgqLvgQ0ZopXkeK3aLaZAY0Ps4SVcFnof26dI8nx0vyQK+VRLOb0ZoNVWltBLeIqZGugopPtnPgvwPDqvOXhtRnnyEFNqoJjIJZaWuJtxQBKyAbPGRAYZD4hHWKMWagMwBQ793DlR8rVZyWEsng= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817555; c=relaxed/simple; bh=7Z5v95LGHs2mjLET21NVji2yunu/nW8nT1iilSCIx10=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZbsG0c4b5CEfRCJVq0B/T8Fa5qOlhwbVUlxotdm0Vp46RR6TS0bQG+hhVpzCJYUTS5dpuQD9usGjn1YT8IObPxeCuGae/Pi9cAyrSb332zcJW6Gjmr36u6DP4/aMrOyEbmCV41V4ZY6E34qI87HGAYZ8yO7LiInYTAcz50oEgy8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=moJlEll/; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="moJlEll/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B430E1F00A3E; Sat, 15 Aug 2026 18:12:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817553; bh=WCMvK5ohQOFymbdY0OzMDpa7FyF9XhAAUoMQkc808Wk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=moJlEll/LoWBuhJimpjctgnUSMd48Rlal3HXbk/EN1lGeg421MCoog3x364lo0qfH ofQaiN3HDD5npxbBnMbscR0jAriSH8JhssOsyiDGba9u5+jNP2ufwb4MRTqWFSeB49 i5iKMmUkFzwc8AfnY37k6KNmBhQKwvwxHSIH4Wsr+McfnI0tNOM6nG8WYqHINAtdhc yVm0KwePY/yKp6F8A+wQVNmgTQCdsiZJIYcYTl/NCxBx5EZjCvOvaXa3juAvkV5Aha iNuRL9OOzBk6Or4WOAnHg9Zw2p2GYSaI8yyu0Yzo3iWuCwvROWD2mt7Mo/PhY3y2Di TdrfKUlWklM+A== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v4 2/8] um: Check for missing AVX and AVX-512 xstate bits Date: Sat, 15 Aug 2026 11:10:18 -0700 Message-ID: <20260815181024.28924-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If the CPU declares AVX or AVX-512 support, verify that all the corresponding bits are set in the XCR0 register. If any are missing, warn and don't set the corresponding X86_FEATURE_* flags. This eliminates the perceived need for UML-supporting AVX and AVX-512 optimized code in the kernel (that is, lib/raid/ currently) to start checking the XCR0 bits in addition to X86_FEATURE_AVX*. This aligns UML with the vast majority of userspace programs, which check the XCR0 bits before considering AVX and AVX-512 to be supported, as per the procedure documented in Intel's CPU manual. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/um/kernel/um_arch.c | 78 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 77 insertions(+), 1 deletion(-) diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c index 2141f5f1f5a2..aafbaef2ae82 100644 --- a/arch/um/kernel/um_arch.c +++ b/arch/um/kernel/um_arch.c @@ -264,12 +264,88 @@ unsigned long brk_start; =20 #define MIN_VMALLOC (32 * 1024 * 1024) =20 +static u64 __init read_xcr0(void) +{ + u32 a, b, c, d; + + asm volatile("cpuid" + : "=3Da"(a), "=3Db"(b), "=3Dc"(c), "=3Dd"(d) + : "a"(0), "c"(0)); + if (a >=3D 1) { /* max_leaf >=3D 1 */ + asm volatile("cpuid" + : "=3Da"(a), "=3Db"(b), "=3Dc"(c), "=3Dd"(d) + : "a"(1), "c"(0)); + if (c & (1 << 27)) { /* XSAVE enabled by OS */ + asm volatile("xgetbv" : "=3Dd"(d), "=3Da"(a) : "c"(0)); + return ((u64)d << 32) | a; + } + } + return 0; +} + +static void __init validate_and_set_cpu_cap(int cap, u64 xcr0) +{ + /* + * Check for missing xstate features right away, so that there's no + * perceived need for all optimized code in the kernel to do so. + */ + switch (cap) { + case X86_FEATURE_AVX: + case X86_FEATURE_AVX2: + case X86_FEATURE_AVX_VNNI: + case X86_FEATURE_FMA: + case X86_FEATURE_VAES: + case X86_FEATURE_VPCLMULQDQ: + if ((xcr0 & 0x7) !=3D 0x7) { + static bool warned; + + if (!warned) { + os_warn("Disabling AVX support due to missing xstate features\n"); + warned =3D true; + } + return; + } + break; + case X86_FEATURE_AVX512F: + case X86_FEATURE_AVX512BW: + case X86_FEATURE_AVX512CD: + case X86_FEATURE_AVX512DQ: + case X86_FEATURE_AVX512ER: + case X86_FEATURE_AVX512IFMA: + case X86_FEATURE_AVX512PF: + case X86_FEATURE_AVX512VBMI: + case X86_FEATURE_AVX512VL: + case X86_FEATURE_AVX512_4FMAPS: + case X86_FEATURE_AVX512_4VNNIW: + case X86_FEATURE_AVX512_BF16: + case X86_FEATURE_AVX512_BITALG: + case X86_FEATURE_AVX512_FP16: + case X86_FEATURE_AVX512_VBMI2: + case X86_FEATURE_AVX512_VNNI: + case X86_FEATURE_AVX512_VP2INTERSECT: + case X86_FEATURE_AVX512_VPOPCNTDQ: + if ((xcr0 & 0xe7) !=3D 0xe7) { + static bool warned; + + if (!warned) { + os_warn("Disabling AVX-512 support due to missing xstate features\n"); + warned =3D true; + } + return; + } + break; + } + set_cpu_cap(&boot_cpu_data, cap); +} + static void __init parse_host_cpu_flags(char *line) { + u64 xcr0 =3D read_xcr0(); int i; + for (i =3D 0; i < 32*NCAPINTS; i++) { if ((x86_cap_flags[i] !=3D NULL) && strstr(line, x86_cap_flags[i])) - set_cpu_cap(&boot_cpu_data, i); + validate_and_set_cpu_cap(i, xcr0); } } =20 --=20 2.55.0 From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9444B377A97; Sat, 15 Aug 2026 18:12:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817557; cv=none; b=IGXxaGap5o73Frr0NqMD/sg8DEOLzqFP18UxbFSOKmTYzEfJbEK6I64+9KpoDo8eSpm5HolDavC7NbT0Wr49+4GSUFjH5HqidWQEIHW/+SNV0WAFcTDmxm45StgDFyFrnzfdIcSOWGRoanfWJuE0grx16qToBqKlrReU29PIE8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817557; c=relaxed/simple; bh=j71FQSXxVyY13IVMc20ZI/TX8wDr4Hvb0YpK4t8lc9s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=THqhPJHeMzTiOx20q/ElGPC3HotGOS+l+n5LuNnS1QHFp4SOnXb1Drzlwm5chIRbcD07RUN77uOFWQ3NnNgkY/DVj0/aTuGQ5o1X/FYACUZQLPKjIJR2rhduc10UXsCYEPwwVspg+yhvnmmNMgRUlFYzl45zYj6j6R9PM/gIVm4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QFl5/EvC; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QFl5/EvC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7BAB81F000E9; Sat, 15 Aug 2026 18:12:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817554; bh=VKmmGWRx3Pfbo+nD8y5eQpPaN0sWZSIp/Ijjeyx4vLE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=QFl5/EvCI9CMmkEQBpw56swdFXpiuMPwyrphmecsee1qR+C+WeZYmbYwsPukZSmZk oBxOT7HLAjpZRff3OD90eAZXg5RV8AGyEqiX2p9jLJ7j25p3UuJ4wIqcicXzJizMwt q6WwPz/+uIJXUp1oxZn6ed4gBl5KtteQn3XuUkRRluZY02DyUE3VldIrW/k42eXXt6 wA6jhn/W0cuG5GRUApS3I2y5fOXb0wEnNmR5dxcdkZnvt2/i3x1pOjypWHpfUs4rgO MjNl3fPK+d3D+WJ7Y0PqXB1cnACezdBm/y55bFhY3mMDQn7SnKE+9wZVpUl15HK1Dk JBn7BGqbnFF3Q== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v4 3/8] crypto: x86 - Stop using cpu_has_xfeatures() Date: Sat, 15 Aug 2026 11:10:19 -0700 Message-ID: <20260815181024.28924-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Checking both boot_cpu_has(X86_FEATURE_AVX*) and cpu_has_xfeatures() has never really been needed in practice, and it's never been universally done (e.g., lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML now explicitly clear the AVX and AVX-512 flags if their xfeatures are missing, which should remove any remaining doubts. Thus, remove all the calls to cpu_has_xfeatures(), as well as the related checks of boot_cpu_has(X86_FEATURE_OSXSAVE). In a few cases there was no corresponding boot_cpu_has(X86_FEATURE_AVX*) check, so add the missing ones. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/x86/crypto/aegis128-aesni-glue.c | 3 +-- arch/x86/crypto/aesni-intel_glue.c | 7 ++----- arch/x86/crypto/aria_aesni_avx2_glue.c | 11 +---------- arch/x86/crypto/aria_aesni_avx_glue.c | 11 +---------- arch/x86/crypto/aria_gfni_avx512_glue.c | 11 +---------- arch/x86/crypto/camellia_aesni_avx2_glue.c | 11 +---------- arch/x86/crypto/camellia_aesni_avx_glue.c | 11 +---------- arch/x86/crypto/cast5_avx_glue.c | 7 ++----- arch/x86/crypto/cast6_avx_glue.c | 7 ++----- arch/x86/crypto/serpent_avx2_glue.c | 9 +-------- arch/x86/crypto/serpent_avx_glue.c | 7 ++----- arch/x86/crypto/sm4_aesni_avx2_glue.c | 11 +---------- arch/x86/crypto/sm4_aesni_avx_glue.c | 11 +---------- arch/x86/crypto/twofish_avx_glue.c | 6 ++---- 14 files changed, 19 insertions(+), 104 deletions(-) diff --git a/arch/x86/crypto/aegis128-aesni-glue.c b/arch/x86/crypto/aegis1= 28-aesni-glue.c index f1adfba1a76e..09fc0b15b0e9 100644 --- a/arch/x86/crypto/aegis128-aesni-glue.c +++ b/arch/x86/crypto/aegis128-aesni-glue.c @@ -265,8 +265,7 @@ static struct aead_alg crypto_aegis128_aesni_alg =3D { static int __init crypto_aegis128_aesni_module_init(void) { if (!boot_cpu_has(X86_FEATURE_XMM4_1) || - !boot_cpu_has(X86_FEATURE_AES) || - !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL)) + !boot_cpu_has(X86_FEATURE_AES)) return -ENODEV; =20 return crypto_register_aead(&crypto_aegis128_aesni_alg); diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-int= el_glue.c index f522fff9231e..f6f899db7482 100644 --- a/arch/x86/crypto/aesni-intel_glue.c +++ b/arch/x86/crypto/aesni-intel_glue.c @@ -1548,8 +1548,7 @@ static int __init register_avx_algs(void) if (!boot_cpu_has(X86_FEATURE_AVX2) || !boot_cpu_has(X86_FEATURE_VAES) || !boot_cpu_has(X86_FEATURE_VPCLMULQDQ) || - !boot_cpu_has(X86_FEATURE_PCLMULQDQ) || - !cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + !boot_cpu_has(X86_FEATURE_PCLMULQDQ)) return 0; err =3D crypto_register_skciphers(skcipher_algs_vaes_avx2, ARRAY_SIZE(skcipher_algs_vaes_avx2)); @@ -1562,9 +1561,7 @@ static int __init register_avx_algs(void) =20 if (!boot_cpu_has(X86_FEATURE_AVX512BW) || !boot_cpu_has(X86_FEATURE_AVX512VL) || - !boot_cpu_has(X86_FEATURE_BMI2) || - !cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | - XFEATURE_MASK_AVX512, NULL)) + !boot_cpu_has(X86_FEATURE_BMI2)) return 0; =20 if (boot_cpu_has(X86_FEATURE_PREFER_YMM)) { diff --git a/arch/x86/crypto/aria_aesni_avx2_glue.c b/arch/x86/crypto/aria_= aesni_avx2_glue.c index 1487a49bfbac..371be2fb6469 100644 --- a/arch/x86/crypto/aria_aesni_avx2_glue.c +++ b/arch/x86/crypto/aria_aesni_avx2_glue.c @@ -195,22 +195,13 @@ static struct skcipher_alg aria_algs[] =3D { =20 static int __init aria_avx2_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX2 or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - if (boot_cpu_has(X86_FEATURE_GFNI)) { aria_ops.aria_encrypt_16way =3D aria_aesni_avx_gfni_encrypt_16way; aria_ops.aria_decrypt_16way =3D aria_aesni_avx_gfni_decrypt_16way; diff --git a/arch/x86/crypto/aria_aesni_avx_glue.c b/arch/x86/crypto/aria_a= esni_avx_glue.c index e4e3d78915a5..d23fc91c0ebd 100644 --- a/arch/x86/crypto/aria_aesni_avx_glue.c +++ b/arch/x86/crypto/aria_aesni_avx_glue.c @@ -182,21 +182,12 @@ static struct skcipher_alg aria_algs[] =3D { =20 static int __init aria_avx_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - if (boot_cpu_has(X86_FEATURE_GFNI)) { aria_ops.aria_encrypt_16way =3D aria_aesni_avx_gfni_encrypt_16way; aria_ops.aria_decrypt_16way =3D aria_aesni_avx_gfni_decrypt_16way; diff --git a/arch/x86/crypto/aria_gfni_avx512_glue.c b/arch/x86/crypto/aria= _gfni_avx512_glue.c index 363cbf4399cc..e05bbeb22d4a 100644 --- a/arch/x86/crypto/aria_gfni_avx512_glue.c +++ b/arch/x86/crypto/aria_gfni_avx512_glue.c @@ -196,24 +196,15 @@ static struct skcipher_alg aria_algs[] =3D { =20 static int __init aria_avx512_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || !boot_cpu_has(X86_FEATURE_AVX512F) || !boot_cpu_has(X86_FEATURE_AVX512VL) || - !boot_cpu_has(X86_FEATURE_GFNI) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_GFNI)) { pr_info("AVX512/GFNI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | - XFEATURE_MASK_AVX512, &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - aria_ops.aria_encrypt_16way =3D aria_aesni_avx_gfni_encrypt_16way; aria_ops.aria_decrypt_16way =3D aria_aesni_avx_gfni_decrypt_16way; aria_ops.aria_ctr_crypt_16way =3D aria_aesni_avx_gfni_ctr_crypt_16way; diff --git a/arch/x86/crypto/camellia_aesni_avx2_glue.c b/arch/x86/crypto/c= amellia_aesni_avx2_glue.c index 2d2f4e16537c..073fa3bb8388 100644 --- a/arch/x86/crypto/camellia_aesni_avx2_glue.c +++ b/arch/x86/crypto/camellia_aesni_avx2_glue.c @@ -97,22 +97,13 @@ static struct skcipher_alg camellia_algs[] =3D { =20 static int __init camellia_aesni_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX2 or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(camellia_algs, ARRAY_SIZE(camellia_algs)); } diff --git a/arch/x86/crypto/camellia_aesni_avx_glue.c b/arch/x86/crypto/ca= mellia_aesni_avx_glue.c index 5c321f255eb7..872e5e07220f 100644 --- a/arch/x86/crypto/camellia_aesni_avx_glue.c +++ b/arch/x86/crypto/camellia_aesni_avx_glue.c @@ -98,21 +98,12 @@ static struct skcipher_alg camellia_algs[] =3D { =20 static int __init camellia_aesni_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(camellia_algs, ARRAY_SIZE(camellia_algs)); } diff --git a/arch/x86/crypto/cast5_avx_glue.c b/arch/x86/crypto/cast5_avx_g= lue.c index 3aca04d43b34..5de35e863370 100644 --- a/arch/x86/crypto/cast5_avx_glue.c +++ b/arch/x86/crypto/cast5_avx_glue.c @@ -92,11 +92,8 @@ static struct skcipher_alg cast5_algs[] =3D { =20 static int __init cast5_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 diff --git a/arch/x86/crypto/cast6_avx_glue.c b/arch/x86/crypto/cast6_avx_g= lue.c index c4dd28c30303..3d7ea48007bc 100644 --- a/arch/x86/crypto/cast6_avx_glue.c +++ b/arch/x86/crypto/cast6_avx_glue.c @@ -92,11 +92,8 @@ static struct skcipher_alg cast6_algs[] =3D { =20 static int __init cast6_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 diff --git a/arch/x86/crypto/serpent_avx2_glue.c b/arch/x86/crypto/serpent_= avx2_glue.c index f5f2121b7956..72a9e2b306d6 100644 --- a/arch/x86/crypto/serpent_avx2_glue.c +++ b/arch/x86/crypto/serpent_avx2_glue.c @@ -93,17 +93,10 @@ static struct skcipher_alg serpent_algs[] =3D { =20 static int __init serpent_avx2_init(void) { - const char *feature_name; - - if (!boot_cpu_has(X86_FEATURE_AVX2) || !boot_cpu_has(X86_FEATURE_OSXSAVE)= ) { + if (!boot_cpu_has(X86_FEATURE_AVX2)) { pr_info("AVX2 instructions are not detected.\n"); return -ENODEV; } - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } =20 return crypto_register_skciphers(serpent_algs, ARRAY_SIZE(serpent_algs)); diff --git a/arch/x86/crypto/serpent_avx_glue.c b/arch/x86/crypto/serpent_a= vx_glue.c index 9c8b3a335d5c..42c4e1569674 100644 --- a/arch/x86/crypto/serpent_avx_glue.c +++ b/arch/x86/crypto/serpent_avx_glue.c @@ -100,11 +100,8 @@ static struct skcipher_alg serpent_algs[] =3D { =20 static int __init serpent_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 diff --git a/arch/x86/crypto/sm4_aesni_avx2_glue.c b/arch/x86/crypto/sm4_ae= sni_avx2_glue.c index fec0ab7a63dd..eef73894e777 100644 --- a/arch/x86/crypto/sm4_aesni_avx2_glue.c +++ b/arch/x86/crypto/sm4_aesni_avx2_glue.c @@ -98,22 +98,13 @@ static struct skcipher_alg sm4_aesni_avx2_skciphers[] = =3D { =20 static int __init sm4_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || !boot_cpu_has(X86_FEATURE_AVX2) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX2 or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(sm4_aesni_avx2_skciphers, ARRAY_SIZE(sm4_aesni_avx2_skciphers)); } diff --git a/arch/x86/crypto/sm4_aesni_avx_glue.c b/arch/x86/crypto/sm4_aes= ni_avx_glue.c index 88caf418a06f..ed383da5ff46 100644 --- a/arch/x86/crypto/sm4_aesni_avx_glue.c +++ b/arch/x86/crypto/sm4_aesni_avx_glue.c @@ -314,21 +314,12 @@ static struct skcipher_alg sm4_aesni_avx_skciphers[] = =3D { =20 static int __init sm4_init(void) { - const char *feature_name; - if (!boot_cpu_has(X86_FEATURE_AVX) || - !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE)) { + !boot_cpu_has(X86_FEATURE_AES)) { pr_info("AVX or AES-NI instructions are not detected.\n"); return -ENODEV; } =20 - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - &feature_name)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); - return -ENODEV; - } - return crypto_register_skciphers(sm4_aesni_avx_skciphers, ARRAY_SIZE(sm4_aesni_avx_skciphers)); } diff --git a/arch/x86/crypto/twofish_avx_glue.c b/arch/x86/crypto/twofish_a= vx_glue.c index 9e20db013750..985bc54a2340 100644 --- a/arch/x86/crypto/twofish_avx_glue.c +++ b/arch/x86/crypto/twofish_avx_glue.c @@ -102,10 +102,8 @@ static struct skcipher_alg twofish_algs[] =3D { =20 static int __init twofish_init(void) { - const char *feature_name; - - if (!cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, &feature_na= me)) { - pr_info("CPU feature '%s' is not supported.\n", feature_name); + if (!boot_cpu_has(X86_FEATURE_AVX)) { + pr_info("AVX instructions are not detected.\n"); return -ENODEV; } =20 --=20 2.55.0 From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A7CB429011; Sat, 15 Aug 2026 18:12:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817557; cv=none; b=b4mKKwAglFFfZFSasQizOrVKnbxdae+Vd83+9TmQgMox8emuWk+mH/8WqT5abdmh3vi4wM2rnoLkyY+Yp5RtAL0RDpkSvjVDrDRdx81fXAPE4oPJNGhjb2JBSBMWYr8qqXVzJkxiIR0j1viDZPLfAYC2wyuDPB+ZjYgYRMf344Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817557; c=relaxed/simple; bh=e0/5EyKvRR38AkedRTxpFtVqo23cOjcvx5sQexInGDU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NFvFLlaYLSA2wWoreSW7VE8QNFG8g/M240PZPGQSGVCiWMmlnaNF64j9K17I7npMxfnQO8gYxWknMQdYPu/DigZ6bAtxjF5XJ98qPqyYhjOePwksK7lKkd/Q+iz3F1mJE/61qtB2d96U5liEKaxpuGOOCXNR2ZEmTtHrElYnScA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QVSOf/P1; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QVSOf/P1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4E6071F00A3A; Sat, 15 Aug 2026 18:12:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817554; bh=DdqWq0+mPRqIC9aeO4PAgts2wuJAHghXz/rr2+6DDJI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=QVSOf/P1bGX0653466SaB69KvzUD86Og8s+ry4GiiYoFtHOdqQNYpzVkL8jHXg38x t2N0e3ugnBdSntzPd9BD7/ozgMdHHAzESKUIMErWAxp7ipxbxKAggB6/J+dyCwcm0K 6qN9iWUQYGGAaLbq3E+vqExnbcQBa+8l43uQlqwrNgg/Soy0bTVzydFaHCwWcOtb6P C5RMLAq+yU3FNKUOcvFZjqCDQi6oqz7xdnUcQa5zbn4GpZ8gNlUBOl9SAnR7TOH+D5 kfa4sb0fAivvISqZU5wo//lIMdUhJH5+r8881yXmUZ70soJmcqugKPTq5lA+SwMhiI ZrJiuJcZoE4kg== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v4 4/8] lib/crypto: x86: Stop using cpu_has_xfeatures() Date: Sat, 15 Aug 2026 11:10:20 -0700 Message-ID: <20260815181024.28924-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Checking both boot_cpu_has() and cpu_has_xfeatures() has never really been needed in practice, and it's never been universally done (e.g., lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML now explicitly clear the AVX and AVX-512 flags if their xfeatures are missing, which should remove any remaining doubts. Thus, remove all the calls to cpu_has_xfeatures(). Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/crypto/x86/blake2s.h | 4 +--- lib/crypto/x86/chacha.h | 3 +-- lib/crypto/x86/nh.h | 4 +--- lib/crypto/x86/poly1305.h | 7 ++----- lib/crypto/x86/sha1.h | 4 +--- lib/crypto/x86/sha256.h | 4 +--- lib/crypto/x86/sha512.h | 3 +-- lib/crypto/x86/sm3.h | 3 +-- 8 files changed, 9 insertions(+), 23 deletions(-) diff --git a/lib/crypto/x86/blake2s.h b/lib/crypto/x86/blake2s.h index f8eed6cb042e..0f7c51f055c8 100644 --- a/lib/crypto/x86/blake2s.h +++ b/lib/crypto/x86/blake2s.h @@ -55,8 +55,6 @@ static void blake2s_mod_init_arch(void) if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX512F) && - boot_cpu_has(X86_FEATURE_AVX512VL) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | - XFEATURE_MASK_AVX512, NULL)) + boot_cpu_has(X86_FEATURE_AVX512VL)) static_branch_enable(&blake2s_use_avx512); } diff --git a/lib/crypto/x86/chacha.h b/lib/crypto/x86/chacha.h index 10cf8f1c569d..c79562aac56b 100644 --- a/lib/crypto/x86/chacha.h +++ b/lib/crypto/x86/chacha.h @@ -165,8 +165,7 @@ static void chacha_mod_init_arch(void) static_branch_enable(&chacha_use_simd); =20 if (boot_cpu_has(X86_FEATURE_AVX) && - boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) { + boot_cpu_has(X86_FEATURE_AVX2)) { static_branch_enable(&chacha_use_avx2); =20 if (boot_cpu_has(X86_FEATURE_AVX512VL) && diff --git a/lib/crypto/x86/nh.h b/lib/crypto/x86/nh.h index 83361c2e9783..342636dcb750 100644 --- a/lib/crypto/x86/nh.h +++ b/lib/crypto/x86/nh.h @@ -37,9 +37,7 @@ static void nh_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_XMM2)) { static_branch_enable(&have_sse2); - if (boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - NULL)) + if (boot_cpu_has(X86_FEATURE_AVX2)) static_branch_enable(&have_avx2); } } diff --git a/lib/crypto/x86/poly1305.h b/lib/crypto/x86/poly1305.h index ee92e3740a78..b061b9926fa5 100644 --- a/lib/crypto/x86/poly1305.h +++ b/lib/crypto/x86/poly1305.h @@ -143,15 +143,12 @@ static void poly1305_emit(const struct poly1305_state= *ctx, #define poly1305_mod_init_arch poly1305_mod_init_arch static void poly1305_mod_init_arch(void) { - if (boot_cpu_has(X86_FEATURE_AVX) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + if (boot_cpu_has(X86_FEATURE_AVX)) static_branch_enable(&poly1305_use_avx); - if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2)) static_branch_enable(&poly1305_use_avx2); if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX512F) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | XFEATURE_MA= SK_AVX512, NULL) && /* Skylake downclocks unacceptably much when using zmm, but later gen= erations are fast. */ boot_cpu_data.x86_vfm !=3D INTEL_SKYLAKE_X) static_branch_enable(&poly1305_use_avx512); diff --git a/lib/crypto/x86/sha1.h b/lib/crypto/x86/sha1.h index c48a0131fd12..6aff433466e7 100644 --- a/lib/crypto/x86/sha1.h +++ b/lib/crypto/x86/sha1.h @@ -59,9 +59,7 @@ static void sha1_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_SHA_NI)) { static_call_update(sha1_blocks_x86, sha1_blocks_ni); - } else if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - NULL) && - boot_cpu_has(X86_FEATURE_AVX)) { + } else if (boot_cpu_has(X86_FEATURE_AVX)) { if (boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_BMI1) && boot_cpu_has(X86_FEATURE_BMI2)) diff --git a/lib/crypto/x86/sha256.h b/lib/crypto/x86/sha256.h index 0ee69d8e39fe..e98ffdaf4b14 100644 --- a/lib/crypto/x86/sha256.h +++ b/lib/crypto/x86/sha256.h @@ -104,9 +104,7 @@ static void sha256_mod_init_arch(void) boot_cpu_has(X86_FEATURE_PHE_EN) && boot_cpu_data.x86 >=3D 0x07) { static_call_update(sha256_blocks_x86, sha256_blocks_phe); - } else if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, - NULL) && - boot_cpu_has(X86_FEATURE_AVX)) { + } else if (boot_cpu_has(X86_FEATURE_AVX)) { if (boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_BMI2)) static_call_update(sha256_blocks_x86, diff --git a/lib/crypto/x86/sha512.h b/lib/crypto/x86/sha512.h index 0213c70cedd0..4e177b4606bd 100644 --- a/lib/crypto/x86/sha512.h +++ b/lib/crypto/x86/sha512.h @@ -37,8 +37,7 @@ static void sha512_blocks(struct sha512_block_state *stat= e, #define sha512_mod_init_arch sha512_mod_init_arch static void sha512_mod_init_arch(void) { - if (cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL) && - boot_cpu_has(X86_FEATURE_AVX)) { + if (boot_cpu_has(X86_FEATURE_AVX)) { if (boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_BMI2)) static_call_update(sha512_blocks_x86, diff --git a/lib/crypto/x86/sm3.h b/lib/crypto/x86/sm3.h index 3834780f2f6a..e06d4a22e4fa 100644 --- a/lib/crypto/x86/sm3.h +++ b/lib/crypto/x86/sm3.h @@ -33,7 +33,6 @@ static void sm3_blocks(struct sm3_block_state *state, #define sm3_mod_init_arch sm3_mod_init_arch static void sm3_mod_init_arch(void) { - if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_BMI2) && - cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM, NULL)) + if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_BMI2)) static_call_update(sm3_blocks_x86, sm3_blocks_avx); } --=20 2.55.0 From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9199B42642D; Sat, 15 Aug 2026 18:12:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817558; cv=none; b=MkEOsqJbUpnlkc2rzAr/onvqp9R8rcHF6gB1Z4mIXhirX+Fv8JBVaGeM0g2wWrFCHvpZ3wRXH/uni8iEmZnHX2XbO1epem7PZuV2+sMz9oXHv5t726BnICX8ekQb32HI0TT46aSIkGd6t46Hvpp7c23McccNpqCQ1HUZXbkjaR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817558; c=relaxed/simple; bh=8EtIjUyWp+Owf958VUqMsdagPce6VSl8WYVhBIP0kNU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WNV6SArXfVDb0BlQM3Q8AXIvSg/5jlTmLdL0B70lmwemMlgVjxaYGDn8/OP7S5xWSqXX3nZ8SSEclbDzHiiSQsB6dwBBHKG3S9cH9G+nhPOXMJsySGPf2FfR3ki7UgXMx7hbV2oixzJuXqioFhXsnlZjdnDdx/AZxFXp/yX3NkM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ED6NroIY; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ED6NroIY" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2757D1F00AC4; Sat, 15 Aug 2026 18:12:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817555; bh=r4lrJRkT6oIO9uz4byVKnhiX3ckXssiVruBvq7U+FfI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ED6NroIYBJzgy6GwIXX/cesOeyUD3VDuuZyQIjm2HwR6sb+3QteysdUjScX1ftRxI Hgp14z/4FXsQ/2Uhuw/3GuxKs7HGwPtmLmt9Xxz8wTH8Oc7NBsOQxOdNlkTLKvAylE JbpJJ6FoLsYjja1a88ouhRH2roiid2VsyU1Dyr0WuqGj3WJkJvulSIbyQPS01w4XsK VgciHopL+mGOoS5XvwQymfKOW7tKjammPIxPKdTepdtZiYaK1v6bzbK8CuX2/FT01Q tvBzaj6V9UyJnAsywJwSO5rVyybgfHgwfqeXMgg8XQLmUngpLhsias4TN0CMrFjBmV UdNgCrnd5UTyg== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v4 5/8] lib/crc: x86: Stop using cpu_has_xfeatures() Date: Sat, 15 Aug 2026 11:10:21 -0700 Message-ID: <20260815181024.28924-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Checking both boot_cpu_has() and cpu_has_xfeatures() has never really been needed in practice, and it's never been universally done (e.g., lib/raid/ omits cpu_has_xfeatures()). Nevertheless, both x86 and UML now explicitly clear the AVX and AVX-512 flags if their xfeatures are missing, which should remove any remaining doubts. Thus, remove all the calls to cpu_has_xfeatures(). Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/crc/x86/crc-pclmul-template.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/lib/crc/x86/crc-pclmul-template.h b/lib/crc/x86/crc-pclmul-tem= plate.h index 02744831c6fa..893119bb7c07 100644 --- a/lib/crc/x86/crc-pclmul-template.h +++ b/lib/crc/x86/crc-pclmul-template.h @@ -27,16 +27,14 @@ DEFINE_STATIC_CALL(prefix##_pclmul, prefix##_pclmul_sse) static inline bool have_vpclmul(void) { return boot_cpu_has(X86_FEATURE_VPCLMULQDQ) && - boot_cpu_has(X86_FEATURE_AVX2) && - cpu_has_xfeatures(XFEATURE_MASK_YMM, NULL); + boot_cpu_has(X86_FEATURE_AVX2); } =20 static inline bool have_avx512(void) { return boot_cpu_has(X86_FEATURE_AVX512BW) && boot_cpu_has(X86_FEATURE_AVX512VL) && - !boot_cpu_has(X86_FEATURE_PREFER_YMM) && - cpu_has_xfeatures(XFEATURE_MASK_AVX512, NULL); + !boot_cpu_has(X86_FEATURE_PREFER_YMM); } =20 /* --=20 2.55.0 From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB626429CC5; Sat, 15 Aug 2026 18:12:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817559; cv=none; b=TUym6EyCZI3epDRjX30CGyrMgOBYjDnJhDcY9fgABFzet8VV2K7hLCkLCyq2KImiYhhEIDDyarH23IpjeCTueO67SqgfUI+I2MAGl87+rJXDyXBmoorZRF+c0W5Q2QPtrfDtLu8EhSKYXEIIfZdk/OzjR8jpiTmySSCkFYTxzOk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817559; c=relaxed/simple; bh=mF5Si+fJ2SqCKuU/60v4aWm6TNcVfLmMHZQC50LpG1A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Nh4snxaYgzS4gvsFkVluWtmhzZz2dsrF8yq3HPvSRkTHcriy+HxT6kyg64df4GjJ9sQ0L+FbGF36RgOKl8cJkbIz3Y54Q+3mFQuheXcqU2anFQKMDQYs46lL/TNoIaWS1QIK+4KZVIBp9cYHz3us3wqhyowGYj6xwsueNIFg628= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=c4wGJ8Vh; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="c4wGJ8Vh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D58F01F00A3D; Sat, 15 Aug 2026 18:12:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817556; bh=Q2A0VpZ3jEW63u/fB0uqkGhkDFeSh86cNwd3FvE985o=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=c4wGJ8VhfbWJcDXsBWiYh+o+RaU1oHSwqYO5XkKUDm3SwYB1bM/Jig7ny6pjVCX+z LWYigBVe0JYLa+bZdolB7OrxOskpY0bDqWsIDBSVS7BrxMuE7l85gE94/PM/ipaLSZ 5h8ZjdKFDpA4HAWuoqogtJvw3SvlVVzrC9h61NdU6w71A/bgrQ9+vwqGDnoZO3OUAi pZkHG+lww+sKuVNmgFH4BcNLtj1gZnEgB2k4aSaJ31HFAesK6PBHb5n34NPfj14glr oa89nzefpeYK5UAs++FLPAUJT4ZkbvGFMu2BlW/oOUIHNaDF/g2BszZ66R34poxAmg ZZu4eY3eouuww== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v4 6/8] x86/fpu: Remove cpu_has_xfeatures() Date: Sat, 15 Aug 2026 11:10:22 -0700 Message-ID: <20260815181024.28924-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The only remaining caller of cpu_has_xfeatures() is print_xstate_features(), which uses it only to check and get the name of a single feature. Remove it and just inline the needed code into print_xstate_features(). This also makes the "unknown xstate feature" entry at index XFEATURE_MAX of xfeature_names[] unnecessary, so remove that too. Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- arch/x86/include/asm/fpu/api.h | 9 ------- arch/x86/kernel/fpu/xstate.c | 44 +++------------------------------- 2 files changed, 3 insertions(+), 50 deletions(-) diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index 90c63fe19c0f..cfed8b24d64f 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -99,15 +99,6 @@ static inline void fpregs_assert_state_consistent(void) = { } */ extern void switch_fpu_return(void); =20 -/* - * Query the presence of one or more xfeatures. Works on any legacy CPU as= well. - * - * If 'feature_name' is set then put a human-readable description of - * the feature there as well - this can be used to print error (or success) - * messages. - */ -extern int cpu_has_xfeatures(u64 xfeatures_mask, const char **feature_name= ); - /* Trap handling */ extern int fpu__exception_code(struct fpu *fpu, int trap_nr); extern void fpu_sync_fpstate(struct fpu *fpu); diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 97cfd4fb6cc0..0b17452d1788 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -66,8 +66,8 @@ static const char *xfeature_names[] =3D "AMX Tile config", "AMX Tile data", "APX registers", - "unknown xstate feature", }; +static_assert(ARRAY_SIZE(xfeature_names) =3D=3D XFEATURE_MAX); =20 static unsigned short xsave_cpuid_features[] __initdata =3D { [XFEATURE_FP] =3D X86_FEATURE_FPU, @@ -122,44 +122,6 @@ static inline unsigned int next_xfeature_order(unsigne= d int i, u64 mask) #define XSTATE_FLAG_SUPERVISOR BIT(0) #define XSTATE_FLAG_ALIGNED64 BIT(1) =20 -/* - * Return whether the system supports a given xfeature. - * - * Also return the name of the (most advanced) feature that the caller req= uested: - */ -int cpu_has_xfeatures(u64 xfeatures_needed, const char **feature_name) -{ - u64 xfeatures_missing =3D xfeatures_needed & ~fpu_kernel_cfg.max_features; - - if (unlikely(feature_name)) { - long xfeature_idx, max_idx; - u64 xfeatures_print; - /* - * So we use FLS here to be able to print the most advanced - * feature that was requested but is missing. So if a driver - * asks about "XFEATURE_MASK_SSE | XFEATURE_MASK_YMM" we'll print the - * missing AVX feature - this is the most informative message - * to users: - */ - if (xfeatures_missing) - xfeatures_print =3D xfeatures_missing; - else - xfeatures_print =3D xfeatures_needed; - - xfeature_idx =3D fls64(xfeatures_print)-1; - max_idx =3D ARRAY_SIZE(xfeature_names)-1; - xfeature_idx =3D min(xfeature_idx, max_idx); - - *feature_name =3D xfeature_names[xfeature_idx]; - } - - if (xfeatures_missing) - return 0; - - return 1; -} -EXPORT_SYMBOL_GPL(cpu_has_xfeatures); - static bool xfeature_is_aligned64(int xfeature_nr) { return xstate_flags[xfeature_nr] & XSTATE_FLAG_ALIGNED64; @@ -302,9 +264,9 @@ static void __init print_xstate_features(void) =20 for (i =3D 0; i < XFEATURE_MAX; i++) { u64 mask =3D BIT_ULL(i); - const char *name; + const char *name =3D xfeature_names[i]; =20 - if (cpu_has_xfeatures(mask, &name)) + if (fpu_kernel_cfg.max_features & mask) pr_info("x86/fpu: Supporting XSAVE feature 0x%03Lx: '%s'\n", mask, name= ); } } --=20 2.55.0 From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DF6C42A155; Sat, 15 Aug 2026 18:12:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817559; cv=none; b=Tv2tznzpP99qK+ZpzPD1rr64cpDP8Q564i+VlMyfo8YFxhI9zpINz4wYHA/hHdHoi0jq7kJQO3mic0lgLsy1VnYO8ZhxRMnJaffN0XMimJdxM6VC2YUgWL65y8NQcFqSLzJUfiAN6sSFd58OBXsjwtIg7fR66MHtA3mWw/YPf/s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817559; c=relaxed/simple; bh=bURSDeOHD7BeXhOUO95qSmh0YND+CXXzB5NqaBliqI4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bpASUIBT7sbxWYI6OWadWt+2qnBVRuWjc14WTXjjOIgVTMYxRdxUgfyg1O7XsDm/fEClYrsM1h+RR3PVeooTj+kmt54dLC0ShRHnBwdI+qRvsv8lqy/oCKgnI5IjFyAPQH3jG8JbtdBWofe6UUsoxZQISeihgzEfsQNnPwE9Tmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ldjMJ+im; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ldjMJ+im" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81EFA1F00ACA; Sat, 15 Aug 2026 18:12:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817556; bh=CGFmBktDb18Ou8jdzee9BRhN2jpwroPuYWGDnSphFl0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ldjMJ+impcxexn1BTzwXKM0ieYKC+W9D1FQGgpAGZUCFZjnldAnVMlyHPjyd6kNrm ceUwaoE+eHz3jmAd+5aWvSLW3jYxwFVNyhz4QlmLOe2jpgpE5UOhNI1UFJYLY2uDTv 1CvCina8Qp/l4r1TlhDDl9CsjMNgxUDvTR8ifzaIwj3qs1/6ky561iQcv8QTbCfHRL jHIYGO/67OninbV3QcYvCqjkgngQsw2u2QzfB82dHTArMxdSkbQOQ6UxLALx2HruZV JV80HbZQ0+Pt6j7jR/ahsiWXylADEMLCz5UMMWTlwWkuh3UfIm6YRRXTl5juK85zrJ v1da6pUA1ddxA== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers Subject: [PATCH v4 7/8] xor: Remove redundant X86_FEATURE_OSXSAVE check Date: Sat, 15 Aug 2026 11:10:23 -0700 Message-ID: <20260815181024.28924-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X86_FEATURE_AVX implies X86_FEATURE_OSXSAVE already. Reviewed-by: Christoph Hellwig Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/raid/xor/x86/xor_arch.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/raid/xor/x86/xor_arch.h b/lib/raid/xor/x86/xor_arch.h index 99fe85a213c6..991abe3f4bbd 100644 --- a/lib/raid/xor/x86/xor_arch.h +++ b/lib/raid/xor/x86/xor_arch.h @@ -18,8 +18,7 @@ extern struct xor_block_template xor_block_avx; */ static __always_inline void __init arch_xor_init(void) { - if (boot_cpu_has(X86_FEATURE_AVX) && - boot_cpu_has(X86_FEATURE_OSXSAVE)) { + if (boot_cpu_has(X86_FEATURE_AVX)) { xor_force(&xor_block_avx); } else if (IS_ENABLED(CONFIG_X86_64) || boot_cpu_has(X86_FEATURE_XMM)) { xor_register(&xor_block_sse); --=20 2.55.0 From nobody Mon Sep 28 23:11:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F14842A15A; Sat, 15 Aug 2026 18:12:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817560; cv=none; b=AY7AUU02ghTk6T8DP8PtSrMN4gW4faNoJ0a+DYF8BXSuPJxIpUfd8zVl+w4Okjbbs7PIgJNvGFp1ohxE1ykwMfvPUkT+ZNXG0/kbSlQQd5xKKFUq6CbJ7WIcTg1B4M23N0SRbTDdx8VaDZOcJIPpZnUSIvHM/OTGLvUgIbUovHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786817560; c=relaxed/simple; bh=iJbcTU68uTqNXKATEqSfJGwN42sXKGD3Ki2fdZOWHD8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s6ewrxrt3V2J0kaz8X5tngwHOofGfDQXie3ECH7HCLAYZjUlbbso36FC/R9YB9tsEg/FPRQQHRRrSLbiaGjidWzn51Y5BWzMVFSLsfB6GmnjKk69UBv90HNifVE6hWYoTSZdXbXhAZAAaVYG5W7kupwU32mu816w7sRrSf53/jg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IXvO2jyq; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IXvO2jyq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 018021F00ACF; Sat, 15 Aug 2026 18:12:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786817557; bh=rdABa6TcIVoYUs+f3mXi/bYXHyjYPbj91ReHiJGt8Nk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=IXvO2jyq3fXC7ctSNsmBcp1/WmuBU2UUXSenUa3Vkouxq5xFNXtkRn0rzUk0HWpV/ +i0DRwowXfQhEblHK9DAFZq5R3IcC1oc3iVD3Aq9YFS4fdw+A6fIbGA/sZ8EWLCjmP 88Amn6G7c2zmAhf/rMT01oj/KMyHmJzQjAMikYCA7augqwta1Cc8tbzIG1My2ZrbRf QzxqUB/l70ERKuJi2QSmJQooEvVEJk7dweQDq4bnIXEqulGuAai6ZVD8KTNBVvabAK xHyAK7tDmU5mNLzcA72PZrU8vuTuVmILTYC+s6OMuVQjiP9mqtLslk4129XZ/nXjOk lYjKqX7eqynqw== From: Eric Biggers To: x86@kernel.org Cc: linux-um@lists.infradead.org, linux-raid@vger.kernel.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Andrew Morton , Eric Biggers , David Laight Subject: [PATCH v4 8/8] xor: Add AVX-512 optimized xor_gen() Date: Sat, 15 Aug 2026 11:10:24 -0700 Message-ID: <20260815181024.28924-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815181024.28924-1-ebiggers@kernel.org> References: <20260815181024.28924-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an implementation of xor_gen() using AVX-512. It uses 512-bit vectors, i.e. ZMM registers. It also uses the vpternlogq instruction to do three-input XORs when applicable. It's enabled on x86_64 CPUs that have AVX512F && !PREFER_YMM. In practice that means: - AMD Zen 4 and later (client and server) - Intel Sapphire Rapids and later (server) - Intel Rocket Lake (client) - Intel Nova Lake and later (client) The !PREFER_YMM condition excludes the older AVX-512 implementations in Intel Skylake Server and Intel Ice Lake. They could run this code, but they're known to have overly-eager downclocking when ZMM registers are used. This is the same policy that the crypto and CRC code uses. Benchmark on AMD Ryzen 9 9950X (Zen 5): src_cnt avx avx512 Improvement =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1 56353 MB/s 75388 MB/s 33% 2 54274 MB/s 68409 MB/s 26% 3 44649 MB/s 64042 MB/s 43% 4 41315 MB/s 55002 MB/s 33% Reviewed-by: David Laight Reviewed-by: Christoph Hellwig Acked-by: Christoph Hellwig Signed-off-by: Eric Biggers --- lib/raid/xor/Makefile | 2 +- lib/raid/xor/x86/xor-avx512.c | 125 ++++++++++++++++++++++++++++++++++ lib/raid/xor/x86/xor_arch.h | 30 +++++--- 3 files changed, 146 insertions(+), 11 deletions(-) create mode 100644 lib/raid/xor/x86/xor-avx512.c diff --git a/lib/raid/xor/Makefile b/lib/raid/xor/Makefile index e8ecec3c09f9..4a0e5c6d8298 100644 --- a/lib/raid/xor/Makefile +++ b/lib/raid/xor/Makefile @@ -29,7 +29,7 @@ xor-$(CONFIG_SPARC32) +=3D sparc/xor-sparc32.o xor-$(CONFIG_SPARC64) +=3D sparc/xor-sparc64.o sparc/xor-sparc64-glue.o xor-$(CONFIG_S390) +=3D s390/xor.o xor-$(CONFIG_X86_32) +=3D x86/xor-avx.o x86/xor-sse.o x86/xor-mmx.o -xor-$(CONFIG_X86_64) +=3D x86/xor-avx.o x86/xor-sse.o +xor-$(CONFIG_X86_64) +=3D x86/xor-avx.o x86/xor-sse.o x86/xor-avx512.o obj-y +=3D tests/ =20 CFLAGS_xor-neon.o +=3D $(CC_FLAGS_FPU) -I$(src)/$(SRCARCH) diff --git a/lib/raid/xor/x86/xor-avx512.c b/lib/raid/xor/x86/xor-avx512.c new file mode 100644 index 000000000000..556ccfea0298 --- /dev/null +++ b/lib/raid/xor/x86/xor-avx512.c @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * AVX-512 optimized implementation of xor_gen() + * + * Copyright 2026 Google LLC + */ + +#include +#include +#include "xor_impl.h" +#include "xor_arch.h" + +/* + * Implementation notes: + * + * Unrolling by the number of buffers (2-5) is very important. + * + * Unrolling by length is less important, especially when using register-i= ndexed + * addressing with negative indices from the end of the buffers. That app= roach + * results in just two loop control instructions being needed per iteratio= n, + * regardless of the number of buffers. + * + * In fact, benchmarks showed that the 2 and 3 buffer cases require only 2x + * unrolling by length, while the 4 and 5 buffer cases don't require any + * unrolling by length. Benchmarks also showed that the register-indexed + * addressing isn't a bottleneck either; i.e., we can't do any better by + * incrementing the pointers as we go along, even with more unrolling. + */ + +static void xor_avx512_2(long bytes, u8 *p1, const u8 *p2) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 64(%1,%0), %%zmm1\n" + "vpxorq (%2,%0), %%zmm0, %%zmm0\n" + "vpxorq 64(%2,%0), %%zmm1, %%zmm1\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "vmovdqa64 %%zmm1, 64(%1,%0)\n" + "add $128, %0\n" + "jnz 1b\n" + "vzeroupper\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes) + : "memory", "cc"); +} + +static void xor_avx512_3(long bytes, u8 *p1, const u8 *p2, const u8 *p3) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 64(%1,%0), %%zmm1\n" + "vmovdqa64 (%2,%0), %%zmm2\n" + "vmovdqa64 64(%2,%0), %%zmm3\n" + "vpternlogq $0x96, (%3,%0), %%zmm2, %%zmm0\n" + "vpternlogq $0x96, 64(%3,%0), %%zmm3, %%zmm1\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "vmovdqa64 %%zmm1, 64(%1,%0)\n" + "add $128, %0\n" + "jnz 1b\n" + "vzeroupper\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes) + : "memory", "cc"); +} + +static void xor_avx512_4(long bytes, u8 *p1, const u8 *p2, const u8 *p3, + const u8 *p4) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 (%2,%0), %%zmm1\n" + "vpxorq (%3,%0), %%zmm0, %%zmm0\n" + "vpternlogq $0x96, (%4,%0), %%zmm1, %%zmm0\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "add $64, %0\n" + "jnz 1b\n" + "vzeroupper\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes), + "r"(p4 + bytes) + : "memory", "cc"); +} + +static void xor_avx512_5(long bytes, u8 *p1, const u8 *p2, const u8 *p3, + const u8 *p4, const u8 *p5) +{ + long i =3D -bytes; + + asm volatile("1: vmovdqa64 (%1,%0), %%zmm0\n" + "vmovdqa64 (%2,%0), %%zmm1\n" + "vpternlogq $0x96, (%3,%0), %%zmm1, %%zmm0\n" + "vmovdqa64 (%4,%0), %%zmm1\n" + "vpternlogq $0x96, (%5,%0), %%zmm1, %%zmm0\n" + "vmovdqa64 %%zmm0, (%1,%0)\n" + "add $64, %0\n" + "jnz 1b\n" + "vzeroupper\n" + : "+&r"(i) + : "r"(p1 + bytes), "r"(p2 + bytes), "r"(p3 + bytes), + "r"(p4 + bytes), "r"(p5 + bytes) + : "memory", "cc"); +} + +DO_XOR_BLOCKS(avx512_inner, xor_avx512_2, xor_avx512_3, xor_avx512_4, + xor_avx512_5); + +/* + * Preconditions: bytes is a nonzero multiple of 512, and all buffers are + * 64-byte aligned. + */ +static void xor_gen_avx512(void *dest, void **srcs, unsigned int src_cnt, + unsigned int bytes) +{ + kernel_fpu_begin(); + xor_gen_avx512_inner(dest, srcs, src_cnt, bytes); + kernel_fpu_end(); +} + +struct xor_block_template xor_block_avx512 =3D { + .name =3D "avx512", + .xor_gen =3D xor_gen_avx512, +}; diff --git a/lib/raid/xor/x86/xor_arch.h b/lib/raid/xor/x86/xor_arch.h index 991abe3f4bbd..ed5921d2e2aa 100644 --- a/lib/raid/xor/x86/xor_arch.h +++ b/lib/raid/xor/x86/xor_arch.h @@ -6,21 +6,31 @@ extern struct xor_block_template xor_block_p5_mmx; extern struct xor_block_template xor_block_sse; extern struct xor_block_template xor_block_sse_pf64; extern struct xor_block_template xor_block_avx; +extern struct xor_block_template xor_block_avx512; =20 -/* - * When SSE is available, use it as it can write around L2. We may also b= e able - * to load into the L1 only depending on how the cpu deals with a load to = a line - * that is being prefetched. - * - * When AVX2 is available, force using it as it is better by all measures. - * - * 32-bit without MMX can fall back to the generic routines. - */ static __always_inline void __init arch_xor_init(void) { - if (boot_cpu_has(X86_FEATURE_AVX)) { + if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_AVX512F) && + !boot_cpu_has(X86_FEATURE_PREFER_YMM)) { + /* + * Use the AVX-512 code on CPUs that support AVX-512 without + * overly-eager downclocking. On such CPUs the AVX-512 code + * should always work at least as well as the AVX code, so + * runtime selection is unnecessary. + * + * The AVX-512 code can work on X86_32. However, due to lack of + * use case for that, for now it's built only for X86_64. + */ + xor_force(&xor_block_avx512); + } else if (boot_cpu_has(X86_FEATURE_AVX)) { + /* AVX will be the best; no need to try others. */ xor_force(&xor_block_avx); } else if (IS_ENABLED(CONFIG_X86_64) || boot_cpu_has(X86_FEATURE_XMM)) { + /* + * When SSE is available, use it as it can write around L2. We + * may also be able to load into the L1 only depending on how + * the cpu deals with a load to a line that is being prefetched. + */ xor_register(&xor_block_sse); xor_register(&xor_block_sse_pf64); } else if (boot_cpu_has(X86_FEATURE_MMX)) { --=20 2.55.0