From nobody Mon Sep 28 23:11:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9747630B529; Sat, 15 Aug 2026 20:58:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827535; cv=none; b=XyXJ0cf09PPPSww6K3aoR+PWyRy1yaM8Pq1OJ7P8A4n5SKVBdwYkQLUWWBgd9mxm3/kxTplyK8ayRF7IFMyDgGZhjx4Of6WMDx7+ofLB5gY/VFEw1D802W7qfDUysafaPavygIJvfedSjP40oM/0U0BAdNBwi8Um4jQIpziUucg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827535; c=relaxed/simple; bh=qpE0SJBd/BU2HephsAzlP5g74fPECM3Mk5F0nbBZYgc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WIf1fMFdYuSmllpw/m9fwhjX/D8XsJvBRZMgxans7UKHxAI/l7WoReKnvSplj+/eczLcemuLnNjYCPyoNDmH0ktVRwDv8ZIEvLdowV8LZfyuWrHr0qNgV9+GoDbY9feGBtosVrZHg4jQ8V/dLTnEJlavbpi5uBLLSZDQnzdC4RE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Culfq8xO; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Culfq8xO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 050B91F00A3F; Sat, 15 Aug 2026 20:58:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786827534; bh=G6Tmug8/NbG6q5O/f3xJp199ph8j+UrYbX3xYD3RUzM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Culfq8xOoK2pwMt/1SYnVBsOSSpWx1gAu02eFHt78VXxFbmX8zd2okqGN+YEUAbrE kiiFe1WbixwJ1TVDYmVuB4JbnoJ7lbUEs+347MUJY083IeoppCsyZtyxu+FhmS/+R1 l2ob47keBcDWt+qTxrcqVZNhX4neHR6kxEXYdPrVuAAPjt/e+MoCo1yxNFo6s8xR+Q IoCGX343o+gXUo2eRH2WOQgWLL3Omq5/PQs3LPzbNowZZFezlH3w9X4Ip0ApUX/TAw wMT6r6yc50EOl8q9RUeIpOAXqPvO8bzLP9IbMG2kBUua6RqRv76yTgcMjZm1ovNTbw gp1l8az9gJkaA== From: Eric Biggers To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, Christoph Hellwig , linux-crypto@vger.kernel.org, Herbert Xu , Taehee Yoo , netfilter-devel@vger.kernel.org, Pablo Neira Ayuso , Florian Westphal , Phil Sutter , Eric Biggers , stable@vger.kernel.org Subject: [PATCH 1/6] xor: add missing vzeroupper to AVX code Date: Sat, 15 Aug 2026 13:57:45 -0700 Message-ID: <20260815205750.169336-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815205750.169336-1-ebiggers@kernel.org> References: <20260815205750.169336-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since the AVX optimized XOR code uses YMM registers, execute vzeroupper before returning from it. This is needed to avoid degrading the performance of any later SSE code that may happen to be executed. Fixes: ea4d26ae24e5 ("raid5: add AVX optimized RAID5 checksumming") Cc: stable@vger.kernel.org Signed-off-by: Eric Biggers --- lib/raid/xor/x86/xor-avx.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/raid/xor/x86/xor-avx.c b/lib/raid/xor/x86/xor-avx.c index f7777d7aa269..95b21e7225e8 100644 --- a/lib/raid/xor/x86/xor-avx.c +++ b/lib/raid/xor/x86/xor-avx.c @@ -147,6 +147,7 @@ static void xor_gen_avx(void *dest, void **srcs, unsign= ed int src_cnt, { kernel_fpu_begin(); xor_gen_avx_inner(dest, srcs, src_cnt, bytes); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 --=20 2.55.0 From nobody Mon Sep 28 23:11:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50B753914EB; Sat, 15 Aug 2026 20:58:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827536; cv=none; b=mUL+04vRRQvGK8bh5D1iySgJPwi1zUxSmCPpajDDdchffLGQU1U9L6V+O8l4V12CtSHWJPJkx+HAMRyqDWiS/jHlbP0TDlMjuWL1oZewZXZqYScwqE08xrZodJqW47gAeaL1TnGn1bJmh3n8oK5eqcsxvs9qzGXxUHqTgyBvQg4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827536; c=relaxed/simple; bh=VUIhCVP18vEs3dKQxig8pMvpDlmYaXiKd+6TmPFTx5A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fiSn1VlGfL//9vg9gzCmQu/9JwIShyVufMwKORbtbRfxrS1T06xGIpEZVvEwfVQsvGM7yd7wCiPoJLDm/8ZeasRxdUmcKz8nJp//N1A9epBWXrzFQGiOYlO4/hSntVGardepEpYsJv9IYn3FCn3ZSvVMxx6jkxwdowucAnPWIO0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=naeOnKRn; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="naeOnKRn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88C891F00AC4; Sat, 15 Aug 2026 20:58:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786827534; bh=krv8XEnhnCImcLWFDlxt4ypVEiMKwgztSgH5l8u1fGg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=naeOnKRnreRJ/F97wOlwAOMe7gIEE5zMTVOoLiD6/mTf6QvpHiObFeKQOnylJY/ij ZaZh3V0zH3qf6y3FRGhzgC4KRhtmfh0x1FH9eNwOWt8fk0ETcTLeOzfCXmqSqA6pNl aYxqj+371WUJf1ZMFsKeOZhZTgyTimJUiKAezWY+gzgGJZWuEufS+HjXPJAw03SwG2 mp9wkTfu8Sj71OAXQgkE67vLAQHxGYHogZejlu2tYJwELNC4ps+9qS0hQqNjCJFf74 QgEjqG6VW8LWy/iojhXG7SkH8W6taX4VHuoODTRdo/yjYMyYmpbGAUzoeYKOWqfPxK /rRgwgOhCIJsg== From: Eric Biggers To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, Christoph Hellwig , linux-crypto@vger.kernel.org, Herbert Xu , Taehee Yoo , netfilter-devel@vger.kernel.org, Pablo Neira Ayuso , Florian Westphal , Phil Sutter , Eric Biggers , stable@vger.kernel.org Subject: [PATCH 2/6] raid6: add missing vzeroupper to AVX2 code Date: Sat, 15 Aug 2026 13:57:46 -0700 Message-ID: <20260815205750.169336-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815205750.169336-1-ebiggers@kernel.org> References: <20260815205750.169336-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since the AVX2 optimized RAID6 code uses YMM registers, execute vzeroupper before returning from it. This is needed to avoid degrading the performance of any later SSE code that may happen to be executed. Fixes: 2c935842bdb4 ("lib/raid6: Add AVX2 optimized gen_syndrome functions") Fixes: 7056741fd9fc ("lib/raid6: Add AVX2 optimized recovery functions") Cc: stable@vger.kernel.org Signed-off-by: Eric Biggers --- lib/raid/raid6/x86/avx2.c | 6 ++++++ lib/raid/raid6/x86/recov_avx2.c | 2 ++ 2 files changed, 8 insertions(+) diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c index 7d829c669ea7..3cc2fe7ac42c 100644 --- a/lib/raid/raid6/x86/avx2.c +++ b/lib/raid/raid6/x86/avx2.c @@ -67,6 +67,7 @@ static void raid6_avx21_gen_syndrome(int disks, size_t by= tes, void **ptrs) } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -115,6 +116,7 @@ static void raid6_avx21_xor_syndrome(int disks, int sta= rt, int stop, } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -175,6 +177,7 @@ static void raid6_avx22_gen_syndrome(int disks, size_t = bytes, void **ptrs) } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -243,6 +246,7 @@ static void raid6_avx22_xor_syndrome(int disks, int sta= rt, int stop, } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -334,6 +338,7 @@ static void raid6_avx24_gen_syndrome(int disks, size_t = bytes, void **ptrs) } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -444,6 +449,7 @@ static void raid6_avx24_xor_syndrome(int disks, int sta= rt, int stop, asm volatile("vmovntdq %%ymm14,%0" : "=3Dm" (q[d+96])); } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx= 2.c index a714a780a2d8..820871046e30 100644 --- a/lib/raid/raid6/x86/recov_avx2.c +++ b/lib/raid/raid6/x86/recov_avx2.c @@ -176,6 +176,7 @@ static void raid6_2data_recov_avx2(int disks, size_t by= tes, int faila, #endif } =20 + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -293,6 +294,7 @@ static void raid6_datap_recov_avx2(int disks, size_t by= tes, int faila, #endif } =20 + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 --=20 2.55.0 From nobody Mon Sep 28 23:11:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 908C0391E59; Sat, 15 Aug 2026 20:58:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827536; cv=none; b=r7rSE4RIVvOtiUJTn6EYaneVLW1HFv21FbOpQFD20dJNs9SuHRL/ZBsT/VsYxd9tj/Um/zgLrR/uxitkGDqP5hXb9X3pynpzOJ8MZGhE1xcgfTjbWTfLekK7yt80el62AjRbhL9zceF7dnMNdr0i1kgDGil54kD3hrtJ+82KUYc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827536; c=relaxed/simple; bh=ljt8f4deHjWMit+6Ii+uv5Z/qUGaSw8AyqGIKX10yqg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rKn4RilKBWcvjSQuCoR/EjgEX2lkSvH/u9ZeSimJJnp9q/6DeESZN6EZwTbS/rCG8cxsTS5FptBLcg2dMi1/SKy4TjvmDYyj7AxhFwDyD+D6bA3oulbYLLCqzLf2VjawVpKNOd5xZ8Rb6iZkha5Dgtk/oZw9O1SlG+GKhDI2KyI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RMalAsAi; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RMalAsAi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 160D71F000E9; Sat, 15 Aug 2026 20:58:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786827535; bh=diYr8gjmYYuc2ZRtM8EL+2DHiYb4p3SsFY0RRdUlGQY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=RMalAsAiFq8PRQUUdBTVEqBbqDbFz+bqVWRTuzA3D8jJ3Og08aSPya2MvP9PtTayR 9Km1CQverFNO9TMs2vjfwqTUbnIvOf8oFVb6ml1X/maPM+Noa0ye/k890IhqmC5tU9 R53fqcDOjP+pVzee+FhVjMJ2RK+dxwD5X1ShG2HdYIUwzHbAPU/czXTtT6N3aQnOC/ H3P+mq7eVuvA5CzNlyGfuaLOZCUtJi3bCgBK9uCgsNBKngxS0pMKaX0iNKlSM30CR5 WZPIVZDUxWX+zfthKTVeWmw3hXWwN3Lpn1LTNKp5sfpguj9Ef0JDVgSN/1jecMf8d+ 9OCt0e5pV2jGQ== From: Eric Biggers To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, Christoph Hellwig , linux-crypto@vger.kernel.org, Herbert Xu , Taehee Yoo , netfilter-devel@vger.kernel.org, Pablo Neira Ayuso , Florian Westphal , Phil Sutter , Eric Biggers , stable@vger.kernel.org Subject: [PATCH 3/6] raid6: add missing vzeroupper to AVX-512 code Date: Sat, 15 Aug 2026 13:57:47 -0700 Message-ID: <20260815205750.169336-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815205750.169336-1-ebiggers@kernel.org> References: <20260815205750.169336-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since the AVX-512 optimized RAID6 code uses ZMM registers, execute vzeroupper before returning from it. This is needed to avoid degrading the performance of any later SSE code that may happen to be executed. Fixes: e0a491c12968 ("lib/raid6: Add AVX512 optimized gen_syndrome function= s") Fixes: 13c520b2993c ("lib/raid6: Add AVX512 optimized recovery functions") Cc: stable@vger.kernel.org Signed-off-by: Eric Biggers --- lib/raid/raid6/x86/avx512.c | 6 ++++++ lib/raid/raid6/x86/recov_avx512.c | 2 ++ 2 files changed, 8 insertions(+) diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c index e671eb5bde63..772bfc4af6df 100644 --- a/lib/raid/raid6/x86/avx512.c +++ b/lib/raid/raid6/x86/avx512.c @@ -78,6 +78,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t = bytes, void **ptrs) } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -137,6 +138,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int s= tart, int stop, } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -208,6 +210,7 @@ static void raid6_avx5122_gen_syndrome(int disks, size_= t bytes, void **ptrs) } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -292,6 +295,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int s= tart, int stop, } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -396,6 +400,7 @@ static void raid6_avx5124_gen_syndrome(int disks, size_= t bytes, void **ptrs) } =20 asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -529,6 +534,7 @@ static void raid6_avx5124_xor_syndrome(int disks, int s= tart, int stop, "m" (q[d+128]), "m" (q[d+192])); } asm volatile("sfence" : : : "memory"); + asm volatile("vzeroupper"); kernel_fpu_end(); } const struct raid6_calls raid6_avx512x4 =3D { diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_a= vx512.c index ec72d5a30c01..299a3f044d61 100644 --- a/lib/raid/raid6/x86/recov_avx512.c +++ b/lib/raid/raid6/x86/recov_avx512.c @@ -211,6 +211,7 @@ static void raid6_2data_recov_avx512(int disks, size_t = bytes, int faila, #endif } =20 + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 @@ -353,6 +354,7 @@ static void raid6_datap_recov_avx512(int disks, size_t = bytes, int faila, #endif } =20 + asm volatile("vzeroupper"); kernel_fpu_end(); } =20 --=20 2.55.0 From nobody Mon Sep 28 23:11:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7B2A396567; Sat, 15 Aug 2026 20:58:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827538; cv=none; b=hxxpsC4Bc+e6HTRn7oTkeU/5svcDUltPik3uOg/uGciipIuw8mikMtnf0GDWISPBPF6QjcZLj50HNQ1tZCxyndDPWdvNPIF8VZfUQAKLmEljr7a2qThG16G5qdIpCULDajE5dUdj9Ew/uW2EqDTYuNkCrpe+Rb0fVeZGsiu4Kt4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827538; c=relaxed/simple; bh=Xg21G8QruOBVYSNsqWHlCp2IipPB/eMbg9nfvIbTqj8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Kz+1+wVf68vd1HZV2otlqpjbTDQMyDllUM/N2/8dx32ix6ZUofk5RkAdW8QjeV2e/T7PqwsJcA/Jd9vH66io6xVvjDqgWgngzjO3XN91SrkARQnbUgYa3LNQV2nKymRWGtnvM3UrTIZKIWiEVnr/TqlE0+qs/Lxq/uTJkl2edVU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DjMOqZ6y; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DjMOqZ6y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 999B51F00A3A; Sat, 15 Aug 2026 20:58:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786827536; bh=XLAdi1TlBOtXW8xgnYx5FHT2dAljIrxrUwmqj950s9Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=DjMOqZ6y4xraZj1WFVvM//1QGmseYfZBjzOqpq5VvLQE3k3BYMwPnv3k2qY5Yb7w6 2foHHdpQ7Wd7ogF7B9M+MRrDU6p2IrOl5dU+Cyddp7J99XkPPF8soi1q+WR6ha+JoX sVVNvXTwHlSh0OWbeZSrk3syGBLwnKvvAAYT95BKDcewwJo4QMGyGp++ofwXmphBeV wWVkwSf0UkM/kmMI72FK8BmmW0RS/VY4jwObIdohXVBMTKOmcJWP18Qi+NagE9Dgv2 4DuCPhnTsouqmZ24oeBjNUk1/wimBivtgwpgrNx2yXaC6TtzYvE08KMuAV9glbV4k3 RmYiH2ZBI87IQ== From: Eric Biggers To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, Christoph Hellwig , linux-crypto@vger.kernel.org, Herbert Xu , Taehee Yoo , netfilter-devel@vger.kernel.org, Pablo Neira Ayuso , Florian Westphal , Phil Sutter , Eric Biggers , stable@vger.kernel.org Subject: [PATCH 4/6] crypto: x86/aria - add missing vzeroupper in AVX2 code Date: Sat, 15 Aug 2026 13:57:48 -0700 Message-ID: <20260815205750.169336-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815205750.169336-1-ebiggers@kernel.org> References: <20260815205750.169336-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since the AVX2 optimized ARIA code uses YMM registers, execute vzeroupper before returning from it. This is needed to avoid degrading the performance of any later SSE code that may happen to be executed. Fixes: 37d8d3ae7a58 ("crypto: x86/aria - implement aria-avx2") Cc: stable@vger.kernel.org Cc: Taehee Yoo Signed-off-by: Eric Biggers --- arch/x86/crypto/aria-aesni-avx2-asm_64.S | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/crypto/aria-aesni-avx2-asm_64.S b/arch/x86/crypto/ari= a-aesni-avx2-asm_64.S index ed53d4f46bd7..fda8cb8a99a8 100644 --- a/arch/x86/crypto/aria-aesni-avx2-asm_64.S +++ b/arch/x86/crypto/aria-aesni-avx2-asm_64.S @@ -982,6 +982,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_encrypt_32way) %ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14, %ymm15, %rax); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_aesni_avx2_encrypt_32way) @@ -1007,6 +1008,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_decrypt_32way) %ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14, %ymm15, %rax); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_aesni_avx2_decrypt_32way) @@ -1209,6 +1211,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_ctr_crypt_32way) %ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14, %ymm15, %r10); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_aesni_avx2_ctr_crypt_32way) @@ -1359,6 +1362,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_encrypt_32w= ay) %ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14, %ymm15, %rax); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_aesni_avx2_gfni_encrypt_32way) @@ -1384,6 +1388,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_decrypt_32w= ay) %ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14, %ymm15, %rax); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_aesni_avx2_gfni_decrypt_32way) @@ -1428,6 +1433,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_ctr_crypt_3= 2way) %ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14, %ymm15, %r10); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_aesni_avx2_gfni_ctr_crypt_32way) --=20 2.55.0 From nobody Mon Sep 28 23:11:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E0993955FC; Sat, 15 Aug 2026 20:58:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827537; cv=none; b=O6ERPQM+FcTsUh9krC5AK0HVCzZ6uhazPq5aBBzLGg5nGkRPgq3wsmQgQ6sRulkFbM203pP0Qk9z1Lh8wM+ge6RKAovFhTkQ2pCzprev87/HIUmjwIcgx8kOMxYteE+lz6Bi82Je8KwJZrPeIsfrHlbh86ou2ISPAkjjkJobhsI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827537; c=relaxed/simple; bh=sDy2xxxUKeTH/KVTJQGSDOxgQWZz4evu7pBOGLdIkDQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=b50/oM/NeGIorI/KdypK/khwmBh37V037GdgcqlyaFUckvYWQANPdijVASLYuj7y+8ILGqXb2cNoZLia+D8ZRSEylqCaLP+3xyFJXw3KBzbt05GyYrFbIh7Xmy/BRKAuZepk+B39CaYZXV0QC2IwVBa84N79Lr0PlCdng/1rf20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Vp5QXjN8; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Vp5QXjN8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26EC61F00A3D; Sat, 15 Aug 2026 20:58:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786827536; bh=MfTGJZLmd6D32z+SzOilmhIPRG29fcdYhDG96KG74Q8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Vp5QXjN80jMRxJMI+Ue2SO3fyaKSJiMf2Q1cZPiyJSLJmvHbYr6ySJh7y/E2egq7z O1uYUE84KgCQC0Y0GOzmzNkxJIc4MRmxtB0r993BfE/PfVpuyvoHGYxx0O5MpPbDTe BA5vfzXuerJLYvC93UN3t3VKrayt8Uf5kX9qNnUhEnPA2tnSQ+ChLAlQmlYR5tPmhC 3IMApT//3K+6eBGv2HNe1OOBrvjGgJ9sTwNZiPD8AMr5lT2zAf+sb7MVOHIewEbfRC gDmU4qZ8d6UqqDWyXiSi2ftrlLxTqupOhCvrakqN9gzWXFbJqefXVuZrj4DM/PWoqO S9tYyby+G5qBA== From: Eric Biggers To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, Christoph Hellwig , linux-crypto@vger.kernel.org, Herbert Xu , Taehee Yoo , netfilter-devel@vger.kernel.org, Pablo Neira Ayuso , Florian Westphal , Phil Sutter , Eric Biggers , stable@vger.kernel.org Subject: [PATCH 5/6] crypto: x86/aria - add missing vzeroupper in AVX-512 code Date: Sat, 15 Aug 2026 13:57:49 -0700 Message-ID: <20260815205750.169336-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815205750.169336-1-ebiggers@kernel.org> References: <20260815205750.169336-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since the AVX-512 optimized ARIA code uses ZMM registers, execute vzeroupper before returning from it. This is needed to avoid degrading the performance of any later SSE code that may happen to be executed. Fixes: c970d42001f2 ("crypto: x86/aria - implement aria-avx512") Cc: stable@vger.kernel.org Cc: Taehee Yoo Signed-off-by: Eric Biggers --- arch/x86/crypto/aria-gfni-avx512-asm_64.S | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/crypto/aria-gfni-avx512-asm_64.S b/arch/x86/crypto/ar= ia-gfni-avx512-asm_64.S index 860887e5d02e..ca83eb126e06 100644 --- a/arch/x86/crypto/aria-gfni-avx512-asm_64.S +++ b/arch/x86/crypto/aria-gfni-avx512-asm_64.S @@ -800,6 +800,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_encrypt_64way) %zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14, %zmm15, %rax); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_gfni_avx512_encrypt_64way) @@ -825,6 +826,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_decrypt_64way) %zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14, %zmm15, %rax); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_gfni_avx512_decrypt_64way) @@ -966,6 +968,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_ctr_crypt_64way) %zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14, %zmm15, %r10); =20 + vzeroupper; FRAME_END RET; SYM_FUNC_END(aria_gfni_avx512_ctr_crypt_64way) --=20 2.55.0 From nobody Mon Sep 28 23:11:42 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D3E139A7F0; Sat, 15 Aug 2026 20:58:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827538; cv=none; b=gEi8te8BB9iI35CO9Kov/GBlr+l+ZaQ+b+R+RJD7RWlKEBZwY0B5EIi4tvSbBY+rwNwd0WnWKk5G5ChUDiQkzWDfkJCkjYNXdes4gdU7zQKWCBALs0AFPJox01K40UGf+vEC8c6BvTqdJpnFcmcU64G3FWgb15teKyQYraTXqho= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786827538; c=relaxed/simple; bh=daENMNWzFwnv/SLyg/LYZ2VdNdrGuELhVjAey20m5cE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AhYvViVphIBMvxqJAQQOMWXRJpfRavIMn8k7XL6KxVT2OKj0Q9F2GwOohNVOELtStn2vdBLJjlbHc7gTz2OV7xaN4wWosfItpzR2LK2eDwoAROPPs/xrtV7yresue21dLF/Vjx85afr0lOyVDU92UdRUHZskb2wEYt4GR8Pncxg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DsWFY2RN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DsWFY2RN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A920B1F00A3E; Sat, 15 Aug 2026 20:58:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786827537; bh=Mli7QknwTOhcn8hn340G0Jnmp+/Q9en83EQ+szYFyZc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=DsWFY2RNQPLVepwFnlzgBVrVP6iOxX4bgYlYgNC4hZXEDd5qZBdOZtC59jFoWev+H 3ihoSLKj7XFAYqyhreN/kWK98aYnf4mQ447IpXPGRkWY8L8kOjIPKUFqw1No9DXAed j5UfPNq3UIm3mW5uH/slW9dglQbjHq5VUVAxy3m7iBczhc6zjt5pLMQUpMaB46Rvvx Q/0qT4WEhNStFGF4Ib3xE9rG0RvtxftGIHJ7cGh6+w83hlgVy8j4/t6GwEV0LR9NpF /azdfhfpeHwSN0CMjWRcevqrBkZ2zeOj5yucqQpAVuBZjcvXT5mtLDiTo7QlXz4Suj wcmabiP2TYjKQ== From: Eric Biggers To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, Christoph Hellwig , linux-crypto@vger.kernel.org, Herbert Xu , Taehee Yoo , netfilter-devel@vger.kernel.org, Pablo Neira Ayuso , Florian Westphal , Phil Sutter , Eric Biggers , stable@vger.kernel.org, Stefano Brivio Subject: [PATCH 6/6] netfilter: nft_set_pipapo_avx2: add missing vzeroupper Date: Sat, 15 Aug 2026 13:57:50 -0700 Message-ID: <20260815205750.169336-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260815205750.169336-1-ebiggers@kernel.org> References: <20260815205750.169336-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since pipapo_get_avx2() uses YMM registers, execute vzeroupper before returning from it. This is needed to avoid degrading the performance of any later SSE code that may happen to be executed. Fixes: 7400b063969b ("nft_set_pipapo: Introduce AVX2-based lookup implement= ation") Cc: stable@vger.kernel.org Cc: Florian Westphal Cc: Stefano Brivio Signed-off-by: Eric Biggers Reviewed-by: Stefano Brivio --- net/netfilter/nft_set_pipapo_avx2.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/net/netfilter/nft_set_pipapo_avx2.c b/net/netfilter/nft_set_pi= papo_avx2.c index b3f105520a85..21f5be68c703 100644 --- a/net/netfilter/nft_set_pipapo_avx2.c +++ b/net/netfilter/nft_set_pipapo_avx2.c @@ -1134,6 +1134,7 @@ struct nft_pipapo_elem *pipapo_get_avx2(const struct = nft_pipapo_match *m, struct nft_pipapo_scratch *scratch; const struct nft_pipapo_field *f; unsigned long *res, *fill, *map; + struct nft_pipapo_elem *e; bool map_index; int ret =3D 0; int i; @@ -1207,14 +1208,11 @@ struct nft_pipapo_elem *pipapo_get_avx2(const struc= t nft_pipapo_match *m, next_match: if (ret < 0) { scratch->map_index =3D map_index; - kernel_fpu_end(); - __local_unlock_nested_bh(&scratch->bh_lock); - return NULL; + e =3D NULL; + goto out; } =20 if (last) { - struct nft_pipapo_elem *e; - e =3D f->mt[ret].e; if (unlikely(__nft_set_elem_expired(&e->ext, tstamp) || !nft_set_elem_active(&e->ext, genmask))) { @@ -1224,9 +1222,7 @@ struct nft_pipapo_elem *pipapo_get_avx2(const struct = nft_pipapo_match *m, } =20 scratch->map_index =3D map_index; - kernel_fpu_end(); - __local_unlock_nested_bh(&scratch->bh_lock); - return e; + goto out; } =20 map_index =3D !map_index; @@ -1234,9 +1230,12 @@ struct nft_pipapo_elem *pipapo_get_avx2(const struct= nft_pipapo_match *m, data +=3D NFT_PIPAPO_GROUPS_PADDED_SIZE(f); } =20 + e =3D NULL; +out: + asm volatile("vzeroupper"); kernel_fpu_end(); __local_unlock_nested_bh(&scratch->bh_lock); - return NULL; + return e; } =20 /** --=20 2.55.0