From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B98332749CB; Sun, 9 Nov 2025 23:49:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732197; cv=none; b=i8KwpVtsMgi9wKh93GnUBru4vjlbaRjomR1yuFxJZZpk2EAKzpkuZXWWH8AxxoLt+9fCKSYZwE2tFW++1qxt2heiO0R7k7Pi2q07uyTHSXfL3ZyL6tzAW34LdvMb3ASiyYMU6Hm6dIjm01xDo12KCnuXwAhRmVAh2VyAT+B7EWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732197; c=relaxed/simple; bh=Wc6TlqQavzVYtanyrw+qZw+ADFSh9UnmNvIiDQ/B1sA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VofbYy/dGL0XiLdig9imcmlS30zO7gZKIjkF1sHcx+KVbAUa66eaIz7dQN8S1QjlzprHxTYRTLJip41t9xbpEyCrV9wo9hKjyEnYlLYqzoeOhio3rDUAtXUmdB9dNOGz1Z4K2nf10qEx95oJDC2f7Xsm0l9RO8d8FXK5nrDITqc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=f9pErZCU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="f9pErZCU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2C7ABC4CEF7; Sun, 9 Nov 2025 23:49:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732197; bh=Wc6TlqQavzVYtanyrw+qZw+ADFSh9UnmNvIiDQ/B1sA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f9pErZCU3vr/wPmWxMCjMcN5UIoJJKvaV4aok4iQLPiHYK09ywK3sqT/5GlnW6E40 zrZ+7Fg5cUYumJ8487nV1zKnE6xRVhI/psbr9H6pjwnLqNBc9noTgL1PkAkpfnHC8Y JuiG7finQtE2EX3QZkvdWZ30PevIqk5tN/qgbF1AZjanbnHkmacbtTq5uL5Mhou3U9 8xT1Jqn2ghdqHqjg4f2Ixe0Ulj16OxOBh0duYzhgDN2zjiuqs6ymNDpGQl83BOQ3Zx aOKptcsseS9CGiO2SjCjfniZG29xRshs105B3Wayq2R+J08JXFkvgrtKov19VTKBtu 8F/5RXLEuKakA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 1/9] crypto: polyval - Rename conflicting functions Date: Sun, 9 Nov 2025 15:47:16 -0800 Message-ID: <20251109234726.638437-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename polyval_init() and polyval_update(), in preparation for adding library functions with the same name to . Note that polyval-generic.c will be removed later, as it will be superseded by the library. This commit just keeps the kernel building for the initial introduction of the library. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- crypto/polyval-generic.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/crypto/polyval-generic.c b/crypto/polyval-generic.c index db8adb56e4ca..fe5b01a4000d 100644 --- a/crypto/polyval-generic.c +++ b/crypto/polyval-generic.c @@ -97,21 +97,21 @@ static int polyval_setkey(struct crypto_shash *tfm, return -ENOMEM; =20 return 0; } =20 -static int polyval_init(struct shash_desc *desc) +static int polyval_generic_init(struct shash_desc *desc) { struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); =20 memset(dctx, 0, sizeof(*dctx)); =20 return 0; } =20 -static int polyval_update(struct shash_desc *desc, - const u8 *src, unsigned int srclen) +static int polyval_generic_update(struct shash_desc *desc, + const u8 *src, unsigned int srclen) { struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); const struct polyval_tfm_ctx *ctx =3D crypto_shash_ctx(desc->tfm); u8 tmp[POLYVAL_BLOCK_SIZE]; =20 @@ -133,11 +133,11 @@ static int polyval_finup(struct shash_desc *desc, con= st u8 *src, =20 if (len) { u8 tmp[POLYVAL_BLOCK_SIZE] =3D {}; =20 memcpy(tmp, src, len); - polyval_update(desc, tmp, POLYVAL_BLOCK_SIZE); + polyval_generic_update(desc, tmp, POLYVAL_BLOCK_SIZE); } copy_and_reverse(dst, dctx->buffer); return 0; } =20 @@ -164,12 +164,12 @@ static void polyval_exit_tfm(struct crypto_shash *tfm) gf128mul_free_4k(ctx->gf128); } =20 static struct shash_alg polyval_alg =3D { .digestsize =3D POLYVAL_DIGEST_SIZE, - .init =3D polyval_init, - .update =3D polyval_update, + .init =3D polyval_generic_init, + .update =3D polyval_generic_update, .finup =3D polyval_finup, .setkey =3D polyval_setkey, .export =3D polyval_export, .import =3D polyval_import, .exit_tfm =3D polyval_exit_tfm, --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22146274B35; Sun, 9 Nov 2025 23:49:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732198; cv=none; b=mXi1DkMtzcByvd833nUTizhK75LSKt9z5GmmbpMD2Se/nZIMZuwzc2zmFoYFqVHB3uu0uO7PF70KENmhmN6bsag/wKFK1luUHpxnzx3eqvUlCVN0V1XT3el5x1rv5VXgpnQCAV408XwvSDGdODEA3zLP6MP9nP9tTRfaduO/ytQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732198; c=relaxed/simple; bh=ZoXcGXBGVlSr3Ho0fOCW8R5LkDPylPl0IcgWUO0IZ7Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SItfqCbi5+A+xTYYz6KcHTJForXzt5SUUvOW66rlC4AwG8b+Pq6C6/M+GAqZlqPHgKil2rNUiyR265gfUul8KuMZuBa6PXFT61HsSw/MqyLuKNZfa7ZJRLEZs+0Yl2s7uBfqMEha3JlU2BFxL9nJrtxkbqyQAzgShywfoHt0MT0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ltPtQ5D7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ltPtQ5D7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87E1AC113D0; Sun, 9 Nov 2025 23:49:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732197; bh=ZoXcGXBGVlSr3Ho0fOCW8R5LkDPylPl0IcgWUO0IZ7Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ltPtQ5D7C46OOur1c0zaKE1wwSXXrQWTzc1bWwcLWUlmUafAEN6NoRIoVLIOTAeGX 6bRgBrTbohN3/p2j05DymcyF1vlGi+3VtDK8whLCWHNMvnQGwXdfu6XLKHz1xNxRhN pQzlf6XrG4q34VBUUno6eTqcAE8gAn7Hk1qCL6rqn8IfIyVBsADHDwhK042NgD6e5Y uR8I0M4q/OkfvtsX6gGZZbXoJDYTdxGkmpKZ32Adbn3k8O5ISU8jiyR/YG1RokSwPo OhTBpsZTNgqjsIKSeDJiEzyC8nxJ4yQ7ME1xCiinSpfwofDt4gUNP5NTgGfb47TsN9 cpVwpeIa/Z4EA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 2/9] lib/crypto: polyval: Add POLYVAL library Date: Sun, 9 Nov 2025 15:47:17 -0800 Message-ID: <20251109234726.638437-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add support for POLYVAL to lib/crypto/. This will replace the polyval crypto_shash algorithm and its use in the hctr2 template, simplifying the code and reducing overhead. Specifically, this commit introduces the POLYVAL library API and a generic implementation of it. Later commits will migrate the existing architecture-optimized implementations of POLYVAL into lib/crypto/ and add a KUnit test suite. I've also rewritten the generic implementation completely, using a more modern approach instead of the traditional table-based approach. It's now constant-time, requires no precomputation or dynamic memory allocations, decreases the per-key memory usage from 4096 bytes to 16 bytes, and is faster than the old polyval-generic even on bulk data reusing the same key (at least on x86_64, where I measured 15% faster). We should do this for GHASH too, but for now just do it for POLYVAL. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel Tested-by: Ard Biesheuvel --- include/crypto/polyval.h | 171 +++++++++++++++++++++- lib/crypto/Kconfig | 10 ++ lib/crypto/Makefile | 8 + lib/crypto/polyval.c | 307 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 493 insertions(+), 3 deletions(-) create mode 100644 lib/crypto/polyval.c diff --git a/include/crypto/polyval.h b/include/crypto/polyval.h index d2e63743e592..5ba4c248cad1 100644 --- a/include/crypto/polyval.h +++ b/include/crypto/polyval.h @@ -1,14 +1,179 @@ -/* SPDX-License-Identifier: GPL-2.0 */ +/* SPDX-License-Identifier: GPL-2.0-or-later */ /* - * Common values for the Polyval hash algorithm + * POLYVAL library API * - * Copyright 2021 Google LLC + * Copyright 2025 Google LLC */ =20 #ifndef _CRYPTO_POLYVAL_H #define _CRYPTO_POLYVAL_H =20 +#include +#include + #define POLYVAL_BLOCK_SIZE 16 #define POLYVAL_DIGEST_SIZE 16 =20 +/** + * struct polyval_elem - An element of the POLYVAL finite field + * @bytes: View of the element as a byte array (unioned with @lo and @hi) + * @lo: The low 64 terms of the element's polynomial + * @hi: The high 64 terms of the element's polynomial + * + * This represents an element of the finite field GF(2^128), using the POL= YVAL + * convention: little-endian byte order and natural bit order. + */ +struct polyval_elem { + union { + u8 bytes[POLYVAL_BLOCK_SIZE]; + struct { + __le64 lo; + __le64 hi; + }; + }; +}; + +/** + * struct polyval_key - Prepared key for POLYVAL + * + * This may contain just the raw key H, or it may contain precomputed key + * powers, depending on the platform's POLYVAL implementation. Use + * polyval_preparekey() to initialize this. + */ +struct polyval_key { +#ifdef CONFIG_CRYPTO_LIB_POLYVAL_ARCH +#error "Unhandled arch" +#else /* CONFIG_CRYPTO_LIB_POLYVAL_ARCH */ + /** @h: The hash key H */ + struct polyval_elem h; +#endif /* !CONFIG_CRYPTO_LIB_POLYVAL_ARCH */ +}; + +/** + * struct polyval_ctx - Context for computing a POLYVAL value + * @key: Pointer to the prepared POLYVAL key. The user of the API is + * responsible for ensuring that the key lives as long as the context. + * @acc: The accumulator + * @partial: Number of data bytes processed so far modulo POLYVAL_BLOCK_SI= ZE + */ +struct polyval_ctx { + const struct polyval_key *key; + struct polyval_elem acc; + size_t partial; +}; + +/** + * polyval_preparekey() - Prepare a POLYVAL key + * @key: (output) The key structure to initialize + * @raw_key: The raw hash key + * + * Initialize a POLYVAL key structure from a raw key. This may be a simple + * copy, or it may involve precomputing powers of the key, depending on the + * platform's POLYVAL implementation. + * + * Context: Any context. + */ +#ifdef CONFIG_CRYPTO_LIB_POLYVAL_ARCH +void polyval_preparekey(struct polyval_key *key, + const u8 raw_key[POLYVAL_BLOCK_SIZE]); + +#else +static inline void polyval_preparekey(struct polyval_key *key, + const u8 raw_key[POLYVAL_BLOCK_SIZE]) +{ + /* Just a simple copy, so inline it. */ + memcpy(key->h.bytes, raw_key, POLYVAL_BLOCK_SIZE); +} #endif + +/** + * polyval_init() - Initialize a POLYVAL context for a new message + * @ctx: The context to initialize + * @key: The key to use. Note that a pointer to the key is saved in the + * context, so the key must live at least as long as the context. + */ +static inline void polyval_init(struct polyval_ctx *ctx, + const struct polyval_key *key) +{ + *ctx =3D (struct polyval_ctx){ .key =3D key }; +} + +/** + * polyval_import_blkaligned() - Import a POLYVAL accumulator value + * @ctx: The context to initialize + * @key: The key to import. Note that a pointer to the key is saved in the + * context, so the key must live at least as long as the context. + * @acc: The accumulator value to import. + * + * This imports an accumulator that was saved by polyval_export_blkaligned= (). + * The same key must be used. + */ +static inline void +polyval_import_blkaligned(struct polyval_ctx *ctx, + const struct polyval_key *key, + const struct polyval_elem *acc) +{ + *ctx =3D (struct polyval_ctx){ .key =3D key, .acc =3D *acc }; +} + +/** + * polyval_export_blkaligned() - Export a POLYVAL accumulator value + * @ctx: The context to export the accumulator value from + * @acc: (output) The exported accumulator value + * + * This exports the accumulator from a POLYVAL context. The number of data + * bytes processed so far must be a multiple of POLYVAL_BLOCK_SIZE. + */ +static inline void polyval_export_blkaligned(const struct polyval_ctx *ctx, + struct polyval_elem *acc) +{ + *acc =3D ctx->acc; +} + +/** + * polyval_update() - Update a POLYVAL context with message data + * @ctx: The context to update; must have been initialized + * @data: The message data + * @len: The data length in bytes. Doesn't need to be block-aligned. + * + * This can be called any number of times. + * + * Context: Any context. + */ +void polyval_update(struct polyval_ctx *ctx, const u8 *data, size_t len); + +/** + * polyval_final() - Finish computing a POLYVAL value + * @ctx: The context to finalize + * @out: The output value + * + * If the total data length isn't a multiple of POLYVAL_BLOCK_SIZE, then t= he + * final block is automatically zero-padded. + * + * After finishing, this zeroizes @ctx. So the caller does not need to do= it. + * + * Context: Any context. + */ +void polyval_final(struct polyval_ctx *ctx, u8 out[POLYVAL_BLOCK_SIZE]); + +/** + * polyval() - Compute a POLYVAL value + * @key: The prepared key + * @data: The message data + * @len: The data length in bytes. Doesn't need to be block-aligned. + * @out: The output value + * + * Context: Any context. + */ +static inline void polyval(const struct polyval_key *key, + const u8 *data, size_t len, + u8 out[POLYVAL_BLOCK_SIZE]) +{ + struct polyval_ctx ctx; + + polyval_init(&ctx, key); + polyval_update(&ctx, data, len); + polyval_final(&ctx, out); +} + +#endif /* _CRYPTO_POLYVAL_H */ diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig index 7445054fc0ad..6545f0e83b83 100644 --- a/lib/crypto/Kconfig +++ b/lib/crypto/Kconfig @@ -133,10 +133,20 @@ config CRYPTO_LIB_POLY1305_RSIZE default 2 if MIPS || RISCV default 11 if X86_64 default 9 if ARM || ARM64 default 1 =20 +config CRYPTO_LIB_POLYVAL + tristate + help + The POLYVAL library functions. Select this if your module uses any of + the functions from . + +config CRYPTO_LIB_POLYVAL_ARCH + bool + depends on CRYPTO_LIB_POLYVAL && !UML + config CRYPTO_LIB_CHACHA20POLY1305 tristate select CRYPTO_LIB_CHACHA select CRYPTO_LIB_POLY1305 select CRYPTO_LIB_UTILS diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 5515e73bfd5e..055e44008805 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -196,10 +196,18 @@ clean-files +=3D arm/poly1305-core.S \ riscv/poly1305-core.S \ x86/poly1305-x86_64-cryptogams.S =20 ##########################################################################= ###### =20 +obj-$(CONFIG_CRYPTO_LIB_POLYVAL) +=3D libpolyval.o +libpolyval-y :=3D polyval.o +ifeq ($(CONFIG_CRYPTO_LIB_POLYVAL_ARCH),y) +CFLAGS_polyval.o +=3D -I$(src)/$(SRCARCH) +endif + +##########################################################################= ###### + obj-$(CONFIG_CRYPTO_LIB_SHA1) +=3D libsha1.o libsha1-y :=3D sha1.o ifeq ($(CONFIG_CRYPTO_LIB_SHA1_ARCH),y) CFLAGS_sha1.o +=3D -I$(src)/$(SRCARCH) ifeq ($(CONFIG_ARM),y) diff --git a/lib/crypto/polyval.c b/lib/crypto/polyval.c new file mode 100644 index 000000000000..5796275f574a --- /dev/null +++ b/lib/crypto/polyval.c @@ -0,0 +1,307 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * POLYVAL library functions + * + * Copyright 2025 Google LLC + */ + +#include +#include +#include +#include +#include + +/* + * POLYVAL is an almost-XOR-universal hash function. Similar to GHASH, PO= LYVAL + * interprets the message as the coefficients of a polynomial in GF(2^128)= and + * evaluates that polynomial at a secret point. POLYVAL has a simple + * mathematical relationship with GHASH, but it uses a better field conven= tion + * which makes it easier and faster to implement. + * + * POLYVAL is not a cryptographic hash function, and it should be used onl= y by + * algorithms that are specifically designed to use it. + * + * POLYVAL is specified by "AES-GCM-SIV: Nonce Misuse-Resistant Authentica= ted + * Encryption" (https://datatracker.ietf.org/doc/html/rfc8452) + * + * POLYVAL is also used by HCTR2. See "Length-preserving encryption with = HCTR2" + * (https://eprint.iacr.org/2021/1441.pdf). + * + * This file provides a library API for POLYVAL. This API can delegate to + * either a generic implementation or an architecture-optimized implementa= tion. + * + * For the generic implementation, we don't use the traditional table appr= oach + * to GF(2^128) multiplication. That approach is not constant-time and re= quires + * a lot of memory. Instead, we use a different approach which emulates + * carryless multiplication using standard multiplications by spreading th= e data + * bits apart using "holes". This allows the carries to spill harmlessly.= This + * approach is borrowed from BoringSSL, which in turn credits BearSSL's + * documentation (https://bearssl.org/constanttime.html#ghash-for-gcm) for= the + * "holes" trick and a presentation by Shay Gueron + * (https://crypto.stanford.edu/RealWorldCrypto/slides/gueron.pdf) for the + * 256-bit =3D> 128-bit reduction algorithm. + */ + +#ifdef CONFIG_ARCH_SUPPORTS_INT128 + +/* Do a 64 x 64 =3D> 128 bit carryless multiplication. */ +static void clmul64(u64 a, u64 b, u64 *out_lo, u64 *out_hi) +{ + /* + * With 64-bit multiplicands and one term every 4 bits, there would be + * up to 64 / 4 =3D 16 one bits per column when each multiplication is + * written out as a series of additions in the schoolbook manner. + * Unfortunately, that doesn't work since the value 16 is 1 too large to + * fit in 4 bits. Carries would sometimes overflow into the next term. + * + * Using one term every 5 bits would work. However, that would cost + * 5 x 5 =3D 25 multiplications instead of 4 x 4 =3D 16. + * + * Instead, mask off 4 bits from one multiplicand, giving a max of 15 + * one bits per column. Then handle those 4 bits separately. + */ + u64 a0 =3D a & 0x1111111111111110; + u64 a1 =3D a & 0x2222222222222220; + u64 a2 =3D a & 0x4444444444444440; + u64 a3 =3D a & 0x8888888888888880; + + u64 b0 =3D b & 0x1111111111111111; + u64 b1 =3D b & 0x2222222222222222; + u64 b2 =3D b & 0x4444444444444444; + u64 b3 =3D b & 0x8888888888888888; + + /* Multiply the high 60 bits of @a by @b. */ + u128 c0 =3D (a0 * (u128)b0) ^ (a1 * (u128)b3) ^ + (a2 * (u128)b2) ^ (a3 * (u128)b1); + u128 c1 =3D (a0 * (u128)b1) ^ (a1 * (u128)b0) ^ + (a2 * (u128)b3) ^ (a3 * (u128)b2); + u128 c2 =3D (a0 * (u128)b2) ^ (a1 * (u128)b1) ^ + (a2 * (u128)b0) ^ (a3 * (u128)b3); + u128 c3 =3D (a0 * (u128)b3) ^ (a1 * (u128)b2) ^ + (a2 * (u128)b1) ^ (a3 * (u128)b0); + + /* Multiply the low 4 bits of @a by @b. */ + u64 e0 =3D -(a & 1) & b; + u64 e1 =3D -((a >> 1) & 1) & b; + u64 e2 =3D -((a >> 2) & 1) & b; + u64 e3 =3D -((a >> 3) & 1) & b; + u64 extra_lo =3D e0 ^ (e1 << 1) ^ (e2 << 2) ^ (e3 << 3); + u64 extra_hi =3D (e1 >> 63) ^ (e2 >> 62) ^ (e3 >> 61); + + /* Add all the intermediate products together. */ + *out_lo =3D (((u64)c0) & 0x1111111111111111) ^ + (((u64)c1) & 0x2222222222222222) ^ + (((u64)c2) & 0x4444444444444444) ^ + (((u64)c3) & 0x8888888888888888) ^ extra_lo; + *out_hi =3D (((u64)(c0 >> 64)) & 0x1111111111111111) ^ + (((u64)(c1 >> 64)) & 0x2222222222222222) ^ + (((u64)(c2 >> 64)) & 0x4444444444444444) ^ + (((u64)(c3 >> 64)) & 0x8888888888888888) ^ extra_hi; +} + +#else /* CONFIG_ARCH_SUPPORTS_INT128 */ + +/* Do a 32 x 32 =3D> 64 bit carryless multiplication. */ +static u64 clmul32(u32 a, u32 b) +{ + /* + * With 32-bit multiplicands and one term every 4 bits, there are up to + * 32 / 4 =3D 8 one bits per column when each multiplication is written + * out as a series of additions in the schoolbook manner. The value 8 + * fits in 4 bits, so the carries don't overflow into the next term. + */ + u32 a0 =3D a & 0x11111111; + u32 a1 =3D a & 0x22222222; + u32 a2 =3D a & 0x44444444; + u32 a3 =3D a & 0x88888888; + + u32 b0 =3D b & 0x11111111; + u32 b1 =3D b & 0x22222222; + u32 b2 =3D b & 0x44444444; + u32 b3 =3D b & 0x88888888; + + u64 c0 =3D (a0 * (u64)b0) ^ (a1 * (u64)b3) ^ + (a2 * (u64)b2) ^ (a3 * (u64)b1); + u64 c1 =3D (a0 * (u64)b1) ^ (a1 * (u64)b0) ^ + (a2 * (u64)b3) ^ (a3 * (u64)b2); + u64 c2 =3D (a0 * (u64)b2) ^ (a1 * (u64)b1) ^ + (a2 * (u64)b0) ^ (a3 * (u64)b3); + u64 c3 =3D (a0 * (u64)b3) ^ (a1 * (u64)b2) ^ + (a2 * (u64)b1) ^ (a3 * (u64)b0); + + /* Add all the intermediate products together. */ + return (c0 & 0x1111111111111111) ^ + (c1 & 0x2222222222222222) ^ + (c2 & 0x4444444444444444) ^ + (c3 & 0x8888888888888888); +} + +/* Do a 64 x 64 =3D> 128 bit carryless multiplication. */ +static void clmul64(u64 a, u64 b, u64 *out_lo, u64 *out_hi) +{ + u32 a_lo =3D (u32)a; + u32 a_hi =3D a >> 32; + u32 b_lo =3D (u32)b; + u32 b_hi =3D b >> 32; + + /* Karatsuba multiplication */ + u64 lo =3D clmul32(a_lo, b_lo); + u64 hi =3D clmul32(a_hi, b_hi); + u64 mi =3D clmul32(a_lo ^ a_hi, b_lo ^ b_hi) ^ lo ^ hi; + + *out_lo =3D lo ^ (mi << 32); + *out_hi =3D hi ^ (mi >> 32); +} +#endif /* !CONFIG_ARCH_SUPPORTS_INT128 */ + +/* Compute @a =3D @a * @b * x^-128 in the POLYVAL field. */ +static void __maybe_unused +polyval_mul_generic(struct polyval_elem *a, const struct polyval_elem *b) +{ + u64 c0, c1, c2, c3, mi0, mi1; + + /* + * Carryless-multiply @a by @b using Karatsuba multiplication. Store + * the 256-bit product in @c0 (low) through @c3 (high). + */ + clmul64(le64_to_cpu(a->lo), le64_to_cpu(b->lo), &c0, &c1); + clmul64(le64_to_cpu(a->hi), le64_to_cpu(b->hi), &c2, &c3); + clmul64(le64_to_cpu(a->lo ^ a->hi), le64_to_cpu(b->lo ^ b->hi), + &mi0, &mi1); + mi0 ^=3D c0 ^ c2; + mi1 ^=3D c1 ^ c3; + c1 ^=3D mi0; + c2 ^=3D mi1; + + /* + * Cancel out the low 128 bits of the product by adding multiples of + * G(x) =3D x^128 + x^127 + x^126 + x^121 + 1. Do this in two steps, each + * of which cancels out 64 bits. Note that we break G(x) into three + * parts: 1, x^64 * (x^63 + x^62 + x^57), and x^128 * 1. + */ + + /* + * First, add G(x) times c0 as follows: + * + * (c0, c1, c2) =3D (0, + * c1 + (c0 * (x^63 + x^62 + x^57) mod x^64), + * c2 + c0 + floor((c0 * (x^63 + x^62 + x^57)) / x^64)) + */ + c1 ^=3D (c0 << 63) ^ (c0 << 62) ^ (c0 << 57); + c2 ^=3D c0 ^ (c0 >> 1) ^ (c0 >> 2) ^ (c0 >> 7); + + /* + * Second, add G(x) times the new c1: + * + * (c1, c2, c3) =3D (0, + * c2 + (c1 * (x^63 + x^62 + x^57) mod x^64), + * c3 + c1 + floor((c1 * (x^63 + x^62 + x^57)) / x^64)) + */ + c2 ^=3D (c1 << 63) ^ (c1 << 62) ^ (c1 << 57); + c3 ^=3D c1 ^ (c1 >> 1) ^ (c1 >> 2) ^ (c1 >> 7); + + /* Return (c2, c3). This implicitly multiplies by x^-128. */ + a->lo =3D cpu_to_le64(c2); + a->hi =3D cpu_to_le64(c3); +} + +static void __maybe_unused +polyval_blocks_generic(struct polyval_elem *acc, const struct polyval_elem= *key, + const u8 *data, size_t nblocks) +{ + do { + acc->lo ^=3D get_unaligned((__le64 *)data); + acc->hi ^=3D get_unaligned((__le64 *)(data + 8)); + polyval_mul_generic(acc, key); + data +=3D POLYVAL_BLOCK_SIZE; + } while (--nblocks); +} + +/* Include the arch-optimized implementation of POLYVAL, if one is availab= le. */ +#ifdef CONFIG_CRYPTO_LIB_POLYVAL_ARCH +#include "polyval.h" /* $(SRCARCH)/polyval.h */ +void polyval_preparekey(struct polyval_key *key, + const u8 raw_key[POLYVAL_BLOCK_SIZE]) +{ + polyval_preparekey_arch(key, raw_key); +} +EXPORT_SYMBOL_GPL(polyval_preparekey); +#endif /* Else, polyval_preparekey() is an inline function. */ + +/* + * polyval_mul_generic() and polyval_blocks_generic() take the key as a + * polyval_elem rather than a polyval_key, so that arch-optimized + * implementations with a different key format can use it as a fallback (i= f they + * have H^1 stored somewhere in their struct). Thus, the following dispat= ch + * code is needed to pass the appropriate key argument. + */ + +static void polyval_mul(struct polyval_ctx *ctx) +{ +#ifdef CONFIG_CRYPTO_LIB_POLYVAL_ARCH + polyval_mul_arch(&ctx->acc, ctx->key); +#else + polyval_mul_generic(&ctx->acc, &ctx->key->h); +#endif +} + +static void polyval_blocks(struct polyval_ctx *ctx, + const u8 *data, size_t nblocks) +{ +#ifdef CONFIG_CRYPTO_LIB_POLYVAL_ARCH + polyval_blocks_arch(&ctx->acc, ctx->key, data, nblocks); +#else + polyval_blocks_generic(&ctx->acc, &ctx->key->h, data, nblocks); +#endif +} + +void polyval_update(struct polyval_ctx *ctx, const u8 *data, size_t len) +{ + if (unlikely(ctx->partial)) { + size_t n =3D min(len, POLYVAL_BLOCK_SIZE - ctx->partial); + + len -=3D n; + while (n--) + ctx->acc.bytes[ctx->partial++] ^=3D *data++; + if (ctx->partial < POLYVAL_BLOCK_SIZE) + return; + polyval_mul(ctx); + } + if (len >=3D POLYVAL_BLOCK_SIZE) { + size_t nblocks =3D len / POLYVAL_BLOCK_SIZE; + + polyval_blocks(ctx, data, nblocks); + data +=3D len & ~(POLYVAL_BLOCK_SIZE - 1); + len &=3D POLYVAL_BLOCK_SIZE - 1; + } + for (size_t i =3D 0; i < len; i++) + ctx->acc.bytes[i] ^=3D data[i]; + ctx->partial =3D len; +} +EXPORT_SYMBOL_GPL(polyval_update); + +void polyval_final(struct polyval_ctx *ctx, u8 out[POLYVAL_BLOCK_SIZE]) +{ + if (unlikely(ctx->partial)) + polyval_mul(ctx); + memcpy(out, &ctx->acc, POLYVAL_BLOCK_SIZE); + memzero_explicit(ctx, sizeof(*ctx)); +} +EXPORT_SYMBOL_GPL(polyval_final); + +#ifdef polyval_mod_init_arch +static int __init polyval_mod_init(void) +{ + polyval_mod_init_arch(); + return 0; +} +subsys_initcall(polyval_mod_init); + +static void __exit polyval_mod_exit(void) +{ +} +module_exit(polyval_mod_exit); +#endif + +MODULE_DESCRIPTION("POLYVAL almost-XOR-universal hash function"); +MODULE_LICENSE("GPL"); --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 794BC274FE8; Sun, 9 Nov 2025 23:49:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732198; cv=none; b=gsMDi1tNaZskBsVKCQ+psR1rhbuV70cr1apwzoHs+5JRsAco1OsRprI25UPSv5CV+H2TuHS0pqsirdq6JYUOoRjWYq5zF0RejbLhloVNMGR7s06dSwIAw6prVsvk0hVyABMK27pZ4LSGgC4z30d+tjlSQqdkkkTa5l7Kzuxmn+U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732198; c=relaxed/simple; bh=y919u1v+MtlZR4u2oYCQjGN0x4jfhBz4y3Z9Fztktow=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bHFGUhJYXGnhlsi9dEbKMg8CYZXw4qogITmwMkbRAp8ojfvJPULvS3OWVGz1ZOpCCaTInWwj9u7xbmtJOeJoT/edEbEWedloJn2m0wUk+WBtnumu8beNCBX5asdj4CanLNk3YCv7HdtT+NpUYh5eUfD8hhfGAXHft7mUZevWUOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EF8tyt0T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EF8tyt0T" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E0957C4CEF7; Sun, 9 Nov 2025 23:49:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732198; bh=y919u1v+MtlZR4u2oYCQjGN0x4jfhBz4y3Z9Fztktow=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EF8tyt0T+HKt81YJjUihiwiM6YKtli8gmXV3BkzrU77IhacgLXG7TwGwMLxhnwkW3 Vhog3eQCw/TZkuYl5Wi4RgyWpfY7wdTDZkogRmztvtop7hqX0qGhtrK+qHf/2HZn4x 2Rln+4TmNzAXkRwvC7abNe5/fpzXHeMkUih/FqyaXLui90ZjpIcXq/Tl/P7DIQMZpn cbITds/gh6HfOhiazi0NT1LHYE/z/8dw3blHSQ4Qbc/RhPZyoBoqEOYsAHYLZGnJ+h DUETTVLFpcWvBJL7Oh3sOd5TxbscxRPLoROCmOc/8m2xHfBfT+sQPKjkcwXy6l7ATW qyzRnjNQfDSSQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 3/9] lib/crypto: tests: Add KUnit tests for POLYVAL Date: Sun, 9 Nov 2025 15:47:18 -0800 Message-ID: <20251109234726.638437-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a test suite for the POLYVAL library, including: - All the standard tests and the benchmark from hash-test-template.h - Comparison with a test vector from the RFC - Test with key and message containing all one bits - Additional tests related to the key struct Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- lib/crypto/tests/Kconfig | 9 ++ lib/crypto/tests/Makefile | 1 + lib/crypto/tests/polyval-testvecs.h | 186 +++++++++++++++++++++++ lib/crypto/tests/polyval_kunit.c | 223 ++++++++++++++++++++++++++++ scripts/crypto/gen-hash-testvecs.py | 47 +++++- 5 files changed, 464 insertions(+), 2 deletions(-) create mode 100644 lib/crypto/tests/polyval-testvecs.h create mode 100644 lib/crypto/tests/polyval_kunit.c diff --git a/lib/crypto/tests/Kconfig b/lib/crypto/tests/Kconfig index 140afd1714ba..61d435c450bb 100644 --- a/lib/crypto/tests/Kconfig +++ b/lib/crypto/tests/Kconfig @@ -45,10 +45,19 @@ config CRYPTO_LIB_POLY1305_KUNIT_TEST select CRYPTO_LIB_BENCHMARK_VISIBLE select CRYPTO_LIB_POLY1305 help KUnit tests for the Poly1305 library functions. =20 +config CRYPTO_LIB_POLYVAL_KUNIT_TEST + tristate "KUnit tests for POLYVAL" if !KUNIT_ALL_TESTS + depends on KUNIT + default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS + select CRYPTO_LIB_BENCHMARK_VISIBLE + select CRYPTO_LIB_POLYVAL + help + KUnit tests for the POLYVAL library functions. + config CRYPTO_LIB_SHA1_KUNIT_TEST tristate "KUnit tests for SHA-1" if !KUNIT_ALL_TESTS depends on KUNIT default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS select CRYPTO_LIB_BENCHMARK_VISIBLE diff --git a/lib/crypto/tests/Makefile b/lib/crypto/tests/Makefile index f7d1392dc847..5109a0651925 100644 --- a/lib/crypto/tests/Makefile +++ b/lib/crypto/tests/Makefile @@ -3,9 +3,10 @@ obj-$(CONFIG_CRYPTO_LIB_BLAKE2B_KUNIT_TEST) +=3D blake2b_kunit.o obj-$(CONFIG_CRYPTO_LIB_BLAKE2S_KUNIT_TEST) +=3D blake2s_kunit.o obj-$(CONFIG_CRYPTO_LIB_CURVE25519_KUNIT_TEST) +=3D curve25519_kunit.o obj-$(CONFIG_CRYPTO_LIB_MD5_KUNIT_TEST) +=3D md5_kunit.o obj-$(CONFIG_CRYPTO_LIB_POLY1305_KUNIT_TEST) +=3D poly1305_kunit.o +obj-$(CONFIG_CRYPTO_LIB_POLYVAL_KUNIT_TEST) +=3D polyval_kunit.o obj-$(CONFIG_CRYPTO_LIB_SHA1_KUNIT_TEST) +=3D sha1_kunit.o obj-$(CONFIG_CRYPTO_LIB_SHA256_KUNIT_TEST) +=3D sha224_kunit.o sha256_kuni= t.o obj-$(CONFIG_CRYPTO_LIB_SHA512_KUNIT_TEST) +=3D sha384_kunit.o sha512_kuni= t.o obj-$(CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST) +=3D sha3_kunit.o diff --git a/lib/crypto/tests/polyval-testvecs.h b/lib/crypto/tests/polyval= -testvecs.h new file mode 100644 index 000000000000..3d33f60d58bb --- /dev/null +++ b/lib/crypto/tests/polyval-testvecs.h @@ -0,0 +1,186 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* This file was generated by: ./scripts/crypto/gen-hash-testvecs.py polyv= al */ + +static const struct { + size_t data_len; + u8 digest[POLYVAL_DIGEST_SIZE]; +} hash_testvecs[] =3D { + { + .data_len =3D 0, + .digest =3D { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + }, + }, + { + .data_len =3D 1, + .digest =3D { + 0xb5, 0x51, 0x69, 0x89, 0xd4, 0x3c, 0x59, 0xca, + 0x6a, 0x1c, 0x2a, 0xe9, 0xa1, 0x9c, 0x6c, 0x83, + }, + }, + { + .data_len =3D 2, + .digest =3D { + 0xf4, 0x50, 0xaf, 0x07, 0xda, 0x42, 0xa7, 0x41, + 0x4d, 0x24, 0x88, 0x87, 0xe3, 0x40, 0x73, 0x7c, + }, + }, + { + .data_len =3D 3, + .digest =3D { + 0x9e, 0x88, 0x78, 0x71, 0x4c, 0x55, 0x87, 0xe8, + 0xb4, 0x96, 0x3d, 0x56, 0xc8, 0xb2, 0xe1, 0x68, + }, + }, + { + .data_len =3D 16, + .digest =3D { + 0x9e, 0x81, 0x37, 0x8f, 0x49, 0xf7, 0xa2, 0xe4, + 0x04, 0x45, 0x12, 0x78, 0x45, 0x42, 0x27, 0xad, + }, + }, + { + .data_len =3D 32, + .digest =3D { + 0x60, 0x19, 0xd0, 0xa4, 0xf0, 0xde, 0x9e, 0xe7, + 0x6a, 0x89, 0x1a, 0xea, 0x80, 0x14, 0xa9, 0xa3, + }, + }, + { + .data_len =3D 48, + .digest =3D { + 0x0c, 0xa2, 0x70, 0x4d, 0x7c, 0x89, 0xac, 0x41, + 0xc2, 0x9e, 0x0d, 0x07, 0x07, 0x6a, 0x7f, 0xd5, + }, + }, + { + .data_len =3D 49, + .digest =3D { + 0x91, 0xd3, 0xa9, 0x5c, 0x79, 0x3d, 0x6b, 0x84, + 0x99, 0x54, 0xa7, 0xb4, 0x06, 0x66, 0xfd, 0x1c, + }, + }, + { + .data_len =3D 63, + .digest =3D { + 0x29, 0x37, 0xb8, 0xe5, 0xd8, 0x27, 0x4d, 0xfb, + 0x83, 0x4f, 0x67, 0xf7, 0xf9, 0xc1, 0x0a, 0x9d, + }, + }, + { + .data_len =3D 64, + .digest =3D { + 0x17, 0xa9, 0x06, 0x2c, 0xf3, 0xe8, 0x2e, 0xa6, + 0x6b, 0xb2, 0x1f, 0x5d, 0x94, 0x3c, 0x02, 0xa2, + }, + }, + { + .data_len =3D 65, + .digest =3D { + 0x7c, 0x80, 0x74, 0xd7, 0xa1, 0x37, 0x30, 0x64, + 0x3b, 0xa4, 0xa3, 0x98, 0xde, 0x47, 0x10, 0x23, + }, + }, + { + .data_len =3D 127, + .digest =3D { + 0x27, 0x3a, 0xcf, 0xf5, 0xaf, 0x9f, 0xd8, 0xd8, + 0x2d, 0x6a, 0x91, 0xfb, 0xb8, 0xfa, 0xbe, 0x0c, + }, + }, + { + .data_len =3D 128, + .digest =3D { + 0x97, 0x6e, 0xc4, 0xbe, 0x6b, 0x15, 0xa6, 0x7c, + 0xc4, 0xa2, 0xb8, 0x0a, 0x0e, 0x9c, 0xc7, 0x3a, + }, + }, + { + .data_len =3D 129, + .digest =3D { + 0x2b, 0xc3, 0x98, 0xba, 0x6e, 0x42, 0xf8, 0x18, + 0x85, 0x69, 0x15, 0x37, 0x10, 0x60, 0xe6, 0xac, + }, + }, + { + .data_len =3D 256, + .digest =3D { + 0x88, 0x21, 0x77, 0x89, 0xd7, 0x93, 0x90, 0xfc, + 0xf3, 0xb0, 0xe3, 0xfb, 0x14, 0xe2, 0xcf, 0x74, + }, + }, + { + .data_len =3D 511, + .digest =3D { + 0x66, 0x3d, 0x3e, 0x08, 0xa0, 0x49, 0x81, 0x68, + 0x3e, 0x3b, 0xc8, 0x80, 0x55, 0xd4, 0x15, 0xe9, + }, + }, + { + .data_len =3D 513, + .digest =3D { + 0x05, 0xf5, 0x06, 0x66, 0xe7, 0x11, 0x08, 0x84, + 0xff, 0x94, 0x50, 0x85, 0x65, 0x95, 0x2a, 0x20, + }, + }, + { + .data_len =3D 1000, + .digest =3D { + 0xd3, 0xa0, 0x51, 0x69, 0xb5, 0x38, 0xae, 0x1b, + 0xe1, 0xa2, 0x89, 0xc6, 0x8d, 0x2b, 0x62, 0x37, + }, + }, + { + .data_len =3D 3333, + .digest =3D { + 0x37, 0x6d, 0x6a, 0x14, 0xdc, 0xa5, 0x37, 0xfc, + 0xfe, 0x67, 0x76, 0xb2, 0x64, 0x68, 0x64, 0x05, + }, + }, + { + .data_len =3D 4096, + .digest =3D { + 0xe3, 0x12, 0x0c, 0x58, 0x46, 0x45, 0x27, 0x7a, + 0x0e, 0xa2, 0xfa, 0x2c, 0x35, 0x73, 0x6c, 0x94, + }, + }, + { + .data_len =3D 4128, + .digest =3D { + 0x63, 0x0d, 0xa1, 0xbc, 0x6e, 0x3e, 0xd3, 0x1d, + 0x28, 0x52, 0xd2, 0xf4, 0x30, 0x2d, 0xff, 0xc4, + }, + }, + { + .data_len =3D 4160, + .digest =3D { + 0xb2, 0x91, 0x49, 0xe2, 0x02, 0x98, 0x00, 0x79, + 0x71, 0xb9, 0xd7, 0xd4, 0xb5, 0x94, 0x6d, 0x7d, + }, + }, + { + .data_len =3D 4224, + .digest =3D { + 0x58, 0x96, 0x48, 0x69, 0x05, 0x17, 0xe1, 0x6d, + 0xbc, 0xf2, 0x3d, 0x10, 0x96, 0x00, 0x74, 0x58, + }, + }, + { + .data_len =3D 16384, + .digest =3D { + 0x99, 0x3c, 0xcb, 0x4d, 0x64, 0xc9, 0xa9, 0x41, + 0x52, 0x93, 0xfd, 0x65, 0xc4, 0xcc, 0xa5, 0xe5, + }, + }, +}; + +static const u8 hash_testvec_consolidated[POLYVAL_DIGEST_SIZE] =3D { + 0xdf, 0x68, 0x52, 0x99, 0x92, 0xc3, 0xe8, 0x88, + 0x29, 0x13, 0xc8, 0x35, 0x67, 0xa3, 0xd3, 0xad, +}; + +static const u8 polyval_allones_hashofhashes[POLYVAL_DIGEST_SIZE] =3D { + 0xd5, 0xf7, 0xfd, 0xb2, 0xa6, 0xef, 0x0b, 0x85, + 0x0d, 0x0a, 0x06, 0x10, 0xbc, 0x64, 0x94, 0x73, +}; diff --git a/lib/crypto/tests/polyval_kunit.c b/lib/crypto/tests/polyval_ku= nit.c new file mode 100644 index 000000000000..e59f598c1572 --- /dev/null +++ b/lib/crypto/tests/polyval_kunit.c @@ -0,0 +1,223 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright 2025 Google LLC + */ +#include +#include "polyval-testvecs.h" + +/* + * A fixed key used when presenting POLYVAL as an unkeyed hash function in= order + * to reuse hash-test-template.h. At the beginning of the test suite, thi= s is + * initialized to a key prepared from bytes generated from a fixed seed. + */ +static struct polyval_key test_key; + +static void polyval_init_withtestkey(struct polyval_ctx *ctx) +{ + polyval_init(ctx, &test_key); +} + +static void polyval_withtestkey(const u8 *data, size_t len, + u8 out[POLYVAL_BLOCK_SIZE]) +{ + polyval(&test_key, data, len, out); +} + +/* Generate the HASH_KUNIT_CASES using hash-test-template.h. */ +#define HASH polyval_withtestkey +#define HASH_CTX polyval_ctx +#define HASH_SIZE POLYVAL_BLOCK_SIZE +#define HASH_INIT polyval_init_withtestkey +#define HASH_UPDATE polyval_update +#define HASH_FINAL polyval_final +#include "hash-test-template.h" + +/* + * Test an example from RFC8452 ("AES-GCM-SIV: Nonce Misuse-Resistant + * Authenticated Encryption") to ensure compatibility with that. + */ +static void test_polyval_rfc8452_testvec(struct kunit *test) +{ + static const u8 raw_key[POLYVAL_BLOCK_SIZE] =3D + "\x31\x07\x28\xd9\x91\x1f\x1f\x38" + "\x37\xb2\x43\x16\xc3\xfa\xb9\xa0"; + static const u8 data[48] =3D + "\x65\x78\x61\x6d\x70\x6c\x65\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x48\x65\x6c\x6c\x6f\x20\x77\x6f" + "\x72\x6c\x64\x00\x00\x00\x00\x00" + "\x38\x00\x00\x00\x00\x00\x00\x00" + "\x58\x00\x00\x00\x00\x00\x00\x00"; + static const u8 expected_hash[POLYVAL_BLOCK_SIZE] =3D + "\xad\x7f\xcf\x0b\x51\x69\x85\x16" + "\x62\x67\x2f\x3c\x5f\x95\x13\x8f"; + u8 hash[POLYVAL_BLOCK_SIZE]; + struct polyval_key key; + + polyval_preparekey(&key, raw_key); + polyval(&key, data, sizeof(data), hash); + KUNIT_ASSERT_MEMEQ(test, hash, expected_hash, sizeof(hash)); +} + +/* + * Test a key and messages containing all one bits. This is useful to det= ect + * overflow bugs in implementations that emulate carryless multiplication = using + * a series of standard multiplications with the bits spread out. + */ +static void test_polyval_allones_key_and_message(struct kunit *test) +{ + struct polyval_key key; + struct polyval_ctx hashofhashes_ctx; + u8 hash[POLYVAL_BLOCK_SIZE]; + + static_assert(TEST_BUF_LEN >=3D 4096); + memset(test_buf, 0xff, 4096); + + polyval_preparekey(&key, test_buf); + polyval_init(&hashofhashes_ctx, &key); + for (size_t len =3D 0; len <=3D 4096; len +=3D 16) { + polyval(&key, test_buf, len, hash); + polyval_update(&hashofhashes_ctx, hash, sizeof(hash)); + } + polyval_final(&hashofhashes_ctx, hash); + KUNIT_ASSERT_MEMEQ(test, hash, polyval_allones_hashofhashes, + sizeof(hash)); +} + +#define MAX_LEN_FOR_KEY_CHECK 1024 + +/* + * Given two prepared keys which should be identical (but may differ in + * alignment and/or whether they are followed by a guard page or not), ver= ify + * that they produce consistent results on various data lengths. + */ +static void check_key_consistency(struct kunit *test, + const struct polyval_key *key1, + const struct polyval_key *key2) +{ + u8 *data =3D test_buf; + u8 hash1[POLYVAL_BLOCK_SIZE]; + u8 hash2[POLYVAL_BLOCK_SIZE]; + + rand_bytes(data, MAX_LEN_FOR_KEY_CHECK); + KUNIT_ASSERT_MEMEQ(test, key1, key2, sizeof(*key1)); + + for (int i =3D 0; i < 100; i++) { + size_t len =3D rand_length(MAX_LEN_FOR_KEY_CHECK); + + polyval(key1, data, len, hash1); + polyval(key2, data, len, hash2); + KUNIT_ASSERT_MEMEQ(test, hash1, hash2, sizeof(hash1)); + } +} + +/* Test that no buffer overreads occur on either raw_key or polyval_key. */ +static void test_polyval_with_guarded_key(struct kunit *test) +{ + u8 raw_key[POLYVAL_BLOCK_SIZE]; + u8 *guarded_raw_key =3D &test_buf[TEST_BUF_LEN - sizeof(raw_key)]; + struct polyval_key key1, key2; + struct polyval_key *guarded_key =3D + (struct polyval_key *)&test_buf[TEST_BUF_LEN - sizeof(key1)]; + + /* Prepare with regular buffers. */ + rand_bytes(raw_key, sizeof(raw_key)); + polyval_preparekey(&key1, raw_key); + + /* Prepare with guarded raw_key, then check that it works. */ + memcpy(guarded_raw_key, raw_key, sizeof(raw_key)); + polyval_preparekey(&key2, guarded_raw_key); + check_key_consistency(test, &key1, &key2); + + /* Prepare guarded polyval_key, then check that it works. */ + polyval_preparekey(guarded_key, raw_key); + check_key_consistency(test, &key1, guarded_key); +} + +/* + * Test that polyval_key only needs to be aligned to + * __alignof__(struct polyval_key), i.e. 8 bytes. The assembly code may p= refer + * 16-byte or higher alignment, but it musn't require it. + */ +static void test_polyval_with_minimally_aligned_key(struct kunit *test) +{ + u8 raw_key[POLYVAL_BLOCK_SIZE]; + struct polyval_key key; + struct polyval_key *minaligned_key =3D + (struct polyval_key *)&test_buf[MAX_LEN_FOR_KEY_CHECK + + __alignof__(struct polyval_key)]; + + KUNIT_ASSERT_TRUE(test, IS_ALIGNED((uintptr_t)minaligned_key, + __alignof__(struct polyval_key))); + KUNIT_ASSERT_TRUE(test, + !IS_ALIGNED((uintptr_t)minaligned_key, + 2 * __alignof__(struct polyval_key))); + + rand_bytes(raw_key, sizeof(raw_key)); + polyval_preparekey(&key, raw_key); + polyval_preparekey(minaligned_key, raw_key); + check_key_consistency(test, &key, minaligned_key); +} + +struct polyval_irq_test_state { + struct polyval_key expected_key; + u8 raw_key[POLYVAL_BLOCK_SIZE]; +}; + +static bool polyval_irq_test_func(void *state_) +{ + struct polyval_irq_test_state *state =3D state_; + struct polyval_key key; + + polyval_preparekey(&key, state->raw_key); + return memcmp(&key, &state->expected_key, sizeof(key)) =3D=3D 0; +} + +/* + * Test that polyval_preparekey() produces the same output regardless of w= hether + * FPU or vector registers are usable when it is called. + */ +static void test_polyval_preparekey_in_irqs(struct kunit *test) +{ + struct polyval_irq_test_state state; + + rand_bytes(state.raw_key, sizeof(state.raw_key)); + polyval_preparekey(&state.expected_key, state.raw_key); + kunit_run_irq_test(test, polyval_irq_test_func, 20000, &state); +} + +static int polyval_suite_init(struct kunit_suite *suite) +{ + u8 raw_key[POLYVAL_BLOCK_SIZE]; + + rand_bytes_seeded_from_len(raw_key, sizeof(raw_key)); + polyval_preparekey(&test_key, raw_key); + return hash_suite_init(suite); +} + +static void polyval_suite_exit(struct kunit_suite *suite) +{ + hash_suite_exit(suite); +} + +static struct kunit_case polyval_test_cases[] =3D { + HASH_KUNIT_CASES, + KUNIT_CASE(test_polyval_rfc8452_testvec), + KUNIT_CASE(test_polyval_allones_key_and_message), + KUNIT_CASE(test_polyval_with_guarded_key), + KUNIT_CASE(test_polyval_with_minimally_aligned_key), + KUNIT_CASE(test_polyval_preparekey_in_irqs), + KUNIT_CASE(benchmark_hash), + {}, +}; + +static struct kunit_suite polyval_test_suite =3D { + .name =3D "polyval", + .test_cases =3D polyval_test_cases, + .suite_init =3D polyval_suite_init, + .suite_exit =3D polyval_suite_exit, +}; +kunit_test_suite(polyval_test_suite); + +MODULE_DESCRIPTION("KUnit tests and benchmark for POLYVAL"); +MODULE_LICENSE("GPL"); diff --git a/scripts/crypto/gen-hash-testvecs.py b/scripts/crypto/gen-hash-= testvecs.py index ae2682882cd1..c1d0517140bd 100755 --- a/scripts/crypto/gen-hash-testvecs.py +++ b/scripts/crypto/gen-hash-testvecs.py @@ -1,9 +1,9 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: GPL-2.0-or-later # -# Script that generates test vectors for the given cryptographic hash func= tion. +# Script that generates test vectors for the given hash function. # # Copyright 2025 Google LLC =20 import hashlib import hmac @@ -48,15 +48,46 @@ class Poly1305: # nondestructive, i.e. not changing any field of self. def digest(self): m =3D (self.h + self.s) % 2**128 return m.to_bytes(16, byteorder=3D'little') =20 +POLYVAL_POLY =3D sum((1 << i) for i in [128, 127, 126, 121, 0]) +POLYVAL_BLOCK_SIZE =3D 16 + +# A straightforward, unoptimized implementation of POLYVAL. +# Reference: https://datatracker.ietf.org/doc/html/rfc8452 +class Polyval: + def __init__(self, key): + assert len(key) =3D=3D 16 + self.h =3D int.from_bytes(key, byteorder=3D'little') + self.acc =3D 0 + + # Note: this supports partial blocks only at the end. + def update(self, data): + for i in range(0, len(data), 16): + # acc +=3D block + self.acc ^=3D int.from_bytes(data[i:i+16], byteorder=3D'little= ') + # acc =3D (acc * h * x^-128) mod POLYVAL_POLY + product =3D 0 + for j in range(128): + if (self.h & (1 << j)) !=3D 0: + product ^=3D self.acc << j + if (product & (1 << j)) !=3D 0: + product ^=3D POLYVAL_POLY << j + self.acc =3D product >> 128 + return self + + def digest(self): + return self.acc.to_bytes(16, byteorder=3D'little') + def hash_init(alg): if alg =3D=3D 'poly1305': # Use a fixed random key here, to present Poly1305 as an unkeyed h= ash. # This allows all the test cases for unkeyed hashes to work on Pol= y1305. return Poly1305(rand_bytes(POLY1305_KEY_SIZE)) + if alg =3D=3D 'polyval': + return Polyval(rand_bytes(POLYVAL_BLOCK_SIZE)) return hashlib.new(alg) =20 def hash_update(ctx, data): ctx.update(data) =20 @@ -163,13 +194,22 @@ def gen_additional_poly1305_testvecs(): data +=3D ctx.digest() print_static_u8_array_definition( 'poly1305_allones_macofmacs[POLY1305_DIGEST_SIZE]', Poly1305(key).update(data).digest()) =20 +def gen_additional_polyval_testvecs(): + key =3D b'\xff' * POLYVAL_BLOCK_SIZE + hashes =3D b'' + for data_len in range(0, 4097, 16): + hashes +=3D Polyval(key).update(b'\xff' * data_len).digest() + print_static_u8_array_definition( + 'polyval_allones_hashofhashes[POLYVAL_DIGEST_SIZE]', + Polyval(key).update(hashes).digest()) + if len(sys.argv) !=3D 2: sys.stderr.write('Usage: gen-hash-testvecs.py ALGORITHM\n') - sys.stderr.write('ALGORITHM may be any supported by Python hashlib, or= poly1305 or sha3.\n') + sys.stderr.write('ALGORITHM may be any supported by Python hashlib; or= poly1305, polyval, or sha3.\n') sys.stderr.write('Example: gen-hash-testvecs.py sha512\n') sys.exit(1) =20 alg =3D sys.argv[1] print('/* SPDX-License-Identifier: GPL-2.0-or-later */') @@ -178,10 +218,13 @@ if alg.startswith('blake2'): gen_unkeyed_testvecs(alg) gen_additional_blake2_testvecs(alg) elif alg =3D=3D 'poly1305': gen_unkeyed_testvecs(alg) gen_additional_poly1305_testvecs() +elif alg =3D=3D 'polyval': + gen_unkeyed_testvecs(alg) + gen_additional_polyval_testvecs() elif alg =3D=3D 'sha3': print() print('/* SHA3-256 test vectors */') gen_unkeyed_testvecs('sha3-256') print() --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99BCF27586C; Sun, 9 Nov 2025 23:49:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732198; cv=none; b=DXz2gEh2oiu/53v1tz+z/TN2InISlH+L9ZlFT6qW/tJu5WKrQIvqY7/7rN0aRZ/ZEiky3SKWSMs7v2XkQxBWWR6zdz2Dqp/xWqJfeuecXVMVydaYZRlSNCsTR0Ro7wKET929LDDzpyzYtIQ/x5g21EKSAFpfPrfSEPQfi/R2McI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732198; c=relaxed/simple; bh=8e2kAyHo72Xgi4j2u5Y3CXtlQQMKjJmSz6jydmNZnxY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lknWclMlcXwq192pLmGvJwzEkNJ3ukW9zUup//6J++wR+3MHFfcv0uQUBTQZVpjTHW2VxzDeKYftATbiXF0Xa30dwmE4aQ1Wz7PVNTwIPLBYueGyvlFQZZv/gMXrY8uv5teHPjACdZlkW8QM0zGS6R51pc9FEjAs5nH5qU7mHCI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ofoRsJIK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ofoRsJIK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 458C1C2BC86; Sun, 9 Nov 2025 23:49:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732198; bh=8e2kAyHo72Xgi4j2u5Y3CXtlQQMKjJmSz6jydmNZnxY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ofoRsJIK5A99kKzkhQCZYQBjBXFcHhBgj3INXraIhnakfH58TtBA+RHWMS8uHJkam g+MQbJK1sIO1GQEyNkUwdYT8nv5VcKf2ncwR3DAdVOzCtDTuZUN4iZwhTjXKnexEGX d6g9S1yVdZKTABsEgUjYpdC/DolxCElel55yHFYt1y5u0dpXqPDCLIGUgQhsnFnJZg kb+ahNHmDWwvS/MPyRLW9+ktbnSMdgQ+E8Fit70rsKf2tE7zKq9Q2FhWhGQsXnXqiu XswnLIbeL/5Hd4hS8l8g7mIFxb22TxaQ2a4mXg+crgo5vEr9fnKTG81bsebDDaAPjf IrHz1f2SE6EnQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 4/9] lib/crypto: arm64/polyval: Migrate optimized code into library Date: Sun, 9 Nov 2025 15:47:19 -0800 Message-ID: <20251109234726.638437-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Migrate the arm64 implementation of POLYVAL into lib/crypto/, wiring it up to the POLYVAL library interface. This makes the POLYVAL library be properly optimized on arm64. This drops the arm64 optimizations of polyval in the crypto_shash API. That's fine, since polyval will be removed from crypto_shash entirely since it is unneeded there. But even if it comes back, the crypto_shash API could just be implemented on top of the library API, as usual. Adjust the names and prototypes of the assembly functions to align more closely with the rest of the library code. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 10 -- arch/arm64/crypto/Makefile | 3 - arch/arm64/crypto/polyval-ce-glue.c | 158 ------------------ include/crypto/polyval.h | 8 + lib/crypto/Kconfig | 1 + lib/crypto/Makefile | 1 + .../crypto/arm64}/polyval-ce-core.S | 38 ++--- lib/crypto/arm64/polyval.h | 82 +++++++++ 8 files changed, 110 insertions(+), 191 deletions(-) delete mode 100644 arch/arm64/crypto/polyval-ce-glue.c rename {arch/arm64/crypto =3D> lib/crypto/arm64}/polyval-ce-core.S (92%) create mode 100644 lib/crypto/arm64/polyval.h diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index 376d6b50743f..bdd276a6e540 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -45,20 +45,10 @@ config CRYPTO_SM3_ARM64_CE SM3 (ShangMi 3) secure hash function (OSCCA GM/T 0004-2012) =20 Architecture: arm64 using: - ARMv8.2 Crypto Extensions =20 -config CRYPTO_POLYVAL_ARM64_CE - tristate "Hash functions: POLYVAL (ARMv8 Crypto Extensions)" - depends on KERNEL_MODE_NEON - select CRYPTO_POLYVAL - help - POLYVAL hash function for HCTR2 - - Architecture: arm64 using: - - ARMv8 Crypto Extensions - config CRYPTO_AES_ARM64 tristate "Ciphers: AES, modes: ECB, CBC, CTR, CTS, XCTR, XTS" select CRYPTO_AES select CRYPTO_LIB_SHA256 help diff --git a/arch/arm64/crypto/Makefile b/arch/arm64/crypto/Makefile index fd3d590fa113..1e330aa08d3f 100644 --- a/arch/arm64/crypto/Makefile +++ b/arch/arm64/crypto/Makefile @@ -27,13 +27,10 @@ obj-$(CONFIG_CRYPTO_SM4_ARM64_NEON_BLK) +=3D sm4-neon.o sm4-neon-y :=3D sm4-neon-glue.o sm4-neon-core.o =20 obj-$(CONFIG_CRYPTO_GHASH_ARM64_CE) +=3D ghash-ce.o ghash-ce-y :=3D ghash-ce-glue.o ghash-ce-core.o =20 -obj-$(CONFIG_CRYPTO_POLYVAL_ARM64_CE) +=3D polyval-ce.o -polyval-ce-y :=3D polyval-ce-glue.o polyval-ce-core.o - obj-$(CONFIG_CRYPTO_AES_ARM64_CE) +=3D aes-ce-cipher.o aes-ce-cipher-y :=3D aes-ce-core.o aes-ce-glue.o =20 obj-$(CONFIG_CRYPTO_AES_ARM64_CE_CCM) +=3D aes-ce-ccm.o aes-ce-ccm-y :=3D aes-ce-ccm-glue.o aes-ce-ccm-core.o diff --git a/arch/arm64/crypto/polyval-ce-glue.c b/arch/arm64/crypto/polyva= l-ce-glue.c deleted file mode 100644 index c4e653688ea0..000000000000 --- a/arch/arm64/crypto/polyval-ce-glue.c +++ /dev/null @@ -1,158 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Glue code for POLYVAL using ARMv8 Crypto Extensions - * - * Copyright (c) 2007 Nokia Siemens Networks - Mikko Herranen - * Copyright (c) 2009 Intel Corp. - * Author: Huang Ying - * Copyright 2021 Google LLC - */ - -/* - * Glue code based on ghash-clmulni-intel_glue.c. - * - * This implementation of POLYVAL uses montgomery multiplication accelerat= ed by - * ARMv8 Crypto Extensions instructions to implement the finite field oper= ations. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define NUM_KEY_POWERS 8 - -struct polyval_tfm_ctx { - /* - * These powers must be in the order h^8, ..., h^1. - */ - u8 key_powers[NUM_KEY_POWERS][POLYVAL_BLOCK_SIZE]; -}; - -struct polyval_desc_ctx { - u8 buffer[POLYVAL_BLOCK_SIZE]; -}; - -asmlinkage void pmull_polyval_update(const struct polyval_tfm_ctx *keys, - const u8 *in, size_t nblocks, u8 *accumulator); -asmlinkage void pmull_polyval_mul(u8 *op1, const u8 *op2); - -static void internal_polyval_update(const struct polyval_tfm_ctx *keys, - const u8 *in, size_t nblocks, u8 *accumulator) -{ - kernel_neon_begin(); - pmull_polyval_update(keys, in, nblocks, accumulator); - kernel_neon_end(); -} - -static void internal_polyval_mul(u8 *op1, const u8 *op2) -{ - kernel_neon_begin(); - pmull_polyval_mul(op1, op2); - kernel_neon_end(); -} - -static int polyval_arm64_setkey(struct crypto_shash *tfm, - const u8 *key, unsigned int keylen) -{ - struct polyval_tfm_ctx *tctx =3D crypto_shash_ctx(tfm); - int i; - - if (keylen !=3D POLYVAL_BLOCK_SIZE) - return -EINVAL; - - memcpy(tctx->key_powers[NUM_KEY_POWERS-1], key, POLYVAL_BLOCK_SIZE); - - for (i =3D NUM_KEY_POWERS-2; i >=3D 0; i--) { - memcpy(tctx->key_powers[i], key, POLYVAL_BLOCK_SIZE); - internal_polyval_mul(tctx->key_powers[i], - tctx->key_powers[i+1]); - } - - return 0; -} - -static int polyval_arm64_init(struct shash_desc *desc) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - - memset(dctx, 0, sizeof(*dctx)); - - return 0; -} - -static int polyval_arm64_update(struct shash_desc *desc, - const u8 *src, unsigned int srclen) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - const struct polyval_tfm_ctx *tctx =3D crypto_shash_ctx(desc->tfm); - unsigned int nblocks; - - do { - /* allow rescheduling every 4K bytes */ - nblocks =3D min(srclen, 4096U) / POLYVAL_BLOCK_SIZE; - internal_polyval_update(tctx, src, nblocks, dctx->buffer); - srclen -=3D nblocks * POLYVAL_BLOCK_SIZE; - src +=3D nblocks * POLYVAL_BLOCK_SIZE; - } while (srclen >=3D POLYVAL_BLOCK_SIZE); - - return srclen; -} - -static int polyval_arm64_finup(struct shash_desc *desc, const u8 *src, - unsigned int len, u8 *dst) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - const struct polyval_tfm_ctx *tctx =3D crypto_shash_ctx(desc->tfm); - - if (len) { - crypto_xor(dctx->buffer, src, len); - internal_polyval_mul(dctx->buffer, - tctx->key_powers[NUM_KEY_POWERS-1]); - } - - memcpy(dst, dctx->buffer, POLYVAL_BLOCK_SIZE); - - return 0; -} - -static struct shash_alg polyval_alg =3D { - .digestsize =3D POLYVAL_DIGEST_SIZE, - .init =3D polyval_arm64_init, - .update =3D polyval_arm64_update, - .finup =3D polyval_arm64_finup, - .setkey =3D polyval_arm64_setkey, - .descsize =3D sizeof(struct polyval_desc_ctx), - .base =3D { - .cra_name =3D "polyval", - .cra_driver_name =3D "polyval-ce", - .cra_priority =3D 200, - .cra_flags =3D CRYPTO_AHASH_ALG_BLOCK_ONLY, - .cra_blocksize =3D POLYVAL_BLOCK_SIZE, - .cra_ctxsize =3D sizeof(struct polyval_tfm_ctx), - .cra_module =3D THIS_MODULE, - }, -}; - -static int __init polyval_ce_mod_init(void) -{ - return crypto_register_shash(&polyval_alg); -} - -static void __exit polyval_ce_mod_exit(void) -{ - crypto_unregister_shash(&polyval_alg); -} - -module_cpu_feature_match(PMULL, polyval_ce_mod_init) -module_exit(polyval_ce_mod_exit); - -MODULE_LICENSE("GPL"); -MODULE_DESCRIPTION("POLYVAL hash function accelerated by ARMv8 Crypto Exte= nsions"); -MODULE_ALIAS_CRYPTO("polyval"); -MODULE_ALIAS_CRYPTO("polyval-ce"); diff --git a/include/crypto/polyval.h b/include/crypto/polyval.h index 5ba4c248cad1..f8aaf4275fbd 100644 --- a/include/crypto/polyval.h +++ b/include/crypto/polyval.h @@ -37,14 +37,22 @@ struct polyval_elem { * struct polyval_key - Prepared key for POLYVAL * * This may contain just the raw key H, or it may contain precomputed key * powers, depending on the platform's POLYVAL implementation. Use * polyval_preparekey() to initialize this. + * + * By H^i we mean H^(i-1) * H * x^-128, with base case H^1 =3D H. I.e. the + * exponentiation repeats the POLYVAL dot operation, with its "extra" x^-1= 28. */ struct polyval_key { #ifdef CONFIG_CRYPTO_LIB_POLYVAL_ARCH +#ifdef CONFIG_ARM64 + /** @h_powers: Powers of the hash key H^8 through H^1 */ + struct polyval_elem h_powers[8]; +#else #error "Unhandled arch" +#endif #else /* CONFIG_CRYPTO_LIB_POLYVAL_ARCH */ /** @h: The hash key H */ struct polyval_elem h; #endif /* !CONFIG_CRYPTO_LIB_POLYVAL_ARCH */ }; diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig index 6545f0e83b83..430723994142 100644 --- a/lib/crypto/Kconfig +++ b/lib/crypto/Kconfig @@ -142,10 +142,11 @@ config CRYPTO_LIB_POLYVAL the functions from . =20 config CRYPTO_LIB_POLYVAL_ARCH bool depends on CRYPTO_LIB_POLYVAL && !UML + default y if ARM64 && KERNEL_MODE_NEON =20 config CRYPTO_LIB_CHACHA20POLY1305 tristate select CRYPTO_LIB_CHACHA select CRYPTO_LIB_POLY1305 diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 055e44008805..2efa96afcb4b 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -200,10 +200,11 @@ clean-files +=3D arm/poly1305-core.S \ =20 obj-$(CONFIG_CRYPTO_LIB_POLYVAL) +=3D libpolyval.o libpolyval-y :=3D polyval.o ifeq ($(CONFIG_CRYPTO_LIB_POLYVAL_ARCH),y) CFLAGS_polyval.o +=3D -I$(src)/$(SRCARCH) +libpolyval-$(CONFIG_ARM64) +=3D arm64/polyval-ce-core.o endif =20 ##########################################################################= ###### =20 obj-$(CONFIG_CRYPTO_LIB_SHA1) +=3D libsha1.o diff --git a/arch/arm64/crypto/polyval-ce-core.S b/lib/crypto/arm64/polyval= -ce-core.S similarity index 92% rename from arch/arm64/crypto/polyval-ce-core.S rename to lib/crypto/arm64/polyval-ce-core.S index b5326540d2e3..7c731a044d02 100644 --- a/arch/arm64/crypto/polyval-ce-core.S +++ b/lib/crypto/arm64/polyval-ce-core.S @@ -25,14 +25,14 @@ */ =20 #include #define STRIDE_BLOCKS 8 =20 -KEY_POWERS .req x0 -MSG .req x1 -BLOCKS_LEFT .req x2 -ACCUMULATOR .req x3 +ACCUMULATOR .req x0 +KEY_POWERS .req x1 +MSG .req x2 +BLOCKS_LEFT .req x3 KEY_START .req x10 EXTRA_BYTES .req x11 TMP .req x13 =20 M0 .req v0 @@ -298,44 +298,42 @@ GSTAR .req v24 karatsuba2 montgomery_reduction SUM .endm =20 /* - * Perform montgomery multiplication in GF(2^128) and store result in op1. + * Computes a =3D a * b * x^{-128} mod x^128 + x^127 + x^126 + x^121 + 1. * - * Computes op1*op2*x^{-128} mod x^128 + x^127 + x^126 + x^121 + 1 - * If op1, op2 are in montgomery form, this computes the montgomery - * form of op1*op2. - * - * void pmull_polyval_mul(u8 *op1, const u8 *op2); + * void polyval_mul_pmull(struct polyval_elem *a, + * const struct polyval_elem *b); */ -SYM_FUNC_START(pmull_polyval_mul) +SYM_FUNC_START(polyval_mul_pmull) adr TMP, .Lgstar ld1 {GSTAR.2d}, [TMP] ld1 {v0.16b}, [x0] ld1 {v1.16b}, [x1] karatsuba1_store v0 v1 karatsuba2 montgomery_reduction SUM st1 {SUM.16b}, [x0] ret -SYM_FUNC_END(pmull_polyval_mul) +SYM_FUNC_END(polyval_mul_pmull) =20 /* * Perform polynomial evaluation as specified by POLYVAL. This computes: * h^n * accumulator + h^n * m_0 + ... + h^1 * m_{n-1} * where n=3Dnblocks, h is the hash key, and m_i are the message blocks. * - * x0 - pointer to precomputed key powers h^8 ... h^1 - * x1 - pointer to message blocks - * x2 - number of blocks to hash - * x3 - pointer to accumulator + * x0 - pointer to accumulator + * x1 - pointer to precomputed key powers h^8 ... h^1 + * x2 - pointer to message blocks + * x3 - number of blocks to hash * - * void pmull_polyval_update(const struct polyval_ctx *ctx, const u8 *in, - * size_t nblocks, u8 *accumulator); + * void polyval_blocks_pmull(struct polyval_elem *acc, + * const struct polyval_key *key, + * const u8 *data, size_t nblocks); */ -SYM_FUNC_START(pmull_polyval_update) +SYM_FUNC_START(polyval_blocks_pmull) adr TMP, .Lgstar mov KEY_START, KEY_POWERS ld1 {GSTAR.2d}, [TMP] ld1 {SUM.16b}, [ACCUMULATOR] subs BLOCKS_LEFT, BLOCKS_LEFT, #STRIDE_BLOCKS @@ -356,6 +354,6 @@ SYM_FUNC_START(pmull_polyval_update) beq .LskipPartial partial_stride .LskipPartial: st1 {SUM.16b}, [ACCUMULATOR] ret -SYM_FUNC_END(pmull_polyval_update) +SYM_FUNC_END(polyval_blocks_pmull) diff --git a/lib/crypto/arm64/polyval.h b/lib/crypto/arm64/polyval.h new file mode 100644 index 000000000000..2486e80750d0 --- /dev/null +++ b/lib/crypto/arm64/polyval.h @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * POLYVAL library functions, arm64 optimized + * + * Copyright 2025 Google LLC + */ +#include +#include +#include + +#define NUM_H_POWERS 8 + +static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pmull); + +asmlinkage void polyval_mul_pmull(struct polyval_elem *a, + const struct polyval_elem *b); +asmlinkage void polyval_blocks_pmull(struct polyval_elem *acc, + const struct polyval_key *key, + const u8 *data, size_t nblocks); + +static void polyval_preparekey_arch(struct polyval_key *key, + const u8 raw_key[POLYVAL_BLOCK_SIZE]) +{ + static_assert(ARRAY_SIZE(key->h_powers) =3D=3D NUM_H_POWERS); + memcpy(&key->h_powers[NUM_H_POWERS - 1], raw_key, POLYVAL_BLOCK_SIZE); + if (static_branch_likely(&have_pmull) && may_use_simd()) { + kernel_neon_begin(); + for (int i =3D NUM_H_POWERS - 2; i >=3D 0; i--) { + key->h_powers[i] =3D key->h_powers[i + 1]; + polyval_mul_pmull(&key->h_powers[i], + &key->h_powers[NUM_H_POWERS - 1]); + } + kernel_neon_end(); + } else { + for (int i =3D NUM_H_POWERS - 2; i >=3D 0; i--) { + key->h_powers[i] =3D key->h_powers[i + 1]; + polyval_mul_generic(&key->h_powers[i], + &key->h_powers[NUM_H_POWERS - 1]); + } + } +} + +static void polyval_mul_arch(struct polyval_elem *acc, + const struct polyval_key *key) +{ + if (static_branch_likely(&have_pmull) && may_use_simd()) { + kernel_neon_begin(); + polyval_mul_pmull(acc, &key->h_powers[NUM_H_POWERS - 1]); + kernel_neon_end(); + } else { + polyval_mul_generic(acc, &key->h_powers[NUM_H_POWERS - 1]); + } +} + +static void polyval_blocks_arch(struct polyval_elem *acc, + const struct polyval_key *key, + const u8 *data, size_t nblocks) +{ + if (static_branch_likely(&have_pmull) && may_use_simd()) { + do { + /* Allow rescheduling every 4 KiB. */ + size_t n =3D min_t(size_t, nblocks, + 4096 / POLYVAL_BLOCK_SIZE); + + kernel_neon_begin(); + polyval_blocks_pmull(acc, key, data, n); + kernel_neon_end(); + data +=3D n * POLYVAL_BLOCK_SIZE; + nblocks -=3D n; + } while (nblocks); + } else { + polyval_blocks_generic(acc, &key->h_powers[NUM_H_POWERS - 1], + data, nblocks); + } +} + +#define polyval_mod_init_arch polyval_mod_init_arch +static void polyval_mod_init_arch(void) +{ + if (cpu_have_named_feature(PMULL)) + static_branch_enable(&have_pmull); +} --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44B45277818; Sun, 9 Nov 2025 23:49:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732199; cv=none; b=DpO3Y/OW2zgNGlvMMMstlH1s3vIbj5xW6I+nkKZ7Ztbp99FLjj8dFsth7DcY2ONNQ6KH7UoQ/5TdYbV0aWydOI3a+OUNWKuJVx+ViDTtmYlhz/7CKXLII75+B84csA1DaNJYdncNol95aNpACoM8k1LPyn8/5GPlIWHdNQkKb5Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732199; c=relaxed/simple; bh=GOujaf6AeVir3vIRStzcGHH3O67wB6Z4gyjbS/4cdVA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r/mg/BbSe9N4tY5fNsRWyCXosagMVWWuk71iKGWqnwjHIHxXAaiNRh44hW+WP0DpeEwPtUSG/raHpf0maJqZ0AGqBU9nm8w6rSd7H5RX1beUcl3n7l6IYEQ6sC0T2KH4OV5MubIMqDCQzlhfqKcL7q/R17xR9m6FSu04V9vbfEc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=V7da6BLt; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="V7da6BLt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E695C19421; Sun, 9 Nov 2025 23:49:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732198; bh=GOujaf6AeVir3vIRStzcGHH3O67wB6Z4gyjbS/4cdVA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=V7da6BLtzTRjctamKJO7nKgshS+quh735NM/0n2RAjRdZA70O9KA4kteQp+yMyh+H 3GmDJU6ndYRnz9XkPXDFC7K5fs4Xf52SVu76FVtgD8uV5ErMTdeTE1ztub7FAIp3Av 7JE89bCftZ74jbeXt1LBFThSdBMgGvjq5hnNaohu4ecN0iIS+ym6yI6ANAE8vZeiWm PC9dtOQghKFcDUGYRrTUeYJu0xCKp64iovnEFJE8FPKym9iPDbNFljlBIDvnXMjFnp Zkiks5iuIy6sSj1Pu+DU9lVuHmu3jdslC5FTAF8cgplpiqL0bGXN4XqPMfcWtIHsFj 1PA+JhHWdo+Ig== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 5/9] lib/crypto: x86/polyval: Migrate optimized code into library Date: Sun, 9 Nov 2025 15:47:20 -0800 Message-ID: <20251109234726.638437-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Migrate the x86_64 implementation of POLYVAL into lib/crypto/, wiring it up to the POLYVAL library interface. This makes the POLYVAL library be properly optimized on x86_64. This drops the x86_64 optimizations of polyval in the crypto_shash API. That's fine, since polyval will be removed from crypto_shash entirely since it is unneeded there. But even if it comes back, the crypto_shash API could just be implemented on top of the library API, as usual. Adjust the names and prototypes of the assembly functions to align more closely with the rest of the library code. Also replace a movaps instruction with movups to remove the assumption that the key struct is 16-byte aligned. Users can still align the key if they want (and at least in this case, movups is just as fast as movaps), but it's inconvenient to require it. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- arch/x86/crypto/Kconfig | 10 - arch/x86/crypto/Makefile | 3 - arch/x86/crypto/polyval-clmulni_glue.c | 180 ------------------ include/crypto/polyval.h | 3 + lib/crypto/Kconfig | 1 + lib/crypto/Makefile | 1 + .../crypto/x86/polyval-pclmul-avx.S | 40 ++-- lib/crypto/x86/polyval.h | 83 ++++++++ 8 files changed, 107 insertions(+), 214 deletions(-) delete mode 100644 arch/x86/crypto/polyval-clmulni_glue.c rename arch/x86/crypto/polyval-clmulni_asm.S =3D> lib/crypto/x86/polyval-p= clmul-avx.S (91%) create mode 100644 lib/crypto/x86/polyval.h diff --git a/arch/x86/crypto/Kconfig b/arch/x86/crypto/Kconfig index 48d3076b6053..3fd2423d3cf8 100644 --- a/arch/x86/crypto/Kconfig +++ b/arch/x86/crypto/Kconfig @@ -351,20 +351,10 @@ config CRYPTO_NHPOLY1305_AVX2 NHPoly1305 hash function for Adiantum =20 Architecture: x86_64 using: - AVX2 (Advanced Vector Extensions 2) =20 -config CRYPTO_POLYVAL_CLMUL_NI - tristate "Hash functions: POLYVAL (CLMUL-NI)" - depends on 64BIT - select CRYPTO_POLYVAL - help - POLYVAL hash function for HCTR2 - - Architecture: x86_64 using: - - CLMUL-NI (carry-less multiplication new instructions) - config CRYPTO_SM3_AVX_X86_64 tristate "Hash functions: SM3 (AVX)" depends on 64BIT select CRYPTO_HASH select CRYPTO_LIB_SM3 diff --git a/arch/x86/crypto/Makefile b/arch/x86/crypto/Makefile index 6409e3009524..5f2fb4f148fe 100644 --- a/arch/x86/crypto/Makefile +++ b/arch/x86/crypto/Makefile @@ -51,13 +51,10 @@ aesni-intel-$(CONFIG_64BIT) +=3D aes-ctr-avx-x86_64.o \ aes-xts-avx-x86_64.o =20 obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) +=3D ghash-clmulni-intel.o ghash-clmulni-intel-y :=3D ghash-clmulni-intel_asm.o ghash-clmulni-intel_g= lue.o =20 -obj-$(CONFIG_CRYPTO_POLYVAL_CLMUL_NI) +=3D polyval-clmulni.o -polyval-clmulni-y :=3D polyval-clmulni_asm.o polyval-clmulni_glue.o - obj-$(CONFIG_CRYPTO_NHPOLY1305_SSE2) +=3D nhpoly1305-sse2.o nhpoly1305-sse2-y :=3D nh-sse2-x86_64.o nhpoly1305-sse2-glue.o obj-$(CONFIG_CRYPTO_NHPOLY1305_AVX2) +=3D nhpoly1305-avx2.o nhpoly1305-avx2-y :=3D nh-avx2-x86_64.o nhpoly1305-avx2-glue.o =20 diff --git a/arch/x86/crypto/polyval-clmulni_glue.c b/arch/x86/crypto/polyv= al-clmulni_glue.c deleted file mode 100644 index 6b466867f91a..000000000000 --- a/arch/x86/crypto/polyval-clmulni_glue.c +++ /dev/null @@ -1,180 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Glue code for POLYVAL using PCMULQDQ-NI - * - * Copyright (c) 2007 Nokia Siemens Networks - Mikko Herranen - * Copyright (c) 2009 Intel Corp. - * Author: Huang Ying - * Copyright 2021 Google LLC - */ - -/* - * Glue code based on ghash-clmulni-intel_glue.c. - * - * This implementation of POLYVAL uses montgomery multiplication - * accelerated by PCLMULQDQ-NI to implement the finite field - * operations. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define POLYVAL_ALIGN 16 -#define POLYVAL_ALIGN_ATTR __aligned(POLYVAL_ALIGN) -#define POLYVAL_ALIGN_EXTRA ((POLYVAL_ALIGN - 1) & ~(CRYPTO_MINALIGN - 1)) -#define POLYVAL_CTX_SIZE (sizeof(struct polyval_tfm_ctx) + POLYVAL_ALIGN_E= XTRA) -#define NUM_KEY_POWERS 8 - -struct polyval_tfm_ctx { - /* - * These powers must be in the order h^8, ..., h^1. - */ - u8 key_powers[NUM_KEY_POWERS][POLYVAL_BLOCK_SIZE] POLYVAL_ALIGN_ATTR; -}; - -struct polyval_desc_ctx { - u8 buffer[POLYVAL_BLOCK_SIZE]; -}; - -asmlinkage void clmul_polyval_update(const struct polyval_tfm_ctx *keys, - const u8 *in, size_t nblocks, u8 *accumulator); -asmlinkage void clmul_polyval_mul(u8 *op1, const u8 *op2); - -static inline struct polyval_tfm_ctx *polyval_tfm_ctx(struct crypto_shash = *tfm) -{ - return PTR_ALIGN(crypto_shash_ctx(tfm), POLYVAL_ALIGN); -} - -static void internal_polyval_update(const struct polyval_tfm_ctx *keys, - const u8 *in, size_t nblocks, u8 *accumulator) -{ - kernel_fpu_begin(); - clmul_polyval_update(keys, in, nblocks, accumulator); - kernel_fpu_end(); -} - -static void internal_polyval_mul(u8 *op1, const u8 *op2) -{ - kernel_fpu_begin(); - clmul_polyval_mul(op1, op2); - kernel_fpu_end(); -} - -static int polyval_x86_setkey(struct crypto_shash *tfm, - const u8 *key, unsigned int keylen) -{ - struct polyval_tfm_ctx *tctx =3D polyval_tfm_ctx(tfm); - int i; - - if (keylen !=3D POLYVAL_BLOCK_SIZE) - return -EINVAL; - - memcpy(tctx->key_powers[NUM_KEY_POWERS-1], key, POLYVAL_BLOCK_SIZE); - - for (i =3D NUM_KEY_POWERS-2; i >=3D 0; i--) { - memcpy(tctx->key_powers[i], key, POLYVAL_BLOCK_SIZE); - internal_polyval_mul(tctx->key_powers[i], - tctx->key_powers[i+1]); - } - - return 0; -} - -static int polyval_x86_init(struct shash_desc *desc) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - - memset(dctx, 0, sizeof(*dctx)); - - return 0; -} - -static int polyval_x86_update(struct shash_desc *desc, - const u8 *src, unsigned int srclen) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - const struct polyval_tfm_ctx *tctx =3D polyval_tfm_ctx(desc->tfm); - unsigned int nblocks; - - do { - /* Allow rescheduling every 4K bytes. */ - nblocks =3D min(srclen, 4096U) / POLYVAL_BLOCK_SIZE; - internal_polyval_update(tctx, src, nblocks, dctx->buffer); - srclen -=3D nblocks * POLYVAL_BLOCK_SIZE; - src +=3D nblocks * POLYVAL_BLOCK_SIZE; - } while (srclen >=3D POLYVAL_BLOCK_SIZE); - - return srclen; -} - -static int polyval_x86_finup(struct shash_desc *desc, const u8 *src, - unsigned int len, u8 *dst) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - const struct polyval_tfm_ctx *tctx =3D polyval_tfm_ctx(desc->tfm); - - if (len) { - crypto_xor(dctx->buffer, src, len); - internal_polyval_mul(dctx->buffer, - tctx->key_powers[NUM_KEY_POWERS-1]); - } - - memcpy(dst, dctx->buffer, POLYVAL_BLOCK_SIZE); - - return 0; -} - -static struct shash_alg polyval_alg =3D { - .digestsize =3D POLYVAL_DIGEST_SIZE, - .init =3D polyval_x86_init, - .update =3D polyval_x86_update, - .finup =3D polyval_x86_finup, - .setkey =3D polyval_x86_setkey, - .descsize =3D sizeof(struct polyval_desc_ctx), - .base =3D { - .cra_name =3D "polyval", - .cra_driver_name =3D "polyval-clmulni", - .cra_priority =3D 200, - .cra_flags =3D CRYPTO_AHASH_ALG_BLOCK_ONLY, - .cra_blocksize =3D POLYVAL_BLOCK_SIZE, - .cra_ctxsize =3D POLYVAL_CTX_SIZE, - .cra_module =3D THIS_MODULE, - }, -}; - -__maybe_unused static const struct x86_cpu_id pcmul_cpu_id[] =3D { - X86_MATCH_FEATURE(X86_FEATURE_PCLMULQDQ, NULL), - {} -}; -MODULE_DEVICE_TABLE(x86cpu, pcmul_cpu_id); - -static int __init polyval_clmulni_mod_init(void) -{ - if (!x86_match_cpu(pcmul_cpu_id)) - return -ENODEV; - - if (!boot_cpu_has(X86_FEATURE_AVX)) - return -ENODEV; - - return crypto_register_shash(&polyval_alg); -} - -static void __exit polyval_clmulni_mod_exit(void) -{ - crypto_unregister_shash(&polyval_alg); -} - -module_init(polyval_clmulni_mod_init); -module_exit(polyval_clmulni_mod_exit); - -MODULE_LICENSE("GPL"); -MODULE_DESCRIPTION("POLYVAL hash function accelerated by PCLMULQDQ-NI"); -MODULE_ALIAS_CRYPTO("polyval"); -MODULE_ALIAS_CRYPTO("polyval-clmulni"); diff --git a/include/crypto/polyval.h b/include/crypto/polyval.h index f8aaf4275fbd..b28b8ef11353 100644 --- a/include/crypto/polyval.h +++ b/include/crypto/polyval.h @@ -46,10 +46,13 @@ struct polyval_elem { struct polyval_key { #ifdef CONFIG_CRYPTO_LIB_POLYVAL_ARCH #ifdef CONFIG_ARM64 /** @h_powers: Powers of the hash key H^8 through H^1 */ struct polyval_elem h_powers[8]; +#elif defined(CONFIG_X86) + /** @h_powers: Powers of the hash key H^8 through H^1 */ + struct polyval_elem h_powers[8]; #else #error "Unhandled arch" #endif #else /* CONFIG_CRYPTO_LIB_POLYVAL_ARCH */ /** @h: The hash key H */ diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig index 430723994142..9d04b3771ce2 100644 --- a/lib/crypto/Kconfig +++ b/lib/crypto/Kconfig @@ -143,10 +143,11 @@ config CRYPTO_LIB_POLYVAL =20 config CRYPTO_LIB_POLYVAL_ARCH bool depends on CRYPTO_LIB_POLYVAL && !UML default y if ARM64 && KERNEL_MODE_NEON + default y if X86_64 =20 config CRYPTO_LIB_CHACHA20POLY1305 tristate select CRYPTO_LIB_CHACHA select CRYPTO_LIB_POLY1305 diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 2efa96afcb4b..6580991f8e12 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -201,10 +201,11 @@ clean-files +=3D arm/poly1305-core.S \ obj-$(CONFIG_CRYPTO_LIB_POLYVAL) +=3D libpolyval.o libpolyval-y :=3D polyval.o ifeq ($(CONFIG_CRYPTO_LIB_POLYVAL_ARCH),y) CFLAGS_polyval.o +=3D -I$(src)/$(SRCARCH) libpolyval-$(CONFIG_ARM64) +=3D arm64/polyval-ce-core.o +libpolyval-$(CONFIG_X86) +=3D x86/polyval-pclmul-avx.o endif =20 ##########################################################################= ###### =20 obj-$(CONFIG_CRYPTO_LIB_SHA1) +=3D libsha1.o diff --git a/arch/x86/crypto/polyval-clmulni_asm.S b/lib/crypto/x86/polyval= -pclmul-avx.S similarity index 91% rename from arch/x86/crypto/polyval-clmulni_asm.S rename to lib/crypto/x86/polyval-pclmul-avx.S index a6ebe4e7dd2b..7f739465ad35 100644 --- a/arch/x86/crypto/polyval-clmulni_asm.S +++ b/lib/crypto/x86/polyval-pclmul-avx.S @@ -34,14 +34,14 @@ #define LO %xmm12 #define HI %xmm13 #define MI %xmm14 #define SUM %xmm15 =20 -#define KEY_POWERS %rdi -#define MSG %rsi -#define BLOCKS_LEFT %rdx -#define ACCUMULATOR %rcx +#define ACCUMULATOR %rdi +#define KEY_POWERS %rsi +#define MSG %rdx +#define BLOCKS_LEFT %rcx #define TMP %rax =20 .section .rodata.cst16.gstar, "aM", @progbits, 16 .align 16 =20 @@ -232,11 +232,11 @@ addq $(16*STRIDE_BLOCKS), KEY_POWERS subq TMP, KEY_POWERS =20 movups (MSG), %xmm0 pxor SUM, %xmm0 - movaps (KEY_POWERS), %xmm1 + movups (KEY_POWERS), %xmm1 schoolbook1_noload dec BLOCKS_LEFT addq $16, MSG addq $16, KEY_POWERS =20 @@ -259,45 +259,43 @@ schoolbook2 montgomery_reduction SUM .endm =20 /* - * Perform montgomery multiplication in GF(2^128) and store result in op1. + * Computes a =3D a * b * x^{-128} mod x^128 + x^127 + x^126 + x^121 + 1. * - * Computes op1*op2*x^{-128} mod x^128 + x^127 + x^126 + x^121 + 1 - * If op1, op2 are in montgomery form, this computes the montgomery - * form of op1*op2. - * - * void clmul_polyval_mul(u8 *op1, const u8 *op2); + * void polyval_mul_pclmul_avx(struct polyval_elem *a, + * const struct polyval_elem *b); */ -SYM_FUNC_START(clmul_polyval_mul) +SYM_FUNC_START(polyval_mul_pclmul_avx) FRAME_BEGIN vmovdqa .Lgstar(%rip), GSTAR movups (%rdi), %xmm0 movups (%rsi), %xmm1 schoolbook1_noload schoolbook2 montgomery_reduction SUM movups SUM, (%rdi) FRAME_END RET -SYM_FUNC_END(clmul_polyval_mul) +SYM_FUNC_END(polyval_mul_pclmul_avx) =20 /* * Perform polynomial evaluation as specified by POLYVAL. This computes: * h^n * accumulator + h^n * m_0 + ... + h^1 * m_{n-1} * where n=3Dnblocks, h is the hash key, and m_i are the message blocks. * - * rdi - pointer to precomputed key powers h^8 ... h^1 - * rsi - pointer to message blocks - * rdx - number of blocks to hash - * rcx - pointer to the accumulator + * rdi - pointer to the accumulator + * rsi - pointer to precomputed key powers h^8 ... h^1 + * rdx - pointer to message blocks + * rcx - number of blocks to hash * - * void clmul_polyval_update(const struct polyval_tfm_ctx *keys, - * const u8 *in, size_t nblocks, u8 *accumulator); + * void polyval_blocks_pclmul_avx(struct polyval_elem *acc, + * const struct polyval_key *key, + * const u8 *data, size_t nblocks); */ -SYM_FUNC_START(clmul_polyval_update) +SYM_FUNC_START(polyval_blocks_pclmul_avx) FRAME_BEGIN vmovdqa .Lgstar(%rip), GSTAR movups (ACCUMULATOR), SUM subq $STRIDE_BLOCKS, BLOCKS_LEFT js .LstrideLoopExit @@ -316,6 +314,6 @@ SYM_FUNC_START(clmul_polyval_update) partial_stride .LskipPartial: movups SUM, (ACCUMULATOR) FRAME_END RET -SYM_FUNC_END(clmul_polyval_update) +SYM_FUNC_END(polyval_blocks_pclmul_avx) diff --git a/lib/crypto/x86/polyval.h b/lib/crypto/x86/polyval.h new file mode 100644 index 000000000000..ef8797521420 --- /dev/null +++ b/lib/crypto/x86/polyval.h @@ -0,0 +1,83 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * POLYVAL library functions, x86_64 optimized + * + * Copyright 2025 Google LLC + */ +#include +#include + +#define NUM_H_POWERS 8 + +static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_pclmul_avx); + +asmlinkage void polyval_mul_pclmul_avx(struct polyval_elem *a, + const struct polyval_elem *b); +asmlinkage void polyval_blocks_pclmul_avx(struct polyval_elem *acc, + const struct polyval_key *key, + const u8 *data, size_t nblocks); + +static void polyval_preparekey_arch(struct polyval_key *key, + const u8 raw_key[POLYVAL_BLOCK_SIZE]) +{ + static_assert(ARRAY_SIZE(key->h_powers) =3D=3D NUM_H_POWERS); + memcpy(&key->h_powers[NUM_H_POWERS - 1], raw_key, POLYVAL_BLOCK_SIZE); + if (static_branch_likely(&have_pclmul_avx) && irq_fpu_usable()) { + kernel_fpu_begin(); + for (int i =3D NUM_H_POWERS - 2; i >=3D 0; i--) { + key->h_powers[i] =3D key->h_powers[i + 1]; + polyval_mul_pclmul_avx( + &key->h_powers[i], + &key->h_powers[NUM_H_POWERS - 1]); + } + kernel_fpu_end(); + } else { + for (int i =3D NUM_H_POWERS - 2; i >=3D 0; i--) { + key->h_powers[i] =3D key->h_powers[i + 1]; + polyval_mul_generic(&key->h_powers[i], + &key->h_powers[NUM_H_POWERS - 1]); + } + } +} + +static void polyval_mul_arch(struct polyval_elem *acc, + const struct polyval_key *key) +{ + if (static_branch_likely(&have_pclmul_avx) && irq_fpu_usable()) { + kernel_fpu_begin(); + polyval_mul_pclmul_avx(acc, &key->h_powers[NUM_H_POWERS - 1]); + kernel_fpu_end(); + } else { + polyval_mul_generic(acc, &key->h_powers[NUM_H_POWERS - 1]); + } +} + +static void polyval_blocks_arch(struct polyval_elem *acc, + const struct polyval_key *key, + const u8 *data, size_t nblocks) +{ + if (static_branch_likely(&have_pclmul_avx) && irq_fpu_usable()) { + do { + /* Allow rescheduling every 4 KiB. */ + size_t n =3D min_t(size_t, nblocks, + 4096 / POLYVAL_BLOCK_SIZE); + + kernel_fpu_begin(); + polyval_blocks_pclmul_avx(acc, key, data, n); + kernel_fpu_end(); + data +=3D n * POLYVAL_BLOCK_SIZE; + nblocks -=3D n; + } while (nblocks); + } else { + polyval_blocks_generic(acc, &key->h_powers[NUM_H_POWERS - 1], + data, nblocks); + } +} + +#define polyval_mod_init_arch polyval_mod_init_arch +static void polyval_mod_init_arch(void) +{ + if (boot_cpu_has(X86_FEATURE_PCLMULQDQ) && + boot_cpu_has(X86_FEATURE_AVX)) + static_branch_enable(&have_pclmul_avx); +} --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93339279334; Sun, 9 Nov 2025 23:49:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732199; cv=none; b=oUpOngq0ajoJZSTohvMzHgjDF1juEl12DcKh6Z0/BTDLgBcWoYX7iYWCen3gtEwEwWTG/2k+C4HwNCVmhaxDNi4/OnfGxQ5KKSxWKssIMjUp8cxmmMOhzb/lWX46+utfdtU4zbirlV44kxN05VsukYh0cBNHbIrqYBXRmSOKP+U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732199; c=relaxed/simple; bh=2leaOrFbqznRLU4+BYb8NBfLyGhNn4xW0et6YGcfWf0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MCosg9xpK0RWU4tCmOXJXcEChU8cexsyP3ma5crY1cu/fF4vMJoAJFf9rOBZYOwn63APVCZ5/AXoQB0cAlc9Dz0l+X1fitINrjLpoby8hXdStZ65RSA0FYJ+7cMUibEJEVYikxQsbTL3l5qK3U6Kqu8TUNteIeKk9e3d0euL0VY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mARffpHe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mARffpHe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02C42C19425; Sun, 9 Nov 2025 23:49:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732199; bh=2leaOrFbqznRLU4+BYb8NBfLyGhNn4xW0et6YGcfWf0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mARffpHeuuaMDLSViepYaTkFEu5QarpqxY0LviMNOxMIwif5k6YxybXIF3mzEbLrl oBsEeqiIOYlzR4Buc/0be/5VePH5lxDqJ3Krx7tuP8kEu7ayaBSrChCAOmPSrmJa6P e8WuAB36oZ4QbNuGb6keNbyNoOcEG9/xiUwRQdNRjXEaa/xRSijQQiSgyJ/VryRJVm 5jjcA5SmsxYkPAgl2JDcQXQgppBebACP9B10DVqhEoBs0m3IBfmuiAga2T/HpxUOT+ ua+W935IHxxn8eQnZeC0tk33I52ZiudGlvKY53ZNkDo6eC82VkisbkZGMT12O5f9uO EuuLLTbdPpCWA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 6/9] crypto: hctr2 - Convert to use POLYVAL library Date: Sun, 9 Nov 2025 15:47:21 -0800 Message-ID: <20251109234726.638437-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The "hash function" in hctr2 is fixed at POLYVAL; it can never vary. Just use the POLYVAL library, which is much easier to use than the crypto_shash API. It's faster, uses fixed-size structs, and never fails (all the functions return void). Note that this eliminates the only known user of the polyval support in crypto_shash. A later commit will remove support for polyval from crypto_shash, given that the library API is sufficient. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- crypto/Kconfig | 2 +- crypto/hctr2.c | 226 ++++++++++++++--------------------------------- crypto/testmgr.c | 3 +- 3 files changed, 66 insertions(+), 165 deletions(-) diff --git a/crypto/Kconfig b/crypto/Kconfig index 57b85e903cf0..805172f75bf1 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -694,11 +694,11 @@ config CRYPTO_ECB ECB (Electronic Codebook) mode (NIST SP800-38A) =20 config CRYPTO_HCTR2 tristate "HCTR2" select CRYPTO_XCTR - select CRYPTO_POLYVAL + select CRYPTO_LIB_POLYVAL select CRYPTO_MANAGER help HCTR2 length-preserving encryption mode =20 A mode for storage encryption that is efficient on processors with diff --git a/crypto/hctr2.c b/crypto/hctr2.c index c8932777bba8..f4cd6c29b4d3 100644 --- a/crypto/hctr2.c +++ b/crypto/hctr2.c @@ -15,11 +15,10 @@ * For more details, see the paper: "Length-preserving encryption with HCT= R2" * (https://eprint.iacr.org/2021/1441.pdf) */ =20 #include -#include #include #include #include #include =20 @@ -35,97 +34,65 @@ #define TWEAK_SIZE 32 =20 struct hctr2_instance_ctx { struct crypto_cipher_spawn blockcipher_spawn; struct crypto_skcipher_spawn xctr_spawn; - struct crypto_shash_spawn polyval_spawn; }; =20 struct hctr2_tfm_ctx { struct crypto_cipher *blockcipher; struct crypto_skcipher *xctr; - struct crypto_shash *polyval; + struct polyval_key poly_key; + struct polyval_elem hashed_tweaklens[2]; u8 L[BLOCKCIPHER_BLOCK_SIZE]; - int hashed_tweak_offset; - /* - * This struct is allocated with extra space for two exported hash - * states. Since the hash state size is not known at compile-time, we - * can't add these to the struct directly. - * - * hashed_tweaklen_divisible; - * hashed_tweaklen_remainder; - */ }; =20 struct hctr2_request_ctx { u8 first_block[BLOCKCIPHER_BLOCK_SIZE]; u8 xctr_iv[BLOCKCIPHER_BLOCK_SIZE]; struct scatterlist *bulk_part_dst; struct scatterlist *bulk_part_src; struct scatterlist sg_src[2]; struct scatterlist sg_dst[2]; + struct polyval_elem hashed_tweak; /* - * Sub-request sizes are unknown at compile-time, so they need to go - * after the members with known sizes. + * skcipher sub-request size is unknown at compile-time, so it needs to + * go after the members with known sizes. */ union { - struct shash_desc hash_desc; + struct polyval_ctx poly_ctx; struct skcipher_request xctr_req; } u; - /* - * This struct is allocated with extra space for one exported hash - * state. Since the hash state size is not known at compile-time, we - * can't add it to the struct directly. - * - * hashed_tweak; - */ }; =20 -static inline u8 *hctr2_hashed_tweaklen(const struct hctr2_tfm_ctx *tctx, - bool has_remainder) -{ - u8 *p =3D (u8 *)tctx + sizeof(*tctx); - - if (has_remainder) /* For messages not a multiple of block length */ - p +=3D crypto_shash_statesize(tctx->polyval); - return p; -} - -static inline u8 *hctr2_hashed_tweak(const struct hctr2_tfm_ctx *tctx, - struct hctr2_request_ctx *rctx) -{ - return (u8 *)rctx + tctx->hashed_tweak_offset; -} - /* * The input data for each HCTR2 hash step begins with a 16-byte block that * contains the tweak length and a flag that indicates whether the input i= s evenly * divisible into blocks. Since this implementation only supports one twe= ak * length, we precompute the two hash states resulting from hashing the two * possible values of this initial block. This reduces by one block the a= mount of * data that needs to be hashed for each encryption/decryption * * These precomputed hashes are stored in hctr2_tfm_ctx. */ -static int hctr2_hash_tweaklen(struct hctr2_tfm_ctx *tctx, bool has_remain= der) +static void hctr2_hash_tweaklens(struct hctr2_tfm_ctx *tctx) { - SHASH_DESC_ON_STACK(shash, tfm->polyval); - __le64 tweak_length_block[2]; - int err; - - shash->tfm =3D tctx->polyval; - memset(tweak_length_block, 0, sizeof(tweak_length_block)); - - tweak_length_block[0] =3D cpu_to_le64(TWEAK_SIZE * 8 * 2 + 2 + has_remain= der); - err =3D crypto_shash_init(shash); - if (err) - return err; - err =3D crypto_shash_update(shash, (u8 *)tweak_length_block, - POLYVAL_BLOCK_SIZE); - if (err) - return err; - return crypto_shash_export(shash, hctr2_hashed_tweaklen(tctx, has_remaind= er)); + struct polyval_ctx ctx; + + for (int has_remainder =3D 0; has_remainder < 2; has_remainder++) { + const __le64 tweak_length_block[2] =3D { + cpu_to_le64(TWEAK_SIZE * 8 * 2 + 2 + has_remainder), + }; + + polyval_init(&ctx, &tctx->poly_key); + polyval_update(&ctx, (const u8 *)&tweak_length_block, + sizeof(tweak_length_block)); + static_assert(sizeof(tweak_length_block) =3D=3D POLYVAL_BLOCK_SIZE); + polyval_export_blkaligned( + &ctx, &tctx->hashed_tweaklens[has_remainder]); + } + memzero_explicit(&ctx, sizeof(ctx)); } =20 static int hctr2_setkey(struct crypto_skcipher *tfm, const u8 *key, unsigned int keylen) { @@ -154,98 +121,75 @@ static int hctr2_setkey(struct crypto_skcipher *tfm, = const u8 *key, =20 memset(tctx->L, 0, sizeof(tctx->L)); tctx->L[0] =3D 0x01; crypto_cipher_encrypt_one(tctx->blockcipher, tctx->L, tctx->L); =20 - crypto_shash_clear_flags(tctx->polyval, CRYPTO_TFM_REQ_MASK); - crypto_shash_set_flags(tctx->polyval, crypto_skcipher_get_flags(tfm) & - CRYPTO_TFM_REQ_MASK); - err =3D crypto_shash_setkey(tctx->polyval, hbar, BLOCKCIPHER_BLOCK_SIZE); - if (err) - return err; + static_assert(sizeof(hbar) =3D=3D POLYVAL_BLOCK_SIZE); + polyval_preparekey(&tctx->poly_key, hbar); memzero_explicit(hbar, sizeof(hbar)); =20 - return hctr2_hash_tweaklen(tctx, true) ?: hctr2_hash_tweaklen(tctx, false= ); + hctr2_hash_tweaklens(tctx); + return 0; } =20 -static int hctr2_hash_tweak(struct skcipher_request *req) +static void hctr2_hash_tweak(struct skcipher_request *req) { struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); const struct hctr2_tfm_ctx *tctx =3D crypto_skcipher_ctx(tfm); struct hctr2_request_ctx *rctx =3D skcipher_request_ctx(req); - struct shash_desc *hash_desc =3D &rctx->u.hash_desc; - int err; + struct polyval_ctx *poly_ctx =3D &rctx->u.poly_ctx; bool has_remainder =3D req->cryptlen % POLYVAL_BLOCK_SIZE; =20 - hash_desc->tfm =3D tctx->polyval; - err =3D crypto_shash_import(hash_desc, hctr2_hashed_tweaklen(tctx, has_re= mainder)); - if (err) - return err; - err =3D crypto_shash_update(hash_desc, req->iv, TWEAK_SIZE); - if (err) - return err; + polyval_import_blkaligned(poly_ctx, &tctx->poly_key, + &tctx->hashed_tweaklens[has_remainder]); + polyval_update(poly_ctx, req->iv, TWEAK_SIZE); =20 // Store the hashed tweak, since we need it when computing both // H(T || N) and H(T || V). - return crypto_shash_export(hash_desc, hctr2_hashed_tweak(tctx, rctx)); + static_assert(TWEAK_SIZE % POLYVAL_BLOCK_SIZE =3D=3D 0); + polyval_export_blkaligned(poly_ctx, &rctx->hashed_tweak); } =20 -static int hctr2_hash_message(struct skcipher_request *req, - struct scatterlist *sgl, - u8 digest[POLYVAL_DIGEST_SIZE]) +static void hctr2_hash_message(struct skcipher_request *req, + struct scatterlist *sgl, + u8 digest[POLYVAL_DIGEST_SIZE]) { - static const u8 padding[BLOCKCIPHER_BLOCK_SIZE] =3D { 0x1 }; + static const u8 padding =3D 0x1; struct hctr2_request_ctx *rctx =3D skcipher_request_ctx(req); - struct shash_desc *hash_desc =3D &rctx->u.hash_desc; + struct polyval_ctx *poly_ctx =3D &rctx->u.poly_ctx; const unsigned int bulk_len =3D req->cryptlen - BLOCKCIPHER_BLOCK_SIZE; struct sg_mapping_iter miter; - unsigned int remainder =3D bulk_len % BLOCKCIPHER_BLOCK_SIZE; int i; - int err =3D 0; int n =3D 0; =20 sg_miter_start(&miter, sgl, sg_nents(sgl), SG_MITER_FROM_SG | SG_MITER_ATOMIC); for (i =3D 0; i < bulk_len; i +=3D n) { sg_miter_next(&miter); n =3D min_t(unsigned int, miter.length, bulk_len - i); - err =3D crypto_shash_update(hash_desc, miter.addr, n); - if (err) - break; + polyval_update(poly_ctx, miter.addr, n); } sg_miter_stop(&miter); =20 - if (err) - return err; - - if (remainder) { - err =3D crypto_shash_update(hash_desc, padding, - BLOCKCIPHER_BLOCK_SIZE - remainder); - if (err) - return err; - } - return crypto_shash_final(hash_desc, digest); + if (req->cryptlen % BLOCKCIPHER_BLOCK_SIZE) + polyval_update(poly_ctx, &padding, 1); + polyval_final(poly_ctx, digest); } =20 static int hctr2_finish(struct skcipher_request *req) { struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); const struct hctr2_tfm_ctx *tctx =3D crypto_skcipher_ctx(tfm); struct hctr2_request_ctx *rctx =3D skcipher_request_ctx(req); + struct polyval_ctx *poly_ctx =3D &rctx->u.poly_ctx; u8 digest[POLYVAL_DIGEST_SIZE]; - struct shash_desc *hash_desc =3D &rctx->u.hash_desc; - int err; =20 // U =3D UU ^ H(T || V) // or M =3D MM ^ H(T || N) - hash_desc->tfm =3D tctx->polyval; - err =3D crypto_shash_import(hash_desc, hctr2_hashed_tweak(tctx, rctx)); - if (err) - return err; - err =3D hctr2_hash_message(req, rctx->bulk_part_dst, digest); - if (err) - return err; + polyval_import_blkaligned(poly_ctx, &tctx->poly_key, + &rctx->hashed_tweak); + hctr2_hash_message(req, rctx->bulk_part_dst, digest); crypto_xor(rctx->first_block, digest, BLOCKCIPHER_BLOCK_SIZE); =20 // Copy U (or M) into dst scatterlist scatterwalk_map_and_copy(rctx->first_block, req->dst, 0, BLOCKCIPHER_BLOCK_SIZE, 1); @@ -267,11 +211,10 @@ static int hctr2_crypt(struct skcipher_request *req, = bool enc) struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); const struct hctr2_tfm_ctx *tctx =3D crypto_skcipher_ctx(tfm); struct hctr2_request_ctx *rctx =3D skcipher_request_ctx(req); u8 digest[POLYVAL_DIGEST_SIZE]; int bulk_len =3D req->cryptlen - BLOCKCIPHER_BLOCK_SIZE; - int err; =20 // Requests must be at least one block if (req->cryptlen < BLOCKCIPHER_BLOCK_SIZE) return -EINVAL; =20 @@ -285,16 +228,12 @@ static int hctr2_crypt(struct skcipher_request *req, = bool enc) rctx->bulk_part_dst =3D scatterwalk_ffwd(rctx->sg_dst, req->dst, BLOCKCIPHER_BLOCK_SIZE); =20 // MM =3D M ^ H(T || N) // or UU =3D U ^ H(T || V) - err =3D hctr2_hash_tweak(req); - if (err) - return err; - err =3D hctr2_hash_message(req, rctx->bulk_part_src, digest); - if (err) - return err; + hctr2_hash_tweak(req); + hctr2_hash_message(req, rctx->bulk_part_src, digest); crypto_xor(digest, rctx->first_block, BLOCKCIPHER_BLOCK_SIZE); =20 // UU =3D E(MM) // or MM =3D D(UU) if (enc) @@ -336,12 +275,10 @@ static int hctr2_init_tfm(struct crypto_skcipher *tfm) struct skcipher_instance *inst =3D skcipher_alg_instance(tfm); struct hctr2_instance_ctx *ictx =3D skcipher_instance_ctx(inst); struct hctr2_tfm_ctx *tctx =3D crypto_skcipher_ctx(tfm); struct crypto_skcipher *xctr; struct crypto_cipher *blockcipher; - struct crypto_shash *polyval; - unsigned int subreq_size; int err; =20 xctr =3D crypto_spawn_skcipher(&ictx->xctr_spawn); if (IS_ERR(xctr)) return PTR_ERR(xctr); @@ -350,35 +287,21 @@ static int hctr2_init_tfm(struct crypto_skcipher *tfm) if (IS_ERR(blockcipher)) { err =3D PTR_ERR(blockcipher); goto err_free_xctr; } =20 - polyval =3D crypto_spawn_shash(&ictx->polyval_spawn); - if (IS_ERR(polyval)) { - err =3D PTR_ERR(polyval); - goto err_free_blockcipher; - } - tctx->xctr =3D xctr; tctx->blockcipher =3D blockcipher; - tctx->polyval =3D polyval; =20 BUILD_BUG_ON(offsetofend(struct hctr2_request_ctx, u) !=3D sizeof(struct hctr2_request_ctx)); - subreq_size =3D max(sizeof_field(struct hctr2_request_ctx, u.hash_desc) + - crypto_shash_descsize(polyval), - sizeof_field(struct hctr2_request_ctx, u.xctr_req) + - crypto_skcipher_reqsize(xctr)); - - tctx->hashed_tweak_offset =3D offsetof(struct hctr2_request_ctx, u) + - subreq_size; - crypto_skcipher_set_reqsize(tfm, tctx->hashed_tweak_offset + - crypto_shash_statesize(polyval)); + crypto_skcipher_set_reqsize( + tfm, max(sizeof(struct hctr2_request_ctx), + offsetofend(struct hctr2_request_ctx, u.xctr_req) + + crypto_skcipher_reqsize(xctr))); return 0; =20 -err_free_blockcipher: - crypto_free_cipher(blockcipher); err_free_xctr: crypto_free_skcipher(xctr); return err; } =20 @@ -386,34 +309,29 @@ static void hctr2_exit_tfm(struct crypto_skcipher *tf= m) { struct hctr2_tfm_ctx *tctx =3D crypto_skcipher_ctx(tfm); =20 crypto_free_cipher(tctx->blockcipher); crypto_free_skcipher(tctx->xctr); - crypto_free_shash(tctx->polyval); } =20 static void hctr2_free_instance(struct skcipher_instance *inst) { struct hctr2_instance_ctx *ictx =3D skcipher_instance_ctx(inst); =20 crypto_drop_cipher(&ictx->blockcipher_spawn); crypto_drop_skcipher(&ictx->xctr_spawn); - crypto_drop_shash(&ictx->polyval_spawn); kfree(inst); } =20 -static int hctr2_create_common(struct crypto_template *tmpl, - struct rtattr **tb, - const char *xctr_name, - const char *polyval_name) +static int hctr2_create_common(struct crypto_template *tmpl, struct rtattr= **tb, + const char *xctr_name) { struct skcipher_alg_common *xctr_alg; u32 mask; struct skcipher_instance *inst; struct hctr2_instance_ctx *ictx; struct crypto_alg *blockcipher_alg; - struct shash_alg *polyval_alg; char blockcipher_name[CRYPTO_MAX_ALG_NAME]; int len; int err; =20 err =3D crypto_check_attr_type(tb, CRYPTO_ALG_TYPE_SKCIPHER, &mask); @@ -455,46 +373,27 @@ static int hctr2_create_common(struct crypto_template= *tmpl, /* Require blocksize of 16 bytes */ err =3D -EINVAL; if (blockcipher_alg->cra_blocksize !=3D BLOCKCIPHER_BLOCK_SIZE) goto err_free_inst; =20 - /* Polyval =CE=B5-=E2=88=86U hash function */ - err =3D crypto_grab_shash(&ictx->polyval_spawn, - skcipher_crypto_instance(inst), - polyval_name, 0, mask); - if (err) - goto err_free_inst; - polyval_alg =3D crypto_spawn_shash_alg(&ictx->polyval_spawn); - - /* Ensure Polyval is being used */ - err =3D -EINVAL; - if (strcmp(polyval_alg->base.cra_name, "polyval") !=3D 0) - goto err_free_inst; - /* Instance fields */ =20 err =3D -ENAMETOOLONG; if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME, "hctr2(%s)", blockcipher_alg->cra_name) >=3D CRYPTO_MAX_ALG_NAME) goto err_free_inst; if (snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME, - "hctr2_base(%s,%s)", - xctr_alg->base.cra_driver_name, - polyval_alg->base.cra_driver_name) >=3D CRYPTO_MAX_ALG_NAME) + "hctr2_base(%s,polyval-lib)", + xctr_alg->base.cra_driver_name) >=3D CRYPTO_MAX_ALG_NAME) goto err_free_inst; =20 inst->alg.base.cra_blocksize =3D BLOCKCIPHER_BLOCK_SIZE; - inst->alg.base.cra_ctxsize =3D sizeof(struct hctr2_tfm_ctx) + - polyval_alg->statesize * 2; + inst->alg.base.cra_ctxsize =3D sizeof(struct hctr2_tfm_ctx); inst->alg.base.cra_alignmask =3D xctr_alg->base.cra_alignmask; - /* - * The hash function is called twice, so it is weighted higher than the - * xctr and blockcipher. - */ inst->alg.base.cra_priority =3D (2 * xctr_alg->base.cra_priority + - 4 * polyval_alg->base.cra_priority + - blockcipher_alg->cra_priority) / 7; + blockcipher_alg->cra_priority) / + 3; =20 inst->alg.setkey =3D hctr2_setkey; inst->alg.encrypt =3D hctr2_encrypt; inst->alg.decrypt =3D hctr2_decrypt; inst->alg.init =3D hctr2_init_tfm; @@ -523,12 +422,15 @@ static int hctr2_create_base(struct crypto_template *= tmpl, struct rtattr **tb) return PTR_ERR(xctr_name); =20 polyval_name =3D crypto_attr_alg_name(tb[2]); if (IS_ERR(polyval_name)) return PTR_ERR(polyval_name); + if (strcmp(polyval_name, "polyval") !=3D 0 && + strcmp(polyval_name, "polyval-lib") !=3D 0) + return -ENOENT; =20 - return hctr2_create_common(tmpl, tb, xctr_name, polyval_name); + return hctr2_create_common(tmpl, tb, xctr_name); } =20 static int hctr2_create(struct crypto_template *tmpl, struct rtattr **tb) { const char *blockcipher_name; @@ -540,11 +442,11 @@ static int hctr2_create(struct crypto_template *tmpl,= struct rtattr **tb) =20 if (snprintf(xctr_name, CRYPTO_MAX_ALG_NAME, "xctr(%s)", blockcipher_name) >=3D CRYPTO_MAX_ALG_NAME) return -ENAMETOOLONG; =20 - return hctr2_create_common(tmpl, tb, xctr_name, "polyval"); + return hctr2_create_common(tmpl, tb, xctr_name); } =20 static struct crypto_template hctr2_tmpls[] =3D { { /* hctr2_base(xctr_name, polyval_name) */ diff --git a/crypto/testmgr.c b/crypto/testmgr.c index 90d06c3ec967..499e979a56dc 100644 --- a/crypto/testmgr.c +++ b/crypto/testmgr.c @@ -5057,12 +5057,11 @@ static const struct alg_test_desc alg_test_descs[] = =3D { .suite =3D { .hash =3D __VECS(ghash_tv_template) } }, { .alg =3D "hctr2(aes)", - .generic_driver =3D - "hctr2_base(xctr(aes-generic),polyval-generic)", + .generic_driver =3D "hctr2_base(xctr(aes-generic),polyval-lib)", .test =3D alg_test_skcipher, .suite =3D { .cipher =3D __VECS(aes_hctr2_tv_template) } }, { --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF2E5279358; Sun, 9 Nov 2025 23:49:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732199; cv=none; b=CnhwxYHfcZY0ng3aw/kIABZMjedsaS2rPZU91iEikCy4cWEstfcMHKH+W+Xz9j60KJyWMkiYL3IXc1CEzZnZyWiXy8CU69xYy2tVB4tPrQtsqUYXQ+Bf/Kz45fUwWmUWMmafTeCY16MC9WsTxkxTzP7kjGvx4rYM4gKEf/67LVo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732199; c=relaxed/simple; bh=Zodgx2IYoNCP1gQCEOxcXaY8nw/zLva62Cv6WrKrveo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NKnEMOXpgJP3gH6qpw+4NArubckvYsJQhzUP8H98PQGXpoENJ/nHDygPycXStpaWDIlzmXyyOGzP2UpGSSV8skzKxpnZS4IMuq/cuZYFY8uVosq2V7HzaZqAUXdFar/3IOEBaoMWYhkqqvSYqJMJ5De1MzZyU0G74PifbzDSlEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bHY/4O64; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bHY/4O64" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5BCA9C113D0; Sun, 9 Nov 2025 23:49:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732199; bh=Zodgx2IYoNCP1gQCEOxcXaY8nw/zLva62Cv6WrKrveo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bHY/4O64tE1Hgu0ldpjTDMtmZm6jm5u7rgsO6X7LaBLYf6YGaIKvRImvfPpEkGKvV vB+Tps9CbLKK5/m/269VfcjRb9jzTi/Q11mSmmLg+1PQeoi3zNypTWcDYZrXODk25N 4rpOMiC0KyNqSd1z9QRPQ8XWw6at6l0ZkTjD7ygoxSVRr8fqYZTJxzyI0M18nt9VN1 Ha9E06n27LxzTrhD4/2tX8e0nL47DQKaJ05hyRxTY8AJbZydqgvnfUqL7twT0x+qFj RtwcFuRfV/4YbSkuOCj9J6wq5KSYeJZolawOA8q7HEwA1WDXLIE2ctKNFTB+PzyZ4d cdyTQrF+Rl8Qg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 7/9] crypto: polyval - Remove the polyval crypto_shash Date: Sun, 9 Nov 2025 15:47:22 -0800 Message-ID: <20251109234726.638437-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove polyval support from crypto_shash. It no longer has any user now that the HCTR2 code uses the POLYVAL library instead. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- crypto/Kconfig | 10 -- crypto/Makefile | 1 - crypto/polyval-generic.c | 205 --------------------------------------- 3 files changed, 216 deletions(-) delete mode 100644 crypto/polyval-generic.c diff --git a/crypto/Kconfig b/crypto/Kconfig index 805172f75bf1..bf8b8a60a0c0 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -946,20 +946,10 @@ config CRYPTO_MICHAEL_MIC known as WPA (Wif-Fi Protected Access). =20 This algorithm is required for TKIP, but it should not be used for other purposes because of the weakness of the algorithm. =20 -config CRYPTO_POLYVAL - tristate - select CRYPTO_HASH - select CRYPTO_LIB_GF128MUL - help - POLYVAL hash function for HCTR2 - - This is used in HCTR2. It is not a general-purpose - cryptographic hash function. - config CRYPTO_RMD160 tristate "RIPEMD-160" select CRYPTO_HASH help RIPEMD-160 hash function (ISO/IEC 10118-3) diff --git a/crypto/Makefile b/crypto/Makefile index 0388ff8d219d..093c56a45d3f 100644 --- a/crypto/Makefile +++ b/crypto/Makefile @@ -170,11 +170,10 @@ KASAN_SANITIZE_jitterentropy.o =3D n UBSAN_SANITIZE_jitterentropy.o =3D n jitterentropy_rng-y :=3D jitterentropy.o jitterentropy-kcapi.o obj-$(CONFIG_CRYPTO_JITTERENTROPY_TESTINTERFACE) +=3D jitterentropy-testin= g.o obj-$(CONFIG_CRYPTO_BENCHMARK) +=3D tcrypt.o obj-$(CONFIG_CRYPTO_GHASH) +=3D ghash-generic.o -obj-$(CONFIG_CRYPTO_POLYVAL) +=3D polyval-generic.o obj-$(CONFIG_CRYPTO_USER_API) +=3D af_alg.o obj-$(CONFIG_CRYPTO_USER_API_HASH) +=3D algif_hash.o obj-$(CONFIG_CRYPTO_USER_API_SKCIPHER) +=3D algif_skcipher.o obj-$(CONFIG_CRYPTO_USER_API_RNG) +=3D algif_rng.o obj-$(CONFIG_CRYPTO_USER_API_AEAD) +=3D algif_aead.o diff --git a/crypto/polyval-generic.c b/crypto/polyval-generic.c deleted file mode 100644 index fe5b01a4000d..000000000000 --- a/crypto/polyval-generic.c +++ /dev/null @@ -1,205 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * POLYVAL: hash function for HCTR2. - * - * Copyright (c) 2007 Nokia Siemens Networks - Mikko Herranen - * Copyright (c) 2009 Intel Corp. - * Author: Huang Ying - * Copyright 2021 Google LLC - */ - -/* - * Code based on crypto/ghash-generic.c - * - * POLYVAL is a keyed hash function similar to GHASH. POLYVAL uses a diffe= rent - * modulus for finite field multiplication which makes hardware accelerated - * implementations on little-endian machines faster. POLYVAL is used in the - * kernel to implement HCTR2, but was originally specified for AES-GCM-SIV - * (RFC 8452). - * - * For more information see: - * Length-preserving encryption with HCTR2: - * https://eprint.iacr.org/2021/1441.pdf - * AES-GCM-SIV: Nonce Misuse-Resistant Authenticated Encryption: - * https://datatracker.ietf.org/doc/html/rfc8452 - * - * Like GHASH, POLYVAL is not a cryptographic hash function and should - * not be used outside of crypto modes explicitly designed to use POLYVAL. - * - * This implementation uses a convenient trick involving the GHASH and POL= YVAL - * fields. This trick allows multiplication in the POLYVAL field to be - * implemented by using multiplication in the GHASH field as a subroutine.= An - * element of the POLYVAL field can be converted to an element of the GHASH - * field by computing x*REVERSE(a), where REVERSE reverses the byte-orderi= ng of - * a. Similarly, an element of the GHASH field can be converted back to the - * POLYVAL field by computing REVERSE(x^{-1}*a). For more information, see: - * https://datatracker.ietf.org/doc/html/rfc8452#appendix-A - * - * By using this trick, we do not need to implement the POLYVAL field for = the - * generic implementation. - * - * Warning: this generic implementation is not intended to be used in prac= tice - * and is not constant time. For practical use, a hardware accelerated - * implementation of POLYVAL should be used instead. - * - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -struct polyval_tfm_ctx { - struct gf128mul_4k *gf128; -}; - -struct polyval_desc_ctx { - union { - u8 buffer[POLYVAL_BLOCK_SIZE]; - be128 buffer128; - }; -}; - -static void copy_and_reverse(u8 dst[POLYVAL_BLOCK_SIZE], - const u8 src[POLYVAL_BLOCK_SIZE]) -{ - u64 a =3D get_unaligned((const u64 *)&src[0]); - u64 b =3D get_unaligned((const u64 *)&src[8]); - - put_unaligned(swab64(a), (u64 *)&dst[8]); - put_unaligned(swab64(b), (u64 *)&dst[0]); -} - -static int polyval_setkey(struct crypto_shash *tfm, - const u8 *key, unsigned int keylen) -{ - struct polyval_tfm_ctx *ctx =3D crypto_shash_ctx(tfm); - be128 k; - - if (keylen !=3D POLYVAL_BLOCK_SIZE) - return -EINVAL; - - gf128mul_free_4k(ctx->gf128); - - BUILD_BUG_ON(sizeof(k) !=3D POLYVAL_BLOCK_SIZE); - copy_and_reverse((u8 *)&k, key); - gf128mul_x_lle(&k, &k); - - ctx->gf128 =3D gf128mul_init_4k_lle(&k); - memzero_explicit(&k, POLYVAL_BLOCK_SIZE); - - if (!ctx->gf128) - return -ENOMEM; - - return 0; -} - -static int polyval_generic_init(struct shash_desc *desc) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - - memset(dctx, 0, sizeof(*dctx)); - - return 0; -} - -static int polyval_generic_update(struct shash_desc *desc, - const u8 *src, unsigned int srclen) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - const struct polyval_tfm_ctx *ctx =3D crypto_shash_ctx(desc->tfm); - u8 tmp[POLYVAL_BLOCK_SIZE]; - - do { - copy_and_reverse(tmp, src); - crypto_xor(dctx->buffer, tmp, POLYVAL_BLOCK_SIZE); - gf128mul_4k_lle(&dctx->buffer128, ctx->gf128); - src +=3D POLYVAL_BLOCK_SIZE; - srclen -=3D POLYVAL_BLOCK_SIZE; - } while (srclen >=3D POLYVAL_BLOCK_SIZE); - - return srclen; -} - -static int polyval_finup(struct shash_desc *desc, const u8 *src, - unsigned int len, u8 *dst) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - - if (len) { - u8 tmp[POLYVAL_BLOCK_SIZE] =3D {}; - - memcpy(tmp, src, len); - polyval_generic_update(desc, tmp, POLYVAL_BLOCK_SIZE); - } - copy_and_reverse(dst, dctx->buffer); - return 0; -} - -static int polyval_export(struct shash_desc *desc, void *out) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - - copy_and_reverse(out, dctx->buffer); - return 0; -} - -static int polyval_import(struct shash_desc *desc, const void *in) -{ - struct polyval_desc_ctx *dctx =3D shash_desc_ctx(desc); - - copy_and_reverse(dctx->buffer, in); - return 0; -} - -static void polyval_exit_tfm(struct crypto_shash *tfm) -{ - struct polyval_tfm_ctx *ctx =3D crypto_shash_ctx(tfm); - - gf128mul_free_4k(ctx->gf128); -} - -static struct shash_alg polyval_alg =3D { - .digestsize =3D POLYVAL_DIGEST_SIZE, - .init =3D polyval_generic_init, - .update =3D polyval_generic_update, - .finup =3D polyval_finup, - .setkey =3D polyval_setkey, - .export =3D polyval_export, - .import =3D polyval_import, - .exit_tfm =3D polyval_exit_tfm, - .statesize =3D sizeof(struct polyval_desc_ctx), - .descsize =3D sizeof(struct polyval_desc_ctx), - .base =3D { - .cra_name =3D "polyval", - .cra_driver_name =3D "polyval-generic", - .cra_priority =3D 100, - .cra_flags =3D CRYPTO_AHASH_ALG_BLOCK_ONLY, - .cra_blocksize =3D POLYVAL_BLOCK_SIZE, - .cra_ctxsize =3D sizeof(struct polyval_tfm_ctx), - .cra_module =3D THIS_MODULE, - }, -}; - -static int __init polyval_mod_init(void) -{ - return crypto_register_shash(&polyval_alg); -} - -static void __exit polyval_mod_exit(void) -{ - crypto_unregister_shash(&polyval_alg); -} - -module_init(polyval_mod_init); -module_exit(polyval_mod_exit); - -MODULE_LICENSE("GPL"); -MODULE_DESCRIPTION("POLYVAL hash function"); -MODULE_ALIAS_CRYPTO("polyval"); -MODULE_ALIAS_CRYPTO("polyval-generic"); --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FC3527EFE9; Sun, 9 Nov 2025 23:50:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732201; cv=none; b=c6oAvDoK7iZObraG3sCd54DDRKboAZenZXxtT2sGTas66u0SBZkgyrGaETrTzuAUJicHboK0rOhH2HFELOCdPDQhG9vgU/XyeNg2KI267yUvVktxDAgDUFpQO14XKIF6zpL96AdVn3JKTyb/6bIqgT3FaVVz+4axRNBEXwuLiYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732201; c=relaxed/simple; bh=nqSfESYD3WdUWSz/fBnUEecChZSD29umtZlw2xLsB0o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KphOvpRtx9Inz//tIz4f2kVhUO0+78ZV1WC1Lb8kt5LbzsgOwGiH74mKjmtmM83c7O04yPIpZtdFfHKQHLDvSVA4EYQ/HRcG1YJnECwJdzkR9Y486H2wl8gg+z207dxY1MAW7Wlbl4uFdD8qgvJq46LNtgiCNpFs5Lqjy2vlQBc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ipyjBVVB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ipyjBVVB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B38CCC4CEF8; Sun, 9 Nov 2025 23:49:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732199; bh=nqSfESYD3WdUWSz/fBnUEecChZSD29umtZlw2xLsB0o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ipyjBVVBmsPbmwitdkyiaqgT4axyARVg2FomhDjAfZkJc1ubHKtY/wftpORy2bCuo g3pkYu7Msd6Ua1c0v+tsP3fY+RprmxzK9s2+KaN6DKW0tl+eBqfxmfSgSq8W0qNhNM /Klt2r7KxtbhBX73y8AxEZZ1Hp1f2gK7VbciFBTvjrZuxAVn0g/+SfegyuxVAaOeT+ cqgQL10S3WCaG5efZKJE5T722jmmCEIha+0RjvYOTfKohjIoS0xgNvupDTVy+gdMfE Ihy86GM1k2fQ+jAF5ZWJYRTlC/5S/GyBuFHdwo9aq7BPf439kTep7b2KvkifiqUUew YbRlwKqxunQbQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 8/9] crypto: testmgr - Remove polyval tests Date: Sun, 9 Nov 2025 15:47:23 -0800 Message-ID: <20251109234726.638437-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" These are no longer used, since polyval support has been removed from the crypto_shash API. POLYVAL remains supported via lib/crypto/, where it has a KUnit test suite instead. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- crypto/tcrypt.c | 4 -- crypto/testmgr.c | 6 -- crypto/testmgr.h | 171 ----------------------------------------------- 3 files changed, 181 deletions(-) diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c index d1d88debbd71..32d9eaf2c8af 100644 --- a/crypto/tcrypt.c +++ b/crypto/tcrypt.c @@ -1688,14 +1688,10 @@ static int do_test(const char *alg, u32 type, u32 m= ask, int m, u32 num_mb) =20 case 56: ret =3D min(ret, tcrypt_test("ccm(sm4)")); break; =20 - case 57: - ret =3D min(ret, tcrypt_test("polyval")); - break; - case 58: ret =3D min(ret, tcrypt_test("gcm(aria)")); break; =20 case 59: diff --git a/crypto/testmgr.c b/crypto/testmgr.c index 499e979a56dc..6fb53978df11 100644 --- a/crypto/testmgr.c +++ b/crypto/testmgr.c @@ -5368,16 +5368,10 @@ static const struct alg_test_desc alg_test_descs[] = =3D { .fips_allowed =3D 1, }, { .alg =3D "pkcs1pad(rsa)", .test =3D alg_test_null, .fips_allowed =3D 1, - }, { - .alg =3D "polyval", - .test =3D alg_test_hash, - .suite =3D { - .hash =3D __VECS(polyval_tv_template) - } }, { .alg =3D "rfc3686(ctr(aes))", .test =3D alg_test_skcipher, .fips_allowed =3D 1, .suite =3D { diff --git a/crypto/testmgr.h b/crypto/testmgr.h index 268231227282..a3e4695945ca 100644 --- a/crypto/testmgr.h +++ b/crypto/testmgr.h @@ -36233,181 +36233,10 @@ static const struct cipher_testvec aes_xctr_tv_t= emplate[] =3D { .len =3D 512, }, =20 }; =20 -/* - * Test vectors generated using https://github.com/google/hctr2 - * - * To ensure compatibility with RFC 8452, some tests were sourced from - * https://datatracker.ietf.org/doc/html/rfc8452 - */ -static const struct hash_testvec polyval_tv_template[] =3D { - { // From RFC 8452 - .key =3D "\x31\x07\x28\xd9\x91\x1f\x1f\x38" - "\x37\xb2\x43\x16\xc3\xfa\xb9\xa0", - .plaintext =3D "\x65\x78\x61\x6d\x70\x6c\x65\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x48\x65\x6c\x6c\x6f\x20\x77\x6f" - "\x72\x6c\x64\x00\x00\x00\x00\x00" - "\x38\x00\x00\x00\x00\x00\x00\x00" - "\x58\x00\x00\x00\x00\x00\x00\x00", - .digest =3D "\xad\x7f\xcf\x0b\x51\x69\x85\x16" - "\x62\x67\x2f\x3c\x5f\x95\x13\x8f", - .psize =3D 48, - .ksize =3D 16, - }, - { // From RFC 8452 - .key =3D "\xd9\xb3\x60\x27\x96\x94\x94\x1a" - "\xc5\xdb\xc6\x98\x7a\xda\x73\x77", - .plaintext =3D "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00", - .digest =3D "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00", - .psize =3D 16, - .ksize =3D 16, - }, - { // From RFC 8452 - .key =3D "\xd9\xb3\x60\x27\x96\x94\x94\x1a" - "\xc5\xdb\xc6\x98\x7a\xda\x73\x77", - .plaintext =3D "\x01\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x40\x00\x00\x00\x00\x00\x00\x00", - .digest =3D "\xeb\x93\xb7\x74\x09\x62\xc5\xe4" - "\x9d\x2a\x90\xa7\xdc\x5c\xec\x74", - .psize =3D 32, - .ksize =3D 16, - }, - { // From RFC 8452 - .key =3D "\xd9\xb3\x60\x27\x96\x94\x94\x1a" - "\xc5\xdb\xc6\x98\x7a\xda\x73\x77", - .plaintext =3D "\x01\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x02\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x03\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x80\x01\x00\x00\x00\x00\x00\x00", - .digest =3D "\x81\x38\x87\x46\xbc\x22\xd2\x6b" - "\x2a\xbc\x3d\xcb\x15\x75\x42\x22", - .psize =3D 64, - .ksize =3D 16, - }, - { // From RFC 8452 - .key =3D "\xd9\xb3\x60\x27\x96\x94\x94\x1a" - "\xc5\xdb\xc6\x98\x7a\xda\x73\x77", - .plaintext =3D "\x01\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x02\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x03\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x04\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x02\x00\x00\x00\x00\x00\x00", - .digest =3D "\x1e\x39\xb6\xd3\x34\x4d\x34\x8f" - "\x60\x44\xf8\x99\x35\xd1\xcf\x78", - .psize =3D 80, - .ksize =3D 16, - }, - { // From RFC 8452 - .key =3D "\xd9\xb3\x60\x27\x96\x94\x94\x1a" - "\xc5\xdb\xc6\x98\x7a\xda\x73\x77", - .plaintext =3D "\x01\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x02\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x03\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x04\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x05\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x08\x00\x00\x00\x00\x00\x00\x00" - "\x00\x02\x00\x00\x00\x00\x00\x00", - .digest =3D "\xff\xcd\x05\xd5\x77\x0f\x34\xad" - "\x92\x67\xf0\xa5\x99\x94\xb1\x5a", - .psize =3D 96, - .ksize =3D 16, - }, - { // Random ( 1) - .key =3D "\x90\xcc\xac\xee\xba\xd7\xd4\x68" - "\x98\xa6\x79\x70\xdf\x66\x15\x6c", - .plaintext =3D "", - .digest =3D "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00", - .psize =3D 0, - .ksize =3D 16, - }, - { // Random ( 1) - .key =3D "\xc1\x45\x71\xf0\x30\x07\x94\xe7" - "\x3a\xdd\xe4\xc6\x19\x2d\x02\xa2", - .plaintext =3D "\xc1\x5d\x47\xc7\x4c\x7c\x5e\x07" - "\x85\x14\x8f\x79\xcc\x73\x83\xf7" - "\x35\xb8\xcb\x73\x61\xf0\x53\x31" - "\xbf\x84\xde\xb6\xde\xaf\xb0\xb8" - "\xb7\xd9\x11\x91\x89\xfd\x1e\x4c" - "\x84\x4a\x1f\x2a\x87\xa4\xaf\x62" - "\x8d\x7d\x58\xf6\x43\x35\xfc\x53" - "\x8f\x1a\xf6\x12\xe1\x13\x3f\x66" - "\x91\x4b\x13\xd6\x45\xfb\xb0\x7a" - "\xe0\x8b\x8e\x99\xf7\x86\x46\x37" - "\xd1\x22\x9e\x52\xf3\x3f\xd9\x75" - "\x2c\x2c\xc6\xbb\x0e\x08\x14\x29" - "\xe8\x50\x2f\xd8\xbe\xf4\xe9\x69" - "\x4a\xee\xf7\xae\x15\x65\x35\x1e", - .digest =3D "\x00\x4f\x5d\xe9\x3b\xc0\xd6\x50" - "\x3e\x38\x73\x86\xc6\xda\xca\x7f", - .psize =3D 112, - .ksize =3D 16, - }, - { // Random ( 1) - .key =3D "\x37\xbe\x68\x16\x50\xb9\x4e\xb0" - "\x47\xde\xe2\xbd\xde\xe4\x48\x09", - .plaintext =3D "\x87\xfc\x68\x9f\xff\xf2\x4a\x1e" - "\x82\x3b\x73\x8f\xc1\xb2\x1b\x7a" - "\x6c\x4f\x81\xbc\x88\x9b\x6c\xa3" - "\x9c\xc2\xa5\xbc\x14\x70\x4c\x9b" - "\x0c\x9f\x59\x92\x16\x4b\x91\x3d" - "\x18\x55\x22\x68\x12\x8c\x63\xb2" - "\x51\xcb\x85\x4b\xd2\xae\x0b\x1c" - "\x5d\x28\x9d\x1d\xb1\xc8\xf0\x77" - "\xe9\xb5\x07\x4e\x06\xc8\xee\xf8" - "\x1b\xed\x72\x2a\x55\x7d\x16\xc9" - "\xf2\x54\xe7\xe9\xe0\x44\x5b\x33" - "\xb1\x49\xee\xff\x43\xfb\x82\xcd" - "\x4a\x70\x78\x81\xa4\x34\x36\xe8" - "\x4c\x28\x54\xa6\x6c\xc3\x6b\x78" - "\xe7\xc0\x5d\xc6\x5d\x81\xab\x70" - "\x08\x86\xa1\xfd\xf4\x77\x55\xfd" - "\xa3\xe9\xe2\x1b\xdf\x99\xb7\x80" - "\xf9\x0a\x4f\x72\x4a\xd3\xaf\xbb" - "\xb3\x3b\xeb\x08\x58\x0f\x79\xce" - "\xa5\x99\x05\x12\x34\xd4\xf4\x86" - "\x37\x23\x1d\xc8\x49\xc0\x92\xae" - "\xa6\xac\x9b\x31\x55\xed\x15\xc6" - "\x05\x17\x37\x8d\x90\x42\xe4\x87" - "\x89\x62\x88\x69\x1c\x6a\xfd\xe3" - "\x00\x2b\x47\x1a\x73\xc1\x51\xc2" - "\xc0\x62\x74\x6a\x9e\xb2\xe5\x21" - "\xbe\x90\xb5\xb0\x50\xca\x88\x68" - "\xe1\x9d\x7a\xdf\x6c\xb7\xb9\x98" - "\xee\x28\x62\x61\x8b\xd1\x47\xf9" - "\x04\x7a\x0b\x5d\xcd\x2b\x65\xf5" - "\x12\xa3\xfe\x1a\xaa\x2c\x78\x42" - "\xb8\xbe\x7d\x74\xeb\x59\xba\xba", - .digest =3D "\xae\x11\xd4\x60\x2a\x5f\x9e\x42" - "\x89\x04\xc2\x34\x8d\x55\x94\x0a", - .psize =3D 256, - .ksize =3D 16, - }, - -}; - /* * Test vectors generated using https://github.com/google/hctr2 */ static const struct cipher_testvec aes_hctr2_tv_template[] =3D { { --=20 2.51.2 From nobody Sat Feb 7 18:26:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A795727B349; Sun, 9 Nov 2025 23:50:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732200; cv=none; b=KWnUb6vTLiSCLg5wGvV7VEstA2wSy0PggS3+yl243JmJz1YPzDv/9GNpcdTyhZn8q+786cQtMAcDyLW/XG4s7B6iSVkExDXB6HW25+FlekpvKcqnmfdZaBQwDqcS3azFks4rETgaDmi4/369atZtPXrQtaQ07E0Jr6SPM53W0WU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762732200; c=relaxed/simple; bh=7gddUMmP2coCrWDBPHW0Qp0T0QzSnROq55ixSxyk3eY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U4+3NhYWxWCbwFSQnxj14N1XzQ0MigArDF2BDSSjGSTQKgfgCXamDkpoVGKjECAIi/DQD8TaGMPqoL5E4Y9wxIIyGKmbQDIpzWwtzN1FnI4wf9ENS/zOEA0W8eNx+Kh8aT3ud/luA9HH90MYUyuTOG5MIwwFDSlMrym8HAX5BkM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aep2Ic4m; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aep2Ic4m" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17266C4CEFB; Sun, 9 Nov 2025 23:50:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1762732200; bh=7gddUMmP2coCrWDBPHW0Qp0T0QzSnROq55ixSxyk3eY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aep2Ic4mAOPEy+UzZYgxrcqU0Vc0IuKyB9GT0ijy2wWrUZCCr0TJjQ4sWV5RrXDBz +z276rb7zAJHWRnhqTchQ4FfGfx/eH+5H+EP7rJNZejjjUClO3qNZ+3jroRdLsC1Cb 9IgCW15wWGZeizwbowEGZKPJR3ZL7i7Ne1uTonvktqbt0XONmQ09aQsgA/9a3xXZBU Q5idrcFJ4beRn05+Sb4Zj8wbQ2/xrH+2QBbKIwc/977EeKhG8ENexjF4xTUxPwJoma Ll0IrZl5KfVIRV0xSBa7f3vXf1tREL6bSy9PlhvtJDyiJjuU79z6DCKfsPHSrbZuQ4 w6Nga/fLzOnOg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , Herbert Xu , linux-arm-kernel@lists.infradead.org, x86@kernel.org, Eric Biggers Subject: [PATCH 9/9] fscrypt: Drop obsolete recommendation to enable optimized POLYVAL Date: Sun, 9 Nov 2025 15:47:24 -0800 Message-ID: <20251109234726.638437-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.51.2 In-Reply-To: <20251109234726.638437-1-ebiggers@kernel.org> References: <20251109234726.638437-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" CONFIG_CRYPTO_POLYVAL_ARM64_CE and CONFIG_CRYPTO_POLYVAL_CLMUL_NI no longer exist. The architecture-optimized POLYVAL code is now just enabled automatically when HCTR2 support is enabled. Update the fscrypt documentation accordingly. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- Documentation/filesystems/fscrypt.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/Documentation/filesystems/fscrypt.rst b/Documentation/filesyst= ems/fscrypt.rst index 696a5844bfa3..70af896822e1 100644 --- a/Documentation/filesystems/fscrypt.rst +++ b/Documentation/filesystems/fscrypt.rst @@ -448,13 +448,11 @@ API, but the filenames mode still does. - AES-256-HCTR2 - Mandatory: - CONFIG_CRYPTO_HCTR2 - Recommended: - arm64: CONFIG_CRYPTO_AES_ARM64_CE_BLK - - arm64: CONFIG_CRYPTO_POLYVAL_ARM64_CE - x86: CONFIG_CRYPTO_AES_NI_INTEL - - x86: CONFIG_CRYPTO_POLYVAL_CLMUL_NI =20 - Adiantum - Mandatory: - CONFIG_CRYPTO_ADIANTUM - Recommended: --=20 2.51.2