From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D36772749E2; Wed, 27 Aug 2025 15:14:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307696; cv=none; b=jDFGazhmz2n02xeEin5hTKNDwVDreN/TpeD1R2BvkQwuWEwVlHB6w5haig3U2Lr+NTHlj1tu8N+9iilv85NSLnVElfmm51Yx+EZjBVv6jY1+ctnSqJ9xdd9CUS6c27u69VQy0L9rUh9GNBY3xsHn9nn6UBjx46m7SvIIvR01SYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307696; c=relaxed/simple; bh=gw2Ah4Rt2D+6bPW1+7M02U96xkL1iHc+3kS1D3Z/L0E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S/bw7M0rsQiKLU+1PPP32wcU+MHT3sT3H/ufwr1BV/qjKHJ5IkuLsPvwDyiPYYKEWU5gn/w0j9KxnJsd+gv7EyFc4aDPPFmfHUtmuEycOmpJAfRGq85EAhbpvhGN4GBj37Y6fNoOXW5G8jETp+713zVuNlT7L1Qm73C5ZF1OHQQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZAYx8mqs; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZAYx8mqs" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70498C4CEF9; Wed, 27 Aug 2025 15:14:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307696; bh=gw2Ah4Rt2D+6bPW1+7M02U96xkL1iHc+3kS1D3Z/L0E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZAYx8mqsUEpyjqV3d0BmRT0pzARb3YUwvqI6cxKolz5P4W0Bf+U/uslnOuw/nQaXN MEtT+p5upyuIPnne/UbnAFfdRtu2Iw5LEKM/JSz7zP5JYhFt+9owmvw5C9/NSLqvaw vn0NgfxGw0d2ikqcCw6jzxucfabdyK8SjBEYEVohweCQmqJSk3IwkzH0L+if56SQNe lZmmAb2BR+Sls6RV/QXEKDdrtv/TX5ZEEn+pzy5gMKSsMvA8KK3Kt8+AE9OX/uItZT EPS+MQB3IcbLUeITEbvSNCX3AONYnf1dHX5eX8zoyP/aYdkcMC9eRR+gWfii0SgMY5 K7V/9i5SWn6rg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 01/12] arm: configs: Remove obsolete assignments to CRYPTO_CHACHA20_NEON Date: Wed, 27 Aug 2025 08:11:20 -0700 Message-ID: <20250827151131.27733-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Since v6.15, CRYPTO_CHACHA20_NEON is a hidden option and is selected automatically. Therefore, assigning a value to it in a defconfig no longer has any effect. Remove it from all files that did this. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- arch/arm/configs/exynos_defconfig | 1 - arch/arm/configs/milbeaut_m10v_defconfig | 1 - arch/arm/configs/multi_v7_defconfig | 1 - arch/arm/configs/omap2plus_defconfig | 1 - 4 files changed, 4 deletions(-) diff --git a/arch/arm/configs/exynos_defconfig b/arch/arm/configs/exynos_de= fconfig index 6915c766923a2..84070e9698e8c 100644 --- a/arch/arm/configs/exynos_defconfig +++ b/arch/arm/configs/exynos_defconfig @@ -362,11 +362,10 @@ CONFIG_CRYPTO_LZ4=3Dm CONFIG_CRYPTO_USER_API_HASH=3Dm CONFIG_CRYPTO_USER_API_SKCIPHER=3Dm CONFIG_CRYPTO_USER_API_RNG=3Dm CONFIG_CRYPTO_USER_API_AEAD=3Dm CONFIG_CRYPTO_AES_ARM_BS=3Dm -CONFIG_CRYPTO_CHACHA20_NEON=3Dm CONFIG_CRYPTO_DEV_EXYNOS_RNG=3Dy CONFIG_CRYPTO_DEV_S5P=3Dy CONFIG_DMA_CMA=3Dy CONFIG_CMA_SIZE_MBYTES=3D96 CONFIG_FONTS=3Dy diff --git a/arch/arm/configs/milbeaut_m10v_defconfig b/arch/arm/configs/mi= lbeaut_m10v_defconfig index a3be0b2ede09c..a2995eb390c60 100644 --- a/arch/arm/configs/milbeaut_m10v_defconfig +++ b/arch/arm/configs/milbeaut_m10v_defconfig @@ -99,11 +99,10 @@ CONFIG_CRYPTO_AES=3Dy CONFIG_CRYPTO_SEQIV=3Dm CONFIG_CRYPTO_GHASH_ARM_CE=3Dm CONFIG_CRYPTO_AES_ARM=3Dm CONFIG_CRYPTO_AES_ARM_BS=3Dm CONFIG_CRYPTO_AES_ARM_CE=3Dm -CONFIG_CRYPTO_CHACHA20_NEON=3Dm # CONFIG_CRYPTO_HW is not set CONFIG_DMA_CMA=3Dy CONFIG_CMA_SIZE_MBYTES=3D64 CONFIG_PRINTK_TIME=3Dy CONFIG_MAGIC_SYSRQ=3Dy diff --git a/arch/arm/configs/multi_v7_defconfig b/arch/arm/configs/multi_v= 7_defconfig index f2822eeefb957..cc0e0e4a879cb 100644 --- a/arch/arm/configs/multi_v7_defconfig +++ b/arch/arm/configs/multi_v7_defconfig @@ -1289,11 +1289,10 @@ CONFIG_CRYPTO_USER_API_RNG=3Dm CONFIG_CRYPTO_USER_API_AEAD=3Dm CONFIG_CRYPTO_GHASH_ARM_CE=3Dm CONFIG_CRYPTO_AES_ARM=3Dm CONFIG_CRYPTO_AES_ARM_BS=3Dm CONFIG_CRYPTO_AES_ARM_CE=3Dm -CONFIG_CRYPTO_CHACHA20_NEON=3Dm CONFIG_CRYPTO_DEV_SUN4I_SS=3Dm CONFIG_CRYPTO_DEV_FSL_CAAM=3Dm CONFIG_CRYPTO_DEV_EXYNOS_RNG=3Dm CONFIG_CRYPTO_DEV_S5P=3Dm CONFIG_CRYPTO_DEV_ATMEL_AES=3Dm diff --git a/arch/arm/configs/omap2plus_defconfig b/arch/arm/configs/omap2p= lus_defconfig index 939913ed9a73b..1d5f752417398 100644 --- a/arch/arm/configs/omap2plus_defconfig +++ b/arch/arm/configs/omap2plus_defconfig @@ -706,11 +706,10 @@ CONFIG_NLS_ISO8859_1=3Dy CONFIG_SECURITY=3Dy CONFIG_CRYPTO_MICHAEL_MIC=3Dy CONFIG_CRYPTO_GHASH_ARM_CE=3Dm CONFIG_CRYPTO_AES_ARM=3Dm CONFIG_CRYPTO_AES_ARM_BS=3Dm -CONFIG_CRYPTO_CHACHA20_NEON=3Dm CONFIG_CRYPTO_DEV_OMAP=3Dm CONFIG_CRYPTO_DEV_OMAP_SHAM=3Dm CONFIG_CRYPTO_DEV_OMAP_AES=3Dm CONFIG_CRYPTO_DEV_OMAP_DES=3Dm CONFIG_DMA_CMA=3Dy --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50FE819FA8D; Wed, 27 Aug 2025 15:14:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307697; cv=none; b=D2JCD/+WKId4PUo9jdYovCWHiQOyCPFiq8acaupCyMSOaywN6tpLxXBautB4CgVdAGoqdT4NU+lW3ItzfopyEsxUhfQy5L1EyVRUG7h8Xz22PAnhXUHEUdCjwnkE83p+FQkI5fcn/sBpKBYaXyjEGcYGpm3xuXn/87aE0YejqGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307697; c=relaxed/simple; bh=PEnNNwlelyhooGpaIQRTUv9wqpiqewuv20VIflQJ328=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i0TwcLCWTuwODxWmWjPKip755dNUyq1sV5rzU2jDbl+EDlZr+KXayUKDcwkeMHhJygcEST7w3vDJ4g+eh/a+EqhBF/BhXSd63uVBRv0zpQa2nG+fCl0wK9oXDLxMXja37ElWRdsOi9Nhm/+wGTzezT7oesL+6mJHccTD40JbStE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Fw1oT9XE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Fw1oT9XE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9775C4CEEB; Wed, 27 Aug 2025 15:14:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307697; bh=PEnNNwlelyhooGpaIQRTUv9wqpiqewuv20VIflQJ328=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fw1oT9XENhcz7u6XoUZEjoyKNpmTN0wpPnqCQgNrdP03++iYsKgS3ZO4FwWNjtlv7 ZR5x8pCRO4Np+NJIvYMhYz9DkKgqx072wkoHkOrq3OyTuuPywHFTBV4cNCoctC9Ctj lrkrd9tmpY830S0ADI1W4XSvoxC/DiQei8EZxhIFo2dOiShiOIekxbvPhihyU1BeYc anb0SLcI5iKJUgvKuf1GaCtHnUdxdaq5TO0W5m1IdUqcogWuA3xRy+71wuuqEP7/Ie +d+91fgr7m36K1+Knm0y1ecAfiik95v/0hsj/L8RGKnb4Fuoxzdki+MjJmfaCv1pll Y9YC9cjJsRQQg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 02/12] crypto: chacha - register only "-lib" drivers Date: Wed, 27 Aug 2025 08:11:21 -0700 Message-ID: <20250827151131.27733-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For the "chacha20", "xchacha20", and "xchacha12" skcipher algorithms, instead of registering "*-generic" drivers as well as conditionally registering "*-$(ARCH)" drivers, instead just register "*-lib" drivers. These just use the regular library functions, so they just do the right thing and are fully accelerated when supported by the CPU. This eliminates the need for the ChaCha library to support chacha_crypt_generic() and hchacha_block_generic() as part of its external interface. A later commit will make chacha_crypt_generic() a static function. Since this commit removes several "*-generic" driver names which crypto/testmgr.c expects to exist, update testmgr.c accordingly. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- crypto/Kconfig | 1 - crypto/chacha.c | 129 ++++++++--------------------------------------- crypto/testmgr.c | 9 +++- 3 files changed, 29 insertions(+), 110 deletions(-) diff --git a/crypto/Kconfig b/crypto/Kconfig index e8ccf5f51b855..09e8fb6ee0813 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -646,11 +646,10 @@ config CRYPTO_ARC4 weakness of the algorithm. =20 config CRYPTO_CHACHA20 tristate "ChaCha" select CRYPTO_LIB_CHACHA - select CRYPTO_LIB_CHACHA_GENERIC select CRYPTO_SKCIPHER help The ChaCha20, XChaCha20, and XChaCha12 stream cipher algorithms =20 ChaCha20 is a 256-bit high-speed stream cipher designed by Daniel J. diff --git a/crypto/chacha.c b/crypto/chacha.c index c3a11f4e2d13d..ec16d5a33f3cd 100644 --- a/crypto/chacha.c +++ b/crypto/chacha.c @@ -45,11 +45,11 @@ static int chacha12_setkey(struct crypto_skcipher *tfm, return chacha_setkey(tfm, key, keysize, 12); } =20 static int chacha_stream_xor(struct skcipher_request *req, const struct chacha_ctx *ctx, - const u8 iv[CHACHA_IV_SIZE], bool arch) + const u8 iv[CHACHA_IV_SIZE]) { struct skcipher_walk walk; struct chacha_state state; int err; =20 @@ -61,200 +61,115 @@ static int chacha_stream_xor(struct skcipher_request = *req, unsigned int nbytes =3D walk.nbytes; =20 if (nbytes < walk.total) nbytes =3D round_down(nbytes, CHACHA_BLOCK_SIZE); =20 - if (arch) - chacha_crypt(&state, walk.dst.virt.addr, - walk.src.virt.addr, nbytes, ctx->nrounds); - else - chacha_crypt_generic(&state, walk.dst.virt.addr, - walk.src.virt.addr, nbytes, - ctx->nrounds); + chacha_crypt(&state, walk.dst.virt.addr, walk.src.virt.addr, + nbytes, ctx->nrounds); err =3D skcipher_walk_done(&walk, walk.nbytes - nbytes); } =20 return err; } =20 -static int crypto_chacha_crypt_generic(struct skcipher_request *req) +static int crypto_chacha_crypt(struct skcipher_request *req) { struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); const struct chacha_ctx *ctx =3D crypto_skcipher_ctx(tfm); =20 - return chacha_stream_xor(req, ctx, req->iv, false); + return chacha_stream_xor(req, ctx, req->iv); } =20 -static int crypto_chacha_crypt_arch(struct skcipher_request *req) -{ - struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); - const struct chacha_ctx *ctx =3D crypto_skcipher_ctx(tfm); - - return chacha_stream_xor(req, ctx, req->iv, true); -} - -static int crypto_xchacha_crypt(struct skcipher_request *req, bool arch) +static int crypto_xchacha_crypt(struct skcipher_request *req) { struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); const struct chacha_ctx *ctx =3D crypto_skcipher_ctx(tfm); struct chacha_ctx subctx; struct chacha_state state; u8 real_iv[16]; =20 /* Compute the subkey given the original key and first 128 nonce bits */ chacha_init(&state, ctx->key, req->iv); - if (arch) - hchacha_block(&state, subctx.key, ctx->nrounds); - else - hchacha_block_generic(&state, subctx.key, ctx->nrounds); + hchacha_block(&state, subctx.key, ctx->nrounds); subctx.nrounds =3D ctx->nrounds; =20 /* Build the real IV */ memcpy(&real_iv[0], req->iv + 24, 8); /* stream position */ memcpy(&real_iv[8], req->iv + 16, 8); /* remaining 64 nonce bits */ =20 /* Generate the stream and XOR it with the data */ - return chacha_stream_xor(req, &subctx, real_iv, arch); -} - -static int crypto_xchacha_crypt_generic(struct skcipher_request *req) -{ - return crypto_xchacha_crypt(req, false); -} - -static int crypto_xchacha_crypt_arch(struct skcipher_request *req) -{ - return crypto_xchacha_crypt(req, true); + return chacha_stream_xor(req, &subctx, real_iv); } =20 static struct skcipher_alg algs[] =3D { { .base.cra_name =3D "chacha20", - .base.cra_driver_name =3D "chacha20-generic", - .base.cra_priority =3D 100, - .base.cra_blocksize =3D 1, - .base.cra_ctxsize =3D sizeof(struct chacha_ctx), - .base.cra_module =3D THIS_MODULE, - - .min_keysize =3D CHACHA_KEY_SIZE, - .max_keysize =3D CHACHA_KEY_SIZE, - .ivsize =3D CHACHA_IV_SIZE, - .chunksize =3D CHACHA_BLOCK_SIZE, - .setkey =3D chacha20_setkey, - .encrypt =3D crypto_chacha_crypt_generic, - .decrypt =3D crypto_chacha_crypt_generic, - }, - { - .base.cra_name =3D "xchacha20", - .base.cra_driver_name =3D "xchacha20-generic", - .base.cra_priority =3D 100, - .base.cra_blocksize =3D 1, - .base.cra_ctxsize =3D sizeof(struct chacha_ctx), - .base.cra_module =3D THIS_MODULE, - - .min_keysize =3D CHACHA_KEY_SIZE, - .max_keysize =3D CHACHA_KEY_SIZE, - .ivsize =3D XCHACHA_IV_SIZE, - .chunksize =3D CHACHA_BLOCK_SIZE, - .setkey =3D chacha20_setkey, - .encrypt =3D crypto_xchacha_crypt_generic, - .decrypt =3D crypto_xchacha_crypt_generic, - }, - { - .base.cra_name =3D "xchacha12", - .base.cra_driver_name =3D "xchacha12-generic", - .base.cra_priority =3D 100, - .base.cra_blocksize =3D 1, - .base.cra_ctxsize =3D sizeof(struct chacha_ctx), - .base.cra_module =3D THIS_MODULE, - - .min_keysize =3D CHACHA_KEY_SIZE, - .max_keysize =3D CHACHA_KEY_SIZE, - .ivsize =3D XCHACHA_IV_SIZE, - .chunksize =3D CHACHA_BLOCK_SIZE, - .setkey =3D chacha12_setkey, - .encrypt =3D crypto_xchacha_crypt_generic, - .decrypt =3D crypto_xchacha_crypt_generic, - }, - { - .base.cra_name =3D "chacha20", - .base.cra_driver_name =3D "chacha20-" __stringify(ARCH), + .base.cra_driver_name =3D "chacha20-lib", .base.cra_priority =3D 300, .base.cra_blocksize =3D 1, .base.cra_ctxsize =3D sizeof(struct chacha_ctx), .base.cra_module =3D THIS_MODULE, =20 .min_keysize =3D CHACHA_KEY_SIZE, .max_keysize =3D CHACHA_KEY_SIZE, .ivsize =3D CHACHA_IV_SIZE, .chunksize =3D CHACHA_BLOCK_SIZE, .setkey =3D chacha20_setkey, - .encrypt =3D crypto_chacha_crypt_arch, - .decrypt =3D crypto_chacha_crypt_arch, + .encrypt =3D crypto_chacha_crypt, + .decrypt =3D crypto_chacha_crypt, }, { .base.cra_name =3D "xchacha20", - .base.cra_driver_name =3D "xchacha20-" __stringify(ARCH), + .base.cra_driver_name =3D "xchacha20-lib", .base.cra_priority =3D 300, .base.cra_blocksize =3D 1, .base.cra_ctxsize =3D sizeof(struct chacha_ctx), .base.cra_module =3D THIS_MODULE, =20 .min_keysize =3D CHACHA_KEY_SIZE, .max_keysize =3D CHACHA_KEY_SIZE, .ivsize =3D XCHACHA_IV_SIZE, .chunksize =3D CHACHA_BLOCK_SIZE, .setkey =3D chacha20_setkey, - .encrypt =3D crypto_xchacha_crypt_arch, - .decrypt =3D crypto_xchacha_crypt_arch, + .encrypt =3D crypto_xchacha_crypt, + .decrypt =3D crypto_xchacha_crypt, }, { .base.cra_name =3D "xchacha12", - .base.cra_driver_name =3D "xchacha12-" __stringify(ARCH), + .base.cra_driver_name =3D "xchacha12-lib", .base.cra_priority =3D 300, .base.cra_blocksize =3D 1, .base.cra_ctxsize =3D sizeof(struct chacha_ctx), .base.cra_module =3D THIS_MODULE, =20 .min_keysize =3D CHACHA_KEY_SIZE, .max_keysize =3D CHACHA_KEY_SIZE, .ivsize =3D XCHACHA_IV_SIZE, .chunksize =3D CHACHA_BLOCK_SIZE, .setkey =3D chacha12_setkey, - .encrypt =3D crypto_xchacha_crypt_arch, - .decrypt =3D crypto_xchacha_crypt_arch, + .encrypt =3D crypto_xchacha_crypt, + .decrypt =3D crypto_xchacha_crypt, } }; =20 -static unsigned int num_algs; - static int __init crypto_chacha_mod_init(void) { - /* register the arch flavours only if they differ from generic */ - num_algs =3D ARRAY_SIZE(algs); - BUILD_BUG_ON(ARRAY_SIZE(algs) % 2 !=3D 0); - if (!chacha_is_arch_optimized()) - num_algs /=3D 2; - - return crypto_register_skciphers(algs, num_algs); + return crypto_register_skciphers(algs, ARRAY_SIZE(algs)); } =20 static void __exit crypto_chacha_mod_fini(void) { - crypto_unregister_skciphers(algs, num_algs); + crypto_unregister_skciphers(algs, ARRAY_SIZE(algs)); } =20 module_init(crypto_chacha_mod_init); module_exit(crypto_chacha_mod_fini); =20 MODULE_LICENSE("GPL"); MODULE_AUTHOR("Martin Willi "); MODULE_DESCRIPTION("Crypto API wrappers for the ChaCha20, XChaCha20, and X= ChaCha12 stream ciphers"); MODULE_ALIAS_CRYPTO("chacha20"); -MODULE_ALIAS_CRYPTO("chacha20-generic"); -MODULE_ALIAS_CRYPTO("chacha20-" __stringify(ARCH)); +MODULE_ALIAS_CRYPTO("chacha20-lib"); MODULE_ALIAS_CRYPTO("xchacha20"); -MODULE_ALIAS_CRYPTO("xchacha20-generic"); -MODULE_ALIAS_CRYPTO("xchacha20-" __stringify(ARCH)); +MODULE_ALIAS_CRYPTO("xchacha20-lib"); MODULE_ALIAS_CRYPTO("xchacha12"); -MODULE_ALIAS_CRYPTO("xchacha12-generic"); -MODULE_ALIAS_CRYPTO("xchacha12-" __stringify(ARCH)); +MODULE_ALIAS_CRYPTO("xchacha12-lib"); diff --git a/crypto/testmgr.c b/crypto/testmgr.c index beab926ba102e..781445f5f56a6 100644 --- a/crypto/testmgr.c +++ b/crypto/testmgr.c @@ -4150,18 +4150,18 @@ static int alg_test_null(const struct alg_test_desc= *desc, =20 /* Please keep this list sorted by algorithm name. */ static const struct alg_test_desc alg_test_descs[] =3D { { .alg =3D "adiantum(xchacha12,aes)", - .generic_driver =3D "adiantum(xchacha12-generic,aes-generic,nhpoly1305-g= eneric)", + .generic_driver =3D "adiantum(xchacha12-lib,aes-generic,nhpoly1305-gener= ic)", .test =3D alg_test_skcipher, .suite =3D { .cipher =3D __VECS(adiantum_xchacha12_aes_tv_template) }, }, { .alg =3D "adiantum(xchacha20,aes)", - .generic_driver =3D "adiantum(xchacha20-generic,aes-generic,nhpoly1305-g= eneric)", + .generic_driver =3D "adiantum(xchacha20-lib,aes-generic,nhpoly1305-gener= ic)", .test =3D alg_test_skcipher, .suite =3D { .cipher =3D __VECS(adiantum_xchacha20_aes_tv_template) }, }, { @@ -4483,10 +4483,11 @@ static const struct alg_test_desc alg_test_descs[] = =3D { .einval_allowed =3D 1, } } }, { .alg =3D "chacha20", + .generic_driver =3D "chacha20-lib", .test =3D alg_test_skcipher, .suite =3D { .cipher =3D __VECS(chacha20_tv_template) }, }, { @@ -5418,16 +5419,18 @@ static const struct alg_test_desc alg_test_descs[] = =3D { .aad_iv =3D 1, } } }, { .alg =3D "rfc7539(chacha20,poly1305)", + .generic_driver =3D "rfc7539(chacha20-lib,poly1305-generic)", .test =3D alg_test_aead, .suite =3D { .aead =3D __VECS(rfc7539_tv_template) } }, { .alg =3D "rfc7539esp(chacha20,poly1305)", + .generic_driver =3D "rfc7539esp(chacha20-lib,poly1305-generic)", .test =3D alg_test_aead, .suite =3D { .aead =3D { ____VECS(rfc7539esp_tv_template), .einval_allowed =3D 1, @@ -5589,16 +5592,18 @@ static const struct alg_test_desc alg_test_descs[] = =3D { .suite =3D { .hash =3D __VECS(sm4_xcbc128_tv_template) } }, { .alg =3D "xchacha12", + .generic_driver =3D "xchacha12-lib", .test =3D alg_test_skcipher, .suite =3D { .cipher =3D __VECS(xchacha12_tv_template) }, }, { .alg =3D "xchacha20", + .generic_driver =3D "xchacha20-lib", .test =3D alg_test_skcipher, .suite =3D { .cipher =3D __VECS(xchacha20_tv_template) }, }, { --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEDD0277CA3; Wed, 27 Aug 2025 15:14:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307697; cv=none; b=lNu4y9fNU4/h37VuteQ7RTbOepPRKllq4uR6uP8ujZdTAAKR5+YM3i/tlnOucOWxkL/qRpNf2eBxk15yOdyytxDtMkFVzTEw+sQb7Ii/1zr5VCHoq00Ko33d73Q04Kv/zW07g9R9t/e8R6yNJjMO0e4yzjjua7ePr+Uok04VNDc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307697; c=relaxed/simple; bh=5zPEaJIPs9FP9Z3db1Xao92WGzZwgzaULtvXVWtymkk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Mcx9tI17uePAgT0ABLLIyAfuHa90XC33pQrzIcMlIfWXAFnRIlToZ4Fj3XpHRjkhFCQyQdso5zrYP9UCOsvVKDKhPJXI5kvX2R7s3PTHrvK/qEtjwObrZ1MOThbtpTo5zaAPAyAnZYG2LJUQNn5O0Sh1d/cq2Zeq6UGCm7Laa74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mJq7KQg5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mJq7KQg5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4ECD4C4CEF9; Wed, 27 Aug 2025 15:14:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307697; bh=5zPEaJIPs9FP9Z3db1Xao92WGzZwgzaULtvXVWtymkk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mJq7KQg563aA9KxgOuA2cg9yHt+bGO4EmyKdT72BXkIjYCFlSMCErB4lMrPirrO8T Z37k8x81IpvbB0lTRaexYKLwFx2YtL9nFpHfnmlKBZPmmf0sUL3h2uHjDB/2kywt8u vVGg6C+HcbfyPsWWmC/qG2NHLt6FphiRSBvhkjioa/35UvumlGdUEhL97L2Wlh8Yeg 6LB4ei6VmKsyJRTkj1Rm4mkT9O9GJQAP6zubtT5HL/nMyYTqlPMhhix5YseYjbxA1G Rwf3jSYP1vyaHN63Ns7Mq2UAEYHIof85IJhCQMWJpbrFzDxElOCAjdSqTtLCbk579f ntJhFpRRsGWwg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 03/12] lib/crypto: chacha: Remove unused function chacha_is_arch_optimized() Date: Wed, 27 Aug 2025 08:11:22 -0700 Message-ID: <20250827151131.27733-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" chacha_is_arch_optimized() is no longer used, so remove it. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- include/crypto/chacha.h | 9 --------- lib/crypto/arm/chacha-glue.c | 7 ------- lib/crypto/arm64/chacha-neon-glue.c | 6 ------ lib/crypto/mips/chacha-glue.c | 6 ------ lib/crypto/powerpc/chacha-p10-glue.c | 6 ------ lib/crypto/riscv/chacha-riscv64-glue.c | 6 ------ lib/crypto/s390/chacha-glue.c | 6 ------ lib/crypto/x86/chacha_glue.c | 6 ------ 8 files changed, 52 deletions(-) diff --git a/include/crypto/chacha.h b/include/crypto/chacha.h index 91f6b4cf561c7..be25a0b65a05f 100644 --- a/include/crypto/chacha.h +++ b/include/crypto/chacha.h @@ -117,15 +117,6 @@ static inline void chacha20_crypt(struct chacha_state = *state, static inline void chacha_zeroize_state(struct chacha_state *state) { memzero_explicit(state, sizeof(*state)); } =20 -#if IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA) -bool chacha_is_arch_optimized(void); -#else -static inline bool chacha_is_arch_optimized(void) -{ - return false; -} -#endif - #endif /* _CRYPTO_CHACHA_H */ diff --git a/lib/crypto/arm/chacha-glue.c b/lib/crypto/arm/chacha-glue.c index 88ec964152831..67ba045cae359 100644 --- a/lib/crypto/arm/chacha-glue.c +++ b/lib/crypto/arm/chacha-glue.c @@ -99,17 +99,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, dst +=3D todo; } while (bytes); } EXPORT_SYMBOL(chacha_crypt_arch); =20 -bool chacha_is_arch_optimized(void) -{ - /* We always can use at least the ARM scalar implementation. */ - return true; -} -EXPORT_SYMBOL(chacha_is_arch_optimized); - static int __init chacha_arm_mod_init(void) { if (IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && (elf_hwcap & HWCAP_NEON)) { switch (read_cpuid_part()) { case ARM_CPU_PART_CORTEX_A7: diff --git a/lib/crypto/arm64/chacha-neon-glue.c b/lib/crypto/arm64/chacha-= neon-glue.c index d0188f974ca5c..48097aa34af7c 100644 --- a/lib/crypto/arm64/chacha-neon-glue.c +++ b/lib/crypto/arm64/chacha-neon-glue.c @@ -93,16 +93,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, dst +=3D todo; } while (bytes); } EXPORT_SYMBOL(chacha_crypt_arch); =20 -bool chacha_is_arch_optimized(void) -{ - return static_key_enabled(&have_neon); -} -EXPORT_SYMBOL(chacha_is_arch_optimized); - static int __init chacha_simd_mod_init(void) { if (cpu_have_named_feature(ASIMD)) static_branch_enable(&have_neon); return 0; diff --git a/lib/crypto/mips/chacha-glue.c b/lib/crypto/mips/chacha-glue.c index 88c097594eb0f..f8390af21dc92 100644 --- a/lib/crypto/mips/chacha-glue.c +++ b/lib/crypto/mips/chacha-glue.c @@ -16,14 +16,8 @@ EXPORT_SYMBOL(chacha_crypt_arch); =20 asmlinkage void hchacha_block_arch(const struct chacha_state *state, u32 out[HCHACHA_OUT_WORDS], int nrounds); EXPORT_SYMBOL(hchacha_block_arch); =20 -bool chacha_is_arch_optimized(void) -{ - return true; -} -EXPORT_SYMBOL(chacha_is_arch_optimized); - MODULE_DESCRIPTION("ChaCha and HChaCha functions (MIPS optimized)"); MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); diff --git a/lib/crypto/powerpc/chacha-p10-glue.c b/lib/crypto/powerpc/chac= ha-p10-glue.c index fcd23c6f1590b..5d3d5506d7f94 100644 --- a/lib/crypto/powerpc/chacha-p10-glue.c +++ b/lib/crypto/powerpc/chacha-p10-glue.c @@ -74,16 +74,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, dst +=3D todo; } while (bytes); } EXPORT_SYMBOL(chacha_crypt_arch); =20 -bool chacha_is_arch_optimized(void) -{ - return static_key_enabled(&have_p10); -} -EXPORT_SYMBOL(chacha_is_arch_optimized); - static int __init chacha_p10_init(void) { if (cpu_has_feature(CPU_FTR_ARCH_31)) static_branch_enable(&have_p10); return 0; diff --git a/lib/crypto/riscv/chacha-riscv64-glue.c b/lib/crypto/riscv/chac= ha-riscv64-glue.c index 8c3f11d79be31..a15f0aca3fc43 100644 --- a/lib/crypto/riscv/chacha-riscv64-glue.c +++ b/lib/crypto/riscv/chacha-riscv64-glue.c @@ -48,16 +48,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, } kernel_vector_end(); } EXPORT_SYMBOL(chacha_crypt_arch); =20 -bool chacha_is_arch_optimized(void) -{ - return static_key_enabled(&use_zvkb); -} -EXPORT_SYMBOL(chacha_is_arch_optimized); - static int __init riscv64_chacha_mod_init(void) { if (riscv_isa_extension_available(NULL, ZVKB) && riscv_vector_vlen() >=3D 128) static_branch_enable(&use_zvkb); diff --git a/lib/crypto/s390/chacha-glue.c b/lib/crypto/s390/chacha-glue.c index c57dc851214fa..d8137387fe28f 100644 --- a/lib/crypto/s390/chacha-glue.c +++ b/lib/crypto/s390/chacha-glue.c @@ -45,13 +45,7 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *d= st, const u8 *src, CHACHA_BLOCK_SIZE; } } EXPORT_SYMBOL(chacha_crypt_arch); =20 -bool chacha_is_arch_optimized(void) -{ - return cpu_has_vx(); -} -EXPORT_SYMBOL(chacha_is_arch_optimized); - MODULE_DESCRIPTION("ChaCha stream cipher (s390 optimized)"); MODULE_LICENSE("GPL v2"); diff --git a/lib/crypto/x86/chacha_glue.c b/lib/crypto/x86/chacha_glue.c index 10b2c945f5412..de7da9d512af2 100644 --- a/lib/crypto/x86/chacha_glue.c +++ b/lib/crypto/x86/chacha_glue.c @@ -158,16 +158,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8= *dst, const u8 *src, dst +=3D todo; } while (bytes); } EXPORT_SYMBOL(chacha_crypt_arch); =20 -bool chacha_is_arch_optimized(void) -{ - return static_key_enabled(&chacha_use_simd); -} -EXPORT_SYMBOL(chacha_is_arch_optimized); - static int __init chacha_simd_mod_init(void) { if (!boot_cpu_has(X86_FEATURE_SSSE3)) return 0; =20 --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 771C3279DDB; Wed, 27 Aug 2025 15:14:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307698; cv=none; b=ENKvbh+1fpPxfNP/G5Vtgw/PfvXKiDD9BLvMeKt6hf7Q6btadbJq6Dg0AFTbI483+jshrD6hKWmN7hdOOaO5xa9vTzneF+5WbINm+Tk17IG+p4savOr5DdVqWr+ScYTaEx+KDb8TVuohTanNwBFHN2cLdT4j2/Db1AuegvB+lnA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307698; c=relaxed/simple; bh=DgqLJLt/N5mdg7z1+FAS46a3j1pnJlIf4tSEAWMOUqk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gi1yfak0gkzt5h8MhaYz2bgDteoy24epA/0d3hB7R/4yJFxJfOsQDVfTzMxVn8aQFhLxpfNVBMhetENXTVzLSjYZ6kuvJZfRhNDjlZXhOgBDiSLwRpPNR+dFbSDsPDzIHeo0lApzdZQlcvooieVaz11RozaP8OhX8NcHRSJzyVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gdCvI0ji; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gdCvI0ji" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7B71C4CEEB; Wed, 27 Aug 2025 15:14:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307698; bh=DgqLJLt/N5mdg7z1+FAS46a3j1pnJlIf4tSEAWMOUqk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gdCvI0jiNwjyyAG+h6jO9lVJzemAuZP8v7aD0Hzp9q8c5BtLEf1GvoHL4qyImWp6p jUq44DX3bCS9q0/MbnW0P+/M/khURXkG/GcXzXSVbF3/jqOa9cfWT3/50aLZOWKmRW YJAauoNQZ9Nk8Aa4SG6J41X8UwY5/xfdADWDuXs+D3PWLM84BqpOd5rQQvW5ZrZiMG cCzj6OJMyAESvKePPjHxpJWkUaUFDjKDe3ZIUXRjE4EbaEhIWScv2Udjb0j17dgdMg hx0mP+Ls2kvWF66F/isj70szONU8a8wgF99zQlbUo3IPTK5IV3iUij8oce03lik72J xbO5SXFia10PA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 04/12] lib/crypto: chacha: Rename chacha.c to chacha-block-generic.c Date: Wed, 27 Aug 2025 08:11:23 -0700 Message-ID: <20250827151131.27733-5-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename chacha.c to chacha-block-generic.c to free up the name chacha.c for the high-level API entry points (chacha_crypt() and hchacha_block()), similar to the other algorithms. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- lib/crypto/Makefile | 4 ++-- lib/crypto/{chacha.c =3D> chacha-block-generic.c} | 0 2 files changed, 2 insertions(+), 2 deletions(-) rename lib/crypto/{chacha.c =3D> chacha-block-generic.c} (100%) diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 8d91572b4d5ef..ca65924e861ff 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -13,12 +13,12 @@ obj-$(CONFIG_KUNIT) +=3D tests/ obj-$(CONFIG_CRYPTO_HASH_INFO) +=3D hash_info.o =20 obj-$(CONFIG_CRYPTO_LIB_UTILS) +=3D libcryptoutils.o libcryptoutils-y :=3D memneq.o utils.o =20 -# chacha is used by the /dev/random driver which is always builtin -obj-y +=3D chacha.o +# chacha20_block() is used by the /dev/random driver which is always built= in +obj-y +=3D chacha-block-generic.o obj-$(CONFIG_CRYPTO_LIB_CHACHA_GENERIC) +=3D libchacha.o =20 obj-$(CONFIG_CRYPTO_LIB_AES) +=3D libaes.o libaes-y :=3D aes.o =20 diff --git a/lib/crypto/chacha.c b/lib/crypto/chacha-block-generic.c similarity index 100% rename from lib/crypto/chacha.c rename to lib/crypto/chacha-block-generic.c --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92E2727A135; Wed, 27 Aug 2025 15:14:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307698; cv=none; b=rb2dSTwqEFqeqpHjWyZyePh2vprKbzbmKsxsksYZsSU2HLiwTR82Yhlei+3B9tmWHqEFVmOvkuTfUM9wz9G0FUHSKwA0OYVcORth+AMTpYDJWhjsvSu4CwijHpODCHObFQqWcb/6NUiYiIWWZDYcqPIeTMA+kctz9g1sFVhB++g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307698; c=relaxed/simple; bh=g1XQPcj5tYdZR7RHqv4B4KYBYDZhxP4CJu6Eu0ySl7M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SApKVj13IqpdsNmV5C3lLdFvisZxZbfxlXYmHrTatfsB4uFyXDmv4H5yU2ANfqDlOPlTeEW5ebdz6MH6PZ7Sh0ucz/+7sstZHTe0d9mc+dKL1ETGLcui1KBvaHKw2lJ34umDBFXpMb/b598CJTcQeHFiGkm2XTMyHlH5QLlmoWo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hAtWYP9p; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hAtWYP9p" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D79CC4CEF6; Wed, 27 Aug 2025 15:14:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307698; bh=g1XQPcj5tYdZR7RHqv4B4KYBYDZhxP4CJu6Eu0ySl7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hAtWYP9pqw9F41rCRT7PMNXh7UMxrQfg6Fh4MWXFo89S9O4tImBUhVwJh8+jJ6cJF 0xTMzFQOQVFUdL86hrGWgt5/AiXIbe955v+pzQ1Q+7VEbIOsE366hHAb9VIPTWihbw Bi/wwjcD5X0UoPjaDeqpeM1JGI5Om2eUWasqEIpI8Jo4trPn/TobDNJqoNUMlbr1Is XOIGY57gXcrP1bODzQ/qQMTDZr4Ri3Jy5BvvAXB3yOri392X02lvWpxXlOQwmxy3aL tkTOfrxoyje0qNAYRhMjbpbV+Ma833YSDVyniSzpjjlCjUoWI+WbNFWmWglwWlBcwV FTUoIFe3+SWOw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 05/12] lib/crypto: chacha: Rename libchacha.c to chacha.c Date: Wed, 27 Aug 2025 08:11:24 -0700 Message-ID: <20250827151131.27733-6-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename libchacha.c to chacha.c to make the naming consistent with other algorithms and allow additional source files to be added to the libchacha module. This file currently contains chacha_crypt_generic(), but it will soon be updated to contain chacha_crypt(). Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- lib/crypto/Makefile | 1 + lib/crypto/{libchacha.c =3D> chacha.c} | 0 2 files changed, 1 insertion(+) rename lib/crypto/{libchacha.c =3D> chacha.c} (100%) diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index ca65924e861ff..5749d176be1c1 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -16,10 +16,11 @@ obj-$(CONFIG_CRYPTO_LIB_UTILS) +=3D libcryptoutils.o libcryptoutils-y :=3D memneq.o utils.o =20 # chacha20_block() is used by the /dev/random driver which is always built= in obj-y +=3D chacha-block-generic.o obj-$(CONFIG_CRYPTO_LIB_CHACHA_GENERIC) +=3D libchacha.o +libchacha-y :=3D chacha.o =20 obj-$(CONFIG_CRYPTO_LIB_AES) +=3D libaes.o libaes-y :=3D aes.o =20 obj-$(CONFIG_CRYPTO_LIB_AESCFB) +=3D libaescfb.o diff --git a/lib/crypto/libchacha.c b/lib/crypto/chacha.c similarity index 100% rename from lib/crypto/libchacha.c rename to lib/crypto/chacha.c --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A4D127E05F; Wed, 27 Aug 2025 15:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307699; cv=none; b=flv5DQBBR9j1uwrLtWp1yzyMRS/GVubA1alWO0WmdjyidY/EjP78MJwH4EZ5T73CKSLvLBl1y+k05/d5MuYo0r159IdqbdzNUrbQbeKFKqn/+kT4WRLD2pEQIYkcf7GKaAMAch8LNYvDWXW4ocAnZBsOHTTHva91fkBPBIj2mj8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307699; c=relaxed/simple; bh=4OQ6L8Z9fiYsl2GeZ8NUQoGRg1AzhJZSgptP9o0Boqk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V6S2cAn7nVjp3j1aD473SlVkEDvh74Y2hurNBfLpkgr5uWmkCFtBwX7HzaqwFOqaED83iXgWfkMhsUymw3mduhH8yvhhRh4ujV57AjGN70qpZFmUDAgFt8C2CTF/vF9erYopBPAI9rC2gZLsAobXdy+ZkH7N6Mv/movNPcbw17o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fcB4W+rk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fcB4W+rk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 978BBC4CEF5; Wed, 27 Aug 2025 15:14:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307698; bh=4OQ6L8Z9fiYsl2GeZ8NUQoGRg1AzhJZSgptP9o0Boqk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fcB4W+rkhc4Um4tzNPOtygXY8UKVEFYewhkSEiKsX/SHTF3y5PoBAA8INcAYePLqZ wh4b0I5aUd+p9VZcw+pSp+74rkyCgQMKfVtYuIOK/yxJbeH97wm5rvRsjNmqbUz+we lpel81AP9m6Wanh+IrdqUW8CmUadHN/T0AhxYmnsPczIE+Jp5Pk9skESIxGYZsigpY UCsEqsnBG50bX7FJBqXqSM20GLShGys4BRL3sjvdTg7YXuH5Kmj2wgzhK+M3ifzmvb HZ6ppx/L2+86fANsO0z4IbByglOTK6QkKKxIrizTbS+IVfo87QzwkSCVhLwPGiQhzH tHY8GX9fkR3ug== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 06/12] lib/crypto: chacha: Consolidate into single module Date: Wed, 27 Aug 2025 08:11:25 -0700 Message-ID: <20250827151131.27733-7-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Consolidate the ChaCha code into a single module (excluding chacha-block-generic.c which remains always built-in for random.c), similar to various other algorithms: - Each arch now provides a header file lib/crypto/$(SRCARCH)/chacha.h, replacing lib/crypto/$(SRCARCH)/chacha*.c. The header defines chacha_crypt_arch() and hchacha_block_arch(). It is included by lib/crypto/chacha.c, and thus the code gets built into the single libchacha module, with improved inlining in some cases. - Whether arch-optimized ChaCha is buildable is now controlled centrally by lib/crypto/Kconfig instead of by lib/crypto/$(SRCARCH)/Kconfig. The conditions for enabling it remain the same as before, and it remains enabled by default. - Any additional arch-specific translation units for the optimized ChaCha code, such as assembly files, are now compiled by lib/crypto/Makefile instead of lib/crypto/$(SRCARCH)/Makefile. This removes the last use for the Makefile and Kconfig files in the arm64, mips, powerpc, riscv, and s390 subdirectories of lib/crypto/. So also remove those files and the references to them. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- include/crypto/chacha.h | 28 ++--------- lib/crypto/Kconfig | 47 +++++-------------- lib/crypto/Makefile | 43 +++++++++++++---- lib/crypto/arm/Kconfig | 5 -- lib/crypto/arm/Makefile | 4 -- lib/crypto/arm/{chacha-glue.c =3D> chacha.h} | 28 +++-------- lib/crypto/arm64/Kconfig | 8 ---- lib/crypto/arm64/Makefile | 4 -- .../arm64/{chacha-neon-glue.c =3D> chacha.h} | 26 +++------- lib/crypto/chacha.c | 41 ++++++++++++++-- lib/crypto/mips/Kconfig | 7 --- lib/crypto/mips/Makefile | 5 -- lib/crypto/mips/{chacha-glue.c =3D> chacha.h} | 11 +---- lib/crypto/powerpc/Kconfig | 8 ---- lib/crypto/powerpc/Makefile | 4 -- .../powerpc/{chacha-p10-glue.c =3D> chacha.h} | 30 +++--------- lib/crypto/riscv/Kconfig | 8 ---- lib/crypto/riscv/Makefile | 4 -- .../riscv/{chacha-riscv64-glue.c =3D> chacha.h} | 30 +++--------- lib/crypto/s390/Kconfig | 7 --- lib/crypto/s390/Makefile | 4 -- lib/crypto/s390/{chacha-glue.c =3D> chacha.h} | 23 ++------- lib/crypto/x86/Kconfig | 7 --- lib/crypto/x86/Makefile | 3 -- lib/crypto/x86/{chacha_glue.c =3D> chacha.h} | 30 ++++-------- 25 files changed, 126 insertions(+), 289 deletions(-) rename lib/crypto/arm/{chacha-glue.c =3D> chacha.h} (79%) delete mode 100644 lib/crypto/arm64/Kconfig delete mode 100644 lib/crypto/arm64/Makefile rename lib/crypto/arm64/{chacha-neon-glue.c =3D> chacha.h} (78%) delete mode 100644 lib/crypto/mips/Kconfig delete mode 100644 lib/crypto/mips/Makefile rename lib/crypto/mips/{chacha-glue.c =3D> chacha.h} (55%) delete mode 100644 lib/crypto/powerpc/Kconfig delete mode 100644 lib/crypto/powerpc/Makefile rename lib/crypto/powerpc/{chacha-p10-glue.c =3D> chacha.h} (66%) delete mode 100644 lib/crypto/riscv/Kconfig delete mode 100644 lib/crypto/riscv/Makefile rename lib/crypto/riscv/{chacha-riscv64-glue.c =3D> chacha.h} (61%) delete mode 100644 lib/crypto/s390/Kconfig delete mode 100644 lib/crypto/s390/Makefile rename lib/crypto/s390/{chacha-glue.c =3D> chacha.h} (55%) rename lib/crypto/x86/{chacha_glue.c =3D> chacha.h} (87%) diff --git a/include/crypto/chacha.h b/include/crypto/chacha.h index be25a0b65a05f..38e26dff27b00 100644 --- a/include/crypto/chacha.h +++ b/include/crypto/chacha.h @@ -43,23 +43,15 @@ static inline void chacha20_block(struct chacha_state *= state, u8 out[CHACHA_BLOCK_SIZE]) { chacha_block_generic(state, out, 20); } =20 -void hchacha_block_arch(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds); void hchacha_block_generic(const struct chacha_state *state, u32 out[HCHACHA_OUT_WORDS], int nrounds); =20 -static inline void hchacha_block(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds) -{ - if (IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA)) - hchacha_block_arch(state, out, nrounds); - else - hchacha_block_generic(state, out, nrounds); -} +void hchacha_block(const struct chacha_state *state, + u32 out[HCHACHA_OUT_WORDS], int nrounds); =20 enum chacha_constants { /* expand 32-byte k */ CHACHA_CONSTANT_EXPA =3D 0x61707865U, CHACHA_CONSTANT_ND_3 =3D 0x3320646eU, CHACHA_CONSTANT_2_BY =3D 0x79622d32U, @@ -91,24 +83,12 @@ static inline void chacha_init(struct chacha_state *sta= te, state->x[13] =3D get_unaligned_le32(iv + 4); state->x[14] =3D get_unaligned_le32(iv + 8); state->x[15] =3D get_unaligned_le32(iv + 12); } =20 -void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds); -void chacha_crypt_generic(struct chacha_state *state, u8 *dst, const u8 *s= rc, - unsigned int bytes, int nrounds); - -static inline void chacha_crypt(struct chacha_state *state, - u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) -{ - if (IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA)) - chacha_crypt_arch(state, dst, src, bytes, nrounds); - else - chacha_crypt_generic(state, dst, src, bytes, nrounds); -} +void chacha_crypt(struct chacha_state *state, u8 *dst, const u8 *src, + unsigned int bytes, int nrounds); =20 static inline void chacha20_crypt(struct chacha_state *state, u8 *dst, const u8 *src, unsigned int bytes) { chacha_crypt(state, dst, src, bytes, 20); diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig index cb4e056a98faf..c1db483bc2306 100644 --- a/lib/crypto/Kconfig +++ b/lib/crypto/Kconfig @@ -42,33 +42,27 @@ config CRYPTO_LIB_BLAKE2S_GENERIC Blake2s library interface that require the generic code as a fallback, e.g., for SIMD implementations. If no arch specific implementation is enabled, this implementation serves the users of CRYPTO_LIB_BLAKE2S. =20 -config CRYPTO_ARCH_HAVE_LIB_CHACHA - bool - help - Declares whether the architecture provides an arch-specific - accelerated implementation of the ChaCha library interface, - either builtin or as a module. - -config CRYPTO_LIB_CHACHA_GENERIC +config CRYPTO_LIB_CHACHA tristate - default CRYPTO_LIB_CHACHA if !CRYPTO_ARCH_HAVE_LIB_CHACHA select CRYPTO_LIB_UTILS help - This symbol can be selected by arch implementations of the ChaCha - library interface that require the generic code as a fallback, e.g., - for SIMD implementations. If no arch specific implementation is - enabled, this implementation serves the users of CRYPTO_LIB_CHACHA. + Enable the ChaCha library interface. Select this if your module uses + chacha_crypt() or hchacha_block(). =20 -config CRYPTO_LIB_CHACHA - tristate - help - Enable the ChaCha library interface. This interface may be fulfilled - by either the generic implementation or an arch-specific one, if one - is available and enabled. +config CRYPTO_LIB_CHACHA_ARCH + bool + depends on CRYPTO_LIB_CHACHA && !UML && !KMSAN + default y if ARM + default y if ARM64 && KERNEL_MODE_NEON + default y if MIPS && CPU_MIPS32_R2 + default y if PPC64 && CPU_LITTLE_ENDIAN && VSX + default y if RISCV && 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + default y if S390 + default y if X86_64 =20 config CRYPTO_ARCH_HAVE_LIB_CURVE25519 bool help Declares whether the architecture provides an arch-specific @@ -216,25 +210,10 @@ source "lib/crypto/tests/Kconfig" =20 if !KMSAN # avoid false positives from assembly if ARM source "lib/crypto/arm/Kconfig" endif -if ARM64 -source "lib/crypto/arm64/Kconfig" -endif -if MIPS -source "lib/crypto/mips/Kconfig" -endif -if PPC -source "lib/crypto/powerpc/Kconfig" -endif -if RISCV -source "lib/crypto/riscv/Kconfig" -endif -if S390 -source "lib/crypto/s390/Kconfig" -endif if X86 source "lib/crypto/x86/Kconfig" endif endif =20 diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 5749d176be1c1..33d7288265db2 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -13,15 +13,10 @@ obj-$(CONFIG_KUNIT) +=3D tests/ obj-$(CONFIG_CRYPTO_HASH_INFO) +=3D hash_info.o =20 obj-$(CONFIG_CRYPTO_LIB_UTILS) +=3D libcryptoutils.o libcryptoutils-y :=3D memneq.o utils.o =20 -# chacha20_block() is used by the /dev/random driver which is always built= in -obj-y +=3D chacha-block-generic.o -obj-$(CONFIG_CRYPTO_LIB_CHACHA_GENERIC) +=3D libchacha.o -libchacha-y :=3D chacha.o - obj-$(CONFIG_CRYPTO_LIB_AES) +=3D libaes.o libaes-y :=3D aes.o =20 obj-$(CONFIG_CRYPTO_LIB_AESCFB) +=3D libaescfb.o libaescfb-y :=3D aescfb.o @@ -38,10 +33,43 @@ obj-$(CONFIG_CRYPTO_LIB_GF128MUL) +=3D gf128mul.o obj-y +=3D libblake2s.o libblake2s-y :=3D blake2s.o libblake2s-$(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC) +=3D blake2s-generic.o libblake2s-$(CONFIG_CRYPTO_SELFTESTS) +=3D blake2s-selftest.o =20 +##########################################################################= ###### + +# chacha20_block() is used by the /dev/random driver which is always built= in +obj-y +=3D chacha-block-generic.o + +obj-$(CONFIG_CRYPTO_LIB_CHACHA) +=3D libchacha.o +libchacha-y :=3D chacha.o + +ifeq ($(CONFIG_CRYPTO_LIB_CHACHA_ARCH),y) +CFLAGS_chacha.o +=3D -I$(src)/$(SRCARCH) + +ifeq ($(CONFIG_ARM),y) +libchacha-y +=3D arm/chacha-scalar-core.o +libchacha-$(CONFIG_KERNEL_MODE_NEON) +=3D arm/chacha-neon-core.o +endif + +libchacha-$(CONFIG_ARM64) +=3D arm64/chacha-neon-core.o + +ifeq ($(CONFIG_MIPS),y) +libchacha-y +=3D mips/chacha-core.o +AFLAGS_mips/chacha-core.o +=3D -O2 # needed to fill branch delay slots +endif + +libchacha-$(CONFIG_PPC) +=3D powerpc/chacha-p10le-8x.o +libchacha-$(CONFIG_RISCV) +=3D riscv/chacha-riscv64-zvkb.o +libchacha-$(CONFIG_S390) +=3D s390/chacha-s390.o +libchacha-$(CONFIG_X86) +=3D x86/chacha-ssse3-x86_64.o \ + x86/chacha-avx2-x86_64.o \ + x86/chacha-avx512vl-x86_64.o +endif # CONFIG_CRYPTO_LIB_CHACHA_ARCH + +##########################################################################= ###### + obj-$(CONFIG_CRYPTO_LIB_CHACHA20POLY1305) +=3D libchacha20poly1305.o libchacha20poly1305-y +=3D chacha20poly1305.o libchacha20poly1305-$(CONFIG_CRYPTO_SELFTESTS) +=3D chacha20poly1305-selft= est.o =20 obj-$(CONFIG_CRYPTO_LIB_CURVE25519_GENERIC) +=3D libcurve25519-generic.o @@ -227,15 +255,10 @@ obj-$(CONFIG_CRYPTO_SELFTESTS_FULL) +=3D simd.o =20 obj-$(CONFIG_CRYPTO_LIB_SM3) +=3D libsm3.o libsm3-y :=3D sm3.o =20 obj-$(CONFIG_ARM) +=3D arm/ -obj-$(CONFIG_ARM64) +=3D arm64/ -obj-$(CONFIG_MIPS) +=3D mips/ -obj-$(CONFIG_PPC) +=3D powerpc/ -obj-$(CONFIG_RISCV) +=3D riscv/ -obj-$(CONFIG_S390) +=3D s390/ obj-$(CONFIG_X86) +=3D x86/ =20 # clean-files must be defined unconditionally clean-files +=3D arm/sha256-core.S arm/sha512-core.S clean-files +=3D arm64/sha256-core.S arm64/sha512-core.S diff --git a/lib/crypto/arm/Kconfig b/lib/crypto/arm/Kconfig index 0d821e282c645..740341aa35d21 100644 --- a/lib/crypto/arm/Kconfig +++ b/lib/crypto/arm/Kconfig @@ -10,10 +10,5 @@ config CRYPTO_BLAKE2S_ARM =20 This is faster than the generic implementations of BLAKE2s and BLAKE2b, but slower than the NEON implementation of BLAKE2b. There is no NEON implementation of BLAKE2s, since NEON doesn't really help with it. - -config CRYPTO_CHACHA20_NEON - tristate - default CRYPTO_LIB_CHACHA - select CRYPTO_ARCH_HAVE_LIB_CHACHA diff --git a/lib/crypto/arm/Makefile b/lib/crypto/arm/Makefile index 9f70e61d419e2..0574b0e9739e3 100644 --- a/lib/crypto/arm/Makefile +++ b/lib/crypto/arm/Makefile @@ -1,8 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only =20 obj-$(CONFIG_CRYPTO_BLAKE2S_ARM) +=3D libblake2s-arm.o libblake2s-arm-y :=3D blake2s-core.o blake2s-glue.o - -obj-$(CONFIG_CRYPTO_CHACHA20_NEON) +=3D chacha-neon.o -chacha-neon-y :=3D chacha-scalar-core.o chacha-glue.o -chacha-neon-$(CONFIG_KERNEL_MODE_NEON) +=3D chacha-neon-core.o diff --git a/lib/crypto/arm/chacha-glue.c b/lib/crypto/arm/chacha.h similarity index 79% rename from lib/crypto/arm/chacha-glue.c rename to lib/crypto/arm/chacha.h index 67ba045cae359..0cae30f8ee5d1 100644 --- a/lib/crypto/arm/chacha-glue.c +++ b/lib/crypto/arm/chacha.h @@ -1,18 +1,16 @@ -// SPDX-License-Identifier: GPL-2.0 +/* SPDX-License-Identifier: GPL-2.0 */ /* * ChaCha and HChaCha functions (ARM optimized) * * Copyright (C) 2016-2019 Linaro, Ltd. * Copyright (C) 2015 Martin Willi */ =20 -#include #include #include #include -#include =20 #include #include #include #include @@ -62,25 +60,24 @@ static void chacha_doneon(struct chacha_state *state, u= 8 *dst, const u8 *src, memcpy(dst, buf, bytes); state->x[12]++; } } =20 -void hchacha_block_arch(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds) +static void hchacha_block_arch(const struct chacha_state *state, + u32 out[HCHACHA_OUT_WORDS], int nrounds) { if (!IS_ENABLED(CONFIG_KERNEL_MODE_NEON) || !neon_usable()) { hchacha_block_arm(state, out, nrounds); } else { kernel_neon_begin(); hchacha_block_neon(state, out, nrounds); kernel_neon_end(); } } -EXPORT_SYMBOL(hchacha_block_arch); =20 -void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) +static void chacha_crypt_arch(struct chacha_state *state, u8 *dst, + const u8 *src, unsigned int bytes, int nrounds) { if (!IS_ENABLED(CONFIG_KERNEL_MODE_NEON) || !neon_usable() || bytes <=3D CHACHA_BLOCK_SIZE) { chacha_doarm(dst, src, bytes, state, nrounds); state->x[12] +=3D DIV_ROUND_UP(bytes, CHACHA_BLOCK_SIZE); @@ -97,13 +94,13 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, bytes -=3D todo; src +=3D todo; dst +=3D todo; } while (bytes); } -EXPORT_SYMBOL(chacha_crypt_arch); =20 -static int __init chacha_arm_mod_init(void) +#define chacha_mod_init_arch chacha_mod_init_arch +static void chacha_mod_init_arch(void) { if (IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && (elf_hwcap & HWCAP_NEON)) { switch (read_cpuid_part()) { case ARM_CPU_PART_CORTEX_A7: case ARM_CPU_PART_CORTEX_A5: @@ -115,17 +112,6 @@ static int __init chacha_arm_mod_init(void) break; default: static_branch_enable(&use_neon); } } - return 0; } -subsys_initcall(chacha_arm_mod_init); - -static void __exit chacha_arm_mod_exit(void) -{ -} -module_exit(chacha_arm_mod_exit); - -MODULE_DESCRIPTION("ChaCha and HChaCha functions (ARM optimized)"); -MODULE_AUTHOR("Ard Biesheuvel "); -MODULE_LICENSE("GPL v2"); diff --git a/lib/crypto/arm64/Kconfig b/lib/crypto/arm64/Kconfig deleted file mode 100644 index 07c8a4f0ab03a..0000000000000 --- a/lib/crypto/arm64/Kconfig +++ /dev/null @@ -1,8 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -config CRYPTO_CHACHA20_NEON - tristate - depends on KERNEL_MODE_NEON - default CRYPTO_LIB_CHACHA - select CRYPTO_LIB_CHACHA_GENERIC - select CRYPTO_ARCH_HAVE_LIB_CHACHA diff --git a/lib/crypto/arm64/Makefile b/lib/crypto/arm64/Makefile deleted file mode 100644 index d49cceca3d1ca..0000000000000 --- a/lib/crypto/arm64/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -obj-$(CONFIG_CRYPTO_CHACHA20_NEON) +=3D chacha-neon.o -chacha-neon-y :=3D chacha-neon-core.o chacha-neon-glue.o diff --git a/lib/crypto/arm64/chacha-neon-glue.c b/lib/crypto/arm64/chacha.h similarity index 78% rename from lib/crypto/arm64/chacha-neon-glue.c rename to lib/crypto/arm64/chacha.h index 48097aa34af7c..ba6c22d460863 100644 --- a/lib/crypto/arm64/chacha-neon-glue.c +++ b/lib/crypto/arm64/chacha.h @@ -16,15 +16,13 @@ * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. */ =20 -#include #include #include #include -#include =20 #include #include #include =20 @@ -59,25 +57,24 @@ static void chacha_doneon(struct chacha_state *state, u= 8 *dst, const u8 *src, dst +=3D l; state->x[12] +=3D DIV_ROUND_UP(l, CHACHA_BLOCK_SIZE); } } =20 -void hchacha_block_arch(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds) +static void hchacha_block_arch(const struct chacha_state *state, + u32 out[HCHACHA_OUT_WORDS], int nrounds) { if (!static_branch_likely(&have_neon) || !crypto_simd_usable()) { hchacha_block_generic(state, out, nrounds); } else { kernel_neon_begin(); hchacha_block_neon(state, out, nrounds); kernel_neon_end(); } } -EXPORT_SYMBOL(hchacha_block_arch); =20 -void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) +static void chacha_crypt_arch(struct chacha_state *state, u8 *dst, + const u8 *src, unsigned int bytes, int nrounds) { if (!static_branch_likely(&have_neon) || bytes <=3D CHACHA_BLOCK_SIZE || !crypto_simd_usable()) return chacha_crypt_generic(state, dst, src, bytes, nrounds); =20 @@ -91,23 +88,12 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, bytes -=3D todo; src +=3D todo; dst +=3D todo; } while (bytes); } -EXPORT_SYMBOL(chacha_crypt_arch); =20 -static int __init chacha_simd_mod_init(void) +#define chacha_mod_init_arch chacha_mod_init_arch +static void chacha_mod_init_arch(void) { if (cpu_have_named_feature(ASIMD)) static_branch_enable(&have_neon); - return 0; } -subsys_initcall(chacha_simd_mod_init); - -static void __exit chacha_simd_mod_exit(void) -{ -} -module_exit(chacha_simd_mod_exit); - -MODULE_DESCRIPTION("ChaCha and HChaCha functions (ARM64 optimized)"); -MODULE_AUTHOR("Ard Biesheuvel "); -MODULE_LICENSE("GPL v2"); diff --git a/lib/crypto/chacha.c b/lib/crypto/chacha.c index 26862ad90a964..e0c7cb4af3180 100644 --- a/lib/crypto/chacha.c +++ b/lib/crypto/chacha.c @@ -9,12 +9,13 @@ #include #include #include #include =20 -void chacha_crypt_generic(struct chacha_state *state, u8 *dst, const u8 *s= rc, - unsigned int bytes, int nrounds) +static void __maybe_unused +chacha_crypt_generic(struct chacha_state *state, u8 *dst, const u8 *src, + unsigned int bytes, int nrounds) { /* aligned to potentially speed up crypto_xor() */ u8 stream[CHACHA_BLOCK_SIZE] __aligned(sizeof(long)); =20 while (bytes >=3D CHACHA_BLOCK_SIZE) { @@ -27,9 +28,43 @@ void chacha_crypt_generic(struct chacha_state *state, u8= *dst, const u8 *src, if (bytes) { chacha_block_generic(state, stream, nrounds); crypto_xor_cpy(dst, src, stream, bytes); } } -EXPORT_SYMBOL(chacha_crypt_generic); + +#ifdef CONFIG_CRYPTO_LIB_CHACHA_ARCH +#include "chacha.h" /* $(SRCARCH)/chacha.h */ +#else +#define chacha_crypt_arch chacha_crypt_generic +#define hchacha_block_arch hchacha_block_generic +#endif + +void chacha_crypt(struct chacha_state *state, u8 *dst, const u8 *src, + unsigned int bytes, int nrounds) +{ + chacha_crypt_arch(state, dst, src, bytes, nrounds); +} +EXPORT_SYMBOL_GPL(chacha_crypt); + +void hchacha_block(const struct chacha_state *state, + u32 out[HCHACHA_OUT_WORDS], int nrounds) +{ + hchacha_block_arch(state, out, nrounds); +} +EXPORT_SYMBOL_GPL(hchacha_block); + +#ifdef chacha_mod_init_arch +static int __init chacha_mod_init(void) +{ + chacha_mod_init_arch(); + return 0; +} +subsys_initcall(chacha_mod_init); + +static void __exit chacha_mod_exit(void) +{ +} +module_exit(chacha_mod_exit); +#endif =20 MODULE_DESCRIPTION("ChaCha stream cipher (RFC7539)"); MODULE_LICENSE("GPL"); diff --git a/lib/crypto/mips/Kconfig b/lib/crypto/mips/Kconfig deleted file mode 100644 index 94c1a0892c203..0000000000000 --- a/lib/crypto/mips/Kconfig +++ /dev/null @@ -1,7 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -config CRYPTO_CHACHA_MIPS - tristate - depends on CPU_MIPS32_R2 - default CRYPTO_LIB_CHACHA - select CRYPTO_ARCH_HAVE_LIB_CHACHA diff --git a/lib/crypto/mips/Makefile b/lib/crypto/mips/Makefile deleted file mode 100644 index b5ea0e25c21ef..0000000000000 --- a/lib/crypto/mips/Makefile +++ /dev/null @@ -1,5 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -obj-$(CONFIG_CRYPTO_CHACHA_MIPS) +=3D chacha-mips.o -chacha-mips-y :=3D chacha-core.o chacha-glue.o -AFLAGS_chacha-core.o +=3D -O2 # needed to fill branch delay slots diff --git a/lib/crypto/mips/chacha-glue.c b/lib/crypto/mips/chacha.h similarity index 55% rename from lib/crypto/mips/chacha-glue.c rename to lib/crypto/mips/chacha.h index f8390af21dc92..0c18c0dc2a406 100644 --- a/lib/crypto/mips/chacha-glue.c +++ b/lib/crypto/mips/chacha.h @@ -1,23 +1,14 @@ -// SPDX-License-Identifier: GPL-2.0 +/* SPDX-License-Identifier: GPL-2.0 */ /* * ChaCha and HChaCha functions (MIPS optimized) * * Copyright (C) 2019 Linaro, Ltd. */ =20 -#include #include -#include =20 asmlinkage void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, unsigned int bytes, int nrounds); -EXPORT_SYMBOL(chacha_crypt_arch); - asmlinkage void hchacha_block_arch(const struct chacha_state *state, u32 out[HCHACHA_OUT_WORDS], int nrounds); -EXPORT_SYMBOL(hchacha_block_arch); - -MODULE_DESCRIPTION("ChaCha and HChaCha functions (MIPS optimized)"); -MODULE_AUTHOR("Ard Biesheuvel "); -MODULE_LICENSE("GPL v2"); diff --git a/lib/crypto/powerpc/Kconfig b/lib/crypto/powerpc/Kconfig deleted file mode 100644 index e41012a61876e..0000000000000 --- a/lib/crypto/powerpc/Kconfig +++ /dev/null @@ -1,8 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -config CRYPTO_CHACHA20_P10 - tristate - depends on PPC64 && CPU_LITTLE_ENDIAN && VSX - default CRYPTO_LIB_CHACHA - select CRYPTO_LIB_CHACHA_GENERIC - select CRYPTO_ARCH_HAVE_LIB_CHACHA diff --git a/lib/crypto/powerpc/Makefile b/lib/crypto/powerpc/Makefile deleted file mode 100644 index 778a04edd226c..0000000000000 --- a/lib/crypto/powerpc/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -obj-$(CONFIG_CRYPTO_CHACHA20_P10) +=3D chacha-p10-crypto.o -chacha-p10-crypto-y :=3D chacha-p10-glue.o chacha-p10le-8x.o diff --git a/lib/crypto/powerpc/chacha-p10-glue.c b/lib/crypto/powerpc/chac= ha.h similarity index 66% rename from lib/crypto/powerpc/chacha-p10-glue.c rename to lib/crypto/powerpc/chacha.h index 5d3d5506d7f94..1df6e1ce31c46 100644 --- a/lib/crypto/powerpc/chacha-p10-glue.c +++ b/lib/crypto/powerpc/chacha.h @@ -1,16 +1,14 @@ -// SPDX-License-Identifier: GPL-2.0-or-later +/* SPDX-License-Identifier: GPL-2.0-or-later */ /* * ChaCha stream cipher (P10 accelerated) * * Copyright 2023- IBM Corp. All rights reserved. */ =20 -#include #include #include -#include #include #include #include #include =20 @@ -46,19 +44,14 @@ static void chacha_p10_do_8x(struct chacha_state *state= , u8 *dst, const u8 *src, =20 if (bytes > 0) chacha_crypt_generic(state, dst, src, bytes, nrounds); } =20 -void hchacha_block_arch(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds) -{ - hchacha_block_generic(state, out, nrounds); -} -EXPORT_SYMBOL(hchacha_block_arch); +#define hchacha_block_arch hchacha_block_generic /* not implemented yet */ =20 -void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) +static void chacha_crypt_arch(struct chacha_state *state, u8 *dst, + const u8 *src, unsigned int bytes, int nrounds) { if (!static_branch_likely(&have_p10) || bytes <=3D CHACHA_BLOCK_SIZE || !crypto_simd_usable()) return chacha_crypt_generic(state, dst, src, bytes, nrounds); =20 @@ -72,23 +65,12 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, bytes -=3D todo; src +=3D todo; dst +=3D todo; } while (bytes); } -EXPORT_SYMBOL(chacha_crypt_arch); =20 -static int __init chacha_p10_init(void) +#define chacha_mod_init_arch chacha_mod_init_arch +static void chacha_mod_init_arch(void) { if (cpu_has_feature(CPU_FTR_ARCH_31)) static_branch_enable(&have_p10); - return 0; -} -subsys_initcall(chacha_p10_init); - -static void __exit chacha_p10_exit(void) -{ } -module_exit(chacha_p10_exit); - -MODULE_DESCRIPTION("ChaCha stream cipher (P10 accelerated)"); -MODULE_AUTHOR("Danny Tsen "); -MODULE_LICENSE("GPL v2"); diff --git a/lib/crypto/riscv/Kconfig b/lib/crypto/riscv/Kconfig deleted file mode 100644 index bc7a43f33eb3a..0000000000000 --- a/lib/crypto/riscv/Kconfig +++ /dev/null @@ -1,8 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -config CRYPTO_CHACHA_RISCV64 - tristate - depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO - default CRYPTO_LIB_CHACHA - select CRYPTO_ARCH_HAVE_LIB_CHACHA - select CRYPTO_LIB_CHACHA_GENERIC diff --git a/lib/crypto/riscv/Makefile b/lib/crypto/riscv/Makefile deleted file mode 100644 index e27b78f317fc8..0000000000000 --- a/lib/crypto/riscv/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) +=3D chacha-riscv64.o -chacha-riscv64-y :=3D chacha-riscv64-glue.o chacha-riscv64-zvkb.o diff --git a/lib/crypto/riscv/chacha-riscv64-glue.c b/lib/crypto/riscv/chac= ha.h similarity index 61% rename from lib/crypto/riscv/chacha-riscv64-glue.c rename to lib/crypto/riscv/chacha.h index a15f0aca3fc43..5c000c6aef4be 100644 --- a/lib/crypto/riscv/chacha-riscv64-glue.c +++ b/lib/crypto/riscv/chacha.h @@ -1,34 +1,27 @@ -// SPDX-License-Identifier: GPL-2.0-only +/* SPDX-License-Identifier: GPL-2.0-only */ /* * ChaCha stream cipher (RISC-V optimized) * * Copyright (C) 2023 SiFive, Inc. * Author: Jerry Shih */ =20 #include #include -#include #include #include -#include =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(use_zvkb); =20 asmlinkage void chacha_zvkb(struct chacha_state *state, const u8 *in, u8 *= out, size_t nblocks, int nrounds); =20 -void hchacha_block_arch(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds) -{ - hchacha_block_generic(state, out, nrounds); -} -EXPORT_SYMBOL(hchacha_block_arch); +#define hchacha_block_arch hchacha_block_generic /* not implemented yet */ =20 -void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) +static void chacha_crypt_arch(struct chacha_state *state, u8 *dst, + const u8 *src, unsigned int bytes, int nrounds) { u8 block_buffer[CHACHA_BLOCK_SIZE]; unsigned int full_blocks =3D bytes / CHACHA_BLOCK_SIZE; unsigned int tail_bytes =3D bytes % CHACHA_BLOCK_SIZE; =20 @@ -46,24 +39,13 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *= dst, const u8 *src, chacha_zvkb(state, block_buffer, block_buffer, 1, nrounds); memcpy(dst, block_buffer, tail_bytes); } kernel_vector_end(); } -EXPORT_SYMBOL(chacha_crypt_arch); =20 -static int __init riscv64_chacha_mod_init(void) +#define chacha_mod_init_arch chacha_mod_init_arch +static void chacha_mod_init_arch(void) { if (riscv_isa_extension_available(NULL, ZVKB) && riscv_vector_vlen() >=3D 128) static_branch_enable(&use_zvkb); - return 0; -} -subsys_initcall(riscv64_chacha_mod_init); - -static void __exit riscv64_chacha_mod_exit(void) -{ } -module_exit(riscv64_chacha_mod_exit); - -MODULE_DESCRIPTION("ChaCha stream cipher (RISC-V optimized)"); -MODULE_AUTHOR("Jerry Shih "); -MODULE_LICENSE("GPL"); diff --git a/lib/crypto/s390/Kconfig b/lib/crypto/s390/Kconfig deleted file mode 100644 index 069b355fe51aa..0000000000000 --- a/lib/crypto/s390/Kconfig +++ /dev/null @@ -1,7 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -config CRYPTO_CHACHA_S390 - tristate - default CRYPTO_LIB_CHACHA - select CRYPTO_LIB_CHACHA_GENERIC - select CRYPTO_ARCH_HAVE_LIB_CHACHA diff --git a/lib/crypto/s390/Makefile b/lib/crypto/s390/Makefile deleted file mode 100644 index 06c2cf77178ef..0000000000000 --- a/lib/crypto/s390/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -obj-$(CONFIG_CRYPTO_CHACHA_S390) +=3D chacha_s390.o -chacha_s390-y :=3D chacha-glue.o chacha-s390.o diff --git a/lib/crypto/s390/chacha-glue.c b/lib/crypto/s390/chacha.h similarity index 55% rename from lib/crypto/s390/chacha-glue.c rename to lib/crypto/s390/chacha.h index d8137387fe28f..fd9c4a4223656 100644 --- a/lib/crypto/s390/chacha-glue.c +++ b/lib/crypto/s390/chacha.h @@ -1,34 +1,23 @@ -// SPDX-License-Identifier: GPL-2.0 +/* SPDX-License-Identifier: GPL-2.0 */ /* * ChaCha stream cipher (s390 optimized) * * Copyright IBM Corp. 2021 */ =20 -#define KMSG_COMPONENT "chacha_s390" -#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt - -#include #include #include #include -#include #include #include #include "chacha-s390.h" =20 -void hchacha_block_arch(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds) -{ - /* TODO: implement hchacha_block_arch() in assembly */ - hchacha_block_generic(state, out, nrounds); -} -EXPORT_SYMBOL(hchacha_block_arch); +#define hchacha_block_arch hchacha_block_generic /* not implemented yet */ =20 -void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) +static void chacha_crypt_arch(struct chacha_state *state, u8 *dst, + const u8 *src, unsigned int bytes, int nrounds) { /* s390 chacha20 implementation has 20 rounds hard-coded, * it cannot handle a block of data or less, but otherwise * it can handle data of arbitrary size */ @@ -43,9 +32,5 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *ds= t, const u8 *src, =20 state->x[12] +=3D round_up(bytes, CHACHA_BLOCK_SIZE) / CHACHA_BLOCK_SIZE; } } -EXPORT_SYMBOL(chacha_crypt_arch); - -MODULE_DESCRIPTION("ChaCha stream cipher (s390 optimized)"); -MODULE_LICENSE("GPL v2"); diff --git a/lib/crypto/x86/Kconfig b/lib/crypto/x86/Kconfig index 24dc9a59b2728..eb47da71aa6b6 100644 --- a/lib/crypto/x86/Kconfig +++ b/lib/crypto/x86/Kconfig @@ -9,12 +9,5 @@ config CRYPTO_BLAKE2S_X86 BLAKE2s cryptographic hash function (RFC 7693) =20 Architecture: x86_64 using: - SSSE3 (Supplemental SSE3) - AVX-512 (Advanced Vector Extensions-512) - -config CRYPTO_CHACHA20_X86_64 - tristate - depends on 64BIT - default CRYPTO_LIB_CHACHA - select CRYPTO_LIB_CHACHA_GENERIC - select CRYPTO_ARCH_HAVE_LIB_CHACHA diff --git a/lib/crypto/x86/Makefile b/lib/crypto/x86/Makefile index 16c9d76f99472..4454556b243e7 100644 --- a/lib/crypto/x86/Makefile +++ b/lib/crypto/x86/Makefile @@ -1,7 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only =20 obj-$(CONFIG_CRYPTO_BLAKE2S_X86) +=3D libblake2s-x86_64.o libblake2s-x86_64-y :=3D blake2s-core.o blake2s-glue.o - -obj-$(CONFIG_CRYPTO_CHACHA20_X86_64) +=3D chacha-x86_64.o -chacha-x86_64-y :=3D chacha-avx2-x86_64.o chacha-ssse3-x86_64.o chacha-avx= 512vl-x86_64.o chacha_glue.o diff --git a/lib/crypto/x86/chacha_glue.c b/lib/crypto/x86/chacha.h similarity index 87% rename from lib/crypto/x86/chacha_glue.c rename to lib/crypto/x86/chacha.h index de7da9d512af2..10cf8f1c569dc 100644 --- a/lib/crypto/x86/chacha_glue.c +++ b/lib/crypto/x86/chacha.h @@ -1,17 +1,15 @@ -// SPDX-License-Identifier: GPL-2.0-or-later +/* SPDX-License-Identifier: GPL-2.0-or-later */ /* * ChaCha and HChaCha functions (x86_64 optimized) * * Copyright (C) 2015 Martin Willi */ =20 #include -#include #include #include -#include #include =20 asmlinkage void chacha_block_xor_ssse3(const struct chacha_state *state, u8 *dst, const u8 *src, unsigned int len, int nrounds); @@ -124,25 +122,24 @@ static void chacha_dosimd(struct chacha_state *state,= u8 *dst, const u8 *src, chacha_block_xor_ssse3(state, dst, src, bytes, nrounds); state->x[12]++; } } =20 -void hchacha_block_arch(const struct chacha_state *state, - u32 out[HCHACHA_OUT_WORDS], int nrounds) +static void hchacha_block_arch(const struct chacha_state *state, + u32 out[HCHACHA_OUT_WORDS], int nrounds) { if (!static_branch_likely(&chacha_use_simd)) { hchacha_block_generic(state, out, nrounds); } else { kernel_fpu_begin(); hchacha_block_ssse3(state, out, nrounds); kernel_fpu_end(); } } -EXPORT_SYMBOL(hchacha_block_arch); =20 -void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src, - unsigned int bytes, int nrounds) +static void chacha_crypt_arch(struct chacha_state *state, u8 *dst, + const u8 *src, unsigned int bytes, int nrounds) { if (!static_branch_likely(&chacha_use_simd) || bytes <=3D CHACHA_BLOCK_SIZE) return chacha_crypt_generic(state, dst, src, bytes, nrounds); =20 @@ -156,16 +153,16 @@ void chacha_crypt_arch(struct chacha_state *state, u8= *dst, const u8 *src, bytes -=3D todo; src +=3D todo; dst +=3D todo; } while (bytes); } -EXPORT_SYMBOL(chacha_crypt_arch); =20 -static int __init chacha_simd_mod_init(void) +#define chacha_mod_init_arch chacha_mod_init_arch +static void chacha_mod_init_arch(void) { if (!boot_cpu_has(X86_FEATURE_SSSE3)) - return 0; + return; =20 static_branch_enable(&chacha_use_simd); =20 if (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_AVX2) && @@ -174,17 +171,6 @@ static int __init chacha_simd_mod_init(void) =20 if (boot_cpu_has(X86_FEATURE_AVX512VL) && boot_cpu_has(X86_FEATURE_AVX512BW)) /* kmovq */ static_branch_enable(&chacha_use_avx512vl); } - return 0; } -subsys_initcall(chacha_simd_mod_init); - -static void __exit chacha_simd_mod_exit(void) -{ -} -module_exit(chacha_simd_mod_exit); - -MODULE_LICENSE("GPL"); -MODULE_AUTHOR("Martin Willi "); -MODULE_DESCRIPTION("ChaCha and HChaCha functions (x86_64 optimized)"); --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8366827E7F9; Wed, 27 Aug 2025 15:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307699; cv=none; b=jbV6zAPDoUveHjrShXKecFi+I0SoVIeFlZUCwO0mEfs3ltWoAolLEdxXQQN2szDuxzJfisiuoXYsNLneWt750Ba+GCYRq+IDuX/G5yjZh6NU3Lf2BRBNzXpky1PKgk7JhM/JPG0W3coA5RncENbj7UIuCVZsPMPCm384NMLryvM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307699; c=relaxed/simple; bh=NFRhzN4wSBfaVwKEFKFe99szQNdk4fuv7Txe2nbhvtk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Nx5f0ELwFbdULK1QRUVmi3mL6O8HTViJ66/hObeDD7vWsiboJxM1TbSDEsTy0I8qNsWYUZp/37lj4CGTMNcYGGqM10ho8AYOJH/i/6sj9ij08+h6mWHOa1qpPsgK8Z1gqG4wCihF7SeCkpqhyjECHGyofm3ZJzbR9mZyjB8clJo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sQRVjO43; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sQRVjO43" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 19953C4CEF9; Wed, 27 Aug 2025 15:14:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307699; bh=NFRhzN4wSBfaVwKEFKFe99szQNdk4fuv7Txe2nbhvtk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sQRVjO43EHxUlBVbNtQiqiymZScScOs2BakOrDe6zxv9CAukxhhGwL5UrzXyai7J4 eeRtxBTdguK+xrbqVLm5YkL9pMSnRselFMCPyjETrvVmuZB7colKipPtz5zNevMymN G50RF3C6C78aVSnYk8SpOZwr+OWs7NmdFvuKF+NHGtvJ3b/MxUKhFRM9AXlXnxa9Tk oN340/40uCL5zzxLs5j8iXgfIchEPlf94CUFJvFDQ3zq3c/QGYSZGjCnERdRKmxsr+ D++1E1ePxwWTtNvYkorsFEzNrKccjoOeH1Ikc/rgzDzqtPynCSm+j43ln9i4dS1oaa QyqJr1BnSW9pg== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 07/12] lib/crypto: x86/blake2s: Reduce size of BLAKE2S_SIGMA2 Date: Wed, 27 Aug 2025 08:11:26 -0700 Message-ID: <20250827151131.27733-8-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Save 480 bytes of .rodata by replacing the .long constants with .bytes, and using the vpmovzxbd instruction to expand them. Also update the code to do the loads before incrementing %rax rather than after. This avoids the need for the first load to use an offset. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- lib/crypto/x86/blake2s-core.S | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/lib/crypto/x86/blake2s-core.S b/lib/crypto/x86/blake2s-core.S index ac1c845445a4d..ef8e9f427aab3 100644 --- a/lib/crypto/x86/blake2s-core.S +++ b/lib/crypto/x86/blake2s-core.S @@ -27,23 +27,23 @@ SIGMA: .byte 2, 6, 0, 8, 12, 10, 11, 3, 1, 4, 7, 15, 9, 13, 5, 14 .byte 12, 1, 14, 4, 5, 15, 13, 10, 8, 0, 6, 9, 11, 7, 3, 2 .byte 13, 7, 12, 3, 11, 14, 1, 9, 2, 5, 15, 8, 10, 0, 4, 6 .byte 6, 14, 11, 0, 15, 9, 3, 8, 10, 12, 13, 1, 5, 2, 7, 4 .byte 10, 8, 7, 1, 2, 4, 6, 5, 13, 15, 9, 3, 0, 11, 14, 12 -.section .rodata.cst64.BLAKE2S_SIGMA2, "aM", @progbits, 640 +.section .rodata.cst64.BLAKE2S_SIGMA2, "aM", @progbits, 160 .align 64 SIGMA2: -.long 0, 2, 4, 6, 1, 3, 5, 7, 14, 8, 10, 12, 15, 9, 11, 13 -.long 8, 2, 13, 15, 10, 9, 12, 3, 6, 4, 0, 14, 5, 11, 1, 7 -.long 11, 13, 8, 6, 5, 10, 14, 3, 2, 4, 12, 15, 1, 0, 7, 9 -.long 11, 10, 7, 0, 8, 15, 1, 13, 3, 6, 2, 12, 4, 14, 9, 5 -.long 4, 10, 9, 14, 15, 0, 11, 8, 1, 7, 3, 13, 2, 5, 6, 12 -.long 2, 11, 4, 15, 14, 3, 10, 8, 13, 6, 5, 7, 0, 12, 1, 9 -.long 4, 8, 15, 9, 14, 11, 13, 5, 3, 2, 1, 12, 6, 10, 7, 0 -.long 6, 13, 0, 14, 12, 2, 1, 11, 15, 4, 5, 8, 7, 9, 3, 10 -.long 15, 5, 4, 13, 10, 7, 3, 11, 12, 2, 0, 6, 9, 8, 1, 14 -.long 8, 7, 14, 11, 13, 15, 0, 12, 10, 4, 5, 6, 3, 2, 1, 9 +.byte 0, 2, 4, 6, 1, 3, 5, 7, 14, 8, 10, 12, 15, 9, 11, 13 +.byte 8, 2, 13, 15, 10, 9, 12, 3, 6, 4, 0, 14, 5, 11, 1, 7 +.byte 11, 13, 8, 6, 5, 10, 14, 3, 2, 4, 12, 15, 1, 0, 7, 9 +.byte 11, 10, 7, 0, 8, 15, 1, 13, 3, 6, 2, 12, 4, 14, 9, 5 +.byte 4, 10, 9, 14, 15, 0, 11, 8, 1, 7, 3, 13, 2, 5, 6, 12 +.byte 2, 11, 4, 15, 14, 3, 10, 8, 13, 6, 5, 7, 0, 12, 1, 9 +.byte 4, 8, 15, 9, 14, 11, 13, 5, 3, 2, 1, 12, 6, 10, 7, 0 +.byte 6, 13, 0, 14, 12, 2, 1, 11, 15, 4, 5, 8, 7, 9, 3, 10 +.byte 15, 5, 4, 13, 10, 7, 3, 11, 12, 2, 0, 6, 9, 8, 1, 14 +.byte 8, 7, 14, 11, 13, 15, 0, 12, 10, 4, 5, 6, 3, 2, 1, 9 =20 .text SYM_FUNC_START(blake2s_compress_ssse3) testq %rdx,%rdx je .Lendofloop @@ -191,13 +191,13 @@ SYM_FUNC_START(blake2s_compress_avx512) vmovdqu 0x20(%rsi),%ymm7 addq $0x40,%rsi leaq SIGMA2(%rip),%rax movb $0xa,%cl .Lblake2s_compress_avx512_roundloop: - addq $0x40,%rax - vmovdqa -0x40(%rax),%ymm8 - vmovdqa -0x20(%rax),%ymm9 + vpmovzxbd (%rax),%ymm8 + vpmovzxbd 0x8(%rax),%ymm9 + addq $0x10,%rax vpermi2d %ymm7,%ymm6,%ymm8 vpermi2d %ymm7,%ymm6,%ymm9 vmovdqa %ymm8,%ymm6 vmovdqa %ymm9,%ymm7 vpaddd %xmm8,%xmm0,%xmm0 --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00E26283C90; Wed, 27 Aug 2025 15:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307700; cv=none; b=RGpSAvY6dztxyXIA6Nb5uX4xb6luqapGYQnbfjw4s2yBhsPHtHVZ6INTs6FG5JEOSn8AUQCuqqwi7VEVz2Q1R50aqtw8SVae3+Fx2ZEanIsXvBS4IJAxHld7UfrVl1hnS4bXYEIcBUY8Pp9yep7bgB3yr/BaaOQwpdRuY2AsNl4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307700; c=relaxed/simple; bh=SpUra98iL3SSp0jMXRLwqWw7shIkHxhCBSHFL/JhHj8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MIMQkQBLiBMHqbodAc4c/4iAZijYfqBQRuHv2Ot8gcyvT4/Z/NZMe31zupgyIQea7F8YrLH5dkRWhk5wjJz7wOZDW8Ccb2+Ydd67alKQEUYOs8loksHEbKUfWM8K6iSviFJlCMP16G/ZREnowLKme4n1DrrwmqaoTrMQS1GkzXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IVruVnoQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IVruVnoQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8A0C9C4CEEB; Wed, 27 Aug 2025 15:14:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307699; bh=SpUra98iL3SSp0jMXRLwqWw7shIkHxhCBSHFL/JhHj8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IVruVnoQpMKRGO66jASVuMctyXtUM4K1gY+Hs8ktTnA2b2I39MDyiZI6S4gyZUQOk O1z9C1/DtGb14eVQ65TmOXTtzqrMeQtMPfIr/ItRIrQ4w6J83ZxYHrXMZ0NKH0+dn9 71PbmyjyA9koFVx/fE5VOTK18+jUz54W4cqsYRPcuJ98gxD2ahzuFaPjrTTuoHbNs0 zKPB04Ov0vVxy+bS+Hn9IhYIpu+a7e3U40X/dLQpK219avJjzVq6qyYpZM0VlAmIY5 XD7QlKI7ORy+yLO53fvGvHBtnO0Eil47GEuagyvFUYV69O36jVyF9wWecgQ7KkpRLZ Je7enAzBdZgxQ== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 08/12] lib/crypto: blake2s: Remove obsolete self-test Date: Wed, 27 Aug 2025 08:11:27 -0700 Message-ID: <20250827151131.27733-9-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the original BLAKE2s self-test, since it will be superseded by blake2s_kunit. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- include/crypto/internal/blake2s.h | 2 - lib/crypto/Makefile | 1 - lib/crypto/blake2s-selftest.c | 651 ------------------------------ lib/crypto/blake2s.c | 10 - 4 files changed, 664 deletions(-) delete mode 100644 lib/crypto/blake2s-selftest.c diff --git a/include/crypto/internal/blake2s.h b/include/crypto/internal/bl= ake2s.h index 506d56530ca93..3b82572c94337 100644 --- a/include/crypto/internal/blake2s.h +++ b/include/crypto/internal/blake2s.h @@ -14,8 +14,6 @@ void blake2s_compress_generic(struct blake2s_state *state= , const u8 *block, size_t nblocks, const u32 inc); =20 void blake2s_compress(struct blake2s_state *state, const u8 *block, size_t nblocks, const u32 inc); =20 -bool blake2s_selftest(void); - #endif /* _CRYPTO_INTERNAL_BLAKE2S_H */ diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 33d7288265db2..eeddb993da033 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -31,11 +31,10 @@ obj-$(CONFIG_CRYPTO_LIB_GF128MUL) +=3D gf128mul.o =20 # blake2s is used by the /dev/random driver which is always builtin obj-y +=3D libblake2s.o libblake2s-y :=3D blake2s.o libblake2s-$(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC) +=3D blake2s-generic.o -libblake2s-$(CONFIG_CRYPTO_SELFTESTS) +=3D blake2s-selftest.o =20 ##########################################################################= ###### =20 # chacha20_block() is used by the /dev/random driver which is always built= in obj-y +=3D chacha-block-generic.o diff --git a/lib/crypto/blake2s-selftest.c b/lib/crypto/blake2s-selftest.c deleted file mode 100644 index d0634ed6a937f..0000000000000 --- a/lib/crypto/blake2s-selftest.c +++ /dev/null @@ -1,651 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 OR MIT -/* - * Copyright (C) 2015-2019 Jason A. Donenfeld . All Right= s Reserved. - */ - -#include -#include -#include -#include - -/* - * blake2s_testvecs[] generated with the program below (using libb2-dev and - * libssl-dev [OpenSSL]) - * - * #include - * #include - * #include - * - * #include - * - * #define BLAKE2S_TESTVEC_COUNT 256 - * - * static void print_vec(const uint8_t vec[], int len) - * { - * int i; - * - * printf(" { "); - * for (i =3D 0; i < len; i++) { - * if (i && (i % 12) =3D=3D 0) - * printf("\n "); - * printf("0x%02x, ", vec[i]); - * } - * printf("},\n"); - * } - * - * int main(void) - * { - * uint8_t key[BLAKE2S_KEYBYTES]; - * uint8_t buf[BLAKE2S_TESTVEC_COUNT]; - * uint8_t hash[BLAKE2S_OUTBYTES]; - * int i, j; - * - * key[0] =3D key[1] =3D 1; - * for (i =3D 2; i < BLAKE2S_KEYBYTES; ++i) - * key[i] =3D key[i - 2] + key[i - 1]; - * - * for (i =3D 0; i < BLAKE2S_TESTVEC_COUNT; ++i) - * buf[i] =3D (uint8_t)i; - * - * printf("static const u8 blake2s_testvecs[][BLAKE2S_HASH_SIZE] __initcon= st =3D {\n"); - * - * for (i =3D 0; i < BLAKE2S_TESTVEC_COUNT; ++i) { - * int outlen =3D 1 + i % BLAKE2S_OUTBYTES; - * int keylen =3D (13 * i) % (BLAKE2S_KEYBYTES + 1); - * - * blake2s(hash, buf, key + BLAKE2S_KEYBYTES - keylen, outlen, i, - * keylen); - * print_vec(hash, outlen); - * } - * printf("};\n\n"); - * - * return 0; - *} - */ -static const u8 blake2s_testvecs[][BLAKE2S_HASH_SIZE] __initconst =3D { - { 0xa1, }, - { 0x7c, 0x89, }, - { 0x74, 0x0e, 0xd4, }, - { 0x47, 0x0c, 0x21, 0x15, }, - { 0x18, 0xd6, 0x9c, 0xa6, 0xc4, }, - { 0x13, 0x5d, 0x16, 0x63, 0x2e, 0xf9, }, - { 0x2c, 0xb5, 0x04, 0xb7, 0x99, 0xe2, 0x73, }, - { 0x9a, 0x0f, 0xd2, 0x39, 0xd6, 0x68, 0x1b, 0x92, }, - { 0xc8, 0xde, 0x7a, 0xea, 0x2f, 0xf4, 0xd2, 0xe3, 0x2b, }, - { 0x5b, 0xf9, 0x43, 0x52, 0x0c, 0x12, 0xba, 0xb5, 0x93, 0x9f, }, - { 0xc6, 0x2c, 0x4e, 0x80, 0xfc, 0x32, 0x5b, 0x33, 0xb8, 0xb8, 0x0a, }, - { 0xa7, 0x5c, 0xfd, 0x3a, 0xcc, 0xbf, 0x90, 0xca, 0xb7, 0x97, 0xde, 0xd8= , }, - { 0x66, 0xca, 0x3c, 0xc4, 0x19, 0xef, 0x92, 0x66, 0x3f, 0x21, 0x8f, 0xda, - 0xb7, }, - { 0xba, 0xe5, 0xbb, 0x30, 0x25, 0x94, 0x6d, 0xc3, 0x89, 0x09, 0xc4, 0x25, - 0x52, 0x3e, }, - { 0xa2, 0xef, 0x0e, 0x52, 0x0b, 0x5f, 0xa2, 0x01, 0x6d, 0x0a, 0x25, 0xbc, - 0x57, 0xe2, 0x27, }, - { 0x4f, 0xe0, 0xf9, 0x52, 0x12, 0xda, 0x84, 0xb7, 0xab, 0xae, 0xb0, 0xa6, - 0x47, 0x2a, 0xc7, 0xf5, }, - { 0x56, 0xe7, 0xa8, 0x1c, 0x4c, 0xca, 0xed, 0x90, 0x31, 0xec, 0x87, 0x43, - 0xe7, 0x72, 0x08, 0xec, 0xbe, }, - { 0x7e, 0xdf, 0x80, 0x1c, 0x93, 0x33, 0xfd, 0x53, 0x44, 0xba, 0xfd, 0x96, - 0xe1, 0xbb, 0xb5, 0x65, 0xa5, 0x00, }, - { 0xec, 0x6b, 0xed, 0xf7, 0x7b, 0x62, 0x1d, 0x7d, 0xf4, 0x82, 0xf3, 0x1e, - 0x18, 0xff, 0x2b, 0xc4, 0x06, 0x20, 0x2a, }, - { 0x74, 0x98, 0xd7, 0x68, 0x63, 0xed, 0x87, 0xe4, 0x5d, 0x8d, 0x9e, 0x1d, - 0xfd, 0x2a, 0xbb, 0x86, 0xac, 0xe9, 0x2a, 0x89, }, - { 0x89, 0xc3, 0x88, 0xce, 0x2b, 0x33, 0x1e, 0x10, 0xd1, 0x37, 0x20, 0x86, - 0x28, 0x43, 0x70, 0xd9, 0xfb, 0x96, 0xd9, 0xb5, 0xd3, }, - { 0xcb, 0x56, 0x74, 0x41, 0x8d, 0x80, 0x01, 0x9a, 0x6b, 0x38, 0xe1, 0x41, - 0xad, 0x9c, 0x62, 0x74, 0xce, 0x35, 0xd5, 0x6c, 0x89, 0x6e, }, - { 0x79, 0xaf, 0x94, 0x59, 0x99, 0x26, 0xe1, 0xc9, 0x34, 0xfe, 0x7c, 0x22, - 0xf7, 0x43, 0xd7, 0x65, 0xd4, 0x48, 0x18, 0xac, 0x3d, 0xfd, 0x93, }, - { 0x85, 0x0d, 0xff, 0xb8, 0x3e, 0x87, 0x41, 0xb0, 0x95, 0xd3, 0x3d, 0x00, - 0x47, 0x55, 0x9e, 0xd2, 0x69, 0xea, 0xbf, 0xe9, 0x7a, 0x2d, 0x61, 0x45= , }, - { 0x03, 0xe0, 0x85, 0xec, 0x54, 0xb5, 0x16, 0x53, 0xa8, 0xc4, 0x71, 0xe9, - 0x6a, 0xe7, 0xcb, 0xc4, 0x15, 0x02, 0xfc, 0x34, 0xa4, 0xa4, 0x28, 0x13, - 0xd1, }, - { 0xe3, 0x34, 0x4b, 0xe1, 0xd0, 0x4b, 0x55, 0x61, 0x8f, 0xc0, 0x24, 0x05, - 0xe6, 0xe0, 0x3d, 0x70, 0x24, 0x4d, 0xda, 0xb8, 0x91, 0x05, 0x29, 0x07, - 0x01, 0x3e, }, - { 0x61, 0xff, 0x01, 0x72, 0xb1, 0x4d, 0xf6, 0xfe, 0xd1, 0xd1, 0x08, 0x74, - 0xe6, 0x91, 0x44, 0xeb, 0x61, 0xda, 0x40, 0xaf, 0xfc, 0x8c, 0x91, 0x6b, - 0xec, 0x13, 0xed, }, - { 0xd4, 0x40, 0xd2, 0xa0, 0x7f, 0xc1, 0x58, 0x0c, 0x85, 0xa0, 0x86, 0xc7, - 0x86, 0xb9, 0x61, 0xc9, 0xea, 0x19, 0x86, 0x1f, 0xab, 0x07, 0xce, 0x37, - 0x72, 0x67, 0x09, 0xfc, }, - { 0x9e, 0xf8, 0x18, 0x67, 0x93, 0x10, 0x9b, 0x39, 0x75, 0xe8, 0x8b, 0x38, - 0x82, 0x7d, 0xb8, 0xb7, 0xa5, 0xaf, 0xe6, 0x6a, 0x22, 0x5e, 0x1f, 0x9c, - 0x95, 0x29, 0x19, 0xf2, 0x4b, }, - { 0xc8, 0x62, 0x25, 0xf5, 0x98, 0xc9, 0xea, 0xe5, 0x29, 0x3a, 0xd3, 0x22, - 0xeb, 0xeb, 0x07, 0x7c, 0x15, 0x07, 0xee, 0x15, 0x61, 0xbb, 0x05, 0x30, - 0x99, 0x7f, 0x11, 0xf6, 0x0a, 0x1d, }, - { 0x68, 0x70, 0xf7, 0x90, 0xa1, 0x8b, 0x1f, 0x0f, 0xbb, 0xce, 0xd2, 0x0e, - 0x33, 0x1f, 0x7f, 0xa9, 0x78, 0xa8, 0xa6, 0x81, 0x66, 0xab, 0x8d, 0xcd, - 0x58, 0x55, 0x3a, 0x0b, 0x7a, 0xdb, 0xb5, }, - { 0xdd, 0x35, 0xd2, 0xb4, 0xf6, 0xc7, 0xea, 0xab, 0x64, 0x24, 0x4e, 0xfe, - 0xe5, 0x3d, 0x4e, 0x95, 0x8b, 0x6d, 0x6c, 0xbc, 0xb0, 0xf8, 0x88, 0x61, - 0x09, 0xb7, 0x78, 0xa3, 0x31, 0xfe, 0xd9, 0x2f, }, - { 0x0a, }, - { 0x6e, 0xd4, }, - { 0x64, 0xe9, 0xd1, }, - { 0x30, 0xdd, 0x71, 0xef, }, - { 0x11, 0xb5, 0x0c, 0x87, 0xc9, }, - { 0x06, 0x1c, 0x6d, 0x04, 0x82, 0xd0, }, - { 0x5c, 0x42, 0x0b, 0xee, 0xc5, 0x9c, 0xb2, }, - { 0xe8, 0x29, 0xd6, 0xb4, 0x5d, 0xf7, 0x2b, 0x93, }, - { 0x18, 0xca, 0x27, 0x72, 0x43, 0x39, 0x16, 0xbc, 0x6a, }, - { 0x39, 0x8f, 0xfd, 0x64, 0xf5, 0x57, 0x23, 0xb0, 0x45, 0xf8, }, - { 0xbb, 0x3a, 0x78, 0x6b, 0x02, 0x1d, 0x0b, 0x16, 0xe3, 0xb2, 0x9a, }, - { 0xb8, 0xb4, 0x0b, 0xe5, 0xd4, 0x1d, 0x0d, 0x85, 0x49, 0x91, 0x35, 0xfa= , }, - { 0x6d, 0x48, 0x2a, 0x0c, 0x42, 0x08, 0xbd, 0xa9, 0x78, 0x6f, 0x18, 0xaf, - 0xe2, }, - { 0x10, 0x45, 0xd4, 0x58, 0x88, 0xec, 0x4e, 0x1e, 0xf6, 0x14, 0x92, 0x64, - 0x7e, 0xb0, }, - { 0x8b, 0x0b, 0x95, 0xee, 0x92, 0xc6, 0x3b, 0x91, 0xf1, 0x1e, 0xeb, 0x51, - 0x98, 0x0a, 0x8d, }, - { 0xa3, 0x50, 0x4d, 0xa5, 0x1d, 0x03, 0x68, 0xe9, 0x57, 0x78, 0xd6, 0x04, - 0xf1, 0xc3, 0x94, 0xd8, }, - { 0xb8, 0x66, 0x6e, 0xdd, 0x46, 0x15, 0xae, 0x3d, 0x83, 0x7e, 0xcf, 0xe7, - 0x2c, 0xe8, 0x8f, 0xc7, 0x34, }, - { 0x2e, 0xc0, 0x1f, 0x29, 0xea, 0xf6, 0xb9, 0xe2, 0xc2, 0x93, 0xeb, 0x41, - 0x0d, 0xf0, 0x0a, 0x13, 0x0e, 0xa2, }, - { 0x71, 0xb8, 0x33, 0xa9, 0x1b, 0xac, 0xf1, 0xb5, 0x42, 0x8f, 0x5e, 0x81, - 0x34, 0x43, 0xb7, 0xa4, 0x18, 0x5c, 0x47, }, - { 0xda, 0x45, 0xb8, 0x2e, 0x82, 0x1e, 0xc0, 0x59, 0x77, 0x9d, 0xfa, 0xb4, - 0x1c, 0x5e, 0xa0, 0x2b, 0x33, 0x96, 0x5a, 0x58, }, - { 0xe3, 0x09, 0x05, 0xa9, 0xeb, 0x48, 0x13, 0xad, 0x71, 0x88, 0x81, 0x9a, - 0x3e, 0x2c, 0xe1, 0x23, 0x99, 0x13, 0x35, 0x9f, 0xb5, }, - { 0xb7, 0x86, 0x2d, 0x16, 0xe1, 0x04, 0x00, 0x47, 0x47, 0x61, 0x31, 0xfb, - 0x14, 0xac, 0xd8, 0xe9, 0xe3, 0x49, 0xbd, 0xf7, 0x9c, 0x3f, }, - { 0x7f, 0xd9, 0x95, 0xa8, 0xa7, 0xa0, 0xcc, 0xba, 0xef, 0xb1, 0x0a, 0xa9, - 0x21, 0x62, 0x08, 0x0f, 0x1b, 0xff, 0x7b, 0x9d, 0xae, 0xb2, 0x95, }, - { 0x85, 0x99, 0xea, 0x33, 0xe0, 0x56, 0xff, 0x13, 0xc6, 0x61, 0x8c, 0xf9, - 0x57, 0x05, 0x03, 0x11, 0xf9, 0xfb, 0x3a, 0xf7, 0xce, 0xbb, 0x52, 0x30= , }, - { 0xb2, 0x72, 0x9c, 0xf8, 0x77, 0x4e, 0x8f, 0x6b, 0x01, 0x6c, 0xff, 0x4e, - 0x4f, 0x02, 0xd2, 0xbc, 0xeb, 0x51, 0x28, 0x99, 0x50, 0xab, 0xc4, 0x42, - 0xe3, }, - { 0x8b, 0x0a, 0xb5, 0x90, 0x8f, 0xf5, 0x7b, 0xdd, 0xba, 0x47, 0x37, 0xc9, - 0x2a, 0xd5, 0x4b, 0x25, 0x08, 0x8b, 0x02, 0x17, 0xa7, 0x9e, 0x6b, 0x6e, - 0xe3, 0x90, }, - { 0x90, 0xdd, 0xf7, 0x75, 0xa7, 0xa3, 0x99, 0x5e, 0x5b, 0x7d, 0x75, 0xc3, - 0x39, 0x6b, 0xa0, 0xe2, 0x44, 0x53, 0xb1, 0x9e, 0xc8, 0xf1, 0x77, 0x10, - 0x58, 0x06, 0x9a, }, - { 0x99, 0x52, 0xf0, 0x49, 0xa8, 0x8c, 0xec, 0xa6, 0x97, 0x32, 0x13, 0xb5, - 0xf7, 0xa3, 0x8e, 0xfb, 0x4b, 0x59, 0x31, 0x3d, 0x01, 0x59, 0x98, 0x5d, - 0x53, 0x03, 0x1a, 0x39, }, - { 0x9f, 0xe0, 0xc2, 0xe5, 0x5d, 0x93, 0xd6, 0x9b, 0x47, 0x8f, 0x9b, 0xe0, - 0x26, 0x35, 0x84, 0x20, 0x1d, 0xc5, 0x53, 0x10, 0x0f, 0x22, 0xb9, 0xb5, - 0xd4, 0x36, 0xb1, 0xac, 0x73, }, - { 0x30, 0x32, 0x20, 0x3b, 0x10, 0x28, 0xec, 0x1f, 0x4f, 0x9b, 0x47, 0x59, - 0xeb, 0x7b, 0xee, 0x45, 0xfb, 0x0c, 0x49, 0xd8, 0x3d, 0x69, 0xbd, 0x90, - 0x2c, 0xf0, 0x9e, 0x8d, 0xbf, 0xd5, }, - { 0x2a, 0x37, 0x73, 0x7f, 0xf9, 0x96, 0x19, 0xaa, 0x25, 0xd8, 0x13, 0x28, - 0x01, 0x29, 0x89, 0xdf, 0x6e, 0x0c, 0x9b, 0x43, 0x44, 0x51, 0xe9, 0x75, - 0x26, 0x0c, 0xb7, 0x87, 0x66, 0x0b, 0x5f, }, - { 0x23, 0xdf, 0x96, 0x68, 0x91, 0x86, 0xd0, 0x93, 0x55, 0x33, 0x24, 0xf6, - 0xba, 0x08, 0x75, 0x5b, 0x59, 0x11, 0x69, 0xb8, 0xb9, 0xe5, 0x2c, 0x77, - 0x02, 0xf6, 0x47, 0xee, 0x81, 0xdd, 0xb9, 0x06, }, - { 0x9d, }, - { 0x9d, 0x7d, }, - { 0xfd, 0xc3, 0xda, }, - { 0xe8, 0x82, 0xcd, 0x21, }, - { 0xc3, 0x1d, 0x42, 0x4c, 0x74, }, - { 0xe9, 0xda, 0xf1, 0xa2, 0xe5, 0x7c, }, - { 0x52, 0xb8, 0x6f, 0x81, 0x5c, 0x3a, 0x4c, }, - { 0x5b, 0x39, 0x26, 0xfc, 0x92, 0x5e, 0xe0, 0x49, }, - { 0x59, 0xe4, 0x7c, 0x93, 0x1c, 0xf9, 0x28, 0x93, 0xde, }, - { 0xde, 0xdf, 0xb2, 0x43, 0x61, 0x0b, 0x86, 0x16, 0x4c, 0x2e, }, - { 0x14, 0x8f, 0x75, 0x51, 0xaf, 0xb9, 0xee, 0x51, 0x5a, 0xae, 0x23, }, - { 0x43, 0x5f, 0x50, 0xd5, 0x70, 0xb0, 0x5b, 0x87, 0xf5, 0xd9, 0xb3, 0x6d= , }, - { 0x66, 0x0a, 0x64, 0x93, 0x79, 0x71, 0x94, 0x40, 0xb7, 0x68, 0x2d, 0xd3, - 0x63, }, - { 0x15, 0x00, 0xc4, 0x0c, 0x7d, 0x1b, 0x10, 0xa9, 0x73, 0x1b, 0x90, 0x6f, - 0xe6, 0xa9, }, - { 0x34, 0x75, 0xf3, 0x86, 0x8f, 0x56, 0xcf, 0x2a, 0x0a, 0xf2, 0x62, 0x0a, - 0xf6, 0x0e, 0x20, }, - { 0xb1, 0xde, 0xc9, 0xf5, 0xdb, 0xf3, 0x2f, 0x4c, 0xd6, 0x41, 0x7d, 0x39, - 0x18, 0x3e, 0xc7, 0xc3, }, - { 0xc5, 0x89, 0xb2, 0xf8, 0xb8, 0xc0, 0xa3, 0xb9, 0x3b, 0x10, 0x6d, 0x7c, - 0x92, 0xfc, 0x7f, 0x34, 0x41, }, - { 0xc4, 0xd8, 0xef, 0xba, 0xef, 0xd2, 0xaa, 0xc5, 0x6c, 0x8e, 0x3e, 0xbb, - 0x12, 0xfc, 0x0f, 0x72, 0xbf, 0x0f, }, - { 0xdd, 0x91, 0xd1, 0x15, 0x9e, 0x7d, 0xf8, 0xc1, 0xb9, 0x14, 0x63, 0x96, - 0xb5, 0xcb, 0x83, 0x1d, 0x35, 0x1c, 0xec, }, - { 0xa9, 0xf8, 0x52, 0xc9, 0x67, 0x76, 0x2b, 0xad, 0xfb, 0xd8, 0x3a, 0xa6, - 0x74, 0x02, 0xae, 0xb8, 0x25, 0x2c, 0x63, 0x49, }, - { 0x77, 0x1f, 0x66, 0x70, 0xfd, 0x50, 0x29, 0xaa, 0xeb, 0xdc, 0xee, 0xba, - 0x75, 0x98, 0xdc, 0x93, 0x12, 0x3f, 0xdc, 0x7c, 0x38, }, - { 0xe2, 0xe1, 0x89, 0x5c, 0x37, 0x38, 0x6a, 0xa3, 0x40, 0xac, 0x3f, 0xb0, - 0xca, 0xfc, 0xa7, 0xf3, 0xea, 0xf9, 0x0f, 0x5d, 0x8e, 0x39, }, - { 0x0f, 0x67, 0xc8, 0x38, 0x01, 0xb1, 0xb7, 0xb8, 0xa2, 0xe7, 0x0a, 0x6d, - 0xd2, 0x63, 0x69, 0x9e, 0xcc, 0xf0, 0xf2, 0xbe, 0x9b, 0x98, 0xdd, }, - { 0x13, 0xe1, 0x36, 0x30, 0xfe, 0xc6, 0x01, 0x8a, 0xa1, 0x63, 0x96, 0x59, - 0xc2, 0xa9, 0x68, 0x3f, 0x58, 0xd4, 0x19, 0x0c, 0x40, 0xf3, 0xde, 0x02= , }, - { 0xa3, 0x9e, 0xce, 0xda, 0x42, 0xee, 0x8c, 0x6c, 0x5a, 0x7d, 0xdc, 0x89, - 0x02, 0x77, 0xdd, 0xe7, 0x95, 0xbb, 0xff, 0x0d, 0xa4, 0xb5, 0x38, 0x1e, - 0xaf, }, - { 0x9a, 0xf6, 0xb5, 0x9a, 0x4f, 0xa9, 0x4f, 0x2c, 0x35, 0x3c, 0x24, 0xdc, - 0x97, 0x6f, 0xd9, 0xa1, 0x7d, 0x1a, 0x85, 0x0b, 0xf5, 0xda, 0x2e, 0xe7, - 0xb1, 0x1d, }, - { 0x84, 0x1e, 0x8e, 0x3d, 0x45, 0xa5, 0xf2, 0x27, 0xf3, 0x31, 0xfe, 0xb9, - 0xfb, 0xc5, 0x45, 0x99, 0x99, 0xdd, 0x93, 0x43, 0x02, 0xee, 0x58, 0xaf, - 0xee, 0x6a, 0xbe, }, - { 0x07, 0x2f, 0xc0, 0xa2, 0x04, 0xc4, 0xab, 0x7c, 0x26, 0xbb, 0xa8, 0xd8, - 0xe3, 0x1c, 0x75, 0x15, 0x64, 0x5d, 0x02, 0x6a, 0xf0, 0x86, 0xe9, 0xcd, - 0x5c, 0xef, 0xa3, 0x25, }, - { 0x2f, 0x3b, 0x1f, 0xb5, 0x91, 0x8f, 0x86, 0xe0, 0xdc, 0x31, 0x48, 0xb6, - 0xa1, 0x8c, 0xfd, 0x75, 0xbb, 0x7d, 0x3d, 0xc1, 0xf0, 0x10, 0x9a, 0xd8, - 0x4b, 0x0e, 0xe3, 0x94, 0x9f, }, - { 0x29, 0xbb, 0x8f, 0x6c, 0xd1, 0xf2, 0xb6, 0xaf, 0xe5, 0xe3, 0x2d, 0xdc, - 0x6f, 0xa4, 0x53, 0x88, 0xd8, 0xcf, 0x4d, 0x45, 0x42, 0x62, 0xdb, 0xdf, - 0xf8, 0x45, 0xc2, 0x13, 0xec, 0x35, }, - { 0x06, 0x3c, 0xe3, 0x2c, 0x15, 0xc6, 0x43, 0x03, 0x81, 0xfb, 0x08, 0x76, - 0x33, 0xcb, 0x02, 0xc1, 0xba, 0x33, 0xe5, 0xe0, 0xd1, 0x92, 0xa8, 0x46, - 0x28, 0x3f, 0x3e, 0x9d, 0x2c, 0x44, 0x54, }, - { 0xea, 0xbb, 0x96, 0xf8, 0xd1, 0x8b, 0x04, 0x11, 0x40, 0x78, 0x42, 0x02, - 0x19, 0xd1, 0xbc, 0x65, 0x92, 0xd3, 0xc3, 0xd6, 0xd9, 0x19, 0xe7, 0xc3, - 0x40, 0x97, 0xbd, 0xd4, 0xed, 0xfa, 0x5e, 0x28, }, - { 0x02, }, - { 0x52, 0xa8, }, - { 0x38, 0x25, 0x0d, }, - { 0xe3, 0x04, 0xd4, 0x92, }, - { 0x97, 0xdb, 0xf7, 0x81, 0xca, }, - { 0x8a, 0x56, 0x9d, 0x62, 0x56, 0xcc, }, - { 0xa1, 0x8e, 0x3c, 0x72, 0x8f, 0x63, 0x03, }, - { 0xf7, 0xf3, 0x39, 0x09, 0x0a, 0xa1, 0xbb, 0x23, }, - { 0x6b, 0x03, 0xc0, 0xe9, 0xd9, 0x83, 0x05, 0x22, 0x01, }, - { 0x1b, 0x4b, 0xf5, 0xd6, 0x4f, 0x05, 0x75, 0x91, 0x4c, 0x7f, }, - { 0x4c, 0x8c, 0x25, 0x20, 0x21, 0xcb, 0xc2, 0x4b, 0x3a, 0x5b, 0x8d, }, - { 0x56, 0xe2, 0x77, 0xa0, 0xb6, 0x9f, 0x81, 0xec, 0x83, 0x75, 0xc4, 0xf9= , }, - { 0x71, 0x70, 0x0f, 0xad, 0x4d, 0x35, 0x81, 0x9d, 0x88, 0x69, 0xf9, 0xaa, - 0xd3, }, - { 0x50, 0x6e, 0x86, 0x6e, 0x43, 0xc0, 0xc2, 0x44, 0xc2, 0xe2, 0xa0, 0x1c, - 0xb7, 0x9a, }, - { 0xe4, 0x7e, 0x72, 0xc6, 0x12, 0x8e, 0x7c, 0xfc, 0xbd, 0xe2, 0x08, 0x31, - 0x3d, 0x47, 0x3d, }, - { 0x08, 0x97, 0x5b, 0x80, 0xae, 0xc4, 0x1d, 0x50, 0x77, 0xdf, 0x1f, 0xd0, - 0x24, 0xf0, 0x17, 0xc0, }, - { 0x01, 0xb6, 0x29, 0xf4, 0xaf, 0x78, 0x5f, 0xb6, 0x91, 0xdd, 0x76, 0x76, - 0xd2, 0xfd, 0x0c, 0x47, 0x40, }, - { 0xa1, 0xd8, 0x09, 0x97, 0x7a, 0xa6, 0xc8, 0x94, 0xf6, 0x91, 0x7b, 0xae, - 0x2b, 0x9f, 0x0d, 0x83, 0x48, 0xf7, }, - { 0x12, 0xd5, 0x53, 0x7d, 0x9a, 0xb0, 0xbe, 0xd9, 0xed, 0xe9, 0x9e, 0xee, - 0x61, 0x5b, 0x42, 0xf2, 0xc0, 0x73, 0xc0, }, - { 0xd5, 0x77, 0xd6, 0x5c, 0x6e, 0xa5, 0x69, 0x2b, 0x3b, 0x8c, 0xd6, 0x7d, - 0x1d, 0xbe, 0x2c, 0xa1, 0x02, 0x21, 0xcd, 0x29, }, - { 0xa4, 0x98, 0x80, 0xca, 0x22, 0xcf, 0x6a, 0xab, 0x5e, 0x40, 0x0d, 0x61, - 0x08, 0x21, 0xef, 0xc0, 0x6c, 0x52, 0xb4, 0xb0, 0x53, }, - { 0xbf, 0xaf, 0x8f, 0x3b, 0x7a, 0x97, 0x33, 0xe5, 0xca, 0x07, 0x37, 0xfd, - 0x15, 0xdf, 0xce, 0x26, 0x2a, 0xb1, 0xa7, 0x0b, 0xb3, 0xac, }, - { 0x16, 0x22, 0xe1, 0xbc, 0x99, 0x4e, 0x01, 0xf0, 0xfa, 0xff, 0x8f, 0xa5, - 0x0c, 0x61, 0xb0, 0xad, 0xcc, 0xb1, 0xe1, 0x21, 0x46, 0xfa, 0x2e, }, - { 0x11, 0x5b, 0x0b, 0x2b, 0xe6, 0x14, 0xc1, 0xd5, 0x4d, 0x71, 0x5e, 0x17, - 0xea, 0x23, 0xdd, 0x6c, 0xbd, 0x1d, 0xbe, 0x12, 0x1b, 0xee, 0x4c, 0x1a= , }, - { 0x40, 0x88, 0x22, 0xf3, 0x20, 0x6c, 0xed, 0xe1, 0x36, 0x34, 0x62, 0x2c, - 0x98, 0x83, 0x52, 0xe2, 0x25, 0xee, 0xe9, 0xf5, 0xe1, 0x17, 0xf0, 0x5c, - 0xae, }, - { 0xc3, 0x76, 0x37, 0xde, 0x95, 0x8c, 0xca, 0x2b, 0x0c, 0x23, 0xe7, 0xb5, - 0x38, 0x70, 0x61, 0xcc, 0xff, 0xd3, 0x95, 0x7b, 0xf3, 0xff, 0x1f, 0x9d, - 0x59, 0x00, }, - { 0x0c, 0x19, 0x52, 0x05, 0x22, 0x53, 0xcb, 0x48, 0xd7, 0x10, 0x0e, 0x7e, - 0x14, 0x69, 0xb5, 0xa2, 0x92, 0x43, 0xa3, 0x9e, 0x4b, 0x8f, 0x51, 0x2c, - 0x5a, 0x2c, 0x3b, }, - { 0xe1, 0x9d, 0x70, 0x70, 0x28, 0xec, 0x86, 0x40, 0x55, 0x33, 0x56, 0xda, - 0x88, 0xca, 0xee, 0xc8, 0x6a, 0x20, 0xb1, 0xe5, 0x3d, 0x57, 0xf8, 0x3c, - 0x10, 0x07, 0x2a, 0xc4, }, - { 0x0b, 0xae, 0xf1, 0xc4, 0x79, 0xee, 0x1b, 0x3d, 0x27, 0x35, 0x8d, 0x14, - 0xd6, 0xae, 0x4e, 0x3c, 0xe9, 0x53, 0x50, 0xb5, 0xcc, 0x0c, 0xf7, 0xdf, - 0xee, 0xa1, 0x74, 0xd6, 0x71, }, - { 0xe6, 0xa4, 0xf4, 0x99, 0x98, 0xb9, 0x80, 0xea, 0x96, 0x7f, 0x4f, 0x33, - 0xcf, 0x74, 0x25, 0x6f, 0x17, 0x6c, 0xbf, 0xf5, 0x5c, 0x38, 0xd0, 0xff, - 0x96, 0xcb, 0x13, 0xf9, 0xdf, 0xfd, }, - { 0xbe, 0x92, 0xeb, 0xba, 0x44, 0x2c, 0x24, 0x74, 0xd4, 0x03, 0x27, 0x3c, - 0x5d, 0x5b, 0x03, 0x30, 0x87, 0x63, 0x69, 0xe0, 0xb8, 0x94, 0xf4, 0x44, - 0x7e, 0xad, 0xcd, 0x20, 0x12, 0x16, 0x79, }, - { 0x30, 0xf1, 0xc4, 0x8e, 0x05, 0x90, 0x2a, 0x97, 0x63, 0x94, 0x46, 0xff, - 0xce, 0xd8, 0x67, 0xa7, 0xac, 0x33, 0x8c, 0x95, 0xb7, 0xcd, 0xa3, 0x23, - 0x98, 0x9d, 0x76, 0x6c, 0x9d, 0xa8, 0xd6, 0x8a, }, - { 0xbe, }, - { 0x17, 0x6c, }, - { 0x1a, 0x42, 0x4f, }, - { 0xba, 0xaf, 0xb7, 0x65, }, - { 0xc2, 0x63, 0x43, 0x6a, 0xea, }, - { 0xe4, 0x4d, 0xad, 0xf2, 0x0b, 0x02, }, - { 0x04, 0xc7, 0xc4, 0x7f, 0xa9, 0x2b, 0xce, }, - { 0x66, 0xf6, 0x67, 0xcb, 0x03, 0x53, 0xc8, 0xf1, }, - { 0x56, 0xa3, 0x60, 0x78, 0xc9, 0x5f, 0x70, 0x1b, 0x5e, }, - { 0x99, 0xff, 0x81, 0x7c, 0x13, 0x3c, 0x29, 0x79, 0x4b, 0x65, }, - { 0x51, 0x10, 0x50, 0x93, 0x01, 0x93, 0xb7, 0x01, 0xc9, 0x18, 0xb7, }, - { 0x8e, 0x3c, 0x42, 0x1e, 0x5e, 0x7d, 0xc1, 0x50, 0x70, 0x1f, 0x00, 0x98= , }, - { 0x5f, 0xd9, 0x9b, 0xc8, 0xd7, 0xb2, 0x72, 0x62, 0x1a, 0x1e, 0xba, 0x92, - 0xe9, }, - { 0x70, 0x2b, 0xba, 0xfe, 0xad, 0x5d, 0x96, 0x3f, 0x27, 0xc2, 0x41, 0x6d, - 0xc4, 0xb3, }, - { 0xae, 0xe0, 0xd5, 0xd4, 0xc7, 0xae, 0x15, 0x5e, 0xdc, 0xdd, 0x33, 0x60, - 0xd7, 0xd3, 0x5e, }, - { 0x79, 0x8e, 0xbc, 0x9e, 0x20, 0xb9, 0x19, 0x4b, 0x63, 0x80, 0xf3, 0x16, - 0xaf, 0x39, 0xbd, 0x92, }, - { 0xc2, 0x0e, 0x85, 0xa0, 0x0b, 0x9a, 0xb0, 0xec, 0xde, 0x38, 0xd3, 0x10, - 0xd9, 0xa7, 0x66, 0x27, 0xcf, }, - { 0x0e, 0x3b, 0x75, 0x80, 0x67, 0x14, 0x0c, 0x02, 0x90, 0xd6, 0xb3, 0x02, - 0x81, 0xf6, 0xa6, 0x87, 0xce, 0x58, }, - { 0x79, 0xb5, 0xe9, 0x5d, 0x52, 0x4d, 0xf7, 0x59, 0xf4, 0x2e, 0x27, 0xdd, - 0xb3, 0xed, 0x57, 0x5b, 0x82, 0xea, 0x6f, }, - { 0xa2, 0x97, 0xf5, 0x80, 0x02, 0x3d, 0xde, 0xa3, 0xf9, 0xf6, 0xab, 0xe3, - 0x57, 0x63, 0x7b, 0x9b, 0x10, 0x42, 0x6f, 0xf2, }, - { 0x12, 0x7a, 0xfc, 0xb7, 0x67, 0x06, 0x0c, 0x78, 0x1a, 0xfe, 0x88, 0x4f, - 0xc6, 0xac, 0x52, 0x96, 0x64, 0x28, 0x97, 0x84, 0x06, }, - { 0xc5, 0x04, 0x44, 0x6b, 0xb2, 0xa5, 0xa4, 0x66, 0xe1, 0x76, 0xa2, 0x51, - 0xf9, 0x59, 0x69, 0x97, 0x56, 0x0b, 0xbf, 0x50, 0xb3, 0x34, }, - { 0x21, 0x32, 0x6b, 0x42, 0xb5, 0xed, 0x71, 0x8d, 0xf7, 0x5a, 0x35, 0xe3, - 0x90, 0xe2, 0xee, 0xaa, 0x89, 0xf6, 0xc9, 0x9c, 0x4d, 0x73, 0xf4, }, - { 0x4c, 0xa6, 0x09, 0xf4, 0x48, 0xe7, 0x46, 0xbc, 0x49, 0xfc, 0xe5, 0xda, - 0xd1, 0x87, 0x13, 0x17, 0x4c, 0x59, 0x71, 0x26, 0x5b, 0x2c, 0x42, 0xb7= , }, - { 0x13, 0x63, 0xf3, 0x40, 0x02, 0xe5, 0xa3, 0x3a, 0x5e, 0x8e, 0xf8, 0xb6, - 0x8a, 0x49, 0x60, 0x76, 0x34, 0x72, 0x94, 0x73, 0xf6, 0xd9, 0x21, 0x6a, - 0x26, }, - { 0xdf, 0x75, 0x16, 0x10, 0x1b, 0x5e, 0x81, 0xc3, 0xc8, 0xde, 0x34, 0x24, - 0xb0, 0x98, 0xeb, 0x1b, 0x8f, 0xa1, 0x9b, 0x05, 0xee, 0xa5, 0xe9, 0x35, - 0xf4, 0x1d, }, - { 0xcd, 0x21, 0x93, 0x6e, 0x5b, 0xa0, 0x26, 0x2b, 0x21, 0x0e, 0xa0, 0xb9, - 0x1c, 0xb5, 0xbb, 0xb8, 0xf8, 0x1e, 0xff, 0x5c, 0xa8, 0xf9, 0x39, 0x46, - 0x4e, 0x29, 0x26, }, - { 0x73, 0x7f, 0x0e, 0x3b, 0x0b, 0x5c, 0xf9, 0x60, 0xaa, 0x88, 0xa1, 0x09, - 0xb1, 0x5d, 0x38, 0x7b, 0x86, 0x8f, 0x13, 0x7a, 0x8d, 0x72, 0x7a, 0x98, - 0x1a, 0x5b, 0xff, 0xc9, }, - { 0xd3, 0x3c, 0x61, 0x71, 0x44, 0x7e, 0x31, 0x74, 0x98, 0x9d, 0x9a, 0xd2, - 0x27, 0xf3, 0x46, 0x43, 0x42, 0x51, 0xd0, 0x5f, 0xe9, 0x1c, 0x5c, 0x69, - 0xbf, 0xf6, 0xbe, 0x3c, 0x40, }, - { 0x31, 0x99, 0x31, 0x9f, 0xaa, 0x43, 0x2e, 0x77, 0x3e, 0x74, 0x26, 0x31, - 0x5e, 0x61, 0xf1, 0x87, 0xe2, 0xeb, 0x9b, 0xcd, 0xd0, 0x3a, 0xee, 0x20, - 0x7e, 0x10, 0x0a, 0x0b, 0x7e, 0xfa, }, - { 0xa4, 0x27, 0x80, 0x67, 0x81, 0x2a, 0xa7, 0x62, 0xf7, 0x6e, 0xda, 0xd4, - 0x5c, 0x39, 0x74, 0xad, 0x7e, 0xbe, 0xad, 0xa5, 0x84, 0x7f, 0xa9, 0x30, - 0x5d, 0xdb, 0xe2, 0x05, 0x43, 0xf7, 0x1b, }, - { 0x0b, 0x37, 0xd8, 0x02, 0xe1, 0x83, 0xd6, 0x80, 0xf2, 0x35, 0xc2, 0xb0, - 0x37, 0xef, 0xef, 0x5e, 0x43, 0x93, 0xf0, 0x49, 0x45, 0x0a, 0xef, 0xb5, - 0x76, 0x70, 0x12, 0x44, 0xc4, 0xdb, 0xf5, 0x7a, }, - { 0x1f, }, - { 0x82, 0x60, }, - { 0xcc, 0xe3, 0x08, }, - { 0x56, 0x17, 0xe4, 0x59, }, - { 0xe2, 0xd7, 0x9e, 0xc4, 0x4c, }, - { 0xb2, 0xad, 0xd3, 0x78, 0x58, 0x5a, }, - { 0xce, 0x43, 0xb4, 0x02, 0x96, 0xab, 0x3c, }, - { 0xe6, 0x05, 0x1a, 0x73, 0x22, 0x32, 0xbb, 0x77, }, - { 0x23, 0xe7, 0xda, 0xfe, 0x2c, 0xef, 0x8c, 0x22, 0xec, }, - { 0xe9, 0x8e, 0x55, 0x38, 0xd1, 0xd7, 0x35, 0x23, 0x98, 0xc7, }, - { 0xb5, 0x81, 0x1a, 0xe5, 0xb5, 0xa5, 0xd9, 0x4d, 0xca, 0x41, 0xe7, }, - { 0x41, 0x16, 0x16, 0x95, 0x8d, 0x9e, 0x0c, 0xea, 0x8c, 0x71, 0x9a, 0xc1= , }, - { 0x7c, 0x33, 0xc0, 0xa4, 0x00, 0x62, 0xea, 0x60, 0x67, 0xe4, 0x20, 0xbc, - 0x5b, }, - { 0xdb, 0xb1, 0xdc, 0xfd, 0x08, 0xc0, 0xde, 0x82, 0xd1, 0xde, 0x38, 0xc0, - 0x90, 0x48, }, - { 0x37, 0x18, 0x2e, 0x0d, 0x61, 0xaa, 0x61, 0xd7, 0x86, 0x20, 0x16, 0x60, - 0x04, 0xd9, 0xd5, }, - { 0xb0, 0xcf, 0x2c, 0x4c, 0x5e, 0x5b, 0x4f, 0x2a, 0x23, 0x25, 0x58, 0x47, - 0xe5, 0x31, 0x06, 0x70, }, - { 0x91, 0xa0, 0xa3, 0x86, 0x4e, 0xe0, 0x72, 0x38, 0x06, 0x67, 0x59, 0x5c, - 0x70, 0x25, 0xdb, 0x33, 0x27, }, - { 0x44, 0x58, 0x66, 0xb8, 0x58, 0xc7, 0x13, 0xed, 0x4c, 0xc0, 0xf4, 0x9a, - 0x1e, 0x67, 0x75, 0x33, 0xb6, 0xb8, }, - { 0x7f, 0x98, 0x4a, 0x8e, 0x50, 0xa2, 0x5c, 0xcd, 0x59, 0xde, 0x72, 0xb3, - 0x9d, 0xc3, 0x09, 0x8a, 0xab, 0x56, 0xf1, }, - { 0x80, 0x96, 0x49, 0x1a, 0x59, 0xa2, 0xc5, 0xd5, 0xa7, 0x20, 0x8a, 0xb7, - 0x27, 0x62, 0x84, 0x43, 0xc6, 0xe1, 0x1b, 0x5d, }, - { 0x6b, 0xb7, 0x2b, 0x26, 0x62, 0x14, 0x70, 0x19, 0x3d, 0x4d, 0xac, 0xac, - 0x63, 0x58, 0x5e, 0x94, 0xb5, 0xb7, 0xe8, 0xe8, 0xa2, }, - { 0x20, 0xa8, 0xc0, 0xfd, 0x63, 0x3d, 0x6e, 0x98, 0xcf, 0x0c, 0x49, 0x98, - 0xe4, 0x5a, 0xfe, 0x8c, 0xaa, 0x70, 0x82, 0x1c, 0x7b, 0x74, }, - { 0xc8, 0xe8, 0xdd, 0xdf, 0x69, 0x30, 0x01, 0xc2, 0x0f, 0x7e, 0x2f, 0x11, - 0xcc, 0x3e, 0x17, 0xa5, 0x69, 0x40, 0x3f, 0x0e, 0x79, 0x7f, 0xcf, }, - { 0xdb, 0x61, 0xc0, 0xe2, 0x2e, 0x49, 0x07, 0x31, 0x1d, 0x91, 0x42, 0x8a, - 0xfc, 0x5e, 0xd3, 0xf8, 0x56, 0x1f, 0x2b, 0x73, 0xfd, 0x9f, 0xb2, 0x8e= , }, - { 0x0c, 0x89, 0x55, 0x0c, 0x1f, 0x59, 0x2c, 0x9d, 0x1b, 0x29, 0x1d, 0x41, - 0x1d, 0xe6, 0x47, 0x8f, 0x8c, 0x2b, 0xea, 0x8f, 0xf0, 0xff, 0x21, 0x70, - 0x88, }, - { 0x12, 0x18, 0x95, 0xa6, 0x59, 0xb1, 0x31, 0x24, 0x45, 0x67, 0x55, 0xa4, - 0x1a, 0x2d, 0x48, 0x67, 0x1b, 0x43, 0x88, 0x2d, 0x8e, 0xa0, 0x70, 0xb3, - 0xc6, 0xbb, }, - { 0xe7, 0xb1, 0x1d, 0xb2, 0x76, 0x4d, 0x68, 0x68, 0x68, 0x23, 0x02, 0x55, - 0x3a, 0xe2, 0xe5, 0xd5, 0x4b, 0x43, 0xf9, 0x34, 0x77, 0x5c, 0xa1, 0xf5, - 0x55, 0xfd, 0x4f, }, - { 0x8c, 0x87, 0x5a, 0x08, 0x3a, 0x73, 0xad, 0x61, 0xe1, 0xe7, 0x99, 0x7e, - 0xf0, 0x5d, 0xe9, 0x5d, 0x16, 0x43, 0x80, 0x2f, 0xd0, 0x66, 0x34, 0xe2, - 0x42, 0x64, 0x3b, 0x1a, }, - { 0x39, 0xc1, 0x99, 0xcf, 0x22, 0xbf, 0x16, 0x8f, 0x9f, 0x80, 0x7f, 0x95, - 0x0a, 0x05, 0x67, 0x27, 0xe7, 0x15, 0xdf, 0x9d, 0xb2, 0xfe, 0x1c, 0xb5, - 0x1d, 0x60, 0x8f, 0x8a, 0x1d, }, - { 0x9b, 0x6e, 0x08, 0x09, 0x06, 0x73, 0xab, 0x68, 0x02, 0x62, 0x1a, 0xe4, - 0xd4, 0xdf, 0xc7, 0x02, 0x4c, 0x6a, 0x5f, 0xfd, 0x23, 0xac, 0xae, 0x6d, - 0x43, 0xa4, 0x7a, 0x50, 0x60, 0x3c, }, - { 0x1d, 0xb4, 0xc6, 0xe1, 0xb1, 0x4b, 0xe3, 0xf2, 0xe2, 0x1a, 0x73, 0x1b, - 0xa0, 0x92, 0xa7, 0xf5, 0xff, 0x8f, 0x8b, 0x5d, 0xdf, 0xa8, 0x04, 0xb3, - 0xb0, 0xf7, 0xcc, 0x12, 0xfa, 0x35, 0x46, }, - { 0x49, 0x45, 0x97, 0x11, 0x0f, 0x1c, 0x60, 0x8e, 0xe8, 0x47, 0x30, 0xcf, - 0x60, 0xa8, 0x71, 0xc5, 0x1b, 0xe9, 0x39, 0x4d, 0x49, 0xb6, 0x12, 0x1f, - 0x24, 0xab, 0x37, 0xff, 0x83, 0xc2, 0xe1, 0x3a, }, - { 0x60, }, - { 0x24, 0x26, }, - { 0x47, 0xeb, 0xc9, }, - { 0x4a, 0xd0, 0xbc, 0xf0, }, - { 0x8e, 0x2b, 0xc9, 0x85, 0x3c, }, - { 0xa2, 0x07, 0x15, 0xb8, 0x12, 0x74, }, - { 0x0f, 0xdb, 0x5b, 0x33, 0x69, 0xfe, 0x4b, }, - { 0xa2, 0x86, 0x54, 0xf4, 0xfd, 0xb2, 0xd4, 0xe6, }, - { 0xbb, 0x84, 0x78, 0x49, 0x27, 0x8e, 0x61, 0xda, 0x60, }, - { 0x04, 0xc3, 0xcd, 0xaa, 0x8f, 0xa7, 0x03, 0xc9, 0xf9, 0xb6, }, - { 0xf8, 0x27, 0x1d, 0x61, 0xdc, 0x21, 0x42, 0xdd, 0xad, 0x92, 0x40, }, - { 0x12, 0x87, 0xdf, 0xc2, 0x41, 0x45, 0x5a, 0x36, 0x48, 0x5b, 0x51, 0x2b= , }, - { 0xbb, 0x37, 0x5d, 0x1f, 0xf1, 0x68, 0x7a, 0xc4, 0xa5, 0xd2, 0xa4, 0x91, - 0x8d, }, - { 0x5b, 0x27, 0xd1, 0x04, 0x54, 0x52, 0x9f, 0xa3, 0x47, 0x86, 0x33, 0x33, - 0xbf, 0xa0, }, - { 0xcf, 0x04, 0xea, 0xf8, 0x03, 0x2a, 0x43, 0xff, 0xa6, 0x68, 0x21, 0x4c, - 0xd5, 0x4b, 0xed, }, - { 0xaf, 0xb8, 0xbc, 0x63, 0x0f, 0x18, 0x4d, 0xe2, 0x7a, 0xdd, 0x46, 0x44, - 0xc8, 0x24, 0x0a, 0xb7, }, - { 0x3e, 0xdc, 0x36, 0xe4, 0x89, 0xb1, 0xfa, 0xc6, 0x40, 0x93, 0x2e, 0x75, - 0xb2, 0x15, 0xd1, 0xb1, 0x10, }, - { 0x6c, 0xd8, 0x20, 0x3b, 0x82, 0x79, 0xf9, 0xc8, 0xbc, 0x9d, 0xe0, 0x35, - 0xbe, 0x1b, 0x49, 0x1a, 0xbc, 0x3a, }, - { 0x78, 0x65, 0x2c, 0xbe, 0x35, 0x67, 0xdc, 0x78, 0xd4, 0x41, 0xf6, 0xc9, - 0xde, 0xde, 0x1f, 0x18, 0x13, 0x31, 0x11, }, - { 0x8a, 0x7f, 0xb1, 0x33, 0x8f, 0x0c, 0x3c, 0x0a, 0x06, 0x61, 0xf0, 0x47, - 0x29, 0x1b, 0x29, 0xbc, 0x1c, 0x47, 0xef, 0x7a, }, - { 0x65, 0x91, 0xf1, 0xe6, 0xb3, 0x96, 0xd3, 0x8c, 0xc2, 0x4a, 0x59, 0x35, - 0x72, 0x8e, 0x0b, 0x9a, 0x87, 0xca, 0x34, 0x7b, 0x63, }, - { 0x5f, 0x08, 0x87, 0x80, 0x56, 0x25, 0x89, 0x77, 0x61, 0x8c, 0x64, 0xa1, - 0x59, 0x6d, 0x59, 0x62, 0xe8, 0x4a, 0xc8, 0x58, 0x99, 0xd1, }, - { 0x23, 0x87, 0x1d, 0xed, 0x6f, 0xf2, 0x91, 0x90, 0xe2, 0xfe, 0x43, 0x21, - 0xaf, 0x97, 0xc6, 0xbc, 0xd7, 0x15, 0xc7, 0x2d, 0x08, 0x77, 0x91, }, - { 0x90, 0x47, 0x9a, 0x9e, 0x3a, 0xdf, 0xf3, 0xc9, 0x4c, 0x1e, 0xa7, 0xd4, - 0x6a, 0x32, 0x90, 0xfe, 0xb7, 0xb6, 0x7b, 0xfa, 0x96, 0x61, 0xfb, 0xa4= , }, - { 0xb1, 0x67, 0x60, 0x45, 0xb0, 0x96, 0xc5, 0x15, 0x9f, 0x4d, 0x26, 0xd7, - 0x9d, 0xf1, 0xf5, 0x6d, 0x21, 0x00, 0x94, 0x31, 0x64, 0x94, 0xd3, 0xa7, - 0xd3, }, - { 0x02, 0x3e, 0xaf, 0xf3, 0x79, 0x73, 0xa5, 0xf5, 0xcc, 0x7a, 0x7f, 0xfb, - 0x79, 0x2b, 0x85, 0x8c, 0x88, 0x72, 0x06, 0xbe, 0xfe, 0xaf, 0xc1, 0x16, - 0xa6, 0xd6, }, - { 0x2a, 0xb0, 0x1a, 0xe5, 0xaa, 0x6e, 0xb3, 0xae, 0x53, 0x85, 0x33, 0x80, - 0x75, 0xae, 0x30, 0xe6, 0xb8, 0x72, 0x42, 0xf6, 0x25, 0x4f, 0x38, 0x88, - 0x55, 0xd1, 0xa9, }, - { 0x90, 0xd8, 0x0c, 0xc0, 0x93, 0x4b, 0x4f, 0x9e, 0x65, 0x6c, 0xa1, 0x54, - 0xa6, 0xf6, 0x6e, 0xca, 0xd2, 0xbb, 0x7e, 0x6a, 0x1c, 0xd3, 0xce, 0x46, - 0xef, 0xb0, 0x00, 0x8d, }, - { 0xed, 0x9c, 0x49, 0xcd, 0xc2, 0xde, 0x38, 0x0e, 0xe9, 0x98, 0x6c, 0xc8, - 0x90, 0x9e, 0x3c, 0xd4, 0xd3, 0xeb, 0x88, 0x32, 0xc7, 0x28, 0xe3, 0x94, - 0x1c, 0x9f, 0x8b, 0xf3, 0xcb, }, - { 0xac, 0xe7, 0x92, 0x16, 0xb4, 0x14, 0xa0, 0xe4, 0x04, 0x79, 0xa2, 0xf4, - 0x31, 0xe6, 0x0c, 0x26, 0xdc, 0xbf, 0x2f, 0x69, 0x1b, 0x55, 0x94, 0x67, - 0xda, 0x0c, 0xd7, 0x32, 0x1f, 0xef, }, - { 0x68, 0x63, 0x85, 0x57, 0x95, 0x9e, 0x42, 0x27, 0x41, 0x43, 0x42, 0x02, - 0xa5, 0x78, 0xa7, 0xc6, 0x43, 0xc1, 0x6a, 0xba, 0x70, 0x80, 0xcd, 0x04, - 0xb6, 0x78, 0x76, 0x29, 0xf3, 0xe8, 0xa0, }, - { 0xe6, 0xac, 0x8d, 0x9d, 0xf0, 0xc0, 0xf7, 0xf7, 0xe3, 0x3e, 0x4e, 0x28, - 0x0f, 0x59, 0xb2, 0x67, 0x9e, 0x84, 0x34, 0x42, 0x96, 0x30, 0x2b, 0xca, - 0x49, 0xb6, 0xc5, 0x9a, 0x84, 0x59, 0xa7, 0x81, }, - { 0x7e, }, - { 0x1e, 0x21, }, - { 0x26, 0xd3, 0xdd, }, - { 0x2c, 0xd4, 0xb3, 0x3d, }, - { 0x86, 0x7b, 0x76, 0x3c, 0xf0, }, - { 0x12, 0xc3, 0x70, 0x1d, 0x55, 0x18, }, - { 0x96, 0xc2, 0xbd, 0x61, 0x55, 0xf4, 0x24, }, - { 0x20, 0x51, 0xf7, 0x86, 0x58, 0x8f, 0x07, 0x2a, }, - { 0x93, 0x15, 0xa8, 0x1d, 0xda, 0x97, 0xee, 0x0e, 0x6c, }, - { 0x39, 0x93, 0xdf, 0xd5, 0x0e, 0xca, 0xdc, 0x7a, 0x92, 0xce, }, - { 0x60, 0xd5, 0xfd, 0xf5, 0x1b, 0x26, 0x82, 0x26, 0x73, 0x02, 0xbc, }, - { 0x98, 0xf2, 0x34, 0xe1, 0xf5, 0xfb, 0x00, 0xac, 0x10, 0x4a, 0x38, 0x9f= , }, - { 0xda, 0x3a, 0x92, 0x8a, 0xd0, 0xcd, 0x12, 0xcd, 0x15, 0xbb, 0xab, 0x77, - 0x66, }, - { 0xa2, 0x92, 0x1a, 0xe5, 0xca, 0x0c, 0x30, 0x75, 0xeb, 0xaf, 0x00, 0x31, - 0x55, 0x66, }, - { 0x06, 0xea, 0xfd, 0x3e, 0x86, 0x38, 0x62, 0x4e, 0xa9, 0x12, 0xa4, 0x12, - 0x43, 0xbf, 0xa1, }, - { 0xe4, 0x71, 0x7b, 0x94, 0xdb, 0xa0, 0xd2, 0xff, 0x9b, 0xeb, 0xad, 0x8e, - 0x95, 0x8a, 0xc5, 0xed, }, - { 0x25, 0x5a, 0x77, 0x71, 0x41, 0x0e, 0x7a, 0xe9, 0xed, 0x0c, 0x10, 0xef, - 0xf6, 0x2b, 0x3a, 0xba, 0x60, }, - { 0xee, 0xe2, 0xa3, 0x67, 0x64, 0x1d, 0xc6, 0x04, 0xc4, 0xe1, 0x68, 0xd2, - 0x6e, 0xd2, 0x91, 0x75, 0x53, 0x07, }, - { 0xe0, 0xf6, 0x4d, 0x8f, 0x68, 0xfc, 0x06, 0x7e, 0x18, 0x79, 0x7f, 0x2b, - 0x6d, 0xef, 0x46, 0x7f, 0xab, 0xb2, 0xad, }, - { 0x3d, 0x35, 0x88, 0x9f, 0x2e, 0xcf, 0x96, 0x45, 0x07, 0x60, 0x71, 0x94, - 0x00, 0x8d, 0xbf, 0xf4, 0xef, 0x46, 0x2e, 0x3c, }, - { 0x43, 0xcf, 0x98, 0xf7, 0x2d, 0xf4, 0x17, 0xe7, 0x8c, 0x05, 0x2d, 0x9b, - 0x24, 0xfb, 0x4d, 0xea, 0x4a, 0xec, 0x01, 0x25, 0x29, }, - { 0x8e, 0x73, 0x9a, 0x78, 0x11, 0xfe, 0x48, 0xa0, 0x3b, 0x1a, 0x26, 0xdf, - 0x25, 0xe9, 0x59, 0x1c, 0x70, 0x07, 0x9f, 0xdc, 0xa0, 0xa6, }, - { 0xe8, 0x47, 0x71, 0xc7, 0x3e, 0xdf, 0xb5, 0x13, 0xb9, 0x85, 0x13, 0xa8, - 0x54, 0x47, 0x6e, 0x59, 0x96, 0x09, 0x13, 0x5f, 0x82, 0x16, 0x0b, }, - { 0xfb, 0xc0, 0x8c, 0x03, 0x21, 0xb3, 0xc4, 0xb5, 0x43, 0x32, 0x6c, 0xea, - 0x7f, 0xa8, 0x43, 0x91, 0xe8, 0x4e, 0x3f, 0xbf, 0x45, 0x58, 0x6a, 0xa3= , }, - { 0x55, 0xf8, 0xf3, 0x00, 0x76, 0x09, 0xef, 0x69, 0x5d, 0xd2, 0x8a, 0xf2, - 0x65, 0xc3, 0xcb, 0x9b, 0x43, 0xfd, 0xb1, 0x7e, 0x7f, 0xa1, 0x94, 0xb0, - 0xd7, }, - { 0xaa, 0x13, 0xc1, 0x51, 0x40, 0x6d, 0x8d, 0x4c, 0x0a, 0x95, 0x64, 0x7b, - 0xd1, 0x96, 0xb6, 0x56, 0xb4, 0x5b, 0xcf, 0xd6, 0xd9, 0x15, 0x97, 0xdd, - 0xb6, 0xef, }, - { 0xaf, 0xb7, 0x36, 0xb0, 0x04, 0xdb, 0xd7, 0x9c, 0x9a, 0x44, 0xc4, 0xf6, - 0x1f, 0x12, 0x21, 0x2d, 0x59, 0x30, 0x54, 0xab, 0x27, 0x61, 0xa3, 0x57, - 0xef, 0xf8, 0x53, }, - { 0x97, 0x34, 0x45, 0x3e, 0xce, 0x7c, 0x35, 0xa2, 0xda, 0x9f, 0x4b, 0x46, - 0x6c, 0x11, 0x67, 0xff, 0x2f, 0x76, 0x58, 0x15, 0x71, 0xfa, 0x44, 0x89, - 0x89, 0xfd, 0xf7, 0x99, }, - { 0x1f, 0xb1, 0x62, 0xeb, 0x83, 0xc5, 0x9c, 0x89, 0xf9, 0x2c, 0xd2, 0x03, - 0x61, 0xbc, 0xbb, 0xa5, 0x74, 0x0e, 0x9b, 0x7e, 0x82, 0x3e, 0x70, 0x0a, - 0xa9, 0x8f, 0x2b, 0x59, 0xfb, }, - { 0xf8, 0xca, 0x5e, 0x3a, 0x4f, 0x9e, 0x10, 0x69, 0x10, 0xd5, 0x4c, 0xeb, - 0x1a, 0x0f, 0x3c, 0x6a, 0x98, 0xf5, 0xb0, 0x97, 0x5b, 0x37, 0x2f, 0x0d, - 0xbd, 0x42, 0x4b, 0x69, 0xa1, 0x82, }, - { 0x12, 0x8c, 0x6d, 0x52, 0x08, 0xef, 0x74, 0xb2, 0xe6, 0xaa, 0xd3, 0xb0, - 0x26, 0xb0, 0xd9, 0x94, 0xb6, 0x11, 0x45, 0x0e, 0x36, 0x71, 0x14, 0x2d, - 0x41, 0x8c, 0x21, 0x53, 0x31, 0xe9, 0x68, }, - { 0xee, 0xea, 0x0d, 0x89, 0x47, 0x7e, 0x72, 0xd1, 0xd8, 0xce, 0x58, 0x4c, - 0x94, 0x1f, 0x0d, 0x51, 0x08, 0xa3, 0xb6, 0x3d, 0xe7, 0x82, 0x46, 0x92, - 0xd6, 0x98, 0x6b, 0x07, 0x10, 0x65, 0x52, 0x65, }, -}; - -static bool __init noinline_for_stack blake2s_digest_test(void) -{ - u8 key[BLAKE2S_KEY_SIZE]; - u8 buf[ARRAY_SIZE(blake2s_testvecs)]; - u8 hash[BLAKE2S_HASH_SIZE]; - struct blake2s_state state; - bool success =3D true; - int i, l; - - key[0] =3D key[1] =3D 1; - for (i =3D 2; i < sizeof(key); ++i) - key[i] =3D key[i - 2] + key[i - 1]; - - for (i =3D 0; i < sizeof(buf); ++i) - buf[i] =3D (u8)i; - - for (i =3D l =3D 0; i < ARRAY_SIZE(blake2s_testvecs); l =3D (l + 37) % ++= i) { - int outlen =3D 1 + i % BLAKE2S_HASH_SIZE; - int keylen =3D (13 * i) % (BLAKE2S_KEY_SIZE + 1); - - blake2s(hash, buf, key + BLAKE2S_KEY_SIZE - keylen, outlen, i, - keylen); - if (memcmp(hash, blake2s_testvecs[i], outlen)) { - pr_err("blake2s self-test %d: FAIL\n", i + 1); - success =3D false; - } - - if (!keylen) - blake2s_init(&state, outlen); - else - blake2s_init_key(&state, outlen, - key + BLAKE2S_KEY_SIZE - keylen, - keylen); - - blake2s_update(&state, buf, l); - blake2s_update(&state, buf + l, i - l); - blake2s_final(&state, hash); - if (memcmp(hash, blake2s_testvecs[i], outlen)) { - pr_err("blake2s init/update/final self-test %d: FAIL\n", - i + 1); - success =3D false; - } - } - - return success; -} - -static bool __init noinline_for_stack blake2s_random_test(void) -{ - struct blake2s_state state; - bool success =3D true; - int i, l; - - for (i =3D 0; i < 32; ++i) { - enum { TEST_ALIGNMENT =3D 16 }; - u8 blocks[BLAKE2S_BLOCK_SIZE * 2 + TEST_ALIGNMENT - 1] - __aligned(TEST_ALIGNMENT); - u8 *unaligned_block =3D blocks + BLAKE2S_BLOCK_SIZE; - struct blake2s_state state1, state2; - - get_random_bytes(blocks, sizeof(blocks)); - get_random_bytes(&state, sizeof(state)); - -#if defined(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC) && \ - defined(CONFIG_CRYPTO_ARCH_HAVE_LIB_BLAKE2S) - memcpy(&state1, &state, sizeof(state1)); - memcpy(&state2, &state, sizeof(state2)); - blake2s_compress(&state1, blocks, 2, BLAKE2S_BLOCK_SIZE); - blake2s_compress_generic(&state2, blocks, 2, BLAKE2S_BLOCK_SIZE); - if (memcmp(&state1, &state2, sizeof(state1))) { - pr_err("blake2s random compress self-test %d: FAIL\n", - i + 1); - success =3D false; - } -#endif - - memcpy(&state1, &state, sizeof(state1)); - blake2s_compress(&state1, blocks, 1, BLAKE2S_BLOCK_SIZE); - for (l =3D 1; l < TEST_ALIGNMENT; ++l) { - memcpy(unaligned_block + l, blocks, - BLAKE2S_BLOCK_SIZE); - memcpy(&state2, &state, sizeof(state2)); - blake2s_compress(&state2, unaligned_block + l, 1, - BLAKE2S_BLOCK_SIZE); - if (memcmp(&state1, &state2, sizeof(state1))) { - pr_err("blake2s random compress align %d self-test %d: FAIL\n", - l, i + 1); - success =3D false; - } - } - } - - return success; -} - -bool __init blake2s_selftest(void) -{ - bool success; - - success =3D blake2s_digest_test(); - success &=3D blake2s_random_test(); - - return success; -} diff --git a/lib/crypto/blake2s.c b/lib/crypto/blake2s.c index f6ec68c3dcdae..51f2dd7a38a4e 100644 --- a/lib/crypto/blake2s.c +++ b/lib/crypto/blake2s.c @@ -9,11 +9,10 @@ */ =20 #include #include #include -#include #include #include #include #include =20 @@ -57,16 +56,7 @@ void blake2s_final(struct blake2s_state *state, u8 *out) memcpy(out, state->h, state->outlen); memzero_explicit(state, sizeof(*state)); } EXPORT_SYMBOL(blake2s_final); =20 -static int __init blake2s_mod_init(void) -{ - if (IS_ENABLED(CONFIG_CRYPTO_SELFTESTS) && - WARN_ON(!blake2s_selftest())) - return -ENODEV; - return 0; -} - -module_init(blake2s_mod_init); MODULE_DESCRIPTION("BLAKE2s hash function"); MODULE_AUTHOR("Jason A. Donenfeld "); --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1996D29D28B; Wed, 27 Aug 2025 15:15:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307701; cv=none; b=lvw7u+A3ad3TUs82g1Tn/xWRe/rR5a9yug3HP4Ch2xTee3Y7KaWGayMjuOFAEzzKx10NdALlhDMPQwJKVC9+RdCyoDHAOhWHjbyaxCueybf1rd/aZ2BVaJQ3aeqsY3ZMBonE9saeoVxidTWhYLMLuWabUEAcJ5p8Ji++Hra8v+M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307701; c=relaxed/simple; bh=/bCBdkNC8TXSkbBvU6E1D99TNf2kUrJ7+Uwzrv5HTbc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XSjy79QynwhXbOWYDMBZHzGZsHJCpQJYqjw8i0etPpsc6Q6k1ZcS0pbHG7fQ30xnjGaDxgTBPDDy9ySF62XXcOlzWphBAeMAVkhmHiiXKIvNE/VnPX5kFD+dRlq2efQbbmx6+8OH0N05JjL3z6Ak2ZYfNuBPWtKuCRq99NrF9zk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=U0+Mrc0G; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="U0+Mrc0G" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2BEBC4CEFA; Wed, 27 Aug 2025 15:14:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307700; bh=/bCBdkNC8TXSkbBvU6E1D99TNf2kUrJ7+Uwzrv5HTbc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U0+Mrc0GvdK47J+B9pHg7oS6KkzCJc+rHz0gATPkd06pRx+qBTm+5j+6ezuV4DAn5 3IEq+i3Nwts/qBGRXqaYGRa4BfVL/XrkHthWHsFIMTSaljOZs2NJumga+k1pdNvan9 IsctHfkMml8XmEL1m7sfvwUxX6OjxmAjvJaw+GJNvl4f+4sKYgWKkh6+DvfjZv7LYe tRjt9fMhk54cROJd/kBpiHp0E0SS6uJmRUKUsDIgT1BLj7AfmfB46tW4PEEeRck9Ff R+2Qlv+J8cj/odJYvH9XboqSmusDiSu0piCQMVegVR0EzzCH3CJ8cNv7XLt9uoHyDH 6C3AclQJBLyIA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 09/12] lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code Date: Wed, 27 Aug 2025 08:11:28 -0700 Message-ID: <20250827151131.27733-10-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When support for a crypto algorithm is enabled, the arch-optimized implementation of that algorithm should be enabled too. We've learned this the hard way many times over the years: people regularly forget to enable the arch-optimized implementations of the crypto algorithms, resulting in significant performance being left on the table. Currently, BLAKE2s support is always enabled ('obj-y'), since random.c uses it. Therefore, the arch-optimized BLAKE2s code, which exists for ARM and x86_64, should be always enabled too. Let's do that. Note that the effect on kernel image size is very small and should not be a concern. On ARM, enabling CRYPTO_BLAKE2S_ARM actually *shrinks* the kernel size by about 1200 bytes, since the ARM-optimized blake2s_compress() completely replaces the generic blake2s_compress(). On x86_64, enabling CRYPTO_BLAKE2S_X86 increases the kernel size by about 1400 bytes, as the generic blake2s_compress() is still included as a fallback; however, for context, that is only about a quarter the size of the generic blake2s_compress(). The x86_64 optimized BLAKE2s code uses much less icache at runtime than the generic code. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- lib/crypto/arm/Kconfig | 2 +- lib/crypto/x86/Kconfig | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/crypto/arm/Kconfig b/lib/crypto/arm/Kconfig index 740341aa35d21..a5607ad079c4f 100644 --- a/lib/crypto/arm/Kconfig +++ b/lib/crypto/arm/Kconfig @@ -1,9 +1,9 @@ # SPDX-License-Identifier: GPL-2.0-only =20 config CRYPTO_BLAKE2S_ARM - bool "Hash functions: BLAKE2s" + def_bool y select CRYPTO_ARCH_HAVE_LIB_BLAKE2S help BLAKE2s cryptographic hash function (RFC 7693) =20 Architecture: arm diff --git a/lib/crypto/x86/Kconfig b/lib/crypto/x86/Kconfig index eb47da71aa6b6..ffa718321369f 100644 --- a/lib/crypto/x86/Kconfig +++ b/lib/crypto/x86/Kconfig @@ -1,9 +1,9 @@ # SPDX-License-Identifier: GPL-2.0-only =20 config CRYPTO_BLAKE2S_X86 - bool "Hash functions: BLAKE2s (SSSE3/AVX-512)" + def_bool y depends on 64BIT select CRYPTO_LIB_BLAKE2S_GENERIC select CRYPTO_ARCH_HAVE_LIB_BLAKE2S help BLAKE2s cryptographic hash function (RFC 7693) --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53A5C27B354; Wed, 27 Aug 2025 15:15:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307701; cv=none; b=W4CbTSMrXP3M7iZuYw25+vTAjagFlLzPB4f5AMzAjmYeEcQXJRAPdtgkSWzUkMez7bE3zyb0bXCkfMWNE8IIMXkhg1TAaJO9SRer8ApSxcqFxus8gPcKfQNRAfO3hmnLUdHImlq75nXf9nqSMtchomBU5U4KEKzNmXo3JcawrJI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307701; c=relaxed/simple; bh=x/NuAW84/cQ0t0p3ZfjB/C473ct6iM59qATu6l/hJrA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gl+cWq21+R5yBbOloJ5f92SQTbJVxnqpUzQ57S3/BlwE9VKMLVyzuSS1nIbiBtQRxgZ49qRqrxBfZgt7/iye40OJHb56XXTxnGEzLC5WS0LqfOVZTVjxQNACwLVLKOIK3Nj87Q56OWasviovrS0gZ/q4DR+/ZycKx0HHWlDpjmc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dg+da+KH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dg+da+KH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D55D6C4CEF0; Wed, 27 Aug 2025 15:15:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307701; bh=x/NuAW84/cQ0t0p3ZfjB/C473ct6iM59qATu6l/hJrA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dg+da+KHJqXKeYUE+nRoDEcrPOzL+VJhQ0dFzQJ2Ry8ITujl+oK7WFeRLyqc57b/G YIbwQjCptgz57cVQrciuVhv19wrndQBmUAS/pNgT5Vz4skqNMo3STV7T3QJxR/GjaK X54SecwHkyqiP3507WtRIPDs8OXukZhl/ztOHDIVSZpMsLyXCrcCWW7kKBrJ5kAOX3 0U/htlHBHgNnoe2MLCZ4DLptCsRbl1Ttzs3sNT4F90QWwjuIdtBentGz31PIuMZ4wt qgspCCosrv6DKtr27Vigg7ppRZCODR53Gb1WuKnIWAGUjr9ul0ECPgTUtRAxnxv4OL hT+GLbBczKjfw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 10/12] lib/crypto: blake2s: Move generic code into blake2s.c Date: Wed, 27 Aug 2025 08:11:29 -0700 Message-ID: <20250827151131.27733-11-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move blake2s_compress_generic() from blake2s-generic.c to blake2s.c. For now it's still guarded by CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC, but this prepares for changing it to a 'static __maybe_unused' function and just using the compiler to automatically decide its inclusion. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- lib/crypto/Makefile | 1 - lib/crypto/blake2s-generic.c | 111 ----------------------------------- lib/crypto/blake2s.c | 94 +++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+), 112 deletions(-) delete mode 100644 lib/crypto/blake2s-generic.c diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index eeddb993da033..be45dcfb8969b 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -30,11 +30,10 @@ libarc4-y :=3D arc4.o obj-$(CONFIG_CRYPTO_LIB_GF128MUL) +=3D gf128mul.o =20 # blake2s is used by the /dev/random driver which is always builtin obj-y +=3D libblake2s.o libblake2s-y :=3D blake2s.o -libblake2s-$(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC) +=3D blake2s-generic.o =20 ##########################################################################= ###### =20 # chacha20_block() is used by the /dev/random driver which is always built= in obj-y +=3D chacha-block-generic.o diff --git a/lib/crypto/blake2s-generic.c b/lib/crypto/blake2s-generic.c deleted file mode 100644 index 9828176a2efec..0000000000000 --- a/lib/crypto/blake2s-generic.c +++ /dev/null @@ -1,111 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 OR MIT -/* - * Copyright (C) 2015-2019 Jason A. Donenfeld . All Right= s Reserved. - * - * This is an implementation of the BLAKE2s hash and PRF functions. - * - * Information: https://blake2.net/ - * - */ - -#include -#include -#include -#include -#include -#include -#include -#include - -static const u8 blake2s_sigma[10][16] =3D { - { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, - { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 }, - { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 }, - { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 }, - { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 }, - { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 }, - { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 }, - { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 }, - { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 }, - { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13, 0 }, -}; - -static inline void blake2s_increment_counter(struct blake2s_state *state, - const u32 inc) -{ - state->t[0] +=3D inc; - state->t[1] +=3D (state->t[0] < inc); -} - -void blake2s_compress(struct blake2s_state *state, const u8 *block, - size_t nblocks, const u32 inc) - __weak __alias(blake2s_compress_generic); - -void blake2s_compress_generic(struct blake2s_state *state, const u8 *block, - size_t nblocks, const u32 inc) -{ - u32 m[16]; - u32 v[16]; - int i; - - WARN_ON(IS_ENABLED(DEBUG) && - (nblocks > 1 && inc !=3D BLAKE2S_BLOCK_SIZE)); - - while (nblocks > 0) { - blake2s_increment_counter(state, inc); - memcpy(m, block, BLAKE2S_BLOCK_SIZE); - le32_to_cpu_array(m, ARRAY_SIZE(m)); - memcpy(v, state->h, 32); - v[ 8] =3D BLAKE2S_IV0; - v[ 9] =3D BLAKE2S_IV1; - v[10] =3D BLAKE2S_IV2; - v[11] =3D BLAKE2S_IV3; - v[12] =3D BLAKE2S_IV4 ^ state->t[0]; - v[13] =3D BLAKE2S_IV5 ^ state->t[1]; - v[14] =3D BLAKE2S_IV6 ^ state->f[0]; - v[15] =3D BLAKE2S_IV7 ^ state->f[1]; - -#define G(r, i, a, b, c, d) do { \ - a +=3D b + m[blake2s_sigma[r][2 * i + 0]]; \ - d =3D ror32(d ^ a, 16); \ - c +=3D d; \ - b =3D ror32(b ^ c, 12); \ - a +=3D b + m[blake2s_sigma[r][2 * i + 1]]; \ - d =3D ror32(d ^ a, 8); \ - c +=3D d; \ - b =3D ror32(b ^ c, 7); \ -} while (0) - -#define ROUND(r) do { \ - G(r, 0, v[0], v[ 4], v[ 8], v[12]); \ - G(r, 1, v[1], v[ 5], v[ 9], v[13]); \ - G(r, 2, v[2], v[ 6], v[10], v[14]); \ - G(r, 3, v[3], v[ 7], v[11], v[15]); \ - G(r, 4, v[0], v[ 5], v[10], v[15]); \ - G(r, 5, v[1], v[ 6], v[11], v[12]); \ - G(r, 6, v[2], v[ 7], v[ 8], v[13]); \ - G(r, 7, v[3], v[ 4], v[ 9], v[14]); \ -} while (0) - ROUND(0); - ROUND(1); - ROUND(2); - ROUND(3); - ROUND(4); - ROUND(5); - ROUND(6); - ROUND(7); - ROUND(8); - ROUND(9); - -#undef G -#undef ROUND - - for (i =3D 0; i < 8; ++i) - state->h[i] ^=3D v[i] ^ v[i + 8]; - - block +=3D BLAKE2S_BLOCK_SIZE; - --nblocks; - } -} - -EXPORT_SYMBOL(blake2s_compress_generic); diff --git a/lib/crypto/blake2s.c b/lib/crypto/blake2s.c index 51f2dd7a38a4e..b5b75ade46588 100644 --- a/lib/crypto/blake2s.c +++ b/lib/crypto/blake2s.c @@ -14,10 +14,104 @@ #include #include #include #include =20 +#ifdef CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC +static const u8 blake2s_sigma[10][16] =3D { + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 }, + { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 }, + { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 }, + { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 }, + { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 }, + { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 }, + { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 }, + { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 }, + { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13, 0 }, +}; + +static inline void blake2s_increment_counter(struct blake2s_state *state, + const u32 inc) +{ + state->t[0] +=3D inc; + state->t[1] +=3D (state->t[0] < inc); +} + +void blake2s_compress(struct blake2s_state *state, const u8 *block, + size_t nblocks, const u32 inc) + __weak __alias(blake2s_compress_generic); + +void blake2s_compress_generic(struct blake2s_state *state, const u8 *block, + size_t nblocks, const u32 inc) +{ + u32 m[16]; + u32 v[16]; + int i; + + WARN_ON(IS_ENABLED(DEBUG) && + (nblocks > 1 && inc !=3D BLAKE2S_BLOCK_SIZE)); + + while (nblocks > 0) { + blake2s_increment_counter(state, inc); + memcpy(m, block, BLAKE2S_BLOCK_SIZE); + le32_to_cpu_array(m, ARRAY_SIZE(m)); + memcpy(v, state->h, 32); + v[ 8] =3D BLAKE2S_IV0; + v[ 9] =3D BLAKE2S_IV1; + v[10] =3D BLAKE2S_IV2; + v[11] =3D BLAKE2S_IV3; + v[12] =3D BLAKE2S_IV4 ^ state->t[0]; + v[13] =3D BLAKE2S_IV5 ^ state->t[1]; + v[14] =3D BLAKE2S_IV6 ^ state->f[0]; + v[15] =3D BLAKE2S_IV7 ^ state->f[1]; + +#define G(r, i, a, b, c, d) do { \ + a +=3D b + m[blake2s_sigma[r][2 * i + 0]]; \ + d =3D ror32(d ^ a, 16); \ + c +=3D d; \ + b =3D ror32(b ^ c, 12); \ + a +=3D b + m[blake2s_sigma[r][2 * i + 1]]; \ + d =3D ror32(d ^ a, 8); \ + c +=3D d; \ + b =3D ror32(b ^ c, 7); \ +} while (0) + +#define ROUND(r) do { \ + G(r, 0, v[0], v[ 4], v[ 8], v[12]); \ + G(r, 1, v[1], v[ 5], v[ 9], v[13]); \ + G(r, 2, v[2], v[ 6], v[10], v[14]); \ + G(r, 3, v[3], v[ 7], v[11], v[15]); \ + G(r, 4, v[0], v[ 5], v[10], v[15]); \ + G(r, 5, v[1], v[ 6], v[11], v[12]); \ + G(r, 6, v[2], v[ 7], v[ 8], v[13]); \ + G(r, 7, v[3], v[ 4], v[ 9], v[14]); \ +} while (0) + ROUND(0); + ROUND(1); + ROUND(2); + ROUND(3); + ROUND(4); + ROUND(5); + ROUND(6); + ROUND(7); + ROUND(8); + ROUND(9); + +#undef G +#undef ROUND + + for (i =3D 0; i < 8; ++i) + state->h[i] ^=3D v[i] ^ v[i + 8]; + + block +=3D BLAKE2S_BLOCK_SIZE; + --nblocks; + } +} +EXPORT_SYMBOL(blake2s_compress_generic); +#endif /* CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC */ + static inline void blake2s_set_lastblock(struct blake2s_state *state) { state->f[0] =3D -1; } =20 --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D85862C2366; Wed, 27 Aug 2025 15:15:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307701; cv=none; b=hk8gGx4ZeaR14Sx7iN1q8uDgEePVrewe/V2Uzhjc1Gg72jHLBk93tTAWKAVn6oJBjeA+YVmzimhd1yLr/f3eldOBNGVni33sxmkvzNRGPG40kS0zb2cJvig93Q+8QZ0ZETiyNJd+0xpQChc4W9YoGBNalYbAVVk+dqIzmW4PFfU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307701; c=relaxed/simple; bh=U49ipLriqZbrr3XNuno8hDtfowuvOtRhW3j9fDfIUdk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f2ZMoR5zitTwTjc5AkvdMhgdVtpTPlx8qHZf4kVZOdi3dG667GUQ/Lrv9e1XA8vsH5XBCxutFw2OHrQljgV3cuGC+AUWWjZXmpQ6cvLVBjYjnOqgOGXnU02U5VQG7R6+5Gro4mSEIQMdva9EM8PgIKwAd0gRh5HTD3CIgwqaTdE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hevibz4Q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hevibz4Q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 58EB5C4CEF7; Wed, 27 Aug 2025 15:15:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307701; bh=U49ipLriqZbrr3XNuno8hDtfowuvOtRhW3j9fDfIUdk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hevibz4Qr5uFxG1U9N17GWmUhPxkuwGxJD3bhrrdTyLyGpwBOhl1vPlU1o+jjJLwY i6S+8d5h28tF+FAEWjM/bsm2qhIC0OdgSbnVuZcnNUaM6vjCVzMVZH/Fm8ycSSmCCy qVZdvO+CCEjukiowRP/IO236eK+C2wNWac/vpZxuwp0tFBABbI2AjPmWJG9GOgQhOv +gStoxC+ULUH3J0WHCXHREFf6Pld8tGjJKgpFU+tPlU7sazQqjSYspT4HcdQOgCTtp hEg2xW41XvW9pVKJqPJgetBYR4VeTyPU55JJojjmimmbcJkr8hhcNdljsieMWHcaEA Wf2cmTg/CuxfA== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 11/12] lib/crypto: blake2s: Consolidate into single C translation unit Date: Wed, 27 Aug 2025 08:11:30 -0700 Message-ID: <20250827151131.27733-12-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" As was done with the other algorithms, reorganize the BLAKE2s code so that the generic implementation and the arch-specific "glue" code is consolidated into a single translation unit, so that the compiler will inline the functions and automatically decide whether to include the generic code in the resulting binary or not. Similarly, also consolidate the build rules into lib/crypto/{Makefile,Kconfig}. This removes the last uses of lib/crypto/{arm,x86}/{Makefile,Kconfig}, so remove those too. Don't keep the !KMSAN dependency. It was needed only for other algorithms such as ChaCha that initialize memory from assembly code. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- include/crypto/internal/blake2s.h | 19 ------------- lib/crypto/Kconfig | 29 ++++---------------- lib/crypto/Makefile | 13 +++++---- lib/crypto/arm/Kconfig | 14 ---------- lib/crypto/arm/Makefile | 4 --- lib/crypto/arm/blake2s-core.S | 5 +++- lib/crypto/arm/blake2s-glue.c | 7 ----- lib/crypto/arm/blake2s.h | 5 ++++ lib/crypto/blake2s.c | 29 +++++++++++++------- lib/crypto/x86/Kconfig | 13 --------- lib/crypto/x86/Makefile | 4 --- lib/crypto/x86/{blake2s-glue.c =3D> blake2s.h} | 16 ++++------- 12 files changed, 47 insertions(+), 111 deletions(-) delete mode 100644 include/crypto/internal/blake2s.h delete mode 100644 lib/crypto/arm/Kconfig delete mode 100644 lib/crypto/arm/Makefile delete mode 100644 lib/crypto/arm/blake2s-glue.c create mode 100644 lib/crypto/arm/blake2s.h delete mode 100644 lib/crypto/x86/Kconfig delete mode 100644 lib/crypto/x86/Makefile rename lib/crypto/x86/{blake2s-glue.c =3D> blake2s.h} (83%) diff --git a/include/crypto/internal/blake2s.h b/include/crypto/internal/bl= ake2s.h deleted file mode 100644 index 3b82572c94337..0000000000000 --- a/include/crypto/internal/blake2s.h +++ /dev/null @@ -1,19 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 OR MIT */ -/* - * Helper functions for BLAKE2s implementations. - * Keep this in sync with the corresponding BLAKE2b header. - */ - -#ifndef _CRYPTO_INTERNAL_BLAKE2S_H -#define _CRYPTO_INTERNAL_BLAKE2S_H - -#include -#include - -void blake2s_compress_generic(struct blake2s_state *state, const u8 *block, - size_t nblocks, const u32 inc); - -void blake2s_compress(struct blake2s_state *state, const u8 *block, - size_t nblocks, const u32 inc); - -#endif /* _CRYPTO_INTERNAL_BLAKE2S_H */ diff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig index c1db483bc2306..37d85e0c9b979 100644 --- a/lib/crypto/Kconfig +++ b/lib/crypto/Kconfig @@ -26,25 +26,17 @@ config CRYPTO_LIB_ARC4 tristate =20 config CRYPTO_LIB_GF128MUL tristate =20 -config CRYPTO_ARCH_HAVE_LIB_BLAKE2S - bool - help - Declares whether the architecture provides an arch-specific - accelerated implementation of the Blake2s library interface, - either builtin or as a module. +# BLAKE2s support is always built-in, so there's no CRYPTO_LIB_BLAKE2S opt= ion. =20 -config CRYPTO_LIB_BLAKE2S_GENERIC - def_bool !CRYPTO_ARCH_HAVE_LIB_BLAKE2S - help - This symbol can be depended upon by arch implementations of the - Blake2s library interface that require the generic code as a - fallback, e.g., for SIMD implementations. If no arch specific - implementation is enabled, this implementation serves the users - of CRYPTO_LIB_BLAKE2S. +config CRYPTO_LIB_BLAKE2S_ARCH + bool + depends on !UML + default y if ARM + default y if X86_64 =20 config CRYPTO_LIB_CHACHA tristate select CRYPTO_LIB_UTILS help @@ -206,15 +198,6 @@ config CRYPTO_LIB_SHA512_ARCH config CRYPTO_LIB_SM3 tristate =20 source "lib/crypto/tests/Kconfig" =20 -if !KMSAN # avoid false positives from assembly -if ARM -source "lib/crypto/arm/Kconfig" -endif -if X86 -source "lib/crypto/x86/Kconfig" -endif -endif - endmenu diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index be45dcfb8969b..2551843694b7e 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -27,13 +27,19 @@ libaesgcm-y :=3D aesgcm.o obj-$(CONFIG_CRYPTO_LIB_ARC4) +=3D libarc4.o libarc4-y :=3D arc4.o =20 obj-$(CONFIG_CRYPTO_LIB_GF128MUL) +=3D gf128mul.o =20 +##########################################################################= ###### + # blake2s is used by the /dev/random driver which is always builtin -obj-y +=3D libblake2s.o -libblake2s-y :=3D blake2s.o +obj-y +=3D blake2s.o +ifeq ($(CONFIG_CRYPTO_LIB_BLAKE2S_ARCH),y) +CFLAGS_blake2s.o +=3D -I$(src)/$(SRCARCH) +obj-$(CONFIG_ARM) +=3D arm/blake2s-core.o +obj-$(CONFIG_X86) +=3D x86/blake2s-core.o +endif =20 ##########################################################################= ###### =20 # chacha20_block() is used by the /dev/random driver which is always built= in obj-y +=3D chacha-block-generic.o @@ -252,11 +258,8 @@ obj-$(CONFIG_MPILIB) +=3D mpi/ obj-$(CONFIG_CRYPTO_SELFTESTS_FULL) +=3D simd.o =20 obj-$(CONFIG_CRYPTO_LIB_SM3) +=3D libsm3.o libsm3-y :=3D sm3.o =20 -obj-$(CONFIG_ARM) +=3D arm/ -obj-$(CONFIG_X86) +=3D x86/ - # clean-files must be defined unconditionally clean-files +=3D arm/sha256-core.S arm/sha512-core.S clean-files +=3D arm64/sha256-core.S arm64/sha512-core.S diff --git a/lib/crypto/arm/Kconfig b/lib/crypto/arm/Kconfig deleted file mode 100644 index a5607ad079c4f..0000000000000 --- a/lib/crypto/arm/Kconfig +++ /dev/null @@ -1,14 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -config CRYPTO_BLAKE2S_ARM - def_bool y - select CRYPTO_ARCH_HAVE_LIB_BLAKE2S - help - BLAKE2s cryptographic hash function (RFC 7693) - - Architecture: arm - - This is faster than the generic implementations of BLAKE2s and - BLAKE2b, but slower than the NEON implementation of BLAKE2b. - There is no NEON implementation of BLAKE2s, since NEON doesn't - really help with it. diff --git a/lib/crypto/arm/Makefile b/lib/crypto/arm/Makefile deleted file mode 100644 index 0574b0e9739e3..0000000000000 --- a/lib/crypto/arm/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -obj-$(CONFIG_CRYPTO_BLAKE2S_ARM) +=3D libblake2s-arm.o -libblake2s-arm-y :=3D blake2s-core.o blake2s-glue.o diff --git a/lib/crypto/arm/blake2s-core.S b/lib/crypto/arm/blake2s-core.S index df40e46601f10..293f44fa8f316 100644 --- a/lib/crypto/arm/blake2s-core.S +++ b/lib/crypto/arm/blake2s-core.S @@ -1,8 +1,11 @@ /* SPDX-License-Identifier: GPL-2.0-or-later */ /* - * BLAKE2s digest algorithm, ARM scalar implementation + * BLAKE2s digest algorithm, ARM scalar implementation. This is faster + * than the generic implementations of BLAKE2s and BLAKE2b, but slower + * than the NEON implementation of BLAKE2b. There is no NEON + * implementation of BLAKE2s, since NEON doesn't really help with it. * * Copyright 2020 Google LLC * * Author: Eric Biggers */ diff --git a/lib/crypto/arm/blake2s-glue.c b/lib/crypto/arm/blake2s-glue.c deleted file mode 100644 index 0238a70d9581e..0000000000000 --- a/lib/crypto/arm/blake2s-glue.c +++ /dev/null @@ -1,7 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later - -#include -#include - -/* defined in blake2s-core.S */ -EXPORT_SYMBOL(blake2s_compress); diff --git a/lib/crypto/arm/blake2s.h b/lib/crypto/arm/blake2s.h new file mode 100644 index 0000000000000..aa7a97139ea74 --- /dev/null +++ b/lib/crypto/arm/blake2s.h @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +/* defined in blake2s-core.S */ +void blake2s_compress(struct blake2s_state *state, const u8 *block, + size_t nblocks, u32 inc); diff --git a/lib/crypto/blake2s.c b/lib/crypto/blake2s.c index b5b75ade46588..5638ed9d882d8 100644 --- a/lib/crypto/blake2s.c +++ b/lib/crypto/blake2s.c @@ -6,19 +6,18 @@ * * Information: https://blake2.net/ * */ =20 -#include +#include #include #include #include #include #include #include =20 -#ifdef CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC static const u8 blake2s_sigma[10][16] =3D { { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 }, { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 }, { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 }, { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 }, @@ -35,16 +34,13 @@ static inline void blake2s_increment_counter(struct bla= ke2s_state *state, { state->t[0] +=3D inc; state->t[1] +=3D (state->t[0] < inc); } =20 -void blake2s_compress(struct blake2s_state *state, const u8 *block, - size_t nblocks, const u32 inc) - __weak __alias(blake2s_compress_generic); - -void blake2s_compress_generic(struct blake2s_state *state, const u8 *block, - size_t nblocks, const u32 inc) +static void __maybe_unused +blake2s_compress_generic(struct blake2s_state *state, const u8 *block, + size_t nblocks, const u32 inc) { u32 m[16]; u32 v[16]; int i; =20 @@ -105,12 +101,16 @@ void blake2s_compress_generic(struct blake2s_state *s= tate, const u8 *block, =20 block +=3D BLAKE2S_BLOCK_SIZE; --nblocks; } } -EXPORT_SYMBOL(blake2s_compress_generic); -#endif /* CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC */ + +#ifdef CONFIG_CRYPTO_LIB_BLAKE2S_ARCH +#include "blake2s.h" /* $(SRCARCH)/blake2s.h */ +#else +#define blake2s_compress blake2s_compress_generic +#endif =20 static inline void blake2s_set_lastblock(struct blake2s_state *state) { state->f[0] =3D -1; } @@ -150,7 +150,16 @@ void blake2s_final(struct blake2s_state *state, u8 *ou= t) memcpy(out, state->h, state->outlen); memzero_explicit(state, sizeof(*state)); } EXPORT_SYMBOL(blake2s_final); =20 +#ifdef blake2s_mod_init_arch +static int __init blake2s_mod_init(void) +{ + blake2s_mod_init_arch(); + return 0; +} +subsys_initcall(blake2s_mod_init); +#endif + MODULE_DESCRIPTION("BLAKE2s hash function"); MODULE_AUTHOR("Jason A. Donenfeld "); diff --git a/lib/crypto/x86/Kconfig b/lib/crypto/x86/Kconfig deleted file mode 100644 index ffa718321369f..0000000000000 --- a/lib/crypto/x86/Kconfig +++ /dev/null @@ -1,13 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -config CRYPTO_BLAKE2S_X86 - def_bool y - depends on 64BIT - select CRYPTO_LIB_BLAKE2S_GENERIC - select CRYPTO_ARCH_HAVE_LIB_BLAKE2S - help - BLAKE2s cryptographic hash function (RFC 7693) - - Architecture: x86_64 using: - - SSSE3 (Supplemental SSE3) - - AVX-512 (Advanced Vector Extensions-512) diff --git a/lib/crypto/x86/Makefile b/lib/crypto/x86/Makefile deleted file mode 100644 index 4454556b243e7..0000000000000 --- a/lib/crypto/x86/Makefile +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only - -obj-$(CONFIG_CRYPTO_BLAKE2S_X86) +=3D libblake2s-x86_64.o -libblake2s-x86_64-y :=3D blake2s-core.o blake2s-glue.o diff --git a/lib/crypto/x86/blake2s-glue.c b/lib/crypto/x86/blake2s.h similarity index 83% rename from lib/crypto/x86/blake2s-glue.c rename to lib/crypto/x86/blake2s.h index adc296cd17c93..b6d30d2fa045e 100644 --- a/lib/crypto/x86/blake2s-glue.c +++ b/lib/crypto/x86/blake2s.h @@ -1,16 +1,14 @@ -// SPDX-License-Identifier: GPL-2.0 OR MIT +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ /* * Copyright (C) 2015-2019 Jason A. Donenfeld . All Right= s Reserved. */ =20 #include #include #include #include -#include -#include #include #include #include =20 asmlinkage void blake2s_compress_ssse3(struct blake2s_state *state, @@ -21,12 +19,12 @@ asmlinkage void blake2s_compress_avx512(struct blake2s_= state *state, const u32 inc); =20 static __ro_after_init DEFINE_STATIC_KEY_FALSE(blake2s_use_ssse3); static __ro_after_init DEFINE_STATIC_KEY_FALSE(blake2s_use_avx512); =20 -void blake2s_compress(struct blake2s_state *state, const u8 *block, - size_t nblocks, const u32 inc) +static void blake2s_compress(struct blake2s_state *state, const u8 *block, + size_t nblocks, const u32 inc) { /* SIMD disables preemption, so relax after processing each page. */ BUILD_BUG_ON(SZ_4K / BLAKE2S_BLOCK_SIZE < 8); =20 if (!static_branch_likely(&blake2s_use_ssse3) || !may_use_simd()) { @@ -47,13 +45,13 @@ void blake2s_compress(struct blake2s_state *state, cons= t u8 *block, =20 nblocks -=3D blocks; block +=3D blocks * BLAKE2S_BLOCK_SIZE; } while (nblocks); } -EXPORT_SYMBOL(blake2s_compress); =20 -static int __init blake2s_mod_init(void) +#define blake2s_mod_init_arch blake2s_mod_init_arch +static void blake2s_mod_init_arch(void) { if (boot_cpu_has(X86_FEATURE_SSSE3)) static_branch_enable(&blake2s_use_ssse3); =20 if (boot_cpu_has(X86_FEATURE_AVX) && @@ -61,10 +59,6 @@ static int __init blake2s_mod_init(void) boot_cpu_has(X86_FEATURE_AVX512F) && boot_cpu_has(X86_FEATURE_AVX512VL) && cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM | XFEATURE_MASK_AVX512, NULL)) static_branch_enable(&blake2s_use_avx512); - - return 0; } - -subsys_initcall(blake2s_mod_init); --=20 2.50.1 From nobody Fri Oct 3 16:47:21 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77E7A2D2384; Wed, 27 Aug 2025 15:15:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307702; cv=none; b=Cy0KX1CE7+W9n/AIq+mzcbpQpyG8WYOTMXj4cMDhud9HdVuNkoDy99LAuzG6WAcK972CG1GSf4u03GZ7GqpKJ6UkiUyKsfilWiqm7GYuX418Cm87wBsfQcZG4Bv5FUMd7vHe1+Fd/WICqHJEl2Xe2Q/gCRA0/euwiXKUMiIh0cc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756307702; c=relaxed/simple; bh=1qgaGsrCssucR1umbiIEhm9E7Chqg0rg24h/vOyJeL0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fXFFBs8T0XmgCbehraGG+lelg1Pf2xA3EoQ4Xj498BtbJJGo9aI898W+vx/lS5eNm18OfZDoY1KDoAkjlRs+DN+qU+36Vv0w4JpD46Mq1o3/MVB/SPH6Ddve6ofQUVndgv8Fddp2d6+Nf2nmx5+i0VCO60Lupq2ZtoBGl54WbrE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WnQf9eW6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WnQf9eW6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DB062C4CEF0; Wed, 27 Aug 2025 15:15:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1756307702; bh=1qgaGsrCssucR1umbiIEhm9E7Chqg0rg24h/vOyJeL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WnQf9eW6/q0WqMUAq3qefEoD22TolFa6J4vbJCvrOsqe5vfeBN4rgBTogvzaUu9Xc K5pT08f0fM3QtdCxYh2prk75RcbPjcm+7/NnaY60tSl4nKAWDtyXGpJDbORZRY2O9L CbpISfeYabjhtHkP0VRl5KsQi/y9YE3BdkOKMKmNMZVlN04AkpZC2UrNjlUCalIsgK uywXCeDZ1B+ePxD1x3f1yebCHUenKQyfGpbw/Zr7UePDGhCg2ZdlyozJmm4VoZ1BkF 52eqCaFQqajyEg8DtyJD3Bz1mkBtb2anViaa2pXPxG4WXzh5aH0sTXeQfl11/FvVkI Bs/air38RLOTw== From: Eric Biggers To: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , "Jason A . Donenfeld" , x86@kernel.org, linux-arm-kernel@lists.infradead.org, Eric Biggers Subject: [PATCH 12/12] lib/crypto: tests: Add KUnit tests for BLAKE2s Date: Wed, 27 Aug 2025 08:11:31 -0700 Message-ID: <20250827151131.27733-13-ebiggers@kernel.org> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20250827151131.27733-1-ebiggers@kernel.org> References: <20250827151131.27733-1-ebiggers@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a KUnit test suite for BLAKE2s. Most of the core test logic is in the previously-added hash-test-template.h. This commit just adds the actual KUnit suite, commits the generated test vectors to the tree so that gen-hash-testvecs.py won't have to be run at build time, and adds a few BLAKE2s-specific test cases. This is the replacement for blake2s-selftest, which an earlier commit removed. Improvements over blake2s-selftest include integration with KUnit, more comprehensive test cases, and support for benchmarking. Signed-off-by: Eric Biggers Reviewed-by: Ard Biesheuvel --- lib/crypto/tests/Kconfig | 10 ++ lib/crypto/tests/Makefile | 1 + lib/crypto/tests/blake2s-testvecs.h | 238 ++++++++++++++++++++++++++++ lib/crypto/tests/blake2s_kunit.c | 134 ++++++++++++++++ scripts/crypto/gen-hash-testvecs.py | 27 +++- 5 files changed, 407 insertions(+), 3 deletions(-) create mode 100644 lib/crypto/tests/blake2s-testvecs.h create mode 100644 lib/crypto/tests/blake2s_kunit.c diff --git a/lib/crypto/tests/Kconfig b/lib/crypto/tests/Kconfig index c21d53fd4b0ce..fd341aa12f157 100644 --- a/lib/crypto/tests/Kconfig +++ b/lib/crypto/tests/Kconfig @@ -1,7 +1,17 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 +config CRYPTO_LIB_BLAKE2S_KUNIT_TEST + tristate "KUnit tests for BLAKE2s" if !KUNIT_ALL_TESTS + depends on KUNIT + default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS + select CRYPTO_LIB_BENCHMARK_VISIBLE + # No need to select CRYPTO_LIB_BLAKE2S here, as that option doesn't + # exist; the BLAKE2s code is always built-in for the /dev/random driver. + help + KUnit tests for the BLAKE2s cryptographic hash function. + config CRYPTO_LIB_MD5_KUNIT_TEST tristate "KUnit tests for MD5" if !KUNIT_ALL_TESTS depends on KUNIT default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS select CRYPTO_LIB_BENCHMARK_VISIBLE diff --git a/lib/crypto/tests/Makefile b/lib/crypto/tests/Makefile index f6f82c6f9cb5d..be7de929af2cc 100644 --- a/lib/crypto/tests/Makefile +++ b/lib/crypto/tests/Makefile @@ -1,7 +1,8 @@ # SPDX-License-Identifier: GPL-2.0-or-later =20 +obj-$(CONFIG_CRYPTO_LIB_BLAKE2S_KUNIT_TEST) +=3D blake2s_kunit.o obj-$(CONFIG_CRYPTO_LIB_MD5_KUNIT_TEST) +=3D md5_kunit.o obj-$(CONFIG_CRYPTO_LIB_POLY1305_KUNIT_TEST) +=3D poly1305_kunit.o obj-$(CONFIG_CRYPTO_LIB_SHA1_KUNIT_TEST) +=3D sha1_kunit.o obj-$(CONFIG_CRYPTO_LIB_SHA256_KUNIT_TEST) +=3D sha224_kunit.o sha256_kuni= t.o obj-$(CONFIG_CRYPTO_LIB_SHA512_KUNIT_TEST) +=3D sha384_kunit.o sha512_kuni= t.o diff --git a/lib/crypto/tests/blake2s-testvecs.h b/lib/crypto/tests/blake2s= -testvecs.h new file mode 100644 index 0000000000000..6f978b79a59b0 --- /dev/null +++ b/lib/crypto/tests/blake2s-testvecs.h @@ -0,0 +1,238 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* This file was generated by: ./scripts/crypto/gen-hash-testvecs.py blake= 2s */ + +static const struct { + size_t data_len; + u8 digest[BLAKE2S_HASH_SIZE]; +} hash_testvecs[] =3D { + { + .data_len =3D 0, + .digest =3D { + 0x69, 0x21, 0x7a, 0x30, 0x79, 0x90, 0x80, 0x94, + 0xe1, 0x11, 0x21, 0xd0, 0x42, 0x35, 0x4a, 0x7c, + 0x1f, 0x55, 0xb6, 0x48, 0x2c, 0xa1, 0xa5, 0x1e, + 0x1b, 0x25, 0x0d, 0xfd, 0x1e, 0xd0, 0xee, 0xf9, + }, + }, + { + .data_len =3D 1, + .digest =3D { + 0x7c, 0xab, 0x53, 0xe2, 0x48, 0x87, 0xdf, 0x64, + 0x98, 0x6a, 0xc1, 0x7e, 0xf0, 0x01, 0x4d, 0xc9, + 0x07, 0x4f, 0xb8, 0x2f, 0x46, 0xd7, 0xee, 0xa9, + 0xad, 0xe5, 0xf8, 0x21, 0xac, 0xfe, 0x17, 0x58, + }, + }, + { + .data_len =3D 2, + .digest =3D { + 0x5e, 0x63, 0x2c, 0xd0, 0xf8, 0x7b, 0xf5, 0xae, + 0x61, 0x97, 0x94, 0x57, 0xc8, 0x76, 0x22, 0xd9, + 0x8b, 0x04, 0x5e, 0xf1, 0x5d, 0xd0, 0xfc, 0xd9, + 0x0c, 0x19, 0x2e, 0xe2, 0xc5, 0xd9, 0x73, 0x51, + }, + }, + { + .data_len =3D 3, + .digest =3D { + 0x33, 0x65, 0xa6, 0x37, 0xbf, 0xf8, 0x4f, 0x15, + 0x4c, 0xac, 0x9e, 0xa4, 0x3b, 0x02, 0x07, 0x0c, + 0x80, 0x86, 0x0d, 0x6c, 0xe4, 0xaf, 0x1c, 0xbc, + 0x0b, 0x9c, 0x0a, 0x98, 0xc2, 0x99, 0x71, 0xcd, + }, + }, + { + .data_len =3D 16, + .digest =3D { + 0x59, 0xd2, 0x10, 0xd3, 0x75, 0xac, 0x48, 0x32, + 0xb1, 0xea, 0xee, 0xcf, 0x0a, 0xd2, 0x8b, 0x15, + 0x5d, 0x72, 0x71, 0x4c, 0xa7, 0x29, 0xb0, 0x7a, + 0x44, 0x48, 0x8a, 0x54, 0x54, 0x54, 0x41, 0xf5, + }, + }, + { + .data_len =3D 32, + .digest =3D { + 0xdc, 0xfc, 0x46, 0x81, 0xc6, 0x1b, 0x2b, 0x47, + 0x8b, 0xed, 0xe0, 0x73, 0x34, 0x38, 0x53, 0x92, + 0x97, 0x2f, 0xfb, 0x51, 0xab, 0x4f, 0x2d, 0x9d, + 0x69, 0x04, 0xa9, 0x5d, 0x33, 0xef, 0xcb, 0x1c, + }, + }, + { + .data_len =3D 48, + .digest =3D { + 0xd6, 0x2a, 0x7f, 0x96, 0x04, 0x4d, 0x16, 0xc8, + 0x49, 0xe0, 0x37, 0x33, 0xe3, 0x7b, 0x34, 0x56, + 0x99, 0xc5, 0x78, 0x57, 0x06, 0x02, 0xb4, 0xea, + 0x80, 0xc4, 0xf8, 0x8f, 0x8d, 0x2b, 0xe4, 0x05, + }, + }, + { + .data_len =3D 49, + .digest =3D { + 0x8b, 0x58, 0x62, 0xb5, 0x85, 0xf6, 0x83, 0x36, + 0xf5, 0x34, 0xb8, 0xd4, 0xbc, 0x5c, 0x8b, 0x38, + 0xfd, 0x15, 0xcd, 0x44, 0x83, 0x25, 0x71, 0xe1, + 0xd5, 0xe8, 0xa1, 0xa4, 0x36, 0x98, 0x7e, 0x68, + }, + }, + { + .data_len =3D 63, + .digest =3D { + 0x7e, 0xeb, 0x06, 0x87, 0xdf, 0x1a, 0xdc, 0xe5, + 0xfb, 0x64, 0xd4, 0xd1, 0x5d, 0x9e, 0x75, 0xc0, + 0xb9, 0xad, 0x55, 0x6c, 0xe6, 0xba, 0x4d, 0x98, + 0x2f, 0xbf, 0x72, 0xad, 0x61, 0x37, 0xf6, 0x11, + }, + }, + { + .data_len =3D 64, + .digest =3D { + 0x72, 0xdb, 0x43, 0x16, 0x57, 0x8e, 0x3a, 0x96, + 0xf3, 0x98, 0x19, 0x24, 0x17, 0x3b, 0xe8, 0xad, + 0xa1, 0x9b, 0xa4, 0x1b, 0x74, 0x85, 0x2e, 0x24, + 0x70, 0xea, 0x31, 0x5a, 0x1c, 0xbe, 0x43, 0xb5, + }, + }, + { + .data_len =3D 65, + .digest =3D { + 0x32, 0x48, 0xb0, 0xf0, 0x3f, 0xbb, 0xd2, 0xa3, + 0xfd, 0xf6, 0x28, 0x4a, 0x2a, 0xc5, 0xbe, 0x4b, + 0x73, 0x50, 0x63, 0xd6, 0x16, 0x00, 0xef, 0xed, + 0xfe, 0x97, 0x41, 0x29, 0xb2, 0x84, 0xc4, 0xa3, + }, + }, + { + .data_len =3D 127, + .digest =3D { + 0x17, 0xda, 0x6b, 0x96, 0x6a, 0xa6, 0xa4, 0xa6, + 0xa6, 0xf3, 0x9d, 0x18, 0x19, 0x8d, 0x98, 0x7c, + 0x66, 0x38, 0xe8, 0x99, 0xe7, 0x0a, 0x50, 0x92, + 0xaf, 0x11, 0x80, 0x05, 0x66, 0xed, 0xab, 0x74, + }, + }, + { + .data_len =3D 128, + .digest =3D { + 0x13, 0xd5, 0x8b, 0x22, 0xae, 0x90, 0x7b, 0x67, + 0x87, 0x4e, 0x3c, 0x35, 0x4e, 0x01, 0xf0, 0xb1, + 0xd3, 0xd1, 0x67, 0xbb, 0x43, 0xdb, 0x7c, 0x75, + 0xa4, 0xc7, 0x64, 0x83, 0x1e, 0x9b, 0x98, 0xad, + }, + }, + { + .data_len =3D 129, + .digest =3D { + 0x6f, 0xe0, 0x5d, 0x9d, 0xd5, 0x78, 0x29, 0xfb, + 0xd0, 0x77, 0xd1, 0x8a, 0xf0, 0x80, 0xcb, 0x81, + 0x71, 0x9e, 0x4d, 0x49, 0xde, 0x74, 0x2a, 0x37, + 0xc0, 0xd5, 0xf0, 0xfa, 0x50, 0xe6, 0x23, 0xfe, + }, + }, + { + .data_len =3D 256, + .digest =3D { + 0x89, 0xac, 0xf6, 0xe7, 0x5e, 0xba, 0x53, 0xf4, + 0x92, 0x32, 0xd5, 0x64, 0xfb, 0xc4, 0x08, 0xac, + 0x2c, 0x19, 0x6e, 0x63, 0x13, 0x75, 0xd0, 0x60, + 0x54, 0x35, 0x82, 0xc4, 0x6d, 0x03, 0x1a, 0x05, + }, + }, + { + .data_len =3D 511, + .digest =3D { + 0x1c, 0xaf, 0x94, 0x7d, 0x9c, 0xce, 0x57, 0x64, + 0xf8, 0xa8, 0x25, 0x45, 0x32, 0x86, 0x2b, 0x04, + 0xb3, 0x2e, 0x67, 0xca, 0x73, 0x04, 0x2f, 0xab, + 0xcc, 0xda, 0x9e, 0x42, 0xa1, 0xaf, 0x83, 0x5a, + }, + }, + { + .data_len =3D 513, + .digest =3D { + 0x21, 0xdf, 0xdc, 0x29, 0xd9, 0xfc, 0x7b, 0xe7, + 0x3a, 0xc4, 0xe1, 0x61, 0xc5, 0xb5, 0xe1, 0xee, + 0x7a, 0x9d, 0x0c, 0x66, 0x36, 0x63, 0xe4, 0x12, + 0x62, 0xe2, 0xf5, 0x68, 0x72, 0xfc, 0x1e, 0x18, + }, + }, + { + .data_len =3D 1000, + .digest =3D { + 0x6e, 0xc7, 0x2e, 0xac, 0xd0, 0xbb, 0x22, 0xe0, + 0xc2, 0x40, 0xb2, 0xfe, 0x8c, 0xaf, 0x9e, 0xcf, + 0x32, 0x06, 0xc6, 0x45, 0x29, 0xbd, 0xe0, 0x7f, + 0x53, 0x32, 0xc3, 0x2b, 0x2f, 0x68, 0x12, 0xcd, + }, + }, + { + .data_len =3D 3333, + .digest =3D { + 0x76, 0xba, 0x52, 0xb5, 0x09, 0xf5, 0x19, 0x09, + 0x70, 0x1c, 0x09, 0x28, 0xb4, 0xaa, 0x98, 0x6a, + 0x79, 0xe7, 0x5e, 0xcd, 0xe8, 0xa4, 0x73, 0x69, + 0x1f, 0xf8, 0x05, 0x0a, 0xb4, 0xfe, 0xf9, 0x63, + }, + }, + { + .data_len =3D 4096, + .digest =3D { + 0xf7, 0xad, 0xf9, 0xc8, 0x0e, 0x04, 0x2f, 0xdf, + 0xbe, 0x39, 0x79, 0x07, 0x0d, 0xd8, 0x1b, 0x06, + 0x42, 0x3a, 0x43, 0x93, 0xf6, 0x7c, 0xc4, 0xe5, + 0xc2, 0xd5, 0xd0, 0xa6, 0x35, 0x6c, 0xbd, 0x17, + }, + }, + { + .data_len =3D 4128, + .digest =3D { + 0x38, 0xd7, 0xab, 0x7e, 0x08, 0xdc, 0x1e, 0xab, + 0x55, 0xbb, 0x3b, 0x7b, 0x6a, 0x17, 0xcc, 0x79, + 0xa7, 0x02, 0x62, 0x66, 0x9b, 0xca, 0xee, 0xc0, + 0x3d, 0x75, 0x34, 0x2e, 0x55, 0x82, 0x26, 0x3c, + }, + }, + { + .data_len =3D 4160, + .digest =3D { + 0xf7, 0xeb, 0x2f, 0x24, 0x98, 0x54, 0x04, 0x5a, + 0x19, 0xe4, 0x12, 0x9d, 0x97, 0xbc, 0x87, 0xa5, + 0x0b, 0x85, 0x29, 0xa1, 0x36, 0x89, 0xc9, 0xba, + 0xa0, 0xe0, 0xac, 0x99, 0x7d, 0xa4, 0x51, 0x9f, + }, + }, + { + .data_len =3D 4224, + .digest =3D { + 0x8f, 0xe8, 0xa7, 0x79, 0x02, 0xbb, 0x4a, 0x56, + 0x66, 0x91, 0xef, 0x22, 0xd1, 0x09, 0x26, 0x6c, + 0xa9, 0x13, 0xd7, 0x44, 0xc7, 0x19, 0x9c, 0x0b, + 0xfb, 0x4f, 0xca, 0x72, 0x8f, 0x34, 0xf7, 0x82, + }, + }, + { + .data_len =3D 16384, + .digest =3D { + 0xaa, 0x21, 0xbb, 0x25, 0x4b, 0x66, 0x6e, 0x29, + 0x71, 0xc1, 0x44, 0x67, 0x19, 0xed, 0xe6, 0xe6, + 0x61, 0x13, 0xf4, 0xb7, 0x02, 0x94, 0x81, 0x0f, + 0xa7, 0x4d, 0xbb, 0x2c, 0xb8, 0xeb, 0x41, 0x0e, + }, + }, +}; + +static const u8 hash_testvec_consolidated[BLAKE2S_HASH_SIZE] =3D { + 0x84, 0x21, 0xbb, 0x73, 0x64, 0x47, 0x45, 0xe0, + 0xc1, 0x83, 0x78, 0xf1, 0xea, 0xe5, 0xfd, 0xdb, + 0x01, 0xda, 0xb7, 0x86, 0x70, 0x3b, 0x83, 0xb3, + 0xbc, 0xd9, 0xfd, 0x96, 0xbd, 0x50, 0x06, 0x67, +}; + +static const u8 blake2s_keyed_testvec_consolidated[BLAKE2S_HASH_SIZE] =3D { + 0xa6, 0xad, 0xcd, 0xb8, 0xd9, 0xdd, 0xc7, 0x70, + 0x07, 0x09, 0x7f, 0x9f, 0x41, 0xa9, 0x70, 0xa4, + 0x1c, 0xca, 0x61, 0xbb, 0x58, 0xb5, 0xb2, 0x1d, + 0xd1, 0x71, 0x16, 0xb0, 0x49, 0x4f, 0x9e, 0x1b, +}; diff --git a/lib/crypto/tests/blake2s_kunit.c b/lib/crypto/tests/blake2s_ku= nit.c new file mode 100644 index 0000000000000..057c40132246f --- /dev/null +++ b/lib/crypto/tests/blake2s_kunit.c @@ -0,0 +1,134 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright 2025 Google LLC + */ +#include +#include "blake2s-testvecs.h" + +/* + * The following are compatibility functions that present BLAKE2s as an un= keyed + * hash function that produces hashes of fixed length BLAKE2S_HASH_SIZE, s= o that + * hash-test-template.h can be reused to test it. + */ + +static void blake2s_default(const u8 *data, size_t len, + u8 out[BLAKE2S_HASH_SIZE]) +{ + blake2s(out, data, NULL, BLAKE2S_HASH_SIZE, len, 0); +} + +static void blake2s_init_default(struct blake2s_state *state) +{ + blake2s_init(state, BLAKE2S_HASH_SIZE); +} + +/* + * Generate the HASH_KUNIT_CASES using hash-test-template.h. These test B= LAKE2s + * with a key length of 0 and a hash length of BLAKE2S_HASH_SIZE. + */ +#define HASH blake2s_default +#define HASH_CTX blake2s_state +#define HASH_SIZE BLAKE2S_HASH_SIZE +#define HASH_INIT blake2s_init_default +#define HASH_UPDATE blake2s_update +#define HASH_FINAL blake2s_final +#include "hash-test-template.h" + +/* + * BLAKE2s specific test case which tests all possible combinations of key + * length and hash length. + */ +static void test_blake2s_all_key_and_hash_lens(struct kunit *test) +{ + const size_t data_len =3D 100; + u8 *data =3D &test_buf[0]; + u8 *key =3D data + data_len; + u8 *hash =3D key + BLAKE2S_KEY_SIZE; + struct blake2s_state main_state; + u8 main_hash[BLAKE2S_HASH_SIZE]; + + rand_bytes_seeded_from_len(data, data_len); + blake2s_init(&main_state, BLAKE2S_HASH_SIZE); + for (int key_len =3D 0; key_len <=3D BLAKE2S_KEY_SIZE; key_len++) { + rand_bytes_seeded_from_len(key, key_len); + for (int out_len =3D 1; out_len <=3D BLAKE2S_HASH_SIZE; out_len++) { + blake2s(hash, data, key, out_len, data_len, key_len); + blake2s_update(&main_state, hash, out_len); + } + } + blake2s_final(&main_state, main_hash); + KUNIT_ASSERT_MEMEQ(test, main_hash, blake2s_keyed_testvec_consolidated, + BLAKE2S_HASH_SIZE); +} + +/* + * BLAKE2s specific test case which tests using a guarded buffer for all a= llowed + * key lengths. Also tests both blake2s() and blake2s_init_key(). + */ +static void test_blake2s_with_guarded_key_buf(struct kunit *test) +{ + const size_t data_len =3D 100; + + rand_bytes(test_buf, data_len); + for (int key_len =3D 0; key_len <=3D BLAKE2S_KEY_SIZE; key_len++) { + u8 key[BLAKE2S_KEY_SIZE]; + u8 *guarded_key =3D &test_buf[TEST_BUF_LEN - key_len]; + u8 hash1[BLAKE2S_HASH_SIZE]; + u8 hash2[BLAKE2S_HASH_SIZE]; + struct blake2s_state state; + + rand_bytes(key, key_len); + memcpy(guarded_key, key, key_len); + + blake2s(hash1, test_buf, key, + BLAKE2S_HASH_SIZE, data_len, key_len); + blake2s(hash2, test_buf, guarded_key, + BLAKE2S_HASH_SIZE, data_len, key_len); + KUNIT_ASSERT_MEMEQ(test, hash1, hash2, BLAKE2S_HASH_SIZE); + + blake2s_init_key(&state, BLAKE2S_HASH_SIZE, + guarded_key, key_len); + blake2s_update(&state, test_buf, data_len); + blake2s_final(&state, hash2); + KUNIT_ASSERT_MEMEQ(test, hash1, hash2, BLAKE2S_HASH_SIZE); + } +} + +/* + * BLAKE2s specific test case which tests using a guarded output buffer fo= r all + * allowed output lengths. + */ +static void test_blake2s_with_guarded_out_buf(struct kunit *test) +{ + const size_t data_len =3D 100; + + rand_bytes(test_buf, data_len); + for (int out_len =3D 1; out_len <=3D BLAKE2S_HASH_SIZE; out_len++) { + u8 hash[BLAKE2S_HASH_SIZE]; + u8 *guarded_hash =3D &test_buf[TEST_BUF_LEN - out_len]; + + blake2s(hash, test_buf, NULL, out_len, data_len, 0); + blake2s(guarded_hash, test_buf, NULL, out_len, data_len, 0); + KUNIT_ASSERT_MEMEQ(test, hash, guarded_hash, out_len); + } +} + +static struct kunit_case blake2s_test_cases[] =3D { + HASH_KUNIT_CASES, + KUNIT_CASE(test_blake2s_all_key_and_hash_lens), + KUNIT_CASE(test_blake2s_with_guarded_key_buf), + KUNIT_CASE(test_blake2s_with_guarded_out_buf), + KUNIT_CASE(benchmark_hash), + {}, +}; + +static struct kunit_suite blake2s_test_suite =3D { + .name =3D "blake2s", + .test_cases =3D blake2s_test_cases, + .suite_init =3D hash_suite_init, + .suite_exit =3D hash_suite_exit, +}; +kunit_test_suite(blake2s_test_suite); + +MODULE_DESCRIPTION("KUnit tests and benchmark for BLAKE2s"); +MODULE_LICENSE("GPL"); diff --git a/scripts/crypto/gen-hash-testvecs.py b/scripts/crypto/gen-hash-= testvecs.py index 4ac927d40cf5c..fc063f2ee95f1 100755 --- a/scripts/crypto/gen-hash-testvecs.py +++ b/scripts/crypto/gen-hash-testvecs.py @@ -82,15 +82,20 @@ def print_static_u8_array_definition(name, value): def print_c_struct_u8_array_field(name, value): print(f'\t\t.{name} =3D {{') print_bytes('\t\t\t', value, 8) print('\t\t},') =20 +def alg_digest_size_const(alg): + if alg =3D=3D 'blake2s': + return 'BLAKE2S_HASH_SIZE' + return f'{alg.upper()}_DIGEST_SIZE' + def gen_unkeyed_testvecs(alg): print('') print('static const struct {') print('\tsize_t data_len;') - print(f'\tu8 digest[{alg.upper()}_DIGEST_SIZE];') + print(f'\tu8 digest[{alg_digest_size_const(alg)}];') print('} hash_testvecs[] =3D {') for data_len in DATA_LENS: data =3D rand_bytes(data_len) print('\t{') print(f'\t\t.data_len =3D {data_len},') @@ -101,11 +106,11 @@ def gen_unkeyed_testvecs(alg): data =3D rand_bytes(4096) ctx =3D hash_init(alg) for data_len in range(len(data) + 1): hash_update(ctx, compute_hash(alg, data[:data_len])) print_static_u8_array_definition( - f'hash_testvec_consolidated[{alg.upper()}_DIGEST_SIZE]', + f'hash_testvec_consolidated[{alg_digest_size_const(alg)}]', hash_final(ctx)) =20 def gen_hmac_testvecs(alg): ctx =3D hmac.new(rand_bytes(32), digestmod=3Dalg) data =3D rand_bytes(4096) @@ -117,10 +122,24 @@ def gen_hmac_testvecs(alg): ctx.update(mac) print_static_u8_array_definition( f'hmac_testvec_consolidated[{alg.upper()}_DIGEST_SIZE]', ctx.digest()) =20 +BLAKE2S_KEY_SIZE =3D 32 +BLAKE2S_HASH_SIZE =3D 32 + +def gen_additional_blake2s_testvecs(): + hashes =3D b'' + for key_len in range(BLAKE2S_KEY_SIZE + 1): + for out_len in range(1, BLAKE2S_HASH_SIZE + 1): + h =3D hashlib.blake2s(digest_size=3Dout_len, key=3Drand_bytes(= key_len)) + h.update(rand_bytes(100)) + hashes +=3D h.digest() + print_static_u8_array_definition( + 'blake2s_keyed_testvec_consolidated[BLAKE2S_HASH_SIZE]', + compute_hash('blake2s', hashes)) + def gen_additional_poly1305_testvecs(): key =3D b'\xff' * POLY1305_KEY_SIZE data =3D b'' ctx =3D Poly1305(key) for _ in range(32): @@ -139,9 +158,11 @@ if len(sys.argv) !=3D 2: =20 alg =3D sys.argv[1] print('/* SPDX-License-Identifier: GPL-2.0-or-later */') print(f'/* This file was generated by: {sys.argv[0]} {" ".join(sys.argv[1:= ])} */') gen_unkeyed_testvecs(alg) -if alg =3D=3D 'poly1305': +if alg =3D=3D 'blake2s': + gen_additional_blake2s_testvecs() +elif alg =3D=3D 'poly1305': gen_additional_poly1305_testvecs() else: gen_hmac_testvecs(alg) --=20 2.50.1