From nobody Wed Sep 10 01:36:57 2025 Received: from fllvem-ot04.ext.ti.com (fllvem-ot04.ext.ti.com [198.47.19.246]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B61A1E260A; Mon, 8 Sep 2025 14:10:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.19.246 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340605; cv=none; b=jFTwzIWMLVnkf9dcR3XKwZC2H20PMv93rFFsz0oOd4nQdir8fnFnIXn+nPrxjiu+94osbZuiL153kt1aRVsn1IykWZX6XyE5OCzSlgXxeIXB9qGXvImgLgDB/8ZRUbVArQlimJnLrqMMlBIR404N6oMviNK/1bEURFbgyKQ6XGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340605; c=relaxed/simple; bh=X06+KYf7g5G3DpMRgGYFx2n380X1Eq8wDc0QXAPf2Lc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=JCBsgj+g5CDrzW5P2gdzDK47WDUS7S2DuN9GDY4mOR03AhizS9IKG2xbif66uoUZfzeNQlE33cg5cQCHHuOAcAF65sgNaqol8s7wmXv81G0V34JwqsNm5GUY6oVupD7ynOGFgT/A8dVj2FXaJ+884SKJTCEzFEDBNOe9E6a15/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=GhCxc0GY; arc=none smtp.client-ip=198.47.19.246 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="GhCxc0GY" Received: from lelvem-sh02.itg.ti.com ([10.180.78.226]) by fllvem-ot04.ext.ti.com (8.15.2/8.15.2) with ESMTP id 588E9vfc066816; Mon, 8 Sep 2025 09:09:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1757340597; bh=dNtpkaHyN6srN/vq/VvKfgni3NKEArpz7XSJccCF30M=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=GhCxc0GYPCJy+T2P7G3Q3CiKTBbsDogobcAuu2SOEbNCDci9C9YMJRfcQyaqrdhbW rPou7ngHqPqzONe6Y/sVW8jkVDiraaztflg2do/0ZP6TRWgN/64I/CMECxxGj2Tpzy Cun+liYD8LEfskzLQeRJY1POBGJZBaasW8C78yN8= Received: from DLEE113.ent.ti.com (dlee113.ent.ti.com [157.170.170.24]) by lelvem-sh02.itg.ti.com (8.18.1/8.18.1) with ESMTPS id 588E9vbm3009109 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA256 bits=128 verify=FAIL); Mon, 8 Sep 2025 09:09:57 -0500 Received: from DLEE106.ent.ti.com (157.170.170.36) by DLEE113.ent.ti.com (157.170.170.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55; Mon, 8 Sep 2025 09:09:57 -0500 Received: from lelvem-mr05.itg.ti.com (10.180.75.9) by DLEE106.ent.ti.com (157.170.170.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55 via Frontend Transport; Mon, 8 Sep 2025 09:09:57 -0500 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr05.itg.ti.com (8.18.1/8.18.1) with ESMTP id 588E9tqD1013827; Mon, 8 Sep 2025 09:09:56 -0500 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Kamlesh Gurudasani , Manorit Chawdhry , Vignesh Raghavendra , Praneeth Bajjuri , Vishal Mahaveer , Kavitha Malarvizhi , , Subject: [PATCH v2 1/4] crypto: ti: Add support for AES-XTS in DTHEv2 driver Date: Mon, 8 Sep 2025 19:28:13 +0530 Message-ID: <20250908140928.2801062-2-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250908140928.2801062-1-t-pratham@ti.com> References: <20250908140928.2801062-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea Content-Type: text/plain; charset="utf-8" Add support for XTS mode of operation for AES algorithm in the AES Engine of the DTHEv2 hardware cryptographic engine. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 1 + drivers/crypto/ti/dthev2-aes.c | 73 ++++++++++++++++++++++++++++++- drivers/crypto/ti/dthev2-common.h | 8 +++- 3 files changed, 80 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index 38d22cab05a9..bdb7f652e093 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -6,6 +6,7 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_SKCIPHER select CRYPTO_ECB select CRYPTO_CBC + select CRYPTO_XTS help This enables support for the TI DTHE V2 hw cryptography engine which can be found on TI K3 SOCs. Selecting this enables use diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index 0431a36d8c4a..21e7e5b8d6da 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -25,6 +25,7 @@ =20 // AES Engine #define DTHE_P_AES_BASE 0x7000 + #define DTHE_P_AES_KEY1_0 0x0038 #define DTHE_P_AES_KEY1_1 0x003C #define DTHE_P_AES_KEY1_2 0x0030 @@ -33,6 +34,16 @@ #define DTHE_P_AES_KEY1_5 0x002C #define DTHE_P_AES_KEY1_6 0x0020 #define DTHE_P_AES_KEY1_7 0x0024 + +#define DTHE_P_AES_KEY2_0 0x0018 +#define DTHE_P_AES_KEY2_1 0x001C +#define DTHE_P_AES_KEY2_2 0x0010 +#define DTHE_P_AES_KEY2_3 0x0014 +#define DTHE_P_AES_KEY2_4 0x0008 +#define DTHE_P_AES_KEY2_5 0x000C +#define DTHE_P_AES_KEY2_6 0x0000 +#define DTHE_P_AES_KEY2_7 0x0004 + #define DTHE_P_AES_IV_IN_0 0x0040 #define DTHE_P_AES_IV_IN_1 0x0044 #define DTHE_P_AES_IV_IN_2 0x0048 @@ -52,6 +63,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_ECB_MASK =3D 0x00, AES_CTRL_CBC_MASK =3D BIT(5), + AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), }; =20 #define DTHE_AES_CTRL_MODE_CLEAR_MASK ~GENMASK(28, 5) @@ -119,6 +131,22 @@ static int dthe_aes_cbc_setkey(struct crypto_skcipher = *tfm, const u8 *key, unsig return dthe_aes_setkey(tfm, key, keylen); } =20 +static int dthe_aes_xts_setkey(struct crypto_skcipher *tfm, const u8 *key,= unsigned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); + + if (keylen !=3D 2 * AES_KEYSIZE_128 && + keylen !=3D 2 * AES_KEYSIZE_192 && + keylen !=3D 2 * AES_KEYSIZE_256) + return -EINVAL; + + ctx->aes_mode =3D DTHE_AES_XTS; + ctx->keylen =3D keylen / 2; + memcpy(ctx->key, key, keylen); + + return 0; +} + static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx *ctx, struct dthe_aes_req_ctx *rctx, u32 *iv_in) @@ -141,6 +169,24 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx = *ctx, writel_relaxed(ctx->key[7], aes_base_reg + DTHE_P_AES_KEY1_7); } =20 + if (ctx->aes_mode =3D=3D DTHE_AES_XTS) { + size_t key2_offset =3D ctx->keylen / sizeof(u32); + + writel_relaxed(ctx->key[key2_offset + 0], aes_base_reg + DTHE_P_AES_KEY2= _0); + writel_relaxed(ctx->key[key2_offset + 1], aes_base_reg + DTHE_P_AES_KEY2= _1); + writel_relaxed(ctx->key[key2_offset + 2], aes_base_reg + DTHE_P_AES_KEY2= _2); + writel_relaxed(ctx->key[key2_offset + 3], aes_base_reg + DTHE_P_AES_KEY2= _3); + + if (ctx->keylen > AES_KEYSIZE_128) { + writel_relaxed(ctx->key[key2_offset + 4], aes_base_reg + DTHE_P_AES_KEY= 2_4); + writel_relaxed(ctx->key[key2_offset + 5], aes_base_reg + DTHE_P_AES_KEY= 2_5); + } + if (ctx->keylen =3D=3D AES_KEYSIZE_256) { + writel_relaxed(ctx->key[key2_offset + 6], aes_base_reg + DTHE_P_AES_KEY= 2_6); + writel_relaxed(ctx->key[key2_offset + 7], aes_base_reg + DTHE_P_AES_KEY= 2_7); + } + } + if (rctx->enc) ctrl_val |=3D DTHE_AES_CTRL_DIR_ENC; =20 @@ -160,6 +206,9 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx *= ctx, case DTHE_AES_CBC: ctrl_val |=3D AES_CTRL_CBC_MASK; break; + case DTHE_AES_XTS: + ctrl_val |=3D AES_CTRL_XTS_MASK; + break; } =20 if (iv_in) { @@ -397,7 +446,29 @@ static struct skcipher_engine_alg cipher_algs[] =3D { .cra_module =3D THIS_MODULE, }, .op.do_one_request =3D dthe_aes_run, - } /* CBC AES */ + }, /* CBC AES */ + { + .base.init =3D dthe_cipher_init_tfm, + .base.setkey =3D dthe_aes_xts_setkey, + .base.encrypt =3D dthe_aes_encrypt, + .base.decrypt =3D dthe_aes_decrypt, + .base.min_keysize =3D AES_MIN_KEY_SIZE * 2, + .base.max_keysize =3D AES_MAX_KEY_SIZE * 2, + .base.ivsize =3D AES_IV_SIZE, + .base.base =3D { + .cra_name =3D "xts(aes)", + .cra_driver_name =3D "xts-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_KERN_DRIVER_ONLY, + .cra_alignmask =3D AES_BLOCK_SIZE - 1, + .cra_blocksize =3D AES_BLOCK_SIZE, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aes_run, + }, /* XTS AES */ }; =20 int dthe_register_aes_algs(void) diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index 68c94acda8aa..e2d901e70fcc 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -27,10 +27,16 @@ =20 #define DTHE_REG_SIZE 4 #define DTHE_DMA_TIMEOUT_MS 2000 +/* + * Size of largest possible key (of all algorithms) to be stored in dthe_t= fm_ctx + * This is currently the keysize of XTS-AES-256 which is 512 bits (64 byte= s) + */ +#define DTHE_MAX_KEYSIZE (AES_KEYSIZE_256 * 2) =20 enum dthe_aes_mode { DTHE_AES_ECB =3D 0, DTHE_AES_CBC, + DTHE_AES_XTS, }; =20 /* Driver specific struct definitions */ @@ -77,7 +83,7 @@ struct dthe_list { struct dthe_tfm_ctx { struct dthe_data *dev_data; unsigned int keylen; - u32 key[AES_KEYSIZE_256 / sizeof(u32)]; + u32 key[DTHE_MAX_KEYSIZE / sizeof(u32)]; enum dthe_aes_mode aes_mode; }; =20 --=20 2.43.0 From nobody Wed Sep 10 01:36:57 2025 Received: from lelvem-ot02.ext.ti.com (lelvem-ot02.ext.ti.com [198.47.23.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 336C730BBA4; Mon, 8 Sep 2025 14:10:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.23.235 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340609; cv=none; b=ky1kM6bWFt2t8NCjF4y5Luax0084+/yB2qjjcTY+DLByX+SKHlriQIgBU7RIB7FipP9WYkkFCAFH7B7+9FJLRSaU8UegA5HrX93Ef5+cfetc/t+hUCqxkSinqSgp8iNkStGGA3PzHtPI8Dmd4W0fkq4xra1j4n3E1Ep/sMH0pFg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340609; c=relaxed/simple; bh=5lagy10vP+ElfLNZ4XPQUjfGJpMlZht7fVKsoNotb30=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Ln1DP5C29Ot8HQR3LQ1wbXnmPnaS4QAcfHLHRF7f4t2qfnnxqAMfPX/4D1BWYIwAATcT+IqrhfyBZoNNUmqHG3bAWAxfTrSUGktC5FSk2XSxq8cvqQ7np2OS1QpAUEzCf/PV0+n0Xuq/ChZjvx1UXfVVUYH8Oh0+1ZfkbQoD0HQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=i0OJDsqn; arc=none smtp.client-ip=198.47.23.235 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="i0OJDsqn" Received: from lelvem-sh02.itg.ti.com ([10.180.78.226]) by lelvem-ot02.ext.ti.com (8.15.2/8.15.2) with ESMTP id 588EA3qn116878; Mon, 8 Sep 2025 09:10:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1757340603; bh=LRS0GaO8Vw+EmQvBRmovB7EJCsf65v1b7tdlOx4p+hY=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=i0OJDsqnZkRoy+Vj2wMWLxpLviy9F8pLZk4IpUyvoU4+fjWusQjIcPpIizRciXqm+ lzKAf5sE05Q9Zct/wzY6LDQ0ZHi5K3W+Qzfih3oOL3ORuJoesyoyqz9aTstRavx8eK XW3xfQjdCP2ORz2U+ZTjW4R/JxAIeadliIPBsp3s= Received: from DLEE101.ent.ti.com (dlee101.ent.ti.com [157.170.170.31]) by lelvem-sh02.itg.ti.com (8.18.1/8.18.1) with ESMTPS id 588EA32I3009212 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA256 bits=128 verify=FAIL); Mon, 8 Sep 2025 09:10:03 -0500 Received: from DLEE114.ent.ti.com (157.170.170.25) by DLEE101.ent.ti.com (157.170.170.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55; Mon, 8 Sep 2025 09:10:02 -0500 Received: from lelvem-mr05.itg.ti.com (10.180.75.9) by DLEE114.ent.ti.com (157.170.170.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55 via Frontend Transport; Mon, 8 Sep 2025 09:10:02 -0500 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr05.itg.ti.com (8.18.1/8.18.1) with ESMTP id 588EA1Hq1013989; Mon, 8 Sep 2025 09:10:02 -0500 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Kamlesh Gurudasani , Manorit Chawdhry , Vignesh Raghavendra , Praneeth Bajjuri , Vishal Mahaveer , Kavitha Malarvizhi , , Subject: [PATCH v2 2/4] crypto: ti: Add support for AES-CTR in DTHEv2 driver Date: Mon, 8 Sep 2025 19:28:14 +0530 Message-ID: <20250908140928.2801062-3-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250908140928.2801062-1-t-pratham@ti.com> References: <20250908140928.2801062-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea Content-Type: text/plain; charset="utf-8" Add support for CTR mode of operation for AES algorithm in the AES Engine of the DTHEv2 hardware cryptographic engine. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 1 + drivers/crypto/ti/dthev2-aes.c | 80 +++++++++++++++++++++++++++++-- drivers/crypto/ti/dthev2-common.c | 19 ++++++++ drivers/crypto/ti/dthev2-common.h | 15 ++++++ 4 files changed, 110 insertions(+), 5 deletions(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index bdb7f652e093..b36733a8c734 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -6,6 +6,7 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_SKCIPHER select CRYPTO_ECB select CRYPTO_CBC + select CRYPTO_CTR select CRYPTO_XTS help This enables support for the TI DTHE V2 hw cryptography engine diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index 21e7e5b8d6da..07787f0ae4fd 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -63,6 +63,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_ECB_MASK =3D 0x00, AES_CTRL_CBC_MASK =3D BIT(5), + AES_CTRL_CTR_MASK =3D BIT(6), AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), }; =20 @@ -74,6 +75,8 @@ enum aes_ctrl_mode_masks { #define DTHE_AES_CTRL_KEYSIZE_24B BIT(4) #define DTHE_AES_CTRL_KEYSIZE_32B (BIT(3) | BIT(4)) =20 +#define DTHE_AES_CTRL_CTR_WIDTH_128B (BIT(7) | BIT(8)) + #define DTHE_AES_CTRL_SAVE_CTX_SET BIT(29) =20 #define DTHE_AES_CTRL_OUTPUT_READY BIT_MASK(0) @@ -131,6 +134,15 @@ static int dthe_aes_cbc_setkey(struct crypto_skcipher = *tfm, const u8 *key, unsig return dthe_aes_setkey(tfm, key, keylen); } =20 +static int dthe_aes_ctr_setkey(struct crypto_skcipher *tfm, const u8 *key,= unsigned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); + + ctx->aes_mode =3D DTHE_AES_CTR; + + return dthe_aes_setkey(tfm, key, keylen); +} + static int dthe_aes_xts_setkey(struct crypto_skcipher *tfm, const u8 *key,= unsigned int keylen) { struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); @@ -206,6 +218,10 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx = *ctx, case DTHE_AES_CBC: ctrl_val |=3D AES_CTRL_CBC_MASK; break; + case DTHE_AES_CTR: + ctrl_val |=3D AES_CTRL_CTR_MASK; + ctrl_val |=3D DTHE_AES_CTRL_CTR_WIDTH_128B; + break; case DTHE_AES_XTS: ctrl_val |=3D AES_CTRL_XTS_MASK; break; @@ -246,6 +262,9 @@ static int dthe_aes_run(struct crypto_engine *engine, v= oid *areq) int src_mapped_nents; int dst_mapped_nents; =20 + u8 pad_buf[AES_BLOCK_SIZE] =3D {0}; + int pad_len =3D 0; + bool diff_dst; enum dma_data_direction src_dir, dst_dir; =20 @@ -265,6 +284,29 @@ static int dthe_aes_run(struct crypto_engine *engine, = void *areq) aes_irqenable_val |=3D DTHE_AES_IRQENABLE_EN_ALL; writel_relaxed(aes_irqenable_val, aes_base_reg + DTHE_P_AES_IRQENABLE); =20 + if (ctx->aes_mode =3D=3D DTHE_AES_CTR) { + /* + * CTR mode can operate on any input length, but the hardware + * requires input length to be a multiple of the block size. + * We need to handle the padding in the driver. + */ + if (req->cryptlen % AES_BLOCK_SIZE) { + /* Need to create a new SG list with padding */ + pad_len =3D ALIGN(req->cryptlen, AES_BLOCK_SIZE) - req->cryptlen; + struct scatterlist *sg; + + src =3D kmalloc_array((src_nents + 1), sizeof(*src), GFP_KERNEL); + if (!src) { + ret =3D -ENOMEM; + goto aes_alloc_err; + } + sg_init_table(src, src_nents + 1); + sg =3D dthe_copy_sg(src, req->src, req->cryptlen); + sg_set_buf(sg, pad_buf, pad_len); + src_nents++; + } + } + if (src =3D=3D dst) { diff_dst =3D false; src_dir =3D DMA_BIDIRECTIONAL; @@ -281,7 +323,7 @@ static int dthe_aes_run(struct crypto_engine *engine, v= oid *areq) src_mapped_nents =3D dma_map_sg(tx_dev, src, src_nents, src_dir); if (src_mapped_nents =3D=3D 0) { ret =3D -EINVAL; - goto aes_err; + goto aes_map_src_err; } =20 if (!diff_dst) { @@ -293,7 +335,7 @@ static int dthe_aes_run(struct crypto_engine *engine, v= oid *areq) if (dst_mapped_nents =3D=3D 0) { dma_unmap_sg(tx_dev, src, src_nents, src_dir); ret =3D -EINVAL; - goto aes_err; + goto aes_map_dst_err; } } =20 @@ -356,11 +398,16 @@ static int dthe_aes_run(struct crypto_engine *engine,= void *areq) } =20 aes_prep_err: - dma_unmap_sg(tx_dev, src, src_nents, src_dir); if (dst_dir !=3D DMA_BIDIRECTIONAL) dma_unmap_sg(rx_dev, dst, dst_nents, dst_dir); +aes_map_dst_err: + dma_unmap_sg(tx_dev, src, src_nents, src_dir); =20 -aes_err: +aes_map_src_err: + if (ctx->aes_mode =3D=3D DTHE_AES_CTR && req->cryptlen % AES_BLOCK_SIZE) + kfree(src); + +aes_alloc_err: local_bh_disable(); crypto_finalize_skcipher_request(dev_data->engine, req, ret); local_bh_enable(); @@ -375,9 +422,10 @@ static int dthe_aes_crypt(struct skcipher_request *req) =20 /* * If data is not a multiple of AES_BLOCK_SIZE, need to return -EINVAL + * except in CTR mode, where any length is supported. * If data length input is zero, no need to do any operation. */ - if (req->cryptlen % AES_BLOCK_SIZE) + if (req->cryptlen % AES_BLOCK_SIZE && ctx->aes_mode !=3D DTHE_AES_CTR) return -EINVAL; =20 if (req->cryptlen =3D=3D 0) @@ -447,6 +495,28 @@ static struct skcipher_engine_alg cipher_algs[] =3D { }, .op.do_one_request =3D dthe_aes_run, }, /* CBC AES */ + { + .base.init =3D dthe_cipher_init_tfm, + .base.setkey =3D dthe_aes_ctr_setkey, + .base.encrypt =3D dthe_aes_encrypt, + .base.decrypt =3D dthe_aes_decrypt, + .base.min_keysize =3D AES_MIN_KEY_SIZE, + .base.max_keysize =3D AES_MAX_KEY_SIZE, + .base.ivsize =3D AES_IV_SIZE, + .base.base =3D { + .cra_name =3D "ctr(aes)", + .cra_driver_name =3D "ctr-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_KERN_DRIVER_ONLY, + .cra_alignmask =3D AES_BLOCK_SIZE - 1, + .cra_blocksize =3D 1, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aes_run, + }, /* CTR AES */ { .base.init =3D dthe_cipher_init_tfm, .base.setkey =3D dthe_aes_xts_setkey, diff --git a/drivers/crypto/ti/dthev2-common.c b/drivers/crypto/ti/dthev2-c= ommon.c index c39d37933b9e..a2ad79bec105 100644 --- a/drivers/crypto/ti/dthev2-common.c +++ b/drivers/crypto/ti/dthev2-common.c @@ -48,6 +48,25 @@ struct dthe_data *dthe_get_dev(struct dthe_tfm_ctx *ctx) return dev_data; } =20 +struct scatterlist *dthe_copy_sg(struct scatterlist *dst, + struct scatterlist *src, + int buflen) +{ + struct scatterlist *from_sg, *to_sg; + int sglen; + + for (to_sg =3D dst, from_sg =3D src; buflen && from_sg; buflen -=3D sglen= ) { + sglen =3D from_sg->length; + if (sglen > buflen) + sglen =3D buflen; + sg_set_buf(to_sg, sg_virt(from_sg), sglen); + from_sg =3D sg_next(from_sg); + to_sg =3D sg_next(to_sg); + } + + return to_sg; +} + static int dthe_dma_init(struct dthe_data *dev_data) { int ret; diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index e2d901e70fcc..629f45a3ab33 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -36,6 +36,7 @@ enum dthe_aes_mode { DTHE_AES_ECB =3D 0, DTHE_AES_CBC, + DTHE_AES_CTR, DTHE_AES_XTS, }; =20 @@ -101,6 +102,20 @@ struct dthe_aes_req_ctx { =20 struct dthe_data *dthe_get_dev(struct dthe_tfm_ctx *ctx); =20 +/** + * dthe_copy_sg - Copy sg entries from src to dst + * @dst: Destination sg to be filled + * @src: Source sg to be copied from + * @buflen: Number of bytes to be copied + * + * Description: + * Copy buflen bytes of data from src to dst. + * + **/ +struct scatterlist *dthe_copy_sg(struct scatterlist *dst, + struct scatterlist *src, + int buflen); + int dthe_register_aes_algs(void); void dthe_unregister_aes_algs(void); =20 --=20 2.43.0 From nobody Wed Sep 10 01:36:57 2025 Received: from lelvem-ot01.ext.ti.com (lelvem-ot01.ext.ti.com [198.47.23.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0B0230C63A; Mon, 8 Sep 2025 14:10:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.23.234 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340614; cv=none; b=tpkLQ7JNzFRLzAP0kESMZKtwUv13iUlztgCFTYBFgbADKslm7aLTwZeCnAXTaFrTNx0EQ3n0G1pJzwxgNNFGYJ8BMW1qM6Sh8WLJRgbPEHzaBZMKRC9Eu75nqeeBloIJHPMAs1hUBLCSLM3p7VzZR/OdL0LAoQZdTPIx9aj6sB8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340614; c=relaxed/simple; bh=9x4YK+NGkgeQJPC3KJaGaK7rZN3IusH0H9n77wqH9Dk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qwnZXFHku7e5QRA66R91ccLwBkYh7CKnwGAIKg2ZLZPvOdXIitFG2U4Vtvkdldur+S/Nbr8flymu+dpdxBvLgDSvaRlsFBlM5E97uabtu79tYpRbVtQ0wUx/BtLCLmShYNysVKezgt6nnWuC3gv2Ou9b/VfP7jmmp2ynoNAmQEA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=DDgq96Jo; arc=none smtp.client-ip=198.47.23.234 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="DDgq96Jo" Received: from fllvem-sh04.itg.ti.com ([10.64.41.54]) by lelvem-ot01.ext.ti.com (8.15.2/8.15.2) with ESMTP id 588EA8Sp3879943; Mon, 8 Sep 2025 09:10:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1757340608; bh=fH9lx0Z3b02tgpza7zAV8SBn/o9GRoQn7g8k8SVUFdM=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=DDgq96JozTS3NMr+7LfJl4xzhT528I6vEDJ8YocPMZ6XoNU08PUWmX2sloZB4YO0w WPt7lc1Bf5aiwydbdedkBvv9SpNvQiVpK8MHK5/f2E8SbF2qVB5fFqw3rI4vs6/7cx vM0pxomzOWTESn3mSzLHzDsegVkxEnw2mxPAfbmY= Received: from DFLE102.ent.ti.com (dfle102.ent.ti.com [10.64.6.23]) by fllvem-sh04.itg.ti.com (8.18.1/8.18.1) with ESMTPS id 588EA87T3823048 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA256 bits=128 verify=FAIL); Mon, 8 Sep 2025 09:10:08 -0500 Received: from DFLE112.ent.ti.com (10.64.6.33) by DFLE102.ent.ti.com (10.64.6.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55; Mon, 8 Sep 2025 09:10:08 -0500 Received: from lelvem-mr06.itg.ti.com (10.180.75.8) by DFLE112.ent.ti.com (10.64.6.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55 via Frontend Transport; Mon, 8 Sep 2025 09:10:08 -0500 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr06.itg.ti.com (8.18.1/8.18.1) with ESMTP id 588EA6fJ719178; Mon, 8 Sep 2025 09:10:07 -0500 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Kamlesh Gurudasani , Manorit Chawdhry , Vignesh Raghavendra , Praneeth Bajjuri , Vishal Mahaveer , Kavitha Malarvizhi , , Subject: [PATCH v2 3/4] crypto: ti: Add support for AES-GCM in DTHEv2 driver Date: Mon, 8 Sep 2025 19:28:15 +0530 Message-ID: <20250908140928.2801062-4-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250908140928.2801062-1-t-pratham@ti.com> References: <20250908140928.2801062-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea Content-Type: text/plain; charset="utf-8" AES-GCM is an AEAD algorithm supporting both encryption and authentication of data. This patch introduces support for AES-GCM as the first AEAD algorithm supported by the DTHEv2 driver. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 2 + drivers/crypto/ti/dthev2-aes.c | 601 +++++++++++++++++++++++++++++- drivers/crypto/ti/dthev2-common.h | 6 + 3 files changed, 608 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index b36733a8c734..e1ef84b39267 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -8,6 +8,8 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_CBC select CRYPTO_CTR select CRYPTO_XTS + select CRYPTO_GCM + select SG_SPLIT help This enables support for the TI DTHE V2 hw cryptography engine which can be found on TI K3 SOCs. Selecting this enables use diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index 07787f0ae4fd..37bc43a1ba73 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include =20 @@ -53,6 +54,7 @@ #define DTHE_P_AES_C_LENGTH_1 0x0058 #define DTHE_P_AES_AUTH_LENGTH 0x005C #define DTHE_P_AES_DATA_IN_OUT 0x0060 +#define DTHE_P_AES_TAG_OUT 0x0070 =20 #define DTHE_P_AES_SYSCONFIG 0x0084 #define DTHE_P_AES_IRQSTATUS 0x008C @@ -65,6 +67,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_CBC_MASK =3D BIT(5), AES_CTRL_CTR_MASK =3D BIT(6), AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), + AES_CTRL_GCM_MASK =3D BIT(17) | BIT(16) | BIT(6), }; =20 #define DTHE_AES_CTRL_MODE_CLEAR_MASK ~GENMASK(28, 5) @@ -91,6 +94,21 @@ enum aes_ctrl_mode_masks { #define AES_IV_SIZE AES_BLOCK_SIZE #define AES_BLOCK_WORDS (AES_BLOCK_SIZE / sizeof(u32)) #define AES_IV_WORDS AES_BLOCK_WORDS +#define DTHE_AES_GCM_AAD_MAXLEN (BIT_ULL(32) - 1) +#define POLL_TIMEOUT_INTERVAL HZ + +static int dthe_poll_reg(struct dthe_data *dev_data, u32 reg, u32 bit) +{ + void __iomem *aes_base_reg =3D dev_data->regs + DTHE_P_AES_BASE; + unsigned long timeout =3D jiffies + POLL_TIMEOUT_INTERVAL; + + while (!(readl_relaxed(aes_base_reg + reg) & bit)) { + if (time_is_before_jiffies(timeout)) + return -ETIMEDOUT; + } + + return 0; +} =20 static int dthe_cipher_init_tfm(struct crypto_skcipher *tfm) { @@ -225,6 +243,9 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx *= ctx, case DTHE_AES_XTS: ctrl_val |=3D AES_CTRL_XTS_MASK; break; + case DTHE_AES_GCM: + ctrl_val |=3D AES_CTRL_GCM_MASK; + break; } =20 if (iv_in) { @@ -451,6 +472,550 @@ static int dthe_aes_decrypt(struct skcipher_request *= req) return dthe_aes_crypt(req); } =20 +static int dthe_aead_init_tfm(struct crypto_aead *tfm) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + + memzero_explicit(ctx, sizeof(*ctx)); + ctx->dev_data =3D dev_data; + + const char *alg_name =3D crypto_tfm_alg_name(&tfm->base); + + ctx->aead_fb =3D crypto_alloc_aead(alg_name, 0, + CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->aead_fb)) { + dev_err(dev_data->dev, "fallback driver %s couldn't be loaded\n", + alg_name); + return PTR_ERR(ctx->aead_fb); + } + + return 0; +} + +static void dthe_aead_exit_tfm(struct crypto_aead *tfm) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + crypto_free_aead(ctx->aead_fb); +} + +/** + * dthe_aead_prep_src - Prepare source scatterlist for AEAD from input req= ->src + * @sg: Input req->src scatterlist + * @assoclen: Input req->assoclen + * @cryptlen: Input req->cryptlen (minus the size of TAG in decryption) + * + * Description: + * For modes with authentication, DTHEv2 hardware requires the input AAD= and + * plaintext/ciphertext to be individually aligned to AES_BLOCK_SIZE. Ho= wever, + * linux crypto's aead_request provides the input with AAD and plaintext= /ciphertext + * contiguously in a single scatterlist. + * + * This helper function takes the input scatterlist and splits it into s= eparate + * scatterlists for AAD and plaintext/ciphertext, ensuring each is align= ed to + * AES_BLOCK_SIZE, and then merges the aligned scatterlists back into a = single + * scatterlist for processing. + * + * Return: + * Pointer to the merged scatterlist, or NULL on failure. + **/ +static struct scatterlist *dthe_aead_prep_src(struct scatterlist *sg, + unsigned int assoclen, + unsigned int cryptlen) +{ + struct scatterlist *in_sg[2]; + struct scatterlist *to_sg; + struct scatterlist *src, *ret; + size_t split_sizes[2] =3D {assoclen, cryptlen}; + int out_mapped_nents[2]; + int crypt_nents =3D 0, assoc_nents =3D 0, src_nents =3D 0; + + /* sg_split does not work properly if one of the split_sizes is 0 */ + if (cryptlen =3D=3D 0 || assoclen =3D=3D 0) { + /* + * Assigning both to sg does not matter as assoclen =3D 0 or cryptlen = =3D 0 + * being passed to dthe_copy_sg will take care to copy the sg correctly + */ + in_sg[0] =3D sg; + in_sg[1] =3D sg; + + src_nents =3D sg_nents_for_len(sg, assoclen + cryptlen); + } else { + sg_split(sg, 0, 0, 2, split_sizes, in_sg, out_mapped_nents, GFP_KERNEL); + assoc_nents =3D sg_nents_for_len(in_sg[0], assoclen); + crypt_nents =3D sg_nents_for_len(in_sg[1], cryptlen); + + src_nents =3D assoc_nents + crypt_nents; + } + + if (assoclen % AES_BLOCK_SIZE) + src_nents++; + if (cryptlen % AES_BLOCK_SIZE) + src_nents++; + + src =3D kmalloc_array(src_nents, sizeof(struct scatterlist), GFP_KERNEL); + if (!src) { + ret =3D NULL; + goto dthe_aead_prep_src_mem_err; + } + + sg_init_table(src, src_nents); + to_sg =3D src; + ret =3D src; + + to_sg =3D dthe_copy_sg(to_sg, in_sg[0], assoclen); + if (assoclen % AES_BLOCK_SIZE) { + unsigned int pad_len =3D AES_BLOCK_SIZE - (assoclen % AES_BLOCK_SIZE); + u8 *pad_buf =3D kzalloc(sizeof(u8) * pad_len, GFP_KERNEL); + + if (!pad_buf) { + kfree(src); + ret =3D NULL; + goto dthe_aead_prep_src_mem_err; + } + + sg_set_buf(to_sg, pad_buf, pad_len); + to_sg =3D sg_next(to_sg); + } + + to_sg =3D dthe_copy_sg(to_sg, in_sg[1], cryptlen); + if (cryptlen % AES_BLOCK_SIZE) { + unsigned int pad_len =3D AES_BLOCK_SIZE - (cryptlen % AES_BLOCK_SIZE); + u8 *pad_buf =3D kzalloc(sizeof(u8) * pad_len, GFP_KERNEL); + + if (!pad_buf) { + kfree(src); + ret =3D NULL; + goto dthe_aead_prep_src_mem_err; + } + + sg_set_buf(to_sg, pad_buf, pad_len); + to_sg =3D sg_next(to_sg); + } + +dthe_aead_prep_src_mem_err: + if (cryptlen !=3D 0 && assoclen !=3D 0) { + kfree(in_sg[0]); + kfree(in_sg[1]); + } + + return ret; +} + +/** + * dthe_aead_prep_dst - Prepare destination scatterlist for AEAD from inpu= t req->dst + * @sg: Input req->dst scatterlist + * @assoclen: Input req->assoclen + * @cryptlen: Input req->cryptlen (minus the size of TAG in decryption) + * + * Description: + * For modes with authentication, DTHEv2 hardware returns encrypted ciph= ertext/decrypted + * plaintext through DMA and TAG through MMRs. However, the dst scatterl= ist in linux + * crypto's aead_request is allocated same as input req->src scatterlist= . That is, it + * contains space for AAD in the beginning and ciphertext/plaintext at t= he end, with no + * alignment padding. This causes issues with DMA engine and DTHEv2 hard= ware. + * + * This helper function takes the output scatterlist and maps the part o= f the buffer + * which holds only the ciphertext/plaintext to a new scatterlist. It al= so adds a padding + * to align it with AES_BLOCK_SIZE. + * + * Return: + * Pointer to the trimmed scatterlist, or NULL on failure. + **/ +static struct scatterlist *dthe_aead_prep_dst(struct scatterlist *sg, + unsigned int assoclen, + unsigned int cryptlen) +{ + struct scatterlist *out_sg[1]; + struct scatterlist *dst; + struct scatterlist *to_sg; + size_t split_sizes[1] =3D {cryptlen}; + int out_mapped_nents[1]; + int dst_nents =3D 0; + + sg_split(sg, 0, assoclen, 1, split_sizes, out_sg, out_mapped_nents, GFP_K= ERNEL); + dst_nents =3D sg_nents_for_len(out_sg[0], cryptlen); + if (cryptlen % AES_BLOCK_SIZE) + dst_nents++; + + dst =3D kmalloc_array(dst_nents, sizeof(struct scatterlist), GFP_KERNEL); + if (!dst) { + kfree(out_sg[0]); + return NULL; + } + sg_init_table(dst, dst_nents); + + to_sg =3D dthe_copy_sg(dst, out_sg[0], cryptlen); + if (cryptlen % AES_BLOCK_SIZE) { + unsigned int pad_len =3D AES_BLOCK_SIZE - (cryptlen % AES_BLOCK_SIZE); + u8 *pad_buf =3D kzalloc(sizeof(u8) * pad_len, GFP_KERNEL); + + if (!pad_buf) { + kfree(dst); + kfree(out_sg[0]); + return NULL; + } + + sg_set_buf(to_sg, pad_buf, pad_len); + to_sg =3D sg_next(to_sg); + } + + kfree(out_sg[0]); + + return dst; +} + +static int dthe_aead_read_tag(struct dthe_tfm_ctx *ctx, u32 *tag) +{ + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + void __iomem *aes_base_reg =3D dev_data->regs + DTHE_P_AES_BASE; + int ret; + + ret =3D dthe_poll_reg(dev_data, DTHE_P_AES_CTRL, DTHE_AES_CTRL_SAVED_CTX_= READY); + if (ret) + return ret; + + for (int i =3D 0; i < AES_BLOCK_WORDS; ++i) + tag[i] =3D readl_relaxed(aes_base_reg + + DTHE_P_AES_TAG_OUT + + DTHE_REG_SIZE * i); + return 0; +} + +static int dthe_aead_enc_get_tag(struct aead_request *req) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + u32 tag[AES_BLOCK_WORDS]; + int nents; + int ret; + + ret =3D dthe_aead_read_tag(ctx, tag); + if (ret) + return ret; + + nents =3D sg_nents_for_len(req->dst, req->cryptlen + req->assoclen + ctx-= >authsize); + + sg_pcopy_from_buffer(req->dst, nents, tag, ctx->authsize, + req->assoclen + req->cryptlen); + + return 0; +} + +static int dthe_aead_dec_verify_tag(struct aead_request *req) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + u32 tag_out[AES_BLOCK_WORDS]; + u32 tag_in[AES_BLOCK_WORDS]; + int nents; + int ret; + + ret =3D dthe_aead_read_tag(ctx, tag_out); + if (ret) + return ret; + + nents =3D sg_nents_for_len(req->src, req->assoclen + req->cryptlen); + + sg_pcopy_to_buffer(req->src, nents, tag_in, ctx->authsize, + req->assoclen + req->cryptlen - ctx->authsize); + + if (memcmp(tag_in, tag_out, ctx->authsize)) + return -EBADMSG; + else + return 0; +} + +static int dthe_aead_setkey(struct crypto_aead *tfm, const u8 *key, unsign= ed int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + if (keylen !=3D AES_KEYSIZE_128 && keylen !=3D AES_KEYSIZE_192 && keylen = !=3D AES_KEYSIZE_256) + return -EINVAL; + + ctx->aes_mode =3D DTHE_AES_GCM; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + crypto_aead_clear_flags(ctx->aead_fb, CRYPTO_TFM_REQ_MASK); + crypto_aead_set_flags(ctx->aead_fb, + crypto_aead_get_flags(tfm) & + CRYPTO_TFM_REQ_MASK); + + return crypto_aead_setkey(ctx->aead_fb, key, keylen); +} + +static int dthe_aead_setauthsize(struct crypto_aead *tfm, unsigned int aut= hsize) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + /* Invalid auth size will be handled by crypto_aead_setauthsize() */ + ctx->authsize =3D authsize; + + return crypto_aead_setauthsize(ctx->aead_fb, authsize); +} + +static void dthe_aead_dma_in_callback(void *data) +{ + struct aead_request *req =3D (struct aead_request *)data; + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + + complete(&rctx->aes_compl); +} + +static int dthe_aead_run(struct crypto_engine *engine, void *areq) +{ + struct aead_request *req =3D container_of(areq, struct aead_request, base= ); + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + + unsigned int cryptlen =3D req->cryptlen; + unsigned int assoclen =3D req->assoclen; + unsigned int authsize =3D ctx->authsize; + unsigned int unpadded_cryptlen; + struct scatterlist *src =3D req->src; + struct scatterlist *dst =3D req->dst; + + int src_nents; + int dst_nents; + int src_mapped_nents, dst_mapped_nents; + + enum dma_data_direction src_dir, dst_dir; + + struct device *tx_dev, *rx_dev; + struct dma_async_tx_descriptor *desc_in, *desc_out; + + int ret; + + void __iomem *aes_base_reg =3D dev_data->regs + DTHE_P_AES_BASE; + + u32 aes_irqenable_val =3D readl_relaxed(aes_base_reg + DTHE_P_AES_IRQENAB= LE); + u32 aes_sysconfig_val =3D readl_relaxed(aes_base_reg + DTHE_P_AES_SYSCONF= IG); + + aes_sysconfig_val |=3D DTHE_AES_SYSCONFIG_DMA_DATA_IN_OUT_EN; + writel_relaxed(aes_sysconfig_val, aes_base_reg + DTHE_P_AES_SYSCONFIG); + + aes_irqenable_val |=3D DTHE_AES_IRQENABLE_EN_ALL; + writel_relaxed(aes_irqenable_val, aes_base_reg + DTHE_P_AES_IRQENABLE); + + /* In decryption, the last authsize bytes are the TAG */ + if (!rctx->enc) + cryptlen -=3D authsize; + unpadded_cryptlen =3D cryptlen; + + /* Prep src and dst scatterlists */ + src =3D dthe_aead_prep_src(req->src, req->assoclen, cryptlen); + if (!src) { + ret =3D -ENOMEM; + goto aead_err; + } + + if (cryptlen !=3D 0) { + dst =3D dthe_aead_prep_dst(req->dst, req->assoclen, cryptlen); + if (!dst) { + ret =3D -ENOMEM; + goto aead_prep_dst_err; + } + } + + if (req->assoclen % AES_BLOCK_SIZE) + assoclen +=3D AES_BLOCK_SIZE - (req->assoclen % AES_BLOCK_SIZE); + if (cryptlen % AES_BLOCK_SIZE) + cryptlen +=3D AES_BLOCK_SIZE - (cryptlen % AES_BLOCK_SIZE); + + src_nents =3D sg_nents_for_len(src, assoclen + cryptlen); + dst_nents =3D sg_nents_for_len(dst, cryptlen); + /* Prep finished */ + + src_dir =3D DMA_TO_DEVICE; + dst_dir =3D DMA_FROM_DEVICE; + + tx_dev =3D dmaengine_get_dma_device(dev_data->dma_aes_tx); + rx_dev =3D dmaengine_get_dma_device(dev_data->dma_aes_rx); + + src_mapped_nents =3D dma_map_sg(tx_dev, src, src_nents, src_dir); + if (src_mapped_nents =3D=3D 0) { + ret =3D -EINVAL; + goto aead_dma_map_src_err; + } + + desc_out =3D dmaengine_prep_slave_sg(dev_data->dma_aes_tx, src, src_mappe= d_nents, + DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!desc_out) { + ret =3D -EINVAL; + goto aead_dma_prep_src_err; + } + + desc_out->callback =3D dthe_aead_dma_in_callback; + desc_out->callback_param =3D req; + + if (cryptlen !=3D 0) { + dst_mapped_nents =3D dma_map_sg(rx_dev, dst, dst_nents, dst_dir); + if (dst_mapped_nents =3D=3D 0) { + ret =3D -EINVAL; + goto aead_dma_prep_src_err; + } + + desc_in =3D dmaengine_prep_slave_sg(dev_data->dma_aes_rx, dst, + dst_mapped_nents, DMA_DEV_TO_MEM, + DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!desc_in) { + ret =3D -EINVAL; + goto aead_dma_prep_dst_err; + } + } + + init_completion(&rctx->aes_compl); + + /* + * HACK: There is an unknown hw issue where if the previous operation had= alen =3D 0 and + * plen !=3D 0, the current operation's tag calculation is incorrect in t= he case where + * plen =3D 0 and alen !=3D 0 currently. This is a workaround for now whi= ch somwhow works; + * by resetting the context by writing a 1 to the C_LENGTH_0 and AUTH_LEN= GTH registers. + */ + if (cryptlen =3D=3D 0) { + writel_relaxed(1, aes_base_reg + DTHE_P_AES_C_LENGTH_0); + writel_relaxed(1, aes_base_reg + DTHE_P_AES_AUTH_LENGTH); + } + + u32 iv_in[AES_IV_WORDS]; + + if (req->iv) { + memcpy(iv_in, req->iv, GCM_AES_IV_SIZE); + } else { + iv_in[0] =3D 0; + iv_in[1] =3D 0; + iv_in[2] =3D 0; + } + iv_in[3] =3D 0x01000000; + + /* Clear key2 to reset previous GHASH intermediate data */ + for (int i =3D 0; i < AES_KEYSIZE_256 / sizeof(u32); ++i) + writel_relaxed(0, aes_base_reg + DTHE_P_AES_KEY2_6 + DTHE_REG_SIZE * i); + + dthe_aes_set_ctrl_key(ctx, rctx, iv_in); + + writel_relaxed(lower_32_bits(unpadded_cryptlen), aes_base_reg + DTHE_P_AE= S_C_LENGTH_0); + writel_relaxed(upper_32_bits(unpadded_cryptlen), aes_base_reg + DTHE_P_AE= S_C_LENGTH_1); + writel_relaxed(req->assoclen, aes_base_reg + DTHE_P_AES_AUTH_LENGTH); + + if (cryptlen !=3D 0) + dmaengine_submit(desc_in); + dmaengine_submit(desc_out); + + if (cryptlen !=3D 0) + dma_async_issue_pending(dev_data->dma_aes_rx); + dma_async_issue_pending(dev_data->dma_aes_tx); + + /* Need to do timeout to ensure finalise gets called if DMA callback fail= s for any reason */ + ret =3D wait_for_completion_timeout(&rctx->aes_compl, msecs_to_jiffies(DT= HE_DMA_TIMEOUT_MS)); + if (!ret) { + ret =3D -ETIMEDOUT; + if (cryptlen !=3D 0) + dmaengine_terminate_sync(dev_data->dma_aes_rx); + dmaengine_terminate_sync(dev_data->dma_aes_tx); + + for (int i =3D 0; i < AES_BLOCK_SIZE / sizeof(int); ++i) + readl_relaxed(aes_base_reg + DTHE_P_AES_DATA_IN_OUT + DTHE_REG_SIZE * i= ); + } else { + ret =3D 0; + } + + if (cryptlen !=3D 0) + dma_sync_sg_for_cpu(rx_dev, dst, dst_nents, dst_dir); + if (rctx->enc) + ret =3D dthe_aead_enc_get_tag(req); + else + ret =3D dthe_aead_dec_verify_tag(req); + +aead_dma_prep_dst_err: + if (cryptlen !=3D 0) + dma_unmap_sg(rx_dev, dst, dst_nents, dst_dir); +aead_dma_prep_src_err: + dma_unmap_sg(tx_dev, src, src_nents, src_dir); + +aead_dma_map_src_err: + if (unpadded_cryptlen % AES_BLOCK_SIZE && cryptlen !=3D 0) + kfree(sg_virt(&dst[dst_nents - 1])); + + if (cryptlen !=3D 0) + kfree(dst); + +aead_prep_dst_err: + if (req->assoclen % AES_BLOCK_SIZE) { + int assoc_nents =3D sg_nents_for_len(src, req->assoclen); + + kfree(sg_virt(&src[assoc_nents])); + } + if (unpadded_cryptlen % AES_BLOCK_SIZE) + kfree(sg_virt(&src[src_nents - 1])); + + kfree(src); + +aead_err: + local_bh_disable(); + crypto_finalize_aead_request(dev_data->engine, req, ret); + local_bh_enable(); + return ret; +} + +static int dthe_aead_crypt(struct aead_request *req) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + struct crypto_engine *engine; + unsigned int cryptlen =3D req->cryptlen; + + /* In decryption, last authsize bytes are the TAG */ + if (!rctx->enc) + cryptlen -=3D ctx->authsize; + + /* + * Need to fallback to software in the following cases due to HW restrict= ions: + * - Both AAD and plaintext/ciphertext are zero length + * - AAD length is more than 2^32 - 1 bytes + * PS: req->cryptlen is currently unsigned int type, which causes the abo= ve condition + * tautologically false. If req->cryptlen were to be changed to a 64-bit = type, + * the check for this would need to be added below. + */ + if (req->assoclen =3D=3D 0 && cryptlen =3D=3D 0) { + struct aead_request *subreq =3D &rctx->fb_req; + int ret; + + aead_request_set_tfm(subreq, ctx->aead_fb); + aead_request_set_callback(subreq, req->base.flags, + req->base.complete, req->base.data); + aead_request_set_crypt(subreq, req->src, req->dst, + req->cryptlen, req->iv); + aead_request_set_ad(subreq, req->assoclen); + + ret =3D rctx->enc ? crypto_aead_encrypt(subreq) : + crypto_aead_decrypt(subreq); + + return ret; + } + + engine =3D dev_data->engine; + return crypto_transfer_aead_request_to_engine(engine, req); +} + +static int dthe_aead_encrypt(struct aead_request *req) +{ + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + + rctx->enc =3D 1; + return dthe_aead_crypt(req); +} + +static int dthe_aead_decrypt(struct aead_request *req) +{ + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + + rctx->enc =3D 0; + return dthe_aead_crypt(req); +} + static struct skcipher_engine_alg cipher_algs[] =3D { { .base.init =3D dthe_cipher_init_tfm, @@ -541,12 +1106,46 @@ static struct skcipher_engine_alg cipher_algs[] =3D { }, /* XTS AES */ }; =20 +static struct aead_engine_alg aead_algs[] =3D { + { + .base.init =3D dthe_aead_init_tfm, + .base.exit =3D dthe_aead_exit_tfm, + .base.setkey =3D dthe_aead_setkey, + .base.setauthsize =3D dthe_aead_setauthsize, + .base.maxauthsize =3D AES_BLOCK_SIZE, + .base.encrypt =3D dthe_aead_encrypt, + .base.decrypt =3D dthe_aead_decrypt, + .base.chunksize =3D AES_BLOCK_SIZE, + .base.ivsize =3D GCM_AES_IV_SIZE, + .base.base =3D { + .cra_name =3D "gcm(aes)", + .cra_driver_name =3D "gcm-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_AEAD | + CRYPTO_ALG_KERN_DRIVER_ONLY | + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D 1, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aead_run, + }, /* GCM AES */ +}; + int dthe_register_aes_algs(void) { - return crypto_engine_register_skciphers(cipher_algs, ARRAY_SIZE(cipher_al= gs)); + int ret =3D 0; + + ret |=3D crypto_engine_register_skciphers(cipher_algs, ARRAY_SIZE(cipher_= algs)); + ret |=3D crypto_engine_register_aeads(aead_algs, ARRAY_SIZE(aead_algs)); + + return ret; } =20 void dthe_unregister_aes_algs(void) { crypto_engine_unregister_skciphers(cipher_algs, ARRAY_SIZE(cipher_algs)); + crypto_engine_unregister_aeads(aead_algs, ARRAY_SIZE(aead_algs)); } diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index 629f45a3ab33..3c9fe0633cca 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -38,6 +38,7 @@ enum dthe_aes_mode { DTHE_AES_CBC, DTHE_AES_CTR, DTHE_AES_XTS, + DTHE_AES_GCM, }; =20 /* Driver specific struct definitions */ @@ -78,14 +79,18 @@ struct dthe_list { * struct dthe_tfm_ctx - Transform ctx struct containing ctx for all sub-c= omponents of DTHE V2 * @dev_data: Device data struct pointer * @keylen: AES key length + * @authsize: Authentication size for modes with authentication * @key: AES key * @aes_mode: AES mode + * @aead_fb: Fallback crypto aead instance for GCM mode */ struct dthe_tfm_ctx { struct dthe_data *dev_data; unsigned int keylen; + unsigned int authsize; u32 key[DTHE_MAX_KEYSIZE / sizeof(u32)]; enum dthe_aes_mode aes_mode; + struct crypto_aead *aead_fb; }; =20 /** @@ -96,6 +101,7 @@ struct dthe_tfm_ctx { struct dthe_aes_req_ctx { int enc; struct completion aes_compl; + struct aead_request fb_req; }; =20 /* Struct definitions end */ --=20 2.43.0 From nobody Wed Sep 10 01:36:57 2025 Received: from lelvem-ot01.ext.ti.com (lelvem-ot01.ext.ti.com [198.47.23.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB9BC30DD02; Mon, 8 Sep 2025 14:10:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.23.234 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340617; cv=none; b=pona/HU2S3DcWxKPgY4Bah+iOgdSjs6Am2bLalLfa3plc2N23dMBeMb9YX77/mrJF0ZTxdsHTLEsu4+/jkByvq44t04vmha79crTmcK+aGRXJHHTCv2ags+2Cxkgmokmf2tvunBzKkIKNDhhTu8YD6dypDck5PXquz7VHdZJq8Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757340617; c=relaxed/simple; bh=5uu5bf0RrsyQKM5y8sWj/85RtaGVjMpoh+8kS3xCVcM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nJmaDKMkUq7f9ejc4D2oZL/jbcUuCMd0UQI2w1tsgGBnxCFD7NG3x5fzL6jtbi/lY8ppmoUrdR9FD/8IOEoQLN9RzIJ9BlFnYk3Rg3duVu/A7ul37XA96gYtbklFcTZWw334Gd7tW9bQzQBoqORBAcUMvwhv0Q70qNBQ8wkzwwA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=nEFzYULN; arc=none smtp.client-ip=198.47.23.234 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="nEFzYULN" Received: from fllvem-sh04.itg.ti.com ([10.64.41.54]) by lelvem-ot01.ext.ti.com (8.15.2/8.15.2) with ESMTP id 588EACVs3879954; Mon, 8 Sep 2025 09:10:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1757340612; bh=9/jgYGag3gWxaLuoifDeLx1geIiL+wOeYcjyrsfrvTg=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=nEFzYULNULcR7/GL8RLvm7kqy/uorFaptoeIEXFOJSiBH7mXBLTNEeGHF/4MOBi+9 am870naAm6x032PfmBEwyTTSqSS+IY1kcQQpE5RtXn8M3Nh1DOjva/joXTVBDPVAKV 9y0j5x9ZzH+LMcUP+TDW/T4BxjyvFvEk3jdHo0TI= Received: from DLEE101.ent.ti.com (dlee101.ent.ti.com [157.170.170.31]) by fllvem-sh04.itg.ti.com (8.18.1/8.18.1) with ESMTPS id 588EACDJ3823081 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA256 bits=128 verify=FAIL); Mon, 8 Sep 2025 09:10:12 -0500 Received: from DLEE114.ent.ti.com (157.170.170.25) by DLEE101.ent.ti.com (157.170.170.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55; Mon, 8 Sep 2025 09:10:11 -0500 Received: from lelvem-mr06.itg.ti.com (10.180.75.8) by DLEE114.ent.ti.com (157.170.170.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.55 via Frontend Transport; Mon, 8 Sep 2025 09:10:11 -0500 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr06.itg.ti.com (8.18.1/8.18.1) with ESMTP id 588EAAQT719446; Mon, 8 Sep 2025 09:10:10 -0500 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Kamlesh Gurudasani , Manorit Chawdhry , Vignesh Raghavendra , Praneeth Bajjuri , Vishal Mahaveer , Kavitha Malarvizhi , , Subject: [PATCH v2 4/4] crypto: ti: Add support for AES-CCM in DTHEv2 driver Date: Mon, 8 Sep 2025 19:28:16 +0530 Message-ID: <20250908140928.2801062-5-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250908140928.2801062-1-t-pratham@ti.com> References: <20250908140928.2801062-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea Content-Type: text/plain; charset="utf-8" AES-CCM is an AEAD algorithm supporting both encryption and authentication of data. This patch introduces support for AES-CCM AEAD algorithm in the DTHEv2 driver. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 1 + drivers/crypto/ti/dthev2-aes.c | 96 +++++++++++++++++++++++++------ drivers/crypto/ti/dthev2-common.h | 1 + 3 files changed, 82 insertions(+), 16 deletions(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index e1ef84b39267..02eff4f0e04b 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -9,6 +9,7 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_CTR select CRYPTO_XTS select CRYPTO_GCM + select CRYPTO_CCM select SG_SPLIT help This enables support for the TI DTHE V2 hw cryptography engine diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index 37bc43a1ba73..47127734f587 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -68,6 +68,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_CTR_MASK =3D BIT(6), AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), AES_CTRL_GCM_MASK =3D BIT(17) | BIT(16) | BIT(6), + AES_CTRL_CCM_MASK =3D BIT(18) | BIT(6), }; =20 #define DTHE_AES_CTRL_MODE_CLEAR_MASK ~GENMASK(28, 5) @@ -80,6 +81,10 @@ enum aes_ctrl_mode_masks { =20 #define DTHE_AES_CTRL_CTR_WIDTH_128B (BIT(7) | BIT(8)) =20 +#define DTHE_AES_CCM_L_FROM_IV_MASK GENMASK(2, 0) +#define DTHE_AES_CTRL_CCM_L_MASK GENMASK(21, 19) +#define DTHE_AES_CTRL_CCM_M_MAXVAL GENMASK(24, 22) + #define DTHE_AES_CTRL_SAVE_CTX_SET BIT(29) =20 #define DTHE_AES_CTRL_OUTPUT_READY BIT_MASK(0) @@ -95,6 +100,8 @@ enum aes_ctrl_mode_masks { #define AES_BLOCK_WORDS (AES_BLOCK_SIZE / sizeof(u32)) #define AES_IV_WORDS AES_BLOCK_WORDS #define DTHE_AES_GCM_AAD_MAXLEN (BIT_ULL(32) - 1) +#define DTHE_AES_CCM_AAD_MAXLEN (BIT(16) - BIT(8)) +#define DTHE_AES_CCM_CRYPT_MAXLEN (BIT_ULL(61) - 1) #define POLL_TIMEOUT_INTERVAL HZ =20 static int dthe_poll_reg(struct dthe_data *dev_data, u32 reg, u32 bit) @@ -246,6 +253,12 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx = *ctx, case DTHE_AES_GCM: ctrl_val |=3D AES_CTRL_GCM_MASK; break; + case DTHE_AES_CCM: + ctrl_val |=3D AES_CTRL_CCM_MASK; + ctrl_val |=3D FIELD_PREP(DTHE_AES_CTRL_CCM_L_MASK, + (iv_in[0] & DTHE_AES_CCM_L_FROM_IV_MASK)); + ctrl_val |=3D DTHE_AES_CTRL_CCM_M_MAXVAL; + break; } =20 if (iv_in) { @@ -732,10 +745,6 @@ static int dthe_aead_setkey(struct crypto_aead *tfm, c= onst u8 *key, unsigned int if (keylen !=3D AES_KEYSIZE_128 && keylen !=3D AES_KEYSIZE_192 && keylen = !=3D AES_KEYSIZE_256) return -EINVAL; =20 - ctx->aes_mode =3D DTHE_AES_GCM; - ctx->keylen =3D keylen; - memcpy(ctx->key, key, keylen); - crypto_aead_clear_flags(ctx->aead_fb, CRYPTO_TFM_REQ_MASK); crypto_aead_set_flags(ctx->aead_fb, crypto_aead_get_flags(tfm) & @@ -744,6 +753,28 @@ static int dthe_aead_setkey(struct crypto_aead *tfm, c= onst u8 *key, unsigned int return crypto_aead_setkey(ctx->aead_fb, key, keylen); } =20 +static int dthe_gcm_aes_setkey(struct crypto_aead *tfm, const u8 *key, uns= igned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + ctx->aes_mode =3D DTHE_AES_GCM; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + return dthe_aead_setkey(tfm, key, keylen); +} + +static int dthe_ccm_aes_setkey(struct crypto_aead *tfm, const u8 *key, uns= igned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + ctx->aes_mode =3D DTHE_AES_CCM; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + return dthe_aead_setkey(tfm, key, keylen); +} + static int dthe_aead_setauthsize(struct crypto_aead *tfm, unsigned int aut= hsize) { struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); @@ -880,14 +911,18 @@ static int dthe_aead_run(struct crypto_engine *engine= , void *areq) =20 u32 iv_in[AES_IV_WORDS]; =20 - if (req->iv) { - memcpy(iv_in, req->iv, GCM_AES_IV_SIZE); + if (ctx->aes_mode =3D=3D DTHE_AES_GCM) { + if (req->iv) { + memcpy(iv_in, req->iv, GCM_AES_IV_SIZE); + } else { + iv_in[0] =3D 0; + iv_in[1] =3D 0; + iv_in[2] =3D 0; + } + iv_in[3] =3D 0x01000000; } else { - iv_in[0] =3D 0; - iv_in[1] =3D 0; - iv_in[2] =3D 0; + memcpy(iv_in, req->iv, AES_IV_SIZE); } - iv_in[3] =3D 0x01000000; =20 /* Clear key2 to reset previous GHASH intermediate data */ for (int i =3D 0; i < AES_KEYSIZE_256 / sizeof(u32); ++i) @@ -974,12 +1009,16 @@ static int dthe_aead_crypt(struct aead_request *req) /* * Need to fallback to software in the following cases due to HW restrict= ions: * - Both AAD and plaintext/ciphertext are zero length - * - AAD length is more than 2^32 - 1 bytes - * PS: req->cryptlen is currently unsigned int type, which causes the abo= ve condition - * tautologically false. If req->cryptlen were to be changed to a 64-bit = type, - * the check for this would need to be added below. + * - For AES-GCM, AAD length is more than 2^32 - 1 bytes + * - For AES-CCM, AAD length is more than 2^16 - 2^8 bytes + * - For AES-CCM, plaintext/ciphertext length is more than 2^61 - 1 bytes + * + * PS: req->cryptlen is currently unsigned int type, which causes the sec= ond and fourth + * cases above tautologically false. If req->cryptlen is to be changed to= a 64-bit + * type, the check for these would also need to be added below. */ - if (req->assoclen =3D=3D 0 && cryptlen =3D=3D 0) { + if ((req->assoclen =3D=3D 0 && cryptlen =3D=3D 0) || + (ctx->aes_mode =3D=3D DTHE_AES_CCM && req->assoclen > DTHE_AES_CCM_AA= D_MAXLEN)) { struct aead_request *subreq =3D &rctx->fb_req; int ret; =20 @@ -1110,7 +1149,7 @@ static struct aead_engine_alg aead_algs[] =3D { { .base.init =3D dthe_aead_init_tfm, .base.exit =3D dthe_aead_exit_tfm, - .base.setkey =3D dthe_aead_setkey, + .base.setkey =3D dthe_gcm_aes_setkey, .base.setauthsize =3D dthe_aead_setauthsize, .base.maxauthsize =3D AES_BLOCK_SIZE, .base.encrypt =3D dthe_aead_encrypt, @@ -1132,6 +1171,31 @@ static struct aead_engine_alg aead_algs[] =3D { }, .op.do_one_request =3D dthe_aead_run, }, /* GCM AES */ + { + .base.init =3D dthe_aead_init_tfm, + .base.exit =3D dthe_aead_exit_tfm, + .base.setkey =3D dthe_ccm_aes_setkey, + .base.setauthsize =3D dthe_aead_setauthsize, + .base.maxauthsize =3D AES_BLOCK_SIZE, + .base.encrypt =3D dthe_aead_encrypt, + .base.decrypt =3D dthe_aead_decrypt, + .base.chunksize =3D AES_BLOCK_SIZE, + .base.ivsize =3D AES_IV_SIZE, + .base.base =3D { + .cra_name =3D "ccm(aes)", + .cra_driver_name =3D "ccm-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_AEAD | + CRYPTO_ALG_KERN_DRIVER_ONLY | + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D 1, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aead_run, + }, /* CCM AES */ }; =20 int dthe_register_aes_algs(void) diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index 3c9fe0633cca..3b3b133fc984 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -39,6 +39,7 @@ enum dthe_aes_mode { DTHE_AES_CTR, DTHE_AES_XTS, DTHE_AES_GCM, + DTHE_AES_CCM, }; =20 /* Driver specific struct definitions */ --=20 2.43.0