From nobody Sun Feb 8 02:55:53 2026 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010032.outbound.protection.outlook.com [52.101.56.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACA293502A6; Tue, 11 Nov 2025 11:22:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860126; cv=fail; b=k5615MELB1VAKLMoqXX2pGMQDF3igIStlEvKvOreP5SGetmSdU14ajTpf4wLVHCf9gxK7X9mxsRkk/3rlIvD1jKbe9/NO77RTFPMwfixzN2604VrLkYyvpALL7PraPduuw06e6Tn3NgQ/W5dVZ7yinKzMTppOBz5/wYdszkt7pY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860126; c=relaxed/simple; bh=BnRhr1yAKF8YTfF2TBoTKGiFYPSnPANfyQxHfAFXBic=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DdDV7A4n/dKOlAr7+f00fyag8iQbZoZ4D51lmgq5ufXyBhK93PSP6/COVARbwaG3ZKhaoAbz2sAtrmUnNYB4+XwRWuI01pvlmrTI0rVIXVERPrDDDDtEfyv7LzPiO5Up8JuvWWQdFFAh4GG3UEUG2X/msVHMAwsG6U8xuRxBYGM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=G+T/GfFd; arc=fail smtp.client-ip=52.101.56.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="G+T/GfFd" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q0FN/gK6XYcvwXoNIPEXEjlkgnZFCc8EbzBE+gY/4yIXbBUCLFfxzOWc6MQDyL604MJPeNrO9faVZShKFWg+VvOp9dLV7cP4D6jVE2nyrfFXpnWIfAQNytmSrC1ZG0vSfIld/AoYb+Cqkl13bxfknqhNGncjoWGhiIqak8k3hTBIYzuUrxJ964Z8OHk16RmYjNBo3F9LFzNK4BFlK0vngfPxOPBimr2GY5HfpGZr8Yz9lQ2xBhcsYOr55/0BlOh/MTo43fMbYC1Wso7546ozbuF2MuQO90eFsLtl1fLgfty9V7nO2bsv+D3MEg4w1IqAG+u+Mdq6/TjyUnAoXfWzTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sk9ntpMnsnJzphDT4iZPd+9W3IQFv2EGnTbW0Awtq2Y=; b=n6AMv5DRLpun7N9uRPShURtkNFaO74EfLAGVKEFYQR+dYiO9hatabXf34YPEC7bH4QNa0soMtTLxZrF9mXSVfAXSQCRZW8J2CVXdNDWc4FIb4s/pZK950gsnPAQ7xd5JMHGWOMkt4OIG2nDodQCWquhEQislomwmsCGhvy6hMfpi57iAul447HHKA67e/XuyDdKbwXf4FBCuHE9bdGsYTe1cXV4WZlnELpZAUba2ooph3TxNB9rhz1RGbe96WFkNaecHGqI+7WnGJODhTh755h9Z2GOfSoHr0FKnVJraoWDX0Vhhgz1jWrXfKl9qJ87h8pAFL0oRG2Ft8yZtelv0zw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 198.47.23.195) smtp.rcpttodomain=gondor.apana.org.au smtp.mailfrom=ti.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=ti.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sk9ntpMnsnJzphDT4iZPd+9W3IQFv2EGnTbW0Awtq2Y=; b=G+T/GfFdoKtUSdj/VA5oI6Z1t7Ri6aM7Hu/IEDUmk3aDBGUN/onSKjuF55VdZOH8P2qQYYZh60DLLL4D3IL6zWfM4pjw29nH0t1F+XLfTL6KHzeXMg5WjyyBtHSeh/51a/8kIT7UkGaKeP+Cuql4iCYn95KEaiVLpAE8yFNej10= Received: from SJ0PR13CA0027.namprd13.prod.outlook.com (2603:10b6:a03:2c0::32) by IA4PR10MB8753.namprd10.prod.outlook.com (2603:10b6:208:55c::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Tue, 11 Nov 2025 11:22:00 +0000 Received: from SJ1PEPF00002315.namprd03.prod.outlook.com (2603:10b6:a03:2c0:cafe::64) by SJ0PR13CA0027.outlook.office365.com (2603:10b6:a03:2c0::32) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9320.15 via Frontend Transport; Tue, 11 Nov 2025 11:21:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 198.47.23.195) smtp.mailfrom=ti.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ti.com; Received-SPF: Pass (protection.outlook.com: domain of ti.com designates 198.47.23.195 as permitted sender) receiver=protection.outlook.com; client-ip=198.47.23.195; helo=lewvzet201.ext.ti.com; pr=C Received: from lewvzet201.ext.ti.com (198.47.23.195) by SJ1PEPF00002315.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9320.13 via Frontend Transport; Tue, 11 Nov 2025 11:21:59 +0000 Received: from DLEE207.ent.ti.com (157.170.170.95) by lewvzet201.ext.ti.com (10.4.14.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:21:54 -0600 Received: from DLEE210.ent.ti.com (157.170.170.112) by DLEE207.ent.ti.com (157.170.170.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:21:53 -0600 Received: from lelvem-mr05.itg.ti.com (10.180.75.9) by DLEE210.ent.ti.com (157.170.170.112) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 11 Nov 2025 05:21:53 -0600 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr05.itg.ti.com (8.18.1/8.18.1) with ESMTP id 5ABBLq3i828340; Tue, 11 Nov 2025 05:21:53 -0600 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Manorit Chawdhry , Kamlesh Gurudasani , Shiva Tripathi , Kavitha Malarvizhi , Vishal Mahaveer , , Subject: [PATCH v6 1/4] crypto: ti - Add support for AES-XTS in DTHEv2 driver Date: Tue, 11 Nov 2025 16:38:30 +0530 Message-ID: <20251111112137.976121-2-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251111112137.976121-1-t-pratham@ti.com> References: <20251111112137.976121-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002315:EE_|IA4PR10MB8753:EE_ X-MS-Office365-Filtering-Correlation-Id: 149e7ea3-88df-4ce6-78ef-08de211487ea X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?F9QdfNhByGbUJOievzYSl5PtChbZsLV0fBrZXtr5Oc6KEC0l8vfnW+A7dxHd?= =?us-ascii?Q?61tToExssCatuNM2aTAB1ZpspNcmczkovZ06tzCLOlN5vcXY+BdLztpv6tDZ?= =?us-ascii?Q?7QbU4gHNORdwvq8ZaEjSIpHo0rXVgE095JKKtkLHP/Jyea852SjZromT3UwB?= =?us-ascii?Q?YY42Hzy9EoUjKQK7rgN6CORab3PHPaTQ+WAox9OORiGzHGY8ACovw30hRxCs?= =?us-ascii?Q?mMFxOpRO7YYOvKI1NAl1yvRR7g5/BXcrMHM/GkugFdzL57Y/OFUIz5YhL5ky?= =?us-ascii?Q?ADtADSnYKQgdNAo82UDr93kKkq+ozmqN0aD33ivbdODHRYJt5P/FjVVANZfh?= =?us-ascii?Q?otVQjDeiBnMQto4nAXMdYowKpvNC4GgqU5LfDdwbQUTWUx01miY3ZPkMkU/k?= =?us-ascii?Q?qUxiGcx34JnbMBqics4OpDursZqAuSlWv9EVSBdsymCdeeBrAPitLNVNUkZc?= =?us-ascii?Q?vIsbCkRiQvIR2djDKZ81sVxYDkFphBl8kYWv4N0j6zNGHjC0I0sfJ2RbvKVp?= =?us-ascii?Q?aWHVExs1izT8vpGL2ACeT+VSFBA766eyn8BFwWlRyXHSmqV/4JU0qnhhKO4b?= =?us-ascii?Q?VPVbotqIIciqbug5oNMwX8y1tYZWWRE9B1O4O2wYc6TJ6GtaWpmTDfibgqIj?= =?us-ascii?Q?Bewd04zcanfJdi1oZ6cL7u2/v0d0GPI7UI/5OOsOfUqrWFDaSAiZTEqq/l+N?= =?us-ascii?Q?v9ehzVaIX/oUWF3D1VvqMEjv3MHYwPxdukKID/Nox9kXtX1JNH/0LqLXF9RP?= =?us-ascii?Q?DzrN8dAY3yfAqVYc5pSMsWgH3nA0Wsv4E30EqN1iABJ+f7EpDE3ANDmFoseB?= =?us-ascii?Q?3n2tdrdjxoEsqdkuIn5x3s8AQdCROUgpGL4jMUAprCRpq2euPeSn9UGs6vYw?= =?us-ascii?Q?Sl5zTMzYC05cX3rn0zRMhWMVyROtjychek5d3s9Yd9K0NkB1CbhGkOIlTVlJ?= =?us-ascii?Q?K0Tdw/Yu6za2w/nSHAJkcxvRCi+gN+ysamBvq8TTZOFyX+Z7Fc6jI7rhVxlq?= =?us-ascii?Q?F5FLmQVTy5J40mG287imO3Pp1/eZsJVE7w9TQbkPJ4tV9AvC4T46UPPM8eik?= =?us-ascii?Q?Zpsei5DVvjtDnCapVyKv9byIoxWeioI7/Q1OhtfjH1g8fVeD+yZFuMQQMLhX?= =?us-ascii?Q?VYMxbVaD1wzudmaD8/xNWVRhe5JMP/HSIk8u8YxX0lqxMpJom6inHhzlxXk5?= =?us-ascii?Q?pFhAHGRxS6ZvKALj2vfDz0yIQKqlyNgZol+vhoDtJ4bUyUUpYAfFko4iotVh?= =?us-ascii?Q?TF4vjxh785QH+YJCE9HaG1sK0NCeeB8jV1eBYyt8jRU8XyJBKM8cEcGxGF1n?= =?us-ascii?Q?bV9DVx2CQjDhES/HuLsAKR+5DXZ8qxUneZlBjSTo7liqaWAs5ZfNV0Px3Nai?= =?us-ascii?Q?0AwFSlyIAkr2QVlqbakml2DwWbdF0fquK8nxvO87+MlKtJpwWtMqtm+FgZT/?= =?us-ascii?Q?SOvOvNiCrajBCvRGAhLnDEAxM516X+G5uhYOvx/Q/UdyGoSVsd9DC4Qfndw6?= =?us-ascii?Q?cPrGjg7SQviJRI3B2aUu9O9Kqm15tY+1pLWSUIdEBkmCXLMfJTL080uqmmfY?= =?us-ascii?Q?eHGWizfQMzDvffEq3SE=3D?= X-Forefront-Antispam-Report: CIP:198.47.23.195;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:lewvzet201.ext.ti.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(82310400026)(1800799024)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: ti.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Nov 2025 11:21:59.5814 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 149e7ea3-88df-4ce6-78ef-08de211487ea X-MS-Exchange-CrossTenant-Id: e5b49634-450b-4709-8abb-1e2b19b982b7 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=e5b49634-450b-4709-8abb-1e2b19b982b7;Ip=[198.47.23.195];Helo=[lewvzet201.ext.ti.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002315.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA4PR10MB8753 Content-Type: text/plain; charset="utf-8" Add support for XTS mode of operation for AES algorithm in the AES Engine of the DTHEv2 hardware cryptographic engine. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 1 + drivers/crypto/ti/dthev2-aes.c | 137 ++++++++++++++++++++++++++++-- drivers/crypto/ti/dthev2-common.h | 10 ++- 3 files changed, 141 insertions(+), 7 deletions(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index d4f91c1e0cb55..a3692ceec49bc 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -6,6 +6,7 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_SKCIPHER select CRYPTO_ECB select CRYPTO_CBC + select CRYPTO_XTS help This enables support for the TI DTHE V2 hw cryptography engine which can be found on TI K3 SOCs. Selecting this enables use diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index 3547c41fa4ed3..7d8123984c1eb 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -25,6 +25,7 @@ =20 // AES Engine #define DTHE_P_AES_BASE 0x7000 + #define DTHE_P_AES_KEY1_0 0x0038 #define DTHE_P_AES_KEY1_1 0x003C #define DTHE_P_AES_KEY1_2 0x0030 @@ -33,6 +34,16 @@ #define DTHE_P_AES_KEY1_5 0x002C #define DTHE_P_AES_KEY1_6 0x0020 #define DTHE_P_AES_KEY1_7 0x0024 + +#define DTHE_P_AES_KEY2_0 0x0018 +#define DTHE_P_AES_KEY2_1 0x001C +#define DTHE_P_AES_KEY2_2 0x0010 +#define DTHE_P_AES_KEY2_3 0x0014 +#define DTHE_P_AES_KEY2_4 0x0008 +#define DTHE_P_AES_KEY2_5 0x000C +#define DTHE_P_AES_KEY2_6 0x0000 +#define DTHE_P_AES_KEY2_7 0x0004 + #define DTHE_P_AES_IV_IN_0 0x0040 #define DTHE_P_AES_IV_IN_1 0x0044 #define DTHE_P_AES_IV_IN_2 0x0048 @@ -52,6 +63,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_ECB_MASK =3D 0x00, AES_CTRL_CBC_MASK =3D BIT(5), + AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), }; =20 #define DTHE_AES_CTRL_MODE_CLEAR_MASK ~GENMASK(28, 5) @@ -88,6 +100,31 @@ static int dthe_cipher_init_tfm(struct crypto_skcipher = *tfm) return 0; } =20 +static int dthe_cipher_xts_init_tfm(struct crypto_skcipher *tfm) +{ + struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + + ctx->dev_data =3D dev_data; + ctx->keylen =3D 0; + + ctx->skcipher_fb =3D crypto_alloc_sync_skcipher("xts(aes)", 0, + CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->skcipher_fb)) { + dev_err(dev_data->dev, "fallback driver xts(aes) couldn't be loaded\n"); + return PTR_ERR(ctx->skcipher_fb); + } + + return 0; +} + +static void dthe_cipher_xts_exit_tfm(struct crypto_skcipher *tfm) +{ + struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); + + crypto_free_sync_skcipher(ctx->skcipher_fb); +} + static int dthe_aes_setkey(struct crypto_skcipher *tfm, const u8 *key, uns= igned int keylen) { struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); @@ -119,6 +156,27 @@ static int dthe_aes_cbc_setkey(struct crypto_skcipher = *tfm, const u8 *key, unsig return dthe_aes_setkey(tfm, key, keylen); } =20 +static int dthe_aes_xts_setkey(struct crypto_skcipher *tfm, const u8 *key,= unsigned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); + + if (keylen !=3D 2 * AES_KEYSIZE_128 && + keylen !=3D 2 * AES_KEYSIZE_192 && + keylen !=3D 2 * AES_KEYSIZE_256) + return -EINVAL; + + ctx->aes_mode =3D DTHE_AES_XTS; + ctx->keylen =3D keylen / 2; + memcpy(ctx->key, key, keylen); + + crypto_sync_skcipher_clear_flags(ctx->skcipher_fb, CRYPTO_TFM_REQ_MASK); + crypto_sync_skcipher_set_flags(ctx->skcipher_fb, + crypto_skcipher_get_flags(tfm) & + CRYPTO_TFM_REQ_MASK); + + return crypto_sync_skcipher_setkey(ctx->skcipher_fb, key, keylen); +} + static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx *ctx, struct dthe_aes_req_ctx *rctx, u32 *iv_in) @@ -141,6 +199,24 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx = *ctx, writel_relaxed(ctx->key[7], aes_base_reg + DTHE_P_AES_KEY1_7); } =20 + if (ctx->aes_mode =3D=3D DTHE_AES_XTS) { + size_t key2_offset =3D ctx->keylen / sizeof(u32); + + writel_relaxed(ctx->key[key2_offset + 0], aes_base_reg + DTHE_P_AES_KEY2= _0); + writel_relaxed(ctx->key[key2_offset + 1], aes_base_reg + DTHE_P_AES_KEY2= _1); + writel_relaxed(ctx->key[key2_offset + 2], aes_base_reg + DTHE_P_AES_KEY2= _2); + writel_relaxed(ctx->key[key2_offset + 3], aes_base_reg + DTHE_P_AES_KEY2= _3); + + if (ctx->keylen > AES_KEYSIZE_128) { + writel_relaxed(ctx->key[key2_offset + 4], aes_base_reg + DTHE_P_AES_KEY= 2_4); + writel_relaxed(ctx->key[key2_offset + 5], aes_base_reg + DTHE_P_AES_KEY= 2_5); + } + if (ctx->keylen =3D=3D AES_KEYSIZE_256) { + writel_relaxed(ctx->key[key2_offset + 6], aes_base_reg + DTHE_P_AES_KEY= 2_6); + writel_relaxed(ctx->key[key2_offset + 7], aes_base_reg + DTHE_P_AES_KEY= 2_7); + } + } + if (rctx->enc) ctrl_val |=3D DTHE_AES_CTRL_DIR_ENC; =20 @@ -160,6 +236,9 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx *= ctx, case DTHE_AES_CBC: ctrl_val |=3D AES_CTRL_CBC_MASK; break; + case DTHE_AES_XTS: + ctrl_val |=3D AES_CTRL_XTS_MASK; + break; } =20 if (iv_in) { @@ -315,24 +394,45 @@ static int dthe_aes_run(struct crypto_engine *engine,= void *areq) local_bh_disable(); crypto_finalize_skcipher_request(dev_data->engine, req, ret); local_bh_enable(); - return ret; + return 0; } =20 static int dthe_aes_crypt(struct skcipher_request *req) { struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(crypto_skcipher_reqtfm(r= eq)); + struct dthe_aes_req_ctx *rctx =3D skcipher_request_ctx(req); struct dthe_data *dev_data =3D dthe_get_dev(ctx); struct crypto_engine *engine; =20 /* - * If data is not a multiple of AES_BLOCK_SIZE, need to return -EINVAL - * If data length input is zero, no need to do any operation. + * If data is not a multiple of AES_BLOCK_SIZE: + * - need to return -EINVAL for ECB, CBC as they are block ciphers + * - need to fallback to software as H/W doesn't support Ciphertext Steal= ing for XTS */ - if (req->cryptlen % AES_BLOCK_SIZE) + if (req->cryptlen % AES_BLOCK_SIZE) { + if (ctx->aes_mode =3D=3D DTHE_AES_XTS) { + SYNC_SKCIPHER_REQUEST_ON_STACK(subreq, ctx->skcipher_fb); + + skcipher_request_set_callback(subreq, skcipher_request_flags(req), + req->base.complete, req->base.data); + skcipher_request_set_crypt(subreq, req->src, req->dst, + req->cryptlen, req->iv); + + return rctx->enc ? crypto_skcipher_encrypt(subreq) : + crypto_skcipher_decrypt(subreq); + } return -EINVAL; + } =20 - if (req->cryptlen =3D=3D 0) + /* + * If data length input is zero, no need to do any operation. + * Except for XTS mode, where data length should be non-zero. + */ + if (req->cryptlen =3D=3D 0) { + if (ctx->aes_mode =3D=3D DTHE_AES_XTS) + return -EINVAL; return 0; + } =20 engine =3D dev_data->engine; return crypto_transfer_skcipher_request_to_engine(engine, req); @@ -399,7 +499,32 @@ static struct skcipher_engine_alg cipher_algs[] =3D { .cra_module =3D THIS_MODULE, }, .op.do_one_request =3D dthe_aes_run, - } /* CBC AES */ + }, /* CBC AES */ + { + .base.init =3D dthe_cipher_xts_init_tfm, + .base.exit =3D dthe_cipher_xts_exit_tfm, + .base.setkey =3D dthe_aes_xts_setkey, + .base.encrypt =3D dthe_aes_encrypt, + .base.decrypt =3D dthe_aes_decrypt, + .base.min_keysize =3D AES_MIN_KEY_SIZE * 2, + .base.max_keysize =3D AES_MAX_KEY_SIZE * 2, + .base.ivsize =3D AES_IV_SIZE, + .base.base =3D { + .cra_name =3D "xts(aes)", + .cra_driver_name =3D "xts-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_KERN_DRIVER_ONLY | + CRYPTO_ALG_NEED_FALLBACK, + .cra_alignmask =3D AES_BLOCK_SIZE - 1, + .cra_blocksize =3D AES_BLOCK_SIZE, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aes_run, + }, /* XTS AES */ }; =20 int dthe_register_aes_algs(void) diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index 68c94acda8aaa..c7a06a4c353ff 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -27,10 +27,16 @@ =20 #define DTHE_REG_SIZE 4 #define DTHE_DMA_TIMEOUT_MS 2000 +/* + * Size of largest possible key (of all algorithms) to be stored in dthe_t= fm_ctx + * This is currently the keysize of XTS-AES-256 which is 512 bits (64 byte= s) + */ +#define DTHE_MAX_KEYSIZE (AES_MAX_KEY_SIZE * 2) =20 enum dthe_aes_mode { DTHE_AES_ECB =3D 0, DTHE_AES_CBC, + DTHE_AES_XTS, }; =20 /* Driver specific struct definitions */ @@ -73,12 +79,14 @@ struct dthe_list { * @keylen: AES key length * @key: AES key * @aes_mode: AES mode + * @skcipher_fb: Fallback crypto skcipher handle for AES-XTS mode */ struct dthe_tfm_ctx { struct dthe_data *dev_data; unsigned int keylen; - u32 key[AES_KEYSIZE_256 / sizeof(u32)]; + u32 key[DTHE_MAX_KEYSIZE / sizeof(u32)]; enum dthe_aes_mode aes_mode; + struct crypto_sync_skcipher *skcipher_fb; }; =20 /** --=20 2.43.0 From nobody Sun Feb 8 02:55:53 2026 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012055.outbound.protection.outlook.com [52.101.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD3F435581D; Tue, 11 Nov 2025 11:22:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.55 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860128; cv=fail; b=B1kBSeSkw5S1f5EF5rnM/dV1UQQqzEvqy3P6C/kornAFIe7WYJwiDuwwuQVR2GPRIIyzXTuUTiMCnExcXhHIF+FS8KIMobAX6AxaqBU0fqXrO+dgGlZ0EF2PJ+y3M1hSSWy7CQ3CMYTSFEh8AGxinxKkPlhP0QhJSbhmwvefL30= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860128; c=relaxed/simple; bh=Oq4yY7DOrd9bjjq4S/jpwSlYLvtP+T+E3kqLoxTnd08=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jQP6/faebSX0giiOvF/awFqqLYHub2LcwF8/844ZN8344m0InWD9uuzb8HR5E6Geo7ln7r6o9SLMWn8vr6czwt7fIDLOHyYVeRM6w9vqttOOUzPwMebKYb+9tlafUZNe/i5qLBLNGetu3+OWg+o4AO66Ebgpl+3qPGstbyx+EyU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=aQv03cCB; arc=fail smtp.client-ip=52.101.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="aQv03cCB" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OR+Dx2/2I1sp8QJwCLNx1Yf7bbbHUyVT5fDcYKxwITsGqCPAweUlUnzStB6prheU1Mm8DEP4pB3QMQue4gShFesatWRJVMpL49P6ANP8ItHsNJq1yCHPb97X+/wb5QxX5qXbiMPwRmr+9bbhkaM6+DciFKLqUKkJJQ5VdBdFw7wQgAJ/ZTbVzu3TKsIabvB8X+zdc0nwKNEN+ZyuVQOKeNcNh1/zivlY2XdTnSH6aTI+9sDzBnMBhhnHADU6ISrDAERoBT7/XP0G1UR/4hzZB/+5o9XTUQN/HcotyOfRQ6xhhpGV7+CuoN2NSXpJpqf1jcy6lz4eLzKBavkwEzegYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WVrc7ta9SwmUd4CLFd0QkzOJcbe7LwqGpkOEvfVvcgM=; b=MMNScpe3Vvb5Hvmfyym0/g2yhTmph2ndwM9S1sAsRtfOgX2zCG2B0/LAOT6k0AioeWucRDBQVJ1oDhx8BKXYRJLhAUc63wys2UpFqRT/jikagfUrR7G/WqEFBBqBiArxha6Nx0gfRjFaUDtOwdMsvXrdN5dqx6cfoLknnoCI1U9dgJcrQY9L/PsQPSOVDLWNEyWgkcA3I5/0d9irrvVjPVEJKPFRyHfRyZB2hW8f/bREZoOb/scbd2U0/wN1du0LI6reB2mP6EqCrRa9Umoz5bagAJhQ5qqJ1/jBWUy4/6MsEZJoAAQMWCsisqB/y4ckepJKnB+9B5AkTWa8IyBtwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 198.47.23.195) smtp.rcpttodomain=gondor.apana.org.au smtp.mailfrom=ti.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=ti.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WVrc7ta9SwmUd4CLFd0QkzOJcbe7LwqGpkOEvfVvcgM=; b=aQv03cCBx/Qjf/uo5l6JWDX/dy2liQtYgEG2RlEPXVLL34S14W0PWBl5k7oDcXPVm+pyjRj3yb/eT/E8JMj5m1vJnhIv4LahGW+CZfMj5H2JYjTz2BMqYVHUEyTrFIv+NSBTjzlIoWqYmRr42pSpNsFru3XTkNBxf0U/HzJcxZw= Received: from SJ0PR13CA0021.namprd13.prod.outlook.com (2603:10b6:a03:2c0::26) by PH7PR10MB5880.namprd10.prod.outlook.com (2603:10b6:510:127::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Tue, 11 Nov 2025 11:22:03 +0000 Received: from SJ1PEPF00002315.namprd03.prod.outlook.com (2603:10b6:a03:2c0:cafe::84) by SJ0PR13CA0021.outlook.office365.com (2603:10b6:a03:2c0::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9320.15 via Frontend Transport; Tue, 11 Nov 2025 11:21:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 198.47.23.195) smtp.mailfrom=ti.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ti.com; Received-SPF: Pass (protection.outlook.com: domain of ti.com designates 198.47.23.195 as permitted sender) receiver=protection.outlook.com; client-ip=198.47.23.195; helo=lewvzet201.ext.ti.com; pr=C Received: from lewvzet201.ext.ti.com (198.47.23.195) by SJ1PEPF00002315.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9320.13 via Frontend Transport; Tue, 11 Nov 2025 11:22:03 +0000 Received: from DLEE206.ent.ti.com (157.170.170.90) by lewvzet201.ext.ti.com (10.4.14.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:21:57 -0600 Received: from DLEE208.ent.ti.com (157.170.170.97) by DLEE206.ent.ti.com (157.170.170.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:21:57 -0600 Received: from lelvem-mr06.itg.ti.com (10.180.75.8) by DLEE208.ent.ti.com (157.170.170.97) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 11 Nov 2025 05:21:57 -0600 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr06.itg.ti.com (8.18.1/8.18.1) with ESMTP id 5ABBLtmf861380; Tue, 11 Nov 2025 05:21:56 -0600 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Manorit Chawdhry , Kamlesh Gurudasani , Shiva Tripathi , Kavitha Malarvizhi , Vishal Mahaveer , , Subject: [PATCH v6 2/4] crypto: ti - Add support for AES-CTR in DTHEv2 driver Date: Tue, 11 Nov 2025 16:38:31 +0530 Message-ID: <20251111112137.976121-3-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251111112137.976121-1-t-pratham@ti.com> References: <20251111112137.976121-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00002315:EE_|PH7PR10MB5880:EE_ X-MS-Office365-Filtering-Correlation-Id: acccc942-23d8-4e61-deac-08de21148a0e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|376014|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?nbYjojY9QLPbsUZnSy5HBAte7NnN1gm/3rFAQgL6Sjqlje7+MgLa1bcarCxC?= =?us-ascii?Q?WiUC4JZCY9c6QBg3VVZWdRec2wDnPCYlZ+NJwhGx3BkaIHJ9UhBn8ZEotGal?= =?us-ascii?Q?B0i2K0WDGOsQl02oBZBjeZdQd0Eq6N6WI3xh3VllADEgpKfkFMKVtO3gagKJ?= =?us-ascii?Q?QzHR1nfwF1sF4frDNGEPetOjvux/VoZAI2HUJsBNK8tdvGwqbnR0jKfX95HE?= =?us-ascii?Q?qP3zqP4l/x+T+ozc3c5XWCDxXFLx5uK55VQcW6k51vPfTdGJ7pCGspElqSs5?= =?us-ascii?Q?vanyz2n2CZs4F/IDDHvt6K+eYImHc1vvJUvX1B+nJqOo+WltmUqPEKBZW1gD?= =?us-ascii?Q?J6/v8sSbBRMoIDOC21CSie8DCBBrcjwH8Qk+7eVRGnWdbbz63VZrRwv9n2Zl?= =?us-ascii?Q?ZFKZxzPcXdz6tj8T1sdCgZc/7kKOY5dDSBPlqa5NvY61X+42LHAQIy0MatkU?= =?us-ascii?Q?ABhzw0i285TtVh7MAPbWRkzfHyx8g+6s0qNmbFvuZ0jhnoseJWIWwTrP6bjb?= =?us-ascii?Q?Eys+RJqKvcdLW8kZpdu7ceVa0RaXwO7WNHNrJODy8ObrSE9l6J5Psq/dPUz8?= =?us-ascii?Q?WbLvqe8pOxQ+nJwuhh/YcH0zeaqAy2mqKaGJCT+bqqiogcoJmK1J/jjwwjVg?= =?us-ascii?Q?/fDkoRyyN/nTL9cH9p6BCx/Ij2REcRxQpLQWg+uGya48QaetFffgIn11a+w/?= =?us-ascii?Q?/lOTSxkEHcxN4FI9aHHAM7ZqONGO6arYKZx3HP+XEcwkEBhm6ZPimmbkXACc?= =?us-ascii?Q?VK2hROOPT3QCZwxY3ntHnSXNDgIfYwVgod5o55AOJ8GVhMcMOJdbnYdWOCoQ?= =?us-ascii?Q?KVZqyOviFuEARuzOwYZrCXY8Ha4cElcVNGH21anRvIcAT5Jpu84iWDg+PFSI?= =?us-ascii?Q?iOtZjS2M9jTxYnBIKdE0ebEejxl2o0WDpk0vpgOtoz6m8mk6IBax6FNK90W2?= =?us-ascii?Q?9XKoueOqcL1ml5G4V37pJ/Gg6tR7mpKop9nS5GZmYDPM5MqxorPX9GGbEteM?= =?us-ascii?Q?cXlFosX+yYtiQC1/gBX62i3NP4hHtpYoC2zpxvQtkKt8DBlEF3XlbEFazb6C?= =?us-ascii?Q?yJz4ET1f50C/FQssrR5U1JWlqk6t3FBMJMgQUBBhrfSuWPL80dnXPPJjFdtI?= =?us-ascii?Q?h8PD3jA0DRNP1iWg5tvGZCsALpDSkGLJHJ6xRgLIDitD8KS/BalAB0lKXmgH?= =?us-ascii?Q?V7yTsJOWYzcRCNWhLIcquamrrv6glxqk+dqruofv5RK0tUu67DChL23xOehT?= =?us-ascii?Q?iUZyJzmOgX2/gSlkonYB49e5rwkE+ibEsaxDBZX1NqQvVbEePkunmF7yGMlB?= =?us-ascii?Q?fpO9z3aPonGPIYVEZ2tpJnVVpzjCW1rUurVnFxF5ljadTkCHAM/nJg9aCfOk?= =?us-ascii?Q?yVtxPixXIAbd5UHt7XzJtxDGpcUoz32bEiQSJp9Pi6LIZPPcWqLOaBW02aNU?= =?us-ascii?Q?FNRDVy1R+HK0buBXg0RFf6sT71km9I9f4qtTMcIxiZtIiuT8lnoipRF/3xJZ?= =?us-ascii?Q?tnKUdFnGbsZPbmcCQcL42ORPojnqTsVDS2IAFSfRPs4M3PlyDc/Wzsam0AYB?= =?us-ascii?Q?2oZW8V2qPYPSV+wJeSg=3D?= X-Forefront-Antispam-Report: CIP:198.47.23.195;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:lewvzet201.ext.ti.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(376014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: ti.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Nov 2025 11:22:03.1752 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acccc942-23d8-4e61-deac-08de21148a0e X-MS-Exchange-CrossTenant-Id: e5b49634-450b-4709-8abb-1e2b19b982b7 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=e5b49634-450b-4709-8abb-1e2b19b982b7;Ip=[198.47.23.195];Helo=[lewvzet201.ext.ti.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00002315.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB5880 Content-Type: text/plain; charset="utf-8" Add support for CTR mode of operation for AES algorithm in the AES Engine of the DTHEv2 hardware cryptographic engine. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 1 + drivers/crypto/ti/dthev2-aes.c | 139 ++++++++++++++++++++++++++++-- drivers/crypto/ti/dthev2-common.c | 19 ++++ drivers/crypto/ti/dthev2-common.h | 15 ++++ 4 files changed, 165 insertions(+), 9 deletions(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index a3692ceec49bc..6027e12de279d 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -6,6 +6,7 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_SKCIPHER select CRYPTO_ECB select CRYPTO_CBC + select CRYPTO_CTR select CRYPTO_XTS help This enables support for the TI DTHE V2 hw cryptography engine diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index 7d8123984c1eb..1f4a953c6e2a0 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -63,6 +63,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_ECB_MASK =3D 0x00, AES_CTRL_CBC_MASK =3D BIT(5), + AES_CTRL_CTR_MASK =3D BIT(6), AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), }; =20 @@ -74,6 +75,8 @@ enum aes_ctrl_mode_masks { #define DTHE_AES_CTRL_KEYSIZE_24B BIT(4) #define DTHE_AES_CTRL_KEYSIZE_32B (BIT(3) | BIT(4)) =20 +#define DTHE_AES_CTRL_CTR_WIDTH_128B (BIT(7) | BIT(8)) + #define DTHE_AES_CTRL_SAVE_CTX_SET BIT(29) =20 #define DTHE_AES_CTRL_OUTPUT_READY BIT_MASK(0) @@ -89,6 +92,46 @@ enum aes_ctrl_mode_masks { #define AES_BLOCK_WORDS (AES_BLOCK_SIZE / sizeof(u32)) #define AES_IV_WORDS AES_BLOCK_WORDS =20 +static struct scatterlist *dthe_chain_pad_sg(struct scatterlist *sg, + unsigned int nents, + struct scatterlist pad_sg[2], + u8 *pad_buf, unsigned int pad_len) +{ + struct scatterlist *sgl; + + sg_init_table(pad_sg, 2); + sgl =3D sg_last(sg, nents); + sg_set_page(&pad_sg[0], sg_page(sgl), sgl->length, sgl->offset); + sg_set_buf(&pad_sg[1], pad_buf, pad_len); + + /* First nent can't be an empty chain nent */ + if (nents =3D=3D 1) + return pad_sg; + + sg_chain(sgl, 1, pad_sg); + return sg; +} + +static void dthe_unchain_padded_sg(struct scatterlist *sg, + struct scatterlist pad_sg[2], + unsigned int nents) +{ + struct scatterlist *sgl; + unsigned int i; + + /* + * The last 2 nents are from our {src,dst}_padded sg. + * Go to the (n-3)th nent. Then the next in memory is + * the chain sg pointing to our {src,dst}_padded sg. + */ + for (i =3D 0, sgl =3D sg; i < nents - 3; ++i) + sgl =3D sg_next(sgl); + sgl++; + sgl->page_link &=3D ~SG_CHAIN; + sg_set_page(sgl, sg_page(&pad_sg[0]), pad_sg[0].length, pad_sg[0].offset); + sg_mark_end(sgl); +} + static int dthe_cipher_init_tfm(struct crypto_skcipher *tfm) { struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); @@ -156,6 +199,15 @@ static int dthe_aes_cbc_setkey(struct crypto_skcipher = *tfm, const u8 *key, unsig return dthe_aes_setkey(tfm, key, keylen); } =20 +static int dthe_aes_ctr_setkey(struct crypto_skcipher *tfm, const u8 *key,= unsigned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); + + ctx->aes_mode =3D DTHE_AES_CTR; + + return dthe_aes_setkey(tfm, key, keylen); +} + static int dthe_aes_xts_setkey(struct crypto_skcipher *tfm, const u8 *key,= unsigned int keylen) { struct dthe_tfm_ctx *ctx =3D crypto_skcipher_ctx(tfm); @@ -236,6 +288,10 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx = *ctx, case DTHE_AES_CBC: ctrl_val |=3D AES_CTRL_CBC_MASK; break; + case DTHE_AES_CTR: + ctrl_val |=3D AES_CTRL_CTR_MASK; + ctrl_val |=3D DTHE_AES_CTRL_CTR_WIDTH_128B; + break; case DTHE_AES_XTS: ctrl_val |=3D AES_CTRL_XTS_MASK; break; @@ -270,12 +326,17 @@ static int dthe_aes_run(struct crypto_engine *engine,= void *areq) struct scatterlist *src =3D req->src; struct scatterlist *dst =3D req->dst; =20 + struct scatterlist src_pad[2], dst_pad[2]; + int src_nents =3D sg_nents_for_len(src, len); - int dst_nents; + int dst_nents =3D sg_nents_for_len(dst, len); =20 int src_mapped_nents; int dst_mapped_nents; =20 + u8 pad_buf[AES_BLOCK_SIZE] =3D {0}; + int pad_len =3D 0; + bool diff_dst; enum dma_data_direction src_dir, dst_dir; =20 @@ -295,6 +356,31 @@ static int dthe_aes_run(struct crypto_engine *engine, = void *areq) aes_irqenable_val |=3D DTHE_AES_IRQENABLE_EN_ALL; writel_relaxed(aes_irqenable_val, aes_base_reg + DTHE_P_AES_IRQENABLE); =20 + if (ctx->aes_mode =3D=3D DTHE_AES_CTR) { + /* + * CTR mode can operate on any input length, but the hardware + * requires input length to be a multiple of the block size. + * We need to handle the padding in the driver. + */ + if (req->cryptlen % AES_BLOCK_SIZE) { + /* Need to create a new SG list with padding */ + pad_len =3D ALIGN(req->cryptlen, AES_BLOCK_SIZE) - req->cryptlen; + + src =3D dthe_chain_pad_sg(req->src, src_nents, src_pad, pad_buf, pad_le= n); + src_nents++; + + if (req->src =3D=3D req->dst) { + /* In-place operation, use same SG for dst */ + dst =3D src; + dst_nents =3D src_nents; + } else { + dst =3D dthe_chain_pad_sg(req->dst, dst_nents, dst_pad, + pad_buf, pad_len); + dst_nents++; + } + } + } + if (src =3D=3D dst) { diff_dst =3D false; src_dir =3D DMA_BIDIRECTIONAL; @@ -311,19 +397,16 @@ static int dthe_aes_run(struct crypto_engine *engine,= void *areq) src_mapped_nents =3D dma_map_sg(tx_dev, src, src_nents, src_dir); if (src_mapped_nents =3D=3D 0) { ret =3D -EINVAL; - goto aes_err; + goto aes_map_src_err; } =20 if (!diff_dst) { - dst_nents =3D src_nents; dst_mapped_nents =3D src_mapped_nents; } else { - dst_nents =3D sg_nents_for_len(dst, len); dst_mapped_nents =3D dma_map_sg(rx_dev, dst, dst_nents, dst_dir); if (dst_mapped_nents =3D=3D 0) { - dma_unmap_sg(tx_dev, src, src_nents, src_dir); ret =3D -EINVAL; - goto aes_err; + goto aes_map_dst_err; } } =20 @@ -386,11 +469,24 @@ static int dthe_aes_run(struct crypto_engine *engine,= void *areq) } =20 aes_prep_err: - dma_unmap_sg(tx_dev, src, src_nents, src_dir); if (dst_dir !=3D DMA_BIDIRECTIONAL) dma_unmap_sg(rx_dev, dst, dst_nents, dst_dir); +aes_map_dst_err: + dma_unmap_sg(tx_dev, src, src_nents, src_dir); + +aes_map_src_err: + if (ctx->aes_mode =3D=3D DTHE_AES_CTR && req->cryptlen % AES_BLOCK_SIZE) { + /* + * Last nent in original sglist is converted to a chain sg. + * Need to revert that to keep the original sglist intact. + */ + if (src_nents > 2) + dthe_unchain_padded_sg(req->src, src_pad, src_nents); + + if (req->src !=3D req->dst && dst_nents > 2) + dthe_unchain_padded_sg(req->dst, dst_pad, dst_nents); + } =20 -aes_err: local_bh_disable(); crypto_finalize_skcipher_request(dev_data->engine, req, ret); local_bh_enable(); @@ -408,6 +504,7 @@ static int dthe_aes_crypt(struct skcipher_request *req) * If data is not a multiple of AES_BLOCK_SIZE: * - need to return -EINVAL for ECB, CBC as they are block ciphers * - need to fallback to software as H/W doesn't support Ciphertext Steal= ing for XTS + * - do nothing for CTR */ if (req->cryptlen % AES_BLOCK_SIZE) { if (ctx->aes_mode =3D=3D DTHE_AES_XTS) { @@ -421,7 +518,8 @@ static int dthe_aes_crypt(struct skcipher_request *req) return rctx->enc ? crypto_skcipher_encrypt(subreq) : crypto_skcipher_decrypt(subreq); } - return -EINVAL; + if (ctx->aes_mode !=3D DTHE_AES_CTR) + return -EINVAL; } =20 /* @@ -500,6 +598,29 @@ static struct skcipher_engine_alg cipher_algs[] =3D { }, .op.do_one_request =3D dthe_aes_run, }, /* CBC AES */ + { + .base.init =3D dthe_cipher_init_tfm, + .base.setkey =3D dthe_aes_ctr_setkey, + .base.encrypt =3D dthe_aes_encrypt, + .base.decrypt =3D dthe_aes_decrypt, + .base.min_keysize =3D AES_MIN_KEY_SIZE, + .base.max_keysize =3D AES_MAX_KEY_SIZE, + .base.ivsize =3D AES_IV_SIZE, + .base.chunksize =3D AES_BLOCK_SIZE, + .base.base =3D { + .cra_name =3D "ctr(aes)", + .cra_driver_name =3D "ctr-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_KERN_DRIVER_ONLY, + .cra_blocksize =3D 1, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aes_run, + }, /* CTR AES */ { .base.init =3D dthe_cipher_xts_init_tfm, .base.exit =3D dthe_cipher_xts_exit_tfm, diff --git a/drivers/crypto/ti/dthev2-common.c b/drivers/crypto/ti/dthev2-c= ommon.c index c39d37933b9ee..a2ad79bec105a 100644 --- a/drivers/crypto/ti/dthev2-common.c +++ b/drivers/crypto/ti/dthev2-common.c @@ -48,6 +48,25 @@ struct dthe_data *dthe_get_dev(struct dthe_tfm_ctx *ctx) return dev_data; } =20 +struct scatterlist *dthe_copy_sg(struct scatterlist *dst, + struct scatterlist *src, + int buflen) +{ + struct scatterlist *from_sg, *to_sg; + int sglen; + + for (to_sg =3D dst, from_sg =3D src; buflen && from_sg; buflen -=3D sglen= ) { + sglen =3D from_sg->length; + if (sglen > buflen) + sglen =3D buflen; + sg_set_buf(to_sg, sg_virt(from_sg), sglen); + from_sg =3D sg_next(from_sg); + to_sg =3D sg_next(to_sg); + } + + return to_sg; +} + static int dthe_dma_init(struct dthe_data *dev_data) { int ret; diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index c7a06a4c353ff..f12b94d64e134 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -36,6 +36,7 @@ enum dthe_aes_mode { DTHE_AES_ECB =3D 0, DTHE_AES_CBC, + DTHE_AES_CTR, DTHE_AES_XTS, }; =20 @@ -103,6 +104,20 @@ struct dthe_aes_req_ctx { =20 struct dthe_data *dthe_get_dev(struct dthe_tfm_ctx *ctx); =20 +/** + * dthe_copy_sg - Copy sg entries from src to dst + * @dst: Destination sg to be filled + * @src: Source sg to be copied from + * @buflen: Number of bytes to be copied + * + * Description: + * Copy buflen bytes of data from src to dst. + * + **/ +struct scatterlist *dthe_copy_sg(struct scatterlist *dst, + struct scatterlist *src, + int buflen); + int dthe_register_aes_algs(void); void dthe_unregister_aes_algs(void); =20 --=20 2.43.0 From nobody Sun Feb 8 02:55:53 2026 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010056.outbound.protection.outlook.com [52.101.56.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B629B3559EB; Tue, 11 Nov 2025 11:22:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860130; cv=fail; b=nGWiZfRxY31sCNrXZMFl2lSfQtwcaaJXP1MTVWRDCZbYLh1KuHgxqwuv/UjhlYNjCPPGtex7/GG3mO7CkWzLJnD4s/PbxLVMHETehso45saV12rmcBzTZQK2cb2wNgaRcm5pWs0glktMLkplXjmZTWZdUcr6b101ITGvcAgVdDg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860130; c=relaxed/simple; bh=gAdux+MO5oIVboxexijWcBcT7Gz8UZ2uCW+g+B9bMro=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=d0AOC1Ut9B3vSQFnwWtABYpS5ptk9Dq+IuPzxVoEmOHgT/ucAjai1eke793QCc2Hu8hyOv26TSJtIVdyQoQaPcd9Xm4/R5BEa/AK/qJpkrCfmaa73oUr/UG3Sq1mseoLT/n6ZIQe2jTnUZZr/DVbRUkuTjVwgPdzb+ogVo+a3Xg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=jAY0JCdg; arc=fail smtp.client-ip=52.101.56.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="jAY0JCdg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=auMBIxyLyIR07Ml9r9WMRMjJaMRDdVZ9pp5G7uBiG6xNDngD35XWq2fcOJwRcIS6mFEViJxQZR9LwDD3EmwRld34LcRD4Eg9HnQSTeeo2jrc47QpPpI2ATF+hB295vzwhuP0t2kMt60Edy5k7RoYzupM9CL/iySSK4dAyxBLeMva4n7ixEsNrv6xzTKLDkmeXAuAe2QRGtTTFY+wVx4M3E0DM80pGMkx+XZ2kpIgqi79hAs831tIFNS/z5kTvywYWI/Xf9vZA2hgLBR4y0jxL9EbxGobDU39LLLkRAY7FepA6+siyq5bw2BfWhllxaQCJCUFbSojejKo0+qVkSmscA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wZYs8JMLnhobHEhiFzVBfDnKMS/7IYckffjCajtOv+0=; b=rcg9fiEJ9uRdwm42F7xUjORDQcfqynFtxy7uoxRNizqzilZAbeZ3jo949X6CooWrKGJVTUraPPuSTcWTqOwjFdbRdYNH+CTHQZdUQ9scP+01ZF0ya3f0d9dsctO5DGVHFx+8WVnEtvGXbeNqKWlfIkmzS4MqeuYuILbcWfo3/mgHGJT4uoSn1z1RurH2fYW1bR3VPcfE/yZYuFQoqAtroki/QwvjnE5XKi5yewp6uYLrRAbqN4QKyDiIe9FI/SY7NLQdfc3N/zSdvPeQen5kZew9kNdZv0lesAw+deS0YMJI2loDqn8PYiTVTz6RMbtUMU3ETXiWGBep13TKW7msag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 198.47.21.194) smtp.rcpttodomain=gondor.apana.org.au smtp.mailfrom=ti.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=ti.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wZYs8JMLnhobHEhiFzVBfDnKMS/7IYckffjCajtOv+0=; b=jAY0JCdgIRLaklFbE8sB7wZDYL9JmwALlTWN0kxoqyABpyisesQmDXFn1C6aZ/ltEZxE5SHaWVfO9cFrlmccXYTJslDxzVTOuObI5EK8Tbx5cVj9lCSoUHjoQwHz/DTPTbcTeN5MQ/blAKwYGfmt9SRygFeHuS/jErCJsVRWumQ= Received: from PH8P220CA0009.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:345::15) by BLAPR10MB4881.namprd10.prod.outlook.com (2603:10b6:208:327::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Tue, 11 Nov 2025 11:22:04 +0000 Received: from SN1PEPF000252A1.namprd05.prod.outlook.com (2603:10b6:510:345:cafe::1d) by PH8P220CA0009.outlook.office365.com (2603:10b6:510:345::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9298.16 via Frontend Transport; Tue, 11 Nov 2025 11:22:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 198.47.21.194) smtp.mailfrom=ti.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ti.com; Received-SPF: Pass (protection.outlook.com: domain of ti.com designates 198.47.21.194 as permitted sender) receiver=protection.outlook.com; client-ip=198.47.21.194; helo=flwvzet200.ext.ti.com; pr=C Received: from flwvzet200.ext.ti.com (198.47.21.194) by SN1PEPF000252A1.mail.protection.outlook.com (10.167.242.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9320.13 via Frontend Transport; Tue, 11 Nov 2025 11:22:03 +0000 Received: from DFLE215.ent.ti.com (10.64.6.73) by flwvzet200.ext.ti.com (10.248.192.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:22:00 -0600 Received: from DFLE203.ent.ti.com (10.64.6.61) by DFLE215.ent.ti.com (10.64.6.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:22:00 -0600 Received: from lelvem-mr05.itg.ti.com (10.180.75.9) by DFLE203.ent.ti.com (10.64.6.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 11 Nov 2025 05:22:00 -0600 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr05.itg.ti.com (8.18.1/8.18.1) with ESMTP id 5ABBLxUk828452; Tue, 11 Nov 2025 05:21:59 -0600 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Manorit Chawdhry , Kamlesh Gurudasani , Shiva Tripathi , Kavitha Malarvizhi , Vishal Mahaveer , , Subject: [PATCH v6 3/4] crypto: ti - Add support for AES-GCM in DTHEv2 driver Date: Tue, 11 Nov 2025 16:38:32 +0530 Message-ID: <20251111112137.976121-4-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251111112137.976121-1-t-pratham@ti.com> References: <20251111112137.976121-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF000252A1:EE_|BLAPR10MB4881:EE_ X-MS-Office365-Filtering-Correlation-Id: 2a7a789b-28a3-45b7-38ab-08de21148a0f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|36860700013|1800799024|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?lRX89cKVqQql/Yt4NChvnuvNTFzP+6iZhw29NmiDuYcsbDTt+5KPc44NvmO8?= =?us-ascii?Q?RYbFOXmd0I5iVD0CS0DixwjNoA4FE+mQtvm1gyRAkr5mHZarLS0bmSqm8NbO?= =?us-ascii?Q?J+MfmO/w9p0YHzBSOa8eG9FRiUiejHiKuqvY/aaTBzfjhOv17ErK3XKhG//l?= =?us-ascii?Q?jFLoky+6mgXaZY4zy16ur6QV4tshjexav8qxtJQlGqbfRQ5DGt/nr6LS3f+2?= =?us-ascii?Q?iwcLMB/W/HsYP2b6KMDOz6ie1A86QdCI32fFq6z7ICiHlaNwNONq5j5EiyOV?= =?us-ascii?Q?a1bZiDjCzsTQqX+I0mg8IW4beoR5eJA7H1xCuqhksv/qLPtp+YgegUFF2/9q?= =?us-ascii?Q?kUIP5cRl29LykheS8qgsRzB/b47jtBebgPW98jSuB9iGnubZ2NfeyP63KrgH?= =?us-ascii?Q?ZbJXxnyEaRyYbsnw9M1G8oFTzzLK7eB8oF7ZCq2GsuPvL6L7m9BcOPDcLcq3?= =?us-ascii?Q?WpZGLnweJLUxuCkh9xnKBebqo8CMcAuPRbQed0KGNlZx2/D/xTNX3OaSfFPh?= =?us-ascii?Q?ZQuWBB5bSSysNEHRc5WYBEFNNeXkfyH9OJ6fYco/Mbxdh+icb3aGteIDCWWi?= =?us-ascii?Q?b4nowjf3rSbksy1+ykjea2tIdIY7nuO5QKpFKemC/mn+SW0f3Te9lea0D5Kc?= =?us-ascii?Q?LgbdDEDSphCYuDCy9OyismuqzCEZkJKHkH8j/+Gm6ojjDiPPMa0rEUD1elto?= =?us-ascii?Q?A5f9NdBqOkXsRamkh2tQyKhep6vo8lze+X7Vambf6IJrDYxWpngOd7nXERg/?= =?us-ascii?Q?J/hvVTz+ROwRNZiKhNu+iU47+aHkO6RnYmTdL/tSh5g5uXDppoK/tooSOYnR?= =?us-ascii?Q?dJC2BSd4Jons/WwNLTUDo0C+hxi25G9x/LHkF8eClpVBuK/o5P+2908JvXbZ?= =?us-ascii?Q?U+CPj6Ej1j2ypLdiF1RjcgkT40KEhnBgU893bozlkBhBy8akA1wEaUJXAFdG?= =?us-ascii?Q?KgMBN+0+vVAUziSWxqkyYBaGRZuq81iMB7E8sA4LdT/NIN8xf2PlNB2QhCNe?= =?us-ascii?Q?v63OBc0dztPozxA20YFq1QNh07iPfShCIjcqAUkiVpBXTUxOEAPk8zI0zYaA?= =?us-ascii?Q?HI2a8DhBZSrKPuh59gJ7xeW1A78Em+rrfj1O/TiwmKcixHlqYFygygCuxSRg?= =?us-ascii?Q?uTzdpfCZmuBmIp/DdxTM3GnxneRupWv/MPJETzh3fq5bFvuYHUw+xIEt2OJn?= =?us-ascii?Q?SR2vbAW0z0hkWOw51QTuXusl9XX5JO0VpN/3uHlUS3XtiyzqX07qv7rhcQwu?= =?us-ascii?Q?XaXAYEnXxhepjDm0JEMAPp9B8a13kMrZ2BbC1O0I/RTnPEUWYWw58gMkTlZK?= =?us-ascii?Q?emKJdC73hHpe98q8pfk9b9Du31/NdMJ95VOEfe7mQin/gkzUmCCq+RVS2mN8?= =?us-ascii?Q?5/nsvA1JawjxMryt2IZ1Kj1ATpPx0n4QDp/C8eaSVR0GlPY3OMxlwAQOOKvS?= =?us-ascii?Q?g07irHyAi5+l7FdKoW4IKD+JAmgdnUFk8qlC/srcXEDbzjW03wbuO2jjKzSy?= =?us-ascii?Q?w4AJSakuB3qH4ygvdH2llg/enxachetYYv9DIyl5Ex+dd0UqUMddgjGLDKxQ?= =?us-ascii?Q?gPw1tYpEzBNKeJ7L45M=3D?= X-Forefront-Antispam-Report: CIP:198.47.21.194;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:flwvzet200.ext.ti.com;PTR:ErrorRetry;CAT:NONE;SFS:(13230040)(376014)(36860700013)(1800799024)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: ti.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Nov 2025 11:22:03.2046 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2a7a789b-28a3-45b7-38ab-08de21148a0f X-MS-Exchange-CrossTenant-Id: e5b49634-450b-4709-8abb-1e2b19b982b7 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=e5b49634-450b-4709-8abb-1e2b19b982b7;Ip=[198.47.21.194];Helo=[flwvzet200.ext.ti.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF000252A1.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB4881 Content-Type: text/plain; charset="utf-8" AES-GCM is an AEAD algorithm supporting both encryption and authentication of data. This patch introduces support for AES-GCM as the first AEAD algorithm supported by the DTHEv2 driver. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 2 + drivers/crypto/ti/dthev2-aes.c | 591 +++++++++++++++++++++++++++++- drivers/crypto/ti/dthev2-common.h | 9 +- 3 files changed, 600 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index 6027e12de279d..221e483737439 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -8,6 +8,8 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_CBC select CRYPTO_CTR select CRYPTO_XTS + select CRYPTO_GCM + select SG_SPLIT help This enables support for the TI DTHE V2 hw cryptography engine which can be found on TI K3 SOCs. Selecting this enables use diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index 1f4a953c6e2a0..f52e888cc7bd9 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include =20 @@ -19,6 +20,7 @@ #include #include #include +#include #include =20 /* Registers */ @@ -53,6 +55,7 @@ #define DTHE_P_AES_C_LENGTH_1 0x0058 #define DTHE_P_AES_AUTH_LENGTH 0x005C #define DTHE_P_AES_DATA_IN_OUT 0x0060 +#define DTHE_P_AES_TAG_OUT 0x0070 =20 #define DTHE_P_AES_SYSCONFIG 0x0084 #define DTHE_P_AES_IRQSTATUS 0x008C @@ -65,6 +68,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_CBC_MASK =3D BIT(5), AES_CTRL_CTR_MASK =3D BIT(6), AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), + AES_CTRL_GCM_MASK =3D BIT(17) | BIT(16) | BIT(6), }; =20 #define DTHE_AES_CTRL_MODE_CLEAR_MASK ~GENMASK(28, 5) @@ -91,6 +95,8 @@ enum aes_ctrl_mode_masks { #define AES_IV_SIZE AES_BLOCK_SIZE #define AES_BLOCK_WORDS (AES_BLOCK_SIZE / sizeof(u32)) #define AES_IV_WORDS AES_BLOCK_WORDS +#define DTHE_AES_GCM_AAD_MAXLEN (BIT_ULL(32) - 1) +#define POLL_TIMEOUT_INTERVAL HZ =20 static struct scatterlist *dthe_chain_pad_sg(struct scatterlist *sg, unsigned int nents, @@ -295,6 +301,9 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx *= ctx, case DTHE_AES_XTS: ctrl_val |=3D AES_CTRL_XTS_MASK; break; + case DTHE_AES_GCM: + ctrl_val |=3D AES_CTRL_GCM_MASK; + break; } =20 if (iv_in) { @@ -552,6 +561,552 @@ static int dthe_aes_decrypt(struct skcipher_request *= req) return dthe_aes_crypt(req); } =20 +static int dthe_aead_init_tfm(struct crypto_aead *tfm) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + + ctx->dev_data =3D dev_data; + + const char *alg_name =3D crypto_tfm_alg_name(crypto_aead_tfm(tfm)); + + ctx->aead_fb =3D crypto_alloc_sync_aead(alg_name, 0, + CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->aead_fb)) { + dev_err(dev_data->dev, "fallback driver %s couldn't be loaded\n", + alg_name); + return PTR_ERR(ctx->aead_fb); + } + + return 0; +} + +static void dthe_aead_exit_tfm(struct crypto_aead *tfm) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + crypto_free_sync_aead(ctx->aead_fb); +} + +/** + * dthe_aead_prep_src - Prepare source scatterlist for AEAD from input req= ->src + * @sg: Input req->src scatterlist + * @assoclen: Input req->assoclen + * @cryptlen: Input req->cryptlen (minus the size of TAG in decryption) + * @assoc_pad_buf: Buffer to hold AAD padding if needed + * @crypt_pad_buf: Buffer to hold ciphertext/plaintext padding if needed + * + * Description: + * For modes with authentication, DTHEv2 hardware requires the input AAD= and + * plaintext/ciphertext to be individually aligned to AES_BLOCK_SIZE. If= either is not + * aligned, it needs to be padded with zeros by the software before pass= ing the data to + * the hardware. However, linux crypto's aead_request provides the input= with AAD and + * plaintext/ciphertext contiguously appended together in a single scatt= erlist. + * + * This helper function takes the input scatterlist and splits it into s= eparate + * scatterlists for AAD and plaintext/ciphertext, ensuring each is align= ed to + * AES_BLOCK_SIZE by adding necessary padding, and then merges the align= ed scatterlists + * back into a single scatterlist for processing. + * + * Return: + * Pointer to the merged scatterlist, or ERR_PTR(error) on failure. + * The calling function needs to free the returned scatterlist when done. + **/ +static struct scatterlist *dthe_aead_prep_src(struct scatterlist *sg, + unsigned int assoclen, + unsigned int cryptlen, + u8 *assoc_pad_buf, + u8 *crypt_pad_buf) +{ + struct scatterlist *in_sg[2]; + struct scatterlist *to_sg; + struct scatterlist *src; + size_t split_sizes[2] =3D {assoclen, cryptlen}; + int out_mapped_nents[2]; + int crypt_nents =3D 0, assoc_nents =3D 0, src_nents =3D 0; + int err =3D 0; + + /* sg_split does not work properly if one of the split_sizes is 0 */ + if (cryptlen =3D=3D 0 || assoclen =3D=3D 0) { + /* + * Assigning both to sg does not matter as assoclen =3D 0 or cryptlen = =3D 0 + * being passed to dthe_copy_sg will take care to copy the sg correctly + */ + in_sg[0] =3D sg; + in_sg[1] =3D sg; + + src_nents =3D sg_nents_for_len(sg, assoclen + cryptlen); + } else { + err =3D sg_split(sg, 0, 0, 2, split_sizes, in_sg, out_mapped_nents, GFP_= ATOMIC); + if (err) + goto dthe_aead_prep_src_split_err; + assoc_nents =3D sg_nents_for_len(in_sg[0], assoclen); + crypt_nents =3D sg_nents_for_len(in_sg[1], cryptlen); + + src_nents =3D assoc_nents + crypt_nents; + } + + if (assoclen % AES_BLOCK_SIZE) + src_nents++; + if (cryptlen % AES_BLOCK_SIZE) + src_nents++; + + src =3D kmalloc_array(src_nents, sizeof(struct scatterlist), GFP_ATOMIC); + if (!src) { + err =3D -ENOMEM; + goto dthe_aead_prep_src_mem_err; + } + + sg_init_table(src, src_nents); + to_sg =3D src; + + to_sg =3D dthe_copy_sg(to_sg, in_sg[0], assoclen); + if (assoclen % AES_BLOCK_SIZE) { + unsigned int pad_len =3D AES_BLOCK_SIZE - (assoclen % AES_BLOCK_SIZE); + + sg_set_buf(to_sg, assoc_pad_buf, pad_len); + to_sg =3D sg_next(to_sg); + } + + to_sg =3D dthe_copy_sg(to_sg, in_sg[1], cryptlen); + if (cryptlen % AES_BLOCK_SIZE) { + unsigned int pad_len =3D AES_BLOCK_SIZE - (cryptlen % AES_BLOCK_SIZE); + + sg_set_buf(to_sg, crypt_pad_buf, pad_len); + to_sg =3D sg_next(to_sg); + } + +dthe_aead_prep_src_mem_err: + if (cryptlen !=3D 0 && assoclen !=3D 0) { + kfree(in_sg[0]); + kfree(in_sg[1]); + } + +dthe_aead_prep_src_split_err: + if (err) + return ERR_PTR(err); + return src; +} + +/** + * dthe_aead_prep_dst - Prepare destination scatterlist for AEAD from inpu= t req->dst + * @sg: Input req->dst scatterlist + * @assoclen: Input req->assoclen + * @cryptlen: Input req->cryptlen (minus the size of TAG in decryption) + * @pad_buf: Buffer to hold ciphertext/plaintext padding if needed + * + * Description: + * For modes with authentication, DTHEv2 hardware returns encrypted ciph= ertext/decrypted + * plaintext through DMA and TAG through MMRs. However, the dst scatterl= ist in linux + * crypto's aead_request is allocated same as input req->src scatterlist= . That is, it + * contains space for AAD in the beginning and ciphertext/plaintext at t= he end, with no + * alignment padding. This causes issues with DMA engine and DTHEv2 hard= ware. + * + * This helper function takes the output scatterlist and maps the part o= f the buffer + * which holds only the ciphertext/plaintext to a new scatterlist. It al= so adds a padding + * to align it with AES_BLOCK_SIZE. + * + * Return: + * Pointer to the trimmed scatterlist, or ERR_PTR(error) on failure. + * The calling function needs to free the returned scatterlist when done. + **/ +static struct scatterlist *dthe_aead_prep_dst(struct scatterlist *sg, + unsigned int assoclen, + unsigned int cryptlen, + u8 *pad_buf) +{ + struct scatterlist *out_sg[1]; + struct scatterlist *dst; + struct scatterlist *to_sg; + size_t split_sizes[1] =3D {cryptlen}; + int out_mapped_nents[1]; + int dst_nents =3D 0; + int err =3D 0; + + err =3D sg_split(sg, 0, assoclen, 1, split_sizes, out_sg, out_mapped_nent= s, GFP_ATOMIC); + if (err) + goto dthe_aead_prep_dst_split_err; + + dst_nents =3D sg_nents_for_len(out_sg[0], cryptlen); + if (cryptlen % AES_BLOCK_SIZE) + dst_nents++; + + dst =3D kmalloc_array(dst_nents, sizeof(struct scatterlist), GFP_ATOMIC); + if (!dst) { + err =3D -ENOMEM; + goto dthe_aead_prep_dst_mem_err; + } + sg_init_table(dst, dst_nents); + + to_sg =3D dthe_copy_sg(dst, out_sg[0], cryptlen); + if (cryptlen % AES_BLOCK_SIZE) { + unsigned int pad_len =3D AES_BLOCK_SIZE - (cryptlen % AES_BLOCK_SIZE); + + sg_set_buf(to_sg, pad_buf, pad_len); + to_sg =3D sg_next(to_sg); + } + +dthe_aead_prep_dst_mem_err: + kfree(out_sg[0]); + +dthe_aead_prep_dst_split_err: + if (err) + return ERR_PTR(err); + return dst; +} + +static int dthe_aead_read_tag(struct dthe_tfm_ctx *ctx, u32 *tag) +{ + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + void __iomem *aes_base_reg =3D dev_data->regs + DTHE_P_AES_BASE; + u32 val; + int ret; + + ret =3D readl_relaxed_poll_timeout(aes_base_reg + DTHE_P_AES_CTRL, val, + (val & DTHE_AES_CTRL_SAVED_CTX_READY), + 0, POLL_TIMEOUT_INTERVAL); + if (ret) + return ret; + + for (int i =3D 0; i < AES_BLOCK_WORDS; ++i) + tag[i] =3D readl_relaxed(aes_base_reg + + DTHE_P_AES_TAG_OUT + + DTHE_REG_SIZE * i); + return 0; +} + +static int dthe_aead_enc_get_tag(struct aead_request *req) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + u32 tag[AES_BLOCK_WORDS]; + int nents; + int ret; + + ret =3D dthe_aead_read_tag(ctx, tag); + if (ret) + return ret; + + nents =3D sg_nents_for_len(req->dst, req->cryptlen + req->assoclen + ctx-= >authsize); + + sg_pcopy_from_buffer(req->dst, nents, tag, ctx->authsize, + req->assoclen + req->cryptlen); + + return 0; +} + +static int dthe_aead_dec_verify_tag(struct aead_request *req) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + u32 tag_out[AES_BLOCK_WORDS]; + u32 tag_in[AES_BLOCK_WORDS]; + int nents; + int ret; + + ret =3D dthe_aead_read_tag(ctx, tag_out); + if (ret) + return ret; + + nents =3D sg_nents_for_len(req->src, req->assoclen + req->cryptlen); + + sg_pcopy_to_buffer(req->src, nents, tag_in, ctx->authsize, + req->assoclen + req->cryptlen - ctx->authsize); + + if (memcmp(tag_in, tag_out, ctx->authsize)) + return -EBADMSG; + else + return 0; +} + +static int dthe_aead_setkey(struct crypto_aead *tfm, const u8 *key, unsign= ed int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + if (keylen !=3D AES_KEYSIZE_128 && keylen !=3D AES_KEYSIZE_192 && keylen = !=3D AES_KEYSIZE_256) + return -EINVAL; + + ctx->aes_mode =3D DTHE_AES_GCM; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + crypto_sync_aead_clear_flags(ctx->aead_fb, CRYPTO_TFM_REQ_MASK); + crypto_sync_aead_set_flags(ctx->aead_fb, + crypto_aead_get_flags(tfm) & + CRYPTO_TFM_REQ_MASK); + + return crypto_sync_aead_setkey(ctx->aead_fb, key, keylen); +} + +static int dthe_aead_setauthsize(struct crypto_aead *tfm, unsigned int aut= hsize) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + /* Invalid auth size will be handled by crypto_aead_setauthsize() */ + ctx->authsize =3D authsize; + + return crypto_sync_aead_setauthsize(ctx->aead_fb, authsize); +} + +static int dthe_aead_do_fallback(struct aead_request *req) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + + SYNC_AEAD_REQUEST_ON_STACK(subreq, ctx->aead_fb); + + aead_request_set_callback(subreq, req->base.flags, + req->base.complete, req->base.data); + aead_request_set_crypt(subreq, req->src, req->dst, req->cryptlen, req->iv= ); + aead_request_set_ad(subreq, req->assoclen); + + return rctx->enc ? crypto_aead_encrypt(subreq) : + crypto_aead_decrypt(subreq); +} + +static void dthe_aead_dma_in_callback(void *data) +{ + struct aead_request *req =3D (struct aead_request *)data; + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + + complete(&rctx->aes_compl); +} + +static int dthe_aead_run(struct crypto_engine *engine, void *areq) +{ + struct aead_request *req =3D container_of(areq, struct aead_request, base= ); + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + + unsigned int cryptlen =3D req->cryptlen; + unsigned int assoclen =3D req->assoclen; + unsigned int authsize =3D ctx->authsize; + unsigned int unpadded_cryptlen; + struct scatterlist *src =3D req->src; + struct scatterlist *dst =3D req->dst; + u32 iv_in[AES_IV_WORDS]; + + int src_nents; + int dst_nents; + int src_mapped_nents, dst_mapped_nents; + + u8 src_assoc_padbuf[AES_BLOCK_SIZE] =3D {0}; + u8 src_crypt_padbuf[AES_BLOCK_SIZE] =3D {0}; + u8 dst_crypt_padbuf[AES_BLOCK_SIZE] =3D {0}; + + enum dma_data_direction src_dir, dst_dir; + + struct device *tx_dev, *rx_dev; + struct dma_async_tx_descriptor *desc_in, *desc_out; + + int ret; + + void __iomem *aes_base_reg =3D dev_data->regs + DTHE_P_AES_BASE; + + u32 aes_irqenable_val =3D readl_relaxed(aes_base_reg + DTHE_P_AES_IRQENAB= LE); + u32 aes_sysconfig_val =3D readl_relaxed(aes_base_reg + DTHE_P_AES_SYSCONF= IG); + + aes_sysconfig_val |=3D DTHE_AES_SYSCONFIG_DMA_DATA_IN_OUT_EN; + writel_relaxed(aes_sysconfig_val, aes_base_reg + DTHE_P_AES_SYSCONFIG); + + aes_irqenable_val |=3D DTHE_AES_IRQENABLE_EN_ALL; + writel_relaxed(aes_irqenable_val, aes_base_reg + DTHE_P_AES_IRQENABLE); + + /* In decryption, the last authsize bytes are the TAG */ + if (!rctx->enc) + cryptlen -=3D authsize; + unpadded_cryptlen =3D cryptlen; + + /* Prep src and dst scatterlists */ + src =3D dthe_aead_prep_src(req->src, req->assoclen, cryptlen, + src_assoc_padbuf, src_crypt_padbuf); + if (IS_ERR(src)) { + ret =3D PTR_ERR(src); + goto aead_prep_src_err; + } + + if (req->assoclen % AES_BLOCK_SIZE) + assoclen +=3D AES_BLOCK_SIZE - (req->assoclen % AES_BLOCK_SIZE); + if (cryptlen % AES_BLOCK_SIZE) + cryptlen +=3D AES_BLOCK_SIZE - (cryptlen % AES_BLOCK_SIZE); + + src_nents =3D sg_nents_for_len(src, assoclen + cryptlen); + + if (cryptlen !=3D 0) { + dst =3D dthe_aead_prep_dst(req->dst, req->assoclen, unpadded_cryptlen, + dst_crypt_padbuf); + if (IS_ERR(dst)) { + ret =3D PTR_ERR(dst); + goto aead_prep_dst_err; + } + + dst_nents =3D sg_nents_for_len(dst, cryptlen); + } + /* Prep finished */ + + src_dir =3D DMA_TO_DEVICE; + dst_dir =3D DMA_FROM_DEVICE; + + tx_dev =3D dmaengine_get_dma_device(dev_data->dma_aes_tx); + rx_dev =3D dmaengine_get_dma_device(dev_data->dma_aes_rx); + + src_mapped_nents =3D dma_map_sg(tx_dev, src, src_nents, src_dir); + if (src_mapped_nents =3D=3D 0) { + ret =3D -EINVAL; + goto aead_dma_map_src_err; + } + + desc_out =3D dmaengine_prep_slave_sg(dev_data->dma_aes_tx, src, src_mappe= d_nents, + DMA_MEM_TO_DEV, DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!desc_out) { + ret =3D -EINVAL; + goto aead_dma_prep_src_err; + } + + desc_out->callback =3D dthe_aead_dma_in_callback; + desc_out->callback_param =3D req; + + if (cryptlen !=3D 0) { + dst_mapped_nents =3D dma_map_sg(rx_dev, dst, dst_nents, dst_dir); + if (dst_mapped_nents =3D=3D 0) { + ret =3D -EINVAL; + goto aead_dma_prep_src_err; + } + + desc_in =3D dmaengine_prep_slave_sg(dev_data->dma_aes_rx, dst, + dst_mapped_nents, DMA_DEV_TO_MEM, + DMA_PREP_INTERRUPT | DMA_CTRL_ACK); + if (!desc_in) { + ret =3D -EINVAL; + goto aead_dma_prep_dst_err; + } + } + + init_completion(&rctx->aes_compl); + + /* + * HACK: There is an unknown hw issue where if the previous operation had= alen =3D 0 and + * plen !=3D 0, the current operation's tag calculation is incorrect in t= he case where + * plen =3D 0 and alen !=3D 0 currently. This is a workaround for now whi= ch somehow works; + * by resetting the context by writing a 1 to the C_LENGTH_0 and AUTH_LEN= GTH registers. + */ + if (cryptlen =3D=3D 0) { + writel_relaxed(1, aes_base_reg + DTHE_P_AES_C_LENGTH_0); + writel_relaxed(1, aes_base_reg + DTHE_P_AES_AUTH_LENGTH); + } + + if (req->iv) { + memcpy(iv_in, req->iv, GCM_AES_IV_SIZE); + } else { + iv_in[0] =3D 0; + iv_in[1] =3D 0; + iv_in[2] =3D 0; + } + iv_in[3] =3D 0x01000000; + + /* Clear key2 to reset previous GHASH intermediate data */ + for (int i =3D 0; i < AES_KEYSIZE_256 / sizeof(u32); ++i) + writel_relaxed(0, aes_base_reg + DTHE_P_AES_KEY2_6 + DTHE_REG_SIZE * i); + + dthe_aes_set_ctrl_key(ctx, rctx, iv_in); + + writel_relaxed(lower_32_bits(unpadded_cryptlen), aes_base_reg + DTHE_P_AE= S_C_LENGTH_0); + writel_relaxed(upper_32_bits(unpadded_cryptlen), aes_base_reg + DTHE_P_AE= S_C_LENGTH_1); + writel_relaxed(req->assoclen, aes_base_reg + DTHE_P_AES_AUTH_LENGTH); + + if (cryptlen !=3D 0) + dmaengine_submit(desc_in); + dmaengine_submit(desc_out); + + if (cryptlen !=3D 0) + dma_async_issue_pending(dev_data->dma_aes_rx); + dma_async_issue_pending(dev_data->dma_aes_tx); + + /* Need to do timeout to ensure finalise gets called if DMA callback fail= s for any reason */ + ret =3D wait_for_completion_timeout(&rctx->aes_compl, msecs_to_jiffies(DT= HE_DMA_TIMEOUT_MS)); + if (!ret) { + ret =3D -ETIMEDOUT; + if (cryptlen !=3D 0) + dmaengine_terminate_sync(dev_data->dma_aes_rx); + dmaengine_terminate_sync(dev_data->dma_aes_tx); + + for (int i =3D 0; i < AES_BLOCK_WORDS; ++i) + readl_relaxed(aes_base_reg + DTHE_P_AES_DATA_IN_OUT + DTHE_REG_SIZE * i= ); + } else { + ret =3D 0; + } + + if (cryptlen !=3D 0) + dma_sync_sg_for_cpu(rx_dev, dst, dst_nents, dst_dir); + if (rctx->enc) + ret =3D dthe_aead_enc_get_tag(req); + else + ret =3D dthe_aead_dec_verify_tag(req); + +aead_dma_prep_dst_err: + if (cryptlen !=3D 0) + dma_unmap_sg(rx_dev, dst, dst_nents, dst_dir); +aead_dma_prep_src_err: + dma_unmap_sg(tx_dev, src, src_nents, src_dir); + +aead_dma_map_src_err: + if (cryptlen !=3D 0) + kfree(dst); + +aead_prep_dst_err: + kfree(src); + +aead_prep_src_err: + if (ret) + ret =3D dthe_aead_do_fallback(req); + local_bh_disable(); + crypto_finalize_aead_request(engine, req, ret); + local_bh_enable(); + return 0; +} + +static int dthe_aead_crypt(struct aead_request *req) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(crypto_aead_reqtfm(req)); + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + struct dthe_data *dev_data =3D dthe_get_dev(ctx); + struct crypto_engine *engine; + unsigned int cryptlen =3D req->cryptlen; + + /* In decryption, last authsize bytes are the TAG */ + if (!rctx->enc) + cryptlen -=3D ctx->authsize; + + /* + * Need to fallback to software in the following cases due to HW restrict= ions: + * - Both AAD and plaintext/ciphertext are zero length + * - AAD length is more than 2^32 - 1 bytes + * PS: req->cryptlen is currently unsigned int type, which causes the abo= ve condition + * tautologically false. If req->cryptlen were to be changed to a 64-bit = type, + * the check for this would need to be added below. + */ + if (req->assoclen =3D=3D 0 && cryptlen =3D=3D 0) + return dthe_aead_do_fallback(req); + + engine =3D dev_data->engine; + return crypto_transfer_aead_request_to_engine(engine, req); +} + +static int dthe_aead_encrypt(struct aead_request *req) +{ + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + + rctx->enc =3D 1; + return dthe_aead_crypt(req); +} + +static int dthe_aead_decrypt(struct aead_request *req) +{ + struct dthe_aes_req_ctx *rctx =3D aead_request_ctx(req); + + rctx->enc =3D 0; + return dthe_aead_crypt(req); +} + static struct skcipher_engine_alg cipher_algs[] =3D { { .base.init =3D dthe_cipher_init_tfm, @@ -648,12 +1203,46 @@ static struct skcipher_engine_alg cipher_algs[] =3D { }, /* XTS AES */ }; =20 +static struct aead_engine_alg aead_algs[] =3D { + { + .base.init =3D dthe_aead_init_tfm, + .base.exit =3D dthe_aead_exit_tfm, + .base.setkey =3D dthe_aead_setkey, + .base.setauthsize =3D dthe_aead_setauthsize, + .base.maxauthsize =3D AES_BLOCK_SIZE, + .base.encrypt =3D dthe_aead_encrypt, + .base.decrypt =3D dthe_aead_decrypt, + .base.chunksize =3D AES_BLOCK_SIZE, + .base.ivsize =3D GCM_AES_IV_SIZE, + .base.base =3D { + .cra_name =3D "gcm(aes)", + .cra_driver_name =3D "gcm-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_AEAD | + CRYPTO_ALG_KERN_DRIVER_ONLY | + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D 1, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aead_run, + }, /* GCM AES */ +}; + int dthe_register_aes_algs(void) { - return crypto_engine_register_skciphers(cipher_algs, ARRAY_SIZE(cipher_al= gs)); + int ret =3D 0; + + ret |=3D crypto_engine_register_skciphers(cipher_algs, ARRAY_SIZE(cipher_= algs)); + ret |=3D crypto_engine_register_aeads(aead_algs, ARRAY_SIZE(aead_algs)); + + return ret; } =20 void dthe_unregister_aes_algs(void) { crypto_engine_unregister_skciphers(cipher_algs, ARRAY_SIZE(cipher_algs)); + crypto_engine_unregister_aeads(aead_algs, ARRAY_SIZE(aead_algs)); } diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index f12b94d64e134..7c54291359bf5 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -38,6 +38,7 @@ enum dthe_aes_mode { DTHE_AES_CBC, DTHE_AES_CTR, DTHE_AES_XTS, + DTHE_AES_GCM, }; =20 /* Driver specific struct definitions */ @@ -78,16 +79,22 @@ struct dthe_list { * struct dthe_tfm_ctx - Transform ctx struct containing ctx for all sub-c= omponents of DTHE V2 * @dev_data: Device data struct pointer * @keylen: AES key length + * @authsize: Authentication size for modes with authentication * @key: AES key * @aes_mode: AES mode + * @aead_fb: Fallback crypto aead handle * @skcipher_fb: Fallback crypto skcipher handle for AES-XTS mode */ struct dthe_tfm_ctx { struct dthe_data *dev_data; unsigned int keylen; + unsigned int authsize; u32 key[DTHE_MAX_KEYSIZE / sizeof(u32)]; enum dthe_aes_mode aes_mode; - struct crypto_sync_skcipher *skcipher_fb; + union { + struct crypto_sync_aead *aead_fb; + struct crypto_sync_skcipher *skcipher_fb; + }; }; =20 /** --=20 2.43.0 From nobody Sun Feb 8 02:55:53 2026 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012044.outbound.protection.outlook.com [52.101.53.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A98C43563F1; Tue, 11 Nov 2025 11:22:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.44 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860136; cv=fail; b=SoHLnwC2neD0EMYd0AP2Vk/tK/oQRF5l/vrXAnm3Nm1ofnJl6824spj7l3GYogvUyvkJcmiF0PgJESNjW7jIPJtY5DqpApc63R5nOywgL/fRfJXnchJGVtPjmVRi4gwbkSX51YBLkA37SJyofKIL5+G/OAlNqg36bH1mn346pjQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762860136; c=relaxed/simple; bh=AS1oGlma6sB5lPPcVownIO9jQwVCnsBEkp4MAEEi67c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=A2FP+Uq6X7SiY1jz49EtDLbZQlE5V1kNbhNNpEBQFM9mHkqauHAi/xug3reAsjqAomwtauGtREgo0H1iip8B13zmWnT6OHpN/WSlxFztvKevnJsG0/Dq3kGRK/zht64L15WkdaIpl/e9CmmAgGjG5OYBVQ7bGOnnE/N1oeDlb4s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=q7zFbJnd; arc=fail smtp.client-ip=52.101.53.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="q7zFbJnd" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SSpVycNn1N2Gey511pGR1e1on/cz85yxWYorpaCGOKY15MjZ6rEY9zZQ05Cqrk3wky+UIuYiIvmNskwngO3gp4SCdT0H74SJh6NOaXpmrFzCfAIBMatJZGbbtPxHVhB2ONufMGl7Q5yLZ3Hf3+7Z1t+u4VnPletoDBXj/nEbbeQEewD2W+E8gm4GzE5cwVWjmZMr9Ssug20pb8wh+RR7gB4K7kC8OC6mvW+om+O3C7qgFpFivZ1PQvPHDZKrT07HDFZnorGtOM528aHnntEr2gNzryPKJJRmr7be//fMeFocQuTwqo1SzoWXnUDg9nssvDJTm+lJ/TTG8GGqJUrRXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BRhZ99t5iP+KnJhFnmSMssKX+OS5Q9bjlarJIZuUX8s=; b=uyPlowcVs3V5UdlSrOH4F6Y+6F2POIiu+W/WjYUzLpwZ3LPike53Jm/a7On0rCqty/uyT3zZenxeHhMwzKuzkQVryAXzmw542veq58A9ScrwcwcX+9ko+a4i69haBTXA+fcjxOfHo0pmOrsHdWu+syCLdxR+r3HJiIbYH/3n23GjeR4Q6c+jIEFSaf7Ci+n8hDS7Si/6GgDYiBXW3npnvZW42cxZWxOZGh+VYjCj2lgDUAKkT44z7wW1+db/IAjoAaPbk6spFhzb0CItWXzFEWa3a4hYPbC+OPjyIba0eFihl4pFfuMoO8iV/MsEIWtbbDz64+9zAF4+cfmgLzW4GQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 198.47.21.194) smtp.rcpttodomain=gondor.apana.org.au smtp.mailfrom=ti.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=ti.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BRhZ99t5iP+KnJhFnmSMssKX+OS5Q9bjlarJIZuUX8s=; b=q7zFbJnd/qHCfcqZnL1wh/8q2wL7gMhvLaJU3BXakZv1VcaIkW9O8ZD+9Ku8NPUyCVMMGPmFXEZqTMr3XdYnFRyPmLcVtHN8L3ru4saCSGPXwwY6xLYXO40amIe5eNk53tA2XYnDi/HQ/5RU7iPqeoLccceeOSnQP5bo3wJKnbA= Received: from PH8P220CA0021.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:345::13) by CH3PR10MB7574.namprd10.prod.outlook.com (2603:10b6:610:180::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9320.15; Tue, 11 Nov 2025 11:22:08 +0000 Received: from SN1PEPF000252A1.namprd05.prod.outlook.com (2603:10b6:510:345:cafe::ea) by PH8P220CA0021.outlook.office365.com (2603:10b6:510:345::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9320.15 via Frontend Transport; Tue, 11 Nov 2025 11:22:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 198.47.21.194) smtp.mailfrom=ti.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ti.com; Received-SPF: Pass (protection.outlook.com: domain of ti.com designates 198.47.21.194 as permitted sender) receiver=protection.outlook.com; client-ip=198.47.21.194; helo=flwvzet200.ext.ti.com; pr=C Received: from flwvzet200.ext.ti.com (198.47.21.194) by SN1PEPF000252A1.mail.protection.outlook.com (10.167.242.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9320.13 via Frontend Transport; Tue, 11 Nov 2025 11:22:06 +0000 Received: from DFLE204.ent.ti.com (10.64.6.62) by flwvzet200.ext.ti.com (10.248.192.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:22:04 -0600 Received: from DFLE201.ent.ti.com (10.64.6.59) by DFLE204.ent.ti.com (10.64.6.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Tue, 11 Nov 2025 05:22:03 -0600 Received: from lelvem-mr05.itg.ti.com (10.180.75.9) by DFLE201.ent.ti.com (10.64.6.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Tue, 11 Nov 2025 05:22:03 -0600 Received: from pratham-Workstation-PC (pratham-workstation-pc.dhcp.ti.com [10.24.69.191]) by lelvem-mr05.itg.ti.com (8.18.1/8.18.1) with ESMTP id 5ABBM29c828548; Tue, 11 Nov 2025 05:22:03 -0600 From: T Pratham To: T Pratham , Herbert Xu , "David S. Miller" CC: Manorit Chawdhry , Kamlesh Gurudasani , Shiva Tripathi , Kavitha Malarvizhi , Vishal Mahaveer , , Subject: [PATCH v6 4/4] crypto: ti - Add support for AES-CCM in DTHEv2 driver Date: Tue, 11 Nov 2025 16:38:33 +0530 Message-ID: <20251111112137.976121-5-t-pratham@ti.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251111112137.976121-1-t-pratham@ti.com> References: <20251111112137.976121-1-t-pratham@ti.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF000252A1:EE_|CH3PR10MB7574:EE_ X-MS-Office365-Filtering-Correlation-Id: 0d8ff193-8581-4443-44b5-08de21148bf8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?foDljOu+GMEwznjL45GyMQMAol/NA0j+ZwtxbCHG344V77kbWnWrZNHXQqEc?= =?us-ascii?Q?sryqMqpX+yjzKW6mzBw86R01RpSSlV2FfFCXtqM7V9rbzvQQUd0CGsYDWcwk?= =?us-ascii?Q?mcdsSVy3ggR15gR7bCeZ6TVzTsY/x+VWg2oOvzalQ0Mh6ZPVNkB0GAxmkNl6?= =?us-ascii?Q?MTNVDbE/Kzea2BoJrJdiZ6QsOVfkExCsr6vcm60KgZ09IESUjctnpEkvYuHy?= =?us-ascii?Q?6VKQJFnQf5nahvJm0E5p1jAf3n9FxZ9dyF/KcF7G3Ct4laJ0Y/iLw5ddMHtC?= =?us-ascii?Q?4J4vUgxvOjJvfTlBA/8ozSktVnacy/ZtwS7Esb9zf8Sx3Zh34os4O8QlwPRS?= =?us-ascii?Q?Rd0bY5t30zjgEIjN5pfgA4KWBEgTH6iEtc/spmfM7ce4I96YirqGdmyaR6bE?= =?us-ascii?Q?oKHzT35p3zHQwrl++KJZ78Jgg/sWc90zwVXX5E5RWQys27pTvbddc8Z4uKv8?= =?us-ascii?Q?3LwdPWRvF03EiId0pbHGlIoDQuW84IkZKREtQsmKrqkGh/Sgh0neELUt9j5o?= =?us-ascii?Q?xGLQAuJBVXLoPxvwplizGUgzdwe0GCEY3aOIbEAux/LZ6tUYJOY2OsJosoJC?= =?us-ascii?Q?EnLI+HUr/jz6auiHpzo8h0wjAKGnjSiUmMDJ8iKIbl2tc3jHEKMusxWyNIsx?= =?us-ascii?Q?icnvS+5Akvu5beINSSXGtsoLU6b383Xf80T52RV0cXlF2lFayr5frovN+71n?= =?us-ascii?Q?EDCxW+HP9YGkGFfHO2lL6hk4bgVPRoXACUrafVxfIzG+ViGIEsqU96CPhRo7?= =?us-ascii?Q?ZmUkBpwJwhyi5vdAY8c6vLVQ+kr3WrQ46HhCeDSt4zxKapFHb/E0DXH7X5tq?= =?us-ascii?Q?fNQjo9kVDBSxgBvjzu9hpRJsV/vGbTBIRt3DbqqmDglYcJwk4q3uAdF5RSFs?= =?us-ascii?Q?fr5H4LPJ89kzbC0Mm+eFOVJsWynLLbVoPawj06G30x+Wim/BGNyuyVmWut2a?= =?us-ascii?Q?v+MSkBf5Sz0Z1Gqk4t+J+HTAAAwA8tpDdo7lPyPpZ8eqHHbvGrpkbMVrVvAY?= =?us-ascii?Q?DBEIykIHC2/0RovpfjXOlIsFaEMCw6BQK84tBl+8n+8RgrIUHEpWRPJJR2ma?= =?us-ascii?Q?pm7tzutGmkXQM0kZ6x4AJAw9dUhvolJIj4mBpJ/lqA/R2zdrNaqr+oB8NYVH?= =?us-ascii?Q?D0cVR+l+b/j7J/uyg5F4QWygD6UDw3ssc5pnTJJ9bIvc+7stqNiv8eCHryO3?= =?us-ascii?Q?9265JKidqo1DSmzqH3FiMSDjlY33SEzK3YLRndRFPCER/6NvlTXrerRUUW2/?= =?us-ascii?Q?rnRrpGvqJCxhQSlX0VAVAteUuw5eonSLgfdKTLAfhi4RrqSfOpaIoiP9JhF9?= =?us-ascii?Q?ewO4oo8mUMjnqcTLLpbX4n/nSE1Io8raabboB3EMeespmI7d4TTd+q4jY4pD?= =?us-ascii?Q?AOyc4zfB7/bk0hWIT/Q+UUTwVdFXjttLZMV8FW+2sxrTYqIbAUB3QM67TOZ7?= =?us-ascii?Q?rEEj8VD/n5ac4mQrrQ9EZNrhhqNGTvPAgtlxjS8t+sJux287lCm7r841cKiI?= =?us-ascii?Q?uhvvjCuMRsdn4h1eDB0hq1C8pQAk2NW7UbJ1anWeSXsw62dHRfaLe89cMXfu?= =?us-ascii?Q?SyuDQAq94i8S4M9J4lE=3D?= X-Forefront-Antispam-Report: CIP:198.47.21.194;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:flwvzet200.ext.ti.com;PTR:ErrorRetry;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: ti.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Nov 2025 11:22:06.4214 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0d8ff193-8581-4443-44b5-08de21148bf8 X-MS-Exchange-CrossTenant-Id: e5b49634-450b-4709-8abb-1e2b19b982b7 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=e5b49634-450b-4709-8abb-1e2b19b982b7;Ip=[198.47.21.194];Helo=[flwvzet200.ext.ti.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF000252A1.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB7574 Content-Type: text/plain; charset="utf-8" AES-CCM is an AEAD algorithm supporting both encryption and authentication of data. This patch introduces support for AES-CCM AEAD algorithm in the DTHEv2 driver. Signed-off-by: T Pratham --- drivers/crypto/ti/Kconfig | 1 + drivers/crypto/ti/dthev2-aes.c | 129 ++++++++++++++++++++++++++---- drivers/crypto/ti/dthev2-common.h | 1 + 3 files changed, 115 insertions(+), 16 deletions(-) diff --git a/drivers/crypto/ti/Kconfig b/drivers/crypto/ti/Kconfig index 221e483737439..1a3a571ac8cef 100644 --- a/drivers/crypto/ti/Kconfig +++ b/drivers/crypto/ti/Kconfig @@ -9,6 +9,7 @@ config CRYPTO_DEV_TI_DTHEV2 select CRYPTO_CTR select CRYPTO_XTS select CRYPTO_GCM + select CRYPTO_CCM select SG_SPLIT help This enables support for the TI DTHE V2 hw cryptography engine diff --git a/drivers/crypto/ti/dthev2-aes.c b/drivers/crypto/ti/dthev2-aes.c index f52e888cc7bd9..5e46733736a7a 100644 --- a/drivers/crypto/ti/dthev2-aes.c +++ b/drivers/crypto/ti/dthev2-aes.c @@ -16,6 +16,7 @@ =20 #include "dthev2-common.h" =20 +#include #include #include #include @@ -69,6 +70,7 @@ enum aes_ctrl_mode_masks { AES_CTRL_CTR_MASK =3D BIT(6), AES_CTRL_XTS_MASK =3D BIT(12) | BIT(11), AES_CTRL_GCM_MASK =3D BIT(17) | BIT(16) | BIT(6), + AES_CTRL_CCM_MASK =3D BIT(18) | BIT(6), }; =20 #define DTHE_AES_CTRL_MODE_CLEAR_MASK ~GENMASK(28, 5) @@ -81,6 +83,11 @@ enum aes_ctrl_mode_masks { =20 #define DTHE_AES_CTRL_CTR_WIDTH_128B (BIT(7) | BIT(8)) =20 +#define DTHE_AES_CCM_L_FROM_IV_MASK GENMASK(2, 0) +#define DTHE_AES_CCM_M_BITS GENMASK(2, 0) +#define DTHE_AES_CTRL_CCM_L_FIELD_MASK GENMASK(21, 19) +#define DTHE_AES_CTRL_CCM_M_FIELD_MASK GENMASK(24, 22) + #define DTHE_AES_CTRL_SAVE_CTX_SET BIT(29) =20 #define DTHE_AES_CTRL_OUTPUT_READY BIT_MASK(0) @@ -96,6 +103,8 @@ enum aes_ctrl_mode_masks { #define AES_BLOCK_WORDS (AES_BLOCK_SIZE / sizeof(u32)) #define AES_IV_WORDS AES_BLOCK_WORDS #define DTHE_AES_GCM_AAD_MAXLEN (BIT_ULL(32) - 1) +#define DTHE_AES_CCM_AAD_MAXLEN (BIT(16) - BIT(8)) +#define DTHE_AES_CCM_CRYPT_MAXLEN (BIT_ULL(61) - 1) #define POLL_TIMEOUT_INTERVAL HZ =20 static struct scatterlist *dthe_chain_pad_sg(struct scatterlist *sg, @@ -304,6 +313,13 @@ static void dthe_aes_set_ctrl_key(struct dthe_tfm_ctx = *ctx, case DTHE_AES_GCM: ctrl_val |=3D AES_CTRL_GCM_MASK; break; + case DTHE_AES_CCM: + ctrl_val |=3D AES_CTRL_CCM_MASK; + ctrl_val |=3D FIELD_PREP(DTHE_AES_CTRL_CCM_L_FIELD_MASK, + (iv_in[0] & DTHE_AES_CCM_L_FROM_IV_MASK)); + ctrl_val |=3D FIELD_PREP(DTHE_AES_CTRL_CCM_M_FIELD_MASK, + ((ctx->authsize - 2) >> 1) & DTHE_AES_CCM_M_BITS); + break; } =20 if (iv_in) { @@ -824,10 +840,6 @@ static int dthe_aead_setkey(struct crypto_aead *tfm, c= onst u8 *key, unsigned int if (keylen !=3D AES_KEYSIZE_128 && keylen !=3D AES_KEYSIZE_192 && keylen = !=3D AES_KEYSIZE_256) return -EINVAL; =20 - ctx->aes_mode =3D DTHE_AES_GCM; - ctx->keylen =3D keylen; - memcpy(ctx->key, key, keylen); - crypto_sync_aead_clear_flags(ctx->aead_fb, CRYPTO_TFM_REQ_MASK); crypto_sync_aead_set_flags(ctx->aead_fb, crypto_aead_get_flags(tfm) & @@ -836,6 +848,28 @@ static int dthe_aead_setkey(struct crypto_aead *tfm, c= onst u8 *key, unsigned int return crypto_sync_aead_setkey(ctx->aead_fb, key, keylen); } =20 +static int dthe_gcm_aes_setkey(struct crypto_aead *tfm, const u8 *key, uns= igned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + ctx->aes_mode =3D DTHE_AES_GCM; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + return dthe_aead_setkey(tfm, key, keylen); +} + +static int dthe_ccm_aes_setkey(struct crypto_aead *tfm, const u8 *key, uns= igned int keylen) +{ + struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); + + ctx->aes_mode =3D DTHE_AES_CCM; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + return dthe_aead_setkey(tfm, key, keylen); +} + static int dthe_aead_setauthsize(struct crypto_aead *tfm, unsigned int aut= hsize) { struct dthe_tfm_ctx *ctx =3D crypto_aead_ctx(tfm); @@ -994,14 +1028,18 @@ static int dthe_aead_run(struct crypto_engine *engin= e, void *areq) writel_relaxed(1, aes_base_reg + DTHE_P_AES_AUTH_LENGTH); } =20 - if (req->iv) { - memcpy(iv_in, req->iv, GCM_AES_IV_SIZE); + if (ctx->aes_mode =3D=3D DTHE_AES_GCM) { + if (req->iv) { + memcpy(iv_in, req->iv, GCM_AES_IV_SIZE); + } else { + iv_in[0] =3D 0; + iv_in[1] =3D 0; + iv_in[2] =3D 0; + } + iv_in[3] =3D 0x01000000; } else { - iv_in[0] =3D 0; - iv_in[1] =3D 0; - iv_in[2] =3D 0; + memcpy(iv_in, req->iv, AES_IV_SIZE); } - iv_in[3] =3D 0x01000000; =20 /* Clear key2 to reset previous GHASH intermediate data */ for (int i =3D 0; i < AES_KEYSIZE_256 / sizeof(u32); ++i) @@ -1071,20 +1109,54 @@ static int dthe_aead_crypt(struct aead_request *req) struct dthe_data *dev_data =3D dthe_get_dev(ctx); struct crypto_engine *engine; unsigned int cryptlen =3D req->cryptlen; + bool is_zero_ctr =3D true; =20 /* In decryption, last authsize bytes are the TAG */ if (!rctx->enc) cryptlen -=3D ctx->authsize; =20 + if (ctx->aes_mode =3D=3D DTHE_AES_CCM) { + /* + * For CCM Mode, the 128-bit IV contains the following: + * | 0 .. 2 | 3 .. 7 | 8 .. (127-8*L) | (128-8*L) .. 127 | + * | L-1 | Zero | Nonce | Counter | + * L needs to be between 2-8 (inclusive), i.e. 1 <=3D (L-1) <=3D 7 + * and the next 5 bits need to be zeroes. Else return -EINVAL + */ + u8 *iv =3D req->iv; + u8 L =3D iv[0]; + + if (L < 1 || L > 7) + return -EINVAL; + /* + * DTHEv2 HW can only work with zero initial counter in CCM mode. + * Check if the initial counter value is zero or not + */ + for (int i =3D 0; i < L + 1; ++i) { + if (iv[AES_IV_SIZE - 1 - i] !=3D 0) { + is_zero_ctr =3D false; + break; + } + } + } + /* * Need to fallback to software in the following cases due to HW restrict= ions: * - Both AAD and plaintext/ciphertext are zero length - * - AAD length is more than 2^32 - 1 bytes - * PS: req->cryptlen is currently unsigned int type, which causes the abo= ve condition - * tautologically false. If req->cryptlen were to be changed to a 64-bit = type, - * the check for this would need to be added below. + * - For AES-GCM, AAD length is more than 2^32 - 1 bytes + * - For AES-CCM, AAD length is more than 2^16 - 2^8 bytes + * - For AES-CCM, plaintext/ciphertext length is more than 2^61 - 1 bytes + * - For AES-CCM, AAD length is non-zero but plaintext/ciphertext length = is zero + * - For AES-CCM, the initial counter (last L+1 bytes of IV) is not all z= eroes + * + * PS: req->cryptlen is currently unsigned int type, which causes the sec= ond and fourth + * cases above tautologically false. If req->cryptlen is to be changed to= a 64-bit + * type, the check for these would also need to be added below. */ - if (req->assoclen =3D=3D 0 && cryptlen =3D=3D 0) + if ((req->assoclen =3D=3D 0 && cryptlen =3D=3D 0) || + (ctx->aes_mode =3D=3D DTHE_AES_CCM && req->assoclen > DTHE_AES_CCM_AA= D_MAXLEN) || + (ctx->aes_mode =3D=3D DTHE_AES_CCM && cryptlen =3D=3D 0) || + (ctx->aes_mode =3D=3D DTHE_AES_CCM && !is_zero_ctr)) return dthe_aead_do_fallback(req); =20 engine =3D dev_data->engine; @@ -1207,7 +1279,7 @@ static struct aead_engine_alg aead_algs[] =3D { { .base.init =3D dthe_aead_init_tfm, .base.exit =3D dthe_aead_exit_tfm, - .base.setkey =3D dthe_aead_setkey, + .base.setkey =3D dthe_gcm_aes_setkey, .base.setauthsize =3D dthe_aead_setauthsize, .base.maxauthsize =3D AES_BLOCK_SIZE, .base.encrypt =3D dthe_aead_encrypt, @@ -1229,6 +1301,31 @@ static struct aead_engine_alg aead_algs[] =3D { }, .op.do_one_request =3D dthe_aead_run, }, /* GCM AES */ + { + .base.init =3D dthe_aead_init_tfm, + .base.exit =3D dthe_aead_exit_tfm, + .base.setkey =3D dthe_ccm_aes_setkey, + .base.setauthsize =3D dthe_aead_setauthsize, + .base.maxauthsize =3D AES_BLOCK_SIZE, + .base.encrypt =3D dthe_aead_encrypt, + .base.decrypt =3D dthe_aead_decrypt, + .base.chunksize =3D AES_BLOCK_SIZE, + .base.ivsize =3D AES_IV_SIZE, + .base.base =3D { + .cra_name =3D "ccm(aes)", + .cra_driver_name =3D "ccm-aes-dthev2", + .cra_priority =3D 299, + .cra_flags =3D CRYPTO_ALG_TYPE_AEAD | + CRYPTO_ALG_KERN_DRIVER_ONLY | + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D 1, + .cra_ctxsize =3D sizeof(struct dthe_tfm_ctx), + .cra_reqsize =3D sizeof(struct dthe_aes_req_ctx), + .cra_module =3D THIS_MODULE, + }, + .op.do_one_request =3D dthe_aead_run, + }, /* CCM AES */ }; =20 int dthe_register_aes_algs(void) diff --git a/drivers/crypto/ti/dthev2-common.h b/drivers/crypto/ti/dthev2-c= ommon.h index 7c54291359bf5..3b8d30b3408a0 100644 --- a/drivers/crypto/ti/dthev2-common.h +++ b/drivers/crypto/ti/dthev2-common.h @@ -39,6 +39,7 @@ enum dthe_aes_mode { DTHE_AES_CTR, DTHE_AES_XTS, DTHE_AES_GCM, + DTHE_AES_CCM, }; =20 /* Driver specific struct definitions */ --=20 2.43.0