From nobody Mon Jun 8 08:28:17 2026 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BE8F331A43 for ; Sat, 30 May 2026 16:07:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157236; cv=none; b=MqllkN7fzOTDX0+kGPoRYZ8xdQ5NHn13jP6mQzdr/fOp3WnV+Js3l5yTyH1ZSBkK96Mi8pRo6sTWGgK03qyqog80YrOnGqAY3D414Ket+5Hyb7QlvLeBITgtE9LdPieFrf1CVT9Gc746ZaIjbSLbN+woSjtKne1kdJ8Bvvq09RQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157236; c=relaxed/simple; bh=oJaLQm7RO4ezlWTTXjCrTAAHg8KhdMd5CaAE1SfLUXM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qWXS20R4jBN/r0Z5+YTD11VB381FDormyo4oj/IGSh6ZPbqUg2atwmFzthZKgjGJ6uMxZTW5zEQNpc7eK9qQoFdPQXThr+xHS2BRKws1elIbjnfYdbqnVVvMb8W9Vz/JpO+UeOjgzFM1/Z6HFXOWJuP6OP+H0iVuTk5wLvHwltg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=pXCyCyEn; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pXCyCyEn" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-45ef779c1c2so632926f8f.1 for ; Sat, 30 May 2026 09:07:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780157233; x=1780762033; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sh+f2AliULr9Rp7wvXUw2a0MafO9SPx6nLUlNFzn0RI=; b=pXCyCyEnOK1AzweldIusjmSZ4u8N4g3zqY9XGPuyJPoJDbQJlDAKHb8kJvznHAU6lc 4c5z7mGBeQWH8a6414G3Yya9GWYHvlt2fTichrvPStlJbINt0niM/hbSSyLP/Q2h7yPw 8MKM7B5QL9UaUwQRqOiNeco4VQSkJXeHDCueaTPyIdKgNV6WW6AfQ+e55WTcaiIv8cKF ypcR0Gm0Lu+6OdkQcuOVeK5WBWdSL1GduDTepD1usypt9ASBfIaSR9nqn/Vp2vjaj0Nu PXLLXIVB0w6DiDAGCPnYf8I9rTNXriSaEVGWhLPKb9cZo7I9qHHKf1sYTjgKGJkFKwK5 Ei2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780157233; x=1780762033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sh+f2AliULr9Rp7wvXUw2a0MafO9SPx6nLUlNFzn0RI=; b=pewatIjcFUEB2VI45ozJfGLMQESwI6LINZqpmeL8fG4GijVZArBd/NzZHKXBfcFPNK GmWCtV+cQABfQ+Qr9gpUba3Thx++l+VI7cYjFzD0CqWX+zDQQAxXYgq+b+aMOft2hisN 0wC08KMheByyAAYEGstbEoDeBwPBHjzcIfe5R/Jgz5PW+4N2Z80txQjm/J5Cbm3l2ZM5 /6a8tW+LhX1ujpmL/EquJwWwMcnRLgKfTnHSH/8blgqdduq/APqfMFSrFTLR/r4KJ6t1 rxeCzQte+RU9OKn3AYeEtMHifAu9JVl1EvDUwrIi4uZmIp0tOZB5ZRXtjAD239bFo69b m6/w== X-Forwarded-Encrypted: i=1; AFNElJ/FmR9b3IHRJmBbf6YfKTchgmyHgcIF39H6KgyRIwg7e8MKCzbQAWRWFJb5TZ4exLrOzBG9du/+uXIacMc=@vger.kernel.org X-Gm-Message-State: AOJu0YwMLbGTTlgdbNkMxuG+usI6V5t3zsRVX5NKUjTxHLpE06Icl04F 4yGFGOJ7vzXBNN2/uUfA7zWrM7jYFBBecZbsLDg5b5+F6OGheeKu6/Wd X-Gm-Gg: Acq92OEVhHD0TOBFGPG4AfMZ0giWsv1Dusl7CdXJm+tq+CJjWPRNTS5+r/Gc/nTQ5nj p93PH3T6WEwEU6Qlm5sMKjImqNJF8Fdq7H+SlGlyCFDoSiIHaTG+7PK2hq3YovbUZac7/n2OM60 H9PjBcXL48DD5VFzruRd2CYUr0HHbzBP5Ip3Ofr9pyH7jAeweR0ogkx8OSOq44OwpUC/s47a18/ bKAT+ZR7Z9ItkoPLkINp69F2CAGO7fYrq6UipN9VQ/DH7r6yXdyzmQtte+IcatyqDKFsbFRFJIi cMUQbYEGhygxbdop3l1yhX/Y3/Es7da6tdy2cOrzbNvVF4b1mGWI9U+hJmrUKbR8qy0GxvG/E1z zha0j9oxgZ9HAUz6kH7b3T1iLFSD618aaXOY7XyHyBpsCS72jKbMdV0UMpTBsyZhp2sa8HaL/ER On3/xrTCbBkMgMKC1hxt0eCkzPs+s= X-Received: by 2002:a05:6000:22c7:b0:43d:762e:76ba with SMTP id ffacd0b85a97d-45ef6b19b8emr8623681f8f.17.1780157232877; Sat, 30 May 2026 09:07:12 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef32fabcasm11667339f8f.0.2026.05.30.09.07.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 May 2026 09:07:12 -0700 (PDT) From: Dawid Olesinski To: herbert@gondor.apana.org.au, davem@davemloft.net, heiko@sntech.de Cc: linux-crypto@vger.kernel.org, linux-rockchip@lists.infradead.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, clabbe@baylibre.com, robh@kernel.org, krzk+dt@kernel.org, conor+dt@kernel.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH 1/4] dt-bindings: crypto: rockchip: Add RK356x/RK3588 crypto engine binding Date: Sat, 30 May 2026 17:06:42 +0100 Message-ID: <20260530160704.3453555-2-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260530160704.3453555-1-dawidro@gmail.com> References: <20260530160704.3453555-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a YAML device tree binding for the Rockchip second-generation (V2) cryptographic hardware accelerator present on the RK3568 and RK3588 SoCs. The IP block exposes AES-ECB, AES-CBC, AES-XTS block ciphers, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512, MD5, and SM3 hash algorithms, each with a hardware DMA engine controlled via linked-list descriptors. The binding covers two compatible strings: - rockchip,rk3568-crypto: clocks and resets are driven directly by the non-secure CRU (accessible to Linux at EL1). - rockchip,rk3588-crypto: clocks and resets live in SECURECRU, a register bank sandboxed to TrustZone. Linux must request them through the ARM SCMI firmware interface (scmi_clk / scmi_reset), as direct MMIO access to SECURECRU from EL1 triggers a bus fault. Signed-off-by: Dawid Olesinski --- .../crypto/rockchip,rk3588-crypto.yaml | 69 +++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 Documentation/devicetree/bindings/crypto/rockchip,rk358= 8-crypto.yaml diff --git a/Documentation/devicetree/bindings/crypto/rockchip,rk3588-crypt= o.yaml b/Documentation/devicetree/bindings/crypto/rockchip,rk3588-crypto.ya= ml new file mode 100644 index 000000000000..4188ed8920db --- /dev/null +++ b/Documentation/devicetree/bindings/crypto/rockchip,rk3588-crypto.yaml @@ -0,0 +1,69 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/crypto/rockchip,rk3588-crypto.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Rockchip cryptographic offloader + +maintainers: + - Heiko Stuebner + - Corentin Labbe + - Dawid Olesinski + +properties: + compatible: + enum: + - rockchip,rk3568-crypto + - rockchip,rk3588-crypto + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: Core clock for the crypto IP internal logic + - description: AXI interconnect clock interface + - description: AHB interface clock + + clock-names: + items: + - const: core + - const: aclk + - const: hclk + + resets: + maxItems: 1 + + reset-names: + items: + - const: core + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - resets + +additionalProperties: false + +examples: + - | + #include + + crypto@fe370000 { + compatible =3D "rockchip,rk3588-crypto"; + reg =3D <0x0 0xfe370000 0x0 0x2000>; + interrupts =3D ; + clocks =3D <&scmi_clk SCMI_CRYPTO_CORE>, + <&scmi_clk SCMI_ACLK_SECURE_NS>, + <&scmi_clk SCMI_HCLK_SECURE_NS>; + clock-names =3D "core", "aclk", "hclk"; + resets =3D <&scmi_reset SCMI_SRST_CRYPTO_CORE>; + reset-names =3D "core"; + }; --=20 2.47.3 From nobody Mon Jun 8 08:28:17 2026 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32EA131E859 for ; Sat, 30 May 2026 16:07:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157242; cv=none; b=rsvO++6i4c/KCb4BsqG9APfQ/UUBmgLNm2bpvjYfUJjX+ZhckmyFxAq2Ew3mwYcutS5514k2vtBH0nIMfOoGrwEMuRyOwmJ+xV4Cl7FZnCtagauHoMp6dMakYPkkIzWdQN4Aw4OIte9RQoUcoC5YbnGChA5irHdkQVzZXKMTen0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157242; c=relaxed/simple; bh=g0XbGgZo3uPbmwJIhG1ZMUBPivsAiTAmF9c+XfSObHs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=USvyvFe/8T673AJS+65hNzW0TwJkYiRSlHtZj22EyNCxxkewwr8sDcDskAjILok7rYJsW57ZRccr7kEDhPgDTC7rvm0k3xd0ocgC7mxGhBGzx/PJ3VVCxW0HsYFtZ6vKXZZzYGEokCWFPimJGVlEABuqFZXWr5WEyVn4UI423LI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hdke2B60; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hdke2B60" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-45ef779c1c2so632965f8f.1 for ; Sat, 30 May 2026 09:07:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780157236; x=1780762036; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y0ylhYf0nuQxp0ap73H8PNlATNySXusRtoiyJhsKGeI=; b=hdke2B60LWIO1Y7o8dHsXchQxzadnfF1xdr8LRgTkSdQ7zA0MWtqzCas6IeYEHT0Uj t87icVU1XaNVYvMLQIRJ8qyFXY9hdYFKsw6PylhYvtRe28U67lH3R5Enxlxn9kHK+V80 b3hbty8rkVsaDkbDZgaZ1fKC4kNX8EAJ1myRhworBg+C6GSjHDV8E47vdkhI1GlTENIG 8L8tK19yyJ6AKmB7RQB3csQ/Fv9pdqQpjsgUVXQTHwv229bjoSXmZ9Rw8ZhIi2+oMLtd x45gXt9DYocS7BJ9vbsGPtrg/saOy+NeLTosLKDbp8UkB1Z1hJwSKN/LKM9z/XoaBfXD 4Qxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780157237; x=1780762037; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Y0ylhYf0nuQxp0ap73H8PNlATNySXusRtoiyJhsKGeI=; b=KD2Lp9igSqywRWmDqMwWD5mMjj9fLvx+M/y+uShg7SfyWGhsRcwR2zl7+S5IILyhcW tgif4AsZVG9R9qn6HKhsYWL4VEv9mGxTUJc1nlwrWK9kWUUX5ocCbUxqLunXwAlUdDn7 YOJftsou5H09180kyo6kO3hLoBseSvueAbILKFB+JUQxh4omorztjv3K9Nm5up2o4KVr zo+Xf6f3V7HL37P42Lk22rT9oj/dnoRvfS1Zw2z3333qdbmrMeLbGu0LFAiMLo4xDJNw H/p5m+edumwMCM6GduRXiGG05x7HDgA4+G6r2RIAaFMVt3M6QjjXTKdfFQ7T1wzwZklx U7oA== X-Forwarded-Encrypted: i=1; AFNElJ+wBsxeYEsWR+U7aucjiXEBxVKRp16N03YiOkv5FkhBr6/vTBbNePvmD5qBIy1Cqs5V+2clbGyfT1SAwDc=@vger.kernel.org X-Gm-Message-State: AOJu0YzLmsgb9HmSfxeR9ETVPymclMxyPPv/tnGKJuen8yydJRO7Lmk0 51c6lgVO6+ReEHT5c/nRqbFuStj82RQcjTJfhYU6RH8ED+PZArui3vFM X-Gm-Gg: Acq92OFR2rNtDpxTDc5IEEXr5L/fKjscFmHOrU6wtqO4WY1A9gY9Kt3E8w/jp6LX9Vz sDNCkG3liLTHTIyRJdwueDMoDsSSAHTSgfdS+/GaBsKtZmFDdWgHPAR5Kof5M2f7MYo/EyWhpwX i5eTD7CJ/w/1rH+zsMjiCiJK05lpXEa7/5+LU/hpBqXOPAqd4eo0xbrEeK8eC5JPv4nwBhv97Dl n2v07smAyg299hJ0tCAkGeBTIhjnsjA2FvLZxBtDRQtZHvmV8E3F4RJRG/UO1FOKnx7HHoXW9q2 q0Fb5MXvoTjsEWOrER60+ZecN0bJp1ezbTMqucjU++kepv74ihOOapWLuatFslgS5AnW8DG0Bby 2m+blaa0jO3ntlMU+BqaUG8Y599H/yIedtseYgJqbapXhCZwQ6mJ3viBPjfafodV/vYUjUClrqd ucXpBwBwnbJTdjpiK0HXPaEMaYXN4= X-Received: by 2002:adf:fad1:0:b0:45e:f5bf:6c25 with SMTP id ffacd0b85a97d-45ef6b6c555mr6199403f8f.32.1780157235885; Sat, 30 May 2026 09:07:15 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef32fabcasm11667339f8f.0.2026.05.30.09.07.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 May 2026 09:07:15 -0700 (PDT) From: Dawid Olesinski To: herbert@gondor.apana.org.au, davem@davemloft.net, heiko@sntech.de Cc: linux-crypto@vger.kernel.org, linux-rockchip@lists.infradead.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, clabbe@baylibre.com, robh@kernel.org, krzk+dt@kernel.org, conor+dt@kernel.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH 2/4] crypto: rockchip: Add RK356x/RK3588 cryptographic offloader driver Date: Sat, 30 May 2026 17:06:43 +0100 Message-ID: <20260530160704.3453555-3-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260530160704.3453555-1-dawidro@gmail.com> References: <20260530160704.3453555-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a driver for the second-generation Rockchip cryptographic hardware accelerator found on RK3568 and RK3588 SoCs (compatible strings "rockchip,rk3568-crypto" and "rockchip,rk3588-crypto"). The hardware provides: - AES block cipher engine: ECB, CBC, and XTS modes, 128/192/256-bit keys. XTS hardware is limited to single-SG requests. - Hash engine: SHA-1, SHA-256, SHA-384, SHA-512, MD5, SM3. The hardware padding engine (HW_PAD) requires the total message length upfront and cannot maintain state across LLI descriptor boundaries, so multi-SG and unaligned requests are routed to a software fallback. - DMA engine: linked-list descriptor (LLI) based, with a 20-entry coherent descriptor table. Design overview: - Built on top of the crypto engine framework (crypto/engine.h) for serialised hardware request dispatch and automatic fallback handling. - Each platform device gets its own private copy of the algorithm descriptor table at probe time (devm_kmemdup), so the dev pointer in each template always refers to the correct hardware instance. This avoids a global device list without any locking overhead. - Runtime PM with autosuspend (2 s idle timeout) gates clocks and asserts reset between requests to save power. - Symmetric software fallback for all registered algorithms handles requests that cannot be processed in hardware (misaligned buffers, multi-SG inputs for hash, zero-length payloads). Co-developed-by: Corentin Labbe Signed-off-by: Corentin Labbe Signed-off-by: Dawid Olesinski --- drivers/crypto/Kconfig | 33 + drivers/crypto/Makefile | 1 + drivers/crypto/rockchip/Makefile | 5 + drivers/crypto/rockchip/rk2_crypto.c | 740 ++++++++++++++++++ drivers/crypto/rockchip/rk2_crypto.h | 243 ++++++ drivers/crypto/rockchip/rk2_crypto_ahash.c | 547 +++++++++++++ drivers/crypto/rockchip/rk2_crypto_skcipher.c | 724 +++++++++++++++++ 7 files changed, 2293 insertions(+) create mode 100644 drivers/crypto/rockchip/rk2_crypto.c create mode 100644 drivers/crypto/rockchip/rk2_crypto.h create mode 100644 drivers/crypto/rockchip/rk2_crypto_ahash.c create mode 100644 drivers/crypto/rockchip/rk2_crypto_skcipher.c diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index d23b58b81ca3..47a891593814 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -709,6 +709,39 @@ config CRYPTO_DEV_ROCKCHIP_DEBUG This will create /sys/kernel/debug/rk3288_crypto/stats for displaying the number of requests per algorithm and other internal stats. =20 +config CRYPTO_DEV_ROCKCHIP2 + tristate "Rockchip's cryptographic offloader" + depends on OF && ARCH_ROCKCHIP + depends on PM + select CRYPTO_ECB + select CRYPTO_CBC + select CRYPTO_AES + select CRYPTO_MD5 + select CRYPTO_SHA1 + select CRYPTO_SHA256 + select CRYPTO_SHA512 + select CRYPTO_SM3_GENERIC + select CRYPTO_HASH + select CRYPTO_XTS + select CRYPTO_SKCIPHER + select CRYPTO_ENGINE + + help + This driver interfaces with the hardware crypto offloader present + on RK3566, RK3568 and RK3588 SoCs. It provides hardware acceleration + for symmetric block ciphers and hashing functions, offloading the + main CPU cores during heavy cryptographic workflows. + +config CRYPTO_DEV_ROCKCHIP2_DEBUG + bool "Enable Rockchip crypto stats" + depends on CRYPTO_DEV_ROCKCHIP2 + depends on DEBUG_FS + help + Say y to enable Rockchip crypto debug stats. + This will create a directory using the device name + (e.g., /sys/kernel/debug/fe370000.crypto/stats) for displaying + the number of requests per algorithm and other internal stats. + config CRYPTO_DEV_TEGRA tristate "Enable Tegra Security Engine" depends on TEGRA_HOST1X diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile index 283bbc650b5b..905538078017 100644 --- a/drivers/crypto/Makefile +++ b/drivers/crypto/Makefile @@ -30,6 +30,7 @@ obj-$(CONFIG_CRYPTO_DEV_PPC4XX) +=3D amcc/ obj-$(CONFIG_CRYPTO_DEV_QCE) +=3D qce/ obj-$(CONFIG_CRYPTO_DEV_QCOM_RNG) +=3D qcom-rng.o obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP) +=3D rockchip/ +obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP2) +=3D rockchip/ obj-$(CONFIG_CRYPTO_DEV_S5P) +=3D s5p-sss.o obj-$(CONFIG_CRYPTO_DEV_SA2UL) +=3D sa2ul.o obj-$(CONFIG_CRYPTO_DEV_SAHARA) +=3D sahara.o diff --git a/drivers/crypto/rockchip/Makefile b/drivers/crypto/rockchip/Mak= efile index 785277aca71e..452a12ff6538 100644 --- a/drivers/crypto/rockchip/Makefile +++ b/drivers/crypto/rockchip/Makefile @@ -3,3 +3,8 @@ obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP) +=3D rk_crypto.o rk_crypto-objs :=3D rk3288_crypto.o \ rk3288_crypto_skcipher.o \ rk3288_crypto_ahash.o + +obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP2) +=3D rk_crypto2.o +rk_crypto2-objs :=3D rk2_crypto.o \ + rk2_crypto_skcipher.o \ + rk2_crypto_ahash.o diff --git a/drivers/crypto/rockchip/rk2_crypto.c b/drivers/crypto/rockchip= /rk2_crypto.c new file mode 100644 index 000000000000..df7dab4d7ca0 --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto.c @@ -0,0 +1,740 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * hardware cryptographic offloader for RK3568/RK3588 SoC + * + * Copyright (c) 2022-2023, Corentin Labbe + */ + +#include "rk2_crypto.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static const struct rk2_variant rk3568_variant =3D { + .num_clks =3D 3, +}; + +static const struct rk2_variant rk3588_variant =3D { + .num_clks =3D 3, +}; + +static int rk2_crypto_get_clks(struct rk2_crypto_dev *dev) +{ + dev->num_clks =3D devm_clk_bulk_get_all(dev->dev, &dev->clks); + if (dev->num_clks < 0) + return dev_err_probe(dev->dev, dev->num_clks, "Failed to get clocks\n"); + if (dev->num_clks < dev->variant->num_clks) + return dev_err_probe(dev->dev, -EINVAL, + "Missing clocks, got %d instead of %d\n", + dev->num_clks, dev->variant->num_clks); + return 0; +} + +static int rk2_crypto_pm_suspend(struct device *dev) +{ + struct rk2_crypto_dev *rkdev =3D dev_get_drvdata(dev); + + reset_control_assert(rkdev->rst); + udelay(10); + clk_bulk_disable_unprepare(rkdev->num_clks, rkdev->clks); + + return 0; +} + +static int rk2_crypto_pm_resume(struct device *dev) +{ + struct rk2_crypto_dev *rkdev =3D dev_get_drvdata(dev); + int ret; + + ret =3D clk_bulk_prepare_enable(rkdev->num_clks, rkdev->clks); + if (ret) + return ret; + + udelay(10); + reset_control_deassert(rkdev->rst); + + return 0; +} + +static const struct dev_pm_ops rk2_crypto_pm_ops =3D { + RUNTIME_PM_OPS(rk2_crypto_pm_suspend, rk2_crypto_pm_resume, NULL) +}; + +static int rk2_crypto_pm_init(struct rk2_crypto_dev *rkdev) +{ + int err; + + pm_runtime_use_autosuspend(rkdev->dev); + pm_runtime_set_autosuspend_delay(rkdev->dev, 2000); + + err =3D pm_runtime_set_suspended(rkdev->dev); + if (err) + return err; + pm_runtime_enable(rkdev->dev); + + return 0; +} + +static void rk2_crypto_pm_exit(struct rk2_crypto_dev *rkdev) +{ + pm_runtime_disable(rkdev->dev); +} + +static irqreturn_t rk2_crypto_irq_handle(int irq, void *dev_id) +{ + struct rk2_crypto_dev *rkc =3D platform_get_drvdata(dev_id); + u32 v; + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_INT_ST); + if (!v) + return IRQ_NONE; + + writel(v, rkc->reg + RK2_CRYPTO_DMA_INT_ST); + + /* + * Only signal completion on list-done or hard DMA error. + * Intermediate SRC_INT (BIT(1)/BIT(2)) fire for every LLI + * entry that has RK2_LLI_DMA_CTRL_SRC_INT set. Completing + * early on those causes the driver to read hash registers + * before all data has been processed, producing wrong results. + */ + if (v & RK2_CRYPTO_DMA_INT_ERR_MASK) { + dev_warn(rkc->dev, "DMA Error\n"); + rkc->status =3D 0; + complete(&rkc->complete); + } else if (v & RK2_CRYPTO_DMA_INT_LISTDONE) { + rkc->status =3D 1; + complete(&rkc->complete); + } + + return IRQ_HANDLED; +} + +static const struct rk2_crypto_template rk2_crypto_algs_template[] =3D { + { + .type =3D CRYPTO_ALG_TYPE_SKCIPHER, + .rk2_mode =3D RK2_CRYPTO_AES_ECB, + .alg.skcipher.base =3D { + .base.cra_name =3D "ecb(aes)", + .base.cra_driver_name =3D "ecb-aes-rk2", + .base.cra_priority =3D 300, + .base.cra_flags =3D + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_blocksize =3D AES_BLOCK_SIZE, + .base.cra_ctxsize =3D sizeof(struct rk2_cipher_ctx), + .base.cra_alignmask =3D 0, + .base.cra_module =3D THIS_MODULE, + .init =3D rk2_cipher_tfm_init, + .exit =3D rk2_cipher_tfm_exit, + .min_keysize =3D AES_MIN_KEY_SIZE, + .max_keysize =3D AES_MAX_KEY_SIZE, + .setkey =3D rk2_aes_setkey, + .encrypt =3D rk2_skcipher_encrypt, + .decrypt =3D rk2_skcipher_decrypt, + }, + .alg.skcipher.op =3D { + .do_one_request =3D rk2_cipher_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_SKCIPHER, + .rk2_mode =3D RK2_CRYPTO_AES_CBC, + .alg.skcipher.base =3D { + .base.cra_name =3D "cbc(aes)", + .base.cra_driver_name =3D "cbc-aes-rk2", + .base.cra_priority =3D 300, + .base.cra_flags =3D + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_blocksize =3D AES_BLOCK_SIZE, + .base.cra_ctxsize =3D + sizeof(struct rk2_cipher_ctx), + .base.cra_alignmask =3D 0, + .base.cra_module =3D THIS_MODULE, + .init =3D rk2_cipher_tfm_init, + .exit =3D rk2_cipher_tfm_exit, + .min_keysize =3D AES_MIN_KEY_SIZE, + .max_keysize =3D AES_MAX_KEY_SIZE, + .ivsize =3D AES_BLOCK_SIZE, + .setkey =3D rk2_aes_setkey, + .encrypt =3D rk2_skcipher_encrypt, + .decrypt =3D rk2_skcipher_decrypt, + }, + .alg.skcipher.op =3D { + .do_one_request =3D rk2_cipher_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_SKCIPHER, + .rk2_mode =3D RK2_CRYPTO_AES_XTS, + .is_xts =3D true, + .alg.skcipher.base =3D { + .base.cra_name =3D "xts(aes)", + .base.cra_driver_name =3D "xts-aes-rk2", + .base.cra_priority =3D 300, + .base.cra_flags =3D + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_blocksize =3D AES_BLOCK_SIZE, + .base.cra_ctxsize =3D + sizeof(struct rk2_cipher_ctx), + .base.cra_alignmask =3D 0, + .base.cra_module =3D THIS_MODULE, + .init =3D rk2_cipher_tfm_init, + .exit =3D rk2_cipher_tfm_exit, + .min_keysize =3D AES_MIN_KEY_SIZE * 2, + .max_keysize =3D AES_MAX_KEY_SIZE * 2, + .ivsize =3D AES_BLOCK_SIZE, + .setkey =3D rk2_aes_xts_setkey, + .encrypt =3D rk2_skcipher_encrypt, + .decrypt =3D rk2_skcipher_decrypt, + }, + .alg.skcipher.op =3D { + .do_one_request =3D rk2_cipher_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_MD5, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D MD5_DIGEST_SIZE, + .statesize =3D sizeof(struct md5_state), + .base =3D { + .cra_name =3D "md5", + .cra_driver_name =3D "rk2-md5", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D + MD5_HMAC_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA1, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA1_DIGEST_SIZE, + .statesize =3D sizeof(struct sha1_state), + .base =3D { + .cra_name =3D "sha1", + .cra_driver_name =3D "rk2-sha1", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA1_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA224, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA224_DIGEST_SIZE, + .statesize =3D sizeof(struct sha256_state), + .base =3D { + .cra_name =3D "sha224", + .cra_driver_name =3D "rk2-sha224", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA256_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA256, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA256_DIGEST_SIZE, + .statesize =3D sizeof(struct sha256_state), + .base =3D { + .cra_name =3D "sha256", + .cra_driver_name =3D "rk2-sha256", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA256_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA384, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA384_DIGEST_SIZE, + .statesize =3D sizeof(struct sha512_state), + .base =3D { + .cra_name =3D "sha384", + .cra_driver_name =3D "rk2-sha384", + .cra_priority =3D 300, + .cra_flags =3D CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA384_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA512, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA512_DIGEST_SIZE, + .statesize =3D sizeof(struct sha512_state), + .base =3D { + .cra_name =3D "sha512", + .cra_driver_name =3D "rk2-sha512", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA512_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SM3, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SM3_DIGEST_SIZE, + .statesize =3D sizeof(struct sm3_state), + .base =3D { + .cra_name =3D "sm3", + .cra_driver_name =3D "rk2-sm3", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SM3_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, +}; + +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG +static int rk2_crypto_debugfs_stats_show(struct seq_file *seq, void *v) +{ + struct rk2_crypto_dev *rkc =3D seq->private; + unsigned int i; + + seq_printf(seq, "%s %s requests: %lu\n", + dev_driver_string(rkc->dev), dev_name(rkc->dev), rkc->nreq); + + for (i =3D 0; i < rkc->num_algs; i++) { + switch (rkc->algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + seq_printf(seq, "%s %s reqs=3D%lu fallback=3D%lu\n", + rkc->algs[i].alg.skcipher.base.base.cra_driver_name, + rkc->algs[i].alg.skcipher.base.base.cra_name, + rkc->algs[i].stat_req, rkc->algs[i].stat_fb); + seq_printf(seq, "\tfallback due to length: %lu\n", + rkc->algs[i].stat_fb_len); + seq_printf(seq, "\tfallback due to alignment: %lu\n", + rkc->algs[i].stat_fb_align); + seq_printf(seq, "\tfallback due to SGs: %lu\n", + rkc->algs[i].stat_fb_sgdiff); + break; + case CRYPTO_ALG_TYPE_AHASH: + seq_printf(seq, "%s %s reqs=3D%lu fallback=3D%lu\n", + rkc->algs[i].alg.hash.base.halg.base.cra_driver_name, + rkc->algs[i].alg.hash.base.halg.base.cra_name, + rkc->algs[i].stat_req, + rkc->algs[i].stat_fb); + break; + } + } + return 0; +} + +static int rk2_crypto_debugfs_info_show(struct seq_file *seq, void *d) +{ + struct rk2_crypto_dev *rkc =3D seq->private; + u32 v; + int err; + + err =3D pm_runtime_resume_and_get(rkc->dev); + if (err) + return err; + + v =3D readl(rkc->reg + RK2_CRYPTO_CLK_CTL); + seq_printf(seq, "CRYPTO_CLK_CTL %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_RST_CTL); + seq_printf(seq, "CRYPTO_RST_CTL %x\n", v); + v =3D readl(rkc->reg + CRYPTO_AES_VERSION); + seq_printf(seq, "CRYPTO_AES_VERSION %x\n", v); + if (v & BIT(17)) + seq_puts(seq, "AES 192\n"); + v =3D readl(rkc->reg + CRYPTO_DES_VERSION); + seq_printf(seq, "CRYPTO_DES_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_SM4_VERSION); + seq_printf(seq, "CRYPTO_SM4_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_HASH_VERSION); + seq_printf(seq, "CRYPTO_HASH_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_HMAC_VERSION); + seq_printf(seq, "CRYPTO_HMAC_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_RNG_VERSION); + seq_printf(seq, "CRYPTO_RNG_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_PKA_VERSION); + seq_printf(seq, "CRYPTO_PKA_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_CRYPTO_VERSION); + seq_printf(seq, "CRYPTO_CRYPTO_VERSION %x\n", v); + + pm_runtime_mark_last_busy(rkc->dev); + pm_runtime_put_autosuspend(rkc->dev); + + return 0; +} + +DEFINE_SHOW_ATTRIBUTE(rk2_crypto_debugfs_stats); +DEFINE_SHOW_ATTRIBUTE(rk2_crypto_debugfs_info); + +#endif + +static void register_debugfs(struct rk2_crypto_dev *rkc) +{ +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + /* Create a directory using the device name + * (e.g., /sys/kernel/debug/fe370000.crypto) + */ + rkc->dbgfs_dir =3D debugfs_create_dir(dev_name(rkc->dev), NULL); + + debugfs_create_file("stats", 0440, rkc->dbgfs_dir, rkc, + &rk2_crypto_debugfs_stats_fops); + debugfs_create_file("info", 0440, rkc->dbgfs_dir, rkc, + &rk2_crypto_debugfs_info_fops); +#endif +} + +static int rk2_crypto_register(struct rk2_crypto_dev *rkc) +{ + int i, k, err =3D 0; + + for (i =3D 0; i < rkc->num_algs; i++) { + rkc->algs[i].dev =3D rkc; /* Tie this alg copy to this device */ + switch (rkc->algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + err =3D crypto_engine_register_skcipher(&rkc->algs[i].alg.skcipher); + break; + case CRYPTO_ALG_TYPE_AHASH: + err =3D crypto_engine_register_ahash(&rkc->algs[i].alg.hash); + break; + } + if (err) + goto err_cipher_algs; + } + return 0; + + err_cipher_algs: + for (k =3D 0; k < i; k++) { + if (rkc->algs[k].type =3D=3D CRYPTO_ALG_TYPE_SKCIPHER) + crypto_engine_unregister_skcipher(&rkc->algs[k].alg.skcipher); + else + crypto_engine_unregister_ahash(&rkc->algs[k].alg.hash); + } + return err; +} + +static void rk2_crypto_unregister(struct rk2_crypto_dev *rkc) +{ + int i; + + for (i =3D 0; i < rkc->num_algs; i++) { + if (rkc->algs[i].type =3D=3D CRYPTO_ALG_TYPE_SKCIPHER) + crypto_engine_unregister_skcipher(&rkc->algs[i].alg.skcipher); + else + crypto_engine_unregister_ahash(&rkc->algs[i].alg.hash); + } +} + +static const struct of_device_id crypto_of_id_table[] =3D { + {.compatible =3D "rockchip,rk3568-crypto", + .data =3D &rk3568_variant, + }, + {.compatible =3D "rockchip,rk3588-crypto", + .data =3D &rk3588_variant, + }, + {} +}; + +MODULE_DEVICE_TABLE(of, crypto_of_id_table); + +static int rk2_crypto_probe(struct platform_device *pdev) +{ + struct device *dev =3D &pdev->dev; + struct rk2_crypto_dev *rkc; + int err =3D 0; + + rkc =3D devm_kzalloc(dev, sizeof(*rkc), GFP_KERNEL); + if (!rkc) + return -ENOMEM; + + rkc->dev =3D dev; + platform_set_drvdata(pdev, rkc); + + /* Duplicate the algorithms locally for this specific device */ + rkc->num_algs =3D ARRAY_SIZE(rk2_crypto_algs_template); + rkc->algs =3D devm_kmemdup(dev, rk2_crypto_algs_template, + sizeof(rk2_crypto_algs_template), GFP_KERNEL); + if (!rkc->algs) + return -ENOMEM; + + rkc->variant =3D of_device_get_match_data(dev); + if (!rkc->variant) + return dev_err_probe(dev, -EINVAL, "Missing variant\n"); + + rkc->rst =3D devm_reset_control_array_get_exclusive(dev); + if (IS_ERR(rkc->rst)) + return dev_err_probe(dev, PTR_ERR(rkc->rst), "Fail to get resets\n"); + + /* Manual DMA allocation requires manual cleanup in error paths */ + rkc->tl =3D dma_alloc_coherent(dev, + sizeof(struct rk2_crypto_lli) * MAX_LLI, + &rkc->t_phy, GFP_KERNEL); + + if (!rkc->tl) + return -ENOMEM; + + rkc->reg =3D devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(rkc->reg)) { + err =3D dev_err_probe(dev, PTR_ERR(rkc->reg), "Fail to get resources\n"); + goto err_dma; + } + + err =3D rk2_crypto_get_clks(rkc); + if (err) + goto err_dma; + + rkc->irq =3D platform_get_irq(pdev, 0); + if (rkc->irq < 0) { + err =3D dev_err_probe(dev, rkc->irq, "Interrupt is not available.\n"); + goto err_dma; + } + + err =3D devm_request_irq(dev, rkc->irq, + rk2_crypto_irq_handle, IRQF_SHARED, + "rk-crypto", pdev); + + if (err) { + err =3D dev_err_probe(dev, err, "irq request failed.\n"); + goto err_dma; + } + + rkc->engine =3D crypto_engine_alloc_init(dev, true); + if (!rkc->engine) { + err =3D -ENOMEM; + goto err_dma; + } + + err =3D crypto_engine_start(rkc->engine); + if (err) { + err =3D dev_err_probe(dev, err, "Failed to start crypto engine\n"); + goto err_engine; + } + + init_completion(&rkc->complete); + + err =3D rk2_crypto_pm_init(rkc); + if (err) { + err =3D dev_err_probe(dev, err, "Failed to initialize runtime PM\n"); + goto err_engine; + } + + err =3D pm_runtime_resume_and_get(dev); + if (err) { + err =3D dev_err_probe(dev, err, "Failed to resume device\n"); + goto err_pm; + } + + /* Register algorithms specific to THIS device */ + err =3D rk2_crypto_register(rkc); + if (err) { + err =3D dev_err_probe(dev, err, "Fail to register crypto algorithms\n"); + goto err_pm_put; + } + + register_debugfs(rkc); + + pm_runtime_mark_last_busy(dev); + pm_runtime_put_autosuspend(dev); + + return 0; + + err_pm_put: + pm_runtime_put_sync(dev); + err_pm: + rk2_crypto_pm_exit(rkc); + err_engine: + crypto_engine_exit(rkc->engine); + err_dma: + dma_free_coherent(dev, sizeof(struct rk2_crypto_lli) * MAX_LLI, + rkc->tl, rkc->t_phy); + return err; +} + +static void rk2_crypto_remove(struct platform_device *pdev) +{ + struct rk2_crypto_dev *rkc =3D platform_get_drvdata(pdev); + + /* Stop engine to prevent new requests */ + crypto_engine_stop(rkc->engine); + + /* Unregister algorithms for this specific device */ + rk2_crypto_unregister(rkc); + +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + debugfs_remove_recursive(rkc->dbgfs_dir); +#endif + + /* Safe to kill the engine completely */ + crypto_engine_exit(rkc->engine); + + rk2_crypto_pm_exit(rkc); + dma_free_coherent(rkc->dev, sizeof(struct rk2_crypto_lli) * MAX_LLI, + rkc->tl, rkc->t_phy); +} + +static struct platform_driver crypto_driver =3D { + .probe =3D rk2_crypto_probe, + .remove =3D rk2_crypto_remove, + .driver =3D { + .name =3D "rk2-crypto", + .pm =3D pm_ptr(&rk2_crypto_pm_ops), + .of_match_table =3D crypto_of_id_table, + }, +}; + +module_platform_driver(crypto_driver); + +MODULE_DESCRIPTION("Rockchip Crypto Engine cryptographic offloader"); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Corentin Labbe "); +MODULE_AUTHOR("Dawid Olesinski "); + diff --git a/drivers/crypto/rockchip/rk2_crypto.h b/drivers/crypto/rockchip= /rk2_crypto.h new file mode 100644 index 000000000000..40e20235cf7e --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto.h @@ -0,0 +1,243 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __RK2_CRYPTO_H__ +#define __RK2_CRYPTO_H__ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define RK2_CRYPTO_CLK_CTL 0x0000 +#define RK2_CRYPTO_RST_CTL 0x0004 + +#define RK2_CRYPTO_DMA_INT_EN 0x0008 + +/* RK2_CRYPTO_DMA_INT_ST / RK2_CRYPTO_DMA_INT_EN bit definitions */ +#define RK2_CRYPTO_DMA_INT_LISTDONE BIT(0) /* LLI list complete */ +#define RK2_CRYPTO_DMA_INT_SRC_ITEM_INT BIT(1) /* per-entry src interru= pt */ +#define RK2_CRYPTO_DMA_INT_DST_ITEM_INT BIT(2) /* per-entry dst interru= pt */ +#define RK2_CRYPTO_DMA_INT_LIST_SRC_ERR BIT(3) /* LLI src error */ +#define RK2_CRYPTO_DMA_INT_LIST_DST_ERR BIT(4) /* LLI dst error */ +#define RK2_CRYPTO_DMA_INT_SRC_ERR BIT(5) /* DMA src error */ +#define RK2_CRYPTO_DMA_INT_DST_ERR BIT(6) /* DMA dst error */ + +#define RK2_CRYPTO_DMA_INT_ERR_MASK (RK2_CRYPTO_DMA_INT_LIST_SRC_ERR = | \ + RK2_CRYPTO_DMA_INT_LIST_DST_ERR | \ + RK2_CRYPTO_DMA_INT_SRC_ERR | \ + RK2_CRYPTO_DMA_INT_DST_ERR) + +#define RK2_CRYPTO_DMA_INT_ALL_MASK 0x7F + +/* The DMA interrupt enable register uses the upper 16 bits as a write-ena= ble mask. + * To enable bits 0-6, we must write 1s to bits 16-22 as well. + */ +#define RK2_CRYPTO_DMA_INT_ENABLE_ALL ((RK2_CRYPTO_DMA_INT_ALL_MASK << 16)= | \ + RK2_CRYPTO_DMA_INT_ALL_MASK) + +/* values in RK2_CRYPTO_DMA_INT_ST are the same than in RK2_CRYPTO_DMA_INT= _EN */ +#define RK2_CRYPTO_DMA_INT_ST 0x000C + +#define RK2_CRYPTO_DMA_CTL 0x0010 +#define RK2_CRYPTO_DMA_CTL_START BIT(0) + +#define RK2_CRYPTO_DMA_LLI_ADDR 0x0014 +#define RK2_CRYPTO_DMA_ST 0x0018 +#define RK2_CRYPTO_DMA_STATE 0x001C +#define RK2_CRYPTO_DMA_LLI_RADDR 0x0020 +#define RK2_CRYPTO_DMA_SRC_RADDR 0x0024 +#define RK2_CRYPTO_DMA_DST_WADDR 0x0028 +#define RK2_CRYPTO_DMA_ITEM_ID 0x002C + +#define RK2_CRYPTO_FIFO_CTL 0x0040 + +#define RK2_CRYPTO_BC_CTL 0x0044 +#define RK2_CRYPTO_AES (0 << 8) +#define RK2_CRYPTO_MODE_ECB (0 << 4) +#define RK2_CRYPTO_MODE_CBC (1 << 4) +#define RK2_CRYPTO_XTS (6 << 4) + +#define RK2_CRYPTO_HASH_CTL 0x0048 +#define RK2_CRYPTO_HW_PAD BIT(2) +#define RK2_CRYPTO_SHA1 (0 << 4) +#define RK2_CRYPTO_MD5 (1 << 4) +#define RK2_CRYPTO_SHA224 (3 << 4) +#define RK2_CRYPTO_SHA256 (2 << 4) +#define RK2_CRYPTO_SHA384 (9 << 4) +#define RK2_CRYPTO_SHA512 (8 << 4) +#define RK2_CRYPTO_SM3 (4 << 4) + +#define RK2_CRYPTO_AES_ECB (RK2_CRYPTO_AES | RK2_CRYPTO_MODE_ECB) +#define RK2_CRYPTO_AES_CBC (RK2_CRYPTO_AES | RK2_CRYPTO_MODE_CBC) +#define RK2_CRYPTO_AES_XTS (RK2_CRYPTO_AES | RK2_CRYPTO_XTS) +#define RK2_CRYPTO_AES_128BIT_key (0 << 2) +#define RK2_CRYPTO_AES_192BIT_key (1 << 2) +#define RK2_CRYPTO_AES_256BIT_key (2 << 2) + +#define RK2_CRYPTO_DEC BIT(1) +#define RK2_CRYPTO_ENABLE BIT(0) + +#define RK2_CRYPTO_CIPHER_ST 0x004C +#define RK2_CRYPTO_CIPHER_STATE 0x0050 + +#define RK2_CRYPTO_CH0_IV_0 0x0100 + +#define RK2_CRYPTO_KEY0 0x0180 +#define RK2_CRYPTO_KEY1 0x0184 +#define RK2_CRYPTO_KEY2 0x0188 +#define RK2_CRYPTO_KEY3 0x018C +#define RK2_CRYPTO_KEY4 0x0190 +#define RK2_CRYPTO_KEY5 0x0194 +#define RK2_CRYPTO_KEY6 0x0198 +#define RK2_CRYPTO_KEY7 0x019C +#define RK2_CRYPTO_CH4_KEY0 0x01c0 + +#define RK2_CRYPTO_CH0_PC_LEN_0 0x0280 + +#define RK2_CRYPTO_CH0_IV_LEN 0x0300 + +#define RK2_CRYPTO_HASH_DOUT_0 0x03A0 +#define RK2_CRYPTO_HASH_VALID 0x03E4 + +#define RK2_CRYPTO_TRNG_CTL 0x0400 +#define RK2_CRYPTO_TRNG_START BIT(0) +#define RK2_CRYPTO_TRNG_ENABLE BIT(1) +#define RK2_CRYPTO_TRNG_256 (0x3 << 4) +#define RK2_CRYPTO_TRNG_SAMPLE_CNT 0x0404 +#define RK2_CRYPTO_TRNG_DOUT 0x0410 + +#define CRYPTO_AES_VERSION 0x0680 +#define CRYPTO_DES_VERSION 0x0684 +#define CRYPTO_SM4_VERSION 0x0688 +#define CRYPTO_HASH_VERSION 0x068C +#define CRYPTO_HMAC_VERSION 0x0690 +#define CRYPTO_RNG_VERSION 0x0694 +#define CRYPTO_PKA_VERSION 0x0698 +#define CRYPTO_CRYPTO_VERSION 0x06F0 + +#define RK2_LLI_DMA_CTRL_SRC_INT BIT(10) +#define RK2_LLI_DMA_CTRL_DST_INT BIT(9) +#define RK2_LLI_DMA_CTRL_LIST_INT BIT(8) +#define RK2_LLI_DMA_CTRL_LAST BIT(0) + +#define RK2_LLI_STRING_LAST BIT(2) +#define RK2_LLI_STRING_FIRST BIT(1) +#define RK2_LLI_CIPHER_START BIT(0) + +#define MAX_LLI 20 + +struct rk2_crypto_lli { + __le32 src_addr; + __le32 src_len; + __le32 dst_addr; + __le32 dst_len; + __le32 user; + __le32 iv; + __le32 dma_ctrl; + __le32 next; +}; + +struct rk2_variant { + int num_clks; +}; + +struct rk2_crypto_dev { + struct device *dev; + struct clk_bulk_data *clks; + int num_clks; + struct reset_control *rst; + void __iomem *reg; + int irq; + const struct rk2_variant *variant; + unsigned long nreq; + struct crypto_engine *engine; + struct completion complete; + int status; + struct rk2_crypto_lli *tl; + dma_addr_t t_phy; + struct rk2_crypto_template *algs; + int num_algs; +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + struct dentry *dbgfs_dir; +#endif +}; + +/* the private variable of hash */ +struct rk2_ahash_ctx { + /* for fallback */ + struct crypto_ahash *fallback_tfm; +}; + +/* the private variable of hash for fallback */ +struct rk2_ahash_rctx { + struct rk2_crypto_dev *dev; + u32 mode; + int nrsgs; + struct ahash_request fallback_req; +}; + +/* the private variable of cipher */ +struct rk2_cipher_ctx { + unsigned int keylen; + u8 key[AES_MAX_KEY_SIZE * 2]; + struct crypto_skcipher *fallback_tfm; +}; + +struct rk2_cipher_rctx { + struct rk2_crypto_dev *dev; + u8 backup_iv[AES_BLOCK_SIZE]; + u32 mode; + /* must be last, see __ctx placement */ + struct skcipher_request fallback_req; +}; + +struct rk2_crypto_template { + u32 type; + u32 rk2_mode; + bool is_xts; + struct rk2_crypto_dev *dev; + union { + struct skcipher_engine_alg skcipher; + struct ahash_engine_alg hash; + } alg; + unsigned long stat_req; + unsigned long stat_fb; + unsigned long stat_fb_len; + unsigned long stat_fb_sglen; + unsigned long stat_fb_align; + unsigned long stat_fb_sgdiff; +}; + +int rk2_cipher_run(struct crypto_engine *engine, void *async_req); +int rk2_hash_run(struct crypto_engine *engine, void *breq); + +int rk2_cipher_tfm_init(struct crypto_skcipher *tfm); +void rk2_cipher_tfm_exit(struct crypto_skcipher *tfm); +int rk2_aes_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen); +int rk2_aes_xts_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen); +int rk2_skcipher_encrypt(struct skcipher_request *req); +int rk2_skcipher_decrypt(struct skcipher_request *req); + +int rk2_ahash_init(struct ahash_request *req); +int rk2_ahash_update(struct ahash_request *req); +int rk2_ahash_final(struct ahash_request *req); +int rk2_ahash_finup(struct ahash_request *req); +int rk2_ahash_import(struct ahash_request *req, const void *in); +int rk2_ahash_export(struct ahash_request *req, void *out); +int rk2_ahash_digest(struct ahash_request *req); +int rk2_hash_init_tfm(struct crypto_ahash *tfm); +void rk2_hash_exit_tfm(struct crypto_ahash *tfm); + +#endif /* __RK2_CRYPTO_H__ */ diff --git a/drivers/crypto/rockchip/rk2_crypto_ahash.c b/drivers/crypto/ro= ckchip/rk2_crypto_ahash.c new file mode 100644 index 000000000000..5aeff32d1402 --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto_ahash.c @@ -0,0 +1,547 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Crypto offloader support for Rockchip RK3568/RK3588 + * + * Copyright (c) 2022-2023 Corentin Labbe + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "rk2_crypto.h" + +static bool rk2_ahash_need_fallback(struct ahash_request *areq) +{ + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(areq); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + struct scatterlist *sg; + int nents =3D sg_nents_for_len(areq->src, areq->nbytes); + + /* + * The hardware's Merkle-Damgard padding engine (HW_PAD) requires the + * total message length to be known upfront via CH0_PC_LEN_0. When a + * request spans multiple LLI descriptors, the hardware either treats + * each descriptor as an independent padded message, or loses running + * hash state at descriptor boundaries. Either way the result is a + * wrong digest. This behaviour is not documented in the RK3588 TRM, + * which advertises LLI chaining but does not specify whether hash + * operations may span multiple linked descriptors. + * Work around this by falling back to software for any multi-SG + * request. Single-SG requests with HW_PAD work correctly. + */ + + if (nents < 0) { + dev_err(algt->dev->dev, "Invalid SG list: length mismatch\n"); + return true; /* force fallback safely */ + } + if (nents > 1) { + algt->stat_fb_sgdiff++; + return true; + } + + sg =3D areq->src; + while (sg) { + if (!IS_ALIGNED(sg->offset, sizeof(u32))) { + algt->stat_fb_align++; + return true; + } + if (sg->length % 4) { + algt->stat_fb_sglen++; + return true; + } + sg =3D sg_next(sg); + } + return false; +} + +static int rk2_ahash_digest_fb(struct ahash_request *areq) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(areq); + struct rk2_ahash_ctx *tfmctx =3D crypto_ahash_ctx(tfm); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + + algt->stat_fb++; + + ahash_request_set_tfm(&rctx->fallback_req, tfmctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, areq->base.flags, + areq->base.complete, areq->base.data); + + rctx->fallback_req.nbytes =3D areq->nbytes; + rctx->fallback_req.src =3D areq->src; + rctx->fallback_req.result =3D areq->result; + + return crypto_ahash_digest(&rctx->fallback_req); +} + +static int zero_message_process(struct ahash_request *req) +{ + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + int digestsize =3D crypto_ahash_digestsize(tfm); + + switch (algt->rk2_mode) { + case RK2_CRYPTO_SHA1: + memcpy(req->result, sha1_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SHA256: + memcpy(req->result, sha256_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SHA384: + memcpy(req->result, sha384_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SHA512: + memcpy(req->result, sha512_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_MD5: + memcpy(req->result, md5_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SM3: + memcpy(req->result, sm3_zero_message_hash, digestsize); + break; + default: + return -EINVAL; + } + + return 0; +} + +/** + * rk2_ahash_init() - Initialize context for a hash request + * @req: The asynchronous hash request structure. + * + * Initializes the software fallback context. The physical hardware engine + * is only utilized during atomic digest operations. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_init(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + return crypto_ahash_init(&rctx->fallback_req); +} + +/** + * rk2_ahash_update() - Feed a message block into the hash stream + * @req: The asynchronous hash request structure. + * + * Passes the message block to the software fallback. The hardware engine + * does not support fragmented streaming updates. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_update(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + rctx->fallback_req.nbytes =3D req->nbytes; + rctx->fallback_req.src =3D req->src; + + return crypto_ahash_update(&rctx->fallback_req); +} + +/** + * rk2_ahash_final() - Finalize the hashing operation + * @req: The asynchronous hash request structure. + * + * Finalizes the hash and extracts the digest via the software fallback. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_final(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + rctx->fallback_req.result =3D req->result; + + return crypto_ahash_final(&rctx->fallback_req); +} + +/** + * rk2_ahash_finup() - Perform update and final hash operations sequential= ly + * @req: The asynchronous hash request structure. + * + * Convenience wrapper that performs update and final operations + * via the software fallback. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_finup(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + rctx->fallback_req.nbytes =3D req->nbytes; + rctx->fallback_req.src =3D req->src; + rctx->fallback_req.result =3D req->result; + + return crypto_ahash_finup(&rctx->fallback_req); +} + +/** + * rk2_ahash_import() - Restore a saved hash context + * @req: The target asynchronous hash request structure. + * @in: Buffer containing the previously exported state. + * + * Restores the software fallback state from an export block. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_import(struct ahash_request *req, const void *in) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + return crypto_ahash_import(&rctx->fallback_req, in); +} + +/** + * rk2_ahash_export() - Serialize an active hash context + * @req: The source asynchronous hash request structure. + * @out: Destination buffer where the state will be written. + * + * Freezes the progression of the software fallback stream into a byte arr= ay. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_export(struct ahash_request *req, void *out) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + return crypto_ahash_export(&rctx->fallback_req, out); +} + +/** + * rk2_ahash_digest() - Compute a complete message digest in a single tran= saction + * @req: The asynchronous hash request structure. + * + * Evaluates hardware constraints (e.g., scatterlist alignment) and either + * routes the atomic request to the hardware engine or diverts to the fall= back. + * + * Return: 0 on synchronous completion, -EINPROGRESS if submitted to the + * hardware engine, or a negative error code on failure. + */ +int rk2_ahash_digest(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt; + struct rk2_crypto_dev *rkc; + struct crypto_engine *engine; + + if (!req->nbytes) + return zero_message_process(req); + + if (rk2_ahash_need_fallback(req)) + return rk2_ahash_digest_fb(req); + + /* Extract the device pointer from the algorithm template! */ + algt =3D container_of(alg, struct rk2_crypto_template, alg.hash.base); + rkc =3D algt->dev; + if (!rkc) + return -ENODEV; + + rctx->dev =3D rkc; + engine =3D rkc->engine; + + return crypto_transfer_hash_request_to_engine(engine, req); +} + +static int rk2_hash_prepare(struct crypto_engine *engine, void *breq) +{ + struct ahash_request *areq =3D + container_of(breq, struct ahash_request, base); + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct rk2_crypto_dev *rkc =3D rctx->dev; + int n =3D sg_nents_for_len(areq->src, areq->nbytes); + int ret; + + if (n < 0) { + dev_err(rkc->dev, "SG list too short for %u bytes\n", areq->nbytes); + return -EINVAL; + } + rctx->nrsgs =3D n; + ret =3D dma_map_sg(rkc->dev, areq->src, rctx->nrsgs, DMA_TO_DEVICE); + if (ret <=3D 0) { + /* + * clear nrsgs on map failure to prevent spurious unmap in unprepare + */ + rctx->nrsgs =3D 0; + return -EINVAL; + } + return 0; +} + +static void rk2_hash_unprepare(struct crypto_engine *engine, void *breq) +{ + struct ahash_request *areq =3D + container_of(breq, struct ahash_request, base); + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct rk2_crypto_dev *rkc =3D rctx->dev; + + if (rctx->nrsgs) + dma_unmap_sg(rkc->dev, areq->src, rctx->nrsgs, DMA_TO_DEVICE); +} + +/** + * rk2_hash_run() - Execute an asynchronous hash request via the hardware + * @engine: The crypto engine queue managing this request. + * @breq: The asynchronous hash request to process. + * + * Configures the hardware hash engine, programs DMA block descriptors, + * and copies the final digest back from the hardware registers. + * + * Return: Always 0. Errors are reported through the crypto engine + * finalization callback. + */ +int rk2_hash_run(struct crypto_engine *engine, void *breq) +{ + struct ahash_request *areq =3D + container_of(breq, struct ahash_request, base); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(areq); + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + struct scatterlist *sgs =3D areq->src; + struct rk2_crypto_dev *rkc =3D rctx->dev; + struct rk2_crypto_lli *dd =3D &rkc->tl[0]; + int ddi =3D 0; + int err =3D 0; + unsigned int len =3D areq->nbytes; + unsigned int todo; + unsigned long timeout; + u32 v; + int i; + + err =3D rk2_hash_prepare(engine, breq); + if (err) + goto exit_unmap; + + err =3D pm_runtime_resume_and_get(rkc->dev); + if (err) + goto exit_unmap; + + dev_dbg(rkc->dev, "%s %s len=3D%u\n", __func__, + crypto_tfm_alg_name(areq->base.tfm), areq->nbytes); + + algt->stat_req++; + rkc->nreq++; + + /* the upper bits are a write enable mask, so we need to write 1 to all + * upper 16 bits to allow write to the 16 lower bits + */ + rctx->mode =3D algt->rk2_mode; + rctx->mode |=3D 0xffff0000; + rctx->mode |=3D RK2_CRYPTO_ENABLE | RK2_CRYPTO_HW_PAD; + writel(rctx->mode, rkc->reg + RK2_CRYPTO_HASH_CTL); + + while (sgs && len > 0) { + if (ddi >=3D MAX_LLI) { + dev_err(rkc->dev, + "Too many SG entries (current: %d, max: %d)\n", + ddi, MAX_LLI); + err =3D -EINVAL; + goto exit; + } + dd =3D &rkc->tl[ddi]; + + todo =3D min(sg_dma_len(sgs), len); + dd->src_addr =3D sg_dma_address(sgs); + dd->src_len =3D todo; + dd->dst_addr =3D 0; + dd->dst_len =3D 0; + dd->dma_ctrl =3D ddi << 24; + dd->iv =3D 0; + dd->next =3D + rkc->t_phy + sizeof(struct rk2_crypto_lli) * (ddi + 1); + + if (ddi =3D=3D 0) + dd->user =3D RK2_LLI_CIPHER_START | RK2_LLI_STRING_FIRST; + else + dd->user =3D 0; + + len -=3D todo; + if (len =3D=3D 0) { + dd->user |=3D RK2_LLI_STRING_LAST; + dd->dma_ctrl |=3D RK2_LLI_DMA_CTRL_LAST | + RK2_LLI_DMA_CTRL_SRC_INT | + RK2_LLI_DMA_CTRL_LIST_INT; + } + dev_dbg(rkc->dev, + "HASH SG %d sglen=3D%u user=3D%x dma=3D%x mode=3D%x len=3D%u todo=3D%u = phy=3D%pad\n", + ddi, sgs->length, dd->user, dd->dma_ctrl, rctx->mode, + len, todo, &rkc->t_phy); + + sgs =3D sg_next(sgs); + ddi++; + } + + /* + * next is ignored by hardware when RK2_LLI_DMA_CTRL_LAST is set in + * dma_ctrl. Set it to an obviously-invalid-but-non-zero sentinel so + * it stands out if ever read in a debug dump. + */ + dd->next =3D 1; + + /* Program total payload length for hardware padding */ + writel(areq->nbytes, rkc->reg + RK2_CRYPTO_CH0_PC_LEN_0); + + /* Clear stale interrupts, then enable with proper write-mask */ + writel(RK2_CRYPTO_DMA_INT_ALL_MASK, rkc->reg + RK2_CRYPTO_DMA_INT_ST); + writel(RK2_CRYPTO_DMA_INT_ENABLE_ALL, rkc->reg + RK2_CRYPTO_DMA_INT_EN); + + writel(rkc->t_phy, rkc->reg + RK2_CRYPTO_DMA_LLI_ADDR); + + reinit_completion(&rkc->complete); + rkc->status =3D 0; + + writel(RK2_CRYPTO_DMA_CTL_START | (RK2_CRYPTO_DMA_CTL_START << 16), + rkc->reg + RK2_CRYPTO_DMA_CTL); + + timeout =3D wait_for_completion_timeout(&rkc->complete, + msecs_to_jiffies(2000)); + if (!timeout) { + dev_err(rkc->dev, "DMA timeout\n"); + err =3D -ETIMEDOUT; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + goto exit; + } + if (!rkc->status) { + dev_err(rkc->dev, "DMA error\n"); + err =3D -EIO; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + goto exit; + } + + err =3D + readl_poll_timeout_atomic(rkc->reg + RK2_CRYPTO_HASH_VALID, v, + v =3D=3D 1, 10, 1000); + if (err) { + dev_err(rkc->dev, "Hash result not valid\n"); + goto exit; + } + + /* + * Hardware outputs digest words in big-endian format. + * Because readl() performs a native little-endian read, + * put_unaligned_be32() is used to store the result correctly + * into the byte array. + */ + for (i =3D 0; i < crypto_ahash_digestsize(tfm) / 4; i++) { + v =3D readl(rkc->reg + RK2_CRYPTO_HASH_DOUT_0 + i * 4); + put_unaligned_be32(v, areq->result + i * 4); + } + exit: + writel(0xffff0000, rkc->reg + RK2_CRYPTO_HASH_CTL); + pm_runtime_mark_last_busy(rkc->dev); + pm_runtime_put_autosuspend(rkc->dev); + + exit_unmap: + rk2_hash_unprepare(engine, breq); + local_bh_disable(); + crypto_finalize_hash_request(engine, breq, err); + local_bh_enable(); + + return 0; +} + +/** + * rk2_hash_init_tfm() - Initialize the transformation context + * @tfm: The crypto ahash handle. + * + * Allocates software fallback transformations required to guarantee + * processing integrity when hardware constraints are violated. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_hash_init_tfm(struct crypto_ahash *tfm) +{ + struct rk2_ahash_ctx *tctx =3D crypto_ahash_ctx(tfm); + const char *alg_name =3D crypto_ahash_alg_name(tfm); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + unsigned int fallback_statesize; + + tctx->fallback_tfm =3D crypto_alloc_ahash(alg_name, 0, + CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(tctx->fallback_tfm)) { + dev_err(algt->dev->dev, "Could not load fallback driver.\n"); + return PTR_ERR(tctx->fallback_tfm); + } + + /* Promote statesize if fallback needs more space for export/import */ + fallback_statesize =3D crypto_ahash_statesize(tctx->fallback_tfm); + if (fallback_statesize > crypto_ahash_statesize(tfm)) + crypto_ahash_set_statesize(tfm, fallback_statesize); + + crypto_ahash_set_reqsize(tfm, + sizeof(struct rk2_ahash_rctx) + + crypto_ahash_reqsize(tctx->fallback_tfm)); + return 0; +} + +/** + * rk2_hash_exit_tfm() - Clean up an ahash transformation context + * @tfm: The crypto ahash handle. + * + * Safely frees internal software fallback transformations. + */ +void rk2_hash_exit_tfm(struct crypto_ahash *tfm) +{ + struct rk2_ahash_ctx *tctx =3D crypto_ahash_ctx(tfm); + + crypto_free_ahash(tctx->fallback_tfm); +} diff --git a/drivers/crypto/rockchip/rk2_crypto_skcipher.c b/drivers/crypto= /rockchip/rk2_crypto_skcipher.c new file mode 100644 index 000000000000..e1a1a1a13096 --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto_skcipher.c @@ -0,0 +1,724 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * hardware cryptographic offloader for RK3568/RK3588 SoC + * + * Copyright (c) 2022-2023 Corentin Labbe + */ +#include +#include +#include +#include +#include +#include +#include +#include "rk2_crypto.h" + +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG +static void rk2_print(struct rk2_crypto_dev *rkc) +{ + u32 v; + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_ST); + dev_info(rkc->dev, "DMA_ST %x\n", v); + switch (v) { + case 0: + dev_info(rkc->dev, "DMA_ST: DMA IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_ST: DMA BUSY\n"); + break; + default: + dev_err(rkc->dev, "DMA_ST: invalid value\n"); + } + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_STATE); + dev_info(rkc->dev, "DMA_STATE %x\n", v); + + switch (v & 0x3) { + case 0: + dev_info(rkc->dev, "DMA_STATE: DMA DST IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_STATE: DMA DST LOAD\n"); + break; + case 2: + dev_info(rkc->dev, "DMA_STATE: DMA DST WORK\n"); + break; + default: + dev_err(rkc->dev, "DMA DST invalid\n"); + break; + } + switch ((v >> 2) & 0x3) { + case 0: + dev_info(rkc->dev, "DMA_STATE: DMA SRC IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_STATE: DMA SRC LOAD\n"); + break; + case 2: + dev_info(rkc->dev, "DMA_STATE: DMA SRC WORK\n"); + break; + default: + dev_err(rkc->dev, "DMA_STATE: DMA SRC invalid\n"); + break; + } + switch ((v >> 4) & 0x3) { + case 0: + dev_info(rkc->dev, "DMA_STATE: DMA LLI IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_STATE: DMA LLI LOAD\n"); + break; + case 2: + dev_info(rkc->dev, "DMA_STATE: LLI WORK\n"); + break; + default: + dev_err(rkc->dev, "DMA_STATE: LLI invalid\n"); + break; + } + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_LLI_RADDR); + dev_info(rkc->dev, "DMA_LLI_RADDR %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_SRC_RADDR); + dev_info(rkc->dev, "DMA_SRC_RADDR %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_DST_WADDR); + dev_info(rkc->dev, "DMA_DST_WADDR %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_ITEM_ID); + dev_info(rkc->dev, "DMA_ITEM_ID %x\n", v); + + v =3D readl(rkc->reg + RK2_CRYPTO_CIPHER_ST); + dev_info(rkc->dev, "CIPHER_ST %x\n", v); + if (v & BIT(0)) + dev_info(rkc->dev, "CIPHER_ST: BLOCK CIPHER BUSY\n"); + else + dev_info(rkc->dev, "CIPHER_ST: BLOCK CIPHER IDLE\n"); + if (v & BIT(2)) + dev_info(rkc->dev, "CIPHER_ST: HASH BUSY\n"); + else + dev_info(rkc->dev, "CIPHER_ST: HASH IDLE\n"); + if (v & BIT(3)) + dev_info(rkc->dev, "CIPHER_ST: OTP KEY VALID\n"); + else + dev_info(rkc->dev, "CIPHER_ST: OTP KEY INVALID\n"); + + v =3D readl(rkc->reg + RK2_CRYPTO_CIPHER_STATE); + dev_info(rkc->dev, "CIPHER_STATE %x\n", v); + switch (v & 0x3) { + case 0: + dev_info(rkc->dev, "serial: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "serial: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "serial: BULK state\n"); + break; + default: + dev_info(rkc->dev, "serial: reserved state\n"); + break; + } + switch ((v >> 2) & 0x3) { + case 0: + dev_info(rkc->dev, "mac_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "mac_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "mac_state: BULK state\n"); + break; + default: + dev_info(rkc->dev, "mac_state: reserved state\n"); + break; + } + switch ((v >> 4) & 0x3) { + case 0: + dev_info(rkc->dev, "parallel_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "parallel_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "parallel_state: BULK state\n"); + break; + default: + dev_info(rkc->dev, "parallel_state: reserved state\n"); + break; + } + switch ((v >> 6) & 0x3) { + case 0: + dev_info(rkc->dev, "ccm_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "ccm_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "ccm_state: NA state\n"); + break; + default: + dev_info(rkc->dev, "ccm_state: reserved state\n"); + break; + } + switch ((v >> 8) & 0xF) { + case 0: + dev_info(rkc->dev, "gcm_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "gcm_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "gcm_state: NA state\n"); + break; + case 3: + dev_info(rkc->dev, "gcm_state: PC state\n"); + break; + } + switch ((v >> 10) & 0x1F) { + case 0x1: + dev_info(rkc->dev, "hash_state: IDLE state\n"); + break; + case 0x2: + dev_info(rkc->dev, "hash_state: IPAD state\n"); + break; + case 0x4: + dev_info(rkc->dev, "hash_state: TEXT state\n"); + break; + case 0x8: + dev_info(rkc->dev, "hash_state: OPAD state\n"); + break; + case 0x10: + dev_info(rkc->dev, "hash_state: OPAD EXT state\n"); + break; + default: + dev_info(rkc->dev, "hash_state: invalid state\n"); + break; + } + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_INT_ST); + dev_info(rkc->dev, "RK2_CRYPTO_DMA_INT_ST %x\n", v); +} +#endif + +static int rk2_cipher_need_fallback(struct skcipher_request *req) +{ + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + struct scatterlist *sgs, *sgd; + unsigned int stodo, dtodo, len; + unsigned int bs =3D crypto_skcipher_blocksize(tfm); + + if (!req->cryptlen) + return true; + + /* + * The hardware XTS implementation programs the tweak once before + * DMA starts and cannot update it at SG boundaries. Restrict to + * exactly one source and one destination SG entry. + */ + if (algt->is_xts) { + if (sg_nents_for_len(req->src, req->cryptlen) !=3D 1) + return true; + if (sg_nents_for_len(req->dst, req->cryptlen) !=3D 1) + return true; + } + + len =3D req->cryptlen; + sgs =3D req->src; + sgd =3D req->dst; + + while (len > 0 && sgs && sgd) { + if (!IS_ALIGNED(sgs->offset, sizeof(u32))) { + algt->stat_fb_align++; + return true; + } + if (!IS_ALIGNED(sgd->offset, sizeof(u32))) { + algt->stat_fb_align++; + return true; + } + + stodo =3D min(len, sgs->length); + if (stodo % bs) { + algt->stat_fb_len++; + return true; + } + + dtodo =3D min(len, sgd->length); + if (dtodo % bs) { + algt->stat_fb_len++; + return true; + } + + /* DMA engines usually require symmetrical source/destination chunks */ + if (stodo !=3D dtodo) { + algt->stat_fb_sgdiff++; + return true; + } + + len -=3D stodo; + sgs =3D sg_next(sgs); + sgd =3D sg_next(sgd); + } + + /* If len > 0, the scatterlist was too short for the request */ + return len > 0; +} + +static int rk2_cipher_fallback(struct skcipher_request *areq) +{ + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(areq); + struct rk2_cipher_ctx *op =3D crypto_skcipher_ctx(tfm); + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(areq); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + int err; + + algt->stat_fb++; + + skcipher_request_set_tfm(&rctx->fallback_req, op->fallback_tfm); + skcipher_request_set_callback(&rctx->fallback_req, areq->base.flags, + areq->base.complete, areq->base.data); + skcipher_request_set_crypt(&rctx->fallback_req, areq->src, areq->dst, + areq->cryptlen, areq->iv); + + if (rctx->mode & RK2_CRYPTO_DEC) + err =3D crypto_skcipher_decrypt(&rctx->fallback_req); + else + err =3D crypto_skcipher_encrypt(&rctx->fallback_req); + return err; +} + +static int rk2_cipher_handle_req(struct skcipher_request *req) +{ + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(req); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + struct rk2_crypto_dev *rkc; + struct crypto_engine *engine; + + if (algt->is_xts && ctx->keylen =3D=3D AES_KEYSIZE_192 * 2) + return rk2_cipher_fallback(req); + + if (rk2_cipher_need_fallback(req)) + return rk2_cipher_fallback(req); + + rkc =3D algt->dev; + if (!rkc) + return -ENODEV; + engine =3D rkc->engine; + rctx->dev =3D rkc; + + return crypto_transfer_skcipher_request_to_engine(engine, req); +} + +/** + * rk2_aes_setkey() - Configure the key for standard AES algorithms + * @cipher: The crypto skcipher handle. + * @key: Buffer containing the raw key material. + * @keylen: Length of the key in bytes. + * + * Validates key length, stores the key in the context, and configures + * the software fallback transformation. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_aes_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen) +{ + struct crypto_tfm *tfm =3D crypto_skcipher_tfm(cipher); + struct rk2_cipher_ctx *ctx =3D crypto_tfm_ctx(tfm); + + if (keylen !=3D AES_KEYSIZE_128 && keylen !=3D AES_KEYSIZE_192 && + keylen !=3D AES_KEYSIZE_256) + return -EINVAL; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + crypto_skcipher_clear_flags(ctx->fallback_tfm, CRYPTO_TFM_REQ_MASK); + crypto_skcipher_set_flags(ctx->fallback_tfm, + crypto_skcipher_get_flags(cipher) & CRYPTO_TFM_REQ_MASK); + + return crypto_skcipher_setkey(ctx->fallback_tfm, key, keylen); +} + +/** + * rk2_aes_xts_setkey() - Configure the key for AES-XTS mode + * @cipher: The crypto skcipher handle. + * @key: Buffer containing both cipher and tweak keys. + * @keylen: Total length of the key in bytes. + * + * Validates XTS-specific bounds (e.g., FIPS requirements) and configures + * both the hardware context and fallback. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_aes_xts_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen) +{ + struct crypto_tfm *tfm =3D crypto_skcipher_tfm(cipher); + struct rk2_cipher_ctx *ctx =3D crypto_tfm_ctx(tfm); + int err; + + err =3D xts_verify_key(cipher, key, keylen); + if (err) + return err; + + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + crypto_skcipher_clear_flags(ctx->fallback_tfm, CRYPTO_TFM_REQ_MASK); + crypto_skcipher_set_flags(ctx->fallback_tfm, + crypto_skcipher_get_flags(cipher) & CRYPTO_TFM_REQ_MASK); + + return crypto_skcipher_setkey(ctx->fallback_tfm, key, keylen); +} + +/** + * rk2_skcipher_encrypt() - General skcipher encryption entry point + * @req: The skcipher request structure. + * + * Evaluates hardware constraints and enqueues the request into the crypto + * engine, or diverts to software fallback. + * + * Return: 0 on success, negative error code, or -EINPROGRESS. + */ +int rk2_skcipher_encrypt(struct skcipher_request *req) +{ + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(req); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + + rctx->mode =3D algt->rk2_mode; + return rk2_cipher_handle_req(req); +} + +/** + * rk2_skcipher_decrypt() - General skcipher decryption entry point + * @req: The skcipher request structure. + * + * Evaluates hardware constraints and enqueues the request into the crypto + * engine, or diverts to software fallback. + * + * Return: 0 on success, negative error code, or -EINPROGRESS. + */ +int rk2_skcipher_decrypt(struct skcipher_request *req) +{ + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(req); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + + rctx->mode =3D algt->rk2_mode | RK2_CRYPTO_DEC; + return rk2_cipher_handle_req(req); +} + +/** + * rk2_cipher_run() - Execute an asynchronous skcipher request + * @engine: The crypto engine queue managing this request. + * @async_req: The asynchronous skcipher request to process. + * + * Prepares the hardware context, configures DMA descriptors, programs + * cipher registers, and triggers the physical cryptographic accelerator. + * + * Return: Always 0. Errors are reported through the crypto engine + * finalization callback. + */ +int rk2_cipher_run(struct crypto_engine *engine, void *async_req) +{ + struct skcipher_request *areq =3D + container_of(async_req, struct skcipher_request, base); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(areq); + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(areq); + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + struct scatterlist *sgs, *sgd; + int err =3D 0; + int ivsize =3D crypto_skcipher_ivsize(tfm); + unsigned int len =3D areq->cryptlen; + unsigned int todo; + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + struct rk2_crypto_dev *rkc =3D rctx->dev; + struct rk2_crypto_lli *dd =3D &rkc->tl[0]; + u32 m, v; + u32 *rkey =3D (u32 *) ctx->key; + u32 *riv =3D (u32 *) areq->iv; + int i; + unsigned int offset; + unsigned long timeout; + + m =3D rctx->mode | RK2_CRYPTO_ENABLE; + if (algt->is_xts) { + switch (ctx->keylen) { + case AES_KEYSIZE_128 * 2: + m |=3D RK2_CRYPTO_AES_128BIT_key; + break; + case AES_KEYSIZE_256 * 2: + m |=3D RK2_CRYPTO_AES_256BIT_key; + break; + default: + dev_err(rkc->dev, "Invalid key length %u\n", + ctx->keylen); + err =3D -EINVAL; + goto exit_no_pm; + } + } else { + switch (ctx->keylen) { + case AES_KEYSIZE_128: + m |=3D RK2_CRYPTO_AES_128BIT_key; + break; + case AES_KEYSIZE_192: + m |=3D RK2_CRYPTO_AES_192BIT_key; + break; + case AES_KEYSIZE_256: + m |=3D RK2_CRYPTO_AES_256BIT_key; + break; + default: + dev_err(rkc->dev, "Invalid key length %u\n", + ctx->keylen); + err =3D -EINVAL; + goto exit_no_pm; + } + } + + err =3D pm_runtime_resume_and_get(rkc->dev); + if (err) + goto exit_no_pm; + + algt->stat_req++; + rkc->nreq++; + + /* the upper bits are a write enable mask, so we need to write 1 to all + * upper 16 bits to allow write to the 16 lower bits + */ + m |=3D 0xffff0000; + + dev_dbg(rkc->dev, "%s %s len=3D%u keylen=3D%u mode=3D%x\n", __func__, + crypto_tfm_alg_name(areq->base.tfm), + areq->cryptlen, ctx->keylen, m); + sgs =3D areq->src; + sgd =3D areq->dst; + + while (sgs && sgd && len) { + if (!sgs->length) { + sgs =3D sg_next(sgs); + sgd =3D sg_next(sgd); + continue; + } + + if (areq->iv && crypto_skcipher_ivsize(tfm) > 0) { + if (rctx->mode & RK2_CRYPTO_DEC) { + offset =3D sgs->length - ivsize; + scatterwalk_map_and_copy(rctx->backup_iv, sgs, + offset, ivsize, 0); + } + } + + dev_dbg(rkc->dev, "SG len=3D%u mode=3D%x ivsize=3D%u\n", sgs->length, + m, ivsize); + + if (sgs =3D=3D sgd) { + err =3D dma_map_sg(rkc->dev, sgs, 1, DMA_BIDIRECTIONAL); + if (err !=3D 1) { + dev_err(rkc->dev, "Invalid sg number %d\n", + err); + err =3D -EINVAL; + goto exit; + } + } else { + err =3D dma_map_sg(rkc->dev, sgs, 1, DMA_TO_DEVICE); + if (err !=3D 1) { + dev_err(rkc->dev, "Invalid sg number %d\n", + err); + err =3D -EINVAL; + goto exit; + } + err =3D dma_map_sg(rkc->dev, sgd, 1, DMA_FROM_DEVICE); + if (err !=3D 1) { + dev_err(rkc->dev, "Invalid sg number %d\n", + err); + err =3D -EINVAL; + dma_unmap_sg(rkc->dev, sgs, 1, DMA_TO_DEVICE); + goto exit; + } + } + err =3D 0; + writel(m, rkc->reg + RK2_CRYPTO_BC_CTL); + + if (algt->is_xts) { + for (i =3D 0; i < ctx->keylen / 8; i++) { + v =3D cpu_to_be32(rkey[i]); + writel(v, rkc->reg + RK2_CRYPTO_KEY0 + i * 4); + } + for (i =3D 0; i < (ctx->keylen / 8); i++) { + v =3D cpu_to_be32(rkey[i + ctx->keylen / 8]); + writel(v, + rkc->reg + RK2_CRYPTO_CH4_KEY0 + i * 4); + } + } else { + for (i =3D 0; i < ctx->keylen / 4; i++) { + v =3D cpu_to_be32(rkey[i]); + writel(v, rkc->reg + RK2_CRYPTO_KEY0 + i * 4); + } + } + + if (ivsize) { + for (i =3D 0; i < ivsize / 4; i++) + writel(cpu_to_be32(riv[i]), + rkc->reg + RK2_CRYPTO_CH0_IV_0 + i * 4); + writel(ivsize, rkc->reg + RK2_CRYPTO_CH0_IV_LEN); + } + + /* + * Process one SG entry per DMA operation. The cipher engine requires + * the IV to be updated between SG entries for CBC and XTS modes; + * the backup_iv mechanism handles this correctly for decryption. + * Building a full multi-descriptor chain is possible but adds + * complexity for no measurable throughput gain on typical workloads. + */ + todo =3D min(sg_dma_len(sgs), len); + len -=3D todo; + dd->src_addr =3D sg_dma_address(sgs); + dd->src_len =3D todo; + dd->dst_addr =3D sg_dma_address(sgd); + dd->dst_len =3D todo; + dd->iv =3D 0; + + /* + * next is ignored by hardware when RK2_LLI_DMA_CTRL_LAST is set in + * dma_ctrl. Set it to an obviously-invalid-but-non-zero sentinel so + * it stands out if ever read in a debug dump. + */ + dd->next =3D 1; + + dd->user =3D RK2_LLI_CIPHER_START | + RK2_LLI_STRING_FIRST | RK2_LLI_STRING_LAST; + dd->dma_ctrl =3D RK2_LLI_DMA_CTRL_DST_INT | + RK2_LLI_DMA_CTRL_LAST | RK2_LLI_DMA_CTRL_LIST_INT; + + /* Clear stale interrupts, then enable with proper write-mask */ + writel(RK2_CRYPTO_DMA_INT_ALL_MASK, rkc->reg + RK2_CRYPTO_DMA_INT_ST); + writel(RK2_CRYPTO_DMA_INT_ENABLE_ALL, rkc->reg + RK2_CRYPTO_DMA_INT_EN); + + writel(rkc->t_phy, rkc->reg + RK2_CRYPTO_DMA_LLI_ADDR); + + reinit_completion(&rkc->complete); + rkc->status =3D 0; + + writel(RK2_CRYPTO_DMA_CTL_START | + (RK2_CRYPTO_DMA_CTL_START << 16), + rkc->reg + RK2_CRYPTO_DMA_CTL); + + timeout =3D wait_for_completion_timeout(&rkc->complete, + msecs_to_jiffies(2000)); + if (sgs =3D=3D sgd) { + dma_unmap_sg(rkc->dev, sgs, 1, DMA_BIDIRECTIONAL); + } else { + dma_unmap_sg(rkc->dev, sgs, 1, DMA_TO_DEVICE); + dma_unmap_sg(rkc->dev, sgd, 1, DMA_FROM_DEVICE); + } + + if (!timeout) { + dev_err(rkc->dev, "DMA timeout\n"); + err =3D -ETIMEDOUT; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + goto exit; + } + + if (!rkc->status) { + dev_err(rkc->dev, "DMA error\n"); +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + rk2_print(rkc); +#endif + err =3D -EIO; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + goto exit; + } + + if (areq->iv && ivsize > 0) { + offset =3D sgd->length - ivsize; + if (rctx->mode & RK2_CRYPTO_DEC) { + memcpy(areq->iv, rctx->backup_iv, ivsize); + memzero_explicit(rctx->backup_iv, ivsize); + } else { + scatterwalk_map_and_copy(areq->iv, sgd, offset, + ivsize, 0); + } + } + sgs =3D sg_next(sgs); + sgd =3D sg_next(sgd); + } + exit: + writel(0xffff0000, rkc->reg + RK2_CRYPTO_BC_CTL); + pm_runtime_mark_last_busy(rkc->dev); + pm_runtime_put_autosuspend(rkc->dev); + exit_no_pm: + local_bh_disable(); + crypto_finalize_skcipher_request(engine, areq, err); + local_bh_enable(); + return 0; +} + +/** + * rk2_cipher_tfm_init() - Initialize the transformation context + * @tfm: The crypto skcipher handle. + * + * Allocates the software fallback transformations required when requests + * fail to meet hardware constraints (e.g., severe scatterlist misalignmen= t). + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_cipher_tfm_init(struct crypto_skcipher *tfm) +{ + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + const char *name =3D crypto_tfm_alg_name(&tfm->base); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + + ctx->fallback_tfm =3D + crypto_alloc_skcipher(name, 0, CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->fallback_tfm)) { + dev_err(algt->dev->dev, + "Cannot allocate fallback for %s %ld\n", name, + PTR_ERR(ctx->fallback_tfm)); + return PTR_ERR(ctx->fallback_tfm); + } + + dev_dbg(algt->dev->dev, "Fallback for %s is %s\n", + crypto_tfm_alg_driver_name(&tfm->base), + crypto_tfm_alg_driver_name(crypto_skcipher_tfm + (ctx->fallback_tfm))); + + tfm->reqsize =3D sizeof(struct rk2_cipher_rctx) + + crypto_skcipher_reqsize(ctx->fallback_tfm); + + return 0; +} + +/** + * rk2_cipher_tfm_exit() - Free skcipher initialization resources + * @tfm: The crypto skcipher handle. + * + * Synchronously releases internal software fallback transformations + * and zeroes out sensitive key material. + */ +void rk2_cipher_tfm_exit(struct crypto_skcipher *tfm) +{ + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + + memzero_explicit(ctx->key, ctx->keylen); + crypto_free_skcipher(ctx->fallback_tfm); +} --=20 2.47.3 From nobody Mon Jun 8 08:28:17 2026 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 844483264D5 for ; Sat, 30 May 2026 16:07:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157242; cv=none; b=ncupx7fT/YBta+72QRkFWm3AjU+zXF7hIN6PufpiTNxQC7/rEuiggOrMrr8ocxtH6kSgGcsrQmolRV3lUwQLYfEHm2iSRN0VXqu9NQf1+7Aj2Gqqu8QO2clCY2e4n5Z8MrSnoyWwZ3QHJtnrH00YhuDBrPCbjf0H2vmfiUOjl8E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157242; c=relaxed/simple; bh=tZfd/ISESKohi7rVf3H86fUzZEpjAxBv1DvbXjeNK2g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XUzC/t6JB0gH8XjRxnl5x+iR5yJ6uQ3DXskyBztvBZq9FKGLo48MqTAzbyyGYOi9YEcoD9g50CpBBdiYRWfIjJKLLseT+TZHZEUy+L/gEZcaBBIf91BHePS0aTx0gGTtSHBjEQBDQJdfDlcsJeb90dpLriGzWotAmt4uGZ7OoHg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EqF2hDXX; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EqF2hDXX" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-45ef1629ff4so1297542f8f.0 for ; Sat, 30 May 2026 09:07:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780157240; x=1780762040; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+SJMrJ4Tyw4rnth10NKoarZEEEB3Q5hOPvF+j2hwsdo=; b=EqF2hDXXWHydBl2C49IydzSYwvEtoKSkO0FF+LuzVOnE1kWfh4XWvFEu1G+AyHSV06 xxB+sA/ULyQhFwTiYcjSH8aBnBN+3OwksCDlKkMU3nS99falyqYcmw3yX0gMMKhokapa UvXPlQ1mC6cVp6oOf+6KBUwM8LYDBY9obHCa70a0hAIouU3RwLrppT/z9S17YRMjYGlZ KK1sWWqjcJTFbUg8FtwDrWOFSemRMC+C0E2WQ5NZnMwcC3use+JS+hCfTWmM2EITsSkm jo79R5GIClBEIum5rDYmckirLtxEVkjVGtdZH5GEIxvTu/kx/SQwyH9VweLqjNm2TFq4 kLZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780157240; x=1780762040; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=+SJMrJ4Tyw4rnth10NKoarZEEEB3Q5hOPvF+j2hwsdo=; b=E/I1T4O0yL3kLQDXKW86vUfPY/Y5dMbuutD4GZSehgrW8jYoZqMjsP3XSG7BsyaP4G JQJB2fLZCslgyiiVT9BT+R1ewWzFmTF5GYUTrJBufFOT+rU+Lz8V6ieyTShpvMsoOO3q TNcn9OvFhLsQhiydE3NgcLkSuHqlxUuKeo22uOPn1Wg2XmUOKugHz16AzbUBhEZOhRoF NtNu2WF8c/461P7os3LgDE7TXbW5QsyWzD1E8hr2fzfz1xZbGSMExBosG35tEi/UffaQ mzZwzmN3VJo1gYomIdTXRJyFGqoOVwCRenL3omgRloePJLbx+TLN2FSMm2aEREiCnO0r sL7Q== X-Forwarded-Encrypted: i=1; AFNElJ+scP5UiLAEfO1+23X3XKwLwmNzU1AwBrVaUz3YwGF3xzgasgjO1CxZygOu4JL7hRh2pX0AQ2G/kdhXJTA=@vger.kernel.org X-Gm-Message-State: AOJu0Yz77/C7i1OYw6m7sSo5HrG1syxow3YnHjCAtSaY0p+asQUoNDEA bCpUTjgJ4jk2lpNxcydiw7aihZOBg1LM9M+afJGMm3EiduY2MhnGGIpr X-Gm-Gg: Acq92OH0bwUjqWOIwFHIq8zLgqnqyv0qhnwjAOE6xnRPU06Rh3yTnMTCx4/Exaf3Qjp 6AbrLiRbw8skmz0KU8MtRqDDvXwVn6NFomWsK9TMzXAELaVaKIfrF6Vo7WNk0K62l1G5s9PDg2u 2NHrnSWz6/zzVzKUi24pCl/mspgkYv5lkkO6g/2vjn+/al1Q4nHh/TplL4bLeFe20Z1TGVsG2wP ynIEAs7OW7nsKhts6kCRgui4OFdzIzMo+G5iZZWFDbJuV1rdzIjwwEityaHHc6l/snjUyaBXI6r cMiDR17f180/pmRrn2LOQtqZ0g37cyJztvJrHB8PBuQDDyl+c9ws/EGMHq0RPlFBg+rkNRgYm1c ulWvZD2wYLBCim+RT62flM5fG4d/k+/GFyI+gak4sD2bn6Z84ECvlaGfOzKSD9qOYgqw2Hgj31j GKACRnHH+WtbhdrzUmj6OheLB7eu8= X-Received: by 2002:a5d:5989:0:b0:45f:68a7:38d0 with SMTP id ffacd0b85a97d-45f68a73a2amr1942009f8f.0.1780157239975; Sat, 30 May 2026 09:07:19 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef32fabcasm11667339f8f.0.2026.05.30.09.07.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 May 2026 09:07:19 -0700 (PDT) From: Dawid Olesinski To: herbert@gondor.apana.org.au, davem@davemloft.net, heiko@sntech.de Cc: linux-crypto@vger.kernel.org, linux-rockchip@lists.infradead.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, clabbe@baylibre.com, robh@kernel.org, krzk+dt@kernel.org, conor+dt@kernel.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH 3/4] arm64: dts: rockchip: Add crypto node to rk356x-base Date: Sat, 30 May 2026 17:06:44 +0100 Message-ID: <20260530160704.3453555-4-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260530160704.3453555-1-dawidro@gmail.com> References: <20260530160704.3453555-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the device tree node for the V2 cryptographic hardware accelerator on RK356x SoCs (RK3566, RK3568). The IP block sits in the non-secure peripheral domain. Its three clocks (core, aclk, hclk) and reset line are accessible directly through the main non-secure CRU, so no firmware intermediary is required. The node is disabled by default; board files that wish to use hardware crypto offload must enable it. Signed-off-by: Dawid Olesinski --- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi b/arch/arm64/boo= t/dts/rockchip/rk356x-base.dtsi index 64bdd8b7754b..3b73a56046e7 100644 --- a/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +++ b/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi @@ -1171,6 +1171,18 @@ gpu_leakage: gpu-leakage@1d { }; }; =20 + crypto: crypto@fe380000 { + compatible =3D "rockchip,rk3568-crypto"; + reg =3D <0x0 0xfe380000 0x0 0x2000>; + interrupts =3D ; + clocks =3D <&cru CLK_CRYPTO_NS_CORE>, <&cru ACLK_CRYPTO_NS>, + <&cru HCLK_CRYPTO_NS>; + clock-names =3D "core", "aclk", "hclk"; + resets =3D <&cru SRST_CRYPTO_NS_CORE>; + reset-names =3D "core"; + status =3D "disabled"; + }; + i2s0_8ch: i2s@fe400000 { compatible =3D "rockchip,rk3568-i2s-tdm"; reg =3D <0x0 0xfe400000 0x0 0x1000>; --=20 2.47.3 From nobody Mon Jun 8 08:28:17 2026 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B22E3246E8 for ; Sat, 30 May 2026 16:07:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157244; cv=none; b=nM02t9XeBvJGT/r0DQoZjLqntjoyQQJdk4bFTkoQ0b7ev8tPpXjuA+6GyT1v+ZTqTEjtuuZNNS7OKCbFEFjt5Zzo1VKVaBph3CcSG3ClbHa6B3zi9vG3q6ej7mLKOCdKKQX3O/N2e+nhY+lnt19+oNIjJSUTMe1ueoGLQWi+ies= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780157244; c=relaxed/simple; bh=7zpQ7IpSRpflbzheGA/K2zsqhy/x5O4W7p1wQDhU9pA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EHzu3fXsgdTKROaSgT/XMi+URcdiccL3iSaadVG25HoFfr4+XqWw1LYiuj+2hEgqFlehK+rS/Fmu8ffcrSNtp5FNNWz/Jz9H7pG5q5tDMKmuYlBkrUmTsAa0RZgLaA9qDR7AGhWhV1xPRNfb4k/pp/+SGGh/WhxdVNhwTRy5TwU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TYytqci6; arc=none smtp.client-ip=209.85.221.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TYytqci6" Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-441209fb77eso9127077f8f.1 for ; Sat, 30 May 2026 09:07:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780157242; x=1780762042; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G5lAYdQwlZ0T0Ie+aG3ceFs/5DSYUa18LTgq9dN2ugk=; b=TYytqci6uT99g6pXncFSv39DifEoqHHwL6G//3XOG+IKVFQXMl85M0Q47DStqsKfY/ tojNzRRiG3svrSvFoeXUGRWbEdYq6drz9MAAiCOp1sm9uZ4C7rndGIOyL02AFkNhRAqX NA83zc96/96B1d4QLk3bfeaZIzver+Q/y8UAigE4Q6VvBPsB99jl1NTBdULOaFEo1Q++ 3HCpaY+728jwLdGEauxm1CbC+pxbn4Zw4BW/e+N92cO88SCu2O6dmnIfl4l/O0oVuC1I YgBXGiklhzJnKUOP576aizxGsS7bwz6kz2tE+NX4BHjrDhg6tRM7m1Q3xj6BStOOZmCl +mTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780157242; x=1780762042; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=G5lAYdQwlZ0T0Ie+aG3ceFs/5DSYUa18LTgq9dN2ugk=; b=ftJ3HZXdOkanGX1b/FicbN8FtqrBCtS0t3MlVQiQEOR/K79DLKW+gnbaeNzdyJtqif FBoNGJNk9FB8lY7NwrExRxWJxqVLaognbONB9Yd3DZa9NVYTQQvl8iLyaUJcQzB70Cwa EwSLnkg4oP3dVjTrZowV56t79rGELeZk5SxQ8ml35uDeeXV8ZS07B4TXaNuheEBQ2krK OQjri25hOSRqeh8/L/If6IEEsH0yP6YBE5WxCSKcfr+9RUtSHDO3azlickN8hAPKn48k 9HGE4KFjeVF0+TUvYpPKAL4K6fK+DyuVipKg4EvkKZV4tBNGLmLg1vGkcLrA8GIVY5VA aqSw== X-Forwarded-Encrypted: i=1; AFNElJ+0BczXv7MaXW22Q5Ju7h0KYdX70ycSLUJ9d1HI5MBsCkqQCn7CTsuFj4vuozPlPrR35iIl7z9n23T4Uy0=@vger.kernel.org X-Gm-Message-State: AOJu0YxJDRe8xyKPRP5NAiLsukoNlO0CFLgD0YBwijCUbUlcehBRBAdc dtS7gKcPH0cPbFLeM/LGcKkE5fpJ7/e6S2DF8cv4ffFPVRFAdaiE1umG X-Gm-Gg: Acq92OEWaBl5MHD2QDPfJzIfw89XvFrINGNyS+ajBXpXpR9OwSedPwbLX6qbYlQCwWO sYOi55K4KqGO+bgnrzw024afSVFjBTMWVN29SayWRYUtTV1zMjfN3yAmFfzAMd1HifYUQNeQfcx EOfv5F3V/RdeeUe7hCNH0FpcqxD61JzObkXvcRG5uwXPJpL6qidd5loLHtwkfV3VLwEWyJhnZci pksNGu+Ro5VuEdEHaBEn/de5bq/qgGB37tSoTim3RWtQx6AoOgnKCTOo/4LJgiXXJ1o19hzu+tE IJTLzZFZbPQYnMDvsBGyyIb/PmUgUbnqBDo4rTHDIB1ZBH3WA9nnO6JKd3FeA+5shd2Ly1Sh1fH DKgKsID0aKu39BBfc3QV4QJZiRws+cs9BpJiPKA/q9hikhod+97Kiz8i688WiWgh1dPvninZok7 VSkqOa5QKl2iNPtcpuBz4zhNfTCLU= X-Received: by 2002:a05:6000:4b1a:b0:45e:8526:7dcb with SMTP id ffacd0b85a97d-45ef6e6cca5mr5660776f8f.7.1780157241724; Sat, 30 May 2026 09:07:21 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef32fabcasm11667339f8f.0.2026.05.30.09.07.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 May 2026 09:07:21 -0700 (PDT) From: Dawid Olesinski To: herbert@gondor.apana.org.au, davem@davemloft.net, heiko@sntech.de Cc: linux-crypto@vger.kernel.org, linux-rockchip@lists.infradead.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, clabbe@baylibre.com, robh@kernel.org, krzk+dt@kernel.org, conor+dt@kernel.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH 4/4] arm64: dts: rockchip: Add crypto node to rk3588-base Date: Sat, 30 May 2026 17:06:45 +0100 Message-ID: <20260530160704.3453555-5-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260530160704.3453555-1-dawidro@gmail.com> References: <20260530160704.3453555-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the device tree node for the V2 cryptographic hardware accelerator on RK3588. On RK3588 the crypto IP sits inside the secure domain controlled by SECURECRU, a register bank that is exclusively accessible to the TrustZone firmware (TF-A). Linux must therefore obtain its clocks and reset line through the ARM SCMI interface provided by the firmware rather than mapping the CRU registers directly. Attempting direct MMIO access to SECURECRU from the non-secure world triggers an asynchronous bus fault. The interrupt uses the four-cell GICv3 format as required by the RK3588 GIC node definition (the fourth cell is the CPU affinity/partition specifier; 0 means no affinity constraint). The node is disabled by default; board files that wish to use hardware crypto offload must enable it. Signed-off-by: Dawid Olesinski --- arch/arm64/boot/dts/rockchip/rk3588-base.dtsi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi b/arch/arm64/boo= t/dts/rockchip/rk3588-base.dtsi index 4fb8888c281c..4f336741d11f 100644 --- a/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi +++ b/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi @@ -2257,6 +2257,18 @@ rng@fe378000 { resets =3D <&scmi_reset SCMI_SRST_H_TRNG_NS>; }; =20 + crypto: crypto@fe370000 { + compatible =3D "rockchip,rk3588-crypto"; + reg =3D <0x0 0xfe370000 0x0 0x2000>; + interrupts =3D ; + clocks =3D <&scmi_clk SCMI_CRYPTO_CORE>, <&scmi_clk SCMI_ACLK_SECURE_NS>, + <&scmi_clk SCMI_HCLK_SECURE_NS>; + clock-names =3D "core", "aclk", "hclk"; + resets =3D <&scmi_reset SCMI_SRST_CRYPTO_CORE>; + reset-names =3D "core"; + status =3D "disabled"; + }; + i2s0_8ch: i2s@fe470000 { compatible =3D "rockchip,rk3588-i2s-tdm"; reg =3D <0x0 0xfe470000 0x0 0x1000>; --=20 2.47.3