From nobody Sun Jul 26 13:25:59 2026 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B1162459EA for ; Wed, 8 Jul 2026 17:59:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533543; cv=none; b=K3vjv6aJb3kf7CeFBhMZ4v2cBzCnd4Y5Ddrjk7DH5p9Tx1qu2SQlRjNiT8LNwvQYZiU7nXrqRse4VDdiVMN0f8JdzY/uk4Px2Zu7vAiR1wHtznmt5SFT4so4+BnwUNsf0EFEnkpGloXARX9k3Wsa1INZKPV6BwHFd4grwx7m1Jg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533543; c=relaxed/simple; bh=qR8da7hKtoo+tPfwG3vADxqc/OIuPfs3p6aDdWf+HJs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m28aKDgw3r0PPVhshJ9CNPMfkMViFm1EUs7VkRv+YWXkyL4mkIug5HJQ+JDwlpQjrMfMuLc2pKa/T9jMMwoSsjuQoIHPQ/wDHfkDSuSaOlwUsvzkqJVuJ6gutJxQNcyjLR2CZcb7ZprpSElRkSCE4k7Wh03+33ekk+Vo9LXoCp0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=okXq95Bu; arc=none smtp.client-ip=209.85.208.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="okXq95Bu" Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-699fbcd23ccso1494625a12.1 for ; Wed, 08 Jul 2026 10:59:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783533541; x=1784138341; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ANaeyZnIlvUZuUb+qcVD79ri5D/Hu+DLtzSwYPDFNAo=; b=okXq95BumxxuPuQcGSoDV1NhQQWS+Pw+q68vEYQuqMONf6t0YyZS6oarm3EO7av1hY sm5bc5QrNtKjLCHBDgwxC9znCd+yqRvDve8Dmp3LGG0JbvgK1wT+2EsmLR7kUkIIicLY W4SvUqdumDi8n1XEg4dTxOEV/EitJsUqQ/gl9vC6r5QZ3fxE+n/rX2mwS3biBPjJEiLA 5MNcIYCZ6Zuo/XRVa2zs99mbc3EB283P9embX+uYa8OJV8MJrt3l6n2Hm2/40rJrDVnH 1vLr2JBNYk0DhkTiU5q6Y0ts5LEvAZNXPqd5tvRxWmlZVJDFKXwOPgnUOxsMDybM/pYr YMYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783533541; x=1784138341; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ANaeyZnIlvUZuUb+qcVD79ri5D/Hu+DLtzSwYPDFNAo=; b=V9pNZqa5snH5VuCcWDfmjtdDqihAB4TS0PFoelWmFmszCMFJPoGnNKVkyQgvq3maJJ gdCifK7X6co4k4sOa/ag0XRbf9wOltJn3/083LmKa85BMjm0JNQzl/wE4/OU/QyQA5Qq 3eM1grIX0eqqHP3iuM+9GT240pkfV/YXjxvtUYvCqiyTlVFKXGh5b3HgMH7UL7ritw0I Hs5nX8voJFF740zBIyw684GLkS27M9T9DorJGG6eprrHbjde4LLS4N2S9ceue223u5dL Ym7DQu1QIzVa0mt8cPiPcwRolSU1NZ2Kd1RHfoZFihpccYbkV5tIpdM0XGOzuHMQg0rm ZJSg== X-Forwarded-Encrypted: i=1; AHgh+RoE9U04s6V6Y+8tgJ5pkmLDbH6mP0xMTLW7pNnaOhMLLzSw+YiJUBb21IsDyRNGEWxubqyzdLPKXtZMEsQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwTL5XLgQBQbh604a9VpcTte6V+Ku+OLVauFn8yxXq+w1X+xy3S 0Ji7qGHJPHJvJzm93v1euwjyrkHODl21Vr+uXLaHYLNJ40w9jIEhZn2i X-Gm-Gg: AfdE7cl2FQCUWxpk7WuplY0lv/nWZuVj4ojTR87nolggkB1N06vUDRZHiB+fYU80vFK MTCXeAQkZpNpUCJd8WLitM/cFKM994YfDMaPDqLfzMPV9Ary+zTQWeTS9fLOHH7Rsgf9vN6yEKn tZB5VYAwu9ZrfZf+Mvp3p9+qKMb6N62nXQvJH8kcSM4yQWHFQRfaPC1jjICQYazKzcJE6it1lCx cqSoF6OhLBCTxpDX65FebcbBSku1GMDfyPWtM6/7dITRyUDPQcKY5sgof6ZoGo/+wz4fVyTF/rr lWBq5Uf44WuWj/LDfVQUsmx3Pw3SmUIh0niajFsYt+VhUWmYKKO9IQm+Mm0AHeD4cGtgO2+263L 65PQU5xxY6SpdOMmbCRwjVasM6mznKxd1klEwJJEfycM/hdt2YLEqPi10kn6lTB6yBenHs9v92n JzXPrnfw== X-Received: by 2002:a17:907:a286:b0:c15:b6bd:cbb8 with SMTP id a640c23a62f3a-c15ce121a12mr164929166b.50.1783533540594; Wed, 08 Jul 2026 10:59:00 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c15ad9bcc26sm357653666b.34.2026.07.08.10.58.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2026 10:59:00 -0700 (PDT) From: Dawid Olesinski To: Herbert Xu , "David S . Miller" , Rob Herring , Krzysztof Kozlowski Cc: Conor Dooley , Heiko Stuebner , Corentin Labbe , linux-crypto@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH v2 1/4] dt-bindings: crypto: rockchip: Add RK356x/RK3588 crypto engine binding Date: Wed, 8 Jul 2026 18:58:22 +0100 Message-ID: <20260708175837.1718437-2-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260708175837.1718437-1-dawidro@gmail.com> References: <20260708175837.1718437-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a YAML device tree binding for the Rockchip second-generation (V2) cryptographic hardware accelerator present on the RK3568 and RK3588 SoCs. The IP block exposes AES-ECB, AES-CBC, AES-XTS block ciphers, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512, MD5, and SM3 hash algorithms, each with a hardware DMA engine controlled via linked-list descriptors. The binding covers two compatible strings: - rockchip,rk3568-crypto: clocks and resets are driven directly by the non-secure CRU (accessible to Linux at EL1). - rockchip,rk3588-crypto: clocks and resets live in SECURECRU, a register bank sandboxed to TrustZone. Linux must request them through the ARM SCMI firmware interface (scmi_clk / scmi_reset), as direct MMIO access to SECURECRU from EL1 triggers a bus fault. Signed-off-by: Dawid Olesinski --- .../crypto/rockchip,rk3588-crypto.yaml | 75 +++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 Documentation/devicetree/bindings/crypto/rockchip,rk358= 8-crypto.yaml diff --git a/Documentation/devicetree/bindings/crypto/rockchip,rk3588-crypt= o.yaml b/Documentation/devicetree/bindings/crypto/rockchip,rk3588-crypto.ya= ml new file mode 100644 index 000000000000..fc09f21b0654 --- /dev/null +++ b/Documentation/devicetree/bindings/crypto/rockchip,rk3588-crypto.yaml @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/crypto/rockchip,rk3588-crypto.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Rockchip cryptographic offloader + +maintainers: + - Heiko Stuebner + - Corentin Labbe + - Dawid Olesinski + +properties: + compatible: + enum: + - rockchip,rk3568-crypto + - rockchip,rk3588-crypto + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: Core clock for the crypto IP internal logic + - description: AXI interconnect clock interface + - description: AHB interface clock + + clock-names: + items: + - const: core + - const: aclk + - const: hclk + + resets: + maxItems: 1 + + reset-names: + items: + - const: core + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - resets + +additionalProperties: false + +examples: + - | + #include + #include + #include + + bus { + #address-cells =3D <2>; + #size-cells =3D <2>; + + crypto@fe370000 { + compatible =3D "rockchip,rk3588-crypto"; + reg =3D <0x0 0xfe370000 0x0 0x2000>; + interrupts =3D ; + clocks =3D <&scmi_clk SCMI_CRYPTO_CORE>, <&scmi_clk SCMI_ACLK_SECU= RE_NS>, + <&scmi_clk SCMI_HCLK_SECURE_NS>; + clock-names =3D "core", "aclk", "hclk"; + resets =3D <&scmi_reset SCMI_SRST_CRYPTO_CORE>; + reset-names =3D "core"; + }; + }; --=20 2.47.3 From nobody Sun Jul 26 13:25:59 2026 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34E0B434404 for ; Wed, 8 Jul 2026 17:59:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533550; cv=none; b=Ktve4Reo8AJu0A1iquRYjYmdrpvP4YYiMDfaT2tElaD79LvjG7cmW4pAFeUt10XRthx6AYt0Z8XOcjJPf/2DyU3cROQTveCAvK6G1kpT1Rh4SnBa2tgFAfq4XbXz5NV4Mfy0HZgFBECViGyP28vOPVlGwbym2Xebo7k3CT4fKIs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533550; c=relaxed/simple; bh=L4ScafNYLjuUYW28tz3sL+9sD+X5NiB0FJ620UdNlTo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Dapna8PQ9wEV517mbweABHu3BQVv9wzpawcdHCcQNt0W8O9VJEDuHzCeYRYxYnd/5V0iEAPtC1fpY0A4IjilSzWPYhoAMFuqer/2uNvzIC4k+Ku0Z06KKkNvVeDKVHn9yXQv4bnXpHL/yS1mgeGpp2V6fOyENyAn0OdZjYQRXA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YoxLi1qQ; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YoxLi1qQ" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-c15b75c9e48so143020266b.1 for ; Wed, 08 Jul 2026 10:59:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783533544; x=1784138344; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=BQ4OXTgwlNwvjhqsYEhDIhN6OkXkfU+ECvQCIBryu7I=; b=YoxLi1qQ5CAkp7pWnBYEFPQHbir0/bYerRAoOxnNnHauBOwsW/VHZT1ZrT444pvdV4 2hlq39PWGc+2K03/Eo3qwfw1JXfO7u2prYgn6gUVtDCzqirUIS63bnf/zh+kLOoHLapZ Qu8g23jrkulVGP7LV/ktRUjnkarVYwM+0ISeGDHoCscGHy5iKITR9frca2y+A44Oaydz RnN174+2CfN7j2RBn7/Hmezc9CKHPMEAjrFxtXIPjH+gJh+BWuAheDCrX9yjZteNQsDu Q1w5jOL1+y9Shs5RH4zToKu/hIXuOO4BGsZGUiKMA9aZEaVO58ghZXHjDwjmsfyDRRTt ZQ/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783533544; x=1784138344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=BQ4OXTgwlNwvjhqsYEhDIhN6OkXkfU+ECvQCIBryu7I=; b=p98qcOH49J+CpVFo6y4HspifQhZZCNXrIoWAb9F9ZUqIrr1Ea6enU8xZK3U+qoSNuo ITZ8XZQqJJ2+DvDcCMDOOpKUVeYM6qNy2UqOcZT7XdFfULWQ978oTSQSKQx0qpVo21my TyzjtE6HaZZLTyTPyCjZRh4XOz8QhDo6CH9EzN9N6ZquLj2/DPuocE7+xMYxOvN+C3uZ vuNYwF0wV+1g+spo06ywUum3P533rLEff5ZgmfOiL5023E84b9aFZJjZx8zH0vZKtb8o E8j9Ap+zovSnhYAxaa0AwLbzBPJvhLzhJnRFuR3nVytcBhXG5BxuMt1uEJDc2YoB1Yo2 AjZQ== X-Forwarded-Encrypted: i=1; AHgh+RrI2cCsW3pIHtEFLJxbSmJdh+U1YB/cMdKuucVuIAeOVTYRqaEVYICRVHPaM3v/LxJvJ86TYu7zgGY5+eQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yx/lSoYRg6Rotm6G6u5N+yOR0hRL2PI2oBdeB5LHVq18jk8T6LQ 0iFXbOSh/atPZjKoC9I/9XuBTWlV0shDiFLY5b3ZZ2hMZLCVWlv0IG9f X-Gm-Gg: AfdE7cmql8qJSFT80RRvNAfCu6i57yMpUTX85i1CYfuwzbUb6iQN9B14/44V7cXMG7I AT4VSG9Mmh2UyCvjXyr6TcBWQj2JY8levuyWTjEDTX+GrXVOehZVucTMk2fjAopIoeEXUdeVz4v JUEPwe2TKzXnIkU1UHGcXnAT38nfJUFlC+6SKRCqbKxIapG85/Uy/VzVMoFKAmEAxUiwE5JKUTX ljBth6DLnxj9q9Zv/vgc7KaSsVqfw+XuVz6oygct5a08C6b36RpQcojl3Ic+4Tbue+OqGFXa4oC sH4x5U46aZlRaFVqMnX3fVo9qIwQDmUNnPFwv8bDo4UI84h6431J+v8pDmcsdMoeVPC5tKYGFQU iVlhny+2wxGG/8dcap0obQStQBZijThclAZ7xDa7PhzsixRrXOGS2HwUShn3G4/AXqlgLOMBahr cFuwK2oQ== X-Received: by 2002:a17:907:60cd:b0:c15:bcb0:fc5e with SMTP id a640c23a62f3a-c15ce006296mr151502066b.23.1783533543186; Wed, 08 Jul 2026 10:59:03 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c15ad9bcc26sm357653666b.34.2026.07.08.10.59.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2026 10:59:02 -0700 (PDT) From: Dawid Olesinski To: Herbert Xu , "David S . Miller" , Rob Herring , Krzysztof Kozlowski Cc: Conor Dooley , Heiko Stuebner , Corentin Labbe , linux-crypto@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH v2 2/4] crypto: rockchip: Add RK356x/RK3588 cryptographic offloader driver Date: Wed, 8 Jul 2026 18:58:23 +0100 Message-ID: <20260708175837.1718437-3-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260708175837.1718437-1-dawidro@gmail.com> References: <20260708175837.1718437-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a driver for the second-generation Rockchip cryptographic hardware accelerator found on RK3568 and RK3588 SoCs (compatible strings "rockchip,rk3568-crypto" and "rockchip,rk3588-crypto"). The hardware provides: - AES block cipher engine: ECB, CBC, and XTS modes, 128/192/256-bit keys. XTS hardware is limited to single-SG requests. - Hash engine: SHA-1, SHA-256, SHA-384, SHA-512, MD5, SM3. The hardware padding engine (HW_PAD) requires the total message length upfront and cannot maintain state across LLI descriptor boundaries, so multi-SG and unaligned requests are routed to a software fallback. - DMA engine: linked-list descriptor (LLI) based, with a 20-entry coherent descriptor table. Design overview: - Built on top of the crypto engine framework (crypto/engine.h) for serialised hardware request dispatch and automatic fallback handling. - Each platform device gets its own private copy of the algorithm descriptor table at probe time (devm_kmemdup), so the dev pointer in each template always refers to the correct hardware instance. This avoids a global device list without any locking overhead. - Runtime PM with autosuspend (2 s idle timeout) gates clocks and asserts reset between requests to save power. - Symmetric software fallback for all registered algorithms handles requests that cannot be processed in hardware (misaligned buffers, multi-SG inputs for hash, zero-length payloads). Co-developed-by: Corentin Labbe Signed-off-by: Corentin Labbe Signed-off-by: Dawid Olesinski --- drivers/crypto/Kconfig | 32 + drivers/crypto/Makefile | 1 + drivers/crypto/rockchip/Makefile | 5 + drivers/crypto/rockchip/rk2_crypto.c | 746 ++++++++++++++++++ drivers/crypto/rockchip/rk2_crypto.h | 249 ++++++ drivers/crypto/rockchip/rk2_crypto_ahash.c | 565 +++++++++++++ drivers/crypto/rockchip/rk2_crypto_skcipher.c | 728 +++++++++++++++++ 7 files changed, 2326 insertions(+) create mode 100644 drivers/crypto/rockchip/rk2_crypto.c create mode 100644 drivers/crypto/rockchip/rk2_crypto.h create mode 100644 drivers/crypto/rockchip/rk2_crypto_ahash.c create mode 100644 drivers/crypto/rockchip/rk2_crypto_skcipher.c diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index 216a00bad5d7..bd1a704ca998 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -690,6 +690,38 @@ config CRYPTO_DEV_ROCKCHIP_DEBUG This will create /sys/kernel/debug/rk3288_crypto/stats for displaying the number of requests per algorithm and other internal stats. =20 +config CRYPTO_DEV_ROCKCHIP2 + tristate "Rockchip's RK356x and RK3588 cryptographic offloader" + depends on OF && ARCH_ROCKCHIP + depends on PM + select CRYPTO_ECB + select CRYPTO_CBC + select CRYPTO_AES + select CRYPTO_MD5 + select CRYPTO_SHA1 + select CRYPTO_SHA256 + select CRYPTO_SHA512 + select CRYPTO_SM3 + select CRYPTO_HASH + select CRYPTO_XTS + select CRYPTO_SKCIPHER + select CRYPTO_ENGINE + help + This driver interfaces with the hardware crypto offloader present + on RK3566, RK3568 and RK3588 SoCs. It provides hardware acceleration + for symmetric block ciphers and hashing functions, offloading the + main CPU cores during heavy cryptographic workflows. + +config CRYPTO_DEV_ROCKCHIP2_DEBUG + bool "Enable Rockchip crypto stats" + depends on CRYPTO_DEV_ROCKCHIP2 + depends on DEBUG_FS + help + Say y to enable Rockchip crypto debug stats. + This will create a directory using the device name + (e.g., /sys/kernel/debug/fe370000.crypto/stats) for displaying + the number of requests per algorithm and other internal stats. + config CRYPTO_DEV_TEGRA tristate "Enable Tegra Security Engine" depends on TEGRA_HOST1X diff --git a/drivers/crypto/Makefile b/drivers/crypto/Makefile index 5a950c7abc39..9e406729ab94 100644 --- a/drivers/crypto/Makefile +++ b/drivers/crypto/Makefile @@ -29,6 +29,7 @@ obj-$(CONFIG_CRYPTO_DEV_PPC4XX) +=3D amcc/ obj-$(CONFIG_CRYPTO_DEV_QCE) +=3D qce/ obj-$(CONFIG_CRYPTO_DEV_QCOM_RNG) +=3D qcom-rng.o obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP) +=3D rockchip/ +obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP2) +=3D rockchip/ obj-$(CONFIG_CRYPTO_DEV_S5P) +=3D s5p-sss.o obj-$(CONFIG_CRYPTO_DEV_SA2UL) +=3D sa2ul.o obj-$(CONFIG_CRYPTO_DEV_SAHARA) +=3D sahara.o diff --git a/drivers/crypto/rockchip/Makefile b/drivers/crypto/rockchip/Mak= efile index 785277aca71e..ea946ba0d4d5 100644 --- a/drivers/crypto/rockchip/Makefile +++ b/drivers/crypto/rockchip/Makefile @@ -3,3 +3,8 @@ obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP) +=3D rk_crypto.o rk_crypto-objs :=3D rk3288_crypto.o \ rk3288_crypto_skcipher.o \ rk3288_crypto_ahash.o + +obj-$(CONFIG_CRYPTO_DEV_ROCKCHIP2) +=3D rk_crypto2.o +rk_crypto2-objs :=3D rk2_crypto.o \ + rk2_crypto_skcipher.o \ + rk2_crypto_ahash.o diff --git a/drivers/crypto/rockchip/rk2_crypto.c b/drivers/crypto/rockchip= /rk2_crypto.c new file mode 100644 index 000000000000..30b39d3b6317 --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto.c @@ -0,0 +1,746 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Hardware cryptographic offloader for RK3568/RK3588 SoC + * + * Copyright (c) 2022-2023 Corentin Labbe + * Copyright (c) 2026 Dawid Olesinski + */ + +#include "rk2_crypto.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static const struct rk2_variant rk3568_variant =3D { + .num_clks =3D 3, +}; + +static const struct rk2_variant rk3588_variant =3D { + .num_clks =3D 3, +}; + +static int rk2_crypto_get_clks(struct rk2_crypto_dev *dev) +{ + dev->num_clks =3D devm_clk_bulk_get_all(dev->dev, &dev->clks); + if (dev->num_clks < 0) + return dev_err_probe(dev->dev, dev->num_clks, "Failed to get clocks\n"); + if (dev->num_clks < dev->variant->num_clks) + return dev_err_probe(dev->dev, -EINVAL, + "Missing clocks, got %d instead of %d\n", + dev->num_clks, dev->variant->num_clks); + return 0; +} + +static int rk2_crypto_pm_suspend(struct device *dev) +{ + struct rk2_crypto_dev *rkdev =3D dev_get_drvdata(dev); + + reset_control_assert(rkdev->rst); + udelay(10); + clk_bulk_disable_unprepare(rkdev->num_clks, rkdev->clks); + + return 0; +} + +static int rk2_crypto_pm_resume(struct device *dev) +{ + struct rk2_crypto_dev *rkdev =3D dev_get_drvdata(dev); + int ret; + + ret =3D clk_bulk_prepare_enable(rkdev->num_clks, rkdev->clks); + if (ret) + return ret; + + udelay(10); + reset_control_deassert(rkdev->rst); + + return 0; +} + +static const struct dev_pm_ops rk2_crypto_pm_ops =3D { + RUNTIME_PM_OPS(rk2_crypto_pm_suspend, rk2_crypto_pm_resume, NULL) +}; + +static int rk2_crypto_pm_init(struct rk2_crypto_dev *rkdev) +{ + int err; + + pm_runtime_use_autosuspend(rkdev->dev); + pm_runtime_set_autosuspend_delay(rkdev->dev, 2000); + + err =3D pm_runtime_set_suspended(rkdev->dev); + if (err) + return err; + pm_runtime_enable(rkdev->dev); + + return 0; +} + +static void rk2_crypto_pm_exit(struct rk2_crypto_dev *rkdev) +{ + pm_runtime_disable(rkdev->dev); +} + +static irqreturn_t rk2_crypto_irq_handle(int irq, void *dev_id) +{ + struct rk2_crypto_dev *rkc =3D platform_get_drvdata(dev_id); + u32 v; + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_INT_ST); + if (!v) + return IRQ_NONE; + + writel(v, rkc->reg + RK2_CRYPTO_DMA_INT_ST); + + /* + * Only signal completion on list-done or hard DMA error. + * Intermediate SRC_INT (BIT(1)/BIT(2)) fire for every LLI + * entry that has RK2_LLI_DMA_CTRL_SRC_INT set. Completing + * early on those causes the driver to read result registers + * before all data has been processed, producing wrong results. + */ + if (v & RK2_CRYPTO_DMA_INT_ERR_MASK) { + dev_warn(rkc->dev, "DMA Error\n"); + rkc->status =3D 0; + complete(&rkc->complete); + } else if (v & RK2_CRYPTO_DMA_INT_LISTDONE) { + rkc->status =3D 1; + complete(&rkc->complete); + } + + return IRQ_HANDLED; +} + +static const struct rk2_crypto_template rk2_crypto_algs_template[] =3D { + { + .type =3D CRYPTO_ALG_TYPE_SKCIPHER, + .rk2_mode =3D RK2_CRYPTO_AES_ECB, + .alg.skcipher.base =3D { + .base.cra_name =3D "ecb(aes)", + .base.cra_driver_name =3D "ecb-aes-rk2", + .base.cra_priority =3D 300, + .base.cra_flags =3D + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_blocksize =3D AES_BLOCK_SIZE, + .base.cra_ctxsize =3D sizeof(struct rk2_cipher_ctx), + .base.cra_alignmask =3D 0, + .base.cra_module =3D THIS_MODULE, + .init =3D rk2_cipher_tfm_init, + .exit =3D rk2_cipher_tfm_exit, + .min_keysize =3D AES_MIN_KEY_SIZE, + .max_keysize =3D AES_MAX_KEY_SIZE, + .setkey =3D rk2_aes_setkey, + .encrypt =3D rk2_skcipher_encrypt, + .decrypt =3D rk2_skcipher_decrypt, + }, + .alg.skcipher.op =3D { + .do_one_request =3D rk2_cipher_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_SKCIPHER, + .rk2_mode =3D RK2_CRYPTO_AES_CBC, + .alg.skcipher.base =3D { + .base.cra_name =3D "cbc(aes)", + .base.cra_driver_name =3D "cbc-aes-rk2", + .base.cra_priority =3D 300, + .base.cra_flags =3D + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_blocksize =3D AES_BLOCK_SIZE, + .base.cra_ctxsize =3D + sizeof(struct rk2_cipher_ctx), + .base.cra_alignmask =3D 0, + .base.cra_module =3D THIS_MODULE, + .init =3D rk2_cipher_tfm_init, + .exit =3D rk2_cipher_tfm_exit, + .min_keysize =3D AES_MIN_KEY_SIZE, + .max_keysize =3D AES_MAX_KEY_SIZE, + .ivsize =3D AES_BLOCK_SIZE, + .setkey =3D rk2_aes_setkey, + .encrypt =3D rk2_skcipher_encrypt, + .decrypt =3D rk2_skcipher_decrypt, + }, + .alg.skcipher.op =3D { + .do_one_request =3D rk2_cipher_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_SKCIPHER, + .rk2_mode =3D RK2_CRYPTO_AES_XTS, + .is_xts =3D true, + .alg.skcipher.base =3D { + .base.cra_name =3D "xts(aes)", + .base.cra_driver_name =3D "xts-aes-rk2", + .base.cra_priority =3D 300, + .base.cra_flags =3D + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_blocksize =3D AES_BLOCK_SIZE, + .base.cra_ctxsize =3D + sizeof(struct rk2_cipher_ctx), + .base.cra_alignmask =3D 0, + .base.cra_module =3D THIS_MODULE, + .init =3D rk2_cipher_tfm_init, + .exit =3D rk2_cipher_tfm_exit, + .min_keysize =3D AES_MIN_KEY_SIZE * 2, + .max_keysize =3D AES_MAX_KEY_SIZE * 2, + .ivsize =3D AES_BLOCK_SIZE, + .setkey =3D rk2_aes_xts_setkey, + .encrypt =3D rk2_skcipher_encrypt, + .decrypt =3D rk2_skcipher_decrypt, + }, + .alg.skcipher.op =3D { + .do_one_request =3D rk2_cipher_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_MD5, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D MD5_DIGEST_SIZE, + .statesize =3D sizeof(struct md5_state), + .base =3D { + .cra_name =3D "md5", + .cra_driver_name =3D "rk2-md5", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D + MD5_HMAC_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA1, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA1_DIGEST_SIZE, + .statesize =3D sizeof(struct sha1_state), + .base =3D { + .cra_name =3D "sha1", + .cra_driver_name =3D "rk2-sha1", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA1_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA224, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA224_DIGEST_SIZE, + .statesize =3D sizeof(struct sha256_state), + .base =3D { + .cra_name =3D "sha224", + .cra_driver_name =3D "rk2-sha224", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA256_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA256, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA256_DIGEST_SIZE, + .statesize =3D sizeof(struct sha256_state), + .base =3D { + .cra_name =3D "sha256", + .cra_driver_name =3D "rk2-sha256", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA256_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA384, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA384_DIGEST_SIZE, + .statesize =3D sizeof(struct sha512_state), + .base =3D { + .cra_name =3D "sha384", + .cra_driver_name =3D "rk2-sha384", + .cra_priority =3D 300, + .cra_flags =3D CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA384_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SHA512, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SHA512_DIGEST_SIZE, + .statesize =3D sizeof(struct sha512_state), + .base =3D { + .cra_name =3D "sha512", + .cra_driver_name =3D "rk2-sha512", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SHA512_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, + { + .type =3D CRYPTO_ALG_TYPE_AHASH, + .rk2_mode =3D RK2_CRYPTO_SM3, + .alg.hash.base =3D { + .init =3D rk2_ahash_init, + .update =3D rk2_ahash_update, + .final =3D rk2_ahash_final, + .finup =3D rk2_ahash_finup, + .export =3D rk2_ahash_export, + .import =3D rk2_ahash_import, + .digest =3D rk2_ahash_digest, + .init_tfm =3D rk2_hash_init_tfm, + .exit_tfm =3D rk2_hash_exit_tfm, + .halg =3D { + .digestsize =3D SM3_DIGEST_SIZE, + .statesize =3D sizeof(struct sm3_ctx), + .base =3D { + .cra_name =3D "sm3", + .cra_driver_name =3D "rk2-sm3", + .cra_priority =3D 300, + .cra_flags =3D + CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_FALLBACK, + .cra_blocksize =3D SM3_BLOCK_SIZE, + .cra_ctxsize =3D + sizeof(struct rk2_ahash_ctx), + .cra_module =3D THIS_MODULE, + } + } + }, + .alg.hash.op =3D { + .do_one_request =3D rk2_hash_run, + }, + }, +}; + +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG +static int rk2_crypto_debugfs_stats_show(struct seq_file *seq, void *v) +{ + struct rk2_crypto_dev *rkc =3D seq->private; + unsigned int i; + + seq_printf(seq, "%s %s requests: %lu\n", + dev_driver_string(rkc->dev), dev_name(rkc->dev), atomic_long_read(&rk= c->nreq)); + + for (i =3D 0; i < rkc->num_algs; i++) { + switch (rkc->algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + seq_printf(seq, "%s %s reqs=3D%lu fallback=3D%lu\n", + rkc->algs[i].alg.skcipher.base.base.cra_driver_name, + rkc->algs[i].alg.skcipher.base.base.cra_name, + atomic_long_read(&rkc->algs[i].stat_req), + atomic_long_read(&rkc->algs[i].stat_fb)); + seq_printf(seq, "\tfallback due to length: %lu\n", + atomic_long_read(&rkc->algs[i].stat_fb_len)); + seq_printf(seq, "\tfallback due to alignment: %lu\n", + atomic_long_read(&rkc->algs[i].stat_fb_align)); + seq_printf(seq, "\tfallback due to SGs: %lu\n", + atomic_long_read(&rkc->algs[i].stat_fb_sgdiff)); + break; + case CRYPTO_ALG_TYPE_AHASH: + seq_printf(seq, "%s %s reqs=3D%lu fallback=3D%lu\n", + rkc->algs[i].alg.hash.base.halg.base.cra_driver_name, + rkc->algs[i].alg.hash.base.halg.base.cra_name, + atomic_long_read(&rkc->algs[i].stat_req), + atomic_long_read(&rkc->algs[i].stat_fb)); + break; + } + } + return 0; +} + +static int rk2_crypto_debugfs_info_show(struct seq_file *seq, void *d) +{ + struct rk2_crypto_dev *rkc =3D seq->private; + u32 v; + int err; + + err =3D pm_runtime_resume_and_get(rkc->dev); + if (err) + return err; + + v =3D readl(rkc->reg + RK2_CRYPTO_CLK_CTL); + seq_printf(seq, "CRYPTO_CLK_CTL %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_RST_CTL); + seq_printf(seq, "CRYPTO_RST_CTL %x\n", v); + v =3D readl(rkc->reg + CRYPTO_AES_VERSION); + seq_printf(seq, "CRYPTO_AES_VERSION %x\n", v); + if (v & BIT(17)) + seq_puts(seq, "AES 192\n"); + v =3D readl(rkc->reg + CRYPTO_DES_VERSION); + seq_printf(seq, "CRYPTO_DES_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_SM4_VERSION); + seq_printf(seq, "CRYPTO_SM4_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_HASH_VERSION); + seq_printf(seq, "CRYPTO_HASH_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_HMAC_VERSION); + seq_printf(seq, "CRYPTO_HMAC_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_RNG_VERSION); + seq_printf(seq, "CRYPTO_RNG_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_PKA_VERSION); + seq_printf(seq, "CRYPTO_PKA_VERSION %x\n", v); + v =3D readl(rkc->reg + CRYPTO_CRYPTO_VERSION); + seq_printf(seq, "CRYPTO_CRYPTO_VERSION %x\n", v); + + pm_runtime_mark_last_busy(rkc->dev); + pm_runtime_put_autosuspend(rkc->dev); + + return 0; +} + +DEFINE_SHOW_ATTRIBUTE(rk2_crypto_debugfs_stats); +DEFINE_SHOW_ATTRIBUTE(rk2_crypto_debugfs_info); + +#endif + +static void register_debugfs(struct rk2_crypto_dev *rkc) +{ +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + /* Create a directory using the device name + * (e.g., /sys/kernel/debug/fe370000.crypto) + */ + rkc->dbgfs_dir =3D debugfs_create_dir(dev_name(rkc->dev), NULL); + + debugfs_create_file("stats", 0440, rkc->dbgfs_dir, rkc, + &rk2_crypto_debugfs_stats_fops); + debugfs_create_file("info", 0440, rkc->dbgfs_dir, rkc, + &rk2_crypto_debugfs_info_fops); +#endif +} + +static int rk2_crypto_register(struct rk2_crypto_dev *rkc) +{ + int i, k, err =3D 0; + + for (i =3D 0; i < rkc->num_algs; i++) { + rkc->algs[i].dev =3D rkc; /* Tie this alg copy to this device */ + switch (rkc->algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + err =3D crypto_engine_register_skcipher(&rkc->algs[i].alg.skcipher); + break; + case CRYPTO_ALG_TYPE_AHASH: + err =3D crypto_engine_register_ahash(&rkc->algs[i].alg.hash); + break; + } + if (err) + goto err_cipher_algs; + } + return 0; + + err_cipher_algs: + for (k =3D 0; k < i; k++) { + if (rkc->algs[k].type =3D=3D CRYPTO_ALG_TYPE_SKCIPHER) + crypto_engine_unregister_skcipher(&rkc->algs[k].alg.skcipher); + else + crypto_engine_unregister_ahash(&rkc->algs[k].alg.hash); + } + return err; +} + +static void rk2_crypto_unregister(struct rk2_crypto_dev *rkc) +{ + int i; + + for (i =3D 0; i < rkc->num_algs; i++) { + if (rkc->algs[i].type =3D=3D CRYPTO_ALG_TYPE_SKCIPHER) + crypto_engine_unregister_skcipher(&rkc->algs[i].alg.skcipher); + else + crypto_engine_unregister_ahash(&rkc->algs[i].alg.hash); + } +} + +static const struct of_device_id crypto_of_id_table[] =3D { + {.compatible =3D "rockchip,rk3568-crypto", + .data =3D &rk3568_variant, + }, + {.compatible =3D "rockchip,rk3588-crypto", + .data =3D &rk3588_variant, + }, + {} +}; + +MODULE_DEVICE_TABLE(of, crypto_of_id_table); + +static int rk2_crypto_probe(struct platform_device *pdev) +{ + struct device *dev =3D &pdev->dev; + struct rk2_crypto_dev *rkc; + int err =3D 0; + + rkc =3D devm_kzalloc(dev, sizeof(*rkc), GFP_KERNEL); + if (!rkc) + return -ENOMEM; + + rkc->dev =3D dev; + platform_set_drvdata(pdev, rkc); + + /* + * Duplicate the algorithms locally for this specific device. + * There is exactly one crypto block per SoC (crypto@fe370000 / crypto@fe= 380000) + * so -EEXIST cannot trigger on real hardware. + */ + rkc->num_algs =3D ARRAY_SIZE(rk2_crypto_algs_template); + rkc->algs =3D devm_kmemdup(dev, rk2_crypto_algs_template, + sizeof(rk2_crypto_algs_template), GFP_KERNEL); + if (!rkc->algs) + return -ENOMEM; + + rkc->variant =3D of_device_get_match_data(dev); + if (!rkc->variant) + return dev_err_probe(dev, -EINVAL, "Missing variant\n"); + + rkc->rst =3D devm_reset_control_array_get_exclusive(dev); + if (IS_ERR(rkc->rst)) + return dev_err_probe(dev, PTR_ERR(rkc->rst), "Fail to get resets\n"); + + err =3D dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); + if (err) + return dev_err_probe(dev, err, "No usable 32-bit DMA configuration\n"); + + /* Manual DMA allocation requires manual cleanup in error paths */ + rkc->tl =3D dma_alloc_coherent(dev, + sizeof(struct rk2_crypto_lli) * MAX_LLI, + &rkc->t_phy, GFP_KERNEL); + + if (!rkc->tl) + return -ENOMEM; + + rkc->reg =3D devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(rkc->reg)) { + err =3D dev_err_probe(dev, PTR_ERR(rkc->reg), "Fail to get resources\n"); + goto err_dma; + } + + err =3D rk2_crypto_get_clks(rkc); + if (err) + goto err_dma; + + rkc->irq =3D platform_get_irq(pdev, 0); + if (rkc->irq < 0) { + err =3D dev_err_probe(dev, rkc->irq, "Interrupt is not available.\n"); + goto err_dma; + } + + rkc->engine =3D crypto_engine_alloc_init(dev, true); + if (!rkc->engine) { + err =3D -ENOMEM; + goto err_dma; + } + + err =3D crypto_engine_start(rkc->engine); + if (err) { + err =3D dev_err_probe(dev, err, "Failed to start crypto engine\n"); + goto err_engine; + } + + init_completion(&rkc->complete); + + err =3D rk2_crypto_pm_init(rkc); + if (err) { + err =3D dev_err_probe(dev, err, "Failed to initialize runtime PM\n"); + goto err_engine; + } + + err =3D pm_runtime_resume_and_get(dev); + if (err) { + err =3D dev_err_probe(dev, err, "Failed to resume device\n"); + goto err_pm; + } + + err =3D devm_request_irq(dev, rkc->irq, + rk2_crypto_irq_handle, 0, + "rk-crypto", pdev); + if (err) { + err =3D dev_err_probe(dev, err, "IRQ request failed.\n"); + goto err_pm_put; + } + + err =3D rk2_crypto_register(rkc); + if (err) { + err =3D dev_err_probe(dev, err, "Fail to register crypto algorithms\n"); + goto err_pm_put; + } + + register_debugfs(rkc); + + pm_runtime_mark_last_busy(dev); + pm_runtime_put_autosuspend(dev); + + return 0; + + err_pm_put: + pm_runtime_put_sync(dev); + err_pm: + rk2_crypto_pm_exit(rkc); + err_engine: + crypto_engine_exit(rkc->engine); + err_dma: + dma_free_coherent(dev, sizeof(struct rk2_crypto_lli) * MAX_LLI, + rkc->tl, rkc->t_phy); + return err; +} + +static void rk2_crypto_remove(struct platform_device *pdev) +{ + struct rk2_crypto_dev *rkc =3D platform_get_drvdata(pdev); + + crypto_engine_stop(rkc->engine); + rk2_crypto_unregister(rkc); +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + debugfs_remove_recursive(rkc->dbgfs_dir); +#endif + crypto_engine_exit(rkc->engine); + + pm_runtime_get_sync(rkc->dev); + pm_runtime_dont_use_autosuspend(rkc->dev); + pm_runtime_put_sync(rkc->dev); + rk2_crypto_pm_exit(rkc); + + dma_free_coherent(rkc->dev, sizeof(struct rk2_crypto_lli) * MAX_LLI, + rkc->tl, rkc->t_phy); +} + +static struct platform_driver crypto_driver =3D { + .probe =3D rk2_crypto_probe, + .remove =3D rk2_crypto_remove, + .driver =3D { + .name =3D "rk2-crypto", + .pm =3D pm_ptr(&rk2_crypto_pm_ops), + .of_match_table =3D crypto_of_id_table, + }, +}; + +module_platform_driver(crypto_driver); + +MODULE_DESCRIPTION("Rockchip Crypto Engine cryptographic offloader"); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Corentin Labbe "); +MODULE_AUTHOR("Dawid Olesinski "); + diff --git a/drivers/crypto/rockchip/rk2_crypto.h b/drivers/crypto/rockchip= /rk2_crypto.h new file mode 100644 index 000000000000..8792d9f1c847 --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto.h @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Hardware cryptographic offloader for RK3568/RK3588 SoC + * + * Copyright (c) 2022-2023 Corentin Labbe + * Copyright (c) 2026 Dawid Olesinski + */ +#ifndef __RK2_CRYPTO_H__ +#define __RK2_CRYPTO_H__ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define RK2_CRYPTO_CLK_CTL 0x0000 +#define RK2_CRYPTO_RST_CTL 0x0004 + +#define RK2_CRYPTO_DMA_INT_EN 0x0008 + +/* RK2_CRYPTO_DMA_INT_ST / RK2_CRYPTO_DMA_INT_EN bit definitions */ +#define RK2_CRYPTO_DMA_INT_LISTDONE BIT(0) /* LLI list complete */ +#define RK2_CRYPTO_DMA_INT_SRC_ITEM_INT BIT(1) /* per-entry src interru= pt */ +#define RK2_CRYPTO_DMA_INT_DST_ITEM_INT BIT(2) /* per-entry dst interru= pt */ +#define RK2_CRYPTO_DMA_INT_LIST_SRC_ERR BIT(3) /* LLI src error */ +#define RK2_CRYPTO_DMA_INT_LIST_DST_ERR BIT(4) /* LLI dst error */ +#define RK2_CRYPTO_DMA_INT_SRC_ERR BIT(5) /* DMA src error */ +#define RK2_CRYPTO_DMA_INT_DST_ERR BIT(6) /* DMA dst error */ + +#define RK2_CRYPTO_DMA_INT_ERR_MASK (RK2_CRYPTO_DMA_INT_LIST_SRC_ERR = | \ + RK2_CRYPTO_DMA_INT_LIST_DST_ERR | \ + RK2_CRYPTO_DMA_INT_SRC_ERR | \ + RK2_CRYPTO_DMA_INT_DST_ERR) + +#define RK2_CRYPTO_DMA_INT_ALL_MASK 0x7F + +/* The DMA interrupt enable register uses the upper 16 bits as a write-ena= ble mask. + * To enable bits 0-6, we must write 1s to bits 16-22 as well. + */ +#define RK2_CRYPTO_DMA_INT_ENABLE_ALL ((RK2_CRYPTO_DMA_INT_ALL_MASK << 16)= | \ + RK2_CRYPTO_DMA_INT_ALL_MASK) + +/* values in RK2_CRYPTO_DMA_INT_ST are the same as in RK2_CRYPTO_DMA_INT_E= N */ +#define RK2_CRYPTO_DMA_INT_ST 0x000C + +#define RK2_CRYPTO_DMA_CTL 0x0010 +#define RK2_CRYPTO_DMA_CTL_START BIT(0) + +#define RK2_CRYPTO_DMA_LLI_ADDR 0x0014 +#define RK2_CRYPTO_DMA_ST 0x0018 +#define RK2_CRYPTO_DMA_STATE 0x001C +#define RK2_CRYPTO_DMA_LLI_RADDR 0x0020 +#define RK2_CRYPTO_DMA_SRC_RADDR 0x0024 +#define RK2_CRYPTO_DMA_DST_WADDR 0x0028 +#define RK2_CRYPTO_DMA_ITEM_ID 0x002C + +#define RK2_CRYPTO_FIFO_CTL 0x0040 + +#define RK2_CRYPTO_BC_CTL 0x0044 +#define RK2_CRYPTO_AES (0 << 8) +#define RK2_CRYPTO_MODE_ECB (0 << 4) +#define RK2_CRYPTO_MODE_CBC (1 << 4) +#define RK2_CRYPTO_XTS (6 << 4) + +#define RK2_CRYPTO_HASH_CTL 0x0048 +#define RK2_CRYPTO_HW_PAD BIT(2) +#define RK2_CRYPTO_SHA1 (0 << 4) +#define RK2_CRYPTO_MD5 (1 << 4) +#define RK2_CRYPTO_SHA224 (3 << 4) +#define RK2_CRYPTO_SHA256 (2 << 4) +#define RK2_CRYPTO_SHA384 (9 << 4) +#define RK2_CRYPTO_SHA512 (8 << 4) +#define RK2_CRYPTO_SM3 (4 << 4) + +#define RK2_CRYPTO_AES_ECB (RK2_CRYPTO_AES | RK2_CRYPTO_MODE_ECB) +#define RK2_CRYPTO_AES_CBC (RK2_CRYPTO_AES | RK2_CRYPTO_MODE_CBC) +#define RK2_CRYPTO_AES_XTS (RK2_CRYPTO_AES | RK2_CRYPTO_XTS) +#define RK2_CRYPTO_AES_128BIT_key (0 << 2) +#define RK2_CRYPTO_AES_192BIT_key (1 << 2) +#define RK2_CRYPTO_AES_256BIT_key (2 << 2) + +#define RK2_CRYPTO_DEC BIT(1) +#define RK2_CRYPTO_ENABLE BIT(0) + +#define RK2_CRYPTO_CIPHER_ST 0x004C +#define RK2_CRYPTO_CIPHER_STATE 0x0050 + +#define RK2_CRYPTO_CH0_IV_0 0x0100 + +#define RK2_CRYPTO_KEY0 0x0180 +#define RK2_CRYPTO_KEY1 0x0184 +#define RK2_CRYPTO_KEY2 0x0188 +#define RK2_CRYPTO_KEY3 0x018C +#define RK2_CRYPTO_KEY4 0x0190 +#define RK2_CRYPTO_KEY5 0x0194 +#define RK2_CRYPTO_KEY6 0x0198 +#define RK2_CRYPTO_KEY7 0x019C +#define RK2_CRYPTO_CH4_KEY0 0x01c0 + +#define RK2_CRYPTO_CH0_PC_LEN_0 0x0280 + +#define RK2_CRYPTO_CH0_IV_LEN 0x0300 + +#define RK2_CRYPTO_HASH_DOUT_0 0x03A0 +#define RK2_CRYPTO_HASH_VALID 0x03E4 + +#define RK2_CRYPTO_TRNG_CTL 0x0400 +#define RK2_CRYPTO_TRNG_START BIT(0) +#define RK2_CRYPTO_TRNG_ENABLE BIT(1) +#define RK2_CRYPTO_TRNG_256 (0x3 << 4) +#define RK2_CRYPTO_TRNG_SAMPLE_CNT 0x0404 +#define RK2_CRYPTO_TRNG_DOUT 0x0410 + +#define CRYPTO_AES_VERSION 0x0680 +#define CRYPTO_DES_VERSION 0x0684 +#define CRYPTO_SM4_VERSION 0x0688 +#define CRYPTO_HASH_VERSION 0x068C +#define CRYPTO_HMAC_VERSION 0x0690 +#define CRYPTO_RNG_VERSION 0x0694 +#define CRYPTO_PKA_VERSION 0x0698 +#define CRYPTO_CRYPTO_VERSION 0x06F0 + +#define RK2_LLI_DMA_CTRL_SRC_INT BIT(10) +#define RK2_LLI_DMA_CTRL_DST_INT BIT(9) +#define RK2_LLI_DMA_CTRL_LIST_INT BIT(8) +#define RK2_LLI_DMA_CTRL_LAST BIT(0) + +#define RK2_LLI_STRING_LAST BIT(2) +#define RK2_LLI_STRING_FIRST BIT(1) +#define RK2_LLI_CIPHER_START BIT(0) + +#define MAX_LLI 20 + +struct rk2_crypto_lli { + __le32 src_addr; + __le32 src_len; + __le32 dst_addr; + __le32 dst_len; + __le32 user; + __le32 iv; + __le32 dma_ctrl; + __le32 next; +}; + +struct rk2_variant { + int num_clks; +}; + +struct rk2_crypto_dev { + struct device *dev; + struct clk_bulk_data *clks; + int num_clks; + struct reset_control *rst; + void __iomem *reg; + int irq; + const struct rk2_variant *variant; + atomic_long_t nreq; + struct crypto_engine *engine; + struct completion complete; + int status; + struct rk2_crypto_lli *tl; + dma_addr_t t_phy; + struct rk2_crypto_template *algs; + int num_algs; +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + struct dentry *dbgfs_dir; +#endif +}; + +/* the private variable of hash */ +struct rk2_ahash_ctx { + /* for fallback */ + struct crypto_ahash *fallback_tfm; +}; + +/* the private variable of hash for fallback */ +struct rk2_ahash_rctx { + struct rk2_crypto_dev *dev; + u32 mode; + int nrsgs; + struct ahash_request fallback_req; +}; + +/* the private variable of cipher */ +struct rk2_cipher_ctx { + unsigned int keylen; + u8 key[AES_MAX_KEY_SIZE * 2]; + struct crypto_skcipher *fallback_tfm; +}; + +struct rk2_cipher_rctx { + struct rk2_crypto_dev *dev; + u8 backup_iv[AES_BLOCK_SIZE]; + u32 mode; + /* must be last, see __ctx placement */ + struct skcipher_request fallback_req; +}; + +struct rk2_crypto_template { + u32 type; + u32 rk2_mode; + bool is_xts; + struct rk2_crypto_dev *dev; + union { + struct skcipher_engine_alg skcipher; + struct ahash_engine_alg hash; + } alg; + atomic_long_t stat_req; + atomic_long_t stat_fb; + atomic_long_t stat_fb_len; + atomic_long_t stat_fb_sglen; + atomic_long_t stat_fb_align; + atomic_long_t stat_fb_sgdiff; +}; + +int rk2_cipher_run(struct crypto_engine *engine, void *async_req); +int rk2_hash_run(struct crypto_engine *engine, void *breq); + +int rk2_cipher_tfm_init(struct crypto_skcipher *tfm); +void rk2_cipher_tfm_exit(struct crypto_skcipher *tfm); +int rk2_aes_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen); +int rk2_aes_xts_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen); +int rk2_skcipher_encrypt(struct skcipher_request *req); +int rk2_skcipher_decrypt(struct skcipher_request *req); + +int rk2_ahash_init(struct ahash_request *req); +int rk2_ahash_update(struct ahash_request *req); +int rk2_ahash_final(struct ahash_request *req); +int rk2_ahash_finup(struct ahash_request *req); +int rk2_ahash_import(struct ahash_request *req, const void *in); +int rk2_ahash_export(struct ahash_request *req, void *out); +int rk2_ahash_digest(struct ahash_request *req); +int rk2_hash_init_tfm(struct crypto_ahash *tfm); +void rk2_hash_exit_tfm(struct crypto_ahash *tfm); + +#endif /* __RK2_CRYPTO_H__ */ diff --git a/drivers/crypto/rockchip/rk2_crypto_ahash.c b/drivers/crypto/ro= ckchip/rk2_crypto_ahash.c new file mode 100644 index 000000000000..475eb82a5f0d --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto_ahash.c @@ -0,0 +1,565 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Hardware cryptographic offloader for RK3568/RK3588 SoC + * + * Copyright (c) 2022-2023 Corentin Labbe + * Copyright (c) 2026 Dawid Olesinski + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "rk2_crypto.h" + +static bool rk2_ahash_need_fallback(struct ahash_request *areq) +{ + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(areq); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + struct scatterlist *sg; + int nents =3D sg_nents_for_len(areq->src, areq->nbytes); + + /* + * The hardware's Merkle-Damgard padding engine (HW_PAD) requires the + * total message length to be known upfront via CH0_PC_LEN_0. When a + * request spans multiple LLI descriptors, the hardware either treats + * each descriptor as an independent padded message, or loses running + * hash state at descriptor boundaries. Either way the result is a + * wrong digest. This behaviour is not documented in the RK3588 TRM, + * which advertises LLI chaining but does not specify whether hash + * operations may span multiple linked descriptors. + * Work around this by falling back to software for any multi-SG + * request. Single-SG requests with HW_PAD work correctly. + */ + + if (nents < 0) { + dev_err(algt->dev->dev, "Invalid SG list: length mismatch\n"); + return true; /* force fallback safely */ + } + if (nents > 1) { + atomic_long_inc(&algt->stat_fb_sgdiff); + return true; + } + + sg =3D areq->src; + while (sg) { + if (!IS_ALIGNED(sg->offset, sizeof(u32))) { + atomic_long_inc(&algt->stat_fb_align); + return true; + } + if (sg->length % 4) { + atomic_long_inc(&algt->stat_fb_sglen); + return true; + } + sg =3D sg_next(sg); + } + return false; +} + +static int rk2_ahash_digest_fb(struct ahash_request *areq) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(areq); + struct rk2_ahash_ctx *tfmctx =3D crypto_ahash_ctx(tfm); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + + atomic_long_inc(&algt->stat_fb); + + ahash_request_set_tfm(&rctx->fallback_req, tfmctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, areq->base.flags, + areq->base.complete, areq->base.data); + + rctx->fallback_req.nbytes =3D areq->nbytes; + rctx->fallback_req.src =3D areq->src; + rctx->fallback_req.result =3D areq->result; + + return crypto_ahash_digest(&rctx->fallback_req); +} + +static int zero_message_process(struct ahash_request *req) +{ + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + int digestsize =3D crypto_ahash_digestsize(tfm); + + switch (algt->rk2_mode) { + case RK2_CRYPTO_SHA1: + memcpy(req->result, sha1_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SHA224: + memcpy(req->result, sha224_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SHA256: + memcpy(req->result, sha256_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SHA384: + memcpy(req->result, sha384_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SHA512: + memcpy(req->result, sha512_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_MD5: + memcpy(req->result, md5_zero_message_hash, digestsize); + break; + case RK2_CRYPTO_SM3: + sm3((const u8 *)"", 0, req->result); + break; + default: + return -EINVAL; + } + + return 0; +} + +/** + * rk2_ahash_init() - Initialize context for a hash request + * @req: The asynchronous hash request structure. + * + * Initializes the software fallback context. The physical hardware engine + * is only utilized during atomic digest operations. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_init(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + return crypto_ahash_init(&rctx->fallback_req); +} + +/** + * rk2_ahash_update() - Feed a message block into the hash stream + * @req: The asynchronous hash request structure. + * + * Passes the message block to the software fallback. The hardware engine + * does not support fragmented streaming updates. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_update(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + rctx->fallback_req.nbytes =3D req->nbytes; + rctx->fallback_req.src =3D req->src; + + return crypto_ahash_update(&rctx->fallback_req); +} + +/** + * rk2_ahash_final() - Finalize the hashing operation + * @req: The asynchronous hash request structure. + * + * Finalizes the hash and extracts the digest via the software fallback. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_final(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + rctx->fallback_req.result =3D req->result; + + return crypto_ahash_final(&rctx->fallback_req); +} + +/** + * rk2_ahash_finup() - Perform update and final hash operations sequential= ly + * @req: The asynchronous hash request structure. + * + * Convenience wrapper that performs update and final operations + * via the software fallback. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_finup(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + rctx->fallback_req.nbytes =3D req->nbytes; + rctx->fallback_req.src =3D req->src; + rctx->fallback_req.result =3D req->result; + + return crypto_ahash_finup(&rctx->fallback_req); +} + +/** + * rk2_ahash_import() - Restore a saved hash context + * @req: The target asynchronous hash request structure. + * @in: Buffer containing the previously exported state. + * + * Restores the software fallback state from an export block. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_import(struct ahash_request *req, const void *in) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + return crypto_ahash_import(&rctx->fallback_req, in); +} + +/** + * rk2_ahash_export() - Serialize an active hash context + * @req: The source asynchronous hash request structure. + * @out: Destination buffer where the state will be written. + * + * Freezes the progression of the software fallback stream into a byte arr= ay. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_ahash_export(struct ahash_request *req, void *out) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct rk2_ahash_ctx *ctx =3D crypto_ahash_ctx(tfm); + + ahash_request_set_tfm(&rctx->fallback_req, ctx->fallback_tfm); + ahash_request_set_callback(&rctx->fallback_req, req->base.flags, + req->base.complete, req->base.data); + + return crypto_ahash_export(&rctx->fallback_req, out); +} + +/** + * rk2_ahash_digest() - Compute a complete message digest in a single tran= saction + * @req: The asynchronous hash request structure. + * + * Evaluates hardware constraints (e.g., scatterlist alignment) and either + * routes the atomic request to the hardware engine or diverts to the fall= back. + * + * Return: 0 on synchronous completion, -EINPROGRESS if submitted to the + * hardware engine, or a negative error code on failure. + */ +int rk2_ahash_digest(struct ahash_request *req) +{ + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(req); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(req); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt; + struct rk2_crypto_dev *rkc; + struct crypto_engine *engine; + + if (!req->nbytes) + return zero_message_process(req); + + if (rk2_ahash_need_fallback(req)) + return rk2_ahash_digest_fb(req); + + /* Extract the device pointer from the algorithm template! */ + algt =3D container_of(alg, struct rk2_crypto_template, alg.hash.base); + rkc =3D algt->dev; + if (!rkc) + return -ENODEV; + + rctx->dev =3D rkc; + engine =3D rkc->engine; + + return crypto_transfer_hash_request_to_engine(engine, req); +} + +static int rk2_hash_prepare(struct crypto_engine *engine, void *breq) +{ + struct ahash_request *areq =3D + container_of(breq, struct ahash_request, base); + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct rk2_crypto_dev *rkc =3D rctx->dev; + int n =3D sg_nents_for_len(areq->src, areq->nbytes); + int ret; + + if (n < 0) { + dev_err(rkc->dev, "SG list too short for %u bytes\n", areq->nbytes); + return -EINVAL; + } + rctx->nrsgs =3D n; + ret =3D dma_map_sg(rkc->dev, areq->src, rctx->nrsgs, DMA_TO_DEVICE); + if (ret <=3D 0) { + /* + * clear nrsgs on map failure to prevent spurious unmap in unprepare + */ + rctx->nrsgs =3D 0; + return -EINVAL; + } + return 0; +} + +static void rk2_hash_unprepare(struct crypto_engine *engine, void *breq) +{ + struct ahash_request *areq =3D + container_of(breq, struct ahash_request, base); + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct rk2_crypto_dev *rkc =3D rctx->dev; + + if (rctx->nrsgs) + dma_unmap_sg(rkc->dev, areq->src, rctx->nrsgs, DMA_TO_DEVICE); +} + +/** + * rk2_hash_run() - Execute an asynchronous hash request via the hardware + * @engine: The crypto engine queue managing this request. + * @breq: The asynchronous hash request to process. + * + * Configures the hardware hash engine, programs DMA block descriptors, + * and copies the final digest back from the hardware registers. + * + * Return: Always 0. Errors are reported through the crypto engine + * finalization callback. + */ +int rk2_hash_run(struct crypto_engine *engine, void *breq) +{ + struct ahash_request *areq =3D + container_of(breq, struct ahash_request, base); + struct crypto_ahash *tfm =3D crypto_ahash_reqtfm(areq); + struct rk2_ahash_rctx *rctx =3D ahash_request_ctx(areq); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + struct scatterlist *sgs =3D areq->src; + struct rk2_crypto_dev *rkc =3D rctx->dev; + struct rk2_crypto_lli *dd =3D &rkc->tl[0]; + int ddi =3D 0; + int err =3D 0; + unsigned int len =3D areq->nbytes; + unsigned int todo; + unsigned long timeout; + u32 v; + u32 user_val; + u32 dma_ctrl_val; + int i; + + err =3D rk2_hash_prepare(engine, breq); + if (err) + goto exit_unmap; + + err =3D pm_runtime_resume_and_get(rkc->dev); + if (err) + goto exit_unmap; + + dev_dbg(rkc->dev, "%s %s len=3D%u\n", __func__, + crypto_tfm_alg_name(areq->base.tfm), areq->nbytes); + + atomic_long_inc(&algt->stat_req); + atomic_long_inc(&rkc->nreq); + + /* + * The upper bits are a write enable mask, so we need to write 1 to all + * upper 16 bits to allow write to the 16 lower bits + */ + rctx->mode =3D algt->rk2_mode; + rctx->mode |=3D 0xffff0000; + rctx->mode |=3D RK2_CRYPTO_ENABLE | RK2_CRYPTO_HW_PAD; + writel(rctx->mode, rkc->reg + RK2_CRYPTO_HASH_CTL); + + while (sgs && len > 0) { + if (ddi >=3D MAX_LLI) { + dev_err(rkc->dev, + "Too many SG entries (current: %d, max: %d)\n", + ddi, MAX_LLI); + err =3D -EINVAL; + goto exit; + } + dd =3D &rkc->tl[ddi]; + + user_val =3D 0; + dma_ctrl_val =3D ddi << 24; + + todo =3D min(sg_dma_len(sgs), len); + dd->src_addr =3D cpu_to_le32(sg_dma_address(sgs)); + dd->src_len =3D cpu_to_le32(todo); + dd->dst_addr =3D 0; + dd->dst_len =3D 0; + dd->iv =3D 0; + dd->next =3D cpu_to_le32(rkc->t_phy + sizeof(struct rk2_crypto_lli) = * (ddi + 1)); + + if (ddi =3D=3D 0) + user_val =3D RK2_LLI_CIPHER_START | RK2_LLI_STRING_FIRST; + + len -=3D todo; + if (len =3D=3D 0) { + user_val |=3D RK2_LLI_STRING_LAST; + dma_ctrl_val |=3D RK2_LLI_DMA_CTRL_LAST | + RK2_LLI_DMA_CTRL_SRC_INT | + RK2_LLI_DMA_CTRL_LIST_INT; + } + + dd->user =3D cpu_to_le32(user_val); + dd->dma_ctrl =3D cpu_to_le32(dma_ctrl_val); + + dev_dbg(rkc->dev, + "HASH SG %d sglen=3D%u user=3D%x dma=3D%x mode=3D%x len=3D%u todo=3D%u = phy=3D%pad\n", + ddi, + sgs->length, + le32_to_cpu(dd->user), + le32_to_cpu(dd->dma_ctrl), + rctx->mode, + len, + todo, + &rkc->t_phy); + + sgs =3D sg_next(sgs); + ddi++; + } + + /* + * next is ignored by hardware when RK2_LLI_DMA_CTRL_LAST is set in + * dma_ctrl. Set it to an obviously-invalid-but-non-zero sentinel so + * it stands out if ever read in a debug dump. + */ + dd->next =3D cpu_to_le32(1); + + /* Program total payload length for hardware padding */ + writel(areq->nbytes, rkc->reg + RK2_CRYPTO_CH0_PC_LEN_0); + + /* Clear stale interrupts, then enable with proper write-mask */ + writel(RK2_CRYPTO_DMA_INT_ALL_MASK, rkc->reg + RK2_CRYPTO_DMA_INT_ST); + writel(RK2_CRYPTO_DMA_INT_ENABLE_ALL, rkc->reg + RK2_CRYPTO_DMA_INT_EN); + + writel(rkc->t_phy, rkc->reg + RK2_CRYPTO_DMA_LLI_ADDR); + + reinit_completion(&rkc->complete); + rkc->status =3D 0; + + writel(RK2_CRYPTO_DMA_CTL_START | (RK2_CRYPTO_DMA_CTL_START << 16), + rkc->reg + RK2_CRYPTO_DMA_CTL); + + timeout =3D wait_for_completion_timeout(&rkc->complete, + msecs_to_jiffies(2000)); + if (!timeout) { + dev_err(rkc->dev, "DMA timeout\n"); + err =3D -ETIMEDOUT; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + synchronize_irq(rkc->irq); + goto exit; + } + if (!rkc->status) { + dev_err(rkc->dev, "DMA error\n"); + err =3D -EIO; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + goto exit; + } + + err =3D + readl_poll_timeout_atomic(rkc->reg + RK2_CRYPTO_HASH_VALID, v, + v =3D=3D 1, 10, 1000); + if (err) { + dev_err(rkc->dev, "Hash result not valid\n"); + goto exit; + } + + /* + * Hardware outputs digest words in big-endian format. + * Because readl() performs a native little-endian read, + * put_unaligned_be32() is used to store the result correctly + * into the byte array. + */ + for (i =3D 0; i < crypto_ahash_digestsize(tfm) / 4; i++) { + v =3D readl(rkc->reg + RK2_CRYPTO_HASH_DOUT_0 + i * 4); + put_unaligned_be32(v, areq->result + i * 4); + } + exit: + writel(0xffff0000, rkc->reg + RK2_CRYPTO_HASH_CTL); + pm_runtime_mark_last_busy(rkc->dev); + pm_runtime_put_autosuspend(rkc->dev); + + exit_unmap: + rk2_hash_unprepare(engine, breq); + local_bh_disable(); + crypto_finalize_hash_request(engine, breq, err); + local_bh_enable(); + + return 0; +} + +/** + * rk2_hash_init_tfm() - Initialize the transformation context + * @tfm: The crypto ahash handle. + * + * Allocates software fallback transformations required to guarantee + * processing integrity when hardware constraints are violated. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_hash_init_tfm(struct crypto_ahash *tfm) +{ + struct rk2_ahash_ctx *tctx =3D crypto_ahash_ctx(tfm); + const char *alg_name =3D crypto_ahash_alg_name(tfm); + struct ahash_alg *alg =3D crypto_ahash_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.hash.base); + unsigned int fallback_statesize; + + tctx->fallback_tfm =3D crypto_alloc_ahash(alg_name, 0, + CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(tctx->fallback_tfm)) { + dev_err(algt->dev->dev, "Could not load fallback driver.\n"); + return PTR_ERR(tctx->fallback_tfm); + } + + /* Promote statesize if fallback needs more space for export/import */ + fallback_statesize =3D crypto_ahash_statesize(tctx->fallback_tfm); + if (fallback_statesize > crypto_ahash_statesize(tfm)) + crypto_ahash_set_statesize(tfm, fallback_statesize); + + crypto_ahash_set_reqsize(tfm, + sizeof(struct rk2_ahash_rctx) + + crypto_ahash_reqsize(tctx->fallback_tfm)); + return 0; +} + +/** + * rk2_hash_exit_tfm() - Clean up an ahash transformation context + * @tfm: The crypto ahash handle. + * + * Safely frees internal software fallback transformations. + */ +void rk2_hash_exit_tfm(struct crypto_ahash *tfm) +{ + struct rk2_ahash_ctx *tctx =3D crypto_ahash_ctx(tfm); + + crypto_free_ahash(tctx->fallback_tfm); +} diff --git a/drivers/crypto/rockchip/rk2_crypto_skcipher.c b/drivers/crypto= /rockchip/rk2_crypto_skcipher.c new file mode 100644 index 000000000000..8ca1e6959538 --- /dev/null +++ b/drivers/crypto/rockchip/rk2_crypto_skcipher.c @@ -0,0 +1,728 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Hardware cryptographic offloader for RK3568/RK3588 SoC + * + * Copyright (c) 2022-2023 Corentin Labbe + * Copyright (c) 2026 Dawid Olesinski + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include "rk2_crypto.h" + +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG +static void rk2_print(struct rk2_crypto_dev *rkc) +{ + u32 v; + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_ST); + dev_info(rkc->dev, "DMA_ST %x\n", v); + switch (v) { + case 0: + dev_info(rkc->dev, "DMA_ST: DMA IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_ST: DMA BUSY\n"); + break; + default: + dev_err(rkc->dev, "DMA_ST: invalid value\n"); + } + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_STATE); + dev_info(rkc->dev, "DMA_STATE %x\n", v); + + switch (v & 0x3) { + case 0: + dev_info(rkc->dev, "DMA_STATE: DMA DST IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_STATE: DMA DST LOAD\n"); + break; + case 2: + dev_info(rkc->dev, "DMA_STATE: DMA DST WORK\n"); + break; + default: + dev_err(rkc->dev, "DMA DST invalid\n"); + break; + } + switch ((v >> 2) & 0x3) { + case 0: + dev_info(rkc->dev, "DMA_STATE: DMA SRC IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_STATE: DMA SRC LOAD\n"); + break; + case 2: + dev_info(rkc->dev, "DMA_STATE: DMA SRC WORK\n"); + break; + default: + dev_err(rkc->dev, "DMA_STATE: DMA SRC invalid\n"); + break; + } + switch ((v >> 4) & 0x3) { + case 0: + dev_info(rkc->dev, "DMA_STATE: DMA LLI IDLE\n"); + break; + case 1: + dev_info(rkc->dev, "DMA_STATE: DMA LLI LOAD\n"); + break; + case 2: + dev_info(rkc->dev, "DMA_STATE: LLI WORK\n"); + break; + default: + dev_err(rkc->dev, "DMA_STATE: LLI invalid\n"); + break; + } + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_LLI_RADDR); + dev_info(rkc->dev, "DMA_LLI_RADDR %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_SRC_RADDR); + dev_info(rkc->dev, "DMA_SRC_RADDR %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_DST_WADDR); + dev_info(rkc->dev, "DMA_DST_WADDR %x\n", v); + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_ITEM_ID); + dev_info(rkc->dev, "DMA_ITEM_ID %x\n", v); + + v =3D readl(rkc->reg + RK2_CRYPTO_CIPHER_ST); + dev_info(rkc->dev, "CIPHER_ST %x\n", v); + if (v & BIT(0)) + dev_info(rkc->dev, "CIPHER_ST: BLOCK CIPHER BUSY\n"); + else + dev_info(rkc->dev, "CIPHER_ST: BLOCK CIPHER IDLE\n"); + if (v & BIT(2)) + dev_info(rkc->dev, "CIPHER_ST: HASH BUSY\n"); + else + dev_info(rkc->dev, "CIPHER_ST: HASH IDLE\n"); + if (v & BIT(3)) + dev_info(rkc->dev, "CIPHER_ST: OTP KEY VALID\n"); + else + dev_info(rkc->dev, "CIPHER_ST: OTP KEY INVALID\n"); + + v =3D readl(rkc->reg + RK2_CRYPTO_CIPHER_STATE); + dev_info(rkc->dev, "CIPHER_STATE %x\n", v); + switch (v & 0x3) { + case 0: + dev_info(rkc->dev, "serial: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "serial: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "serial: BULK state\n"); + break; + default: + dev_info(rkc->dev, "serial: reserved state\n"); + break; + } + switch ((v >> 2) & 0x3) { + case 0: + dev_info(rkc->dev, "mac_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "mac_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "mac_state: BULK state\n"); + break; + default: + dev_info(rkc->dev, "mac_state: reserved state\n"); + break; + } + switch ((v >> 4) & 0x3) { + case 0: + dev_info(rkc->dev, "parallel_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "parallel_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "parallel_state: BULK state\n"); + break; + default: + dev_info(rkc->dev, "parallel_state: reserved state\n"); + break; + } + switch ((v >> 6) & 0x3) { + case 0: + dev_info(rkc->dev, "ccm_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "ccm_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "ccm_state: NA state\n"); + break; + default: + dev_info(rkc->dev, "ccm_state: reserved state\n"); + break; + } + switch ((v >> 8) & 0xF) { + case 0: + dev_info(rkc->dev, "gcm_state: IDLE state\n"); + break; + case 1: + dev_info(rkc->dev, "gcm_state: PRE state\n"); + break; + case 2: + dev_info(rkc->dev, "gcm_state: NA state\n"); + break; + case 3: + dev_info(rkc->dev, "gcm_state: PC state\n"); + break; + } + switch ((v >> 10) & 0x1F) { + case 0x1: + dev_info(rkc->dev, "hash_state: IDLE state\n"); + break; + case 0x2: + dev_info(rkc->dev, "hash_state: IPAD state\n"); + break; + case 0x4: + dev_info(rkc->dev, "hash_state: TEXT state\n"); + break; + case 0x8: + dev_info(rkc->dev, "hash_state: OPAD state\n"); + break; + case 0x10: + dev_info(rkc->dev, "hash_state: OPAD EXT state\n"); + break; + default: + dev_info(rkc->dev, "hash_state: invalid state\n"); + break; + } + + v =3D readl(rkc->reg + RK2_CRYPTO_DMA_INT_ST); + dev_info(rkc->dev, "RK2_CRYPTO_DMA_INT_ST %x\n", v); +} +#endif + +static int rk2_cipher_need_fallback(struct skcipher_request *req) +{ + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + struct scatterlist *sgs, *sgd; + unsigned int stodo, dtodo, len; + unsigned int bs =3D crypto_skcipher_blocksize(tfm); + + if (!req->cryptlen) + return true; + + /* + * The hardware XTS implementation programs the tweak once before + * DMA starts and cannot update it at SG boundaries. Restrict to + * exactly one source and one destination SG entry. + */ + if (algt->is_xts) { + if (sg_nents_for_len(req->src, req->cryptlen) !=3D 1) + return true; + if (sg_nents_for_len(req->dst, req->cryptlen) !=3D 1) + return true; + } + + len =3D req->cryptlen; + sgs =3D req->src; + sgd =3D req->dst; + + while (len > 0 && sgs && sgd) { + if (!IS_ALIGNED(sgs->offset, sizeof(u32))) { + atomic_long_inc(&algt->stat_fb_align); + return true; + } + if (!IS_ALIGNED(sgd->offset, sizeof(u32))) { + atomic_long_inc(&algt->stat_fb_align); + return true; + } + + stodo =3D min(len, sgs->length); + if (stodo % bs) { + atomic_long_inc(&algt->stat_fb_len); + return true; + } + + dtodo =3D min(len, sgd->length); + if (dtodo % bs) { + atomic_long_inc(&algt->stat_fb_len); + return true; + } + + /* DMA engines usually require symmetrical source/destination chunks */ + if (stodo !=3D dtodo) { + atomic_long_inc(&algt->stat_fb_sgdiff); + return true; + } + + len -=3D stodo; + sgs =3D sg_next(sgs); + sgd =3D sg_next(sgd); + } + + /* If len > 0, the scatterlist was too short for the request */ + return len > 0; +} + +static int rk2_cipher_fallback(struct skcipher_request *areq) +{ + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(areq); + struct rk2_cipher_ctx *op =3D crypto_skcipher_ctx(tfm); + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(areq); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + int err; + + atomic_long_inc(&algt->stat_fb); + + skcipher_request_set_tfm(&rctx->fallback_req, op->fallback_tfm); + skcipher_request_set_callback(&rctx->fallback_req, areq->base.flags, + areq->base.complete, areq->base.data); + skcipher_request_set_crypt(&rctx->fallback_req, areq->src, areq->dst, + areq->cryptlen, areq->iv); + + if (rctx->mode & RK2_CRYPTO_DEC) + err =3D crypto_skcipher_decrypt(&rctx->fallback_req); + else + err =3D crypto_skcipher_encrypt(&rctx->fallback_req); + return err; +} + +static int rk2_cipher_handle_req(struct skcipher_request *req) +{ + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(req); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + struct rk2_crypto_dev *rkc; + struct crypto_engine *engine; + + if (algt->is_xts && ctx->keylen =3D=3D AES_KEYSIZE_192 * 2) + return rk2_cipher_fallback(req); + + if (rk2_cipher_need_fallback(req)) + return rk2_cipher_fallback(req); + + rkc =3D algt->dev; + if (!rkc) + return -ENODEV; + engine =3D rkc->engine; + rctx->dev =3D rkc; + + return crypto_transfer_skcipher_request_to_engine(engine, req); +} + +/** + * rk2_aes_setkey() - Configure the key for standard AES algorithms + * @cipher: The crypto skcipher handle. + * @key: Buffer containing the raw key material. + * @keylen: Length of the key in bytes. + * + * Validates key length, stores the key in the context, and configures + * the software fallback transformation. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_aes_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen) +{ + struct crypto_tfm *tfm =3D crypto_skcipher_tfm(cipher); + struct rk2_cipher_ctx *ctx =3D crypto_tfm_ctx(tfm); + + if (keylen !=3D AES_KEYSIZE_128 && keylen !=3D AES_KEYSIZE_192 && + keylen !=3D AES_KEYSIZE_256) + return -EINVAL; + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + crypto_skcipher_clear_flags(ctx->fallback_tfm, CRYPTO_TFM_REQ_MASK); + crypto_skcipher_set_flags(ctx->fallback_tfm, + crypto_skcipher_get_flags(cipher) & + CRYPTO_TFM_REQ_MASK); + + return crypto_skcipher_setkey(ctx->fallback_tfm, key, keylen); +} + +/** + * rk2_aes_xts_setkey() - Configure the key for AES-XTS mode + * @cipher: The crypto skcipher handle. + * @key: Buffer containing both cipher and tweak keys. + * @keylen: Total length of the key in bytes. + * + * Validates XTS-specific bounds (e.g., FIPS requirements) and configures + * both the hardware context and fallback. + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_aes_xts_setkey(struct crypto_skcipher *cipher, const u8 *key, + unsigned int keylen) +{ + struct crypto_tfm *tfm =3D crypto_skcipher_tfm(cipher); + struct rk2_cipher_ctx *ctx =3D crypto_tfm_ctx(tfm); + int err; + + err =3D xts_verify_key(cipher, key, keylen); + if (err) + return err; + + ctx->keylen =3D keylen; + memcpy(ctx->key, key, keylen); + + crypto_skcipher_clear_flags(ctx->fallback_tfm, CRYPTO_TFM_REQ_MASK); + crypto_skcipher_set_flags(ctx->fallback_tfm, + crypto_skcipher_get_flags(cipher) & + CRYPTO_TFM_REQ_MASK); + + return crypto_skcipher_setkey(ctx->fallback_tfm, key, keylen); +} + +/** + * rk2_skcipher_encrypt() - General skcipher encryption entry point + * @req: The skcipher request structure. + * + * Evaluates hardware constraints and enqueues the request into the crypto + * engine, or diverts to software fallback. + * + * Return: 0 on success, negative error code, or -EINPROGRESS. + */ +int rk2_skcipher_encrypt(struct skcipher_request *req) +{ + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(req); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + + rctx->mode =3D algt->rk2_mode; + return rk2_cipher_handle_req(req); +} + +/** + * rk2_skcipher_decrypt() - General skcipher decryption entry point + * @req: The skcipher request structure. + * + * Evaluates hardware constraints and enqueues the request into the crypto + * engine, or diverts to software fallback. + * + * Return: 0 on success, negative error code, or -EINPROGRESS. + */ +int rk2_skcipher_decrypt(struct skcipher_request *req) +{ + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(req); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(req); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + + rctx->mode =3D algt->rk2_mode | RK2_CRYPTO_DEC; + return rk2_cipher_handle_req(req); +} + +/** + * rk2_cipher_run() - Execute an asynchronous skcipher request + * @engine: The crypto engine queue managing this request. + * @async_req: The asynchronous skcipher request to process. + * + * Prepares the hardware context, configures DMA descriptors, programs + * cipher registers, and triggers the physical cryptographic accelerator. + * + * Return: Always 0. Errors are reported through the crypto engine + * finalization callback. + */ +int rk2_cipher_run(struct crypto_engine *engine, void *async_req) +{ + struct skcipher_request *areq =3D + container_of(async_req, struct skcipher_request, base); + struct crypto_skcipher *tfm =3D crypto_skcipher_reqtfm(areq); + struct rk2_cipher_rctx *rctx =3D skcipher_request_ctx(areq); + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + struct scatterlist *sgs, *sgd; + int err =3D 0; + int ivsize =3D crypto_skcipher_ivsize(tfm); + unsigned int len =3D areq->cryptlen; + unsigned int this_len; + unsigned int todo; + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + struct rk2_crypto_dev *rkc =3D rctx->dev; + struct rk2_crypto_lli *dd =3D &rkc->tl[0]; + u32 m, v; + int i; + unsigned int offset; + unsigned long timeout; + + m =3D rctx->mode | RK2_CRYPTO_ENABLE; + if (algt->is_xts) { + switch (ctx->keylen) { + case AES_KEYSIZE_128 * 2: + m |=3D RK2_CRYPTO_AES_128BIT_key; + break; + case AES_KEYSIZE_256 * 2: + m |=3D RK2_CRYPTO_AES_256BIT_key; + break; + default: + dev_err(rkc->dev, "Invalid key length %u\n", + ctx->keylen); + err =3D -EINVAL; + goto exit_no_pm; + } + } else { + switch (ctx->keylen) { + case AES_KEYSIZE_128: + m |=3D RK2_CRYPTO_AES_128BIT_key; + break; + case AES_KEYSIZE_192: + m |=3D RK2_CRYPTO_AES_192BIT_key; + break; + case AES_KEYSIZE_256: + m |=3D RK2_CRYPTO_AES_256BIT_key; + break; + default: + dev_err(rkc->dev, "Invalid key length %u\n", + ctx->keylen); + err =3D -EINVAL; + goto exit_no_pm; + } + } + + err =3D pm_runtime_resume_and_get(rkc->dev); + if (err) + goto exit_no_pm; + + atomic_long_inc(&algt->stat_req); + atomic_long_inc(&rkc->nreq); + + /* the upper bits are a write enable mask, so we need to write 1 to all + * upper 16 bits to allow write to the 16 lower bits + */ + m |=3D 0xffff0000; + + dev_dbg(rkc->dev, "%s %s len=3D%u keylen=3D%u mode=3D%x\n", __func__, + crypto_tfm_alg_name(areq->base.tfm), + areq->cryptlen, ctx->keylen, m); + sgs =3D areq->src; + sgd =3D areq->dst; + + while (sgs && sgd && len) { + if (!sgs->length) { + sgs =3D sg_next(sgs); + sgd =3D sg_next(sgd); + continue; + } + + this_len =3D min(len, sgs->length); + + if (areq->iv && crypto_skcipher_ivsize(tfm) > 0) { + if (rctx->mode & RK2_CRYPTO_DEC) { + offset =3D this_len - ivsize; + scatterwalk_map_and_copy(rctx->backup_iv, sgs, + offset, ivsize, 0); + } + } + + dev_dbg(rkc->dev, "SG len=3D%u mode=3D%x ivsize=3D%u\n", sgs->length, + m, ivsize); + + if (sgs =3D=3D sgd) { + err =3D dma_map_sg(rkc->dev, sgs, 1, DMA_BIDIRECTIONAL); + if (err !=3D 1) { + dev_err(rkc->dev, "Invalid sg number %d\n", + err); + err =3D -EINVAL; + goto exit; + } + } else { + err =3D dma_map_sg(rkc->dev, sgs, 1, DMA_TO_DEVICE); + if (err !=3D 1) { + dev_err(rkc->dev, "Invalid sg number %d\n", + err); + err =3D -EINVAL; + goto exit; + } + err =3D dma_map_sg(rkc->dev, sgd, 1, DMA_FROM_DEVICE); + if (err !=3D 1) { + dev_err(rkc->dev, "Invalid sg number %d\n", + err); + err =3D -EINVAL; + dma_unmap_sg(rkc->dev, sgs, 1, DMA_TO_DEVICE); + goto exit; + } + } + err =3D 0; + writel(m, rkc->reg + RK2_CRYPTO_BC_CTL); + + if (algt->is_xts) { + for (i =3D 0; i < ctx->keylen / 8; i++) { + writel(get_unaligned_be32(ctx->key + i * 4), + rkc->reg + RK2_CRYPTO_KEY0 + i * 4); + } + for (i =3D 0; i < (ctx->keylen / 8); i++) { + writel(get_unaligned_be32(ctx->key + (ctx->keylen / 8) * 4 + i * 4), + rkc->reg + RK2_CRYPTO_CH4_KEY0 + i * 4); + } + } else { + for (i =3D 0; i < ctx->keylen / 4; i++) { + writel(get_unaligned_be32(ctx->key + i * 4), + rkc->reg + RK2_CRYPTO_KEY0 + i * 4); + } + } + + if (ivsize) { + for (i =3D 0; i < ivsize / 4; i++) + writel(get_unaligned_be32(areq->iv + i * 4), + rkc->reg + RK2_CRYPTO_CH0_IV_0 + i * 4); + writel(ivsize, rkc->reg + RK2_CRYPTO_CH0_IV_LEN); + } + + /* + * Process one SG entry per DMA operation. The cipher engine requires + * the IV to be updated between SG entries for CBC and XTS modes; + * the backup_iv mechanism handles this correctly for decryption. + * Building a full multi-descriptor chain is possible but adds + * complexity for no measurable throughput gain on typical workloads. + */ + todo =3D min(sg_dma_len(sgs), len); + len -=3D todo; + dd->src_addr =3D cpu_to_le32(sg_dma_address(sgs)); + dd->src_len =3D cpu_to_le32(todo); + dd->dst_addr =3D cpu_to_le32(sg_dma_address(sgd)); + dd->dst_len =3D cpu_to_le32(todo); + dd->iv =3D 0; + + /* + * next is ignored by hardware when RK2_LLI_DMA_CTRL_LAST is set in + * dma_ctrl. Set it to an obviously-invalid-but-non-zero sentinel so + * it stands out if ever read in a debug dump. + */ + dd->next =3D cpu_to_le32(1); + dd->user =3D cpu_to_le32(RK2_LLI_CIPHER_START | + RK2_LLI_STRING_FIRST | RK2_LLI_STRING_LAST); + dd->dma_ctrl =3D cpu_to_le32(RK2_LLI_DMA_CTRL_DST_INT | + RK2_LLI_DMA_CTRL_LAST | RK2_LLI_DMA_CTRL_LIST_INT); + + /* Clear stale interrupts, then enable with proper write-mask */ + writel(RK2_CRYPTO_DMA_INT_ALL_MASK, rkc->reg + RK2_CRYPTO_DMA_INT_ST); + writel(RK2_CRYPTO_DMA_INT_ENABLE_ALL, rkc->reg + RK2_CRYPTO_DMA_INT_EN); + + writel(rkc->t_phy, rkc->reg + RK2_CRYPTO_DMA_LLI_ADDR); + + reinit_completion(&rkc->complete); + rkc->status =3D 0; + + writel(RK2_CRYPTO_DMA_CTL_START | + (RK2_CRYPTO_DMA_CTL_START << 16), + rkc->reg + RK2_CRYPTO_DMA_CTL); + + timeout =3D wait_for_completion_timeout(&rkc->complete, + msecs_to_jiffies(2000)); + if (sgs =3D=3D sgd) { + dma_unmap_sg(rkc->dev, sgs, 1, DMA_BIDIRECTIONAL); + } else { + dma_unmap_sg(rkc->dev, sgs, 1, DMA_TO_DEVICE); + dma_unmap_sg(rkc->dev, sgd, 1, DMA_FROM_DEVICE); + } + + if (!timeout) { + dev_err(rkc->dev, "DMA timeout\n"); + err =3D -ETIMEDOUT; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + synchronize_irq(rkc->irq); + goto exit; + } + + if (!rkc->status) { + dev_err(rkc->dev, "DMA error\n"); +#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP2_DEBUG + rk2_print(rkc); +#endif + err =3D -EIO; + reset_control_assert(rkc->rst); + udelay(10); + reset_control_deassert(rkc->rst); + goto exit; + } + + if (areq->iv && ivsize > 0) { + offset =3D this_len - ivsize; + if (rctx->mode & RK2_CRYPTO_DEC) { + memcpy(areq->iv, rctx->backup_iv, ivsize); + memzero_explicit(rctx->backup_iv, ivsize); + } else { + scatterwalk_map_and_copy(areq->iv, sgd, offset, + ivsize, 0); + } + } + sgs =3D sg_next(sgs); + sgd =3D sg_next(sgd); + } + exit: + writel(0xffff0000, rkc->reg + RK2_CRYPTO_BC_CTL); + pm_runtime_mark_last_busy(rkc->dev); + pm_runtime_put_autosuspend(rkc->dev); + exit_no_pm: + local_bh_disable(); + crypto_finalize_skcipher_request(engine, areq, err); + local_bh_enable(); + return 0; +} + +/** + * rk2_cipher_tfm_init() - Initialize the transformation context + * @tfm: The crypto skcipher handle. + * + * Allocates the software fallback transformations required when requests + * fail to meet hardware constraints (e.g., severe scatterlist misalignmen= t). + * + * Return: 0 on success, or a negative error code on failure. + */ +int rk2_cipher_tfm_init(struct crypto_skcipher *tfm) +{ + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + const char *name =3D crypto_tfm_alg_name(&tfm->base); + struct skcipher_alg *alg =3D crypto_skcipher_alg(tfm); + struct rk2_crypto_template *algt =3D + container_of(alg, struct rk2_crypto_template, alg.skcipher.base); + + ctx->fallback_tfm =3D + crypto_alloc_skcipher(name, 0, CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(ctx->fallback_tfm)) { + dev_err(algt->dev->dev, + "Cannot allocate fallback for %s %ld\n", name, + PTR_ERR(ctx->fallback_tfm)); + return PTR_ERR(ctx->fallback_tfm); + } + + dev_dbg(algt->dev->dev, "Fallback for %s is %s\n", + crypto_tfm_alg_driver_name(&tfm->base), + crypto_tfm_alg_driver_name(crypto_skcipher_tfm + (ctx->fallback_tfm))); + + tfm->reqsize =3D sizeof(struct rk2_cipher_rctx) + + crypto_skcipher_reqsize(ctx->fallback_tfm); + + return 0; +} + +/** + * rk2_cipher_tfm_exit() - Free skcipher initialization resources + * @tfm: The crypto skcipher handle. + * + * Synchronously releases internal software fallback transformations + * and zeroes out sensitive key material. + */ +void rk2_cipher_tfm_exit(struct crypto_skcipher *tfm) +{ + struct rk2_cipher_ctx *ctx =3D crypto_skcipher_ctx(tfm); + + memzero_explicit(ctx->key, ctx->keylen); + crypto_free_skcipher(ctx->fallback_tfm); +} --=20 2.47.3 From nobody Sun Jul 26 13:25:59 2026 Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3789F478E34 for ; Wed, 8 Jul 2026 17:59:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533554; cv=none; b=QvZBFnM4Kbv18CaEL9ZCm0xwd0ki/FBcLhvs3pot0KTNMtsITPWNmRINUApN9bZSr3Lng5V2TeA/KUDfQk7NmikOccgmF9d+Wb+22yQnvkMxZiuer81cCS4+giAbmJtvVZFY2VRhqjO/oO4OPKqMKIWkaAXkUsl6n4iqhkz7Xuc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533554; c=relaxed/simple; bh=kBDjEzQ80kgpn8wJAREQLaXT5HdibWwvuZRJylSEIBs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Zz5jHI8w2zI65NJmKlZB7FW/dB0FtQQtve904eq1NGZsc1Wy1DvaTWSBOzepzk9xlBt5TPCOALdVRAFlkiErAmkdP/tZit1v3H31oaLErcmWiGoQ8NWKiYgNgYI5/RPllgOR/6p6HQ2D6GLPMWudfKaUHZ+a/aaMq7g8qcNFEZc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NLqYK+Fx; arc=none smtp.client-ip=209.85.218.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NLqYK+Fx" Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-c1254b73d63so113510266b.1 for ; Wed, 08 Jul 2026 10:59:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783533552; x=1784138352; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Sa+hq1ddPX1RzBXoQyE6BmGgQ/wGb1XIxI/SPhDUhIA=; b=NLqYK+FxxhKI3Ept5cV/2oaeL7lvywevLOCb7ZxMk9aRRKRzlPoPPMwY2SK18LNKEy XKRPu37fICoBPkgK1e4pTBiM0DfW1V+i+B44JAlHboWWNBHDXmZjAgPCbcjfISkwMTgL 8NBCsXEkV9m6OeNARyq186dDRDSjDFAUBAPobIr8vmenFb60ggR/iUWOumKu8YmHy56p CCgqOhBjZ+BAxG7RfjlvP7+YfeExv/1UnOvT/5pZdL6S8JLSzB15pGroY8ASM05W55BY +ec5nlGNGAWI9SZ4wv/lx6bt4UeoUDjwgsslLFv+MEjfsLtdixcDkpHk1wXXu7rYmkfc +bTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783533552; x=1784138352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Sa+hq1ddPX1RzBXoQyE6BmGgQ/wGb1XIxI/SPhDUhIA=; b=nADMzdk7urX4bYJmdNvXuey/sdXnwz61fpCu6sFy6cBGXOw+vBtnt9Sdx/EE2E76Wy OqNu2F8WzJpDIPXdLgKaPXMNe9FiPbzSiQzo2zeVtM8uT5poy4id6+S/PpMHCLVlOXIJ UvIIuda77YSy0RTrTYAFF/jTXDA3srdSyboO6DSTpuiBmLjjH2OMe/107snqKeYKyz6R L64zxubD+/vztVn9oFysuHyjtCuWZj9lLnPxUT+NpGY0KBfEz7ODzGjvm69odkGwt03D fqKbqzt5QBzylzDSmCeb328jWF9MM1Nahf34ByhkxqXXerTE4OCV/4+h2kQNlQ0UkL+P EhIQ== X-Forwarded-Encrypted: i=1; AHgh+RpNRSlYMlY6J2zi4tjoTJ2zQBkdUDL6E/JpRKTfNlh4GmgGG/F4XaHaO1jeZ7+ncehk3y8uymznzOydysg=@vger.kernel.org X-Gm-Message-State: AOJu0Yw7hteSoohdfD7b4B/S8nCaJvKqh38f6qTQh07D8c1aUYFvOAhC jgUWKLuP02oh7VBiTHbiMNzMR0UfgURtp6+24k4jlrgRO3/QY7p87z7U X-Gm-Gg: AfdE7cl9ZaBwA7j0sqbwQR2URgXRQoKjrFT8O1VSGsapQsbXVW5p8luiiCiVvMdkU86 Ag55a/fZbdBxUrhu1BxlAOi6PWuOwSLnCjj0OHkSGqJYN3uHB439b7fwelFrAny0SnTt4N19Rd5 rSaSAwLQbd3JltHvAw3y1WKjTAgxcMaWNudzrMk84ogx/WQ+qiM67hwnEjuf55Ct/DjLARb0q/w WWUdfs9yaCTnkl8CTEcbQVwc2TSKW0svyKFoLxHVdkNE5CHw1PIcwGNMyr7aCxj6otF250ftaUa F9ZV+AZeEuIi1oJpIPifviZtLlFcrzk7HpbWALxSOmfZ0HMKhUorjmMwOsYMiJjLlpxwGUJ/Bru N+4SDJxDmdxknygQdwTUkAc9BBBsI3ZqpoHE8/oXZAF71npBYIfFN8Vw7Km1XnubqEvcUfM4pEd zdngHuvg== X-Received: by 2002:a17:907:9406:b0:c12:6a72:5882 with SMTP id a640c23a62f3a-c15cde71af8mr169523166b.5.1783533551631; Wed, 08 Jul 2026 10:59:11 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c15ad9bcc26sm357653666b.34.2026.07.08.10.59.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2026 10:59:11 -0700 (PDT) From: Dawid Olesinski To: Herbert Xu , "David S . Miller" , Rob Herring , Krzysztof Kozlowski Cc: Conor Dooley , Heiko Stuebner , Corentin Labbe , linux-crypto@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH v2 3/4] arm64: dts: rockchip: Add crypto node to rk356x-base Date: Wed, 8 Jul 2026 18:58:24 +0100 Message-ID: <20260708175837.1718437-4-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260708175837.1718437-1-dawidro@gmail.com> References: <20260708175837.1718437-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the device tree node for the V2 cryptographic hardware accelerator on RK356x SoCs (RK3566, RK3568). The IP block sits in the non-secure peripheral domain. Its three clocks (core, aclk, hclk) and reset line are accessible directly through the main non-secure CRU, so no firmware intermediary is required. The node is disabled by default; board files that wish to use hardware crypto offload must enable it. Signed-off-by: Dawid Olesinski --- arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi b/arch/arm64/boo= t/dts/rockchip/rk356x-base.dtsi index a5832895bd39..9de7e7487ca1 100644 --- a/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi +++ b/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi @@ -1112,6 +1112,18 @@ sdhci: mmc@fe310000 { status =3D "disabled"; }; =20 + crypto: crypto@fe380000 { + compatible =3D "rockchip,rk3568-crypto"; + reg =3D <0x0 0xfe380000 0x0 0x2000>; + interrupts =3D ; + clocks =3D <&cru CLK_CRYPTO_NS_CORE>, <&cru ACLK_CRYPTO_NS>, + <&cru HCLK_CRYPTO_NS>; + clock-names =3D "core", "aclk", "hclk"; + resets =3D <&cru SRST_CRYPTO_NS_CORE>; + reset-names =3D "core"; + status =3D "disabled"; + }; + /* * Testing showed that the HWRNG found in RK3566 produces unacceptably * low quality of random data, so the HWRNG isn't enabled for all RK356x --=20 2.47.3 From nobody Sun Jul 26 13:25:59 2026 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B65C49550B for ; Wed, 8 Jul 2026 17:59:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533559; cv=none; b=nN+XR2LJPRKgcmyOB9SHnfOLqQffFlfDURVp7Vxwfr3qxaPVQjKqbRCVFmNuzz8kbkd+jYcf8E8Zc2VtawhvH8SH71DxryInkJIdyq1Yte+eXzSlQtIM2EuNbmNRSIqRg5FXSTI61vHNQ/JFc0xrx9BUdt/uv/7wNy+seaqhmS4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783533559; c=relaxed/simple; bh=g0zBm/HT/JX8+0Ve2noPh/XhrvlAOxSKonPdGSV7PAk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KraETYkpOlWZVFybNnCJLSStvx4+oSkY8Ylngfey39Q8iaS2AoTGm34ndXZNEJYXd4n/wjbxUcEjrVtgkjwLanyCfZD9Fq+GB11SWPeaNWqi4YC3T9LyGyaghtNyNVTVCpucSqr7rCXZsNE15TP7Nk8SINpEO/ye0QufDgC30HQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HTUPiprO; arc=none smtp.client-ip=209.85.218.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HTUPiprO" Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-c1276f8414bso118538966b.0 for ; Wed, 08 Jul 2026 10:59:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783533555; x=1784138355; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=h3RbPrXYdQbVWUvs+F9bureO/gAjNoSjgQ1E6c4QgBI=; b=HTUPiprOuzgdZnrc0eX8kY55b194o0cAPFEbqczJQ+cxhsdwf9tVdkoZ1GdapfWfK7 FlMnBf9AV8J9zM9QfeXGZzMmc7tBB13liJgTwx83Nm6lTDVg+lDWRYQONI3PvpziBPUV pFR4vDE1pqS24Hy+GQGUrf6xF0ioj1zvqaZ6eccK1vRkJbUDkMaXwNtGlyiD3didMm2u HrJlQZz0XNzw/6/c0jRZ4tbSIIqP4R3oa/XKUBvd/wk0D/tS3JefJTRkV8hGMTyaIxSC kgaMZloiBUVTJ1IOSZB5pv55Is1MV0i0MZSMeiSlikQWpsi6MseRIlsrT1CLEP38kAvq n68A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783533555; x=1784138355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=h3RbPrXYdQbVWUvs+F9bureO/gAjNoSjgQ1E6c4QgBI=; b=PpDxLG+W7gx4gyMR4mCE95viJlZ2CBBbI2+pGkyNdf7PwzV10IFO3K6obLWqxvDkov UEDka/w4EU2e3/xKa8c5TqPhFTqO3wP4pbCjxi7Z4ue1wOIWV1w9q2GmrpE1m0UZe5j/ X4tPOMDFJ1VenwCoHwrTQPbirLHKjEsqH5F4d5h+zhrs60J86Gb07X40g21jSi4RUbhr irn6iBE3Fem3KqUnPBKERAsUH8fWnXLmONZ6F1b7BIR9h+UfVBsu1k8D70avLQFXttAA S8w+tAIZl2tldcgUnBDswwi2NQ1030Wq5hdwaiyNkk71fF2cmWxATq4PGUv0HfhynttU ut3g== X-Forwarded-Encrypted: i=1; AHgh+RrbzGd5+ZuhN58r+2aPfdCwE7fZas/cwICUZzuoevEssUBIiOta8vorJUHc3W/wdaWg54RsfHggQcO9Vkc=@vger.kernel.org X-Gm-Message-State: AOJu0Yyus1YocpPST5M2jgw+gXNNRpXBUlNKiLuDbklhazneuUIzSXcc hImuFSAdcpH8Cm0jW9KWbS4ocqRn0+MGe+DXiMPd7nFmIzEAdCYkZVr6/MhOx63C X-Gm-Gg: AfdE7cleMhoqle8E8Or73NSL29aAgqVhpgM3fZfaCMWhWuZ45FCS8odTdreCu3aD9JB zTJmjjqkDFqw4v7qgYGh8+QwddZk2KWifb4K3xQVzsWp5slVHG4bvEqKwTHO6RqGhWdiRnZeUAp mDBX/JIq9pfG5Nj1zUSKFhNQQsbGyuaZMGExiS98z3NrjD/5OxwdIWAp58XsTc9rWs5Zo2FMu1B t7s48iKnFEsXxOgkIZFJigufXVwF74fDZ/tWVP6iw7niODuDvM1/NovTvdn1dX8SP2mP7Er/yNh MZby+uVwF8/pMAnI62BhMLHxaQCfgN53yyqbC6isWxdjFsn5gnfDZ8bIf2Fvjw3h/BwfYsxXSDZ IAwHyksSfUetfmZkCebZb6NcZeV1ZTSyY8uvwxujD4oMe0JQXz1RrLBUQyVp1bjHIJeaKqrr4N1 OpOm/PfnBwtQ1ef0Js X-Received: by 2002:a17:907:3f18:b0:c11:f4d6:48fc with SMTP id a640c23a62f3a-c15ce1c331cmr188941466b.45.1783533555363; Wed, 08 Jul 2026 10:59:15 -0700 (PDT) Received: from olympus.. ([2a0a:ef40:ea3:3f01:2e0:4cff:fe68:285]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-c15ad9bcc26sm357653666b.34.2026.07.08.10.59.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2026 10:59:14 -0700 (PDT) From: Dawid Olesinski To: Herbert Xu , "David S . Miller" , Rob Herring , Krzysztof Kozlowski Cc: Conor Dooley , Heiko Stuebner , Corentin Labbe , linux-crypto@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-kernel@vger.kernel.org, Dawid Olesinski Subject: [PATCH v2 4/4] arm64: dts: rockchip: Add crypto node to rk3588-base Date: Wed, 8 Jul 2026 18:58:25 +0100 Message-ID: <20260708175837.1718437-5-dawidro@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260708175837.1718437-1-dawidro@gmail.com> References: <20260708175837.1718437-1-dawidro@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the device tree node for the V2 cryptographic hardware accelerator on RK3588. On RK3588 the crypto IP sits inside the secure domain controlled by SECURECRU, a register bank that is exclusively accessible to the TrustZone firmware (TF-A). Linux must therefore obtain its clocks and reset line through the ARM SCMI interface provided by the firmware rather than mapping the CRU registers directly. Attempting direct MMIO access to SECURECRU from the non-secure world triggers an asynchronous bus fault. The interrupt uses the four-cell GICv3 format as required by the RK3588 GIC node definition (the fourth cell is the CPU affinity/partition specifier; 0 means no affinity constraint). The node is disabled by default; board files that wish to use hardware crypto offload must enable it. Signed-off-by: Dawid Olesinski --- arch/arm64/boot/dts/rockchip/rk3588-base.dtsi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi b/arch/arm64/boo= t/dts/rockchip/rk3588-base.dtsi index fc1fdbfd3162..a7560b09aeb6 100644 --- a/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi +++ b/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi @@ -2447,6 +2447,18 @@ sdhci: mmc@fe2e0000 { status =3D "disabled"; }; =20 + crypto: crypto@fe370000 { + compatible =3D "rockchip,rk3588-crypto"; + reg =3D <0x0 0xfe370000 0x0 0x2000>; + interrupts =3D ; + clocks =3D <&scmi_clk SCMI_CRYPTO_CORE>, <&scmi_clk SCMI_ACLK_SECURE_NS>, + <&scmi_clk SCMI_HCLK_SECURE_NS>; + clock-names =3D "core", "aclk", "hclk"; + resets =3D <&scmi_reset SCMI_SRST_CRYPTO_CORE>; + reset-names =3D "core"; + status =3D "disabled"; + }; + rng@fe378000 { compatible =3D "rockchip,rk3588-rng"; reg =3D <0x0 0xfe378000 0x0 0x200>; --=20 2.47.3