From nobody Thu Oct 2 06:18:04 2025 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010036.outbound.protection.outlook.com [40.93.198.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE6A531A7E6; Fri, 19 Sep 2025 19:53:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.36 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758311601; cv=fail; b=UgOch0YEYkv4RtlEQR8/2u0oyNkwNXk2/PWn7g0GIND/gPZh5UHTQrbOO+StCLwpD96GHgiW9vIT7v20VJ3VobAg/h1SnSSvwrZithNzEmvz68OBsvE/6npRKZsV69ZLYTW3sEKvhXmcV6h63zKFW42reNQob5XX/FXDhkbjwq0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758311601; c=relaxed/simple; bh=PcPg2b5OhUKmtHs/VTSBZQtLXxZuUIYji+OCpZ4cVjo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pbtsxt4Qn1QSFmqGdGyBa3rIqM92ZmBxx+Ab3WjbxPFexyeE6eHBH1AtL2p5UjKai6VVsVK8oWFZ7gYcHxszORODsUHaY6/FP2phb58GZ602Pio23XexeKFi106/rBdIt6f+Jyd57z0mXNoxaAVTzBil1hfjH0BTqA6L/g2kDig= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ClBEHHF8; arc=fail smtp.client-ip=40.93.198.36 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ClBEHHF8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=q+7y3h/MivMQq7pblrl8nLqJzM05oYnMqSmg/WsYcu4DGZsz3zIV+g0L3cQ+kUmjV80oxcIIgKQJxPUh3B9W+woAQOiHzY2saz7or8Qp7gKwvGXXu/H9O4TvXyiOVOzXABPFcFGjgMv32HTnRU3vwUg6/9Xa0r5uU79ojeG6Smua4UGLsJ6Iaj6ssPqINfiX1c/9anRlaqriS1c8QQduleohMhboaIvTNYMXALc8zntnNj+QOASMGsSjFfsV9uE0C2RryTbKHBFTExq4YVLX//KWmcK8OqPX9XgsFdp4if12Ds5d0T0H5QlNiTMsbBm9MuzUjACo4Fh6dusK23/WkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9GayJZDCZLyH8AF+aL7r3ft7zt59QG2tTZt4jqHQ9Jg=; b=Nx7lFBqhmnHpKuCp+MZOQHnetgNIxgJGYetjkDy1j9er+Z2KfrTvFhctXS3DnXCBF2BbvsUYFCdqkLdC8XOYlWR59NX5DznFuJMJ6aXH1fFpwZU/AvKBCCwvIBm5M+hc7boozr6WXU22ZBq5to2BWnLnn8v6HO+usYyZ/daERfeVn8o+b2xX637YtBNbMuQ/Oj5e21KkuqK6JhhW6NZEoCw+YPDw7yj0QCFgPUxs4x9HgqlfRrDPyM1i/9barZw6XqaZpIAzDHw5CE1C8IlwIm9e5lwhSNdx7cjEPOO+GWIAoHliX1oo3GMVa5f9n0O2eX0rvEjZDXKQlgq0qLkwRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9GayJZDCZLyH8AF+aL7r3ft7zt59QG2tTZt4jqHQ9Jg=; b=ClBEHHF8xlow6ugRkLY+BBEhFvB6XgNMAZcWEK1DwZUkingvUwaXoBWGA4/ldEdoe2V03NFy5erWgKHeQW6O7+7FqXO/Z+icK/44afSaHRSOCkwQcXbjIY75tY8hLLLle4QZHstfJkASXHqrZr+PfsLbFthyrv6jJhcpE9hY1PKQi5lE3Vh3/h/3oLZd/sYhO9g80u/P3AJLySDzjFLmbLjo83prtKeztpMB7AC4fNENUFAdzYojXNtQzNJmr1HDxDsB0P2t0cRo0fcIOCUi84Olr8LnrAkH51KyKyPVG8XbrN5zFO9RZoXhpQljCqYWWtIPM7DuXmvcSZY0AWMkRA== Received: from CH2PR17CA0015.namprd17.prod.outlook.com (2603:10b6:610:53::25) by MW4PR12MB6754.namprd12.prod.outlook.com (2603:10b6:303:1eb::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9137.14; Fri, 19 Sep 2025 19:53:04 +0000 Received: from CH1PEPF0000AD7F.namprd04.prod.outlook.com (2603:10b6:610:53:cafe::73) by CH2PR17CA0015.outlook.office365.com (2603:10b6:610:53::25) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9137.14 via Frontend Transport; Fri, 19 Sep 2025 19:53:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CH1PEPF0000AD7F.mail.protection.outlook.com (10.167.244.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9137.12 via Frontend Transport; Fri, 19 Sep 2025 19:53:03 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.34; Fri, 19 Sep 2025 12:51:38 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Fri, 19 Sep 2025 12:51:37 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.34 via Frontend Transport; Fri, 19 Sep 2025 12:51:37 -0700 From: Ron Li To: , , , , , CC: , , , Ron Li Subject: [PATCH v3 1/3] platform/mellanox/mlxbf_pka: Add core BlueField PKA platform driver Date: Fri, 19 Sep 2025 19:51:30 +0000 Message-ID: <20250919195132.1088515-2-xiangrongl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250919195132.1088515-1-xiangrongl@nvidia.com> References: <20250919195132.1088515-1-xiangrongl@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7F:EE_|MW4PR12MB6754:EE_ X-MS-Office365-Filtering-Correlation-Id: 0f23b014-38e3-4230-6ccf-08ddf7b6258b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?B?QmlRaXA2cm13c2ZxVnFhRTRROU9VWmoySHVCRzlmZ3U3c1JjVHYzTGRiNDFP?= =?utf-8?B?Y0ZxeWtMd1lxVkZraXMybkJ2Q0ZmeU5wSGwvQWt0a1djWTNjYXk3OXNGOEk2?= =?utf-8?B?ZmZ0QUpDbUN6Z1dDdkFKeks1Z2NVREQvUHBJd05pRU5GUHNPZmtOWlZsNk9F?= =?utf-8?B?VVhESFhMVlloM2dhcGFYaDZJTkJoOUhOTklrYW1aUVEwQjZwZGxHbThCK1Av?= =?utf-8?B?NHRCc1J5NTFMTUtQSUhDbHN6SHpjZkU5NmFmcWkwdXBnamUxcUJDU3daM0NS?= =?utf-8?B?QnJTR0N4U1ZyZkRLYldIYTJkekNzZVlmQUs1dzgrcjQ1bUZ3NGptdWFMUGwz?= =?utf-8?B?L1lRZ1h2WEFlS09CcVRDWlB6TkZ6YlMyRmNENGxiWU9TaFBTaFdhWlgxdGRP?= =?utf-8?B?L2VTVHJkWW1aTkJ3OHJnVnJiSVZzdWhBOTk1S1NBOWdxNWVEcFRsWS9Lam1L?= =?utf-8?B?NzBGVEJ5dEJycFBtNjczaFp5NFJDYmpXSElwdnBYVHdNVGsvSEhLWU5zTGx1?= =?utf-8?B?M1Nka1lKV3B2R2k4Nk12RFMzUGQ0NUw5cFhidjh0RG96WHJvYmtxalZ4SGoz?= =?utf-8?B?V3I4VTRxSjJlTFFCK2lGVkluMDk4Zm4zVnovNVA0NUZiM3VkaU9DdTNYbjN1?= =?utf-8?B?UTB5QkV0RjdqbE1HRit5MzUyTnByWmYzREQzMk1Fc3grYXZUWVh0cjZZRDYx?= =?utf-8?B?M3l5V1ZyQ0gzQ3ZSaUZmL2w2Y3haV0ZKVDZ3a0VtdkNwOTVoTXJaNEZJSk1j?= =?utf-8?B?S3FsOSs1YitJaWVvbEo5a3MvRXphTGtGVUF3enVKSDJBRStGVmhZWTRTanhB?= =?utf-8?B?MDQ5VmwvQmtLU2hEalFLZnFiYnNUdzdFR1VjMmhweHpmNEpRcE44Z2NTZUd1?= =?utf-8?B?WHFYcWdXd05STWQ4REVtY2xyV2FtQzdjd2xYd05SRUNoRm9GakFOSUxDVjFY?= =?utf-8?B?NWJ3ZHdvelJ1ZUNHUzhwcEl1UDlybkl4bU8wYnpQMWlVSEdKY0NJVm1Nc0Rl?= =?utf-8?B?NUJ6M3cyMW16YWhKRGxQZjV3TEtwd3RrdlErVE9YdmdJS2toMHpsMnREMjRC?= =?utf-8?B?by9SblhEV1A2bFgxNnhwZm1aajkwUEhaUmNDVnZQUy84OUhuU0hRVW52V0pv?= =?utf-8?B?cHMybVY0aFRBV1VCNUZWT0ZUUUxTamtlNXFSZ2MvWC9uV0VNdHlvbzJJSSs2?= =?utf-8?B?K0FDV1FwUVdBcnpBa0szWVlBa1ZWNnJ3V2J2RnNiQ2tVQ01Rak4yYjdyekpa?= =?utf-8?B?QkF5L1hTZ2VkUkl4WklFZjdCSkUvMjRnSU5OQWVUWFRKazAvaWIvbndueUUv?= =?utf-8?B?ZHYyb2UyQlJJbkdhM2ppY0dhdHhqSUdidnVpenVIZlcyZnVXc3ZuR0Y1aGRa?= =?utf-8?B?U290azU0cjB2blpQV0wvYlFYcWY0MDBSc3FhMXVsZU5kK3BmTC9OZUlCNDQ2?= =?utf-8?B?cGMyaGpwWlVNVXVFU0tVV1hKeGZxMEovYXU4aDkxM25sSmZSSEoxTTRlVlR1?= =?utf-8?B?cVNUdHJaQmdZQThSWm5oWXkwUVgxK1FrY3BtQjhLWHhPbWVXUG9YM3F1UmlP?= =?utf-8?B?Z0pQS2ZKNXE4ZDVQak91eSt4WDVCUm5aRUtoSDBDM2NyV1JVWlY2d09hRnFS?= =?utf-8?B?TFpuTXB6cW1sRE83RXZqK3pUZzRRZFZ5M1dKRU1KTmNoT3NoR0ZOL1A3ZjBw?= =?utf-8?B?cUZoRENYemxteTFEaWgwSG9UdW8xcVNCOWFLZ0ZROW1oMkdZNjJNVnl0cDhF?= =?utf-8?B?MmNQaW5TT0FsYUlZMkhrdTNLL05ubkh6b29WbXEyK1hDWU1EL3poMjNVRzNt?= =?utf-8?B?bGNaOGM3YWZaSlVTekMyY2ErQTNpQlgrSE5UL0pocjNpZzhsWFpNWGVIZis1?= =?utf-8?B?YlpQMCtEMmV5Q2xsSW93Wno2YlBwUVhMK3E0bE5ZcXNRUW01TFpIQWlzaE12?= =?utf-8?B?RUxVaklaVXVSOGgvNExDbUplUHJIVnRtWW5NaERtZW0xNll1dTM5L254WTdo?= =?utf-8?B?MkhVQ1BWRmhoUlc4cklKNmltdU4rSFREQWVaTEpFT1hDUTdKU3h5RW5zWTVj?= =?utf-8?Q?LOG4Zs?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2025 19:53:03.9240 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0f23b014-38e3-4230-6ccf-08ddf7b6258b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD7F.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6754 Add the initial mlxbf_pka driver to support the BlueField DPU Public Key Acceleration (PKA) hardware. The PKA provides a simple, complete framework for crypto public key hardware offload. It supports direct access to the public key hardware resources from the user space, and makes available several arithmetic operations: some basic operations (e.g., addition and multiplication), some complex operations (e.g., modular exponentiation and modular inversion), and high-level operations such as RSA, Diffie-Hellman, Elliptic Curve Cryptography, and the Federal Digital Signature Algorithm (DSA as documented in FIPS-186) public-private key systems. This patch wires up the platform driver and the device/shim layer: - ACPI matching for BF1/BF2/BF3 (MLNXBF10/20/51) - Probe flow to verify boot status, discover EIP154, Window RAM, alt Window= RAM, and CSR resources - Register a PKA "shim" via the device layer, which maps resources and trac= ks per-shim state - Provide shim lifecycle helpers (register/unregister) This driver is placed under drivers/platform/mellanox as it exposes platform resources and does not provide in-kernel crypto offload. Reviewed-by: David Thompson Reviewed-by: Khalil Blaiech Signed-off-by: Ron Li --- .../ABI/testing/sysfs-platform-mellanox-pka | 16 + MAINTAINERS | 8 + drivers/platform/mellanox/Kconfig | 10 + drivers/platform/mellanox/Makefile | 1 + drivers/platform/mellanox/mlxbf_pka/Makefile | 10 + .../mellanox/mlxbf_pka/mlxbf_pka_dev.c | 395 +++++++++++++++++ .../mellanox/mlxbf_pka/mlxbf_pka_dev.h | 298 +++++++++++++ .../mellanox/mlxbf_pka/mlxbf_pka_drv.c | 413 ++++++++++++++++++ 8 files changed, 1151 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-platform-mellanox-pka create mode 100644 drivers/platform/mellanox/mlxbf_pka/Makefile create mode 100644 drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c create mode 100644 drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h create mode 100644 drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c diff --git a/Documentation/ABI/testing/sysfs-platform-mellanox-pka b/Docume= ntation/ABI/testing/sysfs-platform-mellanox-pka new file mode 100644 index 000000000000..cf8dd292c781 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-platform-mellanox-pka @@ -0,0 +1,16 @@ +What: /sys/devices/platform// +Date: Oct 2025 +KernelVersion: 6.18 +Contact: "Ron Li " +Description: + The mlxbf_pka driver to support the BlueField SoC Public Key Acceleratio= n (PKA) + hardware. It supports direct access to the public key hardware resources= from the + user space. + + There are three PKA device IDs that support different BlueField product: + + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D + BlueField-1 MLNXBF10:xx, where xx ranges from '00' to '03' + BlueField-2 MLNXBF20:xx, where xx ranges from '00' to '07' + BlueField-3 MLNXBF51:xx, where xx ranges from '00' to '17' + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/MAINTAINERS b/MAINTAINERS index f6206963efbf..64b995b16d1a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -15755,6 +15755,14 @@ L: linux-i2c@vger.kernel.org S: Supported F: drivers/i2c/busses/i2c-mlxbf.c =20 +MELLANOX BLUEFIELD PKA DRIVER +M: Ron Li +M: Khalil Blaiech +L: platform-driver-x86@vger.kernel.org +S: Supported +F: Documentation/ABI/testing/sysfs-platform-mellanox-pka +F: drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_* + MELLANOX ETHERNET DRIVER (mlx4_en) M: Tariq Toukan L: netdev@vger.kernel.org diff --git a/drivers/platform/mellanox/Kconfig b/drivers/platform/mellanox/= Kconfig index e3afbe62c7f6..b09681c137e6 100644 --- a/drivers/platform/mellanox/Kconfig +++ b/drivers/platform/mellanox/Kconfig @@ -121,4 +121,14 @@ config NVSW_SN2201 C3338R which is one of the Denverton product families. System equipped with Nvidia=C2=AESpectrum-1 32x100GbE Ethernet switch. =20 +config MLXBF_PKA + tristate "Mellanox BlueField Public Key Accelerator driver" + depends on ARM64 && ACPI + help + If you say yes to this option, support will be included for the + Public Key Accelerator device on Mellanox BlueField SoCs. + + This driver can also be built as a module. If so, the module will + be called pka-mlxbf. + endif # MELLANOX_PLATFORM diff --git a/drivers/platform/mellanox/Makefile b/drivers/platform/mellanox= /Makefile index e86723b44c2e..a6535959cde4 100644 --- a/drivers/platform/mellanox/Makefile +++ b/drivers/platform/mellanox/Makefile @@ -5,6 +5,7 @@ # obj-$(CONFIG_MLX_PLATFORM) +=3D mlx-platform.o obj-$(CONFIG_MLXBF_BOOTCTL) +=3D mlxbf-bootctl.o +obj-$(CONFIG_MLXBF_PKA) +=3D mlxbf_pka/ obj-$(CONFIG_MLXBF_PMC) +=3D mlxbf-pmc.o obj-$(CONFIG_MLXBF_TMFIFO) +=3D mlxbf-tmfifo.o obj-$(CONFIG_MLXREG_DPU) +=3D mlxreg-dpu.o diff --git a/drivers/platform/mellanox/mlxbf_pka/Makefile b/drivers/platfor= m/mellanox/mlxbf_pka/Makefile new file mode 100644 index 000000000000..67465a63edb8 --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/Makefile @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause +# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All right= s reserved. +# +# Mellanox BlueField PKA Driver +# + +obj-$(CONFIG_MLXBF_PKA) +=3D mlxbf-pka.o + +mlxbf-pka-objs :=3D mlxbf_pka_drv.o +mlxbf-pka-objs +=3D mlxbf_pka_dev.o diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c new file mode 100644 index 000000000000..0a5db1be6eaa --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c @@ -0,0 +1,395 @@ +// SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause +// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "mlxbf_pka_dev.h" + +struct mlxbf_pka_dev_gbl_config_t mlxbf_pka_gbl_config; + +/* Global PKA shim resource info table. */ +static struct mlxbf_pka_dev_gbl_shim_res_info_t mlxbf_pka_gbl_res_tbl[MLXB= F_PKA_MAX_NUM_IO_BLOCKS]; + +/* Add the resource to the global resource table. */ +static int mlxbf_pka_dev_add_resource(struct mlxbf_pka_dev_res_t *res_ptr,= u32 shim_idx) +{ + u8 res_cnt; + + res_cnt =3D mlxbf_pka_gbl_res_tbl[shim_idx].res_cnt; + if (res_cnt >=3D MLXBF_PKA_DEV_SHIM_RES_CNT) + return -ENOMEM; + + mlxbf_pka_gbl_res_tbl[shim_idx].res_tbl[res_cnt] =3D res_ptr; + mlxbf_pka_gbl_res_tbl[shim_idx].res_cnt++; + + return 0; +} + +/* Remove the resource from the global resource table. */ +static int mlxbf_pka_dev_put_resource(struct mlxbf_pka_dev_res_t *res, u32= shim_idx) +{ + struct mlxbf_pka_dev_res_t *res_ptr; + u8 res_idx; + + for (res_idx =3D 0; res_idx < MLXBF_PKA_DEV_SHIM_RES_CNT; res_idx++) { + res_ptr =3D mlxbf_pka_gbl_res_tbl[shim_idx].res_tbl[res_idx]; + if (!res_ptr || strcmp(res_ptr->name, res->name)) + continue; + + mlxbf_pka_gbl_res_tbl[shim_idx].res_tbl[res_idx] =3D NULL; + mlxbf_pka_gbl_res_tbl[shim_idx].res_cnt--; + break; + } + + /* + * Check whether the resource shares the same memory map; If so, the memo= ry + * map shouldn't be released. + */ + for (res_idx =3D 0; res_idx < MLXBF_PKA_DEV_SHIM_RES_CNT; res_idx++) { + res_ptr =3D mlxbf_pka_gbl_res_tbl[shim_idx].res_tbl[res_idx]; + if (res_ptr && res_ptr->base =3D=3D res->base) + return -EBUSY; + } + + return 0; +} + +static void __iomem *mlxbf_pka_dev_get_resource_ioaddr(u64 res_base, u32 s= him_idx) +{ + struct mlxbf_pka_dev_res_t *res_ptr; + u8 res_cnt, res_idx; + + res_cnt =3D mlxbf_pka_gbl_res_tbl[shim_idx].res_cnt; + if (!res_cnt) + return NULL; + + for (res_idx =3D 0; res_idx < res_cnt; res_idx++) { + res_ptr =3D mlxbf_pka_gbl_res_tbl[shim_idx].res_tbl[res_idx]; + if (res_ptr->base =3D=3D res_base) + return res_ptr->ioaddr; + } + + return NULL; +} + +/* Set PKA device resource config and map io memory if needed. */ +static int mlxbf_pka_dev_set_resource_config(struct device *dev, + struct mlxbf_pka_dev_shim_s *shim, + struct mlxbf_pka_dev_res_t *res_ptr, + u64 res_base, + u64 res_size, + u64 res_type, + char *res_name) +{ + if (res_ptr->status =3D=3D MLXBF_PKA_DEV_RES_STATUS_MAPPED) + return -EPERM; + + switch (res_type) { + case MLXBF_PKA_DEV_RES_TYPE_REG: + res_ptr->base =3D res_base; + break; + case MLXBF_PKA_DEV_RES_TYPE_MEM: + res_ptr->base =3D shim->mem_res.eip154_base + res_base; + break; + default: + return -EINVAL; + } + + res_ptr->size =3D res_size; + res_ptr->type =3D res_type; + res_ptr->name =3D res_name; + res_ptr->status =3D MLXBF_PKA_DEV_RES_STATUS_UNMAPPED; + res_ptr->ioaddr =3D mlxbf_pka_dev_get_resource_ioaddr(res_ptr->base, shim= ->shim_id); + if (!res_ptr->ioaddr) { + if (!devm_request_mem_region(dev, res_ptr->base, res_ptr->size, res_ptr-= >name)) { + dev_err(dev, "failed to get io memory region\n"); + return -EPERM; + } + + res_ptr->ioaddr =3D devm_ioremap(dev, res_ptr->base, res_ptr->size); + if (!res_ptr->ioaddr) { + dev_err(dev, "unable to map io memory into CPU space\n"); + return -ENOMEM; + } + } + + res_ptr->status =3D MLXBF_PKA_DEV_RES_STATUS_MAPPED; + + if (!res_ptr->ioaddr || mlxbf_pka_dev_add_resource(res_ptr, shim->shim_id= )) { + dev_err(dev, "unable to map io memory\n"); + return -ENOMEM; + } + + return 0; +} + +/* Unset PKA device resource config - unmap io memory if needed. */ +void mlxbf_pka_dev_unset_resource_config(struct device *dev, + struct mlxbf_pka_dev_shim_s *shim, + struct mlxbf_pka_dev_res_t *res_ptr) +{ + if (res_ptr->status !=3D MLXBF_PKA_DEV_RES_STATUS_MAPPED) + return; + + if (!res_ptr->ioaddr) + return; + + if (-EBUSY =3D=3D mlxbf_pka_dev_put_resource(res_ptr, shim->shim_id)) + return; + + dev_dbg(dev, "PKA device resource released\n"); + res_ptr->status =3D MLXBF_PKA_DEV_RES_STATUS_UNMAPPED; +} + +/* + * mlxbf_pka_dev_create_shim - Create shim. + * + * Set shim parameters and configure shim resources. + * + * Return: 0 on success, a negative error code on failure. + */ +static int mlxbf_pka_dev_create_shim(struct device *dev, + struct mlxbf_pka_dev_shim_s *shim, + u32 shim_id, + u8 split, + struct mlxbf_pka_dev_mem_res *mem_res) +{ + u64 reg_base; + u64 reg_size; + int ret; + + if (shim->status =3D=3D MLXBF_PKA_SHIM_STATUS_CREATED) + return 0; + + if (shim->status !=3D MLXBF_PKA_SHIM_STATUS_UNDEFINED) { + dev_err(dev, "PKA device must be undefined\n"); + return -EPERM; + } + + if (shim_id > MLXBF_PKA_MAX_NUM_IO_BLOCKS - 1) { + dev_err(dev, "invalid shim identifier\n"); + return -EINVAL; + } + + shim->shim_id =3D shim_id; + shim->mem_res =3D *mem_res; + + if (split) + shim->window_ram_split =3D MLXBF_PKA_SHIM_WINDOW_RAM_SPLIT_ENABLED; + else + shim->window_ram_split =3D MLXBF_PKA_SHIM_WINDOW_RAM_SPLIT_DISABLED; + + /* Set PKA device Buffer RAM config. */ + ret =3D mlxbf_pka_dev_set_resource_config(dev, + shim, + &shim->resources.buffer_ram, + MLXBF_PKA_BUFFER_RAM_BASE, + MLXBF_PKA_BUFFER_RAM_SIZE, + MLXBF_PKA_DEV_RES_TYPE_MEM, + "MLXBF_PKA_BUFFER_RAM"); + if (ret) { + dev_err(dev, "unable to set Buffer RAM config\n"); + return ret; + } + + /* Set PKA device Master Controller register. */ + reg_size =3D PAGE_SIZE; + reg_base =3D mlxbf_pka_dev_get_register_base(shim->mem_res.eip154_base, + MLXBF_PKA_MASTER_SEQ_CTRL_ADDR); + ret =3D mlxbf_pka_dev_set_resource_config(dev, + shim, + &shim->resources.master_seq_ctrl, + reg_base, + reg_size, + MLXBF_PKA_DEV_RES_TYPE_REG, + "MLXBF_PKA_MASTER_SEQ_CTRL"); + if (ret) { + dev_err(dev, "unable to set Master Controller register config\n"); + return ret; + } + + /* Set PKA device AIC registers. */ + reg_size =3D PAGE_SIZE; + reg_base =3D mlxbf_pka_dev_get_register_base(shim->mem_res.eip154_base, + MLXBF_PKA_AIC_POL_CTRL_ADDR); + ret =3D mlxbf_pka_dev_set_resource_config(dev, + shim, + &shim->resources.aic_csr, + reg_base, + reg_size, + MLXBF_PKA_DEV_RES_TYPE_REG, + "MLXBF_PKA_AIC_CSR"); + if (ret) { + dev_err(dev, "unable to set AIC registers config\n"); + return ret; + } + + shim->status =3D MLXBF_PKA_SHIM_STATUS_CREATED; + + return ret; +} + +/* Delete shim and unset shim resources. */ +static int mlxbf_pka_dev_delete_shim(struct device *dev, struct mlxbf_pka_= dev_shim_s *shim) +{ + struct mlxbf_pka_dev_res_t *res_master_seq_ctrl, *res_aic_csr; + struct mlxbf_pka_dev_res_t *res_buffer_ram; + + dev_dbg(dev, "PKA device delete shim\n"); + + if (shim->status =3D=3D MLXBF_PKA_SHIM_STATUS_UNDEFINED) + return 0; + + if (shim->status !=3D MLXBF_PKA_SHIM_STATUS_FINALIZED && + shim->status !=3D MLXBF_PKA_SHIM_STATUS_CREATED) { + dev_err(dev, "PKA device status must be finalized\n"); + return -EPERM; + } + + res_buffer_ram =3D &shim->resources.buffer_ram; + res_master_seq_ctrl =3D &shim->resources.master_seq_ctrl; + res_aic_csr =3D &shim->resources.aic_csr; + + mlxbf_pka_dev_unset_resource_config(dev, shim, res_buffer_ram); + mlxbf_pka_dev_unset_resource_config(dev, shim, res_master_seq_ctrl); + mlxbf_pka_dev_unset_resource_config(dev, shim, res_aic_csr); + + shim->status =3D MLXBF_PKA_SHIM_STATUS_UNDEFINED; + + return 0; +} + +/* + * Initialize PKA IO block referred to as shim. It configures shim's + * parameters and prepares resources by mapping corresponding memory. The + * function also configures shim registers and loads firmware to shim + * internal rams. The struct mlxbf_pka_dev_shim_s passed as input is also + * an output. It returns 0 on success, a negative error code on failure. + */ +static int mlxbf_pka_dev_init_shim(struct device *dev, struct mlxbf_pka_de= v_shim_s *shim) +{ + int ret; + + if (shim->status !=3D MLXBF_PKA_SHIM_STATUS_CREATED) { + dev_err(dev, "PKA device must be created\n"); + return -EPERM; + } + + ret =3D devm_mutex_init(dev, &shim->mutex); + if (ret) + return ret; + + shim->status =3D MLXBF_PKA_SHIM_STATUS_INITIALIZED; + + return ret; +} + +/* Release a given shim. */ +static int mlxbf_pka_dev_release_shim(struct device *dev, struct mlxbf_pka= _dev_shim_s *shim) +{ + int ret =3D 0; + + if (shim->status !=3D MLXBF_PKA_SHIM_STATUS_INITIALIZED && + shim->status !=3D MLXBF_PKA_SHIM_STATUS_STOPPED) { + dev_err(dev, "PKA device must be initialized or stopped\n"); + return -EPERM; + } + + shim->status =3D MLXBF_PKA_SHIM_STATUS_FINALIZED; + + return ret; +} + +/* Return the shim associated with the given identifier. */ +struct mlxbf_pka_dev_shim_s *mlxbf_pka_dev_get_shim(u32 shim_id) +{ + return mlxbf_pka_gbl_config.dev_shims[shim_id]; +} + +int mlxbf_pka_dev_register_shim(struct device *dev, + u32 shim_id, + struct mlxbf_pka_dev_mem_res *mem_res, + struct mlxbf_pka_dev_shim_s **shim) +{ + struct mlxbf_pka_dev_shim_s *shim_ptr; + u8 split; + int ret; + + if (!shim) + return -EINVAL; + + dev_dbg(dev, "register shim id=3D%u\n", shim_id); + + shim_ptr =3D devm_kzalloc(dev, sizeof(*shim_ptr), GFP_KERNEL); + if (!shim_ptr) + return -ENOMEM; + + /* + * Shim state MUST be set to undefined before calling + * 'mlxbf_pka_dev_create_shim' function. + */ + shim_ptr->status =3D MLXBF_PKA_SHIM_STATUS_UNDEFINED; + + /* Set the Window RAM user mode. */ + split =3D MLXBF_PKA_SPLIT_WINDOW_RAM_MODE; + + /* Create PKA shim. */ + ret =3D mlxbf_pka_dev_create_shim(dev, shim_ptr, shim_id, split, mem_res); + if (ret) { + dev_err(dev, "failed to create shim %u\n", shim_id); + mlxbf_pka_dev_delete_shim(dev, shim_ptr); + goto exit_create_shim; + } + + /* Initialize PKA shim. */ + ret =3D mlxbf_pka_dev_init_shim(dev, shim_ptr); + if (ret) { + dev_err(dev, "failed to init shim %u\n", shim_id); + goto exit_init_shim; + } + + mlxbf_pka_gbl_config.dev_shims[shim_ptr->shim_id] =3D shim_ptr; + mlxbf_pka_gbl_config.dev_shims_cnt +=3D 1; + + *shim =3D shim_ptr; + return 0; + +exit_init_shim: + mlxbf_pka_dev_release_shim(dev, shim_ptr); + +exit_create_shim: + mlxbf_pka_dev_delete_shim(dev, shim_ptr); + return ret; +} + +int mlxbf_pka_dev_unregister_shim(struct device *dev, struct mlxbf_pka_dev= _shim_s *shim) +{ + int ret; + + if (!shim) + return -EINVAL; + + mlxbf_pka_gbl_config.dev_shims[shim->shim_id] =3D NULL; + mlxbf_pka_gbl_config.dev_shims_cnt -=3D 1; + + /* Release shim. */ + ret =3D mlxbf_pka_dev_release_shim(dev, shim); + if (ret) + return ret; + + /* Delete shim. */ + return mlxbf_pka_dev_delete_shim(dev, shim); +} diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h new file mode 100644 index 000000000000..df51202f79bd --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h @@ -0,0 +1,298 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause */ +/* SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. */ + +#ifndef __MLXBF_PKA_DEV_H__ +#define __MLXBF_PKA_DEV_H__ + +/* + * @file + * + * API to handle the PKA EIP-154 I/O block (shim). It provides functions a= nd + * data structures to initialize and configure the PKA shim. It's the "sou= thband + * interface" for communication with PKA hardware resources. + */ + +#include +#include +#include +#include +#include +#include +#include + +#define MASTER_CONTROLLER_OUT_OF_RESET 0 + +/* PKA address related definitions. */ + +/* + * Global Control Space CSR addresses/offsets. These are accessed from the= ARM + * as 8 byte reads/writes. However only the bottom 32 bits are implemented. + */ +#define MLXBF_PKA_CLK_FORCE_ADDR 0x11C80 + +/* + * Advanced Interrupt Controller CSR addresses/offsets. These are accessed= from + * the ARM as 8 byte reads/writes. However only the bottom 32 bits are + * implemented. + */ +#define MLXBF_PKA_AIC_POL_CTRL_ADDR 0x11E00 + +/* + * Control register address/offset. This is accessed from the ARM using 8 = byte + * reads/writes. However only the bottom 32 bits are implemented. + */ +#define MLXBF_PKA_MASTER_SEQ_CTRL_ADDR 0x27F90 + +/* PKA buffer RAM */ +#define MLXBF_PKA_BUFFER_RAM_BASE 0x00000 +#define MLXBF_PKA_BUFFER_RAM_SIZE SZ_16K + +/* + * PKA Buffer RAM offsets. These are NOT real CSR's but instead are specif= ic + * offset/addresses within the EIP154 MLXBF_PKA_BUFFER_RAM. + */ + +/* Alternate Window RAM size. */ +#define MLXBF_PKA_WINDOW_RAM_REGION_SIZE SZ_16K + +/* PKA configuration related definitions. */ + +/* The maximum number of PKA shims referred to as IO blocks. */ +#define MLXBF_PKA_MAX_NUM_IO_BLOCKS 24 + +/* + * PKA Window RAM parameters. + * Define whether to split window RAM during PKA device creation phase. + */ +#define MLXBF_PKA_SPLIT_WINDOW_RAM_MODE 0 + +/* Defines for window RAM partition, valid for 16K memory. */ +#define MLXBF_PKA_WINDOW_RAM_DATA_MEM_SIZE 0x3800 /* 14KB. */ + +/* Window RAM/Alternate window RAM offset mask for BF1 and BF2. */ +#define MLXBF_PKA_WINDOW_RAM_OFFSET_BF1_BF2_MASK (GENMASK(17, 16) | GENMAS= K(22, 20)) + +/* Window RAM/Alternate window RAM offset mask for BF3. */ +#define MLXBF_PKA_WINDOW_RAM_OFFSET_BF3_MASK GENMASK(18, 16) + +/* + * PKA Master Sequencer Control/Status Register. + * Writing '1' to bit [31] puts the Master controller Sequencer in a reset + * state. Resetting the Sequencer (in order to load other firmware) should + * only be done when the EIP-154 is not performing any operations. + */ +#define MLXBF_PKA_MASTER_SEQ_CTRL_RESET BIT(31) +/* + * Writing '1' to bit [30] will reset all Command and Result counters. Thi= s bit + * is write-only and self clearing and can only be set if the 'Reset' bit = [31] + * is '1'. + */ +#define MLXBF_PKA_MASTER_SEQ_CTRL_CLEAR_COUNTERS BIT(30) + +/** + * struct mlxbf_pka_dev_res_t - Device resource structure + * @ioaddr: The (iore)mapped version of addr, driver internal use + * @base: Base address of the device's resource + * @size: Size of IO + * @type: Type of resource addr points to + * @status: Status of the resource + * @name: Name of the resource + */ +struct mlxbf_pka_dev_res_t { + void __iomem *ioaddr; + u64 base; + u64 size; + u8 type; + s8 status; + char *name; +}; + +/* Defines for mlxbf_pka_dev_res->type. */ +#define MLXBF_PKA_DEV_RES_TYPE_MEM 1 /* Resource type is memory. */ +#define MLXBF_PKA_DEV_RES_TYPE_REG 2 /* Resource type is register. */ + +/* Defines for mlxbf_pka_dev_res->status. */ +#define MLXBF_PKA_DEV_RES_STATUS_MAPPED 1 /* The resource is (iore)mapped= . */ +#define MLXBF_PKA_DEV_RES_STATUS_UNMAPPED -1 /* The resource is unmapped. = */ + +/** + * struct mlxbf_pka_dev_shim_res_t - PKA Shim resources structure + * @buffer_ram: Buffer RAM + * @master_seq_ctrl: Master sequencer controller CSR + * @aic_csr: Interrupt controller CSRs + */ +struct mlxbf_pka_dev_shim_res_t { + struct mlxbf_pka_dev_res_t buffer_ram; + struct mlxbf_pka_dev_res_t master_seq_ctrl; + struct mlxbf_pka_dev_res_t aic_csr; +}; + +/* Number of PKA device resources. */ +#define MLXBF_PKA_DEV_SHIM_RES_CNT 6 + +/* Platform global shim resource information. */ +struct mlxbf_pka_dev_gbl_shim_res_info_t { + struct mlxbf_pka_dev_res_t *res_tbl[MLXBF_PKA_DEV_SHIM_RES_CNT]; + u8 res_cnt; +}; + +/** + * struct mlxbf_pka_dev_mem_res - PKA device memory resources + * @eip154_base: Base address for EIP154 mmio registers + * @eip154_size: EIP154 mmio register region size + * @wndw_ram_off_mask: Common offset mask for alt window RAM and window RAM + * @wndw_ram_base: Base address for window RAM + * @wndw_ram_size: Window RAM region size + * @alt_wndw_ram_0_base: Base address for alternate window RAM 0 + * @alt_wndw_ram_1_base: Base address for alternate window RAM 1 + * @alt_wndw_ram_2_base: Base address for alternate window RAM 2 + * @alt_wndw_ram_3_base: Base address for alternate window RAM 3 + * @alt_wndw_ram_size: Alternate window RAM regions size + * @csr_base: Base address for CSR registers + * @csr_size: CSR area size + */ +struct mlxbf_pka_dev_mem_res { + u64 eip154_base; + u64 eip154_size; + + u64 wndw_ram_off_mask; + u64 wndw_ram_base; + u64 wndw_ram_size; + + u64 alt_wndw_ram_0_base; + u64 alt_wndw_ram_1_base; + u64 alt_wndw_ram_2_base; + u64 alt_wndw_ram_3_base; + u64 alt_wndw_ram_size; + + u64 csr_base; + u64 csr_size; +}; + +/** + * struct mlxbf_pka_dev_shim_s - PKA Shim structure + * @mem_res: Memory resources + * @shim_id: Shim identifier + * @resources: Shim resources + * @window_ram_split: If non-zero, the split window RAM scheme is used + * @status: Status of the shim + * @mutex: Mutex lock for sharing shim + */ +struct mlxbf_pka_dev_shim_s { + struct mlxbf_pka_dev_mem_res mem_res; + u32 shim_id; + struct mlxbf_pka_dev_shim_res_t resources; + u8 window_ram_split; + s8 status; + struct mutex mutex; +}; + +/* Defines for mlxbf_pka_dev_shim->status. */ +#define MLXBF_PKA_SHIM_STATUS_UNDEFINED -1 +#define MLXBF_PKA_SHIM_STATUS_CREATED 1 +#define MLXBF_PKA_SHIM_STATUS_INITIALIZED 2 +#define MLXBF_PKA_SHIM_STATUS_RUNNING 3 +#define MLXBF_PKA_SHIM_STATUS_STOPPED 4 +#define MLXBF_PKA_SHIM_STATUS_FINALIZED 5 + +/* Defines for mlxbf_pka_dev_shim->window_ram_split. */ + +/* Window RAM is split into 4 * 16KB blocks. */ +#define MLXBF_PKA_SHIM_WINDOW_RAM_SPLIT_ENABLED 1 +/* Window RAM is not split and occupies 64KB. */ +#define MLXBF_PKA_SHIM_WINDOW_RAM_SPLIT_DISABLED 2 + +/** + * struct mlxbf_pka_dev_gbl_config_t - Platform global configuration struc= ture + * @dev_shims_cnt: Number of registered PKA shims + * @dev_shims: Table of registered PKA shims + */ +struct mlxbf_pka_dev_gbl_config_t { + u32 dev_shims_cnt; + struct mlxbf_pka_dev_shim_s *dev_shims[MLXBF_PKA_MAX_NUM_IO_BLOCKS]; +}; + +extern struct mlxbf_pka_dev_gbl_config_t mlxbf_pka_gbl_config; + +/* + * Processor speed in hertz, used in routines which might be called very e= arly + * in boot. + */ +static inline u64 mlxbf_pka_early_cpu_speed(void) +{ + /* + * Initial guess at our CPU speed. We set this to be larger than any + * possible real speed, so that any calculated delays will be too long, + * rather than too short. + * + * CPU Freq for High/Bin Chip - 1.255GHz. + */ + return 1255 * HZ_PER_MHZ; +} + +/* Start a PKA device timer. */ +static inline u64 mlxbf_pka_dev_timer_start_msec(u32 msec) +{ + u64 cur_time =3D get_cycles(); + + return cur_time + mlxbf_pka_early_cpu_speed() * msec / MSEC_PER_SEC; +} + +/* Test a PKA device timer for completion. */ +static inline bool mlxbf_pka_dev_timer_done(u64 timer) +{ + return get_cycles() >=3D timer; +} + +/* Return register base address. */ +static inline u64 mlxbf_pka_dev_get_register_base(u64 base, u64 reg_addr) +{ + return (base + reg_addr) & PAGE_MASK; +} + +/* Return register offset. */ +static inline u64 mlxbf_pka_dev_get_register_offset(u64 base, u64 reg_addr) +{ + return (base + reg_addr) & ~PAGE_MASK; +} + +/* Return word offset within io memory. */ +static inline u64 mlxbf_pka_dev_get_word_offset(u64 mem_base, u64 word_add= r, u64 mem_size) +{ + return (mem_base + word_addr) & (mem_size - 1); +} + +static inline u64 mlxbf_pka_dev_io_read(void __iomem *mem_ptr, u64 mem_off) +{ + return readq_relaxed(mem_ptr + mem_off); +} + +static inline void mlxbf_pka_dev_io_write(void __iomem *mem_ptr, u64 mem_o= ff, u64 value) +{ + writeq_relaxed(value, mem_ptr + mem_off); +} + +/* + * Shim getter for mlxbf_pka_dev_gbl_config_t structure which holds all sy= stem + * global configuration. This configuration is shared and common to kernel + * device driver associated with PKA hardware. + */ +struct mlxbf_pka_dev_shim_s *mlxbf_pka_dev_get_shim(u32 shim_id); + +void mlxbf_pka_dev_unset_resource_config(struct device *dev, + struct mlxbf_pka_dev_shim_s *shim, + struct mlxbf_pka_dev_res_t *res_ptr); + +/* + * Register PKA IO block. This function initializes a shim and configures = its + * related resources, and returns the error code. + */ +int mlxbf_pka_dev_register_shim(struct device *dev, + u32 shim_id, + struct mlxbf_pka_dev_mem_res *mem_res, + struct mlxbf_pka_dev_shim_s **shim); + +/* Unregister PKA IO block. */ +int mlxbf_pka_dev_unregister_shim(struct device *dev, struct mlxbf_pka_dev= _shim_s *shim); + +#endif /* __MLXBF_PKA_DEV_H__ */ diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c new file mode 100644 index 000000000000..42bfe30fbe49 --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c @@ -0,0 +1,413 @@ +// SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause +// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "mlxbf_pka_dev.h" + +#define MLXBF_PKA_DRIVER_DESCRIPTION "BlueField PKA driver" + +#define MLXBF_PKA_DEVICE_ACPIHID_BF1 "MLNXBF10" + +#define MLXBF_PKA_DEVICE_ACPIHID_BF2 "MLNXBF20" + +#define MLXBF_PKA_DEVICE_ACPIHID_BF3 "MLNXBF51" + +#define MLXBF_PKA_DEVICE_ACCESS_MODE 0666 +#define MLXBF_PKA_DEVICE_RES_CNT 7 +#define MLXBF_PKA_DEVICE_NAME_MAX 14 + +enum mlxbf_pka_mem_res_idx { + MLXBF_PKA_ACPI_EIP154_IDX =3D 0, + MLXBF_PKA_ACPI_WNDW_RAM_IDX, + MLXBF_PKA_ACPI_ALT_WNDW_RAM_0_IDX, + MLXBF_PKA_ACPI_ALT_WNDW_RAM_1_IDX, + MLXBF_PKA_ACPI_ALT_WNDW_RAM_2_IDX, + MLXBF_PKA_ACPI_ALT_WNDW_RAM_3_IDX, + MLXBF_PKA_ACPI_CSR_IDX +}; + +enum mlxbf_pka_plat_type { + /* Platform type Bluefield-1. */ + MLXBF_PKA_PLAT_TYPE_BF1 =3D 0, + /* Platform type Bluefield-2. */ + MLXBF_PKA_PLAT_TYPE_BF2, + /* Platform type Bluefield-3. */ + MLXBF_PKA_PLAT_TYPE_BF3 +}; + +struct mlxbf_pka_drv_plat_info { + enum mlxbf_pka_plat_type type; + u64 wndw_ram_off_mask; +}; + +static const struct mlxbf_pka_drv_plat_info mlxbf_pka_bf1_info =3D { + .type =3D MLXBF_PKA_PLAT_TYPE_BF1, + .wndw_ram_off_mask =3D MLXBF_PKA_WINDOW_RAM_OFFSET_BF1_BF2_MASK, +}; + +static const struct mlxbf_pka_drv_plat_info mlxbf_pka_bf2_info =3D { + .type =3D MLXBF_PKA_PLAT_TYPE_BF2, + .wndw_ram_off_mask =3D MLXBF_PKA_WINDOW_RAM_OFFSET_BF1_BF2_MASK, +}; + +static const struct mlxbf_pka_drv_plat_info mlxbf_pka_bf3_info =3D { + .type =3D MLXBF_PKA_PLAT_TYPE_BF3, + .wndw_ram_off_mask =3D MLXBF_PKA_WINDOW_RAM_OFFSET_BF3_MASK, +}; + +static DEFINE_MUTEX(mlxbf_pka_drv_lock); + +static u32 mlxbf_pka_device_cnt; + +static const char mlxbf_pka_acpihid_bf1[] =3D MLXBF_PKA_DEVICE_ACPIHID_BF1; + +static const char mlxbf_pka_acpihid_bf2[] =3D MLXBF_PKA_DEVICE_ACPIHID_BF2; + +static const char mlxbf_pka_acpihid_bf3[] =3D MLXBF_PKA_DEVICE_ACPIHID_BF3; + +static const struct acpi_device_id mlxbf_pka_drv_acpi_ids[] =3D { + { MLXBF_PKA_DEVICE_ACPIHID_BF1, (kernel_ulong_t)&mlxbf_pka_bf1_info, 0, 0= }, + { MLXBF_PKA_DEVICE_ACPIHID_BF2, (kernel_ulong_t)&mlxbf_pka_bf2_info, 0, 0= }, + { MLXBF_PKA_DEVICE_ACPIHID_BF3, (kernel_ulong_t)&mlxbf_pka_bf3_info, 0, 0= }, + {}, +}; + +struct mlxbf_pka_info { + /* The device this info struct belongs to. */ + struct device *dev; + /* Device name. */ + const char *name; + /* Device ACPI HID. */ + const char *acpihid; + /* Device flags. */ + u8 flag; + struct module *module; + /* Optional private data. */ + void *priv; +}; + +/* Defines for mlxbf_pka_info->flags. */ +#define MLXBF_PKA_DRIVER_FLAG_DEVICE 2 + +struct mlxbf_pka_platdata { + struct platform_device *pdev; + struct mlxbf_pka_info *info; + /* Generic spinlock. */ + spinlock_t lock; +}; + +#define MLXBF_PKA_DRIVER_DEV_MAX MLXBF_PKA_MAX_NUM_IO_BLOCKS + +struct mlxbf_pka_device { + struct mlxbf_pka_info *info; + struct device *device; + u32 device_id; + struct resource *resource[MLXBF_PKA_DEVICE_RES_CNT]; + struct mlxbf_pka_dev_shim_s *shim; +}; + +static int mlxbf_pka_drv_verify_bootup_status(struct device *dev) +{ + const char *bootup_status; + int ret; + + ret =3D device_property_read_string(dev, "bootup_done", &bootup_status); + if (ret < 0) { + dev_err(dev, "failed to read bootup_done property\n"); + return ret; + } + + if (strcmp(bootup_status, "true")) { + dev_err(dev, "device bootup required\n"); + return -ENODEV; + } + + return 0; +} + +static void mlxbf_pka_drv_get_mem_res(struct mlxbf_pka_device *mlxbf_pka_d= ev, + struct mlxbf_pka_dev_mem_res *mem_res, + u64 wndw_ram_off_mask) +{ + enum mlxbf_pka_mem_res_idx acpi_mem_idx; + + acpi_mem_idx =3D MLXBF_PKA_ACPI_EIP154_IDX; + mem_res->wndw_ram_off_mask =3D wndw_ram_off_mask; + + /* PKA EIP154 MMIO base address. */ + mem_res->eip154_base =3D mlxbf_pka_dev->resource[acpi_mem_idx]->start; + mem_res->eip154_size =3D resource_size(mlxbf_pka_dev->resource[acpi_mem_i= dx]); + acpi_mem_idx++; + + /* PKA window RAM base address. */ + mem_res->wndw_ram_base =3D mlxbf_pka_dev->resource[acpi_mem_idx]->start; + mem_res->wndw_ram_size =3D resource_size(mlxbf_pka_dev->resource[acpi_mem= _idx]); + acpi_mem_idx++; + + /* + * PKA alternate window RAM base address. + * Note: the size of all the alt window RAM is the same, depicted by + * 'alt_wndw_ram_size' variable. All alt window RAM resources are read + * here even though not all of them are used currently. + */ + mem_res->alt_wndw_ram_0_base =3D mlxbf_pka_dev->resource[acpi_mem_idx]->s= tart; + mem_res->alt_wndw_ram_size =3D resource_size(mlxbf_pka_dev->resource[acpi= _mem_idx]); + + if (mem_res->alt_wndw_ram_size !=3D MLXBF_PKA_WINDOW_RAM_REGION_SIZE) + dev_warn(mlxbf_pka_dev->device, "alternate Window RAM size from ACPI is = wrong.\n"); + + acpi_mem_idx++; + + mem_res->alt_wndw_ram_1_base =3D mlxbf_pka_dev->resource[acpi_mem_idx]->s= tart; + acpi_mem_idx++; + + mem_res->alt_wndw_ram_2_base =3D mlxbf_pka_dev->resource[acpi_mem_idx]->s= tart; + acpi_mem_idx++; + + mem_res->alt_wndw_ram_3_base =3D mlxbf_pka_dev->resource[acpi_mem_idx]->s= tart; + acpi_mem_idx++; + + /* PKA CSR base address. */ + mem_res->csr_base =3D mlxbf_pka_dev->resource[acpi_mem_idx]->start; + mem_res->csr_size =3D resource_size(mlxbf_pka_dev->resource[acpi_mem_idx]= ); +} + +/* + * Note: this function must be serialized because it calls + * 'mlxbf_pka_dev_register_shim' which manipulates common counters for the + * PKA devices. + */ +static int mlxbf_pka_drv_register_device(struct mlxbf_pka_device *mlxbf_pk= a_dev, + u64 wndw_ram_off_mask) +{ + struct mlxbf_pka_dev_mem_res mem_res; + u32 mlxbf_pka_shim_id; + int ret; + + /* Assert that the driver lock is held for serialization */ + lockdep_assert_held(&mlxbf_pka_drv_lock); + + mlxbf_pka_shim_id =3D mlxbf_pka_dev->device_id; + + mlxbf_pka_drv_get_mem_res(mlxbf_pka_dev, &mem_res, wndw_ram_off_mask); + + ret =3D mlxbf_pka_dev_register_shim(mlxbf_pka_dev->device, + mlxbf_pka_shim_id, + &mem_res, + &mlxbf_pka_dev->shim); + if (ret) { + dev_dbg(mlxbf_pka_dev->device, "failed to register shim\n"); + return ret; + } + + return 0; +} + +static int mlxbf_pka_drv_unregister_device(struct mlxbf_pka_device *mlxbf_= pka_dev) +{ + if (!mlxbf_pka_dev || !mlxbf_pka_dev->shim) + return -EINVAL; + + dev_dbg(mlxbf_pka_dev->device, "unregister device shim\n"); + return mlxbf_pka_dev_unregister_shim(mlxbf_pka_dev->device, mlxbf_pka_dev= ->shim); +} + +static int mlxbf_pka_drv_probe_device(struct mlxbf_pka_info *info) +{ + struct mlxbf_pka_drv_plat_info *plat_info; + enum mlxbf_pka_mem_res_idx acpi_mem_idx; + struct mlxbf_pka_device *mlxbf_pka_dev; + const struct acpi_device_id *aid; + struct platform_device *pdev; + u64 wndw_ram_off_mask; + struct device *dev; + int ret; + + if (!info) + return -EINVAL; + + dev =3D info->dev; + pdev =3D to_platform_device(dev); + + mlxbf_pka_dev =3D devm_kzalloc(dev, sizeof(*mlxbf_pka_dev), GFP_KERNEL); + if (!mlxbf_pka_dev) + return -ENOMEM; + + scoped_guard(mutex, &mlxbf_pka_drv_lock) { + mlxbf_pka_device_cnt +=3D 1; + if (mlxbf_pka_device_cnt > MLXBF_PKA_DRIVER_DEV_MAX) { + dev_dbg(dev, "cannot support %u devices\n", mlxbf_pka_device_cnt); + return -ENOSPC; + } + mlxbf_pka_dev->device_id =3D mlxbf_pka_device_cnt - 1; + } + + mlxbf_pka_dev->info =3D info; + mlxbf_pka_dev->device =3D dev; + info->flag =3D MLXBF_PKA_DRIVER_FLAG_DEVICE; + + for (acpi_mem_idx =3D MLXBF_PKA_ACPI_EIP154_IDX; + acpi_mem_idx < MLXBF_PKA_DEVICE_RES_CNT; + acpi_mem_idx++) { + mlxbf_pka_dev->resource[acpi_mem_idx] =3D platform_get_resource(pdev, + IORESOURCE_MEM, + acpi_mem_idx); + } + + /* Verify PKA bootup status. */ + ret =3D mlxbf_pka_drv_verify_bootup_status(dev); + if (ret) + return ret; + + /* Window RAM offset mask is platform dependent. */ + aid =3D acpi_match_device(mlxbf_pka_drv_acpi_ids, dev); + if (!aid) + return -ENODEV; + + plat_info =3D (struct mlxbf_pka_drv_plat_info *)aid->driver_data; + if (!plat_info) { + dev_err(dev, "missing platform data\n"); + return -EINVAL; + } + + wndw_ram_off_mask =3D plat_info->wndw_ram_off_mask; + + scoped_guard(mutex, &mlxbf_pka_drv_lock) { + ret =3D mlxbf_pka_drv_register_device(mlxbf_pka_dev, wndw_ram_off_mask); + if (ret) { + dev_dbg(dev, "failed to register shim\n"); + return ret; + } + } + + info->priv =3D mlxbf_pka_dev; + + return 0; +} + +static void mlxbf_pka_drv_remove_device(struct platform_device *pdev) +{ + struct mlxbf_pka_platdata *priv =3D platform_get_drvdata(pdev); + struct mlxbf_pka_info *info =3D priv->info; + struct mlxbf_pka_device *mlxbf_pka_dev =3D (struct mlxbf_pka_device *)inf= o->priv; + + if (!mlxbf_pka_dev) + return; + + mlxbf_pka_drv_unregister_device(mlxbf_pka_dev); +} + +static int mlxbf_pka_drv_acpi_probe(struct platform_device *pdev, struct m= lxbf_pka_info *info) +{ + struct device *dev =3D &pdev->dev; + struct acpi_device *adev; + int ret; + + if (acpi_disabled) + return -ENOENT; + + adev =3D ACPI_COMPANION(dev); + if (!adev) { + dev_dbg(dev, "ACPI companion device not found for %s\n", pdev->name); + return -ENODEV; + } + + info->acpihid =3D acpi_device_hid(adev); + if (WARN_ON(!info->acpihid)) + return -EINVAL; + + if (!strcmp(info->acpihid, mlxbf_pka_acpihid_bf1) || + !strcmp(info->acpihid, mlxbf_pka_acpihid_bf2) || + !strcmp(info->acpihid, mlxbf_pka_acpihid_bf3)) { + ret =3D mlxbf_pka_drv_probe_device(info); + if (ret) { + dev_dbg(dev, "failed to register device\n"); + return ret; + } + dev_info(dev, "device probed\n"); + } + + return 0; +} + +static int mlxbf_pka_drv_probe(struct platform_device *pdev) +{ + struct mlxbf_pka_platdata *priv; + struct device *dev =3D &pdev->dev; + struct mlxbf_pka_info *info; + int ret; + + priv =3D devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL); + if (!priv) + return -ENOMEM; + + spin_lock_init(&priv->lock); + priv->pdev =3D pdev; + + info =3D devm_kzalloc(dev, sizeof(*info), GFP_KERNEL); + if (!info) + return -ENOMEM; + + info->name =3D pdev->name; + info->module =3D THIS_MODULE; + info->dev =3D dev; + priv->info =3D info; + + platform_set_drvdata(pdev, priv); + + ret =3D mlxbf_pka_drv_acpi_probe(pdev, info); + if (ret) { + dev_dbg(dev, "unknown device\n"); + return ret; + } + + return ret; +} + +static void mlxbf_pka_drv_remove(struct platform_device *pdev) +{ + struct mlxbf_pka_platdata *priv =3D platform_get_drvdata(pdev); + struct mlxbf_pka_info *info =3D priv->info; + + if (info->flag =3D=3D MLXBF_PKA_DRIVER_FLAG_DEVICE) { + dev_info(&pdev->dev, "remove PKA device\n"); + mlxbf_pka_drv_remove_device(pdev); + } +} + +MODULE_DEVICE_TABLE(acpi, mlxbf_pka_drv_acpi_ids); + +static struct platform_driver mlxbf_pka_drv =3D { + .driver =3D { + .name =3D KBUILD_MODNAME, + .acpi_match_table =3D ACPI_PTR(mlxbf_pka_drv_acpi_ids), + }, + .probe =3D mlxbf_pka_drv_probe, + .remove =3D mlxbf_pka_drv_remove, +}; + +module_platform_driver(mlxbf_pka_drv); +MODULE_DESCRIPTION(MLXBF_PKA_DRIVER_DESCRIPTION); +MODULE_AUTHOR("Ron Li "); +MODULE_AUTHOR("Khalil Blaiech "); +MODULE_AUTHOR("Mahantesh Salimath "); +MODULE_AUTHOR("Shih-Yi Chen "); +MODULE_LICENSE("Dual BSD/GPL"); --=20 2.34.1 From nobody Thu Oct 2 06:18:04 2025 Received: from BL2PR02CU003.outbound.protection.outlook.com (mail-eastusazon11011012.outbound.protection.outlook.com [52.101.52.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6D8C2C0F7C; Fri, 19 Sep 2025 19:53:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.52.12 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758311605; cv=fail; b=YNSYvKRsDK9wGNo/YQ1feohgqgjRveYvNoSe/bU0tmiScdsIheGBsBGqaFAho4iMb3nKsjOy4WOURso4Fysxxv2NNUjmq3dDBwO2/Q0xIbRciqa3baLflUPWiXm/VISofmNrBLi1RuoGf2h2reaSRIuqK0eu9HeFC2Q9WmYRNpE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758311605; c=relaxed/simple; bh=rxbYo1zl7+p9PbaxGDtAaA2kWxESX5Hy1K4/Y4ibOK4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Q1zc9s4kzsFVUnpLkseObkP7VVthsZfhWHNh0DLSU+1m4Lkx8409w9qQA1crWph2ldCegIVlGgjS1LbU143zN93GDEpkwFPsWzxfsZOVX21pOgTqZLTyPjOwID7w6wB3P18gdXKr3/q9aeTbqdVzA1m6G8YUlA3G0tzEBlrumw8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=hJ+QaT9Z; arc=fail smtp.client-ip=52.101.52.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="hJ+QaT9Z" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nyZ53/6YCwRBaiiepOZXanepkWNMplKiXk/YdLEjOTQz5R8bW/894fwb2LcB0xmJl9tPOoMQLrNu8ikQ0PnWwIcDduOnPbNS/GrxKG4E6O9nNErpHbu8DnjNuOswa7znt7k0KPmD4OlxD9xylK6YZH8LlcKONhrSFHyvc+lmBs9Rsknk2VrGwc9ZvAlwc7uBo5Jnw3fSuLyCa5UvE0oBktC9pyh/RLHtMYYHljUjcM5TvRJLMDxnJ+oirfrIArlDkDZYgUsi18Oj121sPKgAOn22mHbaR/uFaC9gzaFPhmcpM6cu+sTjU4jPf55dOnqoYr21Fo6E8j3+La8z/MqcAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=smncOxWs2qUUDzJ4uLtgXvU8/DODotSo44onv3xenfw=; b=f2ctkN1Uwc6QjQvqv/yar03Xf0lJwU0U81NLmX2ZJYC4pn+9ALqIeQBryqRfhMesdI0fk8fpENvjypXmiLwPCmoMnxa3cu8nZ2JyvoRir2/yWhIp7GusbTiorUijDjsPzwzwu1g1BDZa0APic2VvK8KX+aEGd2g2mTvfvXrxXWZPxtRKN2JABRKqxlQ5UV2Tqn8vrnUAgBMS/+Igw4Ff9EkL8yy/x0i+H5+K7oCN6fNn+uxfVsayGd2WMr6rCyeKzHn/f+zw/5b3FehXB4bTlWG0372oTC/aCmeT/g/5FJdwFTU0YiXcuS0e0b5yBr28c3MtPB2LbU4BYGSvccK10w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=smncOxWs2qUUDzJ4uLtgXvU8/DODotSo44onv3xenfw=; b=hJ+QaT9Z7W56oPwzg5gVnj62CuXSut+o/5cSrvukShpuNlbDEqJwTXEWIHyec5rKoH815aH37n9+xAMubgIKehPVNCfr9lW8R5rwB9esWYeD7JzYNXSqtyXrKwonE/tJsDirFx6DeubMMAt2JLE52++i/p3RdN/k+4NJiKOss5BOvfdpwKThWazG01mEp+mIu0Fn+e2OQV6H22bF68LhGozKG4IH2zD3kxiNrt81J9izg22C17ZlSZ9faeWJdLy/P+VPdQtbGExTrgi+IYwYdlsJWstYyiVas8xZIu2o1I+9oKY9J4hAuriTASZ8oOyPx7Wk0YirFMzxUYBet7Vhbg== Received: from BL6PEPF00013E0C.NAMP222.PROD.OUTLOOK.COM (2603:10b6:22e:400:0:1001:0:e) by MN0PR12MB6368.namprd12.prod.outlook.com (2603:10b6:208:3d2::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9137.14; Fri, 19 Sep 2025 19:53:07 +0000 Received: from BN1PEPF00004681.namprd03.prod.outlook.com (2a01:111:f403:c803::5) by BL6PEPF00013E0C.outlook.office365.com (2603:1036:903:4::4) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9137.14 via Frontend Transport; Fri, 19 Sep 2025 19:53:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00004681.mail.protection.outlook.com (10.167.243.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9137.12 via Frontend Transport; Fri, 19 Sep 2025 19:53:06 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Fri, 19 Sep 2025 12:51:38 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.34; Fri, 19 Sep 2025 12:51:38 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.34 via Frontend Transport; Fri, 19 Sep 2025 12:51:37 -0700 From: Ron Li To: , , , , , CC: , , , Ron Li Subject: [PATCH v3 2/3] platform/mellanox/mlxbf_pka: Add userspace PKA ring device interface Date: Fri, 19 Sep 2025 19:51:31 +0000 Message-ID: <20250919195132.1088515-3-xiangrongl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250919195132.1088515-1-xiangrongl@nvidia.com> References: <20250919195132.1088515-1-xiangrongl@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00004681:EE_|MN0PR12MB6368:EE_ X-MS-Office365-Filtering-Correlation-Id: 292d4f84-ab0a-48ab-c4db-08ddf7b6275b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?VoDPqDX+9YJ7wcLc0u/UNOCEgRWepIuRTaPkqcCfnK861DJq+p3D8WL7/vH3?= =?us-ascii?Q?5nM825Ex9YPQIKRksaegHezQPpwRiUSvDztLdoFRWmGqKHVLfHsL0uB/g12R?= =?us-ascii?Q?iHvisiXEn5sEzcXFY0ofpmcqaKAPBKWSYsq4oJUWKrqgKdntd63Mk/Y4V4iI?= =?us-ascii?Q?Y/j/wwIKULAd9Fqyqkr8ZE+bKcFwv87K6n3Ub/JAO15Ehb5p92Ny7ZEfM6fm?= =?us-ascii?Q?JmhpGiyga+v0/1muGoctX2E18jQUw89A1NaTmyWVIZRjLS2MAHqUn+l6csuQ?= =?us-ascii?Q?Qht5FfBftqvmZTIY9nmqjomJ7OgnieKDBG94HFW4zU+iq9NDgjFXhOfGID+V?= =?us-ascii?Q?dyjXmEdIQ4b8BKlfOnNWG21CbrG2dSPQg3Hu66EEhEMsGfVDIHzI8vSpalwr?= =?us-ascii?Q?ZZcDleqIiQNcVPaygYgRRHO219dQA4VTcR35NKywOwAFiX4IXSM/tghXK5sQ?= =?us-ascii?Q?tJlZcCQau92L0wIa6NqwTiLYki0PtJs8S+QrCc39zJJdghKxZpRq8u1FIiog?= =?us-ascii?Q?gjvi1iNi+tyHb1YI7hwv1ryUrxvmgIV1kPt2JAXPpSXXYSYPem1b9OrIPwsM?= =?us-ascii?Q?57Dv+CkB6wv5ojkJBOanS/Ee153S5Tj41+r4vqYbvptLKxknSbAxeSan+PO4?= =?us-ascii?Q?Q1DRQOYhk+jXZSMo9O3WiMqBOoRy7+HUbf9Ut01mL4Curd5pWJeGpY6n0PKj?= =?us-ascii?Q?3TZ5JlGDKRUFURrs8MIJIf+94olU1IOSY0z3vGLNOR5vAo0/WuLMctxYm6dQ?= =?us-ascii?Q?QG+PrhwLxQ2CdvASWKEnSabKGNhc5c69dbRmfHBxz72dXmKcKMhOxCcRG0Y4?= =?us-ascii?Q?gyzOgRzLZc21F2PSjs6G4k+GBf9DZgq8yPVb9pHwjOZfWXQ7OPl+2NSmxM8r?= =?us-ascii?Q?KQi28VHQIiHMmDCEvRd7lcsU0eBSKqmjDpXmBA6G3EfClEW+vH6EHMCLytmC?= =?us-ascii?Q?pDPb7lfzoPiIxvqKTvcwcIJu6UxS1CV1d0kQDwntQTGAj/ZwxYvWyhcubPWb?= =?us-ascii?Q?Y+UmW4VwQVp+oWaPW8XgWJTqrauCajphu7D+HoV5f/cQqQ8W7zpcwG9NTFUd?= =?us-ascii?Q?KlpkCyx+0KJ0XntQdBKIF9mTZCV0ETMt1N9klJs5uGlFE0lFM7gJyTN0PZit?= =?us-ascii?Q?pK5F/3DWc1itmF/KxeMsSKb9RL3WFoDanI/KOld6R+S8wUie9GD1AXPF3Ghg?= =?us-ascii?Q?hiBWAWXBJKHSpGPMQJ9M22wMcH3XA3F4ejNkdbWxbd6TF+L0oiw+lg+k79Ws?= =?us-ascii?Q?lcGzd7Xkw/2AKT2b/TVwD59qahDGGf1EhkoLp4aTZLOQt8tyT5jT9BvvCyCs?= =?us-ascii?Q?4qBk3yD+UCMbt3V+gSqmtcR/8By+E1BDp3WF+2bx9JGbEYInhOHG/oVEoSLn?= =?us-ascii?Q?DHBd5Wy+3Xp4b0oa+Rs6t5gJK4geGI6IrrSgDf99ec0hPCLSCtMvdVnmNMIP?= =?us-ascii?Q?8Ib7YbVCk2oedvzrVZNpLTpma/L4V8Oq4DQ27TzCWjfUkE4mEXZCpj92/aSz?= =?us-ascii?Q?UR6x5lUqVleZ5CNrXL5t/Gc4W/VSGKisSMV4?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(82310400026)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2025 19:53:06.9814 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 292d4f84-ab0a-48ab-c4db-08ddf7b6275b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00004681.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6368 Content-Type: text/plain; charset="utf-8" Expose each BlueField PKA ring as a character device for userspace offload. This focuses on per-ring resources, layout, and control, without in-kernel crypto algorithms. - Create ring device nodes and lifecycle: open/close, mmap, ioctl - Partition 16KB Window RAM per ring (1KB cmd, 1KB result, 14KB vectors) - Program ring info words (cmd/rslt bases, size, host_desc_size, in-order) - Provide UAPI ioctls: - MLXBF_PKA_RING_GET_REGION_INFO - MLXBF_PKA_GET_RING_INFO - MLXBF_PKA_CLEAR_RING_COUNTERS - ACPI-based probe for BF1/BF2/BF3 and per-shim ring setup - Document device/ring identifiers and interface in sysfs ABI Reviewed-by: David Thompson Reviewed-by: Khalil Blaiech Signed-off-by: Ron Li --- .../ABI/testing/sysfs-platform-mellanox-pka | 18 + .../userspace-api/ioctl/ioctl-number.rst | 2 + drivers/platform/mellanox/mlxbf_pka/Makefile | 1 + .../mellanox/mlxbf_pka/mlxbf_pka_dev.c | 40 ++ .../mellanox/mlxbf_pka/mlxbf_pka_dev.h | 23 + .../mellanox/mlxbf_pka/mlxbf_pka_drv.c | 528 ++++++++++++++++ .../mellanox/mlxbf_pka/mlxbf_pka_ring.c | 563 ++++++++++++++++++ .../mellanox/mlxbf_pka/mlxbf_pka_ring.h | 255 ++++++++ include/uapi/linux/mlxbf-pka.h | 93 +++ 9 files changed, 1523 insertions(+) create mode 100644 drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_ring.c create mode 100644 drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_ring.h create mode 100644 include/uapi/linux/mlxbf-pka.h diff --git a/Documentation/ABI/testing/sysfs-platform-mellanox-pka b/Docume= ntation/ABI/testing/sysfs-platform-mellanox-pka index cf8dd292c781..e7d331bca8fb 100644 --- a/Documentation/ABI/testing/sysfs-platform-mellanox-pka +++ b/Documentation/ABI/testing/sysfs-platform-mellanox-pka @@ -14,3 +14,21 @@ Description: BlueField-2 MLNXBF20:xx, where xx ranges from '00' to '07' BlueField-3 MLNXBF51:xx, where xx ranges from '00' to '17' =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D + + Each PKA device supports four PKA ring devices. The PKA ring device IDs = are: + + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D + BlueField-1 MLNXBF11:xx, where xx ranges from '00' to '0F' + BlueField-2 MLNXBF21:xx, where xx ranges from '00' to '20' + BlueField-3 MLNXBF52:xx, where xx ranges from '00' to '60' + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D + + For each PKA ring device, the following operation interfaces: + + - open(): open the PKA ring device specified by the device ID, and initi= ate the + related RAM regions. + - release(): close the PKA ring device specified by the device ID, and r= elease the + related RAM regions. + - unlocked_ioctl(): make PKA related system calls, including getting rin= g device or + RAM region information, clearing PKA ring counter. + - mmap(): map the PKA ring device to the virtual memory region. diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documenta= tion/userspace-api/ioctl/ioctl-number.rst index 406a9f4d0869..0bbabf07a495 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -383,6 +383,8 @@ Code Seq# Include File = Comments 0xB8 01-02 uapi/misc/mrvl_cn10k_dpi.h Mar= vell CN10K DPI driver 0xB8 all uapi/linux/mshv.h Mic= rosoft Hyper-V /dev/mshv driver +0xBF 00-0F uapi/linux/mlxbf-pka.h NVI= DIA BlueField PKA driver + 0xC0 00-0F linux/usb/iowarrior.h 0xCA 00-0F uapi/misc/cxl.h Dea= d since 6.15 0xCA 10-2F uapi/misc/ocxl.h diff --git a/drivers/platform/mellanox/mlxbf_pka/Makefile b/drivers/platfor= m/mellanox/mlxbf_pka/Makefile index 67465a63edb8..6e536794d339 100644 --- a/drivers/platform/mellanox/mlxbf_pka/Makefile +++ b/drivers/platform/mellanox/mlxbf_pka/Makefile @@ -8,3 +8,4 @@ obj-$(CONFIG_MLXBF_PKA) +=3D mlxbf-pka.o =20 mlxbf-pka-objs :=3D mlxbf_pka_drv.o mlxbf-pka-objs +=3D mlxbf_pka_dev.o +mlxbf-pka-objs +=3D mlxbf_pka_ring.o diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c index 0a5db1be6eaa..12df11dd1eee 100644 --- a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c @@ -17,6 +17,7 @@ #include =20 #include "mlxbf_pka_dev.h" +#include "mlxbf_pka_ring.h" =20 struct mlxbf_pka_dev_gbl_config_t mlxbf_pka_gbl_config; =20 @@ -192,6 +193,18 @@ static int mlxbf_pka_dev_create_shim(struct device *de= v, else shim->window_ram_split =3D MLXBF_PKA_SHIM_WINDOW_RAM_SPLIT_DISABLED; =20 + shim->ring_type =3D MLXBF_PKA_RING_TYPE_IN_ORDER; + shim->ring_priority =3D MLXBF_PKA_RING_OPTIONS_PRIORITY; + shim->rings_num =3D MLXBF_PKA_MAX_NUM_IO_BLOCK_RINGS; + shim->rings =3D devm_kcalloc(dev, + shim->rings_num, + sizeof(struct mlxbf_pka_dev_ring_t), + GFP_KERNEL); + if (!shim->rings) { + dev_err(dev, "unable to allocate memory for ring\n"); + return -ENOMEM; + } + /* Set PKA device Buffer RAM config. */ ret =3D mlxbf_pka_dev_set_resource_config(dev, shim, @@ -288,10 +301,21 @@ static int mlxbf_pka_dev_init_shim(struct device *dev= , struct mlxbf_pka_dev_shim return -EPERM; } =20 + /* Configure PKA Ring options control word. */ + ret =3D mlxbf_pka_dev_config_ring_options(dev, + &shim->resources.buffer_ram, + shim->rings_num, + shim->ring_priority); + if (ret) { + dev_err(dev, "failed to configure ring options\n"); + return ret; + } + ret =3D devm_mutex_init(dev, &shim->mutex); if (ret) return ret; =20 + shim->busy_ring_num =3D 0; shim->status =3D MLXBF_PKA_SHIM_STATUS_INITIALIZED; =20 return ret; @@ -300,6 +324,7 @@ static int mlxbf_pka_dev_init_shim(struct device *dev, = struct mlxbf_pka_dev_shim /* Release a given shim. */ static int mlxbf_pka_dev_release_shim(struct device *dev, struct mlxbf_pka= _dev_shim_s *shim) { + u32 ring_idx; int ret =3D 0; =20 if (shim->status !=3D MLXBF_PKA_SHIM_STATUS_INITIALIZED && @@ -308,6 +333,21 @@ static int mlxbf_pka_dev_release_shim(struct device *d= ev, struct mlxbf_pka_dev_s return -EPERM; } =20 + /* + * Release rings which belong to the shim. The operating system might + * release ring devices before shim devices. The global configuration + * must be checked before proceeding to the release of ring devices. + */ + if (mlxbf_pka_gbl_config.dev_rings_cnt) { + for (ring_idx =3D 0; ring_idx < shim->rings_num; ring_idx++) { + ret =3D mlxbf_pka_dev_release_ring(dev, shim->rings[ring_idx]); + if (ret) { + dev_err(dev, "failed to release ring %d\n", ring_idx); + return ret; + } + } + } + shim->busy_ring_num =3D 0; shim->status =3D MLXBF_PKA_SHIM_STATUS_FINALIZED; =20 return ret; diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h index df51202f79bd..12512850cf79 100644 --- a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h @@ -14,12 +14,15 @@ =20 #include #include +#include #include #include #include #include #include =20 +#include + #define MASTER_CONTROLLER_OUT_OF_RESET 0 =20 /* PKA address related definitions. */ @@ -60,6 +63,11 @@ /* The maximum number of PKA shims referred to as IO blocks. */ #define MLXBF_PKA_MAX_NUM_IO_BLOCKS 24 =20 +/* The maximum number of rings supported by the IO block (shim). */ +#define MLXBF_PKA_MAX_NUM_IO_BLOCK_RINGS 4 + +#define MLXBF_PKA_MAX_NUM_RINGS (MLXBF_PKA_MAX_NUM_IO_BLOCK_RINGS * MLXBF_= PKA_MAX_NUM_IO_BLOCKS) + /* * PKA Window RAM parameters. * Define whether to split window RAM during PKA device creation phase. @@ -173,16 +181,26 @@ struct mlxbf_pka_dev_mem_res { * struct mlxbf_pka_dev_shim_s - PKA Shim structure * @mem_res: Memory resources * @shim_id: Shim identifier + * @rings_num: Number of supported rings (hardware specific) + * @rings: Pointer to rings which belong to the shim + * @ring_priority: Specify the priority in which rings are handled + * @ring_type: Indicates whether the result ring delivers results strictly= in-order * @resources: Shim resources * @window_ram_split: If non-zero, the split window RAM scheme is used + * @busy_ring_num: Number of active rings (rings in busy state) * @status: Status of the shim * @mutex: Mutex lock for sharing shim */ struct mlxbf_pka_dev_shim_s { struct mlxbf_pka_dev_mem_res mem_res; u32 shim_id; + u32 rings_num; + struct mlxbf_pka_dev_ring_t **rings; + u8 ring_priority; + u8 ring_type; struct mlxbf_pka_dev_shim_res_t resources; u8 window_ram_split; + u32 busy_ring_num; s8 status; struct mutex mutex; }; @@ -205,11 +223,15 @@ struct mlxbf_pka_dev_shim_s { /** * struct mlxbf_pka_dev_gbl_config_t - Platform global configuration struc= ture * @dev_shims_cnt: Number of registered PKA shims + * @dev_rings_cnt: Number of registered Rings * @dev_shims: Table of registered PKA shims + * @dev_rings: Table of registered Rings */ struct mlxbf_pka_dev_gbl_config_t { u32 dev_shims_cnt; + u32 dev_rings_cnt; struct mlxbf_pka_dev_shim_s *dev_shims[MLXBF_PKA_MAX_NUM_IO_BLOCKS]; + struct mlxbf_pka_dev_ring_t *dev_rings[MLXBF_PKA_MAX_NUM_RINGS]; }; =20 extern struct mlxbf_pka_dev_gbl_config_t mlxbf_pka_gbl_config; @@ -279,6 +301,7 @@ static inline void mlxbf_pka_dev_io_write(void __iomem = *mem_ptr, u64 mem_off, u6 */ struct mlxbf_pka_dev_shim_s *mlxbf_pka_dev_get_shim(u32 shim_id); =20 +/* Unset PKA device resource config - unmap io memory if needed. */ void mlxbf_pka_dev_unset_resource_config(struct device *dev, struct mlxbf_pka_dev_shim_s *shim, struct mlxbf_pka_dev_res_t *res_ptr); diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c index 42bfe30fbe49..a009437e4a48 100644 --- a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c @@ -21,14 +21,18 @@ #include =20 #include "mlxbf_pka_dev.h" +#include "mlxbf_pka_ring.h" =20 #define MLXBF_PKA_DRIVER_DESCRIPTION "BlueField PKA driver" =20 #define MLXBF_PKA_DEVICE_ACPIHID_BF1 "MLNXBF10" +#define MLXBF_PKA_RING_DEVICE_ACPIHID_BF1 "MLNXBF11" =20 #define MLXBF_PKA_DEVICE_ACPIHID_BF2 "MLNXBF20" +#define MLXBF_PKA_RING_DEVICE_ACPIHID_BF2 "MLNXBF21" =20 #define MLXBF_PKA_DEVICE_ACPIHID_BF3 "MLNXBF51" +#define MLXBF_PKA_RING_DEVICE_ACPIHID_BF3 "MLNXBF52" =20 #define MLXBF_PKA_DEVICE_ACCESS_MODE 0666 #define MLXBF_PKA_DEVICE_RES_CNT 7 @@ -76,20 +80,33 @@ static const struct mlxbf_pka_drv_plat_info mlxbf_pka_b= f3_info =3D { static DEFINE_MUTEX(mlxbf_pka_drv_lock); =20 static u32 mlxbf_pka_device_cnt; +static u32 mlxbf_pka_ring_device_cnt; =20 static const char mlxbf_pka_acpihid_bf1[] =3D MLXBF_PKA_DEVICE_ACPIHID_BF1; +static const char mlxbf_pka_ring_acpihid_bf1[] =3D MLXBF_PKA_RING_DEVICE_A= CPIHID_BF1; =20 static const char mlxbf_pka_acpihid_bf2[] =3D MLXBF_PKA_DEVICE_ACPIHID_BF2; +static const char mlxbf_pka_ring_acpihid_bf2[] =3D MLXBF_PKA_RING_DEVICE_A= CPIHID_BF2; =20 static const char mlxbf_pka_acpihid_bf3[] =3D MLXBF_PKA_DEVICE_ACPIHID_BF3; +static const char mlxbf_pka_ring_acpihid_bf3[] =3D MLXBF_PKA_RING_DEVICE_A= CPIHID_BF3; =20 static const struct acpi_device_id mlxbf_pka_drv_acpi_ids[] =3D { { MLXBF_PKA_DEVICE_ACPIHID_BF1, (kernel_ulong_t)&mlxbf_pka_bf1_info, 0, 0= }, + { MLXBF_PKA_RING_DEVICE_ACPIHID_BF1, 0, 0, 0 }, { MLXBF_PKA_DEVICE_ACPIHID_BF2, (kernel_ulong_t)&mlxbf_pka_bf2_info, 0, 0= }, + { MLXBF_PKA_RING_DEVICE_ACPIHID_BF2, 0, 0, 0 }, { MLXBF_PKA_DEVICE_ACPIHID_BF3, (kernel_ulong_t)&mlxbf_pka_bf3_info, 0, 0= }, + { MLXBF_PKA_RING_DEVICE_ACPIHID_BF3, 0, 0, 0 }, {}, }; =20 +static struct pka { + struct idr ring_idr; + /* PKA ring device IDR lock mutex. */ + struct mutex idr_lock; +} pka; + struct mlxbf_pka_info { /* The device this info struct belongs to. */ struct device *dev; @@ -105,6 +122,7 @@ struct mlxbf_pka_info { }; =20 /* Defines for mlxbf_pka_info->flags. */ +#define MLXBF_PKA_DRIVER_FLAG_RING_DEVICE 1 #define MLXBF_PKA_DRIVER_FLAG_DEVICE 2 =20 struct mlxbf_pka_platdata { @@ -114,6 +132,53 @@ struct mlxbf_pka_platdata { spinlock_t lock; }; =20 +struct mlxbf_pka_ring_region { + u64 off; + u64 addr; + resource_size_t size; + u32 flags; + u32 type; +}; + +/* Defines for mlxbf_pka_ring_region->flags. */ +/* Region supports read. */ +#define MLXBF_PKA_RING_REGION_FLAG_READ BIT(0) +/* Region supports write. */ +#define MLXBF_PKA_RING_REGION_FLAG_WRITE BIT(1) +/* Region supports mmap. */ +#define MLXBF_PKA_RING_REGION_FLAG_MMAP BIT(2) + +/* Defines for mlxbf_pka_ring_region->type. */ +/* Info control/status words. */ +#define MLXBF_PKA_RING_RES_TYPE_WORDS 1 +/* Count registers. */ +#define MLXBF_PKA_RING_RES_TYPE_CNTRS 2 +/* Window RAM region. */ +#define MLXBF_PKA_RING_RES_TYPE_MEM 4 + +#define MLXBF_PKA_DRIVER_RING_DEV_MAX MLXBF_PKA_MAX_NUM_RINGS + +/* Defines for region index. */ +#define MLXBF_PKA_RING_REGION_WORDS_IDX 0 +#define MLXBF_PKA_RING_REGION_CNTRS_IDX 1 +#define MLXBF_PKA_RING_REGION_MEM_IDX 2 +#define MLXBF_PKA_RING_REGION_OFFSET_SHIFT 40 +#define MLXBF_PKA_RING_REGION_INDEX_TO_OFFSET(index) \ + ((u64)(index) << MLXBF_PKA_RING_REGION_OFFSET_SHIFT) + +struct mlxbf_pka_ring_device { + struct mlxbf_pka_info *info; + struct device *device; + u32 device_id; + u32 parent_device_id; + /* PKA ring device mutex. */ + struct mutex mutex; + struct mlxbf_pka_dev_ring_t *ring; + u32 num_regions; + struct mlxbf_pka_ring_region *regions; + struct miscdevice misc; +}; + #define MLXBF_PKA_DRIVER_DEV_MAX MLXBF_PKA_MAX_NUM_IO_BLOCKS =20 struct mlxbf_pka_device { @@ -143,6 +208,338 @@ static int mlxbf_pka_drv_verify_bootup_status(struct = device *dev) return 0; } =20 +static int mlxbf_pka_drv_ring_regions_init(struct mlxbf_pka_ring_device *r= ing_dev) +{ + struct mlxbf_pka_ring_region *region; + struct mlxbf_pka_dev_ring_t *ring; + struct mlxbf_pka_dev_res_t *res; + u32 num_regions; + + ring =3D ring_dev->ring; + if (!ring || !ring->shim) + return -ENXIO; + + num_regions =3D ring->resources_num; + ring_dev->num_regions =3D num_regions; + ring_dev->regions =3D devm_kcalloc(ring_dev->device, + num_regions, + sizeof(struct mlxbf_pka_ring_region), + GFP_KERNEL); + if (!ring_dev->regions) + return -ENOMEM; + + /* Information words region. */ + res =3D &ring->resources.info_words; + region =3D &ring_dev->regions[MLXBF_PKA_RING_REGION_WORDS_IDX]; + /* Map offset to the physical address. */ + region->off =3D MLXBF_PKA_RING_REGION_INDEX_TO_OFFSET(MLXBF_PKA_RING_REGI= ON_WORDS_IDX); + region->addr =3D res->base; + region->size =3D res->size; + region->type =3D MLXBF_PKA_RING_RES_TYPE_WORDS; + region->flags =3D MLXBF_PKA_RING_REGION_FLAG_MMAP | + MLXBF_PKA_RING_REGION_FLAG_READ | + MLXBF_PKA_RING_REGION_FLAG_WRITE; + + /* Counters registers region. */ + res =3D &ring->resources.counters; + region =3D &ring_dev->regions[MLXBF_PKA_RING_REGION_CNTRS_IDX]; + /* Map offset to the physical address. */ + region->off =3D MLXBF_PKA_RING_REGION_INDEX_TO_OFFSET(MLXBF_PKA_RING_REGI= ON_CNTRS_IDX); + region->addr =3D res->base; + region->size =3D res->size; + region->type =3D MLXBF_PKA_RING_RES_TYPE_CNTRS; + region->flags =3D MLXBF_PKA_RING_REGION_FLAG_MMAP | + MLXBF_PKA_RING_REGION_FLAG_READ | + MLXBF_PKA_RING_REGION_FLAG_WRITE; + + /* Window RAM region. */ + res =3D &ring->resources.window_ram; + region =3D &ring_dev->regions[MLXBF_PKA_RING_REGION_MEM_IDX]; + /* Map offset to the physical address. */ + region->off =3D MLXBF_PKA_RING_REGION_INDEX_TO_OFFSET(MLXBF_PKA_RING_REGI= ON_MEM_IDX); + region->addr =3D res->base; + region->size =3D res->size; + region->type =3D MLXBF_PKA_RING_RES_TYPE_MEM; + region->flags =3D MLXBF_PKA_RING_REGION_FLAG_MMAP | + MLXBF_PKA_RING_REGION_FLAG_READ | + MLXBF_PKA_RING_REGION_FLAG_WRITE; + + return 0; +} + +static void mlxbf_pka_drv_ring_regions_cleanup(struct mlxbf_pka_ring_devic= e *ring_dev) +{ + /* Clear PKA ring device regions. */ + ring_dev->num_regions =3D 0; +} + +static int mlxbf_pka_drv_ring_open(void *device_data) +{ + struct mlxbf_pka_ring_device *ring_dev =3D device_data; + struct mlxbf_pka_info *info =3D ring_dev->info; + struct mlxbf_pka_ring_info_t ring_info; + int ret; + + dev_dbg(ring_dev->device, "open ring device (device_data:%p)\n", ring_dev= ); + + if (!try_module_get(info->module)) + return -ENODEV; + + ring_info.ring_id =3D ring_dev->device_id; + ret =3D mlxbf_pka_dev_open_ring(ring_dev->device, &ring_info); + if (ret) { + dev_dbg(ring_dev->device, "failed to open ring\n"); + goto exit_open_ring; + } + + /* Initialize regions. */ + ret =3D mlxbf_pka_drv_ring_regions_init(ring_dev); + if (ret) + goto exit_ring_regions_init; + + return 0; + +exit_ring_regions_init: + mlxbf_pka_dev_close_ring(&ring_info); + +exit_open_ring: + module_put(info->module); + + return ret; +} + +static void mlxbf_pka_drv_ring_release(void *device_data) +{ + struct mlxbf_pka_ring_device *ring_dev =3D device_data; + struct mlxbf_pka_info *info =3D ring_dev->info; + struct mlxbf_pka_ring_info_t ring_info; + int ret; + + dev_dbg(ring_dev->device, "release ring device (device_data:%p)\n", ring_= dev); + + mlxbf_pka_drv_ring_regions_cleanup(ring_dev); + + ring_info.ring_id =3D ring_dev->device_id; + ret =3D mlxbf_pka_dev_close_ring(&ring_info); + if (ret) + dev_dbg(ring_dev->device, "failed to close ring\n"); + + module_put(info->module); +} + +static int mlxbf_pka_drv_ring_mmap_region(struct mlxbf_pka_ring_region reg= ion, + struct vm_area_struct *vma) +{ + u64 req_len, pgoff, req_start; + + req_len =3D vma->vm_end - vma->vm_start; + pgoff =3D vma->vm_pgoff & ((1U << (MLXBF_PKA_RING_REGION_OFFSET_SHIFT - P= AGE_SHIFT)) - 1); + req_start =3D pgoff << PAGE_SHIFT; + + region.size =3D roundup(region.size, PAGE_SIZE); + + if (req_start + req_len > region.size) + return -EINVAL; + + vma->vm_page_prot =3D pgprot_noncached(vma->vm_page_prot); + vma->vm_pgoff =3D (region.addr >> PAGE_SHIFT) + pgoff; + + return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, req_len, vma->v= m_page_prot); +} + +static int mlxbf_pka_drv_ring_mmap(void *device_data, struct vm_area_struc= t *vma) +{ + struct mlxbf_pka_ring_device *ring_dev =3D device_data; + struct mlxbf_pka_ring_region *region; + unsigned int index; + + dev_dbg(ring_dev->device, "mmap device\n"); + + index =3D vma->vm_pgoff >> (MLXBF_PKA_RING_REGION_OFFSET_SHIFT - PAGE_SHI= FT); + + if (vma->vm_end < vma->vm_start) + return -EINVAL; + if (!(vma->vm_flags & VM_SHARED)) + return -EINVAL; + if (index >=3D ring_dev->num_regions) + return -EINVAL; + if (vma->vm_start & ~PAGE_MASK) + return -EINVAL; + if (vma->vm_end & ~PAGE_MASK) + return -EINVAL; + + region =3D &ring_dev->regions[index]; + + if (!(region->flags & MLXBF_PKA_RING_REGION_FLAG_MMAP)) + return -EINVAL; + + if (!(region->flags & MLXBF_PKA_RING_REGION_FLAG_READ) && (vma->vm_flags = & VM_READ)) + return -EINVAL; + + if (!(region->flags & MLXBF_PKA_RING_REGION_FLAG_WRITE) && (vma->vm_flags= & VM_WRITE)) + return -EINVAL; + + vma->vm_private_data =3D ring_dev; + + if (region->type & MLXBF_PKA_RING_RES_TYPE_CNTRS || + region->type & MLXBF_PKA_RING_RES_TYPE_MEM) + return mlxbf_pka_drv_ring_mmap_region(ring_dev->regions[index], vma); + + if (region->type & MLXBF_PKA_RING_RES_TYPE_WORDS) + /* Currently user space is not allowed to access this region. */ + return -EINVAL; + + return -EINVAL; +} + +static long mlxbf_pka_drv_ring_ioctl(void *device_data, unsigned int cmd, = unsigned long arg) +{ + struct mlxbf_pka_ring_device *ring_dev =3D device_data; + + if (cmd =3D=3D MLXBF_PKA_RING_GET_REGION_INFO) { + struct mlxbf_pka_dev_region_info info; + + info.mem_index =3D MLXBF_PKA_RING_REGION_MEM_IDX; + info.mem_offset =3D ring_dev->regions[info.mem_index].off; + info.mem_size =3D ring_dev->regions[info.mem_index].size; + + info.reg_index =3D MLXBF_PKA_RING_REGION_CNTRS_IDX; + info.reg_offset =3D ring_dev->regions[info.reg_index].off; + info.reg_size =3D ring_dev->regions[info.reg_index].size; + + if (copy_to_user((void __user *)arg, &info, sizeof(info))) + return -EFAULT; + else + return 0; + + } else if (cmd =3D=3D MLXBF_PKA_GET_RING_INFO) { + struct mlxbf_pka_dev_hw_ring_info *this_ring_info; + struct mlxbf_pka_dev_hw_ring_info hw_ring_info; + + this_ring_info =3D ring_dev->ring->ring_info; + + hw_ring_info.cmd_base =3D this_ring_info->cmd_base; + hw_ring_info.rslt_base =3D this_ring_info->rslt_base; + hw_ring_info.size =3D this_ring_info->size; + hw_ring_info.host_desc_size =3D this_ring_info->host_desc_size; + hw_ring_info.in_order =3D this_ring_info->in_order; + hw_ring_info.cmd_rd_ptr =3D this_ring_info->cmd_rd_ptr; + hw_ring_info.rslt_wr_ptr =3D this_ring_info->rslt_wr_ptr; + hw_ring_info.cmd_rd_stats =3D this_ring_info->cmd_rd_ptr; + hw_ring_info.rslt_wr_stats =3D this_ring_info->rslt_wr_stats; + + if (copy_to_user((void __user *)arg, &hw_ring_info, sizeof(hw_ring_info)= )) + return -EFAULT; + else + return 0; + + } else if (cmd =3D=3D MLXBF_PKA_CLEAR_RING_COUNTERS) { + return mlxbf_pka_dev_clear_ring_counters(ring_dev->ring); + } + + return -ENOTTY; +} + +static int mlxbf_pka_drv_open(struct inode *inode, struct file *filep) +{ + struct mlxbf_pka_ring_device *ring_dev; + int ret; + + scoped_guard(mutex, &pka.idr_lock) { + ring_dev =3D idr_find(&pka.ring_idr, iminor(inode)); + } + if (!ring_dev) { + pr_err("mlxbf_pka error: failed to find idr for device\n"); + return -ENODEV; + } + + ret =3D mlxbf_pka_drv_ring_open(ring_dev); + if (ret) + return ret; + + filep->private_data =3D ring_dev; + return ret; +} + +static int mlxbf_pka_drv_release(struct inode *inode, struct file *filep) +{ + struct mlxbf_pka_ring_device *ring_dev =3D filep->private_data; + + filep->private_data =3D NULL; + mlxbf_pka_drv_ring_release(ring_dev); + + return 0; +} + +static int mlxbf_pka_drv_mmap(struct file *filep, struct vm_area_struct *v= ma) +{ + return mlxbf_pka_drv_ring_mmap(filep->private_data, vma); +} + +static long mlxbf_pka_drv_unlocked_ioctl(struct file *filep, unsigned int = cmd, unsigned long arg) +{ + return mlxbf_pka_drv_ring_ioctl(filep->private_data, cmd, arg); +} + +static const struct file_operations mlxbf_pka_ring_fops =3D { + .owner =3D THIS_MODULE, + .open =3D mlxbf_pka_drv_open, + .release =3D mlxbf_pka_drv_release, + .unlocked_ioctl =3D mlxbf_pka_drv_unlocked_ioctl, + .mmap =3D mlxbf_pka_drv_mmap, +}; + +static int mlxbf_pka_drv_add_ring_device(struct mlxbf_pka_ring_device *rin= g_dev) +{ + struct device *dev =3D ring_dev->device; + char name[MLXBF_PKA_DEVICE_NAME_MAX]; + int minor_number; + int ret; + + scnprintf(name, sizeof(name), MLXBF_PKA_DEVFS_RING_DEVICES, ring_dev->dev= ice_id); + + ring_dev->misc.minor =3D MISC_DYNAMIC_MINOR; + ring_dev->misc.name =3D &name[0]; + ring_dev->misc.mode =3D MLXBF_PKA_DEVICE_ACCESS_MODE; + ring_dev->misc.fops =3D &mlxbf_pka_ring_fops; + + ret =3D misc_register(&ring_dev->misc); + if (ret) { + dev_err(dev, "ring device registration failed: ret=3D%d\n", ret); + return ret; + } + + scoped_guard(mutex, &pka.idr_lock) { + minor_number =3D idr_alloc(&pka.ring_idr, ring_dev, ring_dev->misc.minor, + MINORMASK + 1, GFP_KERNEL); + } + if (minor_number !=3D ring_dev->misc.minor) { + dev_err(dev, "failed to allocate minor number %d\n", ring_dev->misc.mino= r); + return minor_number; + } + + dev_dbg(dev, "ring device minor:%d\n", ring_dev->misc.minor); + + return ret; +} + +static struct mlxbf_pka_ring_device *mlxbf_pka_drv_del_ring_device(struct = device *dev) +{ + struct platform_device *pdev =3D container_of(dev, struct platform_device= , dev); + struct mlxbf_pka_platdata *priv =3D platform_get_drvdata(pdev); + struct mlxbf_pka_info *info =3D priv->info; + struct mlxbf_pka_ring_device *ring_dev =3D info->priv; + + if (ring_dev) { + scoped_guard(mutex, &pka.idr_lock) { + idr_remove(&pka.ring_idr, ring_dev->misc.minor); + } + misc_deregister(&ring_dev->misc); + } + + return ring_dev; +} + static void mlxbf_pka_drv_get_mem_res(struct mlxbf_pka_device *mlxbf_pka_d= ev, struct mlxbf_pka_dev_mem_res *mem_res, u64 wndw_ram_off_mask) @@ -230,6 +627,38 @@ static int mlxbf_pka_drv_unregister_device(struct mlxb= f_pka_device *mlxbf_pka_de return mlxbf_pka_dev_unregister_shim(mlxbf_pka_dev->device, mlxbf_pka_dev= ->shim); } =20 +/* + * Note: this function must be serialized because it calls + * 'mlxbf_pka_dev_register_ring' which manipulates common counters for the= PKA + * ring devices. + */ +static int mlxbf_pka_drv_register_ring_device(struct mlxbf_pka_ring_device= *ring_dev) +{ + u32 shim_id =3D ring_dev->parent_device_id; + u32 ring_id =3D ring_dev->device_id; + int ret; + + ret =3D mlxbf_pka_dev_register_ring(ring_dev->device, ring_id, shim_id, &= ring_dev->ring); + if (ret) { + dev_dbg(ring_dev->device, "failed to register ring device\n"); + return ret; + } + + return 0; +} + +static void mlxbf_pka_drv_unregister_ring_device(struct mlxbf_pka_ring_dev= ice *ring_dev) +{ + if (!ring_dev) + return; + + if (!ring_dev->ring) + return; + + dev_dbg(ring_dev->device, "unregister ring device\n"); + mlxbf_pka_dev_unregister_ring(ring_dev->device, ring_dev->ring); +} + static int mlxbf_pka_drv_probe_device(struct mlxbf_pka_info *info) { struct mlxbf_pka_drv_plat_info *plat_info; @@ -315,6 +744,91 @@ static void mlxbf_pka_drv_remove_device(struct platfor= m_device *pdev) mlxbf_pka_drv_unregister_device(mlxbf_pka_dev); } =20 +static int mlxbf_pka_drv_probe_ring_device(struct mlxbf_pka_info *info) +{ + struct mlxbf_pka_ring_device *ring_dev; + struct device *dev =3D info->dev; + int ret; + + if (!info) + return -EINVAL; + + ring_dev =3D devm_kzalloc(dev, sizeof(*ring_dev), GFP_KERNEL); + if (!ring_dev) + return -ENOMEM; + + if (!mlxbf_pka_ring_device_cnt) { + mutex_init(&pka.idr_lock); + scoped_guard(mutex, &pka.idr_lock) { + /* Only initialize IDR if there is no ring device registered. */ + idr_init(&pka.ring_idr); + } + } + + scoped_guard(mutex, &mlxbf_pka_drv_lock) { + mlxbf_pka_ring_device_cnt +=3D 1; + if (mlxbf_pka_ring_device_cnt > MLXBF_PKA_DRIVER_RING_DEV_MAX) { + dev_dbg(dev, "cannot support %u ring devices\n", mlxbf_pka_ring_device_= cnt); + return -ENOSPC; + } + ring_dev->device_id =3D mlxbf_pka_ring_device_cnt - 1; + ring_dev->parent_device_id =3D mlxbf_pka_device_cnt - 1; + } + + ring_dev->info =3D info; + ring_dev->device =3D dev; + info->flag =3D MLXBF_PKA_DRIVER_FLAG_RING_DEVICE; + mutex_init(&ring_dev->mutex); + + /* Verify PKA bootup status. */ + ret =3D mlxbf_pka_drv_verify_bootup_status(dev); + if (ret) + return ret; + + scoped_guard(mutex, &mlxbf_pka_drv_lock) { + /* Add PKA ring device. */ + ret =3D mlxbf_pka_drv_add_ring_device(ring_dev); + if (ret) { + dev_dbg(dev, "failed to add ring device %u\n", ring_dev->device_id); + return ret; + } + + /* Register PKA ring device. */ + ret =3D mlxbf_pka_drv_register_ring_device(ring_dev); + if (ret) { + dev_dbg(dev, "failed to register ring device\n"); + goto err_register_ring; + } + } + + info->priv =3D ring_dev; + + return 0; + + err_register_ring: + mlxbf_pka_drv_del_ring_device(dev); + return ret; +} + +static void mlxbf_pka_drv_remove_ring_device(struct platform_device *pdev) +{ + struct mlxbf_pka_ring_device *ring_dev; + struct device *dev =3D &pdev->dev; + + ring_dev =3D mlxbf_pka_drv_del_ring_device(dev); + if (ring_dev) { + mlxbf_pka_drv_unregister_ring_device(ring_dev); + mlxbf_pka_ring_device_cnt--; + } + + if (!mlxbf_pka_ring_device_cnt) { + scoped_guard(mutex, &pka.idr_lock) { + /* Only destroy IDR if there is no ring device registered. */ + idr_destroy(&pka.ring_idr); + } + } +} + static int mlxbf_pka_drv_acpi_probe(struct platform_device *pdev, struct m= lxbf_pka_info *info) { struct device *dev =3D &pdev->dev; @@ -343,6 +857,15 @@ static int mlxbf_pka_drv_acpi_probe(struct platform_de= vice *pdev, struct mlxbf_p return ret; } dev_info(dev, "device probed\n"); + } else if (!strcmp(info->acpihid, mlxbf_pka_ring_acpihid_bf1) || + !strcmp(info->acpihid, mlxbf_pka_ring_acpihid_bf2) || + !strcmp(info->acpihid, mlxbf_pka_ring_acpihid_bf3)) { + ret =3D mlxbf_pka_drv_probe_ring_device(info); + if (ret) { + dev_dbg(dev, "failed to register ring device\n"); + return ret; + } + dev_dbg(dev, "ring device probed\n"); } =20 return 0; @@ -387,6 +910,11 @@ static void mlxbf_pka_drv_remove(struct platform_devic= e *pdev) struct mlxbf_pka_platdata *priv =3D platform_get_drvdata(pdev); struct mlxbf_pka_info *info =3D priv->info; =20 + if (info->flag =3D=3D MLXBF_PKA_DRIVER_FLAG_RING_DEVICE) { + dev_info(&pdev->dev, "remove ring device\n"); + mlxbf_pka_drv_remove_ring_device(pdev); + } + if (info->flag =3D=3D MLXBF_PKA_DRIVER_FLAG_DEVICE) { dev_info(&pdev->dev, "remove PKA device\n"); mlxbf_pka_drv_remove_device(pdev); diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_ring.c b/drivers= /platform/mellanox/mlxbf_pka/mlxbf_pka_ring.c new file mode 100644 index 000000000000..2f08975223e9 --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_ring.c @@ -0,0 +1,563 @@ +// SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause +// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. + +#include + +#include "mlxbf_pka_dev.h" +#include "mlxbf_pka_ring.h" + +/* + * Mapping PKA Ring address into Window RAM offset address. + * + * It converts the ring address, either physical address or virtual addres= s, to + * valid address into the Window RAM. This is done using the provided Wind= ow RAM + * win_base, ring_addr and win_mask parameters. Here, win_base is the actu= al + * physical address of the Window RAM, with the help of win_mask it is red= uced + * to Window RAM offset within that PKA block. Further, with the help of + * ring_addr and ring_size, we arrive at the Window RAM offset address for= a + * PKA Ring within the given Window RAM. + * + * The hardware encoded the ring size in 32-bit words, not bytes. Therefor= e, + * the ring size is right-shifted to convert bytes into words. + */ +static inline u64 mlxbf_pka_ring_mem_addr(u64 win_base, u64 win_mask, u64 = ring_addr, u64 ring_size) +{ + return (win_base & win_mask) | + FIELD_PREP(MLXBF_PKA_WINDOW_RAM_RING_ADDR_MASK, ring_addr) | + FIELD_PREP(MLXBF_PKA_WINDOW_RAM_RING_SIZE_MASK, + ((ring_addr & ~(ring_size - 1)) >> + MLXBF_PKA_WINDOW_RAM_RING_SIZE_SHIFT)); +} + +int mlxbf_pka_dev_clear_ring_counters(struct mlxbf_pka_dev_ring_t *ring) +{ + struct mlxbf_pka_dev_res_t *master_seq_ctrl_ptr; + u64 master_reg_base, master_reg_off; + struct mlxbf_pka_dev_shim_s *shim; + void __iomem *master_reg_ptr; + + shim =3D ring->shim; + master_seq_ctrl_ptr =3D &shim->resources.master_seq_ctrl; + master_reg_base =3D master_seq_ctrl_ptr->base; + master_reg_ptr =3D master_seq_ctrl_ptr->ioaddr; + master_reg_off =3D mlxbf_pka_dev_get_register_offset(master_reg_base, + MLXBF_PKA_MASTER_SEQ_CTRL_ADDR); + + /* Push the EIP-154 master controller into reset. */ + mlxbf_pka_dev_io_write(master_reg_ptr, master_reg_off, MLXBF_PKA_MASTER_S= EQ_CTRL_RESET); + + /* Clear counters. */ + mlxbf_pka_dev_io_write(master_reg_ptr, master_reg_off, + MLXBF_PKA_MASTER_SEQ_CTRL_CLEAR_COUNTERS); + + /* Take the EIP-154 master controller out of reset. */ + mlxbf_pka_dev_io_write(master_reg_ptr, master_reg_off, MASTER_CONTROLLER_= OUT_OF_RESET); + + return 0; +} + +/* + * Initialize ring. Set ring parameters and configure ring resources. It r= eturns + * 0 on success, a negative error code on failure. + */ +static int mlxbf_pka_dev_init_ring(struct device *dev, + struct mlxbf_pka_dev_ring_t *ring, + u32 ring_id, + struct mlxbf_pka_dev_shim_s *shim) +{ + struct mlxbf_pka_dev_res_t *ring_window_ram_ptr; + struct mlxbf_pka_dev_res_t *ring_info_words_ptr; + struct mlxbf_pka_dev_res_t *ring_counters_ptr; + u8 window_ram_split; + u32 ring_words_off; + u32 ring_cntrs_off; + u32 ring_mem_base; + u32 ring_mem_off; + u32 shim_ring_id; + int ret; + + if (ring->status !=3D MLXBF_PKA_DEV_RING_STATUS_UNDEFINED) { + dev_err(dev, "PKA ring must be undefined\n"); + return -EPERM; + } + + if (ring_id > MLXBF_PKA_MAX_NUM_RINGS - 1) { + dev_err(dev, "invalid ring identifier\n"); + return -EINVAL; + } + + ring->ring_id =3D ring_id; + ring->shim =3D shim; + ring->resources_num =3D MLXBF_PKA_MAX_NUM_RING_RESOURCES; + shim_ring_id =3D ring_id % MLXBF_PKA_MAX_NUM_IO_BLOCK_RINGS; + shim->rings[shim_ring_id] =3D ring; + + /* Configure ring information control/status words resource. */ + ring_info_words_ptr =3D &ring->resources.info_words; + ring_words_off =3D shim_ring_id * MLXBF_PKA_RING_WORDS_SPACING; + ring_info_words_ptr->base =3D ring_words_off + shim->mem_res.eip154_base + + MLXBF_PKA_RING_WORDS_ADDR; + ring_info_words_ptr->size =3D MLXBF_PKA_RING_WORDS_SIZE; + ring_info_words_ptr->type =3D MLXBF_PKA_DEV_RES_TYPE_MEM; + ring_info_words_ptr->status =3D MLXBF_PKA_DEV_RES_STATUS_UNMAPPED; + ring_info_words_ptr->name =3D "MLXBF_PKA_RING_INFO"; + + /* Configure ring counters registers resource. */ + ring_counters_ptr =3D &ring->resources.counters; + ring_cntrs_off =3D shim_ring_id * MLXBF_PKA_RING_CNTRS_SPACING; + ring_counters_ptr->base =3D ring_cntrs_off + shim->mem_res.eip154_base + + MLXBF_PKA_RING_CNTRS_ADDR; + ring_counters_ptr->size =3D MLXBF_PKA_RING_CNTRS_SIZE; + ring_counters_ptr->type =3D MLXBF_PKA_DEV_RES_TYPE_REG; + ring_counters_ptr->status =3D MLXBF_PKA_DEV_RES_STATUS_UNMAPPED; + ring_counters_ptr->name =3D "MLXBF_PKA_RING_CNTRS"; + + /* Configure ring window RAM resource. */ + window_ram_split =3D shim->window_ram_split; + if (window_ram_split =3D=3D MLXBF_PKA_SHIM_WINDOW_RAM_SPLIT_ENABLED) { + ring_mem_off =3D shim_ring_id * MLXBF_PKA_RING_MEM_1_SPACING; + ring_mem_base =3D ring_mem_off + shim->mem_res.alt_wndw_ram_0_base; + } else { + ring_mem_off =3D shim_ring_id * MLXBF_PKA_RING_MEM_0_SPACING; + ring_mem_base =3D ring_mem_off + shim->mem_res.wndw_ram_base; + } + + ring_window_ram_ptr =3D &ring->resources.window_ram; + ring_window_ram_ptr->base =3D ring_mem_base; + ring_window_ram_ptr->size =3D MLXBF_PKA_RING_MEM_SIZE; + ring_window_ram_ptr->type =3D MLXBF_PKA_DEV_RES_TYPE_MEM; + ring_window_ram_ptr->status =3D MLXBF_PKA_DEV_RES_STATUS_UNMAPPED; + ring_window_ram_ptr->name =3D "MLXBF_PKA_RING_WINDOW"; + + ring->ring_info =3D devm_kzalloc(dev, sizeof(*ring->ring_info), GFP_KERNE= L); + if (!ring->ring_info) + return -ENOMEM; + + ret =3D devm_mutex_init(dev, &ring->mutex); + if (ret) + return ret; + + ring->status =3D MLXBF_PKA_DEV_RING_STATUS_INITIALIZED; + + return 0; +} + +/* Release a given Ring. */ +int mlxbf_pka_dev_release_ring(struct device *dev, struct mlxbf_pka_dev_ri= ng_t *ring) +{ + struct mlxbf_pka_dev_shim_s *shim; + u32 shim_ring_id; + + if (ring->status =3D=3D MLXBF_PKA_DEV_RING_STATUS_UNDEFINED) + return 0; + + if (ring->status =3D=3D MLXBF_PKA_DEV_RING_STATUS_BUSY) { + dev_err(dev, "PKA ring is busy\n"); + return -EBUSY; + } + + shim =3D ring->shim; + + if (shim->status =3D=3D MLXBF_PKA_SHIM_STATUS_RUNNING) { + dev_err(dev, "PKA shim is running\n"); + return -EPERM; + } + + mlxbf_pka_dev_unset_resource_config(dev, shim, &ring->resources.info_word= s); + mlxbf_pka_dev_unset_resource_config(dev, shim, &ring->resources.counters); + mlxbf_pka_dev_unset_resource_config(dev, shim, &ring->resources.window_ra= m); + + ring->status =3D MLXBF_PKA_DEV_RING_STATUS_UNDEFINED; + shim_ring_id =3D ring->ring_id % MLXBF_PKA_MAX_NUM_IO_BLOCK_RINGS; + shim->rings[shim_ring_id] =3D NULL; + shim->rings_num--; + + return 0; +} + +/* + * Partition the window RAM for a given PKA ring. Here we statically divid= e the + * 16K memory region into three partitions: First partition is reserved for + * command descriptor ring (1K), second partition is reserved for result + * descriptor ring (1K), and the remaining 14K are reserved for vector dat= a. + * Through this memory partition scheme, command/result descriptor rings h= old a + * total of 1KB/64B =3D 16 descriptors each. The addresses for the rings s= tart at + * offset 0x3800. Also note that it is possible to have rings full while t= he + * vector data can support more data, the opposite can also happen, but it= is + * not suitable. For instance ECC point multiplication requires 8 input ve= ctors + * and 2 output vectors, a total of 10 vectors. If each vector has a lengt= h of + * 24 words (24x4B =3D 96B), we can process 14KB/960B =3D 14 operations wh= ich is + * close to 16 the total descriptors supported by rings. On the other hand, + * using 12K vector data region, allows to process only 12 operations, whi= le + * rings can hold 32 descriptors (ring usage is significantly low). + * + * For ECDSA verify, we have 12 vectors which require 1152B, with 14KB we = can + * handle 12 operations, against 10 operations with 12KB vector data memor= y. We + * believe that the aforementioned memory partition help us to leverage the + * trade-off between supported descriptors and required vectors. Note that= these + * examples give approximative values and does not include buffer word pad= ding + * across vectors. + * + * The function also writes the result descriptor rings base addresses, si= ze and + * type. And initialize the read and write pointers and statistics. It ret= urns + * 0 on success, a negative error code on failure. + * + * This function must be called once per ring, at initialization before any + * other functions are called. + */ +static int mlxbf_pka_dev_partition_mem(struct mlxbf_pka_dev_ring_t *ring) +{ + u64 rslt_desc_ring_base; + u64 cmd_desc_ring_base; + u32 cmd_desc_ring_size; + u64 window_ram_base; + u64 window_ram_size; + u32 ring_mem_base; + + if (!ring->shim || ring->status !=3D MLXBF_PKA_DEV_RING_STATUS_INITIALIZE= D) + return -EPERM; + + window_ram_base =3D ring->resources.window_ram.base; + window_ram_size =3D ring->resources.window_ram.size; + /* + * Partition ring memory. Give ring pair (cmd descriptor ring and rslt + * descriptor ring) an equal portion of the memory. The cmd descriptor + * ring and result descriptor ring are used as "non-overlapping" ring. + * Currently set aside 1/8 of the window RAM for command and result + * descriptor rings - giving a total of 1K/64B =3D 16 descriptors per + * ring. The remaining memory is "Data Memory" - i.e. memory to hold + * the command operands and results - also called input/output vectors + * (in all cases these vectors are just single large integers - often in + * the range of hundreds to thousands of bits long). + */ + ring_mem_base =3D window_ram_base + MLXBF_PKA_WINDOW_RAM_DATA_MEM_SIZE; + cmd_desc_ring_size =3D MLXBF_PKA_WINDOW_RAM_RING_MEM_SIZE / + MLXBF_PKA_WINDOW_RAM_RING_MEM_DIV; + ring->num_cmd_desc =3D MLXBF_PKA_WINDOW_RAM_RING_MEM_SIZE / + MLXBF_PKA_WINDOW_RAM_RING_MEM_DIV / CMD_DESC_SIZE; + /* + * The command and result descriptor rings may be placed at different + * (non-overlapping) locations in Window RAM memory space. PKI command + * interface: Most of the functionality is defined by the EIP-154 master + * firmware on the EIP-154 master controller Sequencer. + */ + cmd_desc_ring_base =3D ring_mem_base; + rslt_desc_ring_base =3D ring_mem_base + cmd_desc_ring_size; + + cmd_desc_ring_base =3D mlxbf_pka_ring_mem_addr(window_ram_base, + ring->shim->mem_res.wndw_ram_off_mask, + cmd_desc_ring_base, + window_ram_size); + rslt_desc_ring_base =3D mlxbf_pka_ring_mem_addr(window_ram_base, + ring->shim->mem_res.wndw_ram_off_mask, + rslt_desc_ring_base, + window_ram_size); + + /* Fill ring information. */ + memset(ring->ring_info, 0, sizeof(*ring->ring_info)); + + ring->ring_info->cmd_base =3D cmd_desc_ring_base; + ring->ring_info->rslt_base =3D rslt_desc_ring_base; + ring->ring_info->size =3D MLXBF_PKA_WINDOW_RAM_RING_MEM_SIZE / + MLXBF_PKA_WINDOW_RAM_RING_MEM_DIV / CMD_DESC_SIZE - 1; + ring->ring_info->host_desc_size =3D CMD_DESC_SIZE / sizeof(u32); + ring->ring_info->in_order =3D ring->shim->ring_type; + + return 0; +} + +/* + * Write the ring base address, ring size and type, and initialize (clear)= the + * read and write pointers and statistics. + */ +static int mlxbf_pka_dev_write_ring_info(struct device *dev, + struct mlxbf_pka_dev_res_t *buffer_ram_ptr, + u8 ring_id, + u32 ring_cmd_base_val, + u32 ring_rslt_base_val, + u32 ring_size_type_val) +{ + u32 ring_spacing; + u64 word_off; + + if (buffer_ram_ptr->status !=3D MLXBF_PKA_DEV_RES_STATUS_MAPPED || + buffer_ram_ptr->type !=3D MLXBF_PKA_DEV_RES_TYPE_MEM) + return -EPERM; + + dev_dbg(dev, "writing ring information control/status words\n"); + + ring_spacing =3D ring_id * MLXBF_PKA_RING_WORDS_SPACING; + + /* Get command descriptors from the Host ring. */ + word_off =3D mlxbf_pka_dev_get_word_offset(buffer_ram_ptr->base, + MLXBF_PKA_RING_CMD_BASE_0_ADDR + ring_spacing, + MLXBF_PKA_BUFFER_RAM_SIZE); + mlxbf_pka_dev_io_write(buffer_ram_ptr->ioaddr, word_off, ring_cmd_base_va= l); + + /* Put the result descriptors in the Host ring. */ + word_off =3D mlxbf_pka_dev_get_word_offset(buffer_ram_ptr->base, + MLXBF_PKA_RING_RSLT_BASE_0_ADDR + ring_spacing, + MLXBF_PKA_BUFFER_RAM_SIZE); + mlxbf_pka_dev_io_write(buffer_ram_ptr->ioaddr, word_off, ring_rslt_base_v= al); + + /* Write the ring size (number of descriptors) */ + word_off =3D mlxbf_pka_dev_get_word_offset(buffer_ram_ptr->base, + MLXBF_PKA_RING_SIZE_TYPE_0_ADDR + ring_spacing, + MLXBF_PKA_BUFFER_RAM_SIZE); + mlxbf_pka_dev_io_write(buffer_ram_ptr->ioaddr, word_off, ring_size_type_v= al); + + /* Write the command and result ring indices. */ + word_off =3D mlxbf_pka_dev_get_word_offset(buffer_ram_ptr->base, + MLXBF_PKA_RING_RW_PTRS_0_ADDR + ring_spacing, + MLXBF_PKA_BUFFER_RAM_SIZE); + mlxbf_pka_dev_io_write(buffer_ram_ptr->ioaddr, word_off, 0); + + /* + * Write the ring statistics (two 16-bit counters, one for commands and + * one for results). + */ + word_off =3D mlxbf_pka_dev_get_word_offset(buffer_ram_ptr->base, + MLXBF_PKA_RING_RW_STAT_0_ADDR + ring_spacing, + MLXBF_PKA_BUFFER_RAM_SIZE); + mlxbf_pka_dev_io_write(buffer_ram_ptr->ioaddr, word_off, 0); + + return 0; +} + +/* + * Set up the control/status words. Upon a PKI command the EIP-154 master + * firmware will read and partially update the ring information. + */ +static int mlxbf_pka_dev_set_ring_info(struct device *dev, struct mlxbf_pk= a_dev_ring_t *ring) +{ + u32 ring_cmd_base_val; + u32 ring_rslt_base_val; + u32 ring_size_type_val; + int ret; + + /* + * Ring info configuration MUST be done when the PKA ring is + * initialized. + */ + if ((ring->shim->status !=3D MLXBF_PKA_SHIM_STATUS_INITIALIZED && + ring->shim->status !=3D MLXBF_PKA_SHIM_STATUS_RUNNING && + ring->shim->status !=3D MLXBF_PKA_SHIM_STATUS_STOPPED) || + ring->status !=3D MLXBF_PKA_DEV_RING_STATUS_INITIALIZED) + return -EPERM; + + /* Partition ring memory. */ + ret =3D mlxbf_pka_dev_partition_mem(ring); + if (ret) { + dev_err(dev, "failed to initialize ring memory\n"); + return ret; + } + + /* Fill ring information. */ + ring_cmd_base_val =3D ring->ring_info->cmd_base; + ring_rslt_base_val =3D ring->ring_info->rslt_base; + ring_size_type_val =3D FIELD_PREP(MLXBF_PKA_RING_INFO_IN_ORDER_MASK, + ring->ring_info->in_order); + ring_size_type_val |=3D FIELD_PREP(MLXBF_PKA_RING_INFO_HOST_DESC_SIZE_MAS= K, + ring->ring_info->host_desc_size); + ring_size_type_val |=3D FIELD_PREP(MLXBF_PKA_RING_NUM_CMD_DESC_MASK, ring= ->num_cmd_desc - 1); + + /* Write ring information status/control words in the PKA Buffer RAM. */ + ret =3D mlxbf_pka_dev_write_ring_info(dev, + &ring->shim->resources.buffer_ram, + ring->ring_id % MLXBF_PKA_MAX_NUM_IO_BLOCK_RINGS, + ring_cmd_base_val, + ring_rslt_base_val, + ring_size_type_val); + if (ret) { + dev_err(dev, "failed to write ring information\n"); + return ret; + } + + ring->status =3D MLXBF_PKA_DEV_RING_STATUS_READY; + + return ret; +} + +/* Configure ring options. */ +int mlxbf_pka_dev_config_ring_options(struct device *dev, + struct mlxbf_pka_dev_res_t *buffer_ram_ptr, + u32 rings_num, + u8 ring_priority) +{ + u64 control_word; + u64 word_off; + + if (buffer_ram_ptr->status !=3D MLXBF_PKA_DEV_RES_STATUS_MAPPED || + buffer_ram_ptr->type !=3D MLXBF_PKA_DEV_RES_TYPE_MEM) + return -EPERM; + + if (rings_num > MLXBF_PKA_MAX_NUM_RINGS || rings_num < 1) { + dev_err(dev, "invalid rings number\n"); + return -EINVAL; + } + + dev_dbg(dev, "configure PKA ring options control word\n"); + + /* + * Write MLXBF_PKA_RING_OPTIONS control word located in the + * MLXBF_PKA_BUFFER_RAM. The value of this word is determined by the + * PKA I/O block (Shim). Set the number of implemented command/result + * ring pairs that is available in this EIP-154, encoded as binary + * value, which is 4. + */ + control_word =3D FIELD_PREP(MLXBF_PKA_RING_OPTIONS_RING_PRIORITY_MASK, ri= ng_priority) | + FIELD_PREP(MLXBF_PKA_RING_OPTIONS_RING_NUM_MASK, (rings_num - 1))= | + FIELD_PREP(MLXBF_PKA_RING_OPTIONS_SIGNATURE_BYTE_MASK, + MLXBF_PKA_RING_OPTIONS_SIGNATURE_BYTE); + word_off =3D mlxbf_pka_dev_get_word_offset(buffer_ram_ptr->base, + MLXBF_PKA_RING_OPTIONS_ADDR, + MLXBF_PKA_BUFFER_RAM_SIZE); + mlxbf_pka_dev_io_write(buffer_ram_ptr->ioaddr, word_off, control_word); + + return 0; +} + +/* Return the ring associated with the given identifier. */ +struct mlxbf_pka_dev_ring_t *mlxbf_pka_dev_get_ring(u32 ring_id) +{ + return mlxbf_pka_gbl_config.dev_rings[ring_id]; +} + +int mlxbf_pka_dev_register_ring(struct device *dev, + u32 ring_id, + u32 shim_id, + struct mlxbf_pka_dev_ring_t **ring) +{ + struct mlxbf_pka_dev_ring_t *ring_ptr; + struct mlxbf_pka_dev_shim_s *shim; + int ret; + + if (!ring) + return -EINVAL; + + shim =3D mlxbf_pka_dev_get_shim(shim_id); + if (!shim) + return -ENODEV; + + ring_ptr =3D devm_kzalloc(dev, sizeof(*ring_ptr), GFP_KERNEL); + if (!ring_ptr) + return -ENOMEM; + + ring_ptr->status =3D MLXBF_PKA_DEV_RING_STATUS_UNDEFINED; + + /* Initialize ring. */ + ret =3D mlxbf_pka_dev_init_ring(dev, ring_ptr, ring_id, shim); + if (ret) { + dev_err(dev, "failed to initialize ring %d\n", ring_id); + mlxbf_pka_dev_release_ring(dev, ring_ptr); + return ret; + } + + mlxbf_pka_gbl_config.dev_rings[ring_ptr->ring_id] =3D ring_ptr; + mlxbf_pka_gbl_config.dev_rings_cnt +=3D 1; + + *ring =3D ring_ptr; + return 0; +} + +int mlxbf_pka_dev_unregister_ring(struct device *dev, struct mlxbf_pka_dev= _ring_t *ring) +{ + if (!ring) + return -EINVAL; + + mlxbf_pka_gbl_config.dev_rings[ring->ring_id] =3D NULL; + mlxbf_pka_gbl_config.dev_rings_cnt -=3D 1; + + /* Release ring. */ + return mlxbf_pka_dev_release_ring(dev, ring); +} + +/* Syscall to open ring. */ +static int __mlxbf_pka_dev_open_ring(struct device *dev, u32 ring_id) +{ + struct mlxbf_pka_dev_shim_s *shim; + struct mlxbf_pka_dev_ring_t *ring; + int ret; + + if (!mlxbf_pka_gbl_config.dev_rings_cnt) + return -EPERM; + + ring =3D mlxbf_pka_dev_get_ring(ring_id); + if (!ring || !ring->shim) + return -ENXIO; + + shim =3D ring->shim; + + guard(mutex)(&ring->mutex); + + if (shim->status =3D=3D MLXBF_PKA_SHIM_STATUS_UNDEFINED || + shim->status =3D=3D MLXBF_PKA_SHIM_STATUS_CREATED || + shim->status =3D=3D MLXBF_PKA_SHIM_STATUS_FINALIZED) + return -EPERM; + + if (ring->status =3D=3D MLXBF_PKA_DEV_RING_STATUS_BUSY) + return -EBUSY; + + if (ring->status !=3D MLXBF_PKA_DEV_RING_STATUS_INITIALIZED) + return -EPERM; + + /* Set ring information words. */ + ret =3D mlxbf_pka_dev_set_ring_info(dev, ring); + if (ret) { + dev_err(dev, "failed to set ring information\n"); + return -EWOULDBLOCK; + } + + if (!shim->busy_ring_num) + shim->status =3D MLXBF_PKA_SHIM_STATUS_RUNNING; + + ring->status =3D MLXBF_PKA_DEV_RING_STATUS_BUSY; + shim->busy_ring_num +=3D 1; + + return ret; +} + +/* Open ring. */ +int mlxbf_pka_dev_open_ring(struct device *dev, struct mlxbf_pka_ring_info= _t *ring_info) +{ + return __mlxbf_pka_dev_open_ring(dev, ring_info->ring_id); +} + +/* Syscall to close ring. */ +static int __mlxbf_pka_dev_close_ring(u32 ring_id) +{ + struct mlxbf_pka_dev_shim_s *shim; + struct mlxbf_pka_dev_ring_t *ring; + + if (!mlxbf_pka_gbl_config.dev_rings_cnt) + return -EPERM; + + ring =3D mlxbf_pka_dev_get_ring(ring_id); + if (!ring || !ring->shim) + return -ENXIO; + + shim =3D ring->shim; + + guard(mutex)(&ring->mutex); + + if (shim->status !=3D MLXBF_PKA_SHIM_STATUS_RUNNING && + ring->status !=3D MLXBF_PKA_DEV_RING_STATUS_BUSY) + return -EPERM; + + ring->status =3D MLXBF_PKA_DEV_RING_STATUS_INITIALIZED; + shim->busy_ring_num -=3D 1; + + if (!shim->busy_ring_num) + shim->status =3D MLXBF_PKA_SHIM_STATUS_STOPPED; + + return 0; +} + +/* Close ring. */ +int mlxbf_pka_dev_close_ring(struct mlxbf_pka_ring_info_t *ring_info) +{ + if (ring_info) + return __mlxbf_pka_dev_close_ring(ring_info->ring_id); + + return 0; +} diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_ring.h b/drivers= /platform/mellanox/mlxbf_pka/mlxbf_pka_ring.h new file mode 100644 index 000000000000..f5be406656c0 --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_ring.h @@ -0,0 +1,255 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause */ +/* SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. */ + +#ifndef __MLXBF_PKA_RING_H__ +#define __MLXBF_PKA_RING_H__ + +/* PKA ring device related definitions. */ +#define CMD_DESC_SIZE 64 + +/** + * struct mlxbf_pka_ring_desc_t - PKA command/result ring descriptor + * @num_descs: Total number of descriptors in the ring + * @cmd_ring_base: Base address of the command ring + * @cmd_idx: Index of the command in a ring + * @rslt_ring_base: Base address of the result ring + * @rslt_idx: Index of the result in a ring + * @operands_base: Operands memory base address + * @operands_end: End address of operands memory + * @desc_size: Size of each element in the ring + * @cmd_desc_mask: Bitmask of free(0)/in_use(1) cmd descriptors + * @cmd_desc_cnt: Number of command descriptors currently in use + * @rslt_desc_cnt: Number of result descriptors currently ready + * + * Describes the PKA command/result ring as used by the hardware. A pair of + * command and result rings in PKA window memory, and the data memory used= by + * the commands. + */ +struct mlxbf_pka_ring_desc_t { + u32 num_descs; + u32 cmd_ring_base; + u32 cmd_idx; + u32 rslt_ring_base; + u32 rslt_idx; + u32 operands_base; + u32 operands_end; + u32 desc_size; + u64 cmd_desc_mask; + u32 cmd_desc_cnt; + u32 rslt_desc_cnt; +}; + +/** + * struct mlxbf_pka_ring_info_t - Ring parameters for user interface + * @fd: File descriptor + * @group: Iommu group + * @container: VFIO container + * @idx: Ring index + * @ring_id: Hardware ring identifier + * @mem_off: Offset specific to window RAM region + * @mem_addr: Window RAM region address + * @mem_size: Window RAM region size + * @reg_off: Offset specific to count registers region + * @reg_addr: Count registers region address + * @reg_size: Count registers region size + * @mem_ptr: Pointer to mapped memory region + * @reg_ptr: Pointer to mapped counters region + * @ring_desc: Ring descriptor + * @big_endian: Big endian byte order when enabled + * + * This structure declares ring parameters which can be used by user inter= face. + */ +struct mlxbf_pka_ring_info_t { + int fd; + int group; + int container; + u32 idx; + u32 ring_id; + u64 mem_off; + u64 mem_addr; + u64 mem_size; + u64 reg_off; + u64 reg_addr; + u64 reg_size; + void *mem_ptr; + void *reg_ptr; + struct mlxbf_pka_ring_desc_t ring_desc; + u8 big_endian; +}; + +/* ring_info related definitions. */ +#define MLXBF_PKA_RING_INFO_IN_ORDER_MASK GENMASK(31, 31) +#define MLXBF_PKA_RING_INFO_HOST_DESC_SIZE_MASK GENMASK(27, 18) +#define MLXBF_PKA_RING_NUM_CMD_DESC_MASK GENMASK(15, 0) + +/* Ring option related definitions. */ +#define MLXBF_PKA_RING_OPTIONS_RING_PRIORITY_MASK GENMASK(7, 0) +#define MLXBF_PKA_RING_OPTIONS_RING_NUM_MASK GENMASK(15, 8) +#define MLXBF_PKA_RING_OPTIONS_SIGNATURE_BYTE_MASK GENMASK(31, 24) + +/* + * Ring CSRs: these are all accessed from the ARM using 8 byte reads/write= s. + * However only the bottom 32 bits are implemented. + */ +/* Ring 0 CSRS. */ +#define MLXBF_PKA_COMMAND_COUNT_0_ADDR 0x80080 + +/* Ring 0. */ +#define MLXBF_PKA_RING_CMD_BASE_0_ADDR 0x00000 +#define MLXBF_PKA_RING_RSLT_BASE_0_ADDR 0x00010 +#define MLXBF_PKA_RING_SIZE_TYPE_0_ADDR 0x00020 +#define MLXBF_PKA_RING_RW_PTRS_0_ADDR 0x00028 +#define MLXBF_PKA_RING_RW_STAT_0_ADDR 0x00030 + +/* Ring Options. */ +#define MLXBF_PKA_RING_OPTIONS_ADDR 0x07FF8 + +/* + * Resources are regions which include info control/status words, count + * registers and host window RAM. + */ +#define MLXBF_PKA_MAX_NUM_RING_RESOURCES 3 + +/* + * PKA Ring resources. + * Define Ring resources parameters including base address, size (in bytes) + * and ring spacing. + */ +#define MLXBF_PKA_RING_WORDS_ADDR MLXBF_PKA_BUFFER_RAM_BASE +#define MLXBF_PKA_RING_CNTRS_ADDR MLXBF_PKA_COMMAND_COUNT_0_ADDR + +#define MLXBF_PKA_RING_WORDS_SIZE SZ_64 +#define MLXBF_PKA_RING_CNTRS_SIZE SZ_32 +#define MLXBF_PKA_RING_MEM_SIZE SZ_16K + +#define MLXBF_PKA_RING_WORDS_SPACING SZ_64 +#define MLXBF_PKA_RING_CNTRS_SPACING SZ_64K +#define MLXBF_PKA_RING_MEM_0_SPACING SZ_16K +#define MLXBF_PKA_RING_MEM_1_SPACING SZ_64K + +/* Defines for window RAM partition, valid for 16K memory. */ +#define MLXBF_PKA_WINDOW_RAM_RING_MEM_SIZE SZ_2K +#define MLXBF_PKA_WINDOW_RAM_RING_MEM_DIV 2 /* Divide into halves. */ +#define MLXBF_PKA_WINDOW_RAM_RING_ADDR_MASK GENMASK(15, 0) +#define MLXBF_PKA_WINDOW_RAM_RING_SIZE_MASK GENMASK(19, 16) +#define MLXBF_PKA_WINDOW_RAM_RING_SIZE_SHIFT 2 + +/* + * MLXBF_PKA_RING_OPTIONS_PRIORITY field to specify the priority in which = rings + * are handled. In this implementation, the priority is full rotating prio= rity, + * with the value of 0x0. + */ +#define MLXBF_PKA_RING_OPTIONS_PRIORITY 0x0 + +/* + * 'Signature' byte used because the ring options are transferred through = RAM + * which does not have a defined reset value. The EIP-154 master controller + * keeps reading the MLXBF_PKA_RING_OPTIONS word at start-up until the + * 'Signature' byte contains 0x46 and the 'Reserved' field contains zero. + */ +#define MLXBF_PKA_RING_OPTIONS_SIGNATURE_BYTE 0x46 + +/* + * Order of the result reporting: two schemas are available: + * InOrder - the results will be reported in the same order as the comman= ds + * were provided. + * OutOfOrder - the results are reported as soon as they are available. + * Note: only the OutOfOrder schema is used in this implementation. + */ +#define MLXBF_PKA_RING_TYPE_OUT_OF_ORDER_BIT 0 +#define MLXBF_PKA_RING_TYPE_IN_ORDER MLXBF_PKA_RING_TYPE_OUT_OF_ORDER_B= IT + +/* PKA device related definitions. */ +#define MLXBF_PKA_DEVFS_RING_DEVICES "mlxbf_pka/%d" + +/** + * struct mlxbf_pka_dev_ring_res_t - PKA Ring resources structure + * @info_words: Ring information words + * @counters: Ring counters + * @window_ram: Window RAM + */ +struct mlxbf_pka_dev_ring_res_t { + struct mlxbf_pka_dev_res_t info_words; + struct mlxbf_pka_dev_res_t counters; + struct mlxbf_pka_dev_res_t window_ram; +}; + +/** + * struct mlxbf_pka_dev_ring_t - PKA Ring structure + * @ring_id: Ring identifier + * @shim: Pointer to the shim associated to the ring + * @resources_num: Number of ring resources + * @resources: Ring resources + * @ring_info: Ring information + * @num_cmd_desc: Number of command descriptors + * @status: Status of the ring + * @mutex: Mutex lock for sharing ring device + */ +struct mlxbf_pka_dev_ring_t { + u32 ring_id; + struct mlxbf_pka_dev_shim_s *shim; + u32 resources_num; + struct mlxbf_pka_dev_ring_res_t resources; + struct mlxbf_pka_dev_hw_ring_info *ring_info; + u32 num_cmd_desc; + s8 status; + struct mutex mutex; +}; + +/* Defines for mlxbf_pka_dev_ring->status. */ +#define MLXBF_PKA_DEV_RING_STATUS_UNDEFINED -1 +#define MLXBF_PKA_DEV_RING_STATUS_INITIALIZED 1 +#define MLXBF_PKA_DEV_RING_STATUS_READY 2 +#define MLXBF_PKA_DEV_RING_STATUS_BUSY 3 + +/* + * Ring getter for mlxbf_pka_dev_gbl_config_t structure which holds all sy= stem + * global configuration. This configuration is shared and common to kernel + * device driver associated with PKA hardware. + */ +struct mlxbf_pka_dev_ring_t *mlxbf_pka_dev_get_ring(u32 ring_id); + +/* Configure ring options. */ +int mlxbf_pka_dev_config_ring_options(struct device *dev, + struct mlxbf_pka_dev_res_t *buffer_ram_ptr, + u32 rings_num, + u8 ring_priority); + + /* Release a given Ring. */ +int mlxbf_pka_dev_release_ring(struct device *dev, struct mlxbf_pka_dev_ri= ng_t *ring); + +/* + * Register a ring. This function initializes a Ring and configures its re= lated + * resources, and returns the error code. + */ +int mlxbf_pka_dev_register_ring(struct device *dev, + u32 ring_id, + u32 shim_id, + struct mlxbf_pka_dev_ring_t **ring); + +/* Unregister a Ring. */ +int mlxbf_pka_dev_unregister_ring(struct device *dev, struct mlxbf_pka_dev= _ring_t *ring); + +/* Reset a Ring. */ +int mlxbf_pka_dev_reset_ring(struct mlxbf_pka_dev_ring_t *ring); + +/* + * Clear ring counters. This function resets the master sequencer controll= er to + * clear the command and result counters. + */ +int mlxbf_pka_dev_clear_ring_counters(struct mlxbf_pka_dev_ring_t *ring); + +/* + * Open the file descriptor associated with ring. It returns an integer va= lue, + * which is used to refer to the file. If not successful, it returns a neg= ative + * error. + */ +int mlxbf_pka_dev_open_ring(struct device *dev, struct mlxbf_pka_ring_info= _t *ring_info); + +/* + * Close the file descriptor associated with ring. The function returns 0 = if + * successful, negative value to indicate an error. + */ +int mlxbf_pka_dev_close_ring(struct mlxbf_pka_ring_info_t *ring_info); + +#endif /* __MLXBF_PKA_RING_H__ */ diff --git a/include/uapi/linux/mlxbf-pka.h b/include/uapi/linux/mlxbf-pka.h new file mode 100644 index 000000000000..9da6c1e5044b --- /dev/null +++ b/include/uapi/linux/mlxbf-pka.h @@ -0,0 +1,93 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. */ + +#ifndef _UAPI_LINUX_MLXBF_PKA_H +#define _UAPI_LINUX_MLXBF_PKA_H + +#include +#include + +/* PKA IOCTL related definitions. */ +#define MLXBF_PKA_IOC_TYPE 0xBF + +/** + * struct mlxbf_pka_dev_region_info - Device region information + * @reg_index: Registers region index + * @reg_size: Registers region size (bytes) + * @reg_offset: Registers region offset from start of device fd + * @mem_index: Memory region index + * @mem_size: Memory region size (bytes) + * @mem_offset: Memory region offset from start of device fd + * + * MLXBF_PKA_RING_GET_REGION_INFO: + * _IOWR(MLXBF_PKA_IOC_TYPE, 0x0, mlxbf_pka_dev_region_info). + * + * Retrieve information about a device region. This is intended to describe + * MMIO, I/O port, as well as bus specific regions (ex. PCI config space).= Zero + * sized regions may be used to describe unimplemented regions. + * + * Return: 0 on success, -errno on failure. + */ +struct mlxbf_pka_dev_region_info { + __u32 reg_index; + __u64 reg_size; + __u64 reg_offset; + __u32 mem_index; + __u64 mem_size; + __u64 mem_offset; +}; + +#define MLXBF_PKA_RING_GET_REGION_INFO \ + _IOWR(MLXBF_PKA_IOC_TYPE, 0x0, struct mlxbf_pka_dev_region_info) + +/** + * struct mlxbf_pka_dev_hw_ring_info - Bluefield specific ring information + * @cmd_base: Base address of the command descriptor ring + * @rslt_base: Base address of the result descriptor ring + * @size: Size of a command ring in number of descriptors, minus 1. Minimum + * value is 0 (for 1 descriptor); maximum value is 65535 (for 64K + * descriptors) + * @host_desc_size: This field specifies the size (in 32-bit words) of the= space + * that PKI command and result descriptor occupies on the Host + * @in_order: Indicates whether the result ring delivers results strictly + * in-order ('1') or that result descriptors are written to the + * result ring as soon as they become available, or out-of-order ('0') + * @cmd_rd_ptr: Read pointer of the command descriptor ring + * @rslt_wr_ptr: Write pointer of the result descriptor ring + * @cmd_rd_stats: Read statistics of the command descriptor ring + * @rslt_wr_stats: Write statistics of the result descriptor ring + * + * MLXBF_PKA_GET_RING_INFO: + * _IOWR(MLXBF_PKA_IOC_TYPE, 0x1, mlxbf_pka_dev_hw_ring_info). + * + * Retrieve information about a ring. This is intended to describe ring + * information words located in MLXBF_PKA_BUFFER_RAM. Ring information + * includes base addresses, size and statistics. + * + * Return: 0 on success, -errno on failure. + */ +struct mlxbf_pka_dev_hw_ring_info { + __u64 cmd_base; + __u64 rslt_base; + __u16 size; + __u16 host_desc_size : 10; + __u8 in_order : 1; + __u16 cmd_rd_ptr; + __u16 rslt_wr_ptr; + __u16 cmd_rd_stats; + __u16 rslt_wr_stats; +}; + +#define MLXBF_PKA_GET_RING_INFO _IOWR(MLXBF_PKA_IOC_TYPE, 0x1, struct mlxb= f_pka_dev_hw_ring_info) + +/** + * MLXBF_PKA_CLEAR_RING_COUNTERS: + * _IO(MLXBF_PKA_IOC_TYPE, 0x2). + * + * Clear counters. This is intended to reset all command and result counte= rs. + * + * Return: 0 on success, -errno on failure. + */ +#define MLXBF_PKA_CLEAR_RING_COUNTERS _IO(MLXBF_PKA_IOC_TYPE, 0x2) + +#endif /* _UAPI_LINUX_MLXBF_PKA_H */ --=20 2.34.1 From nobody Thu Oct 2 06:18:04 2025 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012018.outbound.protection.outlook.com [52.101.43.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78CF132143E; Fri, 19 Sep 2025 19:53:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.18 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758311602; cv=fail; b=lXwI0ZB9c9q0W+TurtVQkZtZstEXoDUuMGLs+Ly/kte5vcmWSWn+kAgMEfVptArLo1HsyznEkw19vGMlj6e1i+f9GsrywI6s+X0goBhxwddwgD3tCBh+urhKmQiy4bu8kO9VQcp5B6Vts2ETS4jhCtOGnXPmIkxztA1GNF4BP6k= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758311602; c=relaxed/simple; bh=PC3pgxfvBJ2p2GI9qLU2qXj3g5+0cCsVg+Sr8pR7Hi0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RlZtOLLlXfy7gljFjtK5zOVpfXxYE5GrPYxU3Agv5jCnRY5r6CsTnPSCRHjcqTV0VdGwCqSwGVRApUUawPbDxpVX+KHWk1D6tLSHBXamSmN4qCZovUr2dF1En/+HX+IBHer+j8eZB2U8fDRAsKrIL9FaDRNZ1TQBCCWIz0f2oxQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SQv0jeaW; arc=fail smtp.client-ip=52.101.43.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SQv0jeaW" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Xx58w32kCACqi4WoErc9gAuy9aA2HT4987Ik/i4yLcz1lM8iYTK4599eLvlx2Xw7nlLV1TuRVCTGAOWS/XEpqmjNmXwlcy64e8QGIDv7FAKKECwBS5WovUYUmMtYLOWoeR36ok3VrpJhsh5R391xu+vNCWvwLuHNYqbBvvOJTAoNR8oIHQWXLV5ootoh0UoLqozguF+olHrjxo7lkwqTNTYaLsQXVxXzgF/xU4keSq+NLFgikXhvtQsjBr+2bFuJzW5lvfQotxZKyvtJY0loEu5AZj9DWRTuJHFG9w/WzupRkRvZaf7lpkTmeW4mukcld7JvGdQrFb83GJ3pN6mSWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=go2a0ObE3Trq89zOLUqiti0Qw7XEHjlsq00e/g8s/80=; b=jzJjRXeWz2xFv2kdi1ZZpQzOliKHedr7agVgHbReQet3rqbrVOnpwrAEdsyKk4iGL6yy7aaSn3qVy/C7e8rGq0/jAAPBpnqwSRMdPs5IHLRplhfIh2eDM1Aq5tPu5HGV9vxTAdETN4LqqpZ2b3zVDIffuIsmDVXBOQVLFPuRl6m2xvEmnlBcoMJSEo+rGJNpTk7l37+yddwjSN9wBLOwgW2JvPjHzK+YlaOChPYbQKMa3ptiRiCAWCRaxDPu5pGZNTyaVpIRtEgiZcMku52Rlfnxe2vxVIJkpby1FbcJSrY5rIPiVRCiVbho8j4B7wVsb1z/WP+FXR/oPbCu0xMWew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=go2a0ObE3Trq89zOLUqiti0Qw7XEHjlsq00e/g8s/80=; b=SQv0jeaWsMjdAhUYo4mHfKzupuyEJmN2FJcn1rt0mymyRHIgAmKz0uMFKXps8g1rPVMRr3Ut91RPupflJmOfjpRtBlVUdjDb0sAlAhvU5BIYDYQhja90DxWd14o4cTTLMJYo6F5FkStry6v48F3thlHhcbaHJumDYmjlU6J5XiHL9PLKc4koDJsLZpOrM3FAVeYFQBQpVHksN4HZZeB5CWporlEbl3yyUourFsfpWsJ+Ue9vD6Wwh0hzcNAryWcZPW45RqSwfZOIVit9KmnJ7z8NVw2tmYUmdjGNkWv+jy2v400nYebe1MTWx0/LiDwbq6MKhEU7TU2j5zlZzuRSbQ== Received: from CH2PR16CA0011.namprd16.prod.outlook.com (2603:10b6:610:50::21) by SJ0PR12MB6880.namprd12.prod.outlook.com (2603:10b6:a03:485::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9137.13; Fri, 19 Sep 2025 19:53:06 +0000 Received: from CH1PEPF0000AD82.namprd04.prod.outlook.com (2603:10b6:610:50:cafe::91) by CH2PR16CA0011.outlook.office365.com (2603:10b6:610:50::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9137.15 via Frontend Transport; Fri, 19 Sep 2025 19:53:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CH1PEPF0000AD82.mail.protection.outlook.com (10.167.244.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9137.12 via Frontend Transport; Fri, 19 Sep 2025 19:53:05 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.34; Fri, 19 Sep 2025 12:51:39 -0700 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Fri, 19 Sep 2025 12:51:38 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server id 15.2.1544.34 via Frontend Transport; Fri, 19 Sep 2025 12:51:38 -0700 From: Ron Li To: , , , , , CC: , , , Ron Li Subject: [PATCH v3 3/3] platform/mellanox/mlxbf_pka: Add TRNG support and hwrng integration Date: Fri, 19 Sep 2025 19:51:32 +0000 Message-ID: <20250919195132.1088515-4-xiangrongl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250919195132.1088515-1-xiangrongl@nvidia.com> References: <20250919195132.1088515-1-xiangrongl@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD82:EE_|SJ0PR12MB6880:EE_ X-MS-Office365-Filtering-Correlation-Id: b3a06956-d6f8-4270-cab3-08ddf7b62672 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|1800799024|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?xPLUPVs1FR6iQ/KrgLULubFMLV9Fnr2PFt9DTiH2ElbuzHAyzOPRdumY7pUW?= =?us-ascii?Q?LQmlDjvGP4WMC0KCI2Z74/KdsyhCtFa7Wdwof5OBB5hgvF+dYITfOUZsyi2B?= =?us-ascii?Q?KLflAQXj2rY2wkW9w/Om/Pj8mBjmUfFiFsOkEm5a7FEYejXOCipB4yFXrarV?= =?us-ascii?Q?Ol/BFi/Bf3wTdibqMuOLE/Y6JbQUhnZxIiZz1jwLgLwDfdjP77zZLsKmWwNs?= =?us-ascii?Q?7lfuiKQRRAg7VBm115RSAEjtlG2WJfmvrcplH8R3nZxzLU9JVygyby5DJOmj?= =?us-ascii?Q?cX4hK+w99yiyd6vMogGwcELo3gGrWWdk5rBMDIS4xl1lMXcTUkDXp4elIXIW?= =?us-ascii?Q?suTbRY68i8jFzEtv+QATrfA6BEYu+9bohP2d5N3mOZZUrWOeaDFjl3K0gsh3?= =?us-ascii?Q?FwVqrjJ8w8WwMn94mAjJFC7u6X0ieGObiKAh0fWlWf8H2jtR64flGKR7Xjmn?= =?us-ascii?Q?6SBmciM6xTtYICKq4FAEE1eOIpjFO0XkqtzjE0FLDL04Kfk7RGAA6bEU/Rxy?= =?us-ascii?Q?FOSk1/ON3celXyut+vxUiAV1cOJZ4q1lEo6a7aby5XLUbFw1+u00PUGcAZZs?= =?us-ascii?Q?JgquAKxoi2Ahshe/u2drOt2Ralhp7YAeX0le0DNAHjU2PqWxm3uWlu+iz6CJ?= =?us-ascii?Q?0o000GghBUczVFDtCDQ+uY5YQz+A2IH2UoMHEIDFwhRoNsZog8k5sFxZkfOr?= =?us-ascii?Q?slmNDusPem/jQHoYsZkx68tDLs9ZNDshFKHrZQmbrLyN/I2ehZI3kY3HLs50?= =?us-ascii?Q?IlTXGnQHm9cFpD3SB0KtLnpSlW4Wr+jc2UkJ4OQX+xFfaqK01AsqSdzm4J8b?= =?us-ascii?Q?SIuat/txQFXYkoROMZQ3BSwyJbPIEoqQR8IBTHzFT5OdN7PqdXmzh7pZke8n?= =?us-ascii?Q?jBaix9mcurG51iZFmUqN9IGxXrRAm3PmV/rGBTmeMsUm1Bxh1eSuD7PrdrGA?= =?us-ascii?Q?OMVAA3QDbUJ5wumHmvsoG2qohi5asmzfpPGImP7mabGLz2qjKqaujPV5j2cJ?= =?us-ascii?Q?bEEnyFG+PdIKkXWLkIIeDYRMGCbjcBRoucYKurm9YfDAfjbQ+kUE/cTK54F2?= =?us-ascii?Q?yBe6/sbrT9QBbSXt+iJoZn+JGxqn/bJs8G3VMIBv80L0wwaTNN0wU01z/NeP?= =?us-ascii?Q?6XRUVubnIoYAb+wt4k3uvxkimR/05x0SdQHOP3q07AnaScIEIMSEVv16dgnl?= =?us-ascii?Q?EbwYjTrFNrwX5Em5wk2P2kPD7zZ+dg5e8Ibi3aaRjVDWfFHeDsPRzK4hSwzm?= =?us-ascii?Q?/RquDKML5jPVj2K6FGED78CGA4bVUF6lN3g17emNd1Vj5KlCerK28r0OeFMU?= =?us-ascii?Q?f8f2TmhEfV2k4pyjCYlpKH7h+DtlUdMcCSFPzSG4AWMg8J95dPpdOMSE45QM?= =?us-ascii?Q?WoOCNQt76JdRrJ8SxGLkq0ADU/C9aDkCdnC8qTK2GsgqcPR3xqQTZ6HT5OY8?= =?us-ascii?Q?rZnuxD9GYzrecVl2IGSHN6Tc+chM67fVBGSm/4fddWBjuJOO0qOmFFUH4wJl?= =?us-ascii?Q?1QKL4a2AiaDZo5m5pDp6+INuW1j1LWgZpYS2?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(376014)(1800799024)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2025 19:53:05.4885 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b3a06956-d6f8-4270-cab3-08ddf7b62672 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD82.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6880 Content-Type: text/plain; charset="utf-8" Add True Random Number Generator support to the BlueField PKA driver and wire it into the HWRNG core to feed kernel entropy (/dev/hwrng). - Enable TRNG clock and configure hardware - Run TRNG health tests (monobit/run/poker) and a sanity check - Implement shutdown-overflow recovery and FRO detune/reenable - Initialize TRNG during shim bring-up and prefetch initial output - Provide read path from TRNG output registers - Register/unregister with the HWRNG core Reviewed-by: David Thompson Reviewed-by: Khalil Blaiech Signed-off-by: Ron Li --- .../ABI/testing/sysfs-platform-mellanox-pka | 3 +- drivers/platform/mellanox/mlxbf_pka/Makefile | 1 + .../mellanox/mlxbf_pka/mlxbf_pka_dev.c | 48 +- .../mellanox/mlxbf_pka/mlxbf_pka_dev.h | 6 + .../mellanox/mlxbf_pka/mlxbf_pka_drv.c | 75 ++ .../mellanox/mlxbf_pka/mlxbf_pka_trng.c | 874 ++++++++++++++++++ .../mellanox/mlxbf_pka/mlxbf_pka_trng.h | 152 +++ include/uapi/linux/mlxbf-pka.h | 19 + 8 files changed, 1176 insertions(+), 2 deletions(-) create mode 100644 drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_trng.c create mode 100644 drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_trng.h diff --git a/Documentation/ABI/testing/sysfs-platform-mellanox-pka b/Docume= ntation/ABI/testing/sysfs-platform-mellanox-pka index e7d331bca8fb..d45575dc0d77 100644 --- a/Documentation/ABI/testing/sysfs-platform-mellanox-pka +++ b/Documentation/ABI/testing/sysfs-platform-mellanox-pka @@ -30,5 +30,6 @@ Description: - release(): close the PKA ring device specified by the device ID, and r= elease the related RAM regions. - unlocked_ioctl(): make PKA related system calls, including getting rin= g device or - RAM region information, clearing PKA ring counter. + RAM region information, clearing PKA ring counter and getting random b= ytes from + the TRNG module from the PKA device. - mmap(): map the PKA ring device to the virtual memory region. diff --git a/drivers/platform/mellanox/mlxbf_pka/Makefile b/drivers/platfor= m/mellanox/mlxbf_pka/Makefile index 6e536794d339..4f6df4de9cb7 100644 --- a/drivers/platform/mellanox/mlxbf_pka/Makefile +++ b/drivers/platform/mellanox/mlxbf_pka/Makefile @@ -9,3 +9,4 @@ obj-$(CONFIG_MLXBF_PKA) +=3D mlxbf-pka.o mlxbf-pka-objs :=3D mlxbf_pka_drv.o mlxbf-pka-objs +=3D mlxbf_pka_dev.o mlxbf-pka-objs +=3D mlxbf_pka_ring.o +mlxbf-pka-objs +=3D mlxbf_pka_trng.o \ No newline at end of file diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c index 12df11dd1eee..f649ae9ef967 100644 --- a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.c @@ -18,6 +18,7 @@ =20 #include "mlxbf_pka_dev.h" #include "mlxbf_pka_ring.h" +#include "mlxbf_pka_trng.h" =20 struct mlxbf_pka_dev_gbl_config_t mlxbf_pka_gbl_config; =20 @@ -250,6 +251,22 @@ static int mlxbf_pka_dev_create_shim(struct device *de= v, return ret; } =20 + /* Set PKA device TRNG registers. */ + reg_size =3D PAGE_SIZE; + reg_base =3D mlxbf_pka_dev_get_register_base(shim->mem_res.eip154_base, + MLXBF_PKA_TRNG_OUTPUT_0_ADDR); + ret =3D mlxbf_pka_dev_set_resource_config(dev, + shim, + &shim->resources.trng_csr, + reg_base, + reg_size, + MLXBF_PKA_DEV_RES_TYPE_REG, + "MLXBF_PKA_TRNG_CSR"); + if (ret) { + dev_err(dev, "unable to setup the TRNG\n"); + return ret; + } + shim->status =3D MLXBF_PKA_SHIM_STATUS_CREATED; =20 return ret; @@ -258,7 +275,7 @@ static int mlxbf_pka_dev_create_shim(struct device *dev, /* Delete shim and unset shim resources. */ static int mlxbf_pka_dev_delete_shim(struct device *dev, struct mlxbf_pka_= dev_shim_s *shim) { - struct mlxbf_pka_dev_res_t *res_master_seq_ctrl, *res_aic_csr; + struct mlxbf_pka_dev_res_t *res_master_seq_ctrl, *res_aic_csr, *res_trng_= csr; struct mlxbf_pka_dev_res_t *res_buffer_ram; =20 dev_dbg(dev, "PKA device delete shim\n"); @@ -275,10 +292,12 @@ static int mlxbf_pka_dev_delete_shim(struct device *d= ev, struct mlxbf_pka_dev_sh res_buffer_ram =3D &shim->resources.buffer_ram; res_master_seq_ctrl =3D &shim->resources.master_seq_ctrl; res_aic_csr =3D &shim->resources.aic_csr; + res_trng_csr =3D &shim->resources.trng_csr; =20 mlxbf_pka_dev_unset_resource_config(dev, shim, res_buffer_ram); mlxbf_pka_dev_unset_resource_config(dev, shim, res_master_seq_ctrl); mlxbf_pka_dev_unset_resource_config(dev, shim, res_aic_csr); + mlxbf_pka_dev_unset_resource_config(dev, shim, res_trng_csr); =20 shim->status =3D MLXBF_PKA_SHIM_STATUS_UNDEFINED; =20 @@ -294,7 +313,9 @@ static int mlxbf_pka_dev_delete_shim(struct device *dev= , struct mlxbf_pka_dev_sh */ static int mlxbf_pka_dev_init_shim(struct device *dev, struct mlxbf_pka_de= v_shim_s *shim) { + u32 data[MLXBF_PKA_TRNG_OUTPUT_CNT]; int ret; + u8 i; =20 if (shim->status !=3D MLXBF_PKA_SHIM_STATUS_CREATED) { dev_err(dev, "PKA device must be created\n"); @@ -311,6 +332,31 @@ static int mlxbf_pka_dev_init_shim(struct device *dev,= struct mlxbf_pka_dev_shim return ret; } =20 + shim->trng_enabled =3D MLXBF_PKA_SHIM_TRNG_ENABLED; + shim->trng_err_cycle =3D 0; + + /* Configure the TRNG. */ + ret =3D mlxbf_pka_dev_config_trng_drbg(dev, + &shim->resources.aic_csr, + &shim->resources.trng_csr); + + /* + * Pull out data from the content of the TRNG buffer RAM and start the + * regeneration of new numbers; read and drop 512 words. The read must + * be done over the 4 TRNG_OUTPUT_X registers at a time. + */ + for (i =3D 0; i < MLXBF_PKA_TRNG_NUM_OF_FOUR_WORD; i++) + mlxbf_pka_dev_trng_read(dev, shim, data, sizeof(data)); + + if (ret) { + /* + * Keep running without TRNG since it does not hurt, but notify + * users. + */ + dev_err(dev, "failed to configure TRNG\n"); + shim->trng_enabled =3D MLXBF_PKA_SHIM_TRNG_DISABLED; + } + ret =3D devm_mutex_init(dev, &shim->mutex); if (ret) return ret; diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h index 12512850cf79..ba0bbc87c832 100644 --- a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_dev.h @@ -128,11 +128,13 @@ struct mlxbf_pka_dev_res_t { * @buffer_ram: Buffer RAM * @master_seq_ctrl: Master sequencer controller CSR * @aic_csr: Interrupt controller CSRs + * @trng_csr: TRNG module CSRs */ struct mlxbf_pka_dev_shim_res_t { struct mlxbf_pka_dev_res_t buffer_ram; struct mlxbf_pka_dev_res_t master_seq_ctrl; struct mlxbf_pka_dev_res_t aic_csr; + struct mlxbf_pka_dev_res_t trng_csr; }; =20 /* Number of PKA device resources. */ @@ -180,6 +182,7 @@ struct mlxbf_pka_dev_mem_res { /** * struct mlxbf_pka_dev_shim_s - PKA Shim structure * @mem_res: Memory resources + * @trng_err_cycle: TRNG error cycle * @shim_id: Shim identifier * @rings_num: Number of supported rings (hardware specific) * @rings: Pointer to rings which belong to the shim @@ -188,11 +191,13 @@ struct mlxbf_pka_dev_mem_res { * @resources: Shim resources * @window_ram_split: If non-zero, the split window RAM scheme is used * @busy_ring_num: Number of active rings (rings in busy state) + * @trng_enabled: Whether the TRNG engine is enabled * @status: Status of the shim * @mutex: Mutex lock for sharing shim */ struct mlxbf_pka_dev_shim_s { struct mlxbf_pka_dev_mem_res mem_res; + u64 trng_err_cycle; u32 shim_id; u32 rings_num; struct mlxbf_pka_dev_ring_t **rings; @@ -201,6 +206,7 @@ struct mlxbf_pka_dev_shim_s { struct mlxbf_pka_dev_shim_res_t resources; u8 window_ram_split; u32 busy_ring_num; + u8 trng_enabled; s8 status; struct mutex mutex; }; diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c b/drivers/= platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c index a009437e4a48..a1ae007ba903 100644 --- a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_drv.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -22,6 +23,7 @@ =20 #include "mlxbf_pka_dev.h" #include "mlxbf_pka_ring.h" +#include "mlxbf_pka_trng.h" =20 #define MLXBF_PKA_DRIVER_DESCRIPTION "BlueField PKA driver" =20 @@ -187,6 +189,7 @@ struct mlxbf_pka_device { u32 device_id; struct resource *resource[MLXBF_PKA_DEVICE_RES_CNT]; struct mlxbf_pka_dev_shim_s *shim; + struct hwrng rng; }; =20 static int mlxbf_pka_drv_verify_bootup_status(struct device *dev) @@ -435,6 +438,44 @@ static long mlxbf_pka_drv_ring_ioctl(void *device_data= , unsigned int cmd, unsign =20 } else if (cmd =3D=3D MLXBF_PKA_CLEAR_RING_COUNTERS) { return mlxbf_pka_dev_clear_ring_counters(ring_dev->ring); + } else if (cmd =3D=3D MLXBF_PKA_GET_RANDOM_BYTES) { + struct mlxbf_pka_dev_trng_info trng_data; + struct mlxbf_pka_dev_shim_s *shim; + bool trng_present; + u32 byte_cnt; + int ret; + + shim =3D ring_dev->ring->shim; + ret =3D copy_from_user(&trng_data, + (void __user *)(arg), + sizeof(struct mlxbf_pka_dev_trng_info)); + if (ret) { + dev_dbg(ring_dev->device, "failed to copy user request.\n"); + return -EFAULT; + } + + /* + * Need byte count which is multiple of 4 as required by the + * mlxbf_pka_dev_trng_read() interface. + */ + byte_cnt =3D round_up(trng_data.count, MLXBF_PKA_TRNG_OUTPUT_CNT); + + u32 *data __free(kfree) =3D kzalloc(byte_cnt, GFP_KERNEL); + if (!data) + return -ENOMEM; + + trng_present =3D mlxbf_pka_dev_has_trng(shim); + if (!trng_present) + return -EAGAIN; + + ret =3D mlxbf_pka_dev_trng_read(ring_dev->device, shim, data, byte_cnt); + if (ret) { + dev_dbg(ring_dev->device, "TRNG failed %d\n", ret); + return ret; + } + + ret =3D copy_to_user((void __user *)(trng_data.data), data, trng_data.co= unt); + return ret ? -EFAULT : 0; } =20 return -ENOTTY; @@ -659,6 +700,23 @@ static void mlxbf_pka_drv_unregister_ring_device(struc= t mlxbf_pka_ring_device *r mlxbf_pka_dev_unregister_ring(ring_dev->device, ring_dev->ring); } =20 +static int mlxbf_pka_drv_rng_read(struct hwrng *rng, void *data, size_t ma= x, bool wait) +{ + struct mlxbf_pka_device *mlxbf_pka_dev =3D container_of(rng, struct mlxbf= _pka_device, rng); + u32 *buffer =3D data; + int ret; + + ret =3D mlxbf_pka_dev_trng_read(mlxbf_pka_dev->device, mlxbf_pka_dev->shi= m, buffer, max); + if (ret) { + dev_dbg(mlxbf_pka_dev->device, + "%s: failed to read random bytes ret=3D%d", + rng->name, ret); + return 0; + } + + return max; +} + static int mlxbf_pka_drv_probe_device(struct mlxbf_pka_info *info) { struct mlxbf_pka_drv_plat_info *plat_info; @@ -667,6 +725,7 @@ static int mlxbf_pka_drv_probe_device(struct mlxbf_pka_= info *info) const struct acpi_device_id *aid; struct platform_device *pdev; u64 wndw_ram_off_mask; + struct hwrng *trng; struct device *dev; int ret; =20 @@ -727,6 +786,19 @@ static int mlxbf_pka_drv_probe_device(struct mlxbf_pka= _info *info) } } =20 + /* Setup the TRNG if needed. */ + if (mlxbf_pka_dev_has_trng(mlxbf_pka_dev->shim)) { + trng =3D &mlxbf_pka_dev->rng; + trng->name =3D pdev->name; + trng->read =3D mlxbf_pka_drv_rng_read; + + ret =3D hwrng_register(&mlxbf_pka_dev->rng); + if (ret) { + dev_err(dev, "failed to register trng\n"); + return ret; + } + } + info->priv =3D mlxbf_pka_dev; =20 return 0; @@ -741,6 +813,9 @@ static void mlxbf_pka_drv_remove_device(struct platform= _device *pdev) if (!mlxbf_pka_dev) return; =20 + if (mlxbf_pka_dev_has_trng(mlxbf_pka_dev->shim)) + hwrng_unregister(&mlxbf_pka_dev->rng); + mlxbf_pka_drv_unregister_device(mlxbf_pka_dev); } =20 diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_trng.c b/drivers= /platform/mellanox/mlxbf_pka/mlxbf_pka_trng.c new file mode 100644 index 000000000000..0a0007fb9e31 --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_trng.c @@ -0,0 +1,874 @@ +// SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause +// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. + +#include +#include +#include + +#include "mlxbf_pka_dev.h" +#include "mlxbf_pka_ring.h" +#include "mlxbf_pka_trng.h" + +/* Personalization string "NVIDIA-MELLANOX-BLUEFIELD-TRUE_RANDOM_NUMBER_GE= N". */ +static u32 mlxbf_pka_trng_drbg_ps_str[] =3D { + 0x4e564944, 0x49412d4d, 0x454c4c41, 0x4e4f582d, + 0x424c5545, 0x4649454c, 0x442d5452, 0x55455f52, + 0x414e444f, 0x4d5f4e55, 0x4d424552, 0x5f47454e +}; + +/* Personalization string for DRBG test. */ +static u32 mlxbf_pka_trng_drbg_test_ps_str[] =3D { + 0x64299d83, 0xc34d7098, 0x5bd1f51d, 0xddccfdc1, + 0xdd0455b7, 0x166279e5, 0x0974cb1b, 0x2f2cd100, + 0x59a5060a, 0xca79940d, 0xd4e29a40, 0x56b7b779 +}; + +/* First Entropy string for DRBG test. */ +static u32 mlxbf_pka_trng_drbg_test_etpy_str1[] =3D { + 0xaa6bbcab, 0xef45e339, 0x136ca1e7, 0xbce1c881, + 0x9fa37b09, 0x63b53667, 0xb36e0053, 0xa202ed81, + 0x4650d90d, 0x8eed6127, 0x666f2402, 0x0dfd3af9 +}; + +/* Second Entropy string for DRBG test. */ +static u32 mlxbf_pka_trng_drbg_test_etpy_str2[] =3D { + 0x35c1b7a1, 0x0154c52b, 0xd5777390, 0x226a4fdb, + 0x5f16080d, 0x06b68369, 0xd0c93d00, 0x3336e27f, + 0x1abf2c37, 0xe6ab006c, 0xa4adc6e1, 0x8e1907a2 +}; + +/* Known answer for DRBG test. */ +static u32 mlxbf_pka_trng_drbg_test_output[] =3D { + 0xb663b9f1, 0x24943e13, 0x80f7dce5, 0xaba1a16f +}; + +/* Known answer for poker test. */ +static u64 poker_test_exp_cnt[] =3D { + 0x20f42bf4, 0xaf415f4, 0xf4f4fff4, 0xfff4f4f4 +}; + +static int mlxbf_pka_dev_config_trng_clk(struct device *dev, + struct mlxbf_pka_dev_res_t *aic_csr_ptr) +{ + u32 trng_clk_en =3D 0; + void __iomem *csr_reg_ptr; + u64 csr_reg_base; + u64 csr_reg_off; + u64 timer; + + if (aic_csr_ptr->status !=3D MLXBF_PKA_DEV_RES_STATUS_MAPPED || + aic_csr_ptr->type !=3D MLXBF_PKA_DEV_RES_TYPE_REG) + return -EPERM; + + dev_dbg(dev, "turn on TRNG clock\n"); + + csr_reg_base =3D aic_csr_ptr->base; + csr_reg_ptr =3D aic_csr_ptr->ioaddr; + + /* + * Enable the TRNG clock in MLXBF_PKA_CLK_FORCE. In general, this + * register should be left in its default state of all zeroes. Only + * when the TRNG is directly controlled via the Host slave interface, + * the engine needs to be turned on using the 'trng_clk_on' bit in + * this register. In case the TRNG is controlled via internal firmware, + * this is not required. + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _CLK_FORCE_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_CLK_FORCE_TRNG= _ON); + /* + * Check whether the system clock for TRNG engine is enabled. The clock + * MUST be running to provide access to the TRNG. + */ + timer =3D mlxbf_pka_dev_timer_start_msec(100); + while (!trng_clk_en) { + trng_clk_en |=3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off) + & MLXBF_PKA_CLK_FORCE_TRNG_ON; + if (mlxbf_pka_dev_timer_done(timer)) { + dev_dbg(dev, "failed to enable TRNG clock\n"); + return -ETIMEDOUT; + } + } + dev_dbg(dev, "trng_clk_on is enabled\n"); + + return 0; +} + +static bool mlxbf_pka_dev_trng_wait_test_ready(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base) +{ + u64 csr_reg_off, timer, csr_reg_val, test_ready =3D 0; + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_STATUS_ADDR); + timer =3D mlxbf_pka_dev_timer_start_msec(MSEC_PER_SEC); + + while (!test_ready) { + csr_reg_val =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + test_ready =3D csr_reg_val & MLXBF_PKA_TRNG_STATUS_TEST_READY; + + if (mlxbf_pka_dev_timer_done(timer)) { + dev_dbg(dev, "TRNG test ready timer done, 0x%llx\n", csr_reg_val); + return false; + } + } + + return true; +} + +static int mlxbf_pka_dev_trng_enable_test(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base, u32 test) +{ + u64 csr_reg_val, csr_reg_off; + int ret; + + /* + * Set the 'test_mode' bit in the TRNG_CONTROL register and the + * 'test_known_noise' bit in the TRNG_TEST register - this will + * immediately set the 'test_ready' bit (in the TRNG_STATUS register) + * to indicate that data can be written. It will also reset the + * 'monobit test', 'run test' and 'poker test' circuits to their + * initial states. Note that the TRNG need not be enabled for this + * test. + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + csr_reg_val =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, + csr_reg_val | MLXBF_PKA_TRNG_CONTROL_TEST_MODE); + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_TEST_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, test); + /* Wait until the 'test_ready' bit is set. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_STATUS_ADDR); + ret =3D read_poll_timeout(mlxbf_pka_dev_io_read, csr_reg_val, + csr_reg_val & MLXBF_PKA_TRNG_STATUS_TEST_READY, + USEC_PER_MSEC / 100, USEC_PER_SEC, false, + csr_reg_ptr, csr_reg_off); + if (ret) { + dev_err(dev, "timeout waiting for test ready\n"); + return -ETIMEDOUT; + } + + /* + * Check whether the 'monobit test', 'run test' and 'poker test' + * are reset. + */ + if (csr_reg_val & MLXBF_PKA_TRNG_STATUS_FAIL_MODES) { + dev_err(dev, "test bits aren't reset, TRNG_STATUS:0x%llx\n", csr_reg_val= ); + return -EIO; + } + + /* + * Set 'stall_run_poker' bit to allow inspecting the state of the + * result counters which would otherwise be reset immediately for the + * next 20,000 bits block to test. + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_ALARMCNT_ADDR); + csr_reg_val =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + mlxbf_pka_dev_io_write(csr_reg_ptr, + csr_reg_off, + csr_reg_val | MLXBF_PKA_TRNG_ALARMCNT_STALL_RUN_POKER); + + return 0; +} + +static int mlxbf_pka_dev_trng_test_circuits(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base, + u64 datal, u64 datah, + int count, u8 add_half, + u64 *monobit_fail_cnt, + u64 *run_fail_cnt, + u64 *poker_fail_cnt) +{ + u64 status, csr_reg_off; + unsigned int test_idx; + int ret; + + if (!monobit_fail_cnt || !run_fail_cnt || !poker_fail_cnt) + return -EINVAL; + + for (test_idx =3D 0; test_idx < count; test_idx++) { + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_RAW_L_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, datal); + + if (!add_half || test_idx < count - 1) { + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_RAW_H_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, datah); + } + + /* + * Wait until the 'test_ready' bit in the TRNG_STATUS register + * becomes '1' again, signalling readiness for the next 64 bits + * of test data. At this point, the previous test data has been + * handled so the counter states can be inspected. + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_STATUS_ADDR); + ret =3D read_poll_timeout(mlxbf_pka_dev_io_read, status, + status & MLXBF_PKA_TRNG_STATUS_TEST_READY, + USEC_PER_MSEC / 100, USEC_PER_SEC, false, + csr_reg_ptr, csr_reg_off); + if (ret) { + dev_err(dev, "timeout waiting for test ready in circuits\n"); + return -ETIMEDOUT; + } + + /* Check test status bits. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_INTACK_ADDR); + if (status & MLXBF_PKA_TRNG_STATUS_MONOBIT_FAIL) { + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, + MLXBF_PKA_TRNG_STATUS_MONOBIT_FAIL); + *monobit_fail_cnt +=3D 1; + } else if (status & MLXBF_PKA_TRNG_STATUS_RUN_FAIL) { + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, + MLXBF_PKA_TRNG_STATUS_RUN_FAIL); + *run_fail_cnt +=3D 1; + } else if (status & MLXBF_PKA_TRNG_STATUS_POKER_FAIL) { + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, + MLXBF_PKA_TRNG_STATUS_POKER_FAIL); + *poker_fail_cnt +=3D 1; + } + } + + return *monobit_fail_cnt || *poker_fail_cnt || *run_fail_cnt ? -EIO : 0; +} + +static void mlxbf_pka_dev_trng_disable_test(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base) +{ + u64 status, val, csr_reg_off; + + /* + * When done, clear the 'test_known_noise' bit in the TRNG_TEST + * register (will immediately clear the 'test_ready' bit in the + * TRNG_STATUS register and reset the 'monobit test', 'run test' + * and 'poker test' circuits) and clear the 'test_mode' bit in the + * TRNG_CONTROL register. + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_TEST_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_STATUS_ADDR); + status =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + + if (status & MLXBF_PKA_TRNG_STATUS_TEST_READY) + dev_info(dev, "test ready bit is still set\n"); + + if (status & MLXBF_PKA_TRNG_STATUS_FAIL_MODES) + dev_info(dev, "test bits are still set, TRNG_STATUS:0x%llx\n", status); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + val =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, val & ~MLXBF_PKA_TRNG_ST= ATUS_TEST_READY); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_ALARMCNT_ADDR); + val =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + mlxbf_pka_dev_io_write(csr_reg_ptr, + csr_reg_off, + val & ~MLXBF_PKA_TRNG_ALARMCNT_STALL_RUN_POKER); +} + +static int mlxbf_pka_dev_trng_test_known_answer_basic(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base) +{ + u64 poker_cnt[MLXBF_PKA_TRNG_POKER_TEST_CNT]; + u64 monobit_fail_cnt =3D 0; + u64 poker_fail_cnt =3D 0; + u64 run_fail_cnt =3D 0; + u64 monobit_cnt; + u64 csr_reg_off; + int cnt_idx; + int cnt_off; + int ret; + + dev_dbg(dev, "run known-answer test circuits\n"); + + ret =3D mlxbf_pka_dev_trng_enable_test(dev, csr_reg_ptr, csr_reg_base, + MLXBF_PKA_TRNG_TEST_KNOWN_NOISE); + if (ret) + return ret; + + ret =3D mlxbf_pka_dev_trng_test_circuits(dev, + csr_reg_ptr, + csr_reg_base, + MLXBF_PKA_TRNG_TEST_DATAL_BASIC_1, + MLXBF_PKA_TRNG_TEST_DATAH_BASIC_1, + MLXBF_PKA_TRNG_TEST_COUNT_BASIC_1, + MLXBF_PKA_TRNG_TEST_HALF_NO, + &monobit_fail_cnt, + &run_fail_cnt, + &poker_fail_cnt); + + ret |=3D mlxbf_pka_dev_trng_test_circuits(dev, + csr_reg_ptr, + csr_reg_base, + MLXBF_PKA_TRNG_TEST_DATAL_BASIC_2, + MLXBF_PKA_TRNG_TEST_DATAH_BASIC_2, + MLXBF_PKA_TRNG_TEST_COUNT_BASIC_2, + MLXBF_PKA_TRNG_TEST_HALF_ADD, + &monobit_fail_cnt, + &run_fail_cnt, + &poker_fail_cnt); + + dev_dbg(dev, "monobit_fail_cnt : 0x%llx\n", monobit_fail_cnt); + dev_dbg(dev, "poker_fail_cnt : 0x%llx\n", poker_fail_cnt); + dev_dbg(dev, "run_fail_cnt : 0x%llx\n", run_fail_cnt); + + for (cnt_idx =3D 0, cnt_off =3D 0; + cnt_idx < MLXBF_PKA_TRNG_POKER_TEST_CNT; + cnt_idx++, cnt_off +=3D 8) { + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_POKER_3_0_ADDR + + cnt_off); + poker_cnt[cnt_idx] =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + } + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_MONOBITCNT_ADDR); + monobit_cnt =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + + if (ret) + goto exit; + + if (memcmp(poker_cnt, poker_test_exp_cnt, sizeof(poker_test_exp_cnt))) { + dev_dbg(dev, "invalid poker counters!\n"); + ret =3D -EIO; + goto exit; + } + + if (monobit_cnt !=3D MLXBF_PKA_TRNG_MONOBITCNT_SUM) { + dev_dbg(dev, "invalid sum of squares!\n"); + ret =3D -EIO; + goto exit; + } + +exit: + mlxbf_pka_dev_trng_disable_test(dev, csr_reg_ptr, csr_reg_base); + return ret; +} + +static int mlxbf_pka_dev_trng_test_known_answer_poker_fail(struct device *= dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base) +{ + u64 monobit_fail_cnt =3D 0; + u64 poker_fail_cnt =3D 0; + u64 run_fail_cnt =3D 0; + + dev_dbg(dev, "run known-answer test circuits (poker fail)\n"); + + mlxbf_pka_dev_trng_enable_test(dev, csr_reg_ptr, csr_reg_base, + MLXBF_PKA_TRNG_TEST_KNOWN_NOISE); + + /* + * Ignore the return value here as it is expected that poker test + * should fail. Check failure counts thereafter to assert only poker + * test has failed. + */ + mlxbf_pka_dev_trng_test_circuits(dev, + csr_reg_ptr, + csr_reg_base, + MLXBF_PKA_TRNG_TEST_DATAL_POKER, + MLXBF_PKA_TRNG_TEST_DATAH_POKER, + MLXBF_PKA_TRNG_TEST_COUNT_POKER, + MLXBF_PKA_TRNG_TEST_HALF_NO, + &monobit_fail_cnt, + &run_fail_cnt, + &poker_fail_cnt); + + dev_dbg(dev, "monobit_fail_cnt : 0x%llx\n", monobit_fail_cnt); + dev_dbg(dev, "poker_fail_cnt : 0x%llx\n", poker_fail_cnt); + dev_dbg(dev, "run_fail_cnt : 0x%llx\n", run_fail_cnt); + + mlxbf_pka_dev_trng_disable_test(dev, csr_reg_ptr, csr_reg_base); + + return poker_fail_cnt && !run_fail_cnt && !monobit_fail_cnt ? 0 : -EIO; +} + +static int mlxbf_pka_dev_trng_test_unknown_answer(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base) +{ + u64 datal =3D 0, datah =3D 0, csr_reg_off; + int ret =3D 0, test_idx; + + dev_dbg(dev, "run unknown-answer self test\n"); + + /* First reset, the RAW registers. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_RAW_L_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_RAW_H_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + /* + * There is a small probability for this test to fail. So run the test + * 10 times, if it succeeds once then assume that the test passed. + */ + for (test_idx =3D 0; test_idx < 10; test_idx++) { + mlxbf_pka_dev_trng_enable_test(dev, csr_reg_ptr, csr_reg_base, + MLXBF_PKA_TRNG_TEST_NOISE); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_RAW_L_ADDR); + datal =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_RAW_H_ADDR); + datah =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + + dev_dbg(dev, "datal=3D0x%llx\n", datal); + dev_dbg(dev, "datah=3D0x%llx\n", datah); + + mlxbf_pka_dev_trng_disable_test(dev, csr_reg_ptr, csr_reg_base); + + if (!datah && !datal) + ret =3D -EIO; + else + return 0; + } + return ret; +} + +/* Test TRNG. */ +static int mlxbf_pka_dev_test_trng(struct device *dev, void __iomem *csr_r= eg_ptr, u64 csr_reg_base) +{ + int ret; + + ret =3D mlxbf_pka_dev_trng_test_known_answer_basic(dev, csr_reg_ptr, csr_= reg_base); + if (ret) + return ret; + + ret =3D mlxbf_pka_dev_trng_test_known_answer_poker_fail(dev, csr_reg_ptr,= csr_reg_base); + if (ret) + return ret; + + ret =3D mlxbf_pka_dev_trng_test_unknown_answer(dev, csr_reg_ptr, csr_reg_= base); + if (ret) + return ret; + + return ret; +} + +static void mlxbf_pka_dev_trng_write_ps_ai_str(void __iomem *csr_reg_ptr, + u64 csr_reg_base, + u32 input_str[]) +{ + u64 csr_reg_off; + u8 i; + + for (i =3D 0; i < MLXBF_PKA_TRNG_PS_AI_REG_COUNT; i++) { + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_PS_AI_0_ADDR + i * + MLXBF_PKA_TRNG_OUTPUT_REG_OFFSET); + + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, input_str[i]); + } +} + +static void mlxbf_pka_dev_trng_drbg_generate(void __iomem *csr_reg_ptr, u6= 4 csr_reg_base) +{ + u64 csr_reg_off; + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_CONTROL_R= EQ_DATA); +} + +static int mlxbf_pka_dev_test_trng_drbg(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base) +{ + u64 csr_reg_off, csr_reg_val; + u8 i; + + /* Make sure the engine is idle. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + /* Enable DRBG, TRNG need not be enabled for this test. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_CONTROL_D= RBG_ENABLE); + + /* Set 'test_sp_800_90' bit in the TRNG_TEST register. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_TEST_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_TEST_DRBG= ); + + /* Wait for 'test_ready' bit to be set. */ + if (!mlxbf_pka_dev_trng_wait_test_ready(dev, csr_reg_ptr, csr_reg_base)) + return -ETIMEDOUT; + + /* 'Instantiate' function. */ + mlxbf_pka_dev_trng_write_ps_ai_str(csr_reg_ptr, + csr_reg_base, + mlxbf_pka_trng_drbg_test_ps_str); + if (!mlxbf_pka_dev_trng_wait_test_ready(dev, csr_reg_ptr, csr_reg_base)) + return -ETIMEDOUT; + + /* 'Generate' function. */ + mlxbf_pka_dev_trng_write_ps_ai_str(csr_reg_ptr, + csr_reg_base, + mlxbf_pka_trng_drbg_test_etpy_str1); + if (!mlxbf_pka_dev_trng_wait_test_ready(dev, csr_reg_ptr, csr_reg_base)) + return -ETIMEDOUT; + + /* + * A standard NIST SP 800-90A DRBG known-answer test discards the + * result of the first 'Generate' function and only checks the result + * of the second 'Generate' function. Hence 'Generate' is performed + * again. + */ + + /* 'Generate' function. */ + mlxbf_pka_dev_trng_write_ps_ai_str(csr_reg_ptr, + csr_reg_base, + mlxbf_pka_trng_drbg_test_etpy_str2); + if (!mlxbf_pka_dev_trng_wait_test_ready(dev, csr_reg_ptr, csr_reg_base)) + return -ETIMEDOUT; + + /* Check output registers. */ + for (i =3D 0; i < MLXBF_PKA_TRNG_OUTPUT_CNT; i++) { + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_OUTPUT_0_ADDR + + (i * MLXBF_PKA_TRNG_OUTPUT_REG_OFFSET)); + + csr_reg_val =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + + if ((u32)csr_reg_val !=3D mlxbf_pka_trng_drbg_test_output[i]) { + dev_dbg(dev, "DRBG known answer test failed: output register:%d, 0x%x\n= ", + i, (u32)csr_reg_val); + return -EIO; + } + } + + /* Clear 'test_sp_800_90' bit in the TRNG_TEST register. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_TEST_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + return 0; +} + +static bool mlxbf_pka_dev_trng_shutdown_oflo(struct mlxbf_pka_dev_res_t *t= rng_csr_ptr, + u64 *err_cycle) +{ + u64 curr_cycle_cnt, fro_stopped_mask, fro_enabled_mask; + u64 csr_reg_base, csr_reg_off, csr_reg_value; + void __iomem *csr_reg_ptr; + + csr_reg_base =3D trng_csr_ptr->base; + csr_reg_ptr =3D trng_csr_ptr->ioaddr; + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_STATUS_ADDR); + csr_reg_value =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + + if (csr_reg_value & MLXBF_PKA_TRNG_STATUS_SHUTDOWN_OFLO) { + curr_cycle_cnt =3D get_cycles(); + /* + * See if any FROs were shut down. If they were, toggle bits in + * the FRO detune register and reenable the FROs. + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_ALARMSTOP_ADDR); + fro_stopped_mask =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + if (fro_stopped_mask) { + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_FROENABLE_ADDR); + fro_enabled_mask =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_FRODETUNE_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, fro_stopped_mask); + + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_FROENABLE_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, + fro_stopped_mask | fro_enabled_mask); + } + + /* Reset the error. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_ALARMMASK_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_ALARMSTOP_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_INTACK_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, + csr_reg_off, + MLXBF_PKA_TRNG_STATUS_SHUTDOWN_OFLO); + + /* + * If this error occurs again within about a second, the hardware + * is malfunctioning. Disable the trng and return an error. + */ + if (*err_cycle && + (curr_cycle_cnt - *err_cycle < MLXBF_PKA_TRNG_TEST_ERR_CYCLE_MAX)) { + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_CONTROL_ADDR); + csr_reg_value =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + csr_reg_value &=3D ~MLXBF_PKA_TRNG_CONTROL; + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, csr_reg_value); + return false; + } + + *err_cycle =3D curr_cycle_cnt; + } + + return true; +} + +static int mlxbf_pka_dev_trng_drbg_reseed(struct device *dev, + void __iomem *csr_reg_ptr, + u64 csr_reg_base) +{ + u64 csr_reg_off; + + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_CONTROL_D= RBG_RESEED); + + if (!mlxbf_pka_dev_trng_wait_test_ready(dev, csr_reg_ptr, csr_reg_base)) + return -ETIMEDOUT; + + /* Write personalization string. */ + mlxbf_pka_dev_trng_write_ps_ai_str(csr_reg_ptr, csr_reg_base, mlxbf_pka_t= rng_drbg_ps_str); + + return 0; +} + +/* Configure the TRNG. */ +int mlxbf_pka_dev_config_trng_drbg(struct device *dev, + struct mlxbf_pka_dev_res_t *aic_csr_ptr, + struct mlxbf_pka_dev_res_t *trng_csr_ptr) +{ + u64 csr_reg_base, csr_reg_off; + void __iomem *csr_reg_ptr; + int ret; + + if (trng_csr_ptr->status !=3D MLXBF_PKA_DEV_RES_STATUS_MAPPED || + trng_csr_ptr->type !=3D MLXBF_PKA_DEV_RES_TYPE_REG) + return -EPERM; + + dev_dbg(dev, "starting up the TRNG\n"); + + ret =3D mlxbf_pka_dev_config_trng_clk(dev, aic_csr_ptr); + if (ret) + return ret; + + csr_reg_base =3D trng_csr_ptr->base; + csr_reg_ptr =3D trng_csr_ptr->ioaddr; + + /* + * Perform NIST known-answer tests on the complete SP 800-90A DRBG + * without BC_DF functionality. + */ + ret =3D mlxbf_pka_dev_test_trng_drbg(dev, csr_reg_ptr, csr_reg_base); + if (ret) + return ret; + + /* Starting up the TRNG with a DRBG. */ + + /* Make sure the engine is idle. */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + /* Disable all FROs initially. */ + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA_TRNG_FROENABLE_= ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA_TRNG_FRODETUNE_= ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + /* + * Write all configuration values in the TRNG_CONFIG and TRNG_ALARMCNT, + * write zeroes to the TRNG_ALARMMASK and TRNG_ALARMSTOP registers. + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONFIG_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_CONFIG); + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_ALARMCNT_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_ALARMCNT); + + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA_TRNG_ALARMMASK_= ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA_TRNG_ALARMSTOP_= ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, 0); + + /* + * Enable all FROs in the TRNG_FROENABLE register. Note that this can + * only be done after clearing the TRNG_ALARMSTOP register. + */ + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA_TRNG_FROENABLE_= ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_FROENABLE= ); + + /* + * Optionally, write 'Personalization string' of up to 384 bits in + * TRNG_PS_AI_xxx registers. + * The contents of these registers will be XOR-ed into the output of the + * SHA-256 'Conditioning Function' to be used as seed value for the + * actual DRBG. + */ + mlxbf_pka_dev_trng_write_ps_ai_str(csr_reg_ptr, csr_reg_base, mlxbf_pka_t= rng_drbg_ps_str); + + /* + * Run TRNG tests after configuring TRNG. + * NOTE: TRNG need not be enabled to carry out these tests. + */ + ret =3D mlxbf_pka_dev_test_trng(dev, csr_reg_ptr, csr_reg_base); + if (ret) + return ret; + + /* + * Start the actual engine by setting the 'enable_trng' and 'drbg_en' + * bit in the TRNG_CONTROL register (also a nice point to set the + * interrupt mask bits). + */ + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA= _TRNG_CONTROL_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, MLXBF_PKA_TRNG_CONTROL_D= RBG); + + /* + * The engine is now ready to handle the first 'Generate' request using + * the 'request_data' bit of the TRNG_CONTROL register. The first output + * for these requests will take a while, as Noise Source and + * Conditioning Function must first generate seed entropy for the DRBG. + * + * Optionally, when buffer RAM is configured: Set a data available + * interrupt threshold using the 'load_thresh' and 'blocks_thresh' + * fields of the TRNG_INTACK register. This allows delaying the data + * available interrupt until the indicated number of 128-bit words a + * available in the buffer RAM. + * + * Start the actual 'Generate' operation using the 'request_data' and + * 'data_blocks' fields of the TRNG_CONTROL register. + */ + mlxbf_pka_dev_trng_drbg_generate(csr_reg_ptr, csr_reg_base); + + /* Delay 200 ms. */ + mdelay(200); + + return 0; +} + +/* Read from DRBG enabled TRNG. */ +int mlxbf_pka_dev_trng_read(struct device *dev, + struct mlxbf_pka_dev_shim_s *shim, + u32 *data, u32 cnt) +{ + u64 csr_reg_base, csr_reg_off, csr_reg_value, timer; + struct mlxbf_pka_dev_res_t *trng_csr_ptr; + u8 output_idx, trng_ready =3D 0; + u32 data_idx, word_cnt; + void __iomem *csr_reg_ptr; + int ret =3D 0; + + if (!shim || !data || (cnt % MLXBF_PKA_TRNG_OUTPUT_CNT !=3D 0)) + return -EINVAL; + + if (!cnt) + return ret; + + guard(mutex)(&shim->mutex); + + trng_csr_ptr =3D &shim->resources.trng_csr; + + if (trng_csr_ptr->status !=3D MLXBF_PKA_DEV_RES_STATUS_MAPPED || + trng_csr_ptr->type !=3D MLXBF_PKA_DEV_RES_TYPE_REG) + return -EPERM; + + csr_reg_base =3D trng_csr_ptr->base; + csr_reg_ptr =3D trng_csr_ptr->ioaddr; + + if (!mlxbf_pka_dev_trng_shutdown_oflo(trng_csr_ptr, &shim->trng_err_cycle= )) + return -EWOULDBLOCK; + + word_cnt =3D cnt >> ilog2(sizeof(u32)); + + for (data_idx =3D 0; data_idx < word_cnt; data_idx++) { + output_idx =3D data_idx % MLXBF_PKA_TRNG_OUTPUT_CNT; + + /* Tell the hardware to advance. */ + if (!output_idx) { + csr_reg_off =3D mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_INTACK_ADDR); + mlxbf_pka_dev_io_write(csr_reg_ptr, csr_reg_off, + MLXBF_PKA_TRNG_STATUS_READY); + trng_ready =3D 0; + + /* + * Check if 'data_blocks' field is zero in TRNG_CONTROL + * register. If it is zero, need to issue a 'Reseed and + * Generate' request for DRBG enabled TRNG. + */ + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_CONTROL_ADDR); + csr_reg_value =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + + if (!((u32)csr_reg_value & MLXBF_PKA_TRNG_DRBG_DATA_BLOCK_MASK)) { + /* Issue reseed. */ + ret =3D mlxbf_pka_dev_trng_drbg_reseed(dev, + csr_reg_ptr, + csr_reg_base); + if (ret) + return -EBUSY; + + /* Issue generate request. */ + mlxbf_pka_dev_trng_drbg_generate(csr_reg_ptr, csr_reg_base); + } + } + + /* + * Wait until a data word is available in the TRNG_OUTPUT_X + * registers, using the interrupt and/or 'ready' status bit + * in the TRNG_STATUS register. The only way this would hang + * is if the TRNG is never initialized. This function cannot + * be called if that happened. + */ + timer =3D mlxbf_pka_dev_timer_start_msec(1000); + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, MLXBF_PKA_TRNG_STATUS_AD= DR); + while (!trng_ready) { + csr_reg_value =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + trng_ready =3D csr_reg_value & MLXBF_PKA_TRNG_STATUS_READY; + + if (mlxbf_pka_dev_timer_done(timer)) { + dev_dbg(dev, "shim %u got error obtaining random number\n", + shim->shim_id); + return -EBUSY; + } + } + + /* Read the registers. */ + csr_reg_off =3D + mlxbf_pka_dev_get_register_offset(csr_reg_base, + MLXBF_PKA_TRNG_OUTPUT_0_ADDR + + (output_idx * MLXBF_PKA_TRNG_OUTPUT_REG_OFFSET)); + csr_reg_value =3D mlxbf_pka_dev_io_read(csr_reg_ptr, csr_reg_off); + data[data_idx] =3D (u32)csr_reg_value; + } + + return ret; +} + +bool mlxbf_pka_dev_has_trng(struct mlxbf_pka_dev_shim_s *shim) +{ + if (!shim) + return false; + + return shim->trng_enabled =3D=3D MLXBF_PKA_SHIM_TRNG_ENABLED; +} diff --git a/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_trng.h b/drivers= /platform/mellanox/mlxbf_pka/mlxbf_pka_trng.h new file mode 100644 index 000000000000..97ed6045781e --- /dev/null +++ b/drivers/platform/mellanox/mlxbf_pka/mlxbf_pka_trng.h @@ -0,0 +1,152 @@ +/* SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause */ +/* SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION. All righ= ts reserved. */ + +#ifndef __MLXBF_PKA_TRNG_H__ +#define __MLXBF_PKA_TRNG_H__ + +/* + * The True Random Number Generator CSR addresses/offsets. These are acces= sed + * from the ARM as 8 byte reads/writes. However only the bottom 32 bits are + * implemented. + */ +#define MLXBF_PKA_TRNG_OUTPUT_0_ADDR 0x12000 +#define MLXBF_PKA_TRNG_STATUS_ADDR 0x12020 +#define MLXBF_PKA_TRNG_INTACK_ADDR MLXBF_PKA_TRNG_STATUS_ADDR +#define MLXBF_PKA_TRNG_CONTROL_ADDR 0x12028 +#define MLXBF_PKA_TRNG_CONFIG_ADDR 0x12030 +#define MLXBF_PKA_TRNG_ALARMCNT_ADDR 0x12038 +#define MLXBF_PKA_TRNG_FROENABLE_ADDR 0x12040 +#define MLXBF_PKA_TRNG_FRODETUNE_ADDR 0x12048 +#define MLXBF_PKA_TRNG_ALARMMASK_ADDR 0x12050 +#define MLXBF_PKA_TRNG_ALARMSTOP_ADDR 0x12058 +#define MLXBF_PKA_TRNG_TEST_ADDR 0x120E0 +#define MLXBF_PKA_TRNG_RAW_L_ADDR 0x12060 +#define MLXBF_PKA_TRNG_RAW_H_ADDR 0x12068 +#define MLXBF_PKA_TRNG_MONOBITCNT_ADDR 0x120B8 +#define MLXBF_PKA_TRNG_POKER_3_0_ADDR 0x120C0 +#define MLXBF_PKA_TRNG_PS_AI_0_ADDR 0x12080 + +/* + * 'trng_clk_on' mask for PKA Clock Switch Forcing Register. Turn on the T= RNG + * clock. When the TRNG is controlled via the host slave interface, this e= ngine + * needs to be turned on by setting bit 11. + */ +#define MLXBF_PKA_CLK_FORCE_TRNG_ON BIT(11) + +/* Number of TRNG output registers. */ +#define MLXBF_PKA_TRNG_OUTPUT_CNT 4 + +/* Number of TRNG poker test counts. */ +#define MLXBF_PKA_TRNG_POKER_TEST_CNT 4 + +/* TRNG configuration. */ +#define MLXBF_PKA_TRNG_CONFIG 0x00020008 +/* TRNG Alarm Counter Register value. */ +#define MLXBF_PKA_TRNG_ALARMCNT 0x000200ff +/* TRNG FRO Enable Register value. */ +#define MLXBF_PKA_TRNG_FROENABLE 0x00ffffff +/* + * TRNG Control Register value. Set bit 10 to start the EIP-76 (i.e. TRNG + * engine), gathering entropy from the Free Running Oscillators (FROs). + */ +#define MLXBF_PKA_TRNG_CONTROL 0x00000400 + +/* TRNG Control bit. */ +#define MLXBF_PKA_TRNG_CONTROL_TEST_MODE BIT(8) + +/* + * TRNG Control Register value. Set bit 10 and 12 to start the EIP-76 (i.e. + * TRNG engine) with DRBG enabled, gathering entropy from the FROs. + */ +#define MLXBF_PKA_TRNG_CONTROL_DRBG 0x00001400 + +/* + * DRBG enabled TRNG 'request_data' value. REQ_DATA (in accordance with + * DATA_BLOCK_MASK) requests 256 blocks of 128-bit random output. 4095 blo= cks + * is the maximum number that can be requested for the TRNG (with DRBG) + * configuration on Bluefield platforms. + */ +#define MLXBF_PKA_TRNG_CONTROL_REQ_DATA 0x10010000 + +/* Mask for 'Data Block' in TRNG Control Register. */ +#define MLXBF_PKA_TRNG_DRBG_DATA_BLOCK_MASK GENMASK(31, 20) + +/* Set bit 12 of TRNG Control Register to enable DRBG functionality. */ +#define MLXBF_PKA_TRNG_CONTROL_DRBG_ENABLE BIT(12) + +/* Set bit 7 (i.e. 'test_sp_800_90 DRBG' bit) in the TRNG Test Register. */ +#define MLXBF_PKA_TRNG_TEST_DRBG BIT(7) + +/* Number of Personalization String/Additional Input Registers. */ +#define MLXBF_PKA_TRNG_PS_AI_REG_COUNT 12 + +/* Offset bytes of Personalization String/Additional Input Registers. */ +#define MLXBF_PKA_TRNG_OUTPUT_REG_OFFSET 0x8 + +/* Maximum TRNG test error cycle, about one second. */ +#define MLXBF_PKA_TRNG_TEST_ERR_CYCLE_MAX (1000 * 1000 * 1000) + +/* DRBG Reseed enable. */ +#define MLXBF_PKA_TRNG_CONTROL_DRBG_RESEED BIT(15) + +/* TRNG Status bits. */ +#define MLXBF_PKA_TRNG_STATUS_READY BIT(0) +#define MLXBF_PKA_TRNG_STATUS_SHUTDOWN_OFLO BIT(1) +#define MLXBF_PKA_TRNG_STATUS_TEST_READY BIT(8) +#define MLXBF_PKA_TRNG_STATUS_MONOBIT_FAIL BIT(7) +#define MLXBF_PKA_TRNG_STATUS_RUN_FAIL BIT(4) +#define MLXBF_PKA_TRNG_STATUS_POKER_FAIL BIT(6) + +#define MLXBF_PKA_TRNG_STATUS_FAIL_MODES (MLXBF_PKA_TRNG_STATUS_MONOBIT_FA= IL | \ + MLXBF_PKA_TRNG_STATUS_RUN_FAIL | \ + MLXBF_PKA_TRNG_STATUS_POKER_FAIL) + +/* TRNG Alarm Counter bits. */ +#define MLXBF_PKA_TRNG_ALARMCNT_STALL_RUN_POKER BIT(15) + +/* TRNG Test bits. */ +#define MLXBF_PKA_TRNG_TEST_KNOWN_NOISE BIT(5) +#define MLXBF_PKA_TRNG_TEST_NOISE BIT(13) + +/* TRNG Test constants*/ +#define MLXBF_PKA_TRNG_MONOBITCNT_SUM 9978 + +#define MLXBF_PKA_TRNG_TEST_HALF_ADD 1 +#define MLXBF_PKA_TRNG_TEST_HALF_NO 0 + +#define MLXBF_PKA_TRNG_TEST_DATAL_BASIC_1 0x11111333 +#define MLXBF_PKA_TRNG_TEST_DATAH_BASIC_1 0x3555779f +#define MLXBF_PKA_TRNG_TEST_COUNT_BASIC_1 11 + +#define MLXBF_PKA_TRNG_TEST_DATAL_BASIC_2 0x01234567 +#define MLXBF_PKA_TRNG_TEST_DATAH_BASIC_2 0x89abcdef +#define MLXBF_PKA_TRNG_TEST_COUNT_BASIC_2 302 + +#define MLXBF_PKA_TRNG_TEST_DATAL_POKER 0xffffffff +#define MLXBF_PKA_TRNG_TEST_DATAH_POKER 0xffffffff +#define MLXBF_PKA_TRNG_TEST_COUNT_POKER 11 + +#define MLXBF_PKA_TRNG_NUM_OF_FOUR_WORD 128 + +/* Defines for mlxbf_pka_dev_shim->trng_enabled. */ +#define MLXBF_PKA_SHIM_TRNG_ENABLED 1 +#define MLXBF_PKA_SHIM_TRNG_DISABLED 0 + +/* Configure the TRNG. */ +int mlxbf_pka_dev_config_trng_drbg(struct device *dev, + struct mlxbf_pka_dev_res_t *aic_csr_ptr, + struct mlxbf_pka_dev_res_t *trng_csr_ptr); + +/* + * Read data from the TRNG. Drivers can fill up to 'cnt' bytes of data int= o the + * buffer 'data'. The buffer 'data' is aligned for any type and 'cnt' is a + * multiple of 4. + */ +int mlxbf_pka_dev_trng_read(struct device *dev, + struct mlxbf_pka_dev_shim_s *shim, + u32 *data, u32 cnt); + +/* Return true if the TRNG engine is enabled, false if not. */ +bool mlxbf_pka_dev_has_trng(struct mlxbf_pka_dev_shim_s *shim); + +#endif /* __MLXBF_PKA_TRNG_H__ */ diff --git a/include/uapi/linux/mlxbf-pka.h b/include/uapi/linux/mlxbf-pka.h index 9da6c1e5044b..2f2b11c2dacd 100644 --- a/include/uapi/linux/mlxbf-pka.h +++ b/include/uapi/linux/mlxbf-pka.h @@ -90,4 +90,23 @@ struct mlxbf_pka_dev_hw_ring_info { */ #define MLXBF_PKA_CLEAR_RING_COUNTERS _IO(MLXBF_PKA_IOC_TYPE, 0x2) =20 +/** + * struct mlxbf_pka_dev_trng_info - TRNG information + * @count: Number of random bytes in the buffer or length of the buffer + * @data: Data buffer to hold the random bytes + * + * MLXBF_PKA_GET_RANDOM_BYTES: + * _IOWR(MLXBF_PKA_IOC_TYPE, 0x3, mlxbf_pka_dev_trng_info). + * + * Get random bytes from True Random Number Generator(TRNG). + * + * Return: 0 on success, -errno on failure. + */ +struct mlxbf_pka_dev_trng_info { + __u32 count; + __u8 *data; +}; + +#define MLXBF_PKA_GET_RANDOM_BYTES _IOWR(MLXBF_PKA_IOC_TYPE, 0x3, struct m= lxbf_pka_dev_trng_info) + #endif /* _UAPI_LINUX_MLXBF_PKA_H */ --=20 2.34.1