From nobody Sat Feb 7 09:30:21 2026 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2056.outbound.protection.outlook.com [40.107.95.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4074C1BBBF7 for ; Mon, 3 Mar 2025 04:44:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.95.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740977049; cv=fail; b=pxcZcmGfIuLCYQHwepeGvar62wxHvd6G1E2wy4fz75jeSsC/iaml2sCGvUdz4/5owHJWHFa0tatE5BfokaTdhUYuHimNdDN+4yLkb8lmK8WluCXMQ0e1sUfjvj1PgbQKmNQV52bFaXrRMLLraPNXIYa7CcogA3Ha8ETlj1Vf7kM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740977049; c=relaxed/simple; bh=qsMk7XPHsuQhkmSxyjsRm+3Zhfbxdtbt4D29KZOe2xA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fUBxtLNGHm26G0ERx6zmCijJeoBuLd3GlrVnqZaEp8wA0X5Q8CWbScXotd5eGzzVIPZiff1vfkZVeP9ONRz+MTC2RDh54JaWZVxUb1C98UrEDV6xbaWgFNYOhNZBBf0PoHiq96e3X333opqfzpwBRQay9pwmduyiuypxjccl3OU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=mVtRmKLH; arc=fail smtp.client-ip=40.107.95.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="mVtRmKLH" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NkD+u4EWKmKLYyE+e4bgpqJV5nuqKXpvI7kQQt7uy3aFQ8ZPHhiOwD2+CdnqSi4QKTK4NOHmwsxOecCDRSMC9PwiWuOL4bfEwCgQDCyp73qWObMl+RFgj18g6Ie5eLHS2ATjL+A2b3oLoKhsd6PpsoY64HuaQUeMPXurPYSbYJRHh+JPnvhFBv2Jf0SyKZySA1bHy7knVFM6W1M/xSyRITp1X7dEuzYmVrsGugEtmr/0jl3S72ZLBTNG/Xuli0XM6psqGF7Hu6uWQE34WH2GouqWXtssooa89vaTnyJZ3mrpWykEJnKSnRdUDf5CguVI2+Uj/76EO4IUEOcdVfFXYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gZIm9DE6jOkf+vOKB/vMSGaIPOAb2WKcIuCsj2Q8/3s=; b=eJTsHhzFsxg3t14O8TQIg2cBxomHVH3PmBiRD+TinNosQziBuis2eAI00n3ssu0LiScsyATWPD+2+5maOXrdjuVC+x6DQhe3ISMGKV68M1Mb9PwUr2driExiKbNhPVZekSngS0CTfHuNi0dhqTMvH4WvcNVvv0CJXtqNl24Ja4yPHBQDmZ/dF3tO7jZljwPtirChwA0L7nnZlw1uHsLDTeV7KBFH4M2TFwIh2wSLG9KJh7eqYL6R8UkcFMzqfpAGXMPcSzcd2cJV/VrPAPNECn1IoBpT2xZXqB+fmubHXY4TlKEuWBNqz1acz7jt+ziJBy00x969SVq3yj0xaSxtZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=arndb.de smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gZIm9DE6jOkf+vOKB/vMSGaIPOAb2WKcIuCsj2Q8/3s=; b=mVtRmKLHEmTPl7hw2a/rYUMefar6cQa2M/SgZguCKHWB9T0Ov3dHu+ZmPMciq0ncFY1hW2ttH5vPLlGz8oMf5FGQvZEuS7xa/T2eHCpxCzdVd6DASG81nWwU3W7de3zVLXalSiKuhdRUr5AvygJ44h0vsgg7cdqEvNAYyX9BACg= Received: from CH0PR13CA0060.namprd13.prod.outlook.com (2603:10b6:610:b2::35) by PH7PR12MB7891.namprd12.prod.outlook.com (2603:10b6:510:27a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.23; Mon, 3 Mar 2025 04:43:55 +0000 Received: from CH2PEPF0000009B.namprd02.prod.outlook.com (2603:10b6:610:b2:cafe::1f) by CH0PR13CA0060.outlook.office365.com (2603:10b6:610:b2::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8489.18 via Frontend Transport; Mon, 3 Mar 2025 04:43:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH2PEPF0000009B.mail.protection.outlook.com (10.167.244.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8511.15 via Frontend Transport; Mon, 3 Mar 2025 04:43:54 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 2 Mar 2025 22:43:54 -0600 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 2 Mar 2025 22:43:54 -0600 Received: from xsjdavidz01.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Sun, 2 Mar 2025 22:43:53 -0600 From: Yidong Zhang To: , , CC: Yidong Zhang , , , DMG Karthik , "Nishad Saraf" Subject: [PATCH V0 1/3] drivers/misc/amd: Add new driver amd versal-pci Date: Sun, 2 Mar 2025 20:43:10 -0800 Message-ID: <20250303044313.679012-2-yidong.zhang@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250303044313.679012-1-yidong.zhang@amd.com> References: <20250303044313.679012-1-yidong.zhang@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF0000009B:EE_|PH7PR12MB7891:EE_ X-MS-Office365-Filtering-Correlation-Id: 6dce3c5b-afc5-4cb1-dc21-08dd5a0e00ef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|376014|13003099007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?VdZRYvNyf0+1ISHOraQuJa15B6JuHSUpJOC1KEpXR2SfhxibTmLHIf0xswvz?= =?us-ascii?Q?ODRTWdo1HggczjpGS4//QcZFmKehg+EPRKo5yesbWbfIPrM/CuS6z+KfvZnM?= =?us-ascii?Q?4HedNVUKdHHq8WVxc5K5qh8uY2KGKF+ABShdazcl9Q+WnFDkj1rl1pAc8MW/?= =?us-ascii?Q?OQV0tGVKpnoWyCkAMe+EKBDMTF3LuwnbIHvq6HJcFi/6igP7FqFwPD9ltKK3?= =?us-ascii?Q?VMUzz1eeqlmjhWZUDOY4saISus7G9leGJ8Ik8rsr1Rj3eA9urGfxdGBZ+dll?= =?us-ascii?Q?Z0McjcFXu2JBAcy9MqAFcS0f+Xq+0mc/6CK7hh4YMqtMtX6OV4yIZiB6fHxI?= =?us-ascii?Q?piuqu1cKH5RGYkH+su/JZ4OudSoMmFhmsxwQ7dEf3fAhfjGh2f1FlI3P5IBj?= =?us-ascii?Q?4MSTF8rWRZSQxhfbjPB2acx4o6MGUNf64v6t/XMiHG9lWBbIZNenZINlXJS6?= =?us-ascii?Q?zsc3CrNosZ7X6p0YpWSS8FO3wlwd9c0WDl/LJKlFY/7sJ2DIuBt1jn7FkuD3?= =?us-ascii?Q?NHn2q7d/Q+R7Z0kzXrtKCcKsBa2di25qjA/bWHyeak5Jul9ALYwVe+xrZPP/?= =?us-ascii?Q?DsEJOq2NOatBPH36R4z60yOqoUNOZgxQMLzMA5HOSoi07iGEr+nbvOdp3uW9?= =?us-ascii?Q?UI6jecAkKGlJfOp80c9iDt5wwsovTSxSjAsck3QbmXiq70fQPrBfUxG0hh1m?= =?us-ascii?Q?X+/sCf+sZ98Ka8NhtjCQnDeidvtcesifv4159IDFuddOcbtUI6iUXiLVTj96?= =?us-ascii?Q?k5PEajB+x/jpEOjylf6uXzpzLfm6nVqA18pDzx+a7LlFqBpM+NIFCSI01s6E?= =?us-ascii?Q?ea2Dg72k6C7CiaPzqQfEX8plBkht3dDJP583uJ1sHtrs5qXuj7PA01CiVIzj?= =?us-ascii?Q?MLjexxhUimIIuO1RGnWeOCqorSNhQuxc8JoY1g3nOTxnv+BLQhMgEVBntdiW?= =?us-ascii?Q?IxAclxgsFsgyT93URekySGgOIe3UxZ/fa1OpTJYyyVCeDc/taPJl9DzBtC4R?= =?us-ascii?Q?dQKYd5kzMiGEpP4IdtwLR32y/kNB+YZUUaNJP08rY9yVZ7H7BBcqHMUT37p7?= =?us-ascii?Q?gLcz1l2bi4rrt1SmPsL5b4zSxdUq48BsVSY3AQh+Lrc7Ll7s0noNtkt8wV61?= =?us-ascii?Q?O1SJmsKE/a/OlVqKp9lPi1f4srsimvTEDc3BNwHCogPNMe0wOdd12lulUOp9?= =?us-ascii?Q?S4wXUVYLqRDQluC8fuIHk08LppsXfh6N425vp0OPgJp2BTKuIlIGWWLz8RSv?= =?us-ascii?Q?CDdsaymQBgQG7EhHjOtZNeG/Yb0+JU2+6C1B/gk1P6sJ/crhcIdVFpEstkG1?= =?us-ascii?Q?Xd8nhtBubbwyJWN3MPadWdSGJ7w4tfDL50BTanNfILIKpCXZORftOAGhV5ID?= =?us-ascii?Q?h4B6nblXRMiH4c7pZmY+KxjgNdK4rNlZ0Q5sqSBsN7oG/i4GiMG+PqrBWgpB?= =?us-ascii?Q?jj/LDl37P+Y=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Mar 2025 04:43:54.7458 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6dce3c5b-afc5-4cb1-dc21-08dd5a0e00ef X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF0000009B.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7891 Content-Type: text/plain; charset="utf-8" AMD Versal based PCIe card, including V70 and RAVE, is designed for AI inference efficiency and is tuned for video analytics and natural language processing applications. The driver architecture: +---------+----+ Remote +-----+------+ | |PCIe| Queue | Embedded | | Mgmt PF |Bar | <=3D=3D=3D=3D=3D=3D=3D> | SoC | | | | +------------+ +---------+----+ | base FW | +-----+------+ This driver provides the following functionalities: - module and PCI device initialization the driver will initialize itself based on bar resources for a hardware queue based ring buffer between PCIe bar and RTOS services VMR [1] running on Embedded SoC hardware firmware for flashing firmware and checking card healthy status. - programming FW The base FW is downloaded onto the flash of the card. [1] https://github.com/Xilinx/VMR.git Co-developed-by: DMG Karthik Signed-off-by: DMG Karthik Co-developed-by: Nishad Saraf Signed-off-by: Nishad Saraf Signed-off-by: Yidong Zhang --- MAINTAINERS | 5 + drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/amd/Kconfig | 15 ++ drivers/misc/amd/Makefile | 5 + drivers/misc/amd/versal-pci-main.c | 272 +++++++++++++++++++++++++++++ drivers/misc/amd/versal-pci.h | 62 +++++++ 7 files changed, 361 insertions(+) create mode 100644 drivers/misc/amd/Kconfig create mode 100644 drivers/misc/amd/Makefile create mode 100644 drivers/misc/amd/versal-pci-main.c create mode 100644 drivers/misc/amd/versal-pci.h diff --git a/MAINTAINERS b/MAINTAINERS index 8e0736dc2ee0..8d79e8b7e963 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1215,6 +1215,11 @@ L: linux-spi@vger.kernel.org S: Supported F: drivers/spi/spi-amd.c =20 +AMD VERSAL PCI DRIVER +M: Yidong Zhang +S: Supported +F: drivers/misc/amd/ + AMD XDNA DRIVER M: Min Ma M: Lizhi Hou diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index 56bc72c7ce4a..d1616f0fb12f 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -649,4 +649,5 @@ source "drivers/misc/uacce/Kconfig" source "drivers/misc/pvpanic/Kconfig" source "drivers/misc/mchp_pci1xxxx/Kconfig" source "drivers/misc/keba/Kconfig" +source "drivers/misc/amd/Kconfig" endmenu diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index 545aad06d088..230a8d6bafaf 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -75,3 +75,4 @@ lan966x-pci-objs :=3D lan966x_pci.o lan966x-pci-objs +=3D lan966x_pci.dtbo.o obj-$(CONFIG_MCHP_LAN966X_PCI) +=3D lan966x-pci.o obj-y +=3D keba/ +obj-$(CONFIG_AMD_VERSAL_PCI) +=3D amd/ diff --git a/drivers/misc/amd/Kconfig b/drivers/misc/amd/Kconfig new file mode 100644 index 000000000000..8b1a9bd0d150 --- /dev/null +++ b/drivers/misc/amd/Kconfig @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config AMD_VERSAL_PCI + tristate "AMD Versal PCIe Management Driver" + select FW_LOADER + select CONFIGFS_FS + depends on HAS_IOMEM + depends on PCI + default m + help + AMD Versal PCIe Management Driver provides management services, + including download firmware, program bitstream, and communicate with + the User function. + + If "M" is selected, the driver module will be versal-pci diff --git a/drivers/misc/amd/Makefile b/drivers/misc/amd/Makefile new file mode 100644 index 000000000000..e3868cb39563 --- /dev/null +++ b/drivers/misc/amd/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0+ + +obj-$(CONFIG_AMD_VERSAL_PCI) +=3D versal-pci.o + +versal-pci-$(CONFIG_AMD_VERSAL_PCI) :=3D versal-pci-main.o diff --git a/drivers/misc/amd/versal-pci-main.c b/drivers/misc/amd/versal-p= ci-main.c new file mode 100644 index 000000000000..1c61c88120b4 --- /dev/null +++ b/drivers/misc/amd/versal-pci-main.c @@ -0,0 +1,272 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Driver for Versal PCIe device + * + * Copyright (C) 2025 Advanced Micro Devices, Inc. All rights reserved. + */ + +#include + +#include "versal-pci.h" + +#define DRV_NAME "amd-versal-pci" + +#define PCI_DEVICE_ID_V70PQ2 0x50B0 +#define PCI_DEVICE_ID_RAVE 0x5700 +#define VERSAL_XCLBIN_MAGIC_ID "xclbin2" + +static inline u32 versal_pci_devid(struct versal_pci_device *vdev) +{ + return ((pci_domain_nr(vdev->pdev->bus) << 16) | + PCI_DEVID(vdev->pdev->bus->number, vdev->pdev->devfn)); +} + +static int versal_pci_load_shell(struct versal_pci_device *vdev, char *fw_= name) +{ + const struct firmware *fw; + struct axlf *xsabin; + int ret; + + strim(fw_name); + + ret =3D request_firmware(&fw, fw_name, &vdev->pdev->dev); + if (ret) { + vdev_warn(vdev, "request xsabin fw %s failed %d", fw_name, ret); + return ret; + } + + xsabin =3D (struct axlf *)fw->data; + if (memcmp(xsabin->magic, VERSAL_XCLBIN_MAGIC_ID, sizeof(VERSAL_XCLBIN_MA= GIC_ID))) { + vdev_err(vdev, "Invalid device firmware"); + ret =3D -EINVAL; + goto release_firmware; + } + + if (!fw->size || + fw->size !=3D xsabin->header.length || + fw->size < sizeof(*xsabin) || + fw->size > SZ_1G) { + vdev_err(vdev, "Invalid device firmware size %zu", fw->size); + ret =3D -EINVAL; + goto release_firmware; + } + + if (!uuid_equal(&vdev->intf_uuid, &xsabin->header.rom_uuid)) { + vdev_err(vdev, "base shell doesn't match uuid %pUb", &xsabin->header.uui= d); + ret =3D -EINVAL; + goto release_firmware; + } + + /* TODO upload fw to card */ + if (ret) { + vdev_err(vdev, "failed to load xsabin %s : %d", fw_name, ret); + goto release_firmware; + } + + vdev_info(vdev, "Downloaded xsabin %pUb of size %lld Bytes", + &xsabin->header.uuid, xsabin->header.length); + +release_firmware: + release_firmware(fw); + + return ret; +} + +static inline struct versal_pci_device *item_to_vdev(struct config_item *i= tem) +{ + return container_of(to_configfs_subsystem(to_config_group(item)), + struct versal_pci_device, cfs_subsys); +} + +static ssize_t versal_pci_cfs_config_store(struct config_item *item, + const char *page, size_t count) +{ + struct versal_pci_device *vdev =3D item_to_vdev(item); + u32 config; + int ret; + + ret =3D kstrtou32(page, 0, &config); + if (ret) + return -EINVAL; + + if (config) + ret =3D versal_pci_load_shell(vdev, vdev->fw.name); + + if (ret) + return -EFAULT; + + return count; +} +CONFIGFS_ATTR_WO(versal_pci_cfs_, config); + +static ssize_t versal_pci_cfs_image_show(struct config_item *item, char *p= age) +{ + struct versal_pci_device *vdev =3D item_to_vdev(item); + + vdev_info(vdev, "fw name: %s", vdev->fw.name); + + return 0; +} + +static ssize_t versal_pci_cfs_image_store(struct config_item *item, + const char *page, size_t count) +{ + struct versal_pci_device *vdev =3D item_to_vdev(item); + + count =3D snprintf(vdev->fw.name, sizeof(vdev->fw.name), "%s", page); + + vdev_info(vdev, "fw name: %s", vdev->fw.name); + return count; +} +CONFIGFS_ATTR(versal_pci_cfs_, image); + +static struct configfs_attribute *versal_pci_cfs_attrs[] =3D { + &versal_pci_cfs_attr_config, + &versal_pci_cfs_attr_image, + NULL, +}; + +static const struct config_item_type versal_pci_cfs_table =3D { + .ct_owner =3D THIS_MODULE, + .ct_attrs =3D versal_pci_cfs_attrs, +}; + +static int versal_pci_cfs_init(struct versal_pci_device *vdev) +{ + struct configfs_subsystem *subsys =3D &vdev->cfs_subsys; + + snprintf(subsys->su_group.cg_item.ci_namebuf, + sizeof(subsys->su_group.cg_item.ci_namebuf), + "%s%x", DRV_NAME, versal_pci_devid(vdev)); + + subsys->su_group.cg_item.ci_type =3D &versal_pci_cfs_table; + + config_group_init(&subsys->su_group); + return configfs_register_subsystem(subsys); +} + +static void versal_pci_fw_fini(struct versal_pci_device *vdev) +{ + uuid_copy(&vdev->intf_uuid, &uuid_null); +} + +static void versal_pci_cfs_fini(struct configfs_subsystem *subsys) +{ + configfs_unregister_subsystem(subsys); +} + +static void versal_pci_device_teardown(struct versal_pci_device *vdev) +{ + versal_pci_cfs_fini(&vdev->cfs_subsys); + versal_pci_fw_fini(vdev); +} + +static void versal_pci_uuid_parse(struct versal_pci_device *vdev, uuid_t *= uuid) +{ + char str[UUID_STRING_LEN]; + u8 i, j; + int len =3D strlen(vdev->fw_id); + + /* parse uuid into a valid uuid string format */ + for (i =3D 0, j =3D 0; i < len && i < sizeof(str); i++) { + str[j++] =3D vdev->fw_id[i]; + if (j =3D=3D 8 || j =3D=3D 13 || j =3D=3D 18 || j =3D=3D 23) + str[j++] =3D '-'; + } + + uuid_parse(str, uuid); + vdev_info(vdev, "Interface uuid %pU", uuid); +} + +static int versal_pci_fw_init(struct versal_pci_device *vdev) +{ + /* TODO request compatible fw_id from card */ + + versal_pci_uuid_parse(vdev, &vdev->intf_uuid); + + return 0; +} + +static int versal_pci_device_setup(struct versal_pci_device *vdev) +{ + int ret; + + ret =3D versal_pci_fw_init(vdev); + if (ret) { + vdev_err(vdev, "Failed to init fw, err %d", ret); + goto comm_chan_fini; + } + + ret =3D versal_pci_cfs_init(vdev); + if (ret) { + vdev_err(vdev, "Failed to init configfs subsys, err %d", ret); + goto comm_chan_fini; + } + + return 0; + +comm_chan_fini: + + return ret; +} + +static void versal_pci_remove(struct pci_dev *pdev) +{ + struct versal_pci_device *vdev =3D pci_get_drvdata(pdev); + + versal_pci_device_teardown(vdev); +} + +static int versal_pci_probe(struct pci_dev *pdev, const struct pci_device_= id *pdev_id) +{ + struct versal_pci_device *vdev; + int ret; + + vdev =3D devm_kzalloc(&pdev->dev, sizeof(*vdev), GFP_KERNEL); + if (!vdev) + return -ENOMEM; + + pci_set_drvdata(pdev, vdev); + vdev->pdev =3D pdev; + + ret =3D pcim_enable_device(pdev); + if (ret) { + vdev_err(vdev, "Failed to enable device %d", ret); + return ret; + } + + vdev->io_regs =3D pcim_iomap_region(vdev->pdev, MGMT_BAR, DRV_NAME); + if (IS_ERR(vdev->io_regs)) { + vdev_err(vdev, "Failed to map RM shared memory BAR%d", MGMT_BAR); + return PTR_ERR(vdev->io_regs); + } + + ret =3D versal_pci_device_setup(vdev); + if (ret) { + vdev_err(vdev, "Failed to setup Versal device %d", ret); + return ret; + } + + return 0; +} + +static const struct pci_device_id versal_pci_ids[] =3D { + { PCI_DEVICE(PCI_VENDOR_ID_XILINX, PCI_DEVICE_ID_V70PQ2), }, + { PCI_DEVICE(PCI_VENDOR_ID_XILINX, PCI_DEVICE_ID_RAVE), }, + { 0 } +}; + +MODULE_DEVICE_TABLE(pci, versal_pci_ids); + +static struct pci_driver versal_pci_driver =3D { + .name =3D DRV_NAME, + .id_table =3D versal_pci_ids, + .probe =3D versal_pci_probe, + .remove =3D versal_pci_remove, +}; + +module_pci_driver(versal_pci_driver); + +MODULE_DESCRIPTION("AMD Versal PCIe Management Driver"); +MODULE_AUTHOR("XRT Team "); +MODULE_LICENSE("GPL"); diff --git a/drivers/misc/amd/versal-pci.h b/drivers/misc/amd/versal-pci.h new file mode 100644 index 000000000000..ca309aee87ad --- /dev/null +++ b/drivers/misc/amd/versal-pci.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Driver for Versal PCIe device + * + * Copyright (C) 2025 Advanced Micro Devices, Inc. All rights reserved. + */ + +#ifndef __VERSAL_PCI_H +#define __VERSAL_PCI_H + +#include +#include + +#define MGMT_BAR 0 + +#define vdev_info(vdev, fmt, args...) \ + dev_info(&(vdev)->pdev->dev, "%s: "fmt, __func__, ##args) + +#define vdev_warn(vdev, fmt, args...) \ + dev_warn(&(vdev)->pdev->dev, "%s: "fmt, __func__, ##args) + +#define vdev_err(vdev, fmt, args...) \ + dev_err(&(vdev)->pdev->dev, "%s: "fmt, __func__, ##args) + +#define vdev_dbg(vdev, fmt, args...) \ + dev_dbg(&(vdev)->pdev->dev, fmt, ##args) + +struct versal_pci_device; + +struct axlf_header { + __u64 length; + __u8 reserved1[24]; + uuid_t rom_uuid; + __u8 reserved2[64]; + uuid_t uuid; + __u8 reserved3[24]; +} __packed; + +struct axlf { + __u8 magic[8]; + __u8 reserved[296]; + struct axlf_header header; +} __packed; + +struct fw_info { + __u32 opcode; + char name[128]; +}; + +struct versal_pci_device { + struct pci_dev *pdev; + + struct fw_info fw; + + void __iomem *io_regs; + uuid_t intf_uuid; + __u8 fw_id[UUID_STRING_LEN + 1]; + + struct configfs_subsystem cfs_subsys; +}; + +#endif /* __VERSAL_PCI_H */ --=20 2.34.1 From nobody Sat Feb 7 09:30:21 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2085.outbound.protection.outlook.com [40.107.236.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 578941DE2BF for ; Mon, 3 Mar 2025 04:44:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.85 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740977058; cv=fail; b=sd6hlKmZekJBZXpgQmLIHpaXSIUR1NM8YIt2OGhb4Ea1nklJCYo3LKBqC7oEd9usAA40jJauWHGmgYhczmfnGZM+529Ipt6Ozz7pOS2nWk9rbb9gGwcEDU5w3NVSAW1UwCJMKeCQ6S/78akOaj0JXVExP0qDeQMi9bttJBuiW5E= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740977058; c=relaxed/simple; bh=Uzbww88kDt5E/rj4OCeUQGTUdmktdatld64cE6B/TSs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GJmA9rrGfc3WTwbTnA6FfnS7UggtyVqR4IWO0Bkrs09SWBYHk5YrFmIykde6qNqhpP+1xHWPyKYXPCRVNHL2fvMTzHw/w4EItjuvV5Ha7g4ICci3kR0BWp5EATseqFdgngw+jbgqkDlx/1F2OVJiM/mvpWvPGCHXLe6fquz5cdA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=J2oL9KPq; arc=fail smtp.client-ip=40.107.236.85 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="J2oL9KPq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LWnpnmIU1QZClX8akkptC+W90jY2psmSR6nlsfXYWykjGVHlBzeWHVBGTp+Hb0cVCNieFEGi1Wmu0oQxH7Sw6U4uNh3zQZZgx6oYSHEGRhJzS40OTcGPBVrVwmwW2EstQnj6LfSOvhsWb8Dngme6Ui6/g3+rzJKVAaJ2jbO/uESDUDsX+nIaGdIjTp7oaSKur64VG05grKBeoa3+5CACv3n6h5JH+jN59RQHPhmUuoUyeXRXxtAMUcRhJadMH11ChIjXoJjS13+8cynsfgqArm5Piq98kS6ByiKt59XJ56231sZTp3wzyEu+0WJMdmq8ujjKbHzqB6sHMRwFO4R+Tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=noXVYckU+zlFaJNZ9RTs0xOP4Sd+Jlo52clo97plhxg=; b=CTsSvXy+GmGpAKIkHhDHzdRpthtXVV9WZp5KSZUhn7PNE1wu9DkCDtOd6bpwoKQFUIO7Re65fQDXH4QfMeksoPK+uqV/BTnz+LCbQqDEfnjx84IewsHw8OsntjeK3wKwx/gxj5/yT2QLxA3mnSj0j813vQSgQP1Exz+yG8sjZiLLlw9Wprajv4qVNOn+sMaTOboNQ7GU8iSNHOLYPnU6WZfHu6BD4ELtwtAWXIPjX87KDMGak+25PFVw2cMsD4ISSJ8UGDTTECXtcOuzE9ET+pm3AirdYkL54Bli5gDNhfJ6y6Rsdq9gAM7huGkS05ZQATFOsBb8UiNPS2PcUjBKpg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=arndb.de smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=noXVYckU+zlFaJNZ9RTs0xOP4Sd+Jlo52clo97plhxg=; b=J2oL9KPqW0kd2Uvdffc3MrVUVEnt/cKu0oyHLhg+j3UBKYWJypCZn9MNQvoun/lwWaOAFGRH90WBVp/bMC48Cqb+bYyBVbV8qh1eQASbu9hN2ns36RsY54h9zkPGFwOqhYKQrhPVqVpN1zbXTlsk+VhmR2n/heJJuYN15tkFCYw= Received: from BN9PR03CA0562.namprd03.prod.outlook.com (2603:10b6:408:138::27) by IA1PR12MB8494.namprd12.prod.outlook.com (2603:10b6:208:44c::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.25; Mon, 3 Mar 2025 04:44:10 +0000 Received: from BL02EPF00021F69.namprd02.prod.outlook.com (2603:10b6:408:138:cafe::40) by BN9PR03CA0562.outlook.office365.com (2603:10b6:408:138::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8489.28 via Frontend Transport; Mon, 3 Mar 2025 04:44:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by BL02EPF00021F69.mail.protection.outlook.com (10.167.249.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8511.15 via Frontend Transport; Mon, 3 Mar 2025 04:44:10 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 2 Mar 2025 22:44:09 -0600 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 2 Mar 2025 22:43:56 -0600 Received: from xsjdavidz01.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Sun, 2 Mar 2025 22:43:55 -0600 From: Yidong Zhang To: , , CC: Yidong Zhang , , , Nishad Saraf Subject: [PATCH V0 2/3] drivers/misc/amd: Add remote queue service Date: Sun, 2 Mar 2025 20:43:11 -0800 Message-ID: <20250303044313.679012-3-yidong.zhang@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250303044313.679012-1-yidong.zhang@amd.com> References: <20250303044313.679012-1-yidong.zhang@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F69:EE_|IA1PR12MB8494:EE_ X-MS-Office365-Filtering-Correlation-Id: fe611fb6-7ea6-4451-87ff-08dd5a0e0a5b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|1800799024|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?3sUnh48t960boKk2ARcCFuqqOSCjEEuvHPIagwTOS9Ltf2+lCNEDqpm1JQFK?= =?us-ascii?Q?sfcmMtesfDw2sWRYcluhwNR9Yzpup6m9cHH73JUlUVCcKNT830ySxBIxR9U6?= =?us-ascii?Q?sTjsdAjb6Aov/psuWVM7yIwSV4kNxZbGW6Leg6xg5yybvdGCgUR3lCZmdwa+?= =?us-ascii?Q?aES+00pLEo5cbq8FfH6mUIsdCKDwlhmSy7/54iLlgpAmyHY8/EvcUvFlR8Xy?= =?us-ascii?Q?kpZcSCSN8Ns3KDAsJ/IP2akvQuQ87sFgZJ1hvO4TtMYEHq/EY2t+/uqplQIe?= =?us-ascii?Q?7wT5XPMIB5E1yDVplxLBq2W6HLTAq9DpCggMaoHgGuiK5d9ESg2Ux8rrMmFB?= =?us-ascii?Q?wePsYuPJ1kuOGd1oj6WjcO3G8Tt1JTjNXeaS5HHZ8sbNs52StjQ9yDFL7Lvu?= =?us-ascii?Q?1KcYzcaiOXyDJp5oVIXSqxyIC4OlfoZe+yXS7kxvTLvZY6b0PFer93lqe6UK?= =?us-ascii?Q?EkYMR7n6DhVRdIfuk/TAB2YaahKXESxZNvxXFg5KwDh68X/cvUGbmN8+qA4f?= =?us-ascii?Q?W58cgO5B/x6NCuwjVxjRN+IkDe3+Z5sN8ggfwF2Klixwf6Dc37CxyDFvKCFH?= =?us-ascii?Q?+QuqJvP00WPJbxby/5ckoi6TIUwC5IPJi2/haD//oL7ePYe0KlqfPmSo2BGi?= =?us-ascii?Q?2LZWjluN+XSMckVtM6wMzZCweBBxO5kGDziMKcv3Xti+D2+nqHok/KsQgihi?= =?us-ascii?Q?nnW7Y5GLesGpGm2eOstbQnZvgEWGUvWUxWaSz7Pb9HI8vbxFpiXCNtwZRP70?= =?us-ascii?Q?Xd2fUx4FEp9FRNmycjZk1nc/uRHPUd6fsaRPn8cQpERG05bJrEz5TwWxlP9S?= =?us-ascii?Q?eqbGI22smgEZXH6H1amdzV03PKq+LNMkuHbVj1nTV3Uqi5sPVB4mwifieZ5W?= =?us-ascii?Q?/khD/UlhwVlfD7oyC6KaiMYgfFc6idG/v22NwvsIaRQEczQhE/AfDI0Jia6T?= =?us-ascii?Q?JDRfZI3tunxWF7F7Zme3CgbffFBLJ0ELGNzLLucKj30ptqD16x+o9KJp6Z9o?= =?us-ascii?Q?+3ggf+Du7lJ9oYPeVrGrluTAQQIX684YFvSjfzFledaVDoDEegX1N8KMMoiD?= =?us-ascii?Q?PXFAnwWtjTCLvCzOQSXfwsW+JKOEnuCb2K33/HIuIuo73M2CYTzzxi9xFesU?= =?us-ascii?Q?R06cnGT0ULO1M9BiTYf6+QDOMtd8Qgjwmrq1Cka/lwrlF4JEbScfgErwh1X7?= =?us-ascii?Q?8kpuvAbXvDZRhNY2BmTAUAY1br4z3u14izHlZnSDYX4Q+NalsxnO2FP++Wcf?= =?us-ascii?Q?e7x1C31G1OXJ3XZu860SfXZLsrtGkGfh1cEOCt2Kktj0XtzCYTaKcVlQ47h7?= =?us-ascii?Q?YGPiyHSVAh4POBPmjW211MihaRFcSf+P5Zm16FIiqwoOTO9agwFZgbTEH9yf?= =?us-ascii?Q?VVg50DXCBDGebArcB5iu4xxpDR82i7+F6c7estIvUSVNueB7UjAfojqY0ouw?= =?us-ascii?Q?glZOvlcqXQ0zeuRQu13VnuDPIq2KoibukYTWnX7+/DjtBcBdGHTuSq1P6KRe?= =?us-ascii?Q?6kGH7WV6sOs9Z/0=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(1800799024)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Mar 2025 04:44:10.5822 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fe611fb6-7ea6-4451-87ff-08dd5a0e0a5b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F69.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8494 Content-Type: text/plain; charset="utf-8" The remote queue is a hardware queue based ring buffer service between mgmt PF and firmware for data and message transfer. Co-developed-by: Nishad Saraf Signed-off-by: Nishad Saraf Signed-off-by: Yidong Zhang --- drivers/misc/amd/Makefile | 3 +- drivers/misc/amd/versal-pci-rm-queue.c | 316 +++++++++++++++++++++++ drivers/misc/amd/versal-pci-rm-queue.h | 21 ++ drivers/misc/amd/versal-pci-rm-service.h | 209 +++++++++++++++ drivers/misc/amd/versal-pci.h | 1 + 5 files changed, 549 insertions(+), 1 deletion(-) create mode 100644 drivers/misc/amd/versal-pci-rm-queue.c create mode 100644 drivers/misc/amd/versal-pci-rm-queue.h create mode 100644 drivers/misc/amd/versal-pci-rm-service.h diff --git a/drivers/misc/amd/Makefile b/drivers/misc/amd/Makefile index e3868cb39563..63879800a3e5 100644 --- a/drivers/misc/amd/Makefile +++ b/drivers/misc/amd/Makefile @@ -2,4 +2,5 @@ =20 obj-$(CONFIG_AMD_VERSAL_PCI) +=3D versal-pci.o =20 -versal-pci-$(CONFIG_AMD_VERSAL_PCI) :=3D versal-pci-main.o +versal-pci-$(CONFIG_AMD_VERSAL_PCI) :=3D versal-pci-main.o \ + versal-pci-rm-queue.o diff --git a/drivers/misc/amd/versal-pci-rm-queue.c b/drivers/misc/amd/vers= al-pci-rm-queue.c new file mode 100644 index 000000000000..8c1aca442253 --- /dev/null +++ b/drivers/misc/amd/versal-pci-rm-queue.c @@ -0,0 +1,316 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Driver for Versal PCIe device + * + * Copyright (C) 2025 Advanced Micro Devices, Inc. All rights reserved. + */ + +#include + +#include "versal-pci.h" +#include "versal-pci-rm-queue.h" +#include "versal-pci-rm-service.h" + +static inline struct rm_device *to_rdev_msg_monitor(struct work_struct *w) +{ + return container_of(w, struct rm_device, msg_monitor); +} + +static inline struct rm_device *to_rdev_msg_timer(struct timer_list *t) +{ + return container_of(t, struct rm_device, msg_timer); +} + +static inline u32 rm_io_read(struct rm_device *rdev, u32 offset) +{ + /* TODO */ + return 0; +} + +static inline int rm_io_write(struct rm_device *rdev, u32 offset, u32 valu= e) +{ + /* TODO */ + return 0; +} + +static inline u32 rm_queue_read(struct rm_device *rdev, u32 offset) +{ + /* TODO */ + return 0; +} + +static inline void rm_queue_write(struct rm_device *rdev, u32 offset, u32 = value) +{ + /* TODO */ +} + +static inline void rm_queue_bulk_read(struct rm_device *rdev, u32 offset, + u32 *value, u32 size) +{ + /* TODO */ +} + +static inline void rm_queue_bulk_write(struct rm_device *rdev, u32 offset, + u32 *value, u32 size) +{ + /* TODO */ +} + +static inline u32 rm_queue_get_cidx(struct rm_device *rdev, enum rm_queue_= type type) +{ + u32 off; + + if (type =3D=3D RM_QUEUE_SQ) + off =3D offsetof(struct rm_queue_header, sq_cidx); + else + off =3D offsetof(struct rm_queue_header, cq_cidx); + + return rm_queue_read(rdev, off); +} + +static inline void rm_queue_set_cidx(struct rm_device *rdev, enum rm_queue= _type type, + u32 value) +{ + u32 off; + + if (type =3D=3D RM_QUEUE_SQ) + off =3D offsetof(struct rm_queue_header, sq_cidx); + else + off =3D offsetof(struct rm_queue_header, cq_cidx); + + rm_queue_write(rdev, off, value); +} + +static inline u32 rm_queue_get_pidx(struct rm_device *rdev, enum rm_queue_= type type) +{ + if (type =3D=3D RM_QUEUE_SQ) + return rm_io_read(rdev, RM_IO_SQ_PIDX_OFF); + else + return rm_io_read(rdev, RM_IO_CQ_PIDX_OFF); +} + +static inline int rm_queue_set_pidx(struct rm_device *rdev, + enum rm_queue_type type, u32 value) +{ + if (type =3D=3D RM_QUEUE_SQ) + return rm_io_write(rdev, RM_IO_SQ_PIDX_OFF, value); + else + return rm_io_write(rdev, RM_IO_CQ_PIDX_OFF, value); +} + +static inline u32 rm_queue_get_sq_slot_offset(struct rm_device *rdev) +{ + u32 index; + + if ((rdev->sq.pidx - rdev->sq.cidx) >=3D rdev->queue_size) + return RM_INVALID_SLOT; + + index =3D rdev->sq.pidx & (rdev->queue_size - 1); + return rdev->sq.offset + RM_CMD_SQ_SLOT_SIZE * index; +} + +static inline u32 rm_queue_get_cq_slot_offset(struct rm_device *rdev) +{ + u32 index; + + index =3D rdev->cq.cidx & (rdev->queue_size - 1); + return rdev->cq.offset + RM_CMD_CQ_SLOT_SIZE * index; +} + +static int rm_queue_submit_cmd(struct rm_cmd *cmd) +{ + struct versal_pci_device *vdev =3D cmd->rdev->vdev; + struct rm_device *rdev =3D cmd->rdev; + u32 offset; + int ret; + + guard(mutex)(&rdev->queue); + + offset =3D rm_queue_get_sq_slot_offset(rdev); + if (!offset) { + vdev_err(vdev, "No SQ slot available"); + return -ENOSPC; + } + + rm_queue_bulk_write(rdev, offset, (u32 *)&cmd->sq_msg, + sizeof(cmd->sq_msg)); + + ret =3D rm_queue_set_pidx(rdev, RM_QUEUE_SQ, ++rdev->sq.pidx); + if (ret) { + vdev_err(vdev, "Failed to update PIDX, ret %d", ret); + return ret; + } + + list_add_tail(&cmd->list, &rdev->submitted_cmds); + return ret; +} + +void rm_queue_withdraw_cmd(struct rm_cmd *cmd) +{ + guard(mutex)(&cmd->rdev->queue); + list_del(&cmd->list); +} + +static int rm_queue_wait_cmd_timeout(struct rm_cmd *cmd, unsigned long tim= eout) +{ + struct versal_pci_device *vdev =3D cmd->rdev->vdev; + int ret; + + if (wait_for_completion_timeout(&cmd->executed, timeout)) { + ret =3D cmd->cq_msg.data.rcode; + if (!ret) + return 0; + + vdev_err(vdev, "CMD returned with a failure: %d", ret); + return ret; + } + + /* + * each cmds will be cleaned up by complete before it times out. + * if we reach here, the cmd should be cleared and hot reset should + * be issued. + */ + vdev_err(vdev, "cmd is timedout after, please reset the card"); + rm_queue_withdraw_cmd(cmd); + return -ETIME; +} + +int rm_queue_send_cmd(struct rm_cmd *cmd, unsigned long timeout) +{ + int ret; + + ret =3D rm_queue_submit_cmd(cmd); + if (ret) + return ret; + + return rm_queue_wait_cmd_timeout(cmd, timeout); +} + +static int rm_process_msg(struct rm_device *rdev) +{ + struct versal_pci_device *vdev =3D rdev->vdev; + struct rm_cmd *cmd, *next; + struct rm_cmd_cq_hdr header; + u32 offset; + + offset =3D rm_queue_get_cq_slot_offset(rdev); + if (!offset) { + vdev_err(vdev, "Invalid CQ offset"); + return -EINVAL; + } + + rm_queue_bulk_read(rdev, offset, (u32 *)&header, sizeof(header)); + + list_for_each_entry_safe(cmd, next, &rdev->submitted_cmds, list) { + u32 value =3D 0; + + if (cmd->sq_msg.hdr.id !=3D header.id) + continue; + + rm_queue_bulk_read(rdev, offset + sizeof(cmd->cq_msg.hdr), + (u32 *)&cmd->cq_msg.data, + sizeof(cmd->cq_msg.data)); + + rm_queue_write(rdev, offset, value); + + list_del(&cmd->list); + complete(&cmd->executed); + return 0; + } + + vdev_err(vdev, "Unknown cmd ID %d found in CQ", header.id); + return -EFAULT; +} + +static void rm_check_msg(struct work_struct *w) +{ + struct rm_device *rdev =3D to_rdev_msg_monitor(w); + int ret; + + guard(mutex)(&rdev->queue); + + rdev->sq.cidx =3D rm_queue_get_cidx(rdev, RM_QUEUE_SQ); + rdev->cq.pidx =3D rm_queue_get_pidx(rdev, RM_QUEUE_CQ); + + while (rdev->cq.cidx < rdev->cq.pidx) { + ret =3D rm_process_msg(rdev); + if (ret) + break; + + rdev->cq.cidx++; + + rm_queue_set_cidx(rdev, RM_QUEUE_CQ, rdev->cq.cidx); + } +} + +static void rm_sched_work(struct timer_list *t) +{ + struct rm_device *rdev =3D to_rdev_msg_timer(t); + + /* Schedule a work in the general workqueue */ + schedule_work(&rdev->msg_monitor); + /* Periodic timer */ + mod_timer(&rdev->msg_timer, jiffies + RM_COMPLETION_TIMER); +} + +void rm_queue_fini(struct rm_device *rdev) +{ + del_timer_sync(&rdev->msg_timer); + cancel_work_sync(&rdev->msg_monitor); +} + +int rm_queue_init(struct rm_device *rdev) +{ + struct versal_pci_device *vdev =3D rdev->vdev; + struct rm_queue_header header =3D {0}; + int ret; + + INIT_LIST_HEAD(&rdev->submitted_cmds); + ret =3D devm_mutex_init(&vdev->pdev->dev, &rdev->queue); + if (ret) + return ret; + + rm_queue_bulk_read(rdev, RM_HDR_OFF, (u32 *)&header, sizeof(header)); + + if (header.magic !=3D RM_QUEUE_HDR_MAGIC_NUM) { + vdev_err(vdev, "Invalid RM queue header"); + return -ENODEV; + } + + if (!header.version) { + vdev_err(vdev, "Invalid RM queue header"); + return -ENODEV; + } + + sema_init(&rdev->sq.data_lock, 1); + sema_init(&rdev->cq.data_lock, 1); + rdev->queue_size =3D header.size; + rdev->sq.offset =3D header.sq_off; + rdev->cq.offset =3D header.cq_off; + rdev->sq.type =3D RM_QUEUE_SQ; + rdev->cq.type =3D RM_QUEUE_CQ; + rdev->sq.data_size =3D rdev->queue_buffer_size - RM_CMD_CQ_BUFFER_SIZE; + rdev->cq.data_size =3D RM_CMD_CQ_BUFFER_SIZE; + rdev->sq.data_offset =3D rdev->queue_buffer_start + + RM_CMD_CQ_BUFFER_OFFSET + RM_CMD_CQ_BUFFER_SIZE; + rdev->cq.data_offset =3D rdev->queue_buffer_start + + RM_CMD_CQ_BUFFER_OFFSET; + rdev->sq.cidx =3D header.sq_cidx; + rdev->cq.cidx =3D header.cq_cidx; + + rdev->sq.pidx =3D rm_queue_get_pidx(rdev, RM_QUEUE_SQ); + rdev->cq.pidx =3D rm_queue_get_pidx(rdev, RM_QUEUE_CQ); + + if (rdev->cq.cidx !=3D rdev->cq.pidx) { + vdev_warn(vdev, "Clearing stale completions"); + rdev->cq.cidx =3D rdev->cq.pidx; + rm_queue_set_cidx(rdev, RM_QUEUE_CQ, rdev->cq.cidx); + } + + /* Create and schedule timer to do recurring work */ + INIT_WORK(&rdev->msg_monitor, &rm_check_msg); + timer_setup(&rdev->msg_timer, &rm_sched_work, 0); + mod_timer(&rdev->msg_timer, jiffies + RM_COMPLETION_TIMER); + + return 0; +} diff --git a/drivers/misc/amd/versal-pci-rm-queue.h b/drivers/misc/amd/vers= al-pci-rm-queue.h new file mode 100644 index 000000000000..d5d991704d5c --- /dev/null +++ b/drivers/misc/amd/versal-pci-rm-queue.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Driver for Versal PCIe device + * + * Copyright (C) 2025 Advanced Micro Devices, Inc. All rights reserved. + */ + +#ifndef __RM_QUEUE_H +#define __RM_QUEUE_H + +struct rm_device; + +/* rm queue hardware setup */ +int rm_queue_init(struct rm_device *rdev); +void rm_queue_fini(struct rm_device *rdev); + +/* rm queue common API */ +int rm_queue_send_cmd(struct rm_cmd *cmd, unsigned long timeout); +void rm_queue_withdraw_cmd(struct rm_cmd *cmd); + +#endif /* __RM_QUEUE_H */ diff --git a/drivers/misc/amd/versal-pci-rm-service.h b/drivers/misc/amd/ve= rsal-pci-rm-service.h new file mode 100644 index 000000000000..d2397a1a672c --- /dev/null +++ b/drivers/misc/amd/versal-pci-rm-service.h @@ -0,0 +1,209 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Driver for Versal PCIe device + * + * Copyright (C) 2025 Advanced Micro Devices, Inc. All rights reserved. + */ + +#ifndef __RM_SERVICE_H +#define __RM_SERVICE_H + +#define RM_HDR_OFF 0x0 +#define RM_HDR_MAGIC_NUM 0x564D5230 +#define RM_QUEUE_HDR_MAGIC_NUM 0x5847513F +#define RM_PCI_IO_BAR_OFF 0x2010000 +#define RM_PCI_IO_SIZE SZ_4K +#define RM_PCI_SHMEM_BAR_OFF 0x8000000 +#define RM_PCI_SHMEM_SIZE SZ_128M +#define RM_PCI_SHMEM_HDR_SIZE 0x28 + +#define RM_QUEUE_HDR_MAGIC_NUM_OFF 0x0 +#define RM_IO_SQ_PIDX_OFF 0x0 +#define RM_IO_CQ_PIDX_OFF 0x100 + +#define RM_CMD_ID_MIN 1 +#define RM_CMD_ID_MAX (BIT(17) - 1) +#define RM_CMD_SQ_HDR_OPS_MSK GENMASK(15, 0) +#define RM_CMD_SQ_HDR_SIZE_MSK GENMASK(14, 0) +#define RM_CMD_SQ_SLOT_SIZE SZ_512 +#define RM_CMD_CQ_SLOT_SIZE SZ_16 +#define RM_CMD_CQ_BUFFER_SIZE (1024 * 1024) +#define RM_CMD_CQ_BUFFER_OFFSET 0x0 +#define RM_CMD_LOG_PAGE_TYPE_MASK GENMASK(15, 0) +#define RM_CMD_VMR_CONTROL_MSK GENMASK(10, 8) +#define RM_CMD_VMR_CONTROL_PS_MASK BIT(9) + +#define RM_CMD_WAIT_CONFIG_TIMEOUT msecs_to_jiffies(10 * 1000) +#define RM_CMD_WAIT_DOWNLOAD_TIMEOUT msecs_to_jiffies(300 * 1000) + +#define RM_COMPLETION_TIMER (HZ / 10) +#define RM_HEALTH_CHECK_TIMER (HZ) + +#define RM_INVALID_SLOT 0 + +enum rm_queue_opcode { + RM_QUEUE_OP_LOAD_XCLBIN =3D 0x0, + RM_QUEUE_OP_GET_LOG_PAGE =3D 0x8, + RM_QUEUE_OP_LOAD_FW =3D 0xA, + RM_QUEUE_OP_LOAD_APU_FW =3D 0xD, + RM_QUEUE_OP_VMR_CONTROL =3D 0xE, + RM_QUEUE_OP_IDENTIFY =3D 0x202, +}; + +struct rm_cmd_sq_hdr { + __u16 opcode; + __u16 msg_size; + __u16 id; + __u16 reserved; +} __packed; + +struct rm_cmd_cq_hdr { + __u16 id; + __u16 reserved; +} __packed; + +struct rm_cmd_sq_bin { + __u64 address; + __u32 size; + __u32 reserved1; + __u32 reserved2; + __u32 reserved3; + __u64 reserved4; +} __packed; + +struct rm_cmd_sq_log_page { + __u64 address; + __u32 size; + __u32 reserved1; + __u32 type; + __u32 reserved2; +} __packed; + +struct rm_cmd_sq_ctrl { + __u32 status; +} __packed; + +struct rm_cmd_sq_data { + union { + struct rm_cmd_sq_log_page page; + struct rm_cmd_sq_bin bin; + struct rm_cmd_sq_ctrl ctrl; + }; +} __packed; + +struct rm_cmd_cq_identify { + __u16 major; + __u16 minor; + __u32 reserved; +} __packed; + +struct rm_cmd_cq_log_page { + __u32 len; + __u32 reserved; +} __packed; + +struct rm_cmd_cq_control { + __u16 status; + __u16 reserved1; + __u32 reserved2; +} __packed; + +struct rm_cmd_cq_data { + union { + struct rm_cmd_cq_identify identify; + struct rm_cmd_cq_log_page page; + struct rm_cmd_cq_control ctrl; + __u32 reserved[2]; + }; + __u32 rcode; +} __packed; + +struct rm_cmd_sq_msg { + struct rm_cmd_sq_hdr hdr; + struct rm_cmd_sq_data data; +} __packed; + +struct rm_cmd_cq_msg { + struct rm_cmd_cq_hdr hdr; + struct rm_cmd_cq_data data; +} __packed; + +struct rm_cmd { + struct rm_device *rdev; + struct list_head list; + struct completion executed; + struct rm_cmd_sq_msg sq_msg; + struct rm_cmd_cq_msg cq_msg; + enum rm_queue_opcode opcode; + __u8 *buffer; + ssize_t size; +}; + +enum rm_queue_type { + RM_QUEUE_SQ, + RM_QUEUE_CQ +}; + +enum rm_cmd_log_page_type { + RM_CMD_LOG_PAGE_AXI_TRIP_STATUS =3D 0x0, + RM_CMD_LOG_PAGE_FW_ID =3D 0xA, +}; + +struct rm_queue { + enum rm_queue_type type; + __u32 pidx; + __u32 cidx; + __u32 offset; + __u32 data_offset; + __u32 data_size; + struct semaphore data_lock; +}; + +struct rm_queue_header { + __u32 magic; + __u32 version; + __u32 size; + __u32 sq_off; + __u32 sq_slot_size; + __u32 cq_off; + __u32 sq_cidx; + __u32 cq_cidx; +}; + +struct rm_header { + __u32 magic; + __u32 queue_base; + __u32 queue_size; + __u32 status_off; + __u32 status_len; + __u32 log_index; + __u32 log_off; + __u32 log_size; + __u32 data_start; + __u32 data_end; +}; + +struct rm_device { + struct versal_pci_device *vdev; + + struct rm_header rm_metadata; + __u32 queue_buffer_start; + __u32 queue_buffer_size; + __u32 queue_base; + + /* Lock to queue access */ + struct mutex queue; + struct rm_queue sq; + struct rm_queue cq; + __u32 queue_size; + + struct timer_list msg_timer; + struct work_struct msg_monitor; + struct timer_list health_timer; + struct work_struct health_monitor; + struct list_head submitted_cmds; + + __u32 firewall_tripped; +}; + +#endif /* __RM_SERVICE_H */ diff --git a/drivers/misc/amd/versal-pci.h b/drivers/misc/amd/versal-pci.h index ca309aee87ad..33f0ef881a33 100644 --- a/drivers/misc/amd/versal-pci.h +++ b/drivers/misc/amd/versal-pci.h @@ -26,6 +26,7 @@ dev_dbg(&(vdev)->pdev->dev, fmt, ##args) =20 struct versal_pci_device; +struct rm_cmd; =20 struct axlf_header { __u64 length; --=20 2.34.1 From nobody Sat Feb 7 09:30:21 2026 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2083.outbound.protection.outlook.com [40.107.244.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA7141AD3E1 for ; Mon, 3 Mar 2025 04:44:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.83 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740977064; cv=fail; b=mgubXGFjz3ngqBJ+lnCStS8tuaOCkJdgg21JhaLea1A27qyHwRM7AisWP8nLfo4ltaHkAzsKWkT5ILL9Prt0p8yQWbQpcUhl1Hj+cKn6hqyzOW0cgW4nypOdSkDoYipGJ07srgrBR5PsfrSCdb6HcrTa6hHCNmw70BmaQcEi9mw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740977064; c=relaxed/simple; bh=XtGjJbAHgyQDywu4sAzTo90MpkCHxk/IlmXCuJPEEIk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aY0RObNlY6kE0Ut5OAPa9Bo8LsHrlzcv7cRtFCcTKJmLcnt9HuWlO8EE5AX+0EVzL09mUUg8FzriSjcvbuHvb+yzVOnI+BFw1gTgFx0FCcJ0MrjNHIhC1HUMxoY4BZN9dHjLSk8nxgxkmyxVYklDcIMV4wxh82FXO0kbDzA786c= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=QohRNlkw; arc=fail smtp.client-ip=40.107.244.83 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="QohRNlkw" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pblDrCjGcufyuZ8fcL+///7svB9qGi+KdqlE+v6/5Tvyza8A45nrjdT4LvLzJcQjeyDoAQL1XvPhXRCbuHkB5RkwrpQNpnTJdUH5XpYht3rr+QVFOc+Uk8JAR47bxUkxjWxYVkJvOzM6rkegc08SM7DT8La7Hxfn0TyHtch03YuFnC5RcnkIkDgQSB6UQBqKTsobY9wbSk59sT5CkrODp90vWr0Cl7yfdK0CrS/sSI49yx2fKFzyoLvYeAs5iQ3yzWonN+CGCZukZysRpmlTzPS9idTEdzB2T35KiKvoQh6YngBNsT6twFNYtb1tbxq0TPCaw6fFJaizwIhTaI9Ntw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=djM+GQuvpvhxQwRb1KrH4iDfNebLIP//8tpauoKvBAM=; b=tOgitb0flS6URv6qzwasWbugC7U7eAxlKkgbzL/VlOK9GX3B3Y1BOqmPrf4c1FTjufuUloRrH59XBrpU3Dqc1Crl5zW2NX+xwDDS8lDerQ6yPeoN8meQlSd8CiTCXqTBri+f7jqmwFz4Sq1vnjDLlSkPzPRt5bYhuM6W5StBFoVL12z9FtGigVEbc9l0GYsi0PPc7THAFIqkDKl/aVn+DC+AcWWTJDsYjih6sUWO16S5+HW0D2yTmQYneI1ocUNcXGZWcRDPXXdtt+QA4V3xBm5zlu9hM7Ofpmny1hNeIADmmhgBfT5FwdnTar3VNFerIFABxildLO17xSHFcBQetw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=arndb.de smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=djM+GQuvpvhxQwRb1KrH4iDfNebLIP//8tpauoKvBAM=; b=QohRNlkwqaI0rA8lbDIjTO/Yx/erPV0YGSQo57yATsmrYWC2GprY6u5WXyDFXz7BHXeeofhh4XnoY126a6PcC0MQU/lebS0Aqu0XX75fFXx9gDHepvEVNRB2zN8+1Zh+aSRtFawjWiTM4UCeCqP9hSIX397wSJ4qFG4wvUUkYdE= Received: from MN2PR17CA0021.namprd17.prod.outlook.com (2603:10b6:208:15e::34) by MW6PR12MB8734.namprd12.prod.outlook.com (2603:10b6:303:249::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.25; Mon, 3 Mar 2025 04:44:16 +0000 Received: from BL02EPF00021F6F.namprd02.prod.outlook.com (2603:10b6:208:15e:cafe::c) by MN2PR17CA0021.outlook.office365.com (2603:10b6:208:15e::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8489.23 via Frontend Transport; Mon, 3 Mar 2025 04:44:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by BL02EPF00021F6F.mail.protection.outlook.com (10.167.249.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8511.15 via Frontend Transport; Mon, 3 Mar 2025 04:44:16 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 2 Mar 2025 22:44:11 -0600 Received: from xsjdavidz01.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.39 via Frontend Transport; Sun, 2 Mar 2025 22:44:11 -0600 From: Yidong Zhang To: , , CC: Yidong Zhang , , , Nishad Saraf Subject: [PATCH V0 3/3] drivers/misc/amd: Add load base shell firmware Date: Sun, 2 Mar 2025 20:43:12 -0800 Message-ID: <20250303044313.679012-4-yidong.zhang@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250303044313.679012-1-yidong.zhang@amd.com> References: <20250303044313.679012-1-yidong.zhang@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Received-SPF: None (SATLEXMB03.amd.com: yidong.zhang@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF00021F6F:EE_|MW6PR12MB8734:EE_ X-MS-Office365-Filtering-Correlation-Id: 24bba117-5bf9-4d6f-abf7-08dd5a0e0d99 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?AJDiFjHcOgsrYgCSOP+GN6avtQhKiwCRKdvoUwQQBHef5cQ3qQfXGLDmCAIz?= =?us-ascii?Q?zc8QHnu5PW4qdgZvtuEhmo6duu+33sPGTjHyXJKbkyos36kwvAwO0clLHuSI?= =?us-ascii?Q?xCcTvkqkclsy9vsUbe3jPGTPX029b3yUEMbf6aL4/oPtsEv7xz0CwatgdhWc?= =?us-ascii?Q?KM1+0mU13g962NdEzR4FrmpP4qrFRmk0gMEh60EczhzMSvTuk61U17+b4q/k?= =?us-ascii?Q?5Ikthkb2fX/xHrXB7EY0go+uirpXSYicn3xsH9ZBRVgX4crPznmg9OxEpK3h?= =?us-ascii?Q?rhsxUEbuRUMs7/qtj1XlEbaWe4g4NR7UWCyIzOv4RherG8ZioI+gf2WfGEtq?= =?us-ascii?Q?5crjn3KPzInfox5kXasc3IDal0ffB8v8c8hShy/DiTn4sOrgt/kYe1BMnlN0?= =?us-ascii?Q?bEaE7pfQ0ZX2inUAilbncgR7ZiN0Ye1UqY+ORl2npSYS++48Xf+tXEzdmenH?= =?us-ascii?Q?FpwGn/vqXJVsl4qPvJOh5VbwIyzNWqtm6gsiXmO5x6iuWuo32VQNg6ysm5LT?= =?us-ascii?Q?71J3bWD9lGXfGyuNRpChZIiqXJR5D7vjJ9tcdFqgrZblbwY7iA0YehicYmDc?= =?us-ascii?Q?CzG7NVQLoyE8o4lj6jdsjPN6jxl9PO+hxTdUhMIv27s65SVj/XTdORoSdE48?= =?us-ascii?Q?N1pUVMcSWhumme4MzgIBQAp0AXlUOHlT+sLgLb9/0fJt1Bagc2dO0IfL6rBm?= =?us-ascii?Q?bUqMUol015E+dDS+/aHxHwqWom5eyF1w5leO7mRMv5hNR7AvIhpsuZJKFGHU?= =?us-ascii?Q?DuyUvs33thVdhXafkZtAWpewYRBPY2a5iq+2mXeiQGVad4IdZDznMKQQT5SE?= =?us-ascii?Q?UrotrDfuJje/g7vLanZsBjlyDj++g7nnNRaChDnLCpF39IvRlpyQhwRwP4Eg?= =?us-ascii?Q?nMBqNLgJw/yxTa0I6/ZY9QKoE7ePMHlYsvDNOYfS87m+T/wn9hGpkVBwygGV?= =?us-ascii?Q?tPCKU18rcrl0zoGqUT5WjQB6wpteazfVH9QqMtL2PHLlDrGhMbvLhPEvO3f9?= =?us-ascii?Q?7cxE6kZMj8yDSfvHKWK3VPm0bpzZI0dY9vwWzv3S5BD/IVJq0ec54YezLvdb?= =?us-ascii?Q?OJh5vwLA4yEKCFjI8zEfD2xAPanK+FxVTiwpjl4/MbA2ysNgNEc/8YHjE/1C?= =?us-ascii?Q?pUGYbODVF95ZlRxah1OBcKE9ptw6jLgBXLsx+Snz8B4igU2ABMk3G1SdeZsZ?= =?us-ascii?Q?yGJ1JMFr2f0fTAIFo1hxRmnoe0BaEv9PpTNswABKBB64iLAkM+biP/UYO1I5?= =?us-ascii?Q?OHxEx4RlnL1te5Y2Ow5ipM090s077cJ5OavkrOk1EnI1ynoPvtYcrmHc5ZpM?= =?us-ascii?Q?YsWeZ3QP9HWvWMIsLdv+FQBuIm1XEpUjki7HFerCjcihRj4eDcC2MhJHChmA?= =?us-ascii?Q?qd1QJ/Gr35wva7qeurrj8t8zVgH+3M5GrnuPBg7Q5YS8pDnEAgv74CWnm76f?= =?us-ascii?Q?laGqH3xGQqe6monTFAOYATiWhb92W/yNj4T0Jw8hR7HRl/Ob7Uk2AqVnig3c?= =?us-ascii?Q?+vtOSkaQeDnyJl0=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Mar 2025 04:44:16.0226 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 24bba117-5bf9-4d6f-abf7-08dd5a0e0d99 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00021F6F.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8734 Content-Type: text/plain; charset="utf-8" - programming FW - The base FW is downloaded onto the flash of the card. - The APU FW is downloaded once after a POR (power on reset). - Reloading the driver will not change any existing hardware. Co-developed-by: Nishad Saraf Signed-off-by: Nishad Saraf Signed-off-by: Yidong Zhang --- drivers/misc/amd/Makefile | 3 +- drivers/misc/amd/versal-pci-main.c | 45 +- drivers/misc/amd/versal-pci-rm-queue.c | 14 +- drivers/misc/amd/versal-pci-rm-service.c | 497 +++++++++++++++++++++++ drivers/misc/amd/versal-pci-rm-service.h | 20 + drivers/misc/amd/versal-pci.h | 1 + 6 files changed, 569 insertions(+), 11 deletions(-) create mode 100644 drivers/misc/amd/versal-pci-rm-service.c diff --git a/drivers/misc/amd/Makefile b/drivers/misc/amd/Makefile index 63879800a3e5..03394b06fd33 100644 --- a/drivers/misc/amd/Makefile +++ b/drivers/misc/amd/Makefile @@ -3,4 +3,5 @@ obj-$(CONFIG_AMD_VERSAL_PCI) +=3D versal-pci.o =20 versal-pci-$(CONFIG_AMD_VERSAL_PCI) :=3D versal-pci-main.o \ - versal-pci-rm-queue.o + versal-pci-rm-queue.o \ + versal-pci-rm-service.o diff --git a/drivers/misc/amd/versal-pci-main.c b/drivers/misc/amd/versal-p= ci-main.c index 1c61c88120b4..6d3a2833f0c4 100644 --- a/drivers/misc/amd/versal-pci-main.c +++ b/drivers/misc/amd/versal-pci-main.c @@ -8,6 +8,8 @@ #include =20 #include "versal-pci.h" +#include "versal-pci-rm-service.h" +#include "versal-pci-rm-queue.h" =20 #define DRV_NAME "amd-versal-pci" =20 @@ -21,6 +23,29 @@ static inline u32 versal_pci_devid(struct versal_pci_dev= ice *vdev) PCI_DEVID(vdev->pdev->bus->number, vdev->pdev->devfn)); } =20 +static int versal_pci_upload_fw(struct versal_pci_device *vdev, + enum rm_queue_opcode opcode, + const char *data, + size_t size) +{ + struct rm_cmd *cmd; + int ret; + + ret =3D rm_queue_create_cmd(vdev->rdev, opcode, &cmd); + if (ret) + return ret; + + ret =3D rm_queue_data_init(cmd, data, size); + if (ret) + goto done; + + ret =3D rm_queue_send_cmd(cmd, RM_CMD_WAIT_DOWNLOAD_TIMEOUT); + +done: + rm_queue_destroy_cmd(cmd); + return ret; +} + static int versal_pci_load_shell(struct versal_pci_device *vdev, char *fw_= name) { const struct firmware *fw; @@ -57,7 +82,8 @@ static int versal_pci_load_shell(struct versal_pci_device= *vdev, char *fw_name) goto release_firmware; } =20 - /* TODO upload fw to card */ + ret =3D versal_pci_upload_fw(vdev, RM_QUEUE_OP_LOAD_FW, + (char *)xsabin, xsabin->header.length); if (ret) { vdev_err(vdev, "failed to load xsabin %s : %d", fw_name, ret); goto release_firmware; @@ -159,6 +185,7 @@ static void versal_pci_device_teardown(struct versal_pc= i_device *vdev) { versal_pci_cfs_fini(&vdev->cfs_subsys); versal_pci_fw_fini(vdev); + versal_pci_rm_fini(vdev->rdev); } =20 static void versal_pci_uuid_parse(struct versal_pci_device *vdev, uuid_t *= uuid) @@ -180,7 +207,13 @@ static void versal_pci_uuid_parse(struct versal_pci_de= vice *vdev, uuid_t *uuid) =20 static int versal_pci_fw_init(struct versal_pci_device *vdev) { - /* TODO request compatible fw_id from card */ + int ret; + + ret =3D rm_queue_get_fw_id(vdev->rdev); + if (ret) { + vdev_warn(vdev, "Failed to get fw_id"); + return -EINVAL; + } =20 versal_pci_uuid_parse(vdev, &vdev->intf_uuid); =20 @@ -191,6 +224,13 @@ static int versal_pci_device_setup(struct versal_pci_d= evice *vdev) { int ret; =20 + vdev->rdev =3D versal_pci_rm_init(vdev); + if (IS_ERR(vdev->rdev)) { + ret =3D PTR_ERR(vdev->rdev); + vdev_err(vdev, "Failed to init remote queue, err %d", ret); + return ret; + } + ret =3D versal_pci_fw_init(vdev); if (ret) { vdev_err(vdev, "Failed to init fw, err %d", ret); @@ -206,6 +246,7 @@ static int versal_pci_device_setup(struct versal_pci_de= vice *vdev) return 0; =20 comm_chan_fini: + versal_pci_rm_fini(vdev->rdev); =20 return ret; } diff --git a/drivers/misc/amd/versal-pci-rm-queue.c b/drivers/misc/amd/vers= al-pci-rm-queue.c index 8c1aca442253..7afc5a1e238c 100644 --- a/drivers/misc/amd/versal-pci-rm-queue.c +++ b/drivers/misc/amd/versal-pci-rm-queue.c @@ -23,37 +23,35 @@ static inline struct rm_device *to_rdev_msg_timer(struc= t timer_list *t) =20 static inline u32 rm_io_read(struct rm_device *rdev, u32 offset) { - /* TODO */ - return 0; + return rm_reg_read(rdev, RM_PCI_IO_BAR_OFF + offset); } =20 static inline int rm_io_write(struct rm_device *rdev, u32 offset, u32 valu= e) { - /* TODO */ + rm_reg_write(rdev, RM_PCI_IO_BAR_OFF + offset, value); return 0; } =20 static inline u32 rm_queue_read(struct rm_device *rdev, u32 offset) { - /* TODO */ - return 0; + return rm_reg_read(rdev, RM_PCI_SHMEM_BAR_OFF + rdev->queue_base + offset= ); } =20 static inline void rm_queue_write(struct rm_device *rdev, u32 offset, u32 = value) { - /* TODO */ + rm_reg_write(rdev, RM_PCI_SHMEM_BAR_OFF + rdev->queue_base + offset, valu= e); } =20 static inline void rm_queue_bulk_read(struct rm_device *rdev, u32 offset, u32 *value, u32 size) { - /* TODO */ + rm_bulk_reg_read(rdev, RM_PCI_SHMEM_BAR_OFF + rdev->queue_base + offset, = value, size); } =20 static inline void rm_queue_bulk_write(struct rm_device *rdev, u32 offset, u32 *value, u32 size) { - /* TODO */ + rm_bulk_reg_write(rdev, RM_PCI_SHMEM_BAR_OFF + rdev->queue_base + offset,= value, size); } =20 static inline u32 rm_queue_get_cidx(struct rm_device *rdev, enum rm_queue_= type type) diff --git a/drivers/misc/amd/versal-pci-rm-service.c b/drivers/misc/amd/ve= rsal-pci-rm-service.c new file mode 100644 index 000000000000..6dbad1c6563d --- /dev/null +++ b/drivers/misc/amd/versal-pci-rm-service.c @@ -0,0 +1,497 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Driver for Versal PCIe device + * + * Copyright (C) 2025 Advanced Micro Devices, Inc. All rights reserved. + */ + +#include +#include +#include + +#include "versal-pci.h" +#include "versal-pci-rm-service.h" +#include "versal-pci-rm-queue.h" + +static DEFINE_IDA(rm_cmd_ids); + +static void rm_uninstall_health_monitor(struct rm_device *rdev); + +static inline struct rm_device *to_rdev_health_monitor(struct work_struct = *w) +{ + return container_of(w, struct rm_device, health_monitor); +} + +static inline struct rm_device *to_rdev_health_timer(struct timer_list *t) +{ + return container_of(t, struct rm_device, health_timer); +} + +u32 rm_reg_read(struct rm_device *rdev, u32 offset) +{ + return readl(rdev->vdev->io_regs + offset); +} + +void rm_reg_write(struct rm_device *rdev, u32 offset, const u32 value) +{ + writel(value, rdev->vdev->io_regs + offset); +} + +void rm_bulk_reg_read(struct rm_device *rdev, u32 offset, u32 *value, size= _t size) +{ + void __iomem *src =3D rdev->vdev->io_regs + offset; + void *dst =3D (void *)value; + + memcpy_fromio(dst, src, size); + /* Barrier of reading data from device */ + rmb(); +} + +void rm_bulk_reg_write(struct rm_device *rdev, u32 offset, const void *val= ue, size_t size) +{ + void __iomem *dst =3D rdev->vdev->io_regs + offset; + + memcpy_toio(dst, value, size); + /* Barrier of writing data to device */ + wmb(); +} + +static inline u32 rm_shmem_read(struct rm_device *rdev, u32 offset) +{ + return rm_reg_read(rdev, RM_PCI_SHMEM_BAR_OFF + offset); +} + +static inline void rm_shmem_bulk_read(struct rm_device *rdev, u32 offset, + u32 *value, u32 size) +{ + rm_bulk_reg_read(rdev, RM_PCI_SHMEM_BAR_OFF + offset, value, size); +} + +static inline void rm_shmem_bulk_write(struct rm_device *rdev, u32 offset, + const void *value, u32 size) +{ + rm_bulk_reg_write(rdev, RM_PCI_SHMEM_BAR_OFF + offset, value, size); +} + +void rm_queue_destroy_cmd(struct rm_cmd *cmd) +{ + ida_free(&rm_cmd_ids, cmd->sq_msg.hdr.id); + kfree(cmd); +} + +static int rm_queue_copy_response(struct rm_cmd *cmd, void *buffer, ssize_= t len) +{ + struct rm_cmd_cq_log_page *result =3D &cmd->cq_msg.data.page; + u64 off =3D cmd->sq_msg.data.page.address; + + if (!result->len || len < result->len) { + vdev_err(cmd->rdev->vdev, "Invalid response or buffer size"); + return -EINVAL; + } + + rm_shmem_bulk_read(cmd->rdev, off, (u32 *)buffer, result->len); + return 0; +} + +static void rm_queue_payload_fini(struct rm_cmd *cmd) +{ + up(&cmd->rdev->cq.data_lock); +} + +static int rm_queue_payload_init(struct rm_cmd *cmd, + enum rm_cmd_log_page_type type) +{ + struct rm_device *rdev =3D cmd->rdev; + int ret; + + ret =3D down_interruptible(&rdev->cq.data_lock); + if (ret) + return ret; + + cmd->sq_msg.data.page.address =3D rdev->cq.data_offset; + cmd->sq_msg.data.page.size =3D rdev->cq.data_size; + cmd->sq_msg.data.page.reserved1 =3D 0; + cmd->sq_msg.data.page.type =3D FIELD_PREP(RM_CMD_LOG_PAGE_TYPE_MASK, + type); + return 0; +} + +void rm_queue_data_fini(struct rm_cmd *cmd) +{ + up(&cmd->rdev->sq.data_lock); +} + +int rm_queue_data_init(struct rm_cmd *cmd, const char *buffer, ssize_t siz= e) +{ + struct rm_device *rdev =3D cmd->rdev; + int ret; + + if (!size || size > rdev->sq.data_size) { + vdev_err(rdev->vdev, "Unsupported file size"); + return -ENOMEM; + } + + ret =3D down_interruptible(&rdev->sq.data_lock); + if (ret) + return ret; + + rm_shmem_bulk_write(cmd->rdev, rdev->sq.data_offset, buffer, size); + + cmd->sq_msg.data.bin.address =3D rdev->sq.data_offset; + cmd->sq_msg.data.bin.size =3D size; + return 0; +} + +int rm_queue_create_cmd(struct rm_device *rdev, enum rm_queue_opcode opcod= e, + struct rm_cmd **cmd_ptr) +{ + struct rm_cmd *cmd =3D NULL; + int ret, id; + u16 size; + + if (rdev->firewall_tripped) + return -ENODEV; + + cmd =3D kzalloc(sizeof(*cmd), GFP_KERNEL); + if (!cmd) + return -ENOMEM; + cmd->rdev =3D rdev; + + switch (opcode) { + case RM_QUEUE_OP_LOAD_XCLBIN: + fallthrough; + case RM_QUEUE_OP_LOAD_FW: + fallthrough; + case RM_QUEUE_OP_LOAD_APU_FW: + size =3D sizeof(struct rm_cmd_sq_bin); + break; + case RM_QUEUE_OP_GET_LOG_PAGE: + size =3D sizeof(struct rm_cmd_sq_log_page); + break; + case RM_QUEUE_OP_IDENTIFY: + size =3D 0; + break; + case RM_QUEUE_OP_VMR_CONTROL: + size =3D sizeof(struct rm_cmd_sq_ctrl); + break; + default: + vdev_err(rdev->vdev, "Invalid cmd opcode %d", opcode); + ret =3D -EINVAL; + goto error; + } + + cmd->opcode =3D opcode; + cmd->sq_msg.hdr.opcode =3D FIELD_PREP(RM_CMD_SQ_HDR_OPS_MSK, opcode); + cmd->sq_msg.hdr.msg_size =3D FIELD_PREP(RM_CMD_SQ_HDR_SIZE_MSK, size); + + id =3D ida_alloc_range(&rm_cmd_ids, RM_CMD_ID_MIN, RM_CMD_ID_MAX, GFP_KER= NEL); + if (id < 0) { + vdev_err(rdev->vdev, "Failed to alloc cmd ID: %d", id); + ret =3D id; + goto error; + } + cmd->sq_msg.hdr.id =3D id; + + init_completion(&cmd->executed); + + *cmd_ptr =3D cmd; + return 0; +error: + kfree(cmd); + return ret; +} + +static int rm_queue_verify(struct rm_device *rdev) +{ + struct versal_pci_device *vdev =3D rdev->vdev; + struct rm_cmd_cq_identify *result; + struct rm_cmd *cmd; + u32 major, minor; + int ret; + + ret =3D rm_queue_create_cmd(rdev, RM_QUEUE_OP_IDENTIFY, &cmd); + if (ret) + return ret; + + ret =3D rm_queue_send_cmd(cmd, RM_CMD_WAIT_CONFIG_TIMEOUT); + if (ret) + goto error; + + result =3D &cmd->cq_msg.data.identify; + major =3D result->major; + minor =3D result->minor; + vdev_dbg(vdev, "VMR version %d.%d", major, minor); + if (!major) { + vdev_err(vdev, "VMR version is unsupported"); + ret =3D -EOPNOTSUPP; + } + +error: + rm_queue_destroy_cmd(cmd); + return ret; +} + +static int rm_check_apu_status(struct rm_device *rdev, bool *status) +{ + struct rm_cmd_cq_control *result; + struct rm_cmd *cmd; + int ret; + + ret =3D rm_queue_create_cmd(rdev, RM_QUEUE_OP_VMR_CONTROL, &cmd); + if (ret) + return ret; + + ret =3D rm_queue_send_cmd(cmd, RM_CMD_WAIT_CONFIG_TIMEOUT); + if (ret) + goto error; + + result =3D &cmd->cq_msg.data.ctrl; + *status =3D FIELD_GET(RM_CMD_VMR_CONTROL_PS_MASK, result->status); + + rm_queue_destroy_cmd(cmd); + return 0; + +error: + rm_queue_destroy_cmd(cmd); + return ret; +} + +static int rm_download_apu_fw(struct rm_device *rdev, char *data, ssize_t = size) +{ + struct rm_cmd *cmd; + int ret; + + ret =3D rm_queue_create_cmd(rdev, RM_QUEUE_OP_LOAD_APU_FW, &cmd); + if (ret) + return ret; + + ret =3D rm_queue_data_init(cmd, data, size); + if (ret) + goto done; + + ret =3D rm_queue_send_cmd(cmd, RM_CMD_WAIT_DOWNLOAD_TIMEOUT); + +done: + rm_queue_destroy_cmd(cmd); + return ret; +} + +int rm_queue_boot_apu(struct rm_device *rdev) +{ + char *bin =3D "xilinx/xrt-versal-apu.xsabin"; + const struct firmware *fw =3D NULL; + bool status; + int ret; + + ret =3D rm_check_apu_status(rdev, &status); + if (ret) { + vdev_err(rdev->vdev, "Failed to get APU status"); + return ret; + } + + if (status) { + vdev_dbg(rdev->vdev, "APU online. Skipping APU FW download"); + return 0; + } + + ret =3D request_firmware(&fw, bin, &rdev->vdev->pdev->dev); + if (ret) { + vdev_warn(rdev->vdev, "Request APU FW %s failed %d", bin, ret); + return ret; + } + + vdev_dbg(rdev->vdev, "Starting... APU FW download"); + ret =3D rm_download_apu_fw(rdev, (char *)fw->data, fw->size); + vdev_dbg(rdev->vdev, "Finished... APU FW download %d", ret); + + if (ret) + vdev_err(rdev->vdev, "Failed to download APU FW, ret:%d", ret); + + release_firmware(fw); + + return ret; +} + +static void rm_check_health(struct work_struct *w) +{ + struct rm_device *rdev =3D to_rdev_health_monitor(w); + u32 max_len =3D PAGE_SIZE; + struct rm_cmd *cmd; + int ret; + + ret =3D rm_queue_create_cmd(rdev, RM_QUEUE_OP_GET_LOG_PAGE, &cmd); + if (ret) + return; + + ret =3D rm_queue_payload_init(cmd, RM_CMD_LOG_PAGE_AXI_TRIP_STATUS); + if (ret) + goto destroy_cmd; + + ret =3D rm_queue_send_cmd(cmd, RM_CMD_WAIT_CONFIG_TIMEOUT); + if (ret =3D=3D -ETIME || ret =3D=3D -EINVAL) + goto payload_fini; + + if (ret) { + u32 log_len =3D cmd->cq_msg.data.page.len; + + if (log_len > max_len) { + vdev_warn(rdev->vdev, "msg size %d is greater than requested %d", + log_len, max_len); + log_len =3D max_len; + } + + if (log_len) { + char *buffer =3D vzalloc(log_len); + + if (!buffer) + goto payload_fini; + + ret =3D rm_queue_copy_response(cmd, buffer, log_len); + if (ret) { + vfree(buffer); + goto payload_fini; + } + + vdev_err(rdev->vdev, "%s", buffer); + vfree(buffer); + + } else { + vdev_err(rdev->vdev, "firewall check ret%d", ret); + } + + rdev->firewall_tripped =3D 1; + } + +payload_fini: + rm_queue_payload_fini(cmd); +destroy_cmd: + rm_queue_destroy_cmd(cmd); + + vdev_dbg(rdev->vdev, "check result: %d", ret); +} + +static void rm_sched_health_check(struct timer_list *t) +{ + struct rm_device *rdev =3D to_rdev_health_timer(t); + + if (rdev->firewall_tripped) { + vdev_err(rdev->vdev, "Firewall tripped, health check paused. Please rese= t card"); + return; + } + /* Schedule a work in the general workqueue */ + schedule_work(&rdev->health_monitor); + /* Periodic timer */ + mod_timer(&rdev->health_timer, jiffies + RM_HEALTH_CHECK_TIMER); +} + +static void rm_uninstall_health_monitor(struct rm_device *rdev) +{ + del_timer_sync(&rdev->health_timer); + cancel_work_sync(&rdev->health_monitor); +} + +static void rm_install_health_monitor(struct rm_device *rdev) +{ + INIT_WORK(&rdev->health_monitor, &rm_check_health); + timer_setup(&rdev->health_timer, &rm_sched_health_check, 0); + mod_timer(&rdev->health_timer, jiffies + RM_HEALTH_CHECK_TIMER); +} + +void versal_pci_rm_fini(struct rm_device *rdev) +{ + rm_uninstall_health_monitor(rdev); + rm_queue_fini(rdev); +} + +struct rm_device *versal_pci_rm_init(struct versal_pci_device *vdev) +{ + struct rm_header *header; + struct rm_device *rdev; + u32 status; + int ret; + + rdev =3D devm_kzalloc(&vdev->pdev->dev, sizeof(*rdev), GFP_KERNEL); + if (!rdev) + return ERR_PTR(-ENOMEM); + + rdev->vdev =3D vdev; + header =3D &rdev->rm_metadata; + + rm_shmem_bulk_read(rdev, RM_HDR_OFF, (u32 *)header, sizeof(*header)); + if (header->magic !=3D RM_HDR_MAGIC_NUM) { + vdev_err(vdev, "Invalid RM header 0x%x", header->magic); + ret =3D -ENODEV; + goto err; + } + + status =3D rm_shmem_read(rdev, header->status_off); + if (!status) { + vdev_err(vdev, "RM status %d is not ready", status); + ret =3D -ENODEV; + goto err; + } + + rdev->queue_buffer_size =3D header->data_end - header->data_start + 1; + rdev->queue_buffer_start =3D header->data_start; + rdev->queue_base =3D header->queue_base; + + ret =3D rm_queue_init(rdev); + if (ret) { + vdev_err(vdev, "Failed to init cmd queue, ret %d", ret); + ret =3D -ENODEV; + goto err; + } + + ret =3D rm_queue_verify(rdev); + if (ret) { + vdev_err(vdev, "Failed to verify cmd queue, ret %d", ret); + ret =3D -ENODEV; + goto queue_fini; + } + + ret =3D rm_queue_boot_apu(rdev); + if (ret) { + vdev_err(vdev, "Failed to bringup APU, ret %d", ret); + ret =3D -ENODEV; + goto queue_fini; + } + + rm_install_health_monitor(rdev); + + return rdev; +queue_fini: + rm_queue_fini(rdev); +err: + return ERR_PTR(ret); +} + +int rm_queue_get_fw_id(struct rm_device *rdev) +{ + struct rm_cmd *cmd; + int ret; + + ret =3D rm_queue_create_cmd(rdev, RM_QUEUE_OP_GET_LOG_PAGE, &cmd); + if (ret) + return ret; + + ret =3D rm_queue_payload_init(cmd, RM_CMD_LOG_PAGE_FW_ID); + if (ret) + goto destroy_cmd; + + ret =3D rm_queue_send_cmd(cmd, RM_CMD_WAIT_CONFIG_TIMEOUT); + if (ret) + goto payload_fini; + + ret =3D rm_queue_copy_response(cmd, rdev->vdev->fw_id, sizeof(rdev->vdev-= >fw_id)); + if (ret) + goto payload_fini; + + vdev_info(rdev->vdev, "fw_id %s", rdev->vdev->fw_id); + +payload_fini: + rm_queue_payload_fini(cmd); +destroy_cmd: + rm_queue_destroy_cmd(cmd); + + return ret; +} diff --git a/drivers/misc/amd/versal-pci-rm-service.h b/drivers/misc/amd/ve= rsal-pci-rm-service.h index d2397a1a672c..57333bb1443c 100644 --- a/drivers/misc/amd/versal-pci-rm-service.h +++ b/drivers/misc/amd/versal-pci-rm-service.h @@ -206,4 +206,24 @@ struct rm_device { __u32 firewall_tripped; }; =20 +/* rm service init api */ +struct rm_device *versal_pci_rm_init(struct versal_pci_device *vdev); +void versal_pci_rm_fini(struct rm_device *rdev); + +/* rm services APIs */ +int rm_queue_create_cmd(struct rm_device *rdev, enum rm_queue_opcode opcod= e, + struct rm_cmd **cmd_ptr); +void rm_queue_destroy_cmd(struct rm_cmd *cmd); + +int rm_queue_data_init(struct rm_cmd *cmd, const char *buffer, ssize_t siz= e); +void rm_queue_data_fini(struct rm_cmd *cmd); +int rm_queue_get_fw_id(struct rm_device *rdev); +int rm_queue_boot_apu(struct rm_device *rdev); + +/* rm bar register operation APIs */ +u32 rm_reg_read(struct rm_device *rdev, u32 offset); +void rm_reg_write(struct rm_device *rdev, u32 offset, const u32 value); +void rm_bulk_reg_read(struct rm_device *rdev, u32 offset, u32 *value, size= _t size); +void rm_bulk_reg_write(struct rm_device *rdev, u32 offset, const void *val= ue, size_t size); + #endif /* __RM_SERVICE_H */ diff --git a/drivers/misc/amd/versal-pci.h b/drivers/misc/amd/versal-pci.h index 33f0ef881a33..89f3590137ce 100644 --- a/drivers/misc/amd/versal-pci.h +++ b/drivers/misc/amd/versal-pci.h @@ -51,6 +51,7 @@ struct fw_info { struct versal_pci_device { struct pci_dev *pdev; =20 + struct rm_device *rdev; struct fw_info fw; =20 void __iomem *io_regs; --=20 2.34.1